Monday, May 21, 2012
 
The best way to analyze your logs! Minimize
 Log4Net Mail archive   

The Log4Net mailing list is a great source of information about using log4Net, in this forum we collect all the messages in the log4net user list and some selected threads from the developer list.

Subject: Best Way to Set Up Massive Number of Loggers
Prev Next
You are not authorized to post a reply.

Author Messages
Eric Marthinsen

12/02/2008 1:31 AM  
Hello-

I'm working on an app that coordinates data transfers among multiple trading partners. I want to create a log file for each trading partner. I had a few ideas of how to do this, but am not sure which would be considered the optimal way. The expected number of trading partners is likely to be around 100, but could be higher. Here are my thoughts:

1. Add a bunch of named loggers to my log4net.config file
2. Do programmatic configuration of the loggers on application start
3. Write my own appender that is based on the RollingFileAppender

Do any of these scream out as being a good or bad solution? Is there a better approach that I might be missing? Thanks in advance.

Regards,
Eric
Walden H. Leverich

12/02/2008 1:39 AM  

Since you're asking for thoughts... bad idea. To me log4net is plumbing, that is it's programmer based stuff. Sounds to me like you're attempting to use it to satisfy a user requirement. I think you'd be better off adding the logging as part of the business processing. Now, having said that, there are still times when it's nice to know which trading partner you're dealing with on a log message. In that case I'd push the partner id into the NDC at the start of the process, and pop it at the end. That way you'll have it in the log (if you format it correctly). You'll have a single log file, but you can always fgrep it. We do something similar with several levels of NDC and can quickly come up with a issue-specific logfile using grep.

 

-Walden

 

--

Walden H Leverich III

Tech Software

(516) 627-3800 x3051

WaldenL@TechSoftInc.com

http://www.TechSoftInc.com

 

Quiquid latine dictum sit altum viditur.

(Whatever is said in Latin seems profound.)

 

Eric Marthinsen

12/02/2008 1:48 AM  
Hi Walden-

I like the grep idea. The motivation for the multiple log files is twofold. First, they get really big really quickly. I realize I can roll them on size, but it's nice to have a daily log file. Second, we have some utilities that can take a log file and replay previous transactions (we are logging a lot of data in custom formats). These are still programmer tools (and QA).

-Eric
Loren Keagle

12/02/2008 8:19 AM  

This is the only way I know how to do this in log4net. We do this to
handle remote logging from embedded controllers. We use .NET remoting
appenders in this case, so we get the HostName property of each remote
host logging to the sink. But if you're local you could do the same
thing with a string filter or other property filter. This gives you
individual layout control over each appender, and hence each log source.

If only log4net would add in a layout-ref tag so we wouldn't have to
repeat ourselves 75 times...


<log4net>
<appender name="ConsoleAppender" type="log4net.Appender.ConsoleAppender">
<layout type="log4net.Layout.PatternLayout">
<param name="ConversionPattern" value="%property{log4net:HostName}
%d{MM/dd/yyyy HH:mm:ss.fff} [%p] %c{1}: %m%n" />
</layout>
</appender>

<appender name="Comp1" type="log4net.Appender.RollingFileAppender">
<file value="Comp1/Comp1.log" />
<staticLogFileName value="true" />
<filter type="log4net.Filter.PropertyFilter">
<Key value="log4net:HostName" />
<StringToMatch value="Comp1" />
</filter>
<filter type="log4net.Filter.DenyAllFilter" />
<appendToFile value="true" />
<rollingStyle value="Date" />
<datePattern value="yyyyMMdd’.log’" />
<layout type="log4net.Layout.PatternLayout">
<param name="ConversionPattern" value="%d{MM/dd/yyyy HH:mm:ss.fff} [%p]
%c{1}: %m%n" />
</layout>
</appender>

<appender name="Comp2" type="log4net.Appender.RollingFileAppender">
<file value="Comp2/Comp2.log" />
<staticLogFileName value="true" />
<filter type="log4net.Filter.PropertyFilter">
<Key value="log4net:HostName" />
<StringToMatch value="Comp2" />
</filter>
<filter type="log4net.Filter.DenyAllFilter" />
<appendToFile value="true" />
<rollingStyle value="Date" />
<datePattern value="yyyyMMdd" />
<layout type="log4net.Layout.PatternLayout">
<param name="ConversionPattern" value="%d{MM/dd/yyyy HH:mm:ss.fff} [%p]
%c{1}: %m%n" />
</layout>
</appender>

<appender name="Comp3" type="log4net.Appender.RollingFileAppender">
<file value="Comp3/Comp3.log" />
<staticLogFileName value="true" />
<filter type="log4net.Filter.PropertyFilter">
<Key value="log4net:HostName" />
<StringToMatch value="Comp3" />
</filter>
<filter type="log4net.Filter.DenyAllFilter" />
<appendToFile value="true" />
<rollingStyle value="Date" />
<datePattern value="yyyyMMdd" />
<layout type="log4net.Layout.PatternLayout">
<param name="ConversionPattern" value="%d{MM/dd/yyyy HH:mm:ss.fff} [%p]
%c{1}: %m%n" />
</layout>
</appender>

...

<root>
<level value="ALL" />
<appender-ref ref="ConsoleAppender" />
<appender-ref ref="Comp1" />
<appender-ref ref="Comp2" />
<appender-ref ref="Comp3" />
...
</root>
</log4net>

Ron Grabowski

12/15/2008 8:51 PM  

If you don't want to type the same pattern over and over, extend PatternLayout and provide your own default pattern:

 <layout type="Company.Logging.MyPatternLayout, Company.Logging" />

That can be overridden when necessary:

 <layout type="Company.Logging.MyPatternLayout, Company.Logging">
  <conversionPattern" value="%d %m%n" />
 </layout>

You can also extend RollingFileAppender and set default values in there:

 <appender name="Comp1" type="Company.Logging.CompRollingFileAppender, Company.Logging">
  <comp value="1"/>
 </appender>
 <appender name="Comp2" type="Company.Logging.CompRollingFileAppender, Company.Logging">

<comp value="2"/>
 </appender>
 <appender name="Comp3" type="Company.Logging.CompRollingFileAppender, Company.Logging">

<comp value="3"/>
  <layout type="Company.Logging.MyPatternLayout, Company.Logging">
   <conversionPattern" value="%d %m%n" />
  </layout>
 </appender>
 <appender name="Comp4" type="Company.Logging.CompRollingFileAppender, Company.Logging">

<comp value="4"/>
 </appender>

Ron Grabowski

12/15/2008 8:55 PM  
It would be nice if the File property on the FileAppender had more robust support for patterns so you could write to a dynamic file. NLog and log4j have had support for this for a while. I think they keep a pool of files open then close a file if it hasn't been used recently.
 
You are not authorized to post a reply.
Forums > Log4Net > Log4Net Mail archive > Best Way to Set Up Massive Number of Loggers



ActiveForums 3.7

 

 

 

 

 

 

 

 

Log4Net Dashboard

Log analysis and monitoring made easy!

Log4Net Dashboard is a log viewer that can read log statements from a variety of logging output targets.

You can download a free developer version.

  

Check it out!

On the demonstration site you can try it  with live data.demo.l4ndash.com - Try Log4Net Dashboard with live data

The mail archive is a copy of all the mail sent to the mail address: log4net-user@logging.apache.org, organized as a forum.

If you would like to participate in the mail list, send a mail to log4net-user-subscribe@logging.apache.org.

More information about the mailing list is available on: http://logging.apache.org/log4net/support.html

 

A complete topic list is available and can be viewed here (warning, it takes some time to load)

 

Copyright 2005-2008 by FaktNet AS Terms Of Use Privacy Statement