Hands-create the wheel: Write a logging framework

Hands-create the wheel: Write a logging framework

Intro

There are a lot of logging frameworks, such as log4net/ nlog/ serilog/ microsoft.extensions.loggingand so on, how to do without having to modify the code, just switch to a different frame when switching logs loggingProvideron it, reduce the cost of the lowest-cost switch logging framework, in this consideration wrote a logging framework for different logging framework write an adapter, need to use what logging framework configuration click on it, businesses do not need to change the code.

V0

The initial log strongly dependent on the log4net, log4net logging framework is the first one I use, so for a long time are using it for logging, but due to the strong dependence, when want to change the log frame will be very hard to accept, to change a lot of code, does not comply with the basic principles of the open closed, so there will be the first version of the log.

V1

The first edition of the journal reference implementation of Microsoft's logging framework, probably structured as follows:

public interface ILogHelperLogFactory
{
    ILogger CreateLogger(string categoryName);
    
    bool AddProvider(ILogHelperProvider provider);
}
public interface ILogHelperLogger
{
    bool IsEnabled(LogHelperLogLevel logLevel);
    void Log(LogHelperLogLevel logLevel, Exception exception, string message);
}
public enum LogHelperLogLevel
{
    /// <summary>
    /// All logging levels
    /// </summary>
    All = 0,

    /// <summary>
    /// A trace logging level
    /// </summary>
    Trace = 1,

    /// <summary>
    /// A debug logging level
    /// </summary>
    Debug = 2,

    /// <summary>
    /// A info logging level
    /// </summary>
    Info = 4,

    /// <summary>
    /// A warn logging level
    /// </summary>
    Warn = 8,

    /// <summary>
    /// An error logging level
    /// </summary>
    Error = 16,

    /// <summary>
    /// A fatal logging level
    /// </summary>
    Fatal = 32,

    /// <summary>
    /// None
    /// </summary>
    None = 64
}
public interface ILogHelperProvider
{
    ILogHelperLogger CreateLogger(string categoryName);
}

For ease of use the Logger, defines extensions to the method, may be used as such logger.Info/ logger.Erroror the like, expansion is defined as follows:

public static void Log(this ILogHelperLogger logger, LogHelperLevel loggerLevel, string msg) => logger.Log(loggerLevel, null, msg);

#region Info

    public static void Info(this ILogHelperLogger logger, string msg, params object[] parameters)
{
    if (parameters == null || parameters.Length == 0)
    {
        logger.Log(LogHelperLevel.Info, msg);
    }
    else
    {
        logger.Log(LogHelperLevel.Info, null, msg.FormatWith(parameters));
    }
}

public static void Info(this ILogHelperLogger logger, Exception ex, string msg) => logger.Log(LogHelperLevel.Info, ex, msg);

public static void Info(this ILogHelperLogger logger, Exception ex) => logger.Log(LogHelperLevel.Info, ex, ex?.Message);

#endregion Info
// ...其他的类似，这里就不详细展开了

If you want to customize the logging defined, then it is a realization ILogHelperProvidercan achieve a ILogHelperProviderwill to achieve a ILogHelperLogger, originally strong dependence of log4net can achieve a Log4NetLogHelperProviderso change other logging framework only when needed to achieve corresponding ILogHelperProviderto, but from a functional on sex is still weak

If you want some logs do not record, for example, Debug level logs do not record, for example, only Error level logging under certain Logger, now is a bit hard, only through the log4net configuration limits, so there a second version, an increase LoggingFiltercan be set filter for Provider / Logger / LogLevel / Exception, filter unwanted record of the log, which is the reference to the Microsoft filter logging framework, but to achieve quite the same, there is little interest in the partnership you can own in-depth study.

V2

Version V2, the ILogFactoryinterface increases the AddFiltermethod is defined as follows:

/// <summary>   
/// Add logs filter 
/// </summary>  
/// <param name="filterFunc">filterFunc, logProviderType/categoryName/Exception, whether to write log</param>   
bool AddFilter(Func<Type, string, LogHelperLogLevel, Exception, bool> filterFunc);

Then defines some extensions ways to facilitate the use of:

public static ILogHelperFactory WithMinimumLevel(this ILogHelperFactory logHelperFactory, LogHelperLevel logLevel)
{
    return logHelperFactory.WithFilter(level => level >= logLevel);
}

public static ILogHelperFactory WithFilter(this ILogHelperFactory logHelperFactory, Func<LogHelperLevel, bool> filterFunc)
{
    logHelperFactory.AddFilter((type, categoryName, logLevel, exception) => filterFunc.Invoke(logLevel));
    return logHelperFactory;
}

public static ILogHelperFactory WithFilter(this ILogHelperFactory logHelperFactory, Func<string, LogHelperLevel, bool> filterFunc)
{
    logHelperFactory.AddFilter((type, categoryName, logLevel, exception) => filterFunc.Invoke(categoryName, logLevel));
    return logHelperFactory;
}

public static ILogHelperFactory WithFilter(this ILogHelperFactory logHelperFactory, Func<Type, string, LogHelperLevel, bool> filterFunc)
{
    logHelperFactory.AddFilter((type, categoryName, logLevel, exception) => filterFunc.Invoke(type, categoryName, logLevel));
    return logHelperFactory;
}

public static ILogHelperFactory WithFilter(this ILogHelperFactory logHelperFactory, Func<Type, string, LogHelperLevel, Exception, bool> filterFunc)
{
    logHelperFactory.AddFilter(filterFunc);
    return logHelperFactory;
}

This facilitates We just want to define for the Logger Filter and Filter Provider, do not all parameters are used, logging filter has already been realized, this time has been used Serilogto do the logging for some time, feeling Serilogin some very good design , very elegant, so the want Serilogin some of the design used in the framework of their own log, for example:

1. SerilogThe extension is called the Sinklocal log output, Seriloga custom Sink, very simple just need to implement an interface, do not need to implement a Logger, from this point, I feel Serilogmore better than Microsoft's logging framework, and LogEventmaking the log easier batch operation, there is a need to understand what Serilogthe PeriodBatching https://github.com/serilog/serilog-sinks-periodicbatching

   /// <summary>
   /// A destination for log events.
   /// </summary>
   public interface ILogEventSink
   {
       /// <summary>
       /// Emit the provided log event to the sink.
       /// </summary>
       /// <param name="logEvent">The log event to write.</param>
       void Emit(LogEvent logEvent);
   }

2. SerilogSome can be customized Enricherin order to enrich the content of the log record, such as the request context log, log environment, or may be some fixed attribute information

3. MessageTemplateIn fact logging framework, Microsoft also has a similar concept, but very obviously, with Serilogbefore I rarely use Microsoft's logging framework can do with logger.LogInfo("hello {name}", "world")such wording in fact be regarded as the first parameter MessageTemplateor within it it is calledFormat

Given so many benefits, it intended bringing these features to my log frame

V3

Introduced LoggingEvent

She went ahead, to introduce a first LogHelperLoggingEvent, corresponding Serilogto LogEventdefined as follows:

public class LogHelperLoggingEvent : ICloneable
{
    public string CategoryName { get; set; }

    public DateTimeOffset DateTime { get; set; }

    public string MessageTemplate { get; set; }

    public string Message { get; set; }

    public Exception Exception { get; set; }

    public LogHelperLogLevel LogLevel { get; set; }

    public Dictionary<string, object> Properties { get; set; }

    public LogHelperLoggingEvent Copy => (LogHelperLoggingEvent)Clone();

    public object Clone()
    {
        var newEvent = (LogHelperLoggingEvent)MemberwiseClone();
        if (Properties != null)
        {
            newEvent.Properties = new Dictionary<string, object>();
            foreach (var property in Properties)
            {
                newEvent.Properties[property.Key] = property.Value;
            }
        }
        return newEvent;
    }
}

Event Properties defined in the dictionary for a rich log additionally implements ICloneableinterface to facilitate copying of objects, for strongly typed, adds a Copymethod, the object returns a strongly typed

Transformation LogProvider

In order to reduce expand a ILogProvidercomplexity, we want to ILogProvidermake a simplified, just like the expansion Serilogof Sink as logging can be, do not care whether you want to create Logger

After transformation, defined as follows:

public interface ILogHelperProvider
{
    Task Log(LogHelperLoggingEvent loggingEvent);
}

(Here returns a Task, may void return type is enough to see to their needs)

In this way realized LogProviderwhen only need to implement this interface can be, and do not need to implement a Logger up

Increase Enricher

Enricher definition:

public interface ILogHelperLoggingEnricher
{
    void Enrich(LogHelperLoggingEvent loggingEvent);
}

A built-in PropertyEnricher, easy to add some simple properties

internal class PropertyLoggingEnricher : ILogHelperLoggingEnricher
{
    private readonly string _propertyName;
    private readonly Func<LogHelperLoggingEvent, object> _propertyValueFactory;
    private readonly bool _overwrite;
    private readonly Func<LogHelperLoggingEvent, bool> _logPropertyPredict = null;

    public PropertyLoggingEnricher(string propertyName, object propertyValue, bool overwrite = false) : this(propertyName, (loggingEvent) => propertyValue, overwrite)
    {
    }

    public PropertyLoggingEnricher(string propertyName, Func<LogHelperLoggingEvent, object> propertyValueFactory,
                                   bool overwrite = false) : this(propertyName, propertyValueFactory, null, overwrite)
    {
    }

    public PropertyLoggingEnricher(string propertyName, Func<LogHelperLoggingEvent, object> propertyValueFactory, Func<LogHelperLoggingEvent, bool> logPropertyPredict,
                                   bool overwrite = false)
    {
        _propertyName = propertyName;
        _propertyValueFactory = propertyValueFactory;
        _logPropertyPredict = logPropertyPredict;
        _overwrite = overwrite;
    }

    public void Enrich(LogHelperLoggingEvent loggingEvent)
    {
        if (_logPropertyPredict?.Invoke(loggingEvent) != false)
        {
            loggingEvent.AddProperty(_propertyName, _propertyValueFactory, _overwrite);
        }
    }
}

To ILogFactoryadd a AddEnrichermethod

/// <summary>
/// add log enricher
/// </summary>
/// <param name="enricher">log enricher</param>
/// <returns></returns>
bool AddEnricher(ILogHelperLoggingEnricher enricher);

So when we log on through these Enricher rich LoggingEventin the Properties

In order to facilitate operation of Property, we added some extension methods:

public static ILogHelperFactory WithEnricher<TEnricher>(this ILogHelperFactory logHelperFactory,
                                                        TEnricher enricher) where TEnricher : ILogHelperLoggingEnricher
{
    logHelperFactory.AddEnricher(enricher);
    return logHelperFactory;
}

public static ILogHelperFactory WithEnricher<TEnricher>(this ILogHelperFactory logHelperFactory) where TEnricher : ILogHelperLoggingEnricher, new()
{
    logHelperFactory.AddEnricher(new TEnricher());
    return logHelperFactory;
}

public static ILogHelperFactory EnrichWithProperty(this ILogHelperFactory logHelperFactory, string propertyName, object value, bool overwrite = false)
{
    logHelperFactory.AddEnricher(new PropertyLoggingEnricher(propertyName, value, overwrite));
    return logHelperFactory;
}

public static ILogHelperFactory EnrichWithProperty(this ILogHelperFactory logHelperFactory, string propertyName, Func<LogHelperLoggingEvent> valueFactory, bool overwrite = false)
{
    logHelperFactory.AddEnricher(new PropertyLoggingEnricher(propertyName, valueFactory, overwrite));
    return logHelperFactory;
}

public static ILogHelperFactory EnrichWithProperty(this ILogHelperFactory logHelperFactory, string propertyName, object value, Func<LogHelperLoggingEvent, bool> predict, bool overwrite = false)
{
    logHelperFactory.AddEnricher(new PropertyLoggingEnricher(propertyName, e => value, predict, overwrite));
    return logHelperFactory;
}

public static ILogHelperFactory EnrichWithProperty(this ILogHelperFactory logHelperFactory, string propertyName, Func<LogHelperLoggingEvent, object> valueFactory, Func<LogHelperLoggingEvent, bool> predict, bool overwrite = false)
{
    logHelperFactory.AddEnricher(new PropertyLoggingEnricher(propertyName, valueFactory, predict, overwrite));
    return logHelperFactory;
}

MessageTemplate

From the above LoggingEventit has been added MessageTemplate, so we introduced the Microsoft log messages formatted frame, and converting the parameters into messageTemplate Message and Properties, specific reference https://github.com/WeihanLi/WeihanLi.Common/blob/276cc49cfda511f9b7b3bb8344ee52441c4a3b23 /src/WeihanLi.Common/Logging/LoggingFormatter.cs

internal struct FormattedLogValue
{
    public string Msg { get; set; }

    public Dictionary<string, object> Values { get; set; }

    public FormattedLogValue(string msg, Dictionary<string, object> values)
    {
        Msg = msg;
        Values = values;
    }
}

internal static class LoggingFormatter
{
    public static FormattedLogValue Format(string msgTemplate, object[] values)
    {
        if (values == null || values.Length == 0)
            return new FormattedLogValue(msgTemplate, null);

        var formatter = new LogValuesFormatter(msgTemplate);
        var msg = formatter.Format(values);
        var dic = formatter.GetValues(values)
            .ToDictionary(x => x.Key, x => x.Value);

        return new FormattedLogValue(msg, dic);
    }
}

So that we can support messageTemplate, and then to transform what our Logger

public interface ILogHelperLogger
{
    void Log(LogHelperLogLevel logLevel, Exception exception, string messageTemplate, params object[] parameters);

    bool IsEnabled(LogHelperLogLevel logLevel);
}

The difference is that with the above, we added parameters

Again updated about our extension methods, the above expansion method is used directly string.Formatformatted the way we want to update it here

public static void Info(this ILogHelperLogger logger, string msg, params object[] parameters)
{
    logger.Log(LogHelperLogLevel.Info, null, msg, parameters);
}

public static void Info(this ILogHelperLogger logger, Exception ex, string msg) => logger.Log(LogHelperLogLevel.Info, ex, msg);

public static void Info(this ILogHelperLogger logger, Exception ex) => logger.Log(LogHelperLogLevel.Info, ex, ex?.Message);

Thus, the function basically completed, but from the perspective of the API, the feeling now ILogFactorytoo heavy, these AddProvider/ AddEnricher/ AddFiltershould attribute ILogFactoryof internal properties, through the configuration, it should not be the interface methods, so there will be next Version

V4

The moderator if introduced LoggingBuilder, through LoggingBuilderto the internal configuration LogFactoryrequired Provider/ Enricher/ Filter, their original configuration and extension methods have becomeILogHelperLoggingBuilder

public interface ILogHelperLoggingBuilder
{
    /// <summary>
    /// Adds an ILogHelperProvider to the logging system.
    /// </summary>
    /// <param name="provider">The ILogHelperProvider.</param>
    bool AddProvider(ILogHelperProvider provider);

    /// <summary>
    /// add log enricher
    /// </summary>
    /// <param name="enricher">log enricher</param>
    /// <returns></returns>
    bool AddEnricher(ILogHelperLoggingEnricher enricher);

    /// <summary>
    /// Add logs filter
    /// </summary>
    /// <param name="filterFunc">filterFunc, logProviderType/categoryName/Exception, whether to write log</param>
    bool AddFilter(Func<Type, string, LogHelperLogLevel, Exception, bool> filterFunc);

    ///// <summary>
    ///// config period batching
    ///// </summary>
    ///// <param name="period">period</param>
    ///// <param name="batchSize">batchSize</param>
    //void PeriodBatchingConfig(TimeSpan period, int batchSize);

    /// <summary>
    /// Build for LogFactory
    /// </summary>
    /// <returns></returns>
    ILogHelperFactory Build();
}

Increase logging configuration:

public static class LogHelper
{
    private static ILogHelperFactory LogFactory { get; private set; } = NullLogHelperFactory.Instance;

    public static void ConfigureLogging(Action<ILogHelperLoggingBuilder> configureAction)
    {
        var loggingBuilder = new LogHelperLoggingBuilder();
        configureAction?.Invoke(loggingBuilder);
        LogFactory = loggingBuilder.Build();
    }

    public static ILogHelperLogger GetLogger<T>() => LogFactory.GetLogger(typeof(T));

    public static ILogHelperLogger GetLogger(Type type) => LogFactory.GetLogger(type);

    public static ILogHelperLogger GetLogger(string categoryName)
    {
        return LogFactory.CreateLogger(categoryName);
    }
}

The final use:

internal class LoggingTest
{
    private static readonly ILogHelperLogger Logger = LogHelper.GetLogger<LoggingTest>();

    public static void MainTest()
    {
        var abc = "1233";
        LogHelper.ConfigureLogging(builder =>
                                   {
                                       builder
                                           .AddLog4Net()
                                           //.AddSerilog(loggerConfig => loggerConfig.WriteTo.Console())
                                           .WithMinimumLevel(LogHelperLogLevel.Info)
                                           .WithFilter((category, level) => level > LogHelperLogLevel.Error && category.StartsWith("System"))
                                           .EnrichWithProperty("Entry0", ApplicationHelper.ApplicationName)
                                           .EnrichWithProperty("Entry1", ApplicationHelper.ApplicationName, e => e.LogLevel >= LogHelperLogLevel.Error)// 当 LogLevel 是 Error 及以上级别时才增加 Property
                                           ;
                                   });

        Logger.Debug("12333 {abc}", abc);
        Logger.Trace("122334334");
        Logger.Info($"122334334 {abc}");

        Logger.Warn("12333, err:{err}", "hahaha");
        Logger.Error("122334334");
        Logger.Fatal("12333");
    }
}

More

Increase LoggingEventalso want to do a batch commit log, as defined above PeriodBatchingConfig, as batch synchronization to Provider but the actual use down, some time to set the log provider does not support internal time is recorded, so time is not allowed to log the , and most do not support batch write the log, so the back to give up, but if you just use your own extensions without external logging framework like log4net of it, I think it can be done, can improve efficiency, mainly used Serilogand log4net, not being updated, on first so be it

The next version to fix things

• ILogProvider Logging Task returns a sense of some tasteless, did not make much sense, and then change it back now
• SerilogThe Filter is based LogEvent, the need to change it back to see whether, based on the words of LogEvent more concise, and can do filtering based on Properties in the LogEvent, so the API can be updated at AddFilterAddFilter(Func<LogHelperLoggingEvent, bool> filter)

Reference

• https://github.com/serilog/serilog
• https://github.com/serilog/serilog-sinks-periodicbatching
• https://github.com/aspnet/Logging
• https://github.com/aspnet/Extensions/tree/master/src/Logging
• https://github.com/WeihanLi/WeihanLi.Common/tree/dev/src/WeihanLi.Common/Logging