Building Semantic Logging Library as Out of Process Service

What is Semantic Logging Application Block or SLAB

The Semantic Logging Application Block is a framework for capturing and manipulating events raised by applications, and storing the typed and structured information they contain in log files or other logging stores. It takes advantage of features of the .NET Framework (version 4.5 and above) and Event Tracing for Windows (ETW). Here's an extract from MSDN:

ETW is a fast, lightweight, strongly typed, extensible logging system that is built into the Windows operating system.

The Semantic Logging Application Block consumes events raised by event sources, and provides features to help you sample, filter, correlate, format, and store these events in a wide range of target logging stores. The block can be used both in-process within your applications, or out-of-process to capture events generated by one or more processes.

How does the SLAB work

The process by which events messages are passed to the event sinks depends on whether you are using the block just to collect events within a single application, or to capture and log events from more than one application (including applications running in different locations on the network). The Semantic Logging Application Block can be used in two ways:

• In-process: In this scenario, you just need to collect events from within a single application. The event sinks run in the same process as the application and subscribe to events exposed by a trace listener that is part of the application block.
  
  
• Out-of-process: In this scenario, you want to maximize logging throughput and improve the reliability of logging should the application fail. Your custom event source running within the application writes events to the ETW infrastructure. However, your event sinks run within a separate logging application and subscribe to the events exposed by a trace event service, which is notified of events by ETW.

Read more information of SLAB introduction and its design.

What we will learn

1. Creating class library and adding SLAB packages to it.
2. Creating “EveryLogSource” custom EventSource class.
3. Adding Event Sinks for database.
4. Setting SLAB Out of Process logging.
5. Consuming SLAB library in Console application to check logging.

Creating class library and adding slab packages to it

Create C# class library under Infrastructure Solution Folder with name “EveryLogSource”.

Using Nuget Package manager, install the following package: “Semantic Logging Application Block”. This will install, add reference to “EnterpriseLibrary.SemanticLogging” DLL and installs NewtonSoft.Json. Wondering why it installs JSON package because we can convert logging information to JSON format.



Creating “EVERYLOGSOURCE” custom event source class

1. Create a folder “Events”, in that create class “EveryLoggerSource” which inherits EventSource class of System.Diagnotics.Tracing. Give an attribute name as “AdvWrksLogs” as shown below:
   

   using System.Diagnostics.Tracing;
   namespace EveryLogSource.Events
   {
       [EventSource(Name = "AdvWrksLogs")]
       public class EveryLoggerSource: EventSource
       {}
   }

2. Create another folder “LogTypes”, inside it create class “LogType”. Add the following code in it. We can add more classes like Keywords, Tasks, OpCodes if we are creating more strongly typed logging framework.
   

   using System.Diagnostics.Tracing;
   namespace EveryLogSource.LogTypes
   {
       public class GlobalType
       {
           public const int GlobalInformational = 1;
           public const int GlobalCritical = 2;
           public const int GlobalError = 3;
           public const int GlobalLogAlways = 4;
           public const int GlobalVerbose = 5;
           public const int GlobalWarning = 6;
       }
   }

3. Copy the following code in “EventLoggerSource” class. Its work is to write the logging to ETW either as Critical, Informational, Error, LogAlways, Verbose and Warning.
   

   using EveryLogSource.LogTypes;
   using System.Diagnostics.Tracing;
   namespace EveryLogSource.Events
   {
       [EventSource(Name = "AdvWrksLogs")]
       public class EveryLoggerSource: EventSource
       {
           public static readonlyEveryLoggerSource Log = new EveryLoggerSource();
           [Event(GlobalType.GlobalCritical, Message = "Global Critical: {0}", Level = EventLevel.Critical)]
           public void Critical(string message)
           {
               if (IsEnabled()) WriteEvent(GlobalType.GlobalCritical, message);
           }
           [Event(GlobalType.GlobalError, Message = "Global Error {0}", Level = EventLevel.Error)]
           public void Error(string message)
           {
               if (IsEnabled()) WriteEvent(GlobalType.GlobalError, message);
           }
           [Event(GlobalType.GlobalInformational, Message = "Global Informational {0}", Level = EventLevel.Informational)]
           public void Informational(string message)
           {
               if (IsEnabled()) WriteEvent(GlobalType.GlobalInformational, message);
           }
           [Event(GlobalType.GlobalLogAlways, Message = "Global LogAlways {0}", Level = EventLevel.LogAlways)]
           public void LogAlways(string message)
           {
               if (IsEnabled()) WriteEvent(GlobalType.GlobalLogAlways, message);
           }
           [Event(GlobalType.GlobalVerbose, Message = "Global Verbose {0}", Level = EventLevel.Verbose)]
           public void Verbose(string message)
           {
               if (IsEnabled()) WriteEvent(GlobalType.GlobalVerbose, message);
           }
           [Event(GlobalType.GlobalWarning, Message = "Global Warning {0}", Level = EventLevel.Warning)]
           public void Warning(string message)
           {
               if (IsEnabled()) WriteEvent(GlobalType.GlobalWarning, message);
           }
       }
   }

Since the Event source class must be Singleton in application, it’s required to create wrapper kind of class whenever we create EventSource class.

Let’s create C# class file in folder “Events” and name it “EveryLoggerSourceWrapper". Copy the following code in this class.

using EveryLogSource.LogTypes;
using System;
using System.Collections.Generic;
namespace EveryLogSource.Events
{
    public class EveryLoggerSourceWrapper
    {
        public void RegisterLogger(Dictionary < int, Action < string >> exectueLogDict)
        {
            exectueLogDict.Add(GlobalType.GlobalCritical, Critical);
            exectueLogDict.Add(GlobalType.GlobalError, Error);
            exectueLogDict.Add(GlobalType.GlobalInformational, Informational);
            exectueLogDict.Add(GlobalType.GlobalLogAlways, LogAlways);
            exectueLogDict.Add(GlobalType.GlobalVerbose, Verbose);
            exectueLogDict.Add(GlobalType.GlobalWarning, Warning);
        }
        public void Critical(string message)
        {
            EveryLoggerSource.Log.Critical(message);
        }
        public void Error(string message)
        {
            EveryLoggerSource.Log.Error(message);
        }
        public void Informational(string message)
        {
            EveryLoggerSource.Log.Informational(message);
        }
        public void LogAlways(string message)
        {
            EveryLoggerSource.Log.LogAlways(message);
        }
        public void Verbose(string message)
        {
            EveryLoggerSource.Log.Verbose(message);
        }
        public void Warning(string message)
        {
            EveryLoggerSource.Log.Warning(message);
        }
    }
}

With thoughts of using IoC containers or just plain Dependency Injections, lets create interface “IEveryLogger” with one simple Log method which takes EventId (integer) and logging message.

namespace EveryLogSource
{
    public interface IEveryLogger
    {
        void Log(int log, stringLogMessages);
    }
}

Now we have Event Source class, its wrapper classes, Event Types and interface. Let’s implement this interface in logger class implementations so that we have used SLAB logging code we developed here.

This implementation is a simple logging strategy, if we need more strongly typed then we create Event Source, Wrap and use this interface so that any client will consume based on its requirement.

Create C# class “EveryLogger.cs”, implement “IEveryLogger” interface and copy this code:

using EveryLogSource.Events;
using EveryLogSource.LogTypes;
using System;
using System.Collections.Generic;
namespace EveryLogSource
{
    public class EveryLogger: IEveryLogger
    {
        private readonlyDictionary < int, Action < string >> _LogDict = newDictionary < int, Action < string >> ();
        public EveryLogger()
        {
            varlogSourceWrapper = newEveryLoggerSourceWrapper();
            logSourceWrapper.RegisterLogger(_LogDict);
        }
        public void Log(int log, stringLogMessages)
        {
            if (_LogDict.ContainsKey(log))
            {
                _LogDict[log].Invoke(LogMessages);
                return;
            }
            _LogDict[GlobalType.GlobalWarning].Invoke(LogMessages);
        }
    }
}

Adding event sinks for database

The Semantic Logging Application Block includes sinks that enable you to write log messages to the Console, and save log messages to a database, a Microsoft Azure storage table, and to flat files.

The following event sinks are available as part of the Semantic Logging Application Block: 

• Console event sink: This event sink writes formatted log entries to the console.
  
  
• Flat File event sink: This event sink writes log entries to a text file.
  
  
• Rolling Flat File event sink: This event sink writes log entries to a text file and creates a new log file depending on the current log file age and/or size.
  
  
• SQL Database event sink: Azure Table Storage event sink. This event sink writes log entries to Azure table storage. By default, the sink buffers the log messages.
  
  
• SQL Database Event Sink: This event sink writes log entries to SQL Server or Azure SQL database. By default, the sink buffers the log messages.

Using the “Manage NuGet packages”, install the “Semantic Logging – Sql Server Database Sink”.

Let’s examine what gets installed after this:

• SemanticLogging.Database – Persisting the logging into SQL Server database.
• TransientFaultHandling – Retry logic mechanisms for more resilient logging.

In the folder “EnterpriseLibrary.SemanticLogging.Database.2.0.1406.1”, we found SQL scripts to create a database for persisting all the logging information. It’s highly recommended to run these scripts in SQL server either manually or through batch file provided (please check connection details before running).



Setting SLAB out of process logging

Till now we have created class library for generic logging using SLAB, install database to store all the logging information. It’s time to create OUT OF PROCESS setting for semantic logging.

• Out of process enables us to keep the logging infrastructure out application being used.
• Multiple applications can log information just by adding Event Source without restarting anything.
• It can be long running either as console or Windows Services. We will run that as windows services.
• We can use multiple sinks for same applications i.e. logging information can be stored in files as well as database.

Install OUT OF PROCESS for SLAB using the Nuget packages. After installation if you just observe packages folder (this folder contains all Nuget packages). We can see folder “EnterpriseLibrary.SemanticLogging.Service.2.0.1406.1”, go to Tools folder and open SemanticLogging-svc.xml.

SemanticLogging-svc.xml – Entries for sinks details, filters and formatters.

As we want to log information to store in database we need to add database sink details. Check this for reference:

<?xml version="1.0" encoding="utf-8" ?>
    <configuration xmlns="http://schemas.microsoft.com/practices/2013/entlib/semanticlogging/etw" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://schemas.microsoft.com/practices/2013/entlib/semanticlogging/etw SemanticLogging-svc.xsd">
        <!-- Optional settings for fine tuning performance and Trace Event Session identification-->
        <traceEventService/>
        <!-- Sinks reference definitons used by this host to listen ETW events -->
        <sinks>
            <sqlDatabaseSink name="logDb" instanceName="AdvWrksLogging" connectionString="Data Source=.;Initial Catalog=Logging;Integrated Security=True">
                <sources>
                    <!-- The name attribute is from the EventSource.Name Property -->
                    <eventSource name="AdvWrksLogs" level="LogAlways" /> </sources>
            </sqlDatabaseSink>
        </sinks>
    </configuration>

SLAB needs more additional files for working as OUT OF PROCESS, to accomplish this it has provided “install-packages.ps1”, a power shell script for installing all dependencies for it to run properly.

Just open Power Shell command prompt, run the “install-packages.ps1” in command line. This will install all dependencies for us to run SLAB as Out of Process. Refer below image.

Sometimes script execution is blocked, run the command (red line indicated) to override restriction. Ensure you revert back the settings. Refer using the Set-ExecutionPolicy.



Once everything is installed using powershell script, we are ready to host SLAB as an OUT OF PROCESS. We can run it in CONSOLE or as WINDOWS SERVICE.

We will run it as console window here, running it as windows service is fairly simple. 

• As Console: SemanticLogging-svc.exe –console.
• As Windows Service: SemanticLogging-svc.exe –service.

We will see similar console display if we succeed in it.



Consuming SLAB library in console application to check logging

Now that we have SLAB out of process running which consumes “AdvWrksLogs” Event source, it’s time to consume that in console application, we can do this in any application like ASP.NET MVC, WPF,  etc.

Just copy these files after you create dummy console application to test. Make sure you reference “EveryLog” class library and Semantic logging DLL.

using System;
using EveryLogSource;
using EveryLogSource.LogTypes;
namespace AdWrksConsoleTest
{
    class Program
    {
        static void Main(string[] args)
        {
            IEveryLogger _logger = newEveryLogger();
            _logger.Log(GlobalType.GlobalInformational, "Hello World");
            _logger.Log(GlobalType.GlobalError, "This is an ERROR !!");
            _logger.Log(GlobalType.GlobalWarning, "Hello World, This a WARNING");
            Console.ReadLine();
        }
    }
}

It’s time to check in Logging database, if we have got logging information. Open “Logging” database, query traces table. We would get these rows.



Note from MSDN

In the out-of-process scenario, both the application that generates log messages and the Out-of-Process Host application that collects the messages must run on the same physical computer. In a distributed application, you must install the Out-of-Process Host application on every computer. To collate log messages in this scenario, each instances of the Out-of-Process host application can write them to a suitable single destination such as a database, Azure table storage, or—if you create or obtain suitable custom extensions for the block—to other stores such as ElasticSearch and Splunk.