Plug-in from scratch to achieve the development of ASP.NET Core MVC (6) - how to load plug-references

Title: ASP.NET Core MVC from scratch to achieve the development of plug-in (f) - how to load plug-reference.
Author: Lamond Lu
Address: https://www.cnblogs.com/lwqlun/p/11717254.html
Source: https://github.com/lamondlu/DynamicPlugins

Prospects Review

• ASP.NET Core MVC from scratch to achieve the development of plug-in (a) - Use Application Part dynamic load controllers and views
• ASP.NET Core MVC from scratch to achieve the development of plug-in (two) - How to create a project template
• ASP.NET Core MVC from scratch to achieve plug-in development (c) - how to enable components at runtime
• ASP.NET Core MVC from scratch to achieve the development of plug-in (four) - plug-in installation
• ASP.NET Core MVC from scratch to achieve the development of plug-in (e) - use AssemblyLoadContext upgrade and remove plug-ins

Brief introduction

In the first one, let us demonstrate how to use .NET Core 3.0 introduced a new AssemblyLoadContextupgrade to achieve operation and remove the plug. After completing this post, I got a lot of feedback park Friends, I am glad that so many people can be involved, I will be based on your feedback to improve the project. Benpian it, I will solve the major problems loading the plug-referenced, this feedback is being asked the most questions.

Problem use case

Before doing plug-in, we do is very, very simple function, does not introduce any third-party libraries. But under normal circumstances, will quote some third-party libraries plug-ins we created more or less, so let's try to use our previous project, the use of third-party program loads a set, and see what the results obtained .

To simulate this, I created a new Class Library project DemoReferenceLibrary, and before the DemoPlugin1reference to the project in DemoReferenceLibrarythe project.

In the DemoReferenceLibrarymiddle, I created a new class Demo.cs file, its code is as follows:

    public class Demo
    {
        public string SayHello()
        {
            return "Hello World. Version 1";
        }
    }

This is simply by SayHelloway of a return string.

Then in the DemoPlugin1project, we modify previously created Plugin1Controller, from Democlass by SayHelloa method to get the string to be displayed on the page.

    [Area("DemoPlugin1")]
    public class Plugin1Controller : Controller
    {
        public IActionResult HelloWorld()
        {
            var content = new Demo().SayHello();
            ViewBag.Content = content;
            return View();
        }
    }

Finally, we look at the plug-packaged and re-install it into the system, after a visit to the plug-route, you will get the following error.

Here is the problem encountered by most of the students can not load the assembly DemoReferenceLibrary.

How to load plugins references?

The reason for this is very simple, that is, when by AssemblyLoadContextthe time the assembly is loaded, we only load the plug-in assembly, the assembly did not load it references.

For example, our DemoPlugin1example, this plug-in directory is as follows

In this directory, in addition to our well-known DemoPlugin1.dll, DemoPlugin1.Views.dlloutside, there is a DemoReferenceLibrary.dllfile. This file is loaded when we did not enable plug-ins to the current AssemblyLoadContextin it when accessing the plug-routing, the system can not find the dll file this component.

Why Mystique.Core.dll, System.Data.SqlClient.dll, Newtonsoft.Json.dllthe DLL will not be a problem it?

There are two in the .NET Core LoadContext. One is before us AssemblyLoadContext, it is a custom LoadContext. Another is the system default DefaultLoadContext. When a .NET Core application startup, and will create a reference DefaultLoadContext.

If not specified LoadContext, the system will default assemblies are loaded into DefaultLoadContextthe. Here we can look at our main site project, which we have quoted Mystique.Core.dll, System.Data.SqlClient.dll, Newtonsoft.Json.dll.

In .NET Core design documentation for the assembly loading such a description

If the assembly was already present in A1's context, either because we had successfully loaded it earlier, or because we failed to load it for some reason, we return the corresponding status (and assembly reference for the success case).

However, if C1 was not found in A1's context, the Load method override in A1's context is invoked.

• For Custom LoadContext, this override is an opportunity to load an assembly before the fallback (see below) to Default LoadContext is attempted to resolve the load.
• For Default LoadContext, this override always returns null since Default Context cannot override itself.

Here simple terms, means that when a custom LoadContexttime to load an assembly, if the assembly can not be found, the program will automatically default to LoadContextfind, if the default LoadContextcan not be found, it will return null.

Thus, the question before us is solved, here precisely because the primary site is already loaded assembly required, although the plugin AssemblyLoadContextcan not find this assembly, the program still can default LoadContextto load the assembly.

It is not really no issue of it?

In fact, I do not really recommend using more than a way to load third-party assembly. There are two main reasons

• Different plug-ins can reference different versions of third-party assembly, may implement different versions of third-party assembly. The default LoadContextcan only load one version, resulting in a total reference feature a plug-in that assembly failure.
• The default LoadContextmay be set to load third-party programs and other plug-ins are different, lead to other plug-in function reference function of the assembly failure.

So here is the most correct way, or give up using the default LoadContextload an assembly, to ensure that each plug AssemblyLoadContextassembly required are fully loaded.

So how do these third-party assemblies loaded it? Here we introduce two ways

• Original way
• Use plug-in cache

Original way

Compare the original way of violence, we can choose to load the plug assembly, to load an assembly where all the dll files in the directory.

Here we first created a reference library loader plug-in interface IReferenceLoader.

    public interface IRefenerceLoader
    {
        public void LoadStreamsIntoContext(CollectibleAssemblyLoadContext context, 
            string folderName, 
            string excludeFile);
    }

Then we create a default reference library loader plug-in DefaultReferenceLoader, its code is as follows:

    public class DefaultReferenceLoader : IRefenerceLoader
    {
        public void LoadStreamsIntoContext(CollectibleAssemblyLoadContext context, 
            string folderName, 
            string excludeFile)
        {
            var streams = new List<Stream>();
            var di = new DirectoryInfo(folderName);
            var allReferences = di.GetFiles("*.dll").Where(p => p.Name != excludeFile);

            foreach (var file in allReferences)
            {
                using (var sr = new StreamReader(file.OpenRead()))
                {
                    context.LoadFromStream(sr.BaseStream);
                }
            }
        }
    }

Code explanation

• Here I was to exclude the currently loaded plug-in assembly, so add a excludeFileparameter.
• folderNameThat is the current directory plug-ins, where we pass DirectoryInfothe class GetFilesmethod to get the currently specified folderNameall the dll files in the directory.
• I'm still here to load third-party plug-ins required for the assembly by way of the file stream.

After completion of the above code, we also need to modify the code to enable plug-ins in two parts

• [MystiqueStartup.cs] - When the program starts, into the IReferenceLoaderservice, to enable plug-ins
• [MvcModuleSetup.cs] - in the plugin admin page to enable plug-trigger operation

MystiqueStartup.cs

    public static void MystiqueSetup(this IServiceCollection services, IConfiguration configuration)
    {

        ...
            
        services.AddSingleton<IReferenceLoader, DefaultReferenceLoader>();

        var mvcBuilder = services.AddMvc();

        var provider = services.BuildServiceProvider();
        using (var scope = provider.CreateScope())
        {
            ...

            foreach (var plugin in allEnabledPlugins)
            {
                var context = new CollectibleAssemblyLoadContext();
                var moduleName = plugin.Name;
                var filePath = $"{AppDomain.CurrentDomain.BaseDirectory}Modules\\{moduleName}\\{moduleName}.dll";
                var referenceFolderPath = $"{AppDomain.CurrentDomain.BaseDirectory}Modules\\{moduleName}";

                _presets.Add(filePath);
                using (var fs = new FileStream(filePath, FileMode.Open))
                {
                    var assembly = context.LoadFromStream(fs);
                    loader.LoadStreamsIntoContext(context, 
                          referenceFolderPath,
                          $"{moduleName}.dll");

                   ...
                }
            }
        }

        ...
    }

MvcModuleSetup.cs

    public void EnableModule(string moduleName)
    {
        if (!PluginsLoadContexts.Any(moduleName))
        {
            var context = new CollectibleAssemblyLoadContext();

            var filePath = $"{AppDomain.CurrentDomain.BaseDirectory}Modules\\{moduleName}\\{moduleName}.dll";
            var referenceFolderPath = $"{AppDomain.CurrentDomain.BaseDirectory}Modules\\{moduleName}";
            using (var fs = new FileStream(filePath, FileMode.Open))
            {
                var assembly = context.LoadFromStream(fs);
                _referenceLoader.LoadStreamsIntoContext(context, 
                      referenceFolderPath, 
                      $"{moduleName}.dll");

                ...
            }
        }
        else
        {
            var context = PluginsLoadContexts.GetContext(moduleName);
            var controllerAssemblyPart = new MystiqueAssemblyPart(context.Assemblies.First());
            _partManager.ApplicationParts.Add(controllerAssemblyPart);
        }

        ResetControllActions();
    }

Now before we re-run the project, and the access routes plug 1, you'll find the page displayed properly, and page content but also from the DemoReferenceLibraryloader out of focus.

Use plug-in cache

Although the original way can help us successfully loaded plug-reference the assembly, but it is not efficient, if the plug-in 1 and 2 refer to the same plug assembly, when the plug 1 AssemblyLoadContextafter loading all reference assemblies, plug-ins will plug 1 2 've done repeat. This is not what we want, we want if multiple plug-ins at the same time using the same set of programs you do not need to repeat read dll files.

How to avoid duplicate dll file to read it? Here we can use a static dictionary to cache file stream information to avoid duplication reading dll file.

If you feel in using a static dictionary in ASP.NET Core MVC stream information in the file to cache unsafe, you can switch to other caches way here just for a simple demonstration.

Here we first create a reference to an interface assembly cache container IReferenceContainer, its code is as follows:

    public interface IReferenceContainer
    {
        List<CachedReferenceItemKey> GetAll();

        bool Exist(string name, string version);

        void SaveStream(string name, string version, Stream stream);

        Stream GetStream(string name, string version);
    }

Code explanation

• GetAllAll methods will follow the use, the system used to obtain the reference assemblies loaded
• ExistThe method of determining the specified version of the assembly if there is a file stream
• SaveStreamIs the designated version of an assembly file stream stored into the static dictionary
• GetStreamIs pulling specified version of the assembly from a static dictionary file stream

Then we can create a reference to the default implementation assembly cache container DefaultReferenceContainerclass, its code is as follows:

    public class DefaultReferenceContainer : IReferenceContainer
    {
        private static Dictionary<CachedReferenceItemKey, Stream> _cachedReferences = new Dictionary<CachedReferenceItemKey, Stream>();

        public List<CachedReferenceItemKey> GetAll()
        {
            return _cachedReferences.Keys.ToList();
        }

        public bool Exist(string name, string version)
        {
            return _cachedReferences.Keys.Any(p => p.ReferenceName == name
                && p.Version == version);
        }

        public void SaveStream(string name, string version, Stream stream)
        {
            if (Exist(name, version))
            {
                return;
            }

            _cachedReferences.Add(new CachedReferenceItemKey { ReferenceName = name, Version = version }, stream);
        }

        public Stream GetStream(string name, string version)
        {
            var key = _cachedReferences.Keys.FirstOrDefault(p => p.ReferenceName == name
                && p.Version == version);

            if (key != null)
            {
                _cachedReferences[key].Position = 0;
                return _cachedReferences[key];
            }

            return null;
        }
    }

This class is relatively simple, I do not do too much explanation.

After completing the referral cache container, I modified the previously created IReferenceLoaderinterface and default implementation DefaultReferenceLoader.

    public interface IReferenceLoader
    {
        public void LoadStreamsIntoContext(CollectibleAssemblyLoadContext context, string moduleFolder, Assembly assembly);
    }

    public class DefaultReferenceLoader : IReferenceLoader
    {
        private IReferenceContainer _referenceContainer = null;
        private readonly ILogger<DefaultReferenceLoader> _logger = null;

        public DefaultReferenceLoader(IReferenceContainer referenceContainer, ILogger<DefaultReferenceLoader> logger)
        {
            _referenceContainer = referenceContainer;
            _logger = logger;
        }

        public void LoadStreamsIntoContext(CollectibleAssemblyLoadContext context, string moduleFolder, Assembly assembly)
        {
            var references = assembly.GetReferencedAssemblies();

            foreach (var item in references)
            {
                var name = item.Name;

                var version = item.Version.ToString();

                var stream = _referenceContainer.GetStream(name, version);

                if (stream != null)
                {
                    _logger.LogDebug($"Found the cached reference '{name}' v.{version}");
                    context.LoadFromStream(stream);
                }
                else
                {

                    if (IsSharedFreamwork(name))
                    {
                        continue;
                    }

                    var dllName = $"{name}.dll";
                    var filePath = $"{moduleFolder}\\{dllName}";

                    if (!File.Exists(filePath))
                    {
                        _logger.LogWarning($"The package '{dllName}' is missing.");
                        continue;
                    }

                    using (var fs = new FileStream(filePath, FileMode.Open))
                    {
                        var referenceAssembly = context.LoadFromStream(fs);

                        var memoryStream = new MemoryStream();

                        fs.Position = 0;
                        fs.CopyTo(memoryStream);
                        fs.Position = 0;
                        memoryStream.Position = 0;
                        _referenceContainer.SaveStream(name, version, memoryStream);

                        LoadStreamsIntoContext(context, moduleFolder, referenceAssembly);
                    }
                }
            }
        }

        private bool IsSharedFreamwork(string name)
        {
            return SharedFrameworkConst.SharedFrameworkDLLs.Contains($"{name}.dll");
        }
    }

Code explanation:

• Here LoadStreamsIntoContextprocess assemblyparameters, i.e., the current plug-in assembly.
• Here I pass GetReferencedAssembliesmethod, get all the assembly plug-in referenced assemblies.
• If the referenced assemblies do not exist in a reference vessel, we are loading it with a file stream and save it to a reference container, if the referenced assembly already exists in the reference container, it is loaded directly into the current plug- AssemblyLoadContextin. Here To test the effect, if the assembly from the cache, I use the log component outputs a log.
• Because the plug assembly references, may have come from Shared Framework, this assembly is no need to load, so I chose to skip the loading of these assemblies. (Here I have not considered the case of Self-Contained released, there may be subsequent changes)

Finally, we need to modify MystiqueStartup.csand MvcModuleSetup.csplug-in code is enabled.

MystiqueStartup.cs

    public static void MystiqueSetup(this IServiceCollection services, IConfiguration configuration)
    {

        ...
        services.AddSingleton<IReferenceContainer, DefaultReferenceContainer>();
        services.AddSingleton<IReferenceLoader, DefaultReferenceLoader>();
        ...

        var mvcBuilder = services.AddMvc();

        var provider = services.BuildServiceProvider();
        using (var scope = provider.CreateScope())
        {
            ...

            foreach (var plugin in allEnabledPlugins)
            {
                ...
               
                using (var fs = new FileStream(filePath, FileMode.Open))
                {
                    var assembly = context.LoadFromStream(fs);
                    loader.LoadStreamsIntoContext(context, referenceFolderPath, assembly);

                    ...
                }
            }
        }

        ...
    }

MvcModuleSetup.cs

    public void EnableModule(string moduleName)
    {
        if (!PluginsLoadContexts.Any(moduleName))
        {
            ...
            using (var fs = new FileStream(filePath, FileMode.Open))
            {
                var assembly = context.LoadFromStream(fs);
                _referenceLoader.LoadStreamsIntoContext(context, referenceFolderPath, assembly);
               ...
            }
        }
        else
        {
            ...
        }

        ResetControllActions();
    }

After the completion of the code, to test the effect, I created another plugin DemoPlugin2, the project code and DemoPlugin1basically the same. When the program starts, you will find DemoPlugin2reference assemblies used are loaded from the cache, but DemoPlugin2the routing can be normal access.

Add a page to load third-party display assembly

In order to show what the system here which programs are loading set, I added a new page Assembilies, the page is to call the IReferenceContainerinterface defined in the GetAllmethod, shows the static dictionary, all assemblies loaded.

Results are as follows:

Several test scenarios

Finally, after the completion of the preparation of the code above function, we use the following scenarios to test, take a look at AssemblyLoadContextthe power of our offer.

scene 1

2 plug-ins, a reference DemoReferenceLibraryversion 1.0.0.0, another reference to DemoReferenceLibrarythe 1.0.1.0 version. 1.0.0.0 version of which, SayHellothe string method returns "Hello World. Version 1", 1.0.1.0 version, SayHellothe string method returns "Hello World. Version 2".

Start the project, install the plug 1 and plug 2, respectively, to run plug-ins and plug-ins 1 Route 2, you'll get different results. This means AssemblyLoadContextfor us to do good isolation, plug-ins and plug-ins 2 1 Although references to different versions of the same plug-in, but did not affect each other.

Scene 2

When the two plug-ins use the same third-party libraries, and load is complete, disable the plug 1. Although they referenced assembly the same, but you will find plug-ins can be accessed or 2, which shows the plug 1 of the AssemblyLoadContextrelease of the plug-2 AssemblyLoadContextdid not affect.

to sum up

Benpian I showed how to solve the problem plug-loading referenced assemblies for everyone, where we explained in two ways, original mode and cache mode. The net effect of these two methods, although the same, but the way the cache efficiency is significantly higher. I will follow-up based on feedback, continue to add new content, we stay tuned.