OSC invites you to party! 1028 Suzhou Yuan Founding Association, let’s hunt for treasures in the AI era

Preface

This article will introduce from the source code perspective how SpringBoot customizes ClassLoader to load FatJar, and how FatJar reads classes in nested jars in the jar. Some knowledge of ClassLoader will be involved in the process.

Class loader basics

Parental delegation mechanism

Parental delegation means that when loading a class, it is loaded first by the parent class loader. If the parent class loader cannot load in the loading directory, then it is loaded by the child loader. The purpose of this is to ensure that the same class is only loaded once.

BootStrap Classloader: Starts the class loader, mainly loading the core class library
Extension ClassLoader: extension class loader, under java.ext.dirs or jre/lib/ext
Application ClassLoader: AppClassLoader, application class loader, ordinary Java startup project is loaded by AppClassLoader

<p>Some containers are based on AppClassLoader to extend custom ClassLoader, with AppClassLoader as the parent class loader. For example, Tomcat uses WebAppClassLoader, and SpringBoot uses LaunchedURLClassLoader. </p>

In addition, our customized ClassLoader is generally extended based on AppClassLoader.

Parental delegation refers to the ClassLoader object (ClassLoader instance), not the inheritance of the ClassLoader class. It is an object member that implements parental delegation. The source code loadClass method of the ClassLoader loading class is as follows:

protected Class<?> loadClass(String name, boolean resolve)
        throws ClassNotFoundException
    {
        synchronized (getClassLoadingLock(name)) {
            // 加载过的类不再加载
            Class<?> c = findLoadedClass(name);
            if (c == null) {
                long t0 = System.nanoTime();
                //从父类加载器加载
                try {
                    if (parent != null) {
                        c = parent.loadClass(name, false);
                    } else {
                        c = findBootstrapClassOrNull(name);
                    }
                } catch (ClassNotFoundException e) {
                }

                if (c == null) {
                    //父类加载器找不到，调用findClass方法
                    long t1 = System.nanoTime();
                    c = findClass(name);

                    // this is the defining class loader; record the stats
                    sun.misc.PerfCounter.getParentDelegationTime().addTime(t1 - t0);
                    sun.misc.PerfCounter.getFindClassTime().addElapsedTimeFrom(t1);
                    sun.misc.PerfCounter.getFindClasses().increment();
                }
            }
            if (resolve) {
                resolveClass(c);
            }
            return c;
        }
    }

First, determine whether the class has been loaded. If not, the parent class will load it. If the parent class cannot be loaded, the findClass method will be called.

URLClassLoader

As its name suggests, the URLClassLoader class is mainly used to load some URL resources such as classes and jar packages. You can reuse URLClassLoader to extend your own ClassLoader. In fact, Tomcat and SpringBoot also do this.

What is the URL?

Refers to a resource, including but not limited to class files, jar packages, and even customized files.

How does URL handle byte stream?

Since the URL can be any file or resource, the way to read the file is also different.

When we open the URL.openConnection function, we can see that the logic is processed by the URLStreamHandler handler.

    public URLConnection openConnection() throws java.io.IOException {
        return handler.openConnection(this);
    }

In this way, the extended URL can implement the custom loading logic of the URL by rewriting the URLStreamHandler class. In fact, SpringBoot's FatJAR also does this.

There is a getInputStream function in URLConnection

public InputStream getInputStream() throws IOException;

What is returned is a binary byte stream. With InputStream, resources and classes can be constructed.

The ins and outs of URLClassLoader

URLClassLoader is a derived class of ClassLoader. Jar packages can be loaded through addURL.

    protected void addURL(URL url) {
        ucp.addURL(url);
    }

And ucp is URLClassPath, which manages all added URLs.

URLClassLoader complies with the parent delegation mechanism and overrides the findClass method to find Class.

    protected Class<?> findClass(final String name)
        throws ClassNotFoundException
    {
        final Class<?> result;
        try {
            result = AccessController.doPrivileged(
                new PrivilegedExceptionAction<Class<?>>() {
                    public Class<?> run() throws ClassNotFoundException {
                        String path = name.replace('.', '/').concat(".class");
                        Resource res = ucp.getResource(path, false);
                        if (res != null) {
                            try {
                                return defineClass(name, res);
                            } catch (IOException e) {
                                throw new ClassNotFoundException(name, e);
                            }
                        } else {
                            return null;
                        }
                    }
                }, acc);
        } catch (java.security.PrivilegedActionException pae) {
            throw (ClassNotFoundException) pae.getException();
        }
        if (result == null) {
            throw new ClassNotFoundException(name);
        }
        return result;
    }

The resource class is obtained from Resource res = ucp.getResource(path, false);, ucp is the URLClassPath above.

Go into the ucp.getResource method and you can see the implementation details of getResource, which is the loading of the URL I mentioned above.

The principle of SpringBoot loading FatJar

FatJar structure

Unzip the jar package of a SpringBoot project and you can see the FatJar structure.

BOOT-INF is the file packaged by the project, BOOT-INF/classes stores the project code and configuration files, and BOOT-INF/lib stores the dependent nested jar packages (i.e. jar in jar).
org.springframework.boot. * is the SpringBoot startup class

Open META-INF/MANIFEST.MF and you can see the startup class of the jar package

Manifest-Version: 1.0
Implementation-Title: LittleBoy
Implementation-Version: 1.0-SNAPSHOT
Archiver-Version: Plexus Archiver
Built-By: liubs
Implementation-Vendor-Id: com.thanple.little.boy
Spring-Boot-Version: 1.5.1.RELEASE
Implementation-Vendor: Pivotal Software, Inc.
Main-Class: org.springframework.boot.loader.JarLauncher
Start-Class: com.thanple.little.boy.web.WebApplication
Spring-Boot-Classes: BOOT-INF/classes/
Spring-Boot-Lib: BOOT-INF/lib/
Created-By: Apache Maven 3.6.3
Build-Jdk: 1.8.0_191
Implementation-URL: http://projects.spring.io/spring-boot/LittleBoy/

Main-Class is the startup class of the Jar package, org.springframework.boot.loader.JarLauncher. Start-Class is the main function for starting the project you write.

FatJar is not a standard jar package structure. Only org.springframework.boot conforms to the jar package standard. Everything in the BOOT-INF directory cannot be loaded, so SpringBoot needs a custom loader.

SpringBoot starts the loading class process

The org.springframework.boot.loader.JarLauncher mentioned just now is the real startup main function of SpringBoot. What is the difference between it and the locally written main function class (assumed to be WebApplication)? The ordinary local main function, without nested Jar, is directly loaded and started by AppClassLoader, while SpringBoot writes its own set of class loaders, LaunchedURLClassLoader.

The main function of JarLauncher serves as the entry point, and the Launcher.launch function is called when creating an instance.

public static void main(String[] args) throws Exception {
    new JarLauncher().launch(args);
}

When you enter Launcher.launch, you can see that a LaunchedURLClassLoader is created, and the LaunchedURLClassLoader loads the classes and jars nested in FatJar (that is, jar in jar).

    protected void launch(String[] args) throws Exception {
        if (!this.isExploded()) {
            JarFile.registerUrlProtocolHandler();
        }
        //加载URL，由LaunchedURLClassLoader加载
        ClassLoader classLoader = this.createClassLoader(this.getClassPathArchivesIterator());
        String jarMode = System.getProperty("jarmode");
        String launchClass = jarMode != null && !jarMode.isEmpty() ? "org.springframework.boot.loader.jarmode.JarModeLauncher" : this.getMainClass();
        this.launch(args, launchClass, classLoader);
    }
    //创建LaunchedURLClassLoader
    protected ClassLoader createClassLoader(URL[] urls) throws Exception {
        return new LaunchedURLClassLoader(this.isExploded(), this.getArchive(), urls, this.getClass().getClassLoader());
    }

Then use LaunchedURLClassLoader in this.launch(args, launchClass, classLoader) to call the main function WebApplication written by yourself

public class MainMethodRunner {
    //...
    public void run() throws Exception {
        Class<?> mainClass = Class.forName(this.mainClassName, false, Thread.currentThread().getContextClassLoader());
        Method mainMethod = mainClass.getDeclaredMethod("main", String[].class);
        mainMethod.setAccessible(true);
        mainMethod.invoke((Object)null, this.args);
    }
}

Here, the main function of WebApplication, the main function written by myself, is called reflectively, thus completing a set of loading processes.

The complete process link is shown in the figure:

LaunchedURLClassLoader

LaunchedURLClassLoader, as the main class loader for SpringBoot to load FatJar, inherits URLClassLoader and rewrites the loadClass method, but this class does not see much core logic.

	@Override
	protected Class<?> loadClass(String name, boolean resolve) throws ClassNotFoundException {
		if (name.startsWith("org.springframework.boot.loader.jarmode.")) {
			try {
				Class<?> result = loadClassInLaunchedClassLoader(name);
				if (resolve) {
					resolveClass(result);
				}
				return result;
			}
			catch (ClassNotFoundException ex) {
			}
		}
		if (this.exploded) {
			return super.loadClass(name, resolve);
		}
		Handler.setUseFastConnectionExceptions(true);
		try {
			try {
				definePackageIfNecessary(name);
			}
			catch (IllegalArgumentException ex) {
				// Tolerate race condition due to being parallel capable
				if (getPackage(name) == null) {
					// This should never happen as the IllegalArgumentException indicates
					// that the package has already been defined and, therefore,
					// getPackage(name) should not return null.
					throw new AssertionError("Package " + name + " has already been defined but it could not be found");
				}
			}
			return super.loadClass(name, resolve);
		}
		finally {
			Handler.setUseFastConnectionExceptions(false);
		}
	}

It distinguishes the difference between org.springframework.boot.loader.jarmode and other jars, defines the package name, and the core logic still calls super.loadClass, which is URLClassLoader.loadClass.

URLClassLoader.loadClass is the process of loading URL above. So how do FatJar and ordinary Jar distinguish the logic?

All the difference lies in the process of loading the URL in Launcher.launch:

 ClassLoader classLoader = this.createClassLoader(this.getClassPathArchivesIterator());

You can see the logic of getClassPathArchivesIterator in ExecutableArchiveLauncher

    protected Iterator<Archive> getClassPathArchivesIterator() throws Exception {
        Archive.EntryFilter searchFilter = this::isSearchCandidate;
        Iterator<Archive> archives = this.archive.getNestedArchives(searchFilter, (entry) -> {
            return this.isNestedArchive(entry) && !this.isEntryIndexed(entry);
        });
        if (this.isPostProcessingClassPathArchives()) {
            archives = this.applyClassPathArchivePostProcessing(archives);
        }

        return archives;
    }

Among them, getNestedArchives obtains the archive information in the jar package (jar in jar) in FatJar, BOOT-INF/classes and BOOT-INF/lib/*.jar. Each element is treated as an Archive and an iterator list is returned.

FatJar loading

SpringBoot redefines (to be precise, extends) the jar implementation in org.springframework.boot.loader.jar, including JavaFile, JavaEntry, JarURLConnection, etc.

This.archive.getNestedArchives goes all the way in to view the source code. You can see that there is a function in JarFile.

  private JarFile createJarFileFromEntry(org.springframework.boot.loader.jar.JarEntry entry) throws IOException {
        return entry.isDirectory() ? this.createJarFileFromDirectoryEntry(entry) : this.createJarFileFromFileEntry(entry);
    }

BOOT-INF/classes as a directory will construct a JavaFile, and BOOT-INF/lib/*.jar as a nested jar (jar in jar) will construct another JavaFile

So how does this nested Jar (jar in jar) read data?

We can find the logic in JavaFile.createJarFileFromFileEntry

    private JarFile createJarFileFromFileEntry(org.springframework.boot.loader.jar.JarEntry entry) throws IOException {
        if (entry.getMethod() != 0) {
            throw new IllegalStateException("Unable to open nested entry '" + entry.getName() + "'. It has been compressed and nested jar files must be stored without compression. Please check the mechanism used to create your executable jar file");
        } else {
            RandomAccessData entryData = this.entries.getEntryData(entry.getName());
            return new JarFile(this.rootFile, this.pathFromRoot + "!/" + entry.getName(), entryData, JarFile.JarFileType.NESTED_JAR);
        }
    }

The entries refer to each piece of data in the outer FatJar, such as BOOT-INF/lib/a.jar, BOOT-INF/lib/b.jar. As jar in jar, first obtain the RandomAccessData data stream in this jar in jar. .

Then construct a JavaFile through the RandomAccessData data stream, so that the constructed Jar in jar can be read and accessed.

The path of jar in jar probably looks like this:

LittleBoy-1.0.jar!/BOOT-INF/lib/a.jar

LittleBoy-1.0.jar!/BOOT-INF/lib/b.jar

LittleBoy-1.0.jar!/BOOT-INF/lib/c.jar

The URL obtained by JavaFile is the URL that can be loaded by URLClassLoader.

    public URL getUrl() throws MalformedURLException {
        if (this.url == null) {
            String file = this.rootFile.getFile().toURI() + this.pathFromRoot + "!/";
            file = file.replace("file:////", "file://");
            this.url = new URL("jar", "", -1, file, new Handler(this));
        }

        return this.url;
    }

When returning the URL here, a new Handler is added to implement the logic of URLStreamHandler processing the URL.

We open the source code of this Handler, org.springframework.boot.loader.jar.Handler

You can see the processing logic for obtaining URLConnection

    protected URLConnection openConnection(URL url) throws IOException {
        if (this.jarFile != null && this.isUrlInJarFile(url, this.jarFile)) {
            return JarURLConnection.get(url, this.jarFile);
        } else {
            try {
                return JarURLConnection.get(url, this.getRootJarFileFromUrl(url));
            } catch (Exception var3) {
                return this.openFallbackConnection(url, var3);
            }
        }
    }

The obtained JarURLConnection has a getInputStream method that returns the searched byte stream.

How is the returned byte stream loaded into LaunchedURLClassLoader?

As mentioned earlier, LaunchedURLClassLoader inherits URLClassLoader. According to the byte stream returned by the URL, the Resource can be obtained based on ClassName. The following is the URLClassLoader.getResource method.

And for this Resource, you can defineClass, that is, define a class

The following code is excerpted from URLClassLoader.findClass

    protected Class<?> findClass(final String name)
        throws ClassNotFoundException
    {
        //省略部分代码
        ...
        
        String path = name.replace('.', '/').concat(".class");
        Resource res = ucp.getResource(path, false);
        if (res != null) {
            try {
                return defineClass(name, res);
            } catch (IOException e) {
                throw new ClassNotFoundException(name, e);
            }
        } else{
            return null;
        }
        
    }

So far, the loading process of a FatJar has been fully explained.

In-depth understanding of the principle of SpringBoot loading FatJar