Cyberborean Chronicles

Custom ClassLoaders: The Black Art of Java

Java is great platform for component development but there are some odd and counterintuitive things on the way. One of these hidden pitfalls waiting for a developer who is going to replace default system ClassLoader with a custom one.

A common case when you might have an idea of using a custom ClassLoader is to avoid the classpath issues using URLClassLoader for loading classes instead of the default one. Classpath is a list of locations (JAR-files or directories) where default ClassLoader is looking for the classes to load. Declaring Classpath explicitly (either with CLASSPATH environment variable, or as a command-line argument of Java virtual machine) is tedious and error-prone job and it works only in the case of monolithic, statically-linked applications with known set of libraries.

But imagine a complex component-based system with a number of replaceable modules (plug-ins). It is obvious that classpath for that system should be built automatically to reflect the configuration changes before each running. It is often done with system-specific startup scripts which analyzes the modules configuration and generates an appropriate classpath before starting the application. However, this method has its own drawbacks – especially in the case of poor Microsoft systems as their rudimentary command interpreter makes this task hardly possible. And this is a way system-dependent, of course.

URLClassLoader, from a distant point of view, looks like an elegant, simple and “100%-pure-java” solution for the classpath headache. It is instantiated with an array of locations (URL’s) so each class loaded with that ClassLoader will use this array as a classpath. An advantage is that the classpath can be generated on the fly, automatically by an application itself.

So, let’s see how it works. Imagine we have an application with main executable class Foo (in package com.example.foo) and a lot of other classes in different locations. We will develop an invocation wrapper for loading Foo using URLClassLoader with an array of locations, generated automatically:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
/*
 * FooInvoker.java
 */
 
package com.example.foo;
 
import java.net.URL;
import java.net.URLClassLoader;
import java.lang.reflect.Method;
import java.lang.reflect.Modifier;
 
public class FooInvoker {
    public static String FOO_CLASS_NAME = "com.example.foo.Foo";
 
    public static void main(String[] args) {
        URL[] classpathURLs = getClasspathURLs();
        ClassLoader loader = new URLClassLoader(classpathURLs);
        try {
            Class fooClass = loader.loadClass(FOO_CLASS_NAME);
            Method main = fooClass.getMethod("main", new Class[] {
                String[].class
            });
            int modifiers = main.getModifiers();
            if (Modifier.isPublic(modifiers) && Modifier.isStatic(modifiers)) {
                main.invoke(null, new Object[] {
                    new String[] {}
                });
                // In this example, we assume Foo.main() takes no
                // command-line arguments
            } else {
                throw new NoSuchMethodException();
            }
        } catch (Exception e) {
            System.err.println("Error running class " + FOO_CLASS_NAME
                    + e.getMessage());
            e.printStackTrace();
        }
    }
 
    private static URL[] getClasspathURLs() {
        URL[] urls = new URL[] {};
        // Here we're looking for all project-related libraries
        // and constructing the classpath array
        // ...
        return urls;
    }
}

Run it (assuming Foo and FooInvoker are packaged both into foo.jar file, which is the core library of our app):

$ java -classpath ./foo.jar com.example.foo.FooInvoker

Alas! Everything located outside the foo.jar (that is, the classpath set in JVM arguments) is failed to load. For instance, if Foo uses class Bar in it’s own bar.jar library, Foo will throw ClassNotFoundException despite of the fact that bar.jar has been passed to URLClassLoader argument. It seems we have no progress at all – all classes are loaded from default JVM classpath as with the system ClassLoader.

Let’s do some debugging. Add this to Foo.main() code to see which ClassLoader loads Foo:

System.out.println("Foo is loaded with: " +
                           Foo.class.getClassLoader().getClass().getName());

Instead of “java.net.URLClassLoader” in output (as we hoped), we will see:

Foo is loaded with: sun.misc.Launcher$AppClassLoader

This is the default system ClassLoader. We really have no progress – it seems our URLClassLoader simply does not come into play. Our code is right but it doesn’t work. Is it a Java bug? Should we give up? Or report to Sun?

Wait a minute. The problem is not in code but in how Java VM executes it. Let’s try a trick: change FooInvoker.getClasspathURLs() algorythm to include foo.jar path in the resulted array (to be passed to URLClassLoader among other locations) and compile FooInvoker as a separate class file (not inside the foo.jar). Then run it (no classpath is declared!):

$ java com.example.foo.FooInvoker
...
Foo is loaded with: java.net.URLClassLoader

Voila! Our URLClassLoader has been used to load Foo as well as other classes, including those from locations in our dynamic classpath. It works! So what is happened?

In this case we found ourselves in one of the dark and odd Java territories ruled by the Masters of The Black Java Art (also known as the Heavy Java Geeks) who believe the programs should not behave as their author expects, but following the mysterious Rules instead. One of those Rules is called “ClassLoader Delegation Model” and it reads:

The ClassLoader class uses a delegation model to search for classes and resources. Each instance of ClassLoader has an associated parent class loader. When requested to find a class or resource, a ClassLoader instance will delegate the search for the class or resource to its parent class loader before attempting to find the class or resource itself.

A parent of URLClassLoader is the default system ClassLoader (as we used single-argument constructor). When we tried to load Foo with an instance of URLClassLoader, it asked the parent if the latter could load Foo. As Foo was in the default classpath, the system ClassLoader, of course, could. By this way, it took a position of a default ClassLoader in a chain of subsequent class invocations and left our URLClassLoader out of business.

When we moved Foo and everything out the default classpath, the system ClassLoader was not able to load anything (except FooInvoker itself). So, it had no chance to took precedence over our custom ClassLoader.

Well, if you like rules, there is another one:

If you use custom ClassLoader, do not give the system one a chance to come into play.

Happy coding!

4 Responses to “Custom ClassLoaders: The Black Art of Java”

  1. osmadja · December 11, 2007 at 4:44 pm · Reply

    Great article. It helped me.

  2. rimu · December 17, 2007 at 11:03 am · Reply

    This is a shoot in the dark, but here it goes. Is there some way to unload a static class in order to load it again without using reflection?

    I have an application that uses a static class and in my unit testing I want to unload that static class in order to load it again going through its constructor and everything.

    Is this possible?

  3. Alex · December 18, 2007 at 2:34 am · Reply

    Not sure. Java Language Specification does not specify a mechanism for explicit class unloading (12.8).

  4. rimu · December 18, 2007 at 12:28 pm · Reply

    I think I found the answer. Basically you can’t unload a class loaded by the system class loader, but if you load a class with your own custom class loader and from within that class you load a static class with the same class loader you can. But that means that you unload both the original class and the static class. Anyway I have come to the conclusion that custom classloaders isn’t the best way to solve the problem.

    Good post btw.

Leave a Reply