Introduction to the Java Platform Module System

Introduction to the Java Platform Module System - Part 1

Bhaskar S

05/29/2020

The Java Development Kit (JDK) was first released in the year 1995 and has evolved since then. All the packages and classes of the Java runtime are bundled into one big uber JAR file called rt.jar. As with any monolith, it becames challenging to develop, maintain, refactor, secure, and/or evolve over time.

Between classes, Java implements good encapsulation mechanism using the visibility modifiers such as public, protected, private, etc.

On the other hand, Java packages implement a weak encapsulation mechanism - all public classes are visible to all other classes. There is no way to control this public visibility of claas(es) in a package. This implies, if a library developer implements some class(es) for their own internal use, there is no way to prevent an application developer from using those class(es). How many of us are guilty of using the classes sun.misc.BASE64Encoder or sun.misc.BASE64Decoder ???

A JAR file is nothing more than an archive of one or more Java packages. Even though it may appear to be modular because each library is in its own JAR, there is no mechanism to indicate JAR dependencies. How many of us have run into the java.lang.ClassNotFoundException issue because of missing JAR file(s) ??? Or worse yet, the presence of the same class in different JAR files - ever encounter the java.lang.NoSuchMethodError issue ???

What we were missing was a strong encapsulation mechanism around Java packages. This is exactly what the Java Platform Module System in the Java 9 release set out to address.

Modules are independent, deployable containers for Java package(s), with strong encapsulation mechanisms in place. Within a module, some packages are marked as available for external consumption, while others are strictly for internal use, which is enforced by Java at both the compile time and the run time. In addition, a module indicates the other module(s) it is dependent on.

The following diagram illustrates the typical list of files and directories from Java 8:

Figure.1

The following diagram illustrates the contents of the directory jre/lib from Java 8:

Figure.2

Starting Java 9, the platform itself has been refactored into a set of modules. The module java.base is at the heart of the Java platform and is the primordial module that exposes the fundamental and core Java language packages java.lang and java.util. It is the base parent module and does not depend on any other module. All the other modules in the Java platform implicitly depend on this core module.

The following diagram illustrates the typical list of files and directories from Java 9 (or above):

Figure.3

The following diagram illustrates the contents of the directory jmods (which contains all the platform modules) from Java 9 (or above):

Figure.4

The Java platform modules are divided into two categories:

Standard :: modules with names that are prefixed with java.. For example, java.base, java.logging, etc
Non-Standard :: modules with names that are prefixed with jdk.. For example, jdk.compiler, jdk.net, etc. The Java packages from these modules are not part of the Java specification

To list all the modules provided with the Java platform, execute the following command using Java 11:

$ java --list-modules

The following would be a typical output:

Output.1

java.base@11.0.7
java.compiler@11.0.7
java.datatransfer@11.0.7
java.desktop@11.0.7
java.instrument@11.0.7
java.logging@11.0.7
java.management@11.0.7
java.management.rmi@11.0.7
java.naming@11.0.7
java.net.http@11.0.7
java.prefs@11.0.7
java.rmi@11.0.7
java.scripting@11.0.7
java.se@11.0.7
java.security.jgss@11.0.7
java.security.sasl@11.0.7
java.smartcardio@11.0.7
java.sql@11.0.7
java.sql.rowset@11.0.7
java.transaction.xa@11.0.7
java.xml@11.0.7
java.xml.crypto@11.0.7
jdk.accessibility@11.0.7
jdk.aot@11.0.7
jdk.attach@11.0.7
jdk.charsets@11.0.7
jdk.compiler@11.0.7
jdk.crypto.cryptoki@11.0.7
jdk.crypto.ec@11.0.7
jdk.dynalink@11.0.7
jdk.editpad@11.0.7
jdk.hotspot.agent@11.0.7
jdk.httpserver@11.0.7
jdk.internal.ed@11.0.7
jdk.internal.jvmstat@11.0.7
jdk.internal.le@11.0.7
jdk.internal.opt@11.0.7
jdk.internal.vm.ci@11.0.7
jdk.internal.vm.compiler@11.0.7
jdk.internal.vm.compiler.management@11.0.7
jdk.jartool@11.0.7
jdk.javadoc@11.0.7
jdk.jcmd@11.0.7
jdk.jconsole@11.0.7
jdk.jdeps@11.0.7
jdk.jdi@11.0.7
jdk.jdwp.agent@11.0.7
jdk.jfr@11.0.7
jdk.jlink@11.0.7
jdk.jshell@11.0.7
jdk.jsobject@11.0.7
jdk.jstatd@11.0.7
jdk.localedata@11.0.7
jdk.management@11.0.7
jdk.management.agent@11.0.7
jdk.management.jfr@11.0.7
jdk.naming.dns@11.0.7
jdk.naming.rmi@11.0.7
jdk.net@11.0.7
jdk.pack@11.0.7
jdk.rmic@11.0.7
jdk.scripting.nashorn@11.0.7
jdk.scripting.nashorn.shell@11.0.7
jdk.sctp@11.0.7
jdk.security.auth@11.0.7
jdk.security.jgss@11.0.7
jdk.unsupported@11.0.7
jdk.unsupported.desktop@11.0.7
jdk.xml.dom@11.0.7
jdk.zipfs@11.0.7

The following illustration depicts the composition of a module:

Figure.5

A module has a globally unique name associated with it along with the following elements:

exports :: indicates the package(s) available for external access
opens :: indicates the package(s) available through reflective access
provides :: indicates the package(s) as implementation(s) of some service interface
requires :: indicates the modules it depends on
uses :: indicates the service interface(s) it leverages

To display all the information about the Java platform a module called java.sql, execute the following command using Java 11:

$ java --describe-module java.sql

The following will be the typical output:

Output.2

java.sql@11.0.7
exports java.sql
exports javax.sql
requires java.transaction.xa transitive
requires java.xml transitive
requires java.logging transitive
requires java.base mandated
uses java.sql.Driver

We will now demonstrate the behavior of a simple Java program in both Java 8 and Java 11.

The following simple Java program BASE64 encodes a string of characters:

Listing.1

package com.polarsparc.jm;

import sun.misc.BASE64Encoder;

public class MyB64Enc {
    public static void main(String[] args) {
        byte[] secret = "MySup3rS3cr3t#".getBytes();
        
        BASE64Encoder encoder = new BASE64Encoder();
        
        System.out.printf("Encoded secret: %s\n", encoder.encode(secret));
    }
}

To compile the above Java program using the Java 8 compiler, execute the following command:

$ javac com/polarsparc/jm/MyB64Enc.java

The following will be the typical output:

Output.3

com/polarsparc/jm/MyB64Enc.java:4: warning: BASE64Encoder is internal proprietary API and may be removed in a future release
import sun.misc.BASE64Encoder;
               ^
com/polarsparc/jm/MyB64Enc.java:10: warning: BASE64Encoder is internal proprietary API and may be removed in a future release
        BASE64Encoder encoder = new BASE64Encoder();
        ^
com/polarsparc/jm/MyB64Enc.java:10: warning: BASE64Encoder is internal proprietary API and may be removed in a future release
        BASE64Encoder encoder = new BASE64Encoder();
                                    ^
3 warnings

⛔ ATTENTION ⛔

Always PAY attention to the *warnings*.

To execute the compiled Java program using the Java 8 compiler, execute the following command:

$ java com/polarsparc/jm/MyB64Enc

The following will be the typical output:

Output.4

Encoded secret: TXlTdXAzclMzY3IzdCM=

Now let us try to run the same Java program using the Java 11 compiler by executing the following command:

$ java com/polarsparc/jm/MyB64Enc

The following will be the typical output:

Output.5

Exception in thread "main" java.lang.NoClassDefFoundError: sun/misc/BASE64Encoder
	at com.polarsparc.jm.MyB64Enc.main(MyB64Enc.java:10)
Caused by: java.lang.ClassNotFoundException: sun.misc.BASE64Encoder
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
	... 1 more

We will use the dependency analyzer utility program called jdeps provided with Java 11 to analyze and understand the Java package or class level dependencies. To do that, execute the following command:

$ jdeps --jdk-internals --class-path ./com/polarsparc/jm

The following will be the typical output:

Output.6

jm -> JDK removed internal API
   com.polarsparc.jm.MyB64Enc          -> sun.misc.BASE64Encoder          JDK internal API (JDK removed internal API)

Warning: JDK internal APIs are unsupported and private to JDK implementation that are
subject to be removed or changed incompatibly and could break your application.
Please modify your code to eliminate dependence on any JDK internal APIs.
For the most recent update on JDK internal API replacements, please check:
https://wiki.openjdk.java.net/display/JDK8/Java+Dependency+Analysis+Tool

JDK Internal API                         Suggested Replacement
----------------                         ---------------------
sun.misc.BASE64Encoder                   Use java.util.Base64 @since 1.8

From the Output.6 above, it is clear that the internal class sun.misc.BASE64Encoder has been removed and the recommendation is to use the class java.util.Base64.

The following is the fixed Java program that uses the recommended approach for BASE64 encoding a string of characters that works in both Java 8 and Java 11:

Listing.2

package com.polarsparc.jm;

import java.util.Base64;

public class MyB64Enc2 {
    public static void main(String[] args) {
        byte[] secret = "MySup3rS3cr3t#".getBytes();
        
        Base64.Encoder encoder = Base64.getEncoder();
        
        System.out.printf("Encoded secret: %s\n", encoder.encodeToString(secret));
    }
}

Let us look at another example. The following simple Java program creates an SSL context the old way:

Listing.3

package com.polarsparc.jm;

import com.sun.net.ssl.SSLContext;
import com.sun.net.ssl.TrustManager;
import com.sun.net.ssl.X509TrustManager;
import java.security.cert.X509Certificate;

public class MyOldSslStuff {
    static TrustManager[] trustAllCerts = new TrustManager[] {
        new X509TrustManager() {
            public X509Certificate[] getAcceptedIssuers() {
                return null;
            }
        
            public boolean isClientTrusted(X509Certificate[] chain) {
                return true;
            }
            
            public boolean isServerTrusted(X509Certificate[] chain) {
                return true;
            }
        }
    };
    
    public static void main(String[] args) {
        try {
            SSLContext context = SSLContext.getInstance("SSL");
            context.init(null, trustAllCerts, null);
            
            System.out.println("Success !!!");
        }
        catch (Exception ex) {
            System.err.println(ex);
        }
    }
}

To compile the above Java program using the Java 8 compiler, execute the following command:

$ javac com/polarsparc/jm/MyOldSslStuff.java

The following will be the typical output:

Output.7

Note: com/polarsparc/jm/MyOldSslStuff.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.

Now let us try to compile the same Java program using the Java 11 compiler by executing the following command:

$ javac com/polarsparc/jm/MyOldSslStuff.java

The following will be the typical output:

Output.8

com/polarsparc/jm/MyOldSslStuff.java:3: error: package com.sun.net.ssl is not visible
import com.sun.net.ssl.SSLContext;
                  ^
  (package com.sun.net.ssl is declared in module java.base, which does not export it)
com/polarsparc/jm/MyOldSslStuff.java:4: error: package com.sun.net.ssl is not visible
import com.sun.net.ssl.TrustManager;
                  ^
  (package com.sun.net.ssl is declared in module java.base, which does not export it)
com/polarsparc/jm/MyOldSslStuff.java:5: error: package com.sun.net.ssl is not visible
import com.sun.net.ssl.X509TrustManager;
                  ^
  (package com.sun.net.ssl is declared in module java.base, which does not export it)
3 errors

From the Output.8 above, we see learn that the package com.sun.net.ssl is declared in the module java.base and is *NOT* exported.

To resolve this issue, we need to use the command-line option --add-exports in the Java 11 compiler to force the export of the desired package by breaking the encapsulation of internal API and make it accessible.

Now, let us re-try the compile by executing the following command:

$ javac --add-exports java.base/com.sun.net.ssl=ALL-UNNAMED com/polarsparc/jm/MyOldSslStuff.java

The following will be the typical output:

Output.9

Note: com/polarsparc/jm/MyOldSslStuff.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.

The option --add-exports has the following format:

<source-module>/<package-name>=<target-module>

The keyword ALL-UNNAMED for the <target-module> means, all the class(es) from the classpath can access the package <package-name> from <source-module>.

References

Modular JDK - Project Jigsaw

The State of the Module System