The Java mascot Duke, by Joe Palrang, licensed under the New BSD license

How to assemble a Java Program

Notes published the 24 Mar, 2024
14 - 17 minutes to read, 3446 words
Categories: build system, java, reproducible, shell
Keywords: archive, build system, bytecode, java, reproducible, shell, validation

Some notes on how to compile Java source code, and modify and execute jar files without using a build system.

Thus no ant, maven, or gradle.

Compile and execute a single file

I personally find it useful to compare how a minimal Java program is compiled in comparison to a minimal program in another language, like C.

Compile and execute a "Hello World" C program

This is how a hello world in C is generally introduced; a single file and a compiler invocation:

echo '#include <stdio.h>

int main(){
  puts("Hello World!");
  return 0;
}' > main.c;
gcc main.c -o main;
./main;

Compile and execute a "Hello World" Java program

A Java program can be compiled very similarly to a C program; one source file and a compiler invocation.

The syntax is obviously different, and how the program is executed is also slightly different:

echo 'class Main {
  public static void main(String[] args) {
    System.out.println("Hello, World!");
  }
}' > Main.java;
javac Main.java;
java Main;

Note that the output of javac Main.java is Main.class because the class name is Main. The file name does not play a role; you can rename Main.java to Dummy.java, and notice that the output of javac Dummy.java is still Main.class.

Another thing to note is that java takes as argument the class name, not the .class file (thus java Main and not java Main.class), and it expects that the file name and class name coincide.

Thus renaming a .class file is generally asking for trouble.

You also really want to keep the class name and file name consistent. It is not necessary, but it will avoid headaches.

Compiler warnings

Similar to GCC, also javac can diagnose code that is valid but probably incorrect or that follows some unexpected pattern.

With javac --help-lint it is possible to see which diagnostics are supported, normally with -Xlint:all, all warnings are enabled

Compile and execute multiple source files

In C

A minimal C program that consists of multiple source files:

echo '#pragma once
const char* hello();' > hello.h;
echo '#include "hello.h"
const char* hello(){return "Hello World";}' > hello.c;

echo '#include <stdio.h>
#include "hello.h"

int main(){
  puts(hello());
  return 0;
}' > main.c;
gcc main.c hello.c -o main;
./main;

In Java

A minimal Java program that consists of multiple source files

echo 'class HelloWorld {
  public static String hello() {return "Hello, World!";}
}' > HelloWorld.java;

echo 'class Main {
  public static void main(String[] args) {
    System.out.println(HelloWorld.hello());
  }
}' > Main.java;
javac -Xlint:all Main.java HelloWorld.java;
java Main;

Again, the process is similar to how a C program is assembled and executed, with the differences already mentioned before.

Project structure convention and directory conventions

For good or bad, Java projects consist of multiple directories and are not as "flat" as projects in other languages.

Another useful convention is having the source code located in one folder, like src or source, and the compiled class files somewhere else, like in a folder named out, bld, or build. Having the output in a separate folder makes it easier to automate different tasks, like restoring a clean environment, or creating an archive.

mkdir -p src/com/example/lib;
echo 'package com.example.lib;

public class HelloWorld {
  public static String hello() {
    return "Hello, World!";
  }
}' > src/com/example/lib/HelloWorld.java;

echo 'package com.example;

import com.example.lib.HelloWorld;

public class Main {
  public static void main(String[] args) {
    System.out.println(HelloWorld.hello());
  }
}' > src/com/example/Main.java;
javac -Xlint:all -d out src/com/example/Main.java src/com/example/lib/HelloWorld.java;
(cd out && java com.example.Main);

The parameter -d is used for specifying an output directory, otherwise the program javac would create the .class file in the same folder as the .java file, like in the previous section.

The fact that src/com/example/Main.java is compiled to out/com/example/Main.class and not out/src/com/example/Main.class, depends on the fact that in the source file, the first line is package com.example.

Changing the first line to package src.com.example, means that the class file will be created at out/src/com/example/.

Thus, similarly to the class name, keeping the like with package synchronized to where the file is located will avoid a lot of headaches, as the location of the file and filename do not play a role where the output of javac is stored.

Note that for executing the Java program, we had to change directory (cd out), and java takes as a parameter the class name prefixed by the package name.

Doing cd out/com/example && java Main would trigger a java.lang.NoClassDefFoundError, with the error message com/example/Main (wrong name: Main)

Classpath

When invoking java with the Main class, we did not specify all classes that Main depends on.

Java searches in the current directory (and subdirectories) where to search for .class file to load.

Thus, instead of (cd out && java com.example.Main) one can write java --class-path out com.example.Main.

With --class-path, it is possible to define a semicolon-separated list of directories where java should search for .class files.

Compile Java files one-by-one

It is possible to compile Java files one by one.

javac -d out src/com/example/lib/HelloWorld.java creates out/com/example/lib/HelloWorld.class, but javac -d out src/com/example/Main.java fails with the following error message

src/com/example/Main.java:3: error: package com.example.lib does not exist
import com.example.lib.HelloWorld;
                      ^
src/com/example/Main.java:7: error: cannot find symbol
    System.out.println(HelloWorld.hello());
                       ^
  symbol:   variable HelloWorld
  location: class Main
2 errors

The compiler needs to find all symbols used, not only at compile-time but also at runtime.

Also in this case, one can use the --class-path parameter, and in fact with javac --class-path src -d out src/com/example/Main.java the Main.class is created successfully.

One could also use javac --class-path out -d out src/com/example/Main.java as in this case HelloWorld.class has already been generated, but I guess that it is better to use the source folder, as it is up-to-date, while the compiled .class file could be out-of-date.

But compiling as much as possible in one go has also another advantage: performance.

Even if there are no dependencies between .java files, there is a measurable difference when compiling them together or one-by-one sequentially

touch files.txt;
for i in $(seq 1 25); do :;
  file="HelloWorld$i.java";
  printf '
public class HelloWorld%s {
  public static String hello%s() {
    return "Hello, World %s!";
  }
}' "$i" "$i" "$i"> "$file";
  echo "$file">> files.txt;
done
echo 'build everything together';
javac '@files.txt';

echo 'build one-by-one';
for i in $(seq 1 25); do :;
  file="HelloWorld$i.java";
  javac "$file";
done;

Assemble a jar file

A jar file is an archive containing a collection of .class files.

It can be used similarly to a library (like a .so, .lib, or .dll file ) or an executable.

The program jar is used to create a jar file, which is, at the end of the day, a renamed .zip archive:

jar --create --file Hello.jar -C out .;

It is also possible to enlist the single class files:

# those two commands create an archive functionally equivalent to "jar --create --file Hello.jar -C out ."
jar --create --file Hello.jar -C out com/example/lib/HelloWorld.class -C out com/example/Main.class;
jar --create --file Hello.jar -C out com/example/lib/HelloWorld.class out/com/example/Main.class; // (1)

# those two commands create something else and are "wrong"
jar --create --file Hello.jar -C out com/example/lib/HelloWorld.class com/example/Main.class;
jar --create --file Hello.jar out/com/example/lib/HelloWorld.class out/com/example/Main.class;

I have no idea why the first file is treated differently, the documentation is not clear about it.

For "executing" the .jar file, use

java --class-path Hello.jar come.example.Main;

Manifest file

Having to type out the classpath and where the main method is located, is not as practical as just writing the class name.

Embed the manifest with

mkdir -p src/META-INF/;
echo 'Main-Class: com.example.Main' > src/META-INF/MANIFEST.MF;
jar --create --file Hello.jar --manifest src/META-INF/MANIFEST.MF -C out .;

or with

jar --main-class=com.example.Main --create --file Hello.jar -C out .;

A third alternative is to write

mkdir -p out/META-INF;
echo 'Main-Class: com.example.Main' > out/META-INF/MANIFEST.MF;
jar --create --file Hello.jar --no-manifest -C out .;

In the first case, jar will use src/META-INF/MANIFEST.MF as manifest, but also add some additional information, in my case

Manifest-Version: 1.0
Created-By: 23-ea (Debian)

in the second case, jar will create a manifest that looks like the one in the first case-

I prefer adding as little information as possible from the current system and thus would want to avoid having the Created-By line, the third approach seems to be most sensible.

At this point, no matter if you use --manifest, --no-manifest, or --main-class; if you have an appropriate manifest file, you can execute the Hello.jar file with

java -jar Hello.jar;

Reproducible Builds

The output of javac is generally reproducible, the .class file does not have embedded timestamps or references to the path to the project.

The archive created with jar is generally not reproducible, as it contains at least the timestamp of the files.

The newer version of jar of OpenJDK (version 23) supports the --date parameter, thus by using it one can ensure that the build does not depend on the current time.

Note 📝

the value "1980-01-01T00:00:02Z" is the first timestamp that is supported by the --date parameter. SOURCE_DATE_EPOCH is defined as the number of seconds since 01 Jan 1970 00:00:00, thus 10 years before, but zip archives use a timestamp relative to 1980 with two seconds precision ^🗄️, hence the difference. The tool strip-non-determinism uses 1980-01-01T12:01:00Z as a timestamp, thus it might make sense to use that value.

The generated manifest file is another source of non-reproducibility, which is one more reason to manually create one, and use --no-manifest.

Modify a jar archive

Sometimes, during or after the build process, one would like to modify an existing jar archive.

`R8` and `proguard`

I’ve looked at how Java code is optimized, and the TLDR is that it is not.

Tools like R8 ^🗄️ and Proguard take as input a .jar file, optimize some contained .class files, and create a new .jar file.

proguard -dontobfuscate -libraryjars /usr/lib/jvm/java-23-openjdk-amd64/jmods/java.base.jmod -injars Hello.jar -outjars Hello.opt.jar -keep "public class com.example.Main { public static void main(java.lang.String[]);}"

Note that you must define at least one -keep parameter, the manifest file is not taken into account.

You must also define -libraryjars pointing to the JDK used for creating your .jar file, otherwise you’ll get errors like can’t find superclass or interface java.lang.Object.

Modify or add a file

As mentioned previously, a .jar file is a renamed .zip archive, thus any tool that can manipulate .zip files could be used for modifying .jar archive.

Since the jar executable is already able to modify archives, there is generally no need to resort to other tools.

# optionally extract a file
(
  mkdir newfiles && cd newfiles;
  jar --extract --file ../Hello.jar "file to extract";
)

# modify the extracted file (or create a new file)

# insert the new or modified file in the archive
jar --update --file Hello.jar -C newfiles .;

But this will not work if you are trying to modify the META-INF/MANIFEST.MF file. In this case, you’ll need to add the --no-manifest option:

mkdir -p newmanifest/META-INF/;

# create or edit newmanifest/META-INF/MANIFEST.MF

jar --update --no-manifest --file Hello.jar -C newfiles META-INF/MANIFEST.MF;

Note that always adding --no-manifest would be wrong, because if there is no replacement for the manifest file, then the modified jar archive will be without manifest.

Sign jar file

jarsigner should be used for adding a cryptographic signature to your jar archives.

# generate a key; keep it secret, and choose a password more secure than 123456
keytool -genkey -alias ALIAS -keyalg RSA -keypass 123456 -storepass 123456 -keystore store.jks;

# sign the jar file
jarsigner -keystore store.jks -storepass 123456 -signedjar Hello.signed.jar Hello.jar ALIAS;

# verify the signature
jarsigner -verify Hello.signed.jar;

Remove file

The jar executable, nor any other tool of the JDK seems to be able to remove a file from a .jar file.

Thus the "official" way would be to extract everything, remove the offending files, and create a new jar file

# extract everything
(
  mkdir newfiles && cd newfiles;
  jar --extract --file ../Hello.jar .;
)

# remove the files we do not want from newfiles

# create new archive
rm Hello.jar;
jar --create --file Hello.jar --no-manifest -C newfiles .;

A sensible alternative might be to resort to an external tool

zip -d Hello.jar unwanted_file;
zip -d Hello.jar unwanted_folder/;

Dependencies

Often, when writing more complex programs, it makes sense to reuse existing libraries instead of reinventing the wheel.

Most libraries for the Java ecosystem can be found at repositories like the Maven Repository.

Build systems like Maven and Gradle can download and manage transitive dependencies for you, doing this manually is generally error-prone, as external libraries make liberal use of other libraries too.

Download dependencies

Since the JDK does not have an integrated test framework, let’s download Junit5, one of the most used testing frameworks in Java, and integrate it by hand.

After looking at the page at mvnrepository I tracked the dependencies down, and wrote a couple of tests:

wget \
  'https://repo1.maven.org/maven2/org/junit/jupiter/junit-jupiter-api/5.10.2/junit-jupiter-api-5.10.2.jar' \
  'https://repo1.maven.org/maven2/org/junit/platform/junit-platform-launcher/1.10.2/junit-platform-launcher-1.10.2.jar' \
  'https://repo1.maven.org/maven2/org/junit/platform/junit-platform-engine/1.10.2/junit-platform-engine-1.10.2.jar' \
  'https://repo1.maven.org/maven2/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.jar' \
  ;

mkdir -p src/com/example/;

echo 'package com.example;

import static org.junit.jupiter.api.Assertions.assertEquals;
import org.junit.jupiter.api.Test;

public class TestArithmetic {
  @Test
  public void addition() {
    assertEquals(2, 1+1);
  }
  @Test
  public void subtraction() {
    assertEquals(0, 1+1);
  }
}' > src/com/example/TestArithmetic.java;

javac --class-path junit-jupiter-api-5.10.2.jar:junit-platform-launcher-1.10.2.jar:junit-platform-engine-1.10.2.jar:apiguardian-api-1.1.2.jar -d out src/com/example/TestArithmetic.java;

After compiling everything together, I had difficulties understanding how to execute the tests from the command line.

Only at that point, I’ve noticed that the documentation mentions another library for executing the tests from the command line ^🗄️, at least it does not seem to have external dependencies

wget \
  'https://repo1.maven.org/maven2/org/junit/jupiter/junit-jupiter-api/5.10.2/junit-jupiter-api-5.10.2.jar' \
  'https://repo1.maven.org/maven2/org/junit/platform/junit-platform-launcher/1.10.2/junit-platform-launcher-1.10.2.jar' \
  'https://repo1.maven.org/maven2/org/junit/platform/junit-platform-engine/1.10.2/junit-platform-engine-1.10.2.jar' \
  'https://repo1.maven.org/maven2/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.jar' \
  'https://repo1.maven.org/maven2/org/junit/platform/junit-platform-console-standalone/1.10.2/junit-platform-console-standalone-1.10.2.jar' \
  ;

mkdir -p src/com/example/;

echo 'package com.example;

import static org.junit.jupiter.api.Assertions.assertEquals;
import org.junit.jupiter.api.Test;

public class TestArithmetic {
  @Test
  public void addition() {
    assertEquals(2, 1+1);
  }
  @Test
  public void subtraction() {
    assertEquals(0, 1+1);
  }
}' > src/com/example/TestArithmetic.java;

javac --class-path junit-platform-console-standalone-1.10.2.jar -d out src/com/example/TestArithmetic.java;

java -jar junit-platform-console-standalone-1.10.2.jar execute --disable-banner --class-path out --scan-class-path --fail-if-no-tests;

The result will look similar too

╷
├─ JUnit Jupiter ✔
│  └─ Main ✔
│     ├─ subtraction() ✘ expected: <0> but was: (2)
│     └─ addition() ✔
├─ JUnit Vintage ✔
└─ JUnit Platform Suite ✔

Failures (1):
  JUnit Jupiter:Main:subtraction()
    MethodSource [className = 'com.example.Main', methodName = 'subtraction', methodParameterTypes = '']
    => org.opentest4j.AssertionFailedError: expected: <0> but was: (2)
       org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
       org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
       org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:197)
       org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:150)
       org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:145)
       org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:531)
       com.example.Main.subtraction(Main.java:17)
       java.base/java.lang.reflect.Method.invoke(Method.java:568)
       java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
       java.base/java.util.ArrayList.forEach(ArrayList.java:1511)

Test run finished after 159 ms
[         4 containers found      ]
[         0 containers skipped    ]
[         4 containers started    ]
[         0 containers aborted    ]
[         4 containers successful ]
[         0 containers failed     ]
[         2 tests found           ]
[         0 tests skipped         ]
[         2 tests started         ]
[         0 tests aborted         ]
[         1 tests successful      ]
[         1 tests failed          ]

It is also possible to package all unit tests to be executed in a jar file

jar --create --file Tests.jar -C out com/example/TestArithmetic.class;
java -jar junit-platform-console-standalone-1.10.2.jar execute --disable-banner --class-path Tests.jar --scan-class-path --fail-if-no-tests;

fat jar

A "fat jar" is a Java archive file that does not depend on external files.

Which means it avoids the need to set --class-path when executing a Java program.

The simplest approach is to merge all jar files together by extracting the content in a common location and create a new jar file

(
  mkdir newfiles && cd newfiles;
  jar --extract --file ../Hello.jar .;
  jar --extract --file ../dep1.jar .;
  jar --extract --file ../dep2.jar .;
)
rm Hello.jar;
jar --create --file Hello.jar --no-manifest -C newfiles .;

Care must be taken for files that are present in multiple archives, like the manifest, and merge them together.

In the case of the unit-tests, wouldn’t it be nice to have only one jar file that automatically executes the tests, instead of having Tests.jar and junit-platform-console-standalone-1.10.2.jar?

wget \
  'https://repo1.maven.org/maven2/org/junit/jupiter/junit-jupiter-api/5.10.2/junit-jupiter-api-5.10.2.jar' \
  'https://repo1.maven.org/maven2/org/junit/platform/junit-platform-launcher/1.10.2/junit-platform-launcher-1.10.2.jar' \
  'https://repo1.maven.org/maven2/org/junit/platform/junit-platform-engine/1.10.2/junit-platform-engine-1.10.2.jar' \
  'https://repo1.maven.org/maven2/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.jar' \
  'https://repo1.maven.org/maven2/org/junit/platform/junit-platform-console-standalone/1.10.2/junit-platform-console-standalone-1.10.2.jar' \
  ;

mkdir -p src/com/example/;

echo 'package com.example;

import static org.junit.jupiter.api.Assertions.assertEquals;
import org.junit.jupiter.api.Test;
import org.junit.platform.console.ConsoleLauncher;

public class Main {
  public static void main(String[] args) {
    ConsoleLauncher.main("execute", "--disable-banner", "--select-class", "com.example.TestArithmetic", "--fail-if-no-tests");
  }
}' > src/com/example/Main.java;

echo 'package com.example;

import static org.junit.jupiter.api.Assertions.assertEquals;
import org.junit.jupiter.api.Test;

public class TestArithmetic {
  @Test
  public void addition() {
    assertEquals(2, 1+1);
  }
  @Test
  public void subtraction() {
    assertEquals(0, 1+1);
  }
}' > src/com/example/TestArithmetic.java;

javac --class-path junit-platform-console-standalone-1.10.2.jar -d out src/com/example/Main.java src/com/example/TestArithmetic.java;
(cd out && jar --extract --file=../junit-platform-console-standalone-1.10.2.jar);
mkdir -p out/META-INF;
echo 'Main-Class: com.example.Main' > out/META-INF/MANIFEST.MF;
jar --create --file Tests.jar --no-manifest -C out .;

java -jar Tests.jar;

I’m not sure why, in this case --scan-class-path does not work. At least for this example, using --select-class com.example.TestArithmetic is good enough.

I assume there is a better way to execute the tests, the main use case of ConsoleLauncher is to be used from the console.

Validate jar file

Only recently ^🗄️ jar has a --validate option. The main use case is for validating a multi-release archive, but it will do some sanity checks on normal archives too.

If working with an older version of jar, there is still --list. It will not validate any content, but it can be used to verify that jar can at least read the content.

Why those notes

One might be wondering, why would one want to compile Java code by hand, download dependencies with wget, and create .jar manually.

There are two main reasons; the first one, is that build systems hide a lot of details, but when something does not work, it helps to know the basics in order to diagnose the issue.

The second main reason is projects that consist of multiple languages.

If one program uses Java and "another language", integrating different build systems might be problematic, just as it can be difficult to compile code of another language out of its environment.

Especially if the Java code is only a subset of a bigger project, it might be easier to extend the "other build system" to support Java too.

Do you want to share your opinion? Or is there an error, some parts that are not clear enough?

You can contact me anytime.