The Android logo, released under public domain

Download Java dependencies

Notes published the 22 Jul, 2025
11 - 14 minutes to read, 2746 words
Categories: android, build system, java
Keywords: android, archive, build system, java

Table of Contents

Parallel use from multiple processes
annotation vs annotation-jvm
Dependency hell
Extract proguard rules
Handle dependency conflicts
Create the correct classpath
- Create classpath
- Copy relevant files in separate folder
Configuration file instead of command-line arguments
Other useful settings
Dependency tree
Conclusion

While building a minimal Android application without Gradle, I avoided bringing in any dependencies, as it brings it’s own set of issues.

The most used way to publish dependencies is through Maven repositories, and the build systems Gradle and Maven are both able to interact with them, and download during the build the declared dependencies.

Normally I would prefer to download the dependencies before building the software. It makes it easier to have a more predictable build process.

Unfortunately both Maven and Gradle are not able to download a given list of dependencies in a given folder. And if you are not using those build system, tracking the dependencies by hand is too error-prone.

Fortunately there already exist a tool that can work with Maven repositories, is not a build system, and does not depend on a build system (don’t be fooled by the URL): Apache Ivy.

While there is a strong focus about Ant, another build system for Java, it can be used from the command line directly.

It main job is to download dependencies in a folder, transitive dependencies included. Perfect.

This is a minimum configuration file for downloading dependencies from https://mvnrepository.com and https://maven.google.com:

ivysettings.xml

<ivysettings>
  <settings defaultResolver="chain"/>
  <resolvers>
    <chain name="chain">
      <ibiblio name="maven-central" m2compatible="true"/>
      <ibiblio name="google-maven"  m2compatible="true" root="https://maven.google.com"/>
    </chain>
  </resolvers>
</ivysettings>

For downloading a specific package from the command-line, you need to specify the organisation (in this example androidx.annotation), the module (annotation) and the revision (1.9.1):

java -jar /usr/share/java/ivy.jar -settings ivysettings.xml -cache $PWD/cache -dependency androidx.annotation annotation 1.9.1

The output looks like

:: loading settings :: file = /home/df0/Workspace/snippets/cmake-android/ivysettings.xml
:: resolving dependencies :: androidx.annotation#annotation-caller;working
        confs: [default]
        found androidx.annotation#annotation;1.9.1 in google-maven
        found androidx.annotation#annotation-jvm;1.9.1 in google-maven
        found org.jetbrains.kotlin#kotlin-stdlib;1.9.24 in maven-central
        found org.jetbrains#annotations;13.0 in maven-central
downloading https://maven.google.com/androidx/annotation/annotation/1.9.1/annotation-1.9.1.jar ...
.......... (11kB)
.. (0kB)
        [SUCCESSFUL ] androidx.annotation#annotation;1.9.1!annotation.jar (791ms)
downloading https://maven.google.com/androidx/annotation/annotation/1.9.1/annotation-1.9.1-sources.jar ...
....................... (28kB)
.. (0kB)
        [SUCCESSFUL ] androidx.annotation#annotation;1.9.1!annotation.jar(source) (796ms)
downloading https://maven.google.com/androidx/annotation/annotation-jvm/1.9.1/annotation-jvm-1.9.1.jar ...
............................................... (59kB)
.. (0kB)
        [SUCCESSFUL ] androidx.annotation#annotation-jvm;1.9.1!annotation-jvm.jar (701ms)
downloading https://repo1.maven.org/maven2/org/jetbrains/kotlin/kotlin-stdlib/1.9.24/kotlin-stdlib-1.9.24.jar ...
........................................................................................................... (1678kB)
.. (0kB)
        [SUCCESSFUL ] org.jetbrains.kotlin#kotlin-stdlib;1.9.24!kotlin-stdlib.jar (363ms)
downloading https://repo1.maven.org/maven2/org/jetbrains/annotations/13.0/annotations-13.0.jar ...
... (17kB)
.. (0kB)
        [SUCCESSFUL ] org.jetbrains#annotations;13.0!annotations.jar (104ms)
:: resolution report :: resolve 5733ms :: artifacts dl 2774ms
        ---------------------------------------------------------------------
        |                  |            modules            ||   artifacts   |
        |       conf       | number| search|dwnlded|evicted|| number|dwnlded|
        ---------------------------------------------------------------------
        |      default     |   4   |   4   |   4   |   0   ||   5   |   5   |
        ---------------------------------------------------------------------

Note 📝

you might want to create a wrapper script ivy instead of typing java -jar /usr/share/java/ivy.jar every time.

Parallel use from multiple processes

One thing I like to criticize about Maven ^🗄️ and Gradle ^🗄️ is the inability to use a shared cache folder for storing the downloaded data for multiple independent build processes.

So one of the first thing I wanted to know was if it is fine to start multiple ivy processes and let the download concurrently dependencies in a shared folder.

Since I could not find anything in the documentation, I believe such feature is not supported. This is not necessarily an issue, because ivy does not handle the whole build process. Thus serializing the download process should be good enough for most use-cases.

On POSIX systems, you can use flock

(
  flock --wait "$TIMEOUT"  --exclusive 9 || exit 1;
  # execute apache ivy for downloading data
) 9>file.lock

or, if uoy just need to execute a single command, or the logic is already packaged in a script:

flock --wait "$TIMEOUT"  --exclusive $COMMAND

while on Windows system, you can open from PowerShell a file with exclusive access, unfortunately there is no built-in timeout mechanism

for($i = 0; $i -lt $max_retries; $i++){
  try{
    $fs = New-Object System.IO.FileStream("file.lock", [System.IO.FileMode]::OpenOrCreate, [System.IO.FileAccess]::Write, [System.IO.FileShare]::None);
  } catch {
    Start-Sleep -Second $interval;
  }
}
if( $i -ge $max_retries ){
  Write-Host "Another instance is already running";
  exit 1;
}
# execute apache ivy for downloading data

If you are using a more high-level language, chances are that there already exists a multiplatform solution; for example python has the portallocker library:

import portalocker

with portalocker.Lock('test') as fh:
  # execute apache ivy for downloading data

From a higher-level perspective, the workflow looks like the following

serialized download of dependencies, read-write access to cache folder
concurrent build, read-only access to cache folder

As long as serialized download of dependencies does not modify already downloaded archives, there are no race conditions.

With Gradle and Maven, I found no way to set up such workflow.

Contrary to Gradle, the cache is not cleaned up automatically and periodically. If you plan to clean it up, ensure that no one is building in the meantime, or at least accessing the relevant files.

`annotation` vs `annotation-jvm`

The annotation package depends on the package annotation-jvm.

I’m not sure why there is this second package, but the Keep class is in annotation-jvm. You can avoid thus thus to download one package by using following command

java -jar /usr/share/java/ivy.jar -settings ivysettings.xml -cache $PWD/cache -dependency androidx.annotation annotation-jvm 1.9.1

Dependency hell

If you want to use even a simple annotation like @Keep, you need to bring in the whole Kotlin standard library.

For @Keep alone, as it offers no functionality, this sounds like an overkill. I guess some other annotation might need the Kotlin standard library.

This is the reason why, if possible, I would prefer to avoid dependencies, and by using Maven or Gradle, one might not even realize what dependencies are pulled in, and why.

Extract proguard rules

I’ve tried to integrate annotation-jvm in my CMake project, and use the @Keep annotation.

One just needs to add the .jar file to the class-path, and the code compiles successfully.

But if you remove following line parameter from the proguard invocation:

-keep "public class com.example.hello.HelloFromCPP{public static void log();}"

then the function log of HelloFromCPP will be removed, breaking the functionality of the test application.

It turns out, that proguard does not know anything about the @Keep annotation. What is important, is the proguard configuration file located inside the annotation-jvm-1.9.1.jar file, in the META-INF/proguard/ folder.

proguard, when using -libraryjars and -injars, does not automatically acknowledge those files, thus one needs to extract them, and use -include parameter explicitly.

A helper script like

extract-proguard-rules.sh

#!/bin/sh
set -e
set -u

FILE="$1"; shift
OUTFILE="$1"; shift

# assume no space in output
PROFILES=$(jar --list --file="$FILE" "META-INF/proguard/" | grep ".*\\.pro");
if [ -z "$PROFILES" ]; then :;
  touch "$OUTFILE";
else :;
  TMPDIR=${XDG_RUNTIME_DIR:-${TMPDIR:-${TMP:-${TEMP:-/tmp}}}}
  TMPDIR="$(mktemp -p "$TMPDIR" -d ff.XXXXXX.d)";
  cd "${TMPDIR}";
  jar --extract --file="$FILE" $PROFILES;
  cat $PROFILES > "$OUTFILE";
fi

can be used to extract all proguard rules in a single file with jar and cat:

extract-proguard-rules.sh file.jar out.pro

If you have unzip at your disposal, you do not even need to create temporary files:

unzip -p file.jar "META-INF/proguard/*.pro" > out.pro

In CMake, this can be summarized more or less like

set(PROGUARD_RULES)
foreach(DEP IN LISTS DEPENDENCIES) # DEPENDENCIES contains list of jar files
  cmake_path(GET DEP FILENAME filename)
  add_custom_command(
    OUTPUT  ${CMAKE_BINARY_DIR}/${filename}.pro
    DEPENDS ${DEP}
    COMMAND  ${CMAKE_BINARY_DIR}/extract_proguard_from_jar ${DEP} ${CMAKE_BINARY_DIR}/${filename}.pro
    COMMENT "extract proguard rules of ${DEP} to ${CMAKE_BINARY_DIR}/${filename}.pro"
    VERBATIM
  )
  set(PROGUARD_RULES "${PROGUARD_RULES};${CMAKE_BINARY_DIR}/${filename}.pro")
endforeach()


add_custom_command(
  OUTPUT  ${OPTIMIZED_CLASS}
  DEPENDS ${CLASS_FILES} ${PROGUARD_RULES} ${DEPENDENCIES} ${APP_GEN_PROGUARD_RULES}
  COMMAND ${PROGUARD} -dontobfuscate
                      -injars ${CLASS_FILES}
                      -include ${APP_GEN_PROGUARD_RULES} # references com.example.MainActivity, generated by aapt
                      # additional dependencies and theyr proguard rules
                      # note might generate the "You don't need to keep library classes; they are already left unchanged." warning
                      -libraryjars ${DEPENDENCIES}
                      -include ${PROGUARD_RULES}
                      -outjars ${OPTIMIZED_CLASS}
  COMMENT "optimize .class files"
  VERBATIM
)

Handle dependency conflicts

In my case, I’ve used the Kotlin compiler packaged by the system, with it’s own standard library. Now through annotation-jvm there is a second Kotlin standard library. The build system needs to choose one library to use.

Gradle and Maven ^🗄️ use two different strategies, Ivy seems to be configurable ^🗄️. I did not test yet what happens and how the conflict manager works.

No matter which approach is used, with my current approach CMake should be instructed to analyze the dependencies, and see it the Kotlin standard library is added transitively. If it does, it needs to decide which standard library to use.

A similar issue also exists when creating and consuming own libraries, although I assume in that case it is clear which library one should prefer, while for the Kotlin library it might not be the case.

Create the correct classpath

There is another thing I’ve forgot to handle correctly at the beginning.

Not all jars in the cache folder needs to be added to the classpath, especially if the cache folder is used between multiple projects.

At first I did a simple find cache -name '*.jar' to find all downloaded archives, but if the same folder is used for multiple projects, it will also contains libraries that are not relevant for a given project.

I’ve found two ways to fix this issue with ivy.

Create classpath

Use -cachepath to create the classpath:

java -jar /usr/share/java/ivy.jar -cache "$PWD/cache" -cachepath classpath.txt -dependency androidx.annotation annotation 1.9.1

Once the command finishes, classpath.txt will contain the classpath for adding annotation and the transitive dependencies.

Note 📝

when downloading annotation, and asking classpath for annotation-jvm, then no error. Otherwise error but classpath is there.

Copy relevant files in separate folder

Use -retrieve:

java -jar /usr/share/java/ivy.jar -cache "$PWD/cache" -retrieve "dependencies/[organisation]-[artifact]-[revision]-[type].[ext]" -dependency androidx.annotation annotation 1.9.1

This should copy the relevant .jar files from $PWD/cache to dependencies.

If copying is to expensive, one could consider using the -symlink option.

You might also want to remove the -sources.jar and keep only the -jar.jar files.

Unfortunately there is no option for using hard links, otherwise it would have made it even easier to create a workflow that periodically cleans the cache folder, without the overhead of a copy.

Configuration file instead of command-line arguments

In all examples I’ve specified the dependency on the command-line.

If you have more than one dependency, you are out of luck.

The way to go would be to create a ivy.xml configuration file, and specify there all dependencies.

This is a minimal configuration file for downloading the annotation package:

ivy.xml

<ivy-module version="2.0">
  <info module="" />
  <dependencies>
    <dependency org="androidx.annotation" name="annotation" rev="1.9.1" />
  </dependencies>
</ivy-module>

To use it, replace -dependency androidx.annotation annotation 1.9.1 with -ivy ivy.xml

Other useful settings

ivysettings.xml

<ivysettings>
  <settings defaultResolver="chain"/>
  <property name="ivy.default.ivy.user.dir" value="/tmp/test/ivywd" override="true"/>
  <caches defaultCacheDir="${ivy.default.ivy.user.dir}/cache"/>
  <resolvers>
    <chain name="chain">
      <ibiblio name="maven-central" m2compatible="true"/>
      <ibiblio name="google-maven"  m2compatible="true" root="https://maven.google.com"/>
    </chain>
  </resolvers>
</ivysettings>

By setting ivy.default.ivy.user.dir, Apache Ivy wont create the folder ~/.ivy2. I’m not sure what this folder should contain anything if a cache folder is always specified on the command-line.

But I would like to to use ivysettings only when downloading files; I want to avoid downloading something by accident. When using -verbose on -retrieve, one can see how it searches inside ~/.ivy2.

Thus either create a separate ivysettings.xml, or specify as command-line parameter -Divy.home=/nop or another invalid (or valid) path.

Dependency tree

There seem to be no direct way to pretty-print a dependency graph, but all information seem to be contained in the top-level *-default.xml file.

After cleaning up by hand the xml file (I’ve removed the license nodex, most attributes) The xml file looks like (I’ve cleaned up the copied part by a lot)

cache/*-default.xml

<ivy-report version="1.0">
  <dependencies>
    <module organisation="androidx.annotation" name="annotation">
      <revision name="1.9.1" homepage="https://developer.android.com/jetpack/androidx/releases/annotation#1.9.1">
      <caller/>
      <artifacts>
        <artifact name="annotation" />
        <artifact name="annotation" location="/tmp/test/cache/androidx.annotation/annotation/sources/annotation-1.9.1-sources.jar"/>
      </artifacts>
      </revision>
    </module>
    <module organisation="androidx.annotation" name="annotation-jvm">
      <revision name="1.9.1" homepage="https://developer.android.com/jetpack/androidx/releases/annotation#1.9.1">
        <caller organisation="androidx.annotation" name="annotation" />
        <artifacts>
          <artifact name="annotation-jvm" />
        </artifacts>
      </revision>
    </module>
    <module organisation="org.jetbrains.kotlin" name="kotlin-stdlib">
      <revision name="1.9.24" homepage="https://kotlinlang.org/">
        <caller organisation="androidx.annotation" name="annotation"/>
        <caller organisation="androidx.annotation" name="annotation-jvm"/>
        <artifacts>
          <artifact name="kotlin-stdlib"/>
        </artifacts>
      </revision>
    </module>
    <module organisation="org.jetbrains" name="annotations">
      <revision name="13.0" homepage="http://www.jetbrains.org">
        <caller organisation="org.jetbrains.kotlin">
        <artifacts>
          <artifact name="annotations" />
        </artifacts>
      </revision>
    </module>
  </dependencies>
</ivy-report>

The XML files does indicate what are the dependencies of an artifact, but does the opposite; why and artifact is there at all. This is described by the caller node.

I assume that with XSLT it is possible to create an svg image with a nice tree.

An alternate approach is to look at the corresponding xml file of every artifact; for example androidx.annotation/annotation/ivy-1.9.1.xml contains following entries

<?xml version="1.0" encoding="UTF-8"?>
<ivy-module version="2.0" xmlns:m="http://ant.apache.org/ivy/maven">
  <!-- ... -->
  <dependencies>
    <dependency org="org.jetbrains.kotlin" name="kotlin-stdlib" rev="1.9.24" force="true" conf="runtime->compile(*),runtime(*),master(*)"/>
    <dependency org="androidx.annotation" name="annotation-jvm" rev="1.9.1" force="true" conf="compile->compile(*),master(*);runtime->runtime(*)"/>
  </dependencies>
</ivy-module>

Conclusion

I’ve never used Apache Ivy before, but at least from what I could see it seems like an appropriate tool for managing dependencies for Android and Java projects.

Unfortunately Apache Ivy alone is not sufficient. Even ignoring the fact that one needs to extract the proguard rules by hand, there are still other features that one might expect whe coming from Gradle, or at least third party libraries are taking advantage of.

At least one feature I am aware of is merging manifests files ^🗄️.

If you have questions, comments, or found typos, the notes are not clear, or there are some errors; then just contact me.