The Java mascot Duke, by Joe Palrang, licensed under the New BSD license

Resource handling in Java

Notes published the 13 Jun, 2017
10 - 13 minutes to read, 2528 words
Categories: java
Keywords: java, leak, resources

Introduction

With the release of Java 7, Oracle presented a new way to handle resources: through the try-with-resources statement ^🗄️. Even if Java SE 7 is not supported since April 2015 anymore, there are a lot of people who still do not know this technique for managing resources, and those who use it are not always aware of the pitfalls.

Here’s a simple sample that uses a BufferedReader and a FileReader for reading the first line of a file.

static String readFirstLineFromFile(String path) throws IOException {
    try(BufferedReader br = new BufferedReader(new FileReader(path))){
        return br.readLine();
    }
}

and if we want to handle the resources manually, the following code should be equivalent.

static String readFirstLineFromFile(String path) throws IOException {
    final BufferedReader br = new BufferedReader(new FileReader(path));
    try{
        return br.readLine();
    }finally{
        try{
            br.close();
        }catch(IOException ex){
            // dismiss
        }
    }
}

The first version is shorter, it works out of the box since 2011, and has no runtime penalties. In fact, the first snippet is compiled to the same bytecode as the second version, so there seem to be no drawbacks. It’s a win-win situation.

The ugly truth

It seems to be a win-win situation unless you read the mentioned article till the end, where some pitfalls are explained, but no general solution is provided.

Who owns the resource?

When dealing with garbage-collected languages (not only Java), very often we are not interested in who is going to own the resources, let it be memory or some file handles. The GC (Garbage Collector) is going to release them for us, we do not need to worry about it. Except that there are some intrinsic limits about how many files we can open simultaneously, and on many situations, not releasing a file handle immediately after we are done, may provide a bad user experience. Think for example about a file on a thumb drive that we want to expel. This is why we should call the close method in the finally block or, since 2011, use the try-with-resources statement. Unfortunately, there are some edge cases, maybe they did never occur in your software, but they are still possible, where the resources are not released correctly.

Since we do not know who owns the resource, and therefore assure that it will be released and since releasing a resource more than once, may cause some error, there is no way to write some generic boilerplate code to cover all cases.

How does the try-with-resources statement work

The try-with-resources statement is just syntactic sugar for writing automatically a finally clause where the resource is released and the exception dismissed (even if we can query it later). It really shines when using more multiple or nested resources, since in those cases the required code for covering all scenarios increases exponentially, whereas with the try-with-resources statement only linearly. You can find a lot of examples, this is taken from the documentation:

public static void writeToFileZipFileContents(
    String zipFileName, String outputFileName) throws IOException {

    Charset charset = StandardCharsets.US_ASCII;
    Path outputFilePath = Paths.get(outputFileName);

    // Open the zip file and create an output file

    try(
        ZipFile zf = new ZipFile(zipFileName);
        BufferedWriter writer = new BufferedWriter(outputFilePath, charset)
    ){
        // Enumerate each entry
        for(Enumeration entries = zf.entries(); entries.hasMoreElements();){
            // Get the entry name and write it to the output file
            String newLine = System.getProperty("line.separator");
            String zipEntryName =((ZipEntry)entries.nextElement()).getName() + newLine;
            writer.write(zipEntryName, 0, zipEntryName.length());
        }
    }
}

One pitfall that is very often overlooked, but clearly documented, is that if we take my first example, readFirstLineFromFile, if the close method of the BufferedReader throws, the FileReader resource may be leaked. A simple solution would be to write

static String readFirstLineFromFile(String path) throws IOException {
    try(
        FileReader fr = new FileReader(path);
        BufferedReader br = new BufferedReader(fr)
    ){
        return br.readLine();
    }
}

This way, we can ensure that the FileReader object will be closed. If no error happens, then the FileReader object will be closed twice, which is not a problem for this class, but it may be a problem with other classes.

The try-with-resources statement works with all Classes that inherit the AutoCloseable interface ^🗄️. Those classes need to provide a close method, which can throw an Exception and does not need to be idempotent. This last statement makes it really hard to write code where we can be sure that resource leaks are avoided on all possible execution paths.

Code samples

I’ve experimented with different scenarios, the whole source code can be downloaded from HandleResourceTest.7z. In the next paragraph, I’ll show how to handle resources in specific scenarios, some generic solutions, and experiments, but as I’ve mentioned before, we have no way to ensure that a class that implements AutoCloseable will not be leaked.

First of all, let’s define some AutoCloseable classes that can throw an exception during construction, or while executing the close method, before or after releasing the internal resource, and of course a custom Exception class. For simplicity, we will simply print to the console if the close method has been called.

public class TestException extends RuntimeException{
    private static final long serialVersionUID = 1L;
}

public enum ThrowModus{
    doNotThrow, throwOnCreate, throwOnCloseBeforeInner, throwOnCloseAfterInner
}

public class MyAutoCloseableIntern implements AutoCloseable{
    final ThrowModus dothrow;

    public MyAutoCloseableIntern(final ThrowModus dothrow_){
        dothrow = dothrow_;
        if(dothrow == ThrowModus.throwOnCreate){
            throw new TestException();
        }
    }

    public MyAutoCloseableIntern(){
        dothrow = ThrowModus.doNotThrow;
    }

    @Override
    public void close(){
        System.out.println("Close MyAutoCloseableIntern");
        if(dothrow == ThrowModus.throwOnCloseAfterInner || dothrow == ThrowModus.throwOnCloseBeforeInner){
            throw new TestException();
        }
    }
}


public class MyAutoCloseableExtern implements AutoCloseable{
    final ThrowModus dothrow;
    final AutoCloseable in;

    public MyAutoCloseableExtern(final AutoCloseable in_, final ThrowModus dothrow_){
        in = in_;
        dothrow = dothrow_;
        if(dothrow == ThrowModus.throwOnCreate){
            throw new TestException();
        }
    }

    public MyAutoCloseableExtern(final AutoCloseable in_){
        in = in_;
        dothrow = ThrowModus.doNotThrow;
    }

    @Override
    public void close() throws Exception{
        System.out.println("Close MyAutoCloseableExtern");
        if(dothrow == ThrowModus.throwOnCloseBeforeInner){
            throw new TestException();
        }
        if(in != null){
            in.close();
        }
        if(dothrow == ThrowModus.throwOnCloseAfterInner){
            throw new TestException();
        }
    }
}

So, let’s try to use the close-with-resource-statement and see how it behaves

try(MyAutoCloseableExtern br = new MyAutoCloseableExtern(new MyAutoCloseableIntern())){
    // do nothing
}

Great, all resources are released exactly once. It couldn’t be any better.

try(MyAutoCloseableExtern br = new MyAutoCloseableExtern(new MyAutoCloseableIntern(), ThrowModus.throwOnCreate)){
    // do nothing
}catch(final TestException ex){
    // dismiss
}

If MyAutoCloseableExtern throws during construction, then MyAutoCloseableIntern is leaked, but we know how to handle it, right? Just use two instances.

try(
  MyAutoCloseableIntern fr = new MyAutoCloseableIntern();
  MyAutoCloseableExtern br = new MyAutoCloseableExtern(fr, ThrowModus.throwOnCreate)
){
    // do nothing
}catch(final TestException ex){
    // dismiss
}

try(
  MyAutoCloseableIntern fr = new MyAutoCloseableIntern();
  MyAutoCloseableExtern br = new MyAutoCloseableExtern(fr)
){
    // do nothing
}

If MyAutoCloseableExtern throws on construction, MyAutoCloseableIntern intern is closed, but if MyAutoCloseableExtern works as expected, then MyAutoCloseableIntern is closed twice. Aargh! Since all instances are declared as final we can’t reassign the variable fr to null to prevent the double closing. Maybe some helper function can help?

public static MyAutoCloseableExtern createMyAutoCloseableExtern(final ThrowModus throwModus){
    try(MyAutoCloseableIntern fr = new MyAutoCloseableIntern()){
        return new MyAutoCloseableExtern(fr, throwModus);
    }
}
...
try (MyAutoCloseableExtern br = createMyAutoCloseableExtern(ThrowModus.doNotThrow)) {
    // do nothing
}

Not there yet, this function always returns an already closed resource. This is the first place where we clearly see that there are use cases where we need, at least partially, to handle resources manually.

@SuppressWarnings("resource")
public static MyAutoCloseableExtern createMyAutoCloseableExtern(final ThrowModus throwModus){
    final MyAutoCloseableIntern fr = new MyAutoCloseableIntern();
    try{
        return new MyAutoCloseableExtern(fr, throwModus);
    }catch(final Throwable ex){
        try{
            fr.close();
        }catch(final Throwable ex2){
            ex.addSuppressed(ex2);
        }
        throw ex;
    }
}

...

try(MyAutoCloseableExtern br = createMyAutoCloseableExtern(ThrowModus.doNotThrow)){
    // do nothing
}

try(MyAutoCloseableExtern br = createMyAutoCloseableExtern(ThrowModus.throwOnCreate)){
    // do nothing
}catch(final TestException ex){
    // dismiss
}

try(MyAutoCloseableExtern br = createMyAutoCloseableExtern(ThrowModus.throwOnCloseBeforeInner)){
    // do nothing
}catch(final TestException ex){
    // dismiss
}

try(MyAutoCloseableExtern br = createMyAutoCloseableExtern(ThrowModus.throwOnCloseAfterInner)){
    // do nothing
}catch(final TestException ex){
    // dismiss
}

So, if everything works, then fine, but we still have problems if close throws. So far this is our best solution.

Managing resources manually

At this point I was out of ideas, let’s try to handle those resources manually and see if we can do better! After a little bit, this is what I got:

final ThrowModus throwModus = ...;
final boolean preferLearktoSideEffect = true;
MyAutoCloseableIntern fr = null;
try{
    fr = new MyAutoCloseableIntern();
    MyAutoCloseableExtern br = null;
    try{
        br = new MyAutoCloseableExtern(fr, throwModus);
        if(preferLearktoSideEffect && externOwnsIntern){
            fr = null;
        }

        // do nothing

        {
            // try to close the resources, not inside finally block in
            // order to avoid hiding a previous exception
            final MyAutoCloseableExtern brtmp = br;
            br = null;
            brtmp.close();

            if(!externOwnsIntern && fr != null){
                final MyAutoCloseableIntern frtmp = fr;
                fr = null;
                frtmp.close();
            }
        }
    }catch(final Throwable thr){
        if(br != null){
            try{
                br.close();
            }catch(final Throwable ex){
                thr.addSuppressed(ex);
            }
        }
        throw thr;
    }
}catch(final Throwable thr){
    if(fr != null){
        try{
            fr.close();
        }catch(final Throwable ex){
            thr.addSuppressed(ex);
        }
    }
    throw thr;
}

All this code and

We did not even do something sensible with those resources.
We still assume that the outer resource closes the internal resource, if this is true or not, it should be clearly stated in the documentation, see the BufferedReader close method ^🗄️ as an example. As far as I know, all classes in the JDK that take some Closeable in the constructor also release the resources when calling close.
If the close method of MyAutoCloseableExtern throws, since there is no method to query the status of AutoCloseable in its interface, we do not know if MyAutoCloseableExtern has been closed or not. If ThrowModus is set on throwOnCloseBeforeInner, we have still a leak.

We can at least try to wrap this code in a function. The code gets a lot cleaner since opening and closing the resource is encapsulated, but writing this function without the try-with-resource statement is cumbersome. Notice that the Consumer Interface ^🗄️ comes from Java 8, if we are still working on an older JVM we can simply define our interface with an accept method.

public class ConsumeMyAutoCloseableExtern implements Consumer<AutoCloseable>{
    @Override
    public void accept(final AutoCloseable r){
        // do nothing, do not close fr!
    }
}

...

@SuppressWarnings("resource")
public static void manually(final ThrowModus throwModus, final Consumer<AutoCloseable> cons, final boolean externOwnsIntern) throws Exception{
    final boolean preferLearktoSideEffect = true;
    MyAutoCloseableIntern fr = null;
    try{
        fr = new MyAutoCloseableIntern();
        MyAutoCloseableExtern br = null;
        try{
            br = new MyAutoCloseableExtern(fr, throwModus);
            if(preferLearktoSideEffect && externOwnsIntern){
                fr = null;
            }

            cons.accept(br);

            {
                // try to close the resources, not inside finally block in
                // order to avoid hiding a previous exception
                final MyAutoCloseableExtern brtmp = br;
                br = null;
                brtmp.close();

                if(!externOwnsIntern && fr != null){
                    final MyAutoCloseableIntern frtmp = fr;
                    fr = null;
                    frtmp.close();
                }
            }

        }catch(final Throwable thr){
            if(br != null){
                try{
                    br.close();
                }catch(final Throwable ex){
                    thr.addSuppressed(ex);
                }
            }
            throw thr;
        }
    }catch(final Throwable thr){
        if(fr != null){
            try{
                fr.close();
            }catch(final Throwable ex){
                thr.addSuppressed(ex);
            }
        }
        throw thr;
    }
}

...

final ConsumeMyAutoCloseableExtern call = new ConsumeMyAutoCloseableExtern();

manually(ThrowModus.doNotThrow, call, true);

try{
    manually(ThrowModus.throwOnCloseBeforeInner, call, true);
}catch(final TestException ex){
    // dismiss
}

try{
    manually(ThrowModus.throwOnCreate, call, true);
}catch(final TestException ex){
    // dismiss
}

try{
    manually(ThrowModus.throwOnCloseAfterInner, call, true);
}catch(final TestException ex){
    // dismiss
}

Long story short, if close throws, we can’t do a lot for the internal resource, mostly nothing. We may close it, maybe a second time, and take the risk that there are some unknown side-effects, or just leave it, and maybe get a resource leak, that at least should be handled by the GC if it ever gets called.

Conclusion

Time to draw some conclusions

Make interfaces easy to use correctly and hard to use incorrectly.

— Scott Meyers
author of Effective C++

If we are going to implement some class that implements AutoCloseable, let’s take a step further and just implement the Closeable interface ^🗄️. This way we are telling our consumers that our close method won’t have side effects if the resource has already been closed. If close is idempotent, managing resources is a lot easier since we can simply use the try-with-resource statement with multiple resources, even if those are nested. The resource will be closed multiple times, but it won’t be a problem.

If we need to use an AutoCloseable class, and it’s not stated that the close method is idempotent, we may wrap it in a Closeable class, or subclass it, and modify the close method to close the resource only once. This should be done with care since the class itself may close and reopen its instance and leverage on side effects. This method is therefore risky, and I wouldn’t recommend it unless we knew something more about the class itself.

If the class is not marked as final and nothing relies on calling close multiple times (may be difficult to prove, since there may be a reason why the method is not idempotent), you can override the close method in such a way that the super.close() is not called if the class has already been closed. Notice that close may be called by some private method and the class may be opened by some internal method too. Even if we could prove that close is not called internally, we have no way to prevent an update from changing this behavior without noticing. If overriding is not an option you can wrap it in another class and implement every member function you need, but this implementation is also brittle since close may be called internally, and the wrapping layer would not know if the resource has been closed or not.

If the class implements Closeable, or the documentation states that close does not have side effects on already-closed resources, use try-with-resources with multiple resources to ensure close is called in all cases.
If the external resource does not take ownership, try-with-resources with multiple instances is still the way to go.
If the external resource takes ownership, a helper function that ensures that the internal resource won’t get closed twice if there are no exceptions and that the internal resource will not be leaked if we are unable to create the external gives us the most robust solution.

The manual method does not scale at all, and it is not superior to the try-with-resource statement with the helper function unless you need to implement the helper function or return some non-closed resource. The code of the manual method was also written only for two resources, and it is much more complicated than it needs to be.

Therefore it is nearly impossible to write code in Java that never leaks resources, without knowing some implementation details. But should always remember that

if the application is not going to run for a long time, you will probably not reach the maximum connection or file handle limit.
if the application is going to run a long time (like a service), the garbage collector will help you, unless those exceptions when closing a resource happen very frequently (which hopefully won’t be your case).

You may think that normally when working with resources, you have more information. You normally do not pass instances of AutoCloseable around. Until you notice that BaseStream ^🗄️ implements AutoCloseable, and thus when doing some operations with streams, we are passing and using instances of AutoCloseable without knowing the actual implementation of the Stream class. Lucky us most of the time the close method is simply empty, but it may bite you. It would have been so much easier letting BaseStream extend Closeable, or even better not even extending that interface if most streams do nothing on close.

Do you want to share your opinion? Or is there an error, some parts that are not clear enough?

You can contact me anytime.