JUnit 5 is amazing, and after all these years since its initial release, there is hardly any reason why you would ever need to fall back to an older revision of the framework - unless you exercise a bunch of Robolectric tests, that is. As Android developers, we are burdened by the duality of our system, however.

Let me back up a little. The Android testing ecosystem divides itself into two sub-categories: Unit tests and instrumentation tests. The former ideally do not interact with Android platform types directly, making it possible to execute them on any JVM without the need for a real device. Most of your testing efforts should be spent on these tests, as they are small in scope, high in number, and quick to execute.

But then, there’s the other stuff. You know, the tests that actually require a physical Android device. The sort of solution that usually comes up when trying to answer any of the following questions:

• Will that one button show correctly on this screen?
• Does the migration of my Realm database work correctly?
• Is my “Share” Intent picking up the correct result?

Android instrumentation testing currently does not support anything newer than JUnit 4, as the entire system is built on top of it. Despite this bleak statement, there is an effort to make it possible to run JUnit 5 on your Android device, and it’s being attempted by yours truly. In this post, I’d like to outline how the Android JUnit 5 Instrumentation Test Library can fake its way into the hostile environment of your (reasonably modern) Android phone.

Enhance the Old with Something New

One of the steps you need to take when using JUnit 5 for instrumentation tests is to add the following line to your build.gradle:

android {
    defaultConfig {
        testInstrumentationRunnerArgument(
            "runnerBuilder", 
            "de.mannodermaus.junit5.AndroidJUnit5Builder"
        )
    }

The runnerBuilder argument is one of the more obscure things that we can pass to the Android testing runtime. Basically, it allows us to hook in custom execution logic into the AndroidJUnitRunner at runtime. An extension of the abstract RunnerBuilder class must be public itself and requires a public, no-argument constructor, and is called for any class in a test run. The only abstract method in the class can return a JUnit 4 Runner object, or null if the RunnerBuilder doesn’t recognize the test class.

Our implementation for Android JUnit 5 uses this as an entry point into the JUnit 4 world, bridging the gap to the JUnit Platform via a custom Runner.

class AndroidJUnit5Builder : RunnerBuilder() {
    override fun runnerForClass(testClass: Class<*>): Runner? {
        if (!junit5Available) {
            return null
        }

        // Omitted: Check for JUnit 5 annotations...

        return createJUnit5Runner(testClass)
    }
}

Sneaky API Check without causing Trouble

The junit5Available check in the last code snippet uses reflection to determine the availability of JUnit 5 APIs at runtime. The reasoning is simple: JUnit 5 requires Java 8, which Android devices didn’t completely support before API Level 26. We have to be very careful not to include most of the JUnit Platform libraries on the compilation classpath, since otherwise your apps would require a minSdkVersion of 26 as well, just for running the tests. Instead, these dependencies are bundled as “runtime-only” dependencies in order to bypass this restriction.

The effect? If you executed your instrumentation test suite on a phone running API Level 26 or later, the JUnit 5 tests would be executed alongside their wrinkly older cousins, written against the older JUnit 4 API. If you executed the same suite on a very old phone without support for Java 8, then the JUnit 5 stuff would be effectively disabled with a log message. Looking behind the curtain of the createJUnit5Runner() method above, we can see how this works:

internal fun createJUnit5Runner(klass: Class<*>): Runner =
    if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {
      AndroidJUnit5(klass)
    } else {
      DummyJUnit5(klass)
    }

The DummyJUnit5 doesn’t actually attempt to load any of the Java 8 code (which would crash the older phone), but instead it disables the test class altogether. On the other hand, the AndroidJUnit5 implementation is free to access any of the JUnit Platform code.

Test Discovery and the Way Back

The actual implementation of the JUnit 5 Runner for Android is written in Java - gasp! This has historical reasons, as it originated from the built-in backwards compatibility layer for JUnit 4, included in the JUnit Platform as a separate library. Over time, the Android version evolved beyond the original scope, adding support for more and more JUnit 5 features in the context of an older environment, but the Java language kept sticking to it.

For all of you Minecraft players out there, we haven’t let go of the Shift key just yet, so we’re still in sneaky territory here. In order to trick the Android instrumentation further, we generate a custom version of the TestPlan class, because the Android instrumentation would otherwise mangle the test names of dynamic tests into an unreadable mess. This is done by means of the AndroidJUnitPlatformTestTree class, again written in Java to leverage the package-private visibility of some JUnit types.

public final class AndroidJUnitPlatformTestTree {
    private final ModifiedTestPlan testPlan;
}

// Launcher: JUnit 5 type
// RunNotifier: JUnit 4 type
public final class AndroidJUnit5 extends Runner {
    private final AndroidJUnitPlatformTestTree testTree;
    private final Launcher launcher = LauncherFactory.create();

    @Override
    public void run(RunNotifier notifier) {
        // Omitted: Environment variables etc...
        
        launcher.execute(
            testTree.getTestPlan(),
            new AndroidJUnitPlatformRunnerListener(testTree, notifier)
        );
    }
}

Under the hood, the JUnit Platform’s Launcher API is used to actually execute the tests on the platform. It’s possible to add listeners to the JUnit 5 execution, so we do just that and receive the results via a TestExecutionListener, mapping the events back for the JUnit 4 RunNotifier. Quite a ride, but we managed to get there.

Conclusion

I’m not sure if this overview of the innards of Android JUnit 5’s instrumentation library is of use to anybody, but hopefully you still had a good time. Next, I might talk about some of the actually useful features that we can leverage in a world where Android and JUnit 5 coexist.

Get started with JUnit 5 today!