The Java Microbenchmarking Harness (JMH) is a nice tool to alleviate most of the problems that arise with (micro-)benchmarking in Java. It is fairly easy to use if you use the recommended setup of using two separate maven projects - one for the application code, and one for the benchmarks. It is, however, more convenient to keep the benchmarks in a separate source set within the same project and I personally prefer Gradle over Maven anytime.

Luckily there is a plugin for running JMH benchmarks from Gradle, but it can be a bit clunky and is actually a little bit of an overkill if you just want to run some small benchmarks from within your project. This left me to wonder: How hard can it actually be to write a manual Gradle config to include the necessary dependencies for JMH in your project? A quick google search actually revealed several examples where others successfully implemented such a build script with seemingly little effort.

However, life is never easy if you like shiny new things. I wanted to use Gradle 5, since I really like the idea to write my build scripts with the Gradle Kotlin DSL instead of pure dynamically typed Groovy. Of course, none of the examples I found online used Kotlin, so I translated this example from Groovy to Kotlin to get a minimal working build configuration to use JMH in a Java project with Gradle 5 and the Gradle Kotlin DSL.

plugins {
    `java-library`
}

repositories {
    jcenter()
}

sourceSets.create("jmh") {
    java.setSrcDirs(listOf("src/jmh/java"))
}

dependencies {
    "jmhImplementation"(project)
    "jmhImplementation"("org.openjdk.jmh:jmh-core:1.21")
    "jmhAnnotationProcessor"("org.openjdk.jmh:jmh-generator-annprocess:1.21")
}

tasks {
    register("jmh", type=JavaExec::class) {
        dependsOn("jmhClasses")
        group = "benchmark"
        main = "org.openjdk.jmh.Main"
        classpath = sourceSets["jmh"].runtimeClasspath
        // To pass parameters ("-h" gives a list of possible parameters)
        // args(listOf("-h"))
    }
}

This build configuration does the following:

  • Create a new source set jmh, so that you can have your JMH benchmarks in a separate directory src/jmh/java.
  • Make the project itself and jmh-core available as compile time dependency for the benchmarks.
  • Register jmh-generator-annprocess as annotation processor for the jmh source set. This one took me a while since it was not apparent from the Gradle 4 example or the Maven configuration generated by the recommended setup. Since Gradle 4.6 there exists a separate jmhAnnotationProcessor dependency, so you cannot just add the annotation processor as jmhImplementation dependency. This would lead to the error message Unable to find the resource: /META-INF/BenchmarkList since the benchmark annotations where not processed.
  • Add a task jmh that runs the benchmarks (without creating an uber JAR as Maven or the JMH-Plugin would do). In this task you can (and probably will have to) specify several command line parameters that can be passed to the benchmarks. To get a list of possible parameters, you can use args(listOf("-h")).