State objects encapsulate the state on which benchmark is working on.
For example, I want to know how many times a method gets executed while performing the benchmark. To do this, I can maintain a counter that can be shared across all the benchmark execution.
State variables are declared in special state classes, and an instance of that state class can then be provided as a parameter to the benchmark method.
Step 1: Define a State class.
@State(Scope.Benchmark)
public static class BenchmarkState {
private AtomicLong totalMethodExecutionCount;
@Setup(Level.Trial)
public void doSetup() {
totalMethodExecutionCount = new AtomicLong(0);
System.out.println("Do Setup");
}
@TearDown(Level.Trial)
public void doTearDown() {
System.out.println("Benchmark Method executed " + totalMethodExecutionCount.longValue() + " times");
}
}
Step 2: Pass the BenchmarkState variable as an argument to benchmark method.
@Benchmark
@BenchmarkMode(Mode.AverageTime)
public void measureShared1(BenchmarkState state) throws InterruptedException {
TimeUnit.MILLISECONDS.sleep(100);
state.totalMethodExecutionCount.getAndIncrement();
}
With this logic, I can check how many times the method ‘measureShared1’ is called by benchmarking.
Find the below working application.
BenchmarkStateDemo.java
package com.sample.app;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicLong;
import org.openjdk.jmh.annotations.Benchmark;
import org.openjdk.jmh.annotations.BenchmarkMode;
import org.openjdk.jmh.annotations.Level;
import org.openjdk.jmh.annotations.Mode;
import org.openjdk.jmh.annotations.Scope;
import org.openjdk.jmh.annotations.Setup;
import org.openjdk.jmh.annotations.State;
import org.openjdk.jmh.annotations.TearDown;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.RunnerException;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;
public class BenchmarkStateDemo {
@State(Scope.Benchmark)
public static class BenchmarkState {
private AtomicLong totalMethodExecutionCount;
private String methodName = null;
@Setup(Level.Trial)
public void doSetup() {
totalMethodExecutionCount = new AtomicLong(0);
System.out.println("Do Setup");
}
@TearDown(Level.Trial)
public void doTearDown() {
System.out.println("Benchmark Method " + methodName + " executed " + totalMethodExecutionCount.longValue() + " times");
}
}
@Benchmark
@BenchmarkMode(Mode.AverageTime)
public void measureShared1(BenchmarkState state) throws InterruptedException {
TimeUnit.MILLISECONDS.sleep(100);
state.methodName = "measureShared1";
state.totalMethodExecutionCount.getAndIncrement();
}
@Benchmark
@BenchmarkMode(Mode.AverageTime)
public void measureShared2(BenchmarkState state) throws InterruptedException {
TimeUnit.MILLISECONDS.sleep(200);
state.methodName = "measureShared2";
state.totalMethodExecutionCount.getAndIncrement();
}
public static void main(String[] args) throws RunnerException {
Options opt = new OptionsBuilder()
.include(BenchmarkStateDemo.class.getSimpleName())
.forks(1)
.measurementIterations(2)
.warmupIterations(2).build();
Runner runner = new Runner(opt);
runner.run();
}
}
Output
# JMH version: 1.35
# VM version: JDK 15.0.2, Java HotSpot(TM) 64-Bit Server VM, 15.0.2+7-27
# VM invoker: /Library/Java/JavaVirtualMachines/jdk-15.0.2.jdk/Contents/Home/bin/java
# VM options: -Dfile.encoding=UTF-8 -XX:+ShowCodeDetailsInExceptionMessages
# Blackhole mode: full + dont-inline hint (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
# Warmup: 2 iterations, 10 s each
# Measurement: 2 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: com.sample.app.BenchmarkStateDemo.measureShared1
# Run progress: 0.00% complete, ETA 00:01:20
# Fork: 1 of 1
# Warmup Iteration 1: Do Setup
0.103 s/op
# Warmup Iteration 2: 0.103 s/op
Iteration 1: 0.103 s/op
Iteration 2: Benchmark Method measureShared1 executed 392 times
0.103 s/op
Result "com.sample.app.BenchmarkStateDemo.measureShared1":
0.103 s/op
# JMH version: 1.35
# VM version: JDK 15.0.2, Java HotSpot(TM) 64-Bit Server VM, 15.0.2+7-27
# VM invoker: /Library/Java/JavaVirtualMachines/jdk-15.0.2.jdk/Contents/Home/bin/java
# VM options: -Dfile.encoding=UTF-8 -XX:+ShowCodeDetailsInExceptionMessages
# Blackhole mode: full + dont-inline hint (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
# Warmup: 2 iterations, 10 s each
# Measurement: 2 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: com.sample.app.BenchmarkStateDemo.measureShared2
# Run progress: 50.00% complete, ETA 00:00:45
# Fork: 1 of 1
# Warmup Iteration 1: Do Setup
0.203 s/op
# Warmup Iteration 2: 0.203 s/op
Iteration 1: 0.203 s/op
Iteration 2: Benchmark Method measureShared2 executed 200 times
0.203 s/op
Result "com.sample.app.BenchmarkStateDemo.measureShared2":
0.203 s/op
# Run complete. Total time: 00:01:31
REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
experiments, perform baseline and negative tests that provide experimental control, make sure
the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
Do not assume the numbers tell you what you want them to tell.
Benchmark Mode Cnt Score Error Units
BenchmarkStateDemo.measureShared1 avgt 2 0.103 s/op
BenchmarkStateDemo.measureShared2 avgt 2 0.203 s/op
From the output, you can see the below lines.
Iteration 2: Benchmark Method measureShared1 executed 392 times Iteration 2: Benchmark Method measureShared2 executed 200 times
A state object can have following scopes.
a. Benchmark: Benchmark state scope. All threads running the benchmark share the same state object.
b. Group: Group state scope. Each thread group running the benchmark will create its own instance of the state object.
c. Thread: Each thread running the benchmark will create its own instance of the state object.
Let’s confirm it by below example.
StatesDemo.java
package com.sample.app;
import java.util.concurrent.TimeUnit;
import org.openjdk.jmh.annotations.Benchmark;
import org.openjdk.jmh.annotations.BenchmarkMode;
import org.openjdk.jmh.annotations.Mode;
import org.openjdk.jmh.annotations.Scope;
import org.openjdk.jmh.annotations.State;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.RunnerException;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;
public class StatesDemo {
@State(Scope.Benchmark)
public static class BenchmarkState {
}
@State(Scope.Thread)
public static class ThreadState {
}
@Benchmark
@BenchmarkMode(Mode.AverageTime)
public void benchmarkState(BenchmarkState benchmarkState) throws InterruptedException {
TimeUnit.SECONDS.sleep(1);
String threadName = Thread.currentThread().getName();
String threadGroupName = Thread.currentThread().getThreadGroup().getName();
System.out.println("threadName : %s, threadGroupName %s, benchmarkState : %s".formatted(threadName,
threadGroupName, benchmarkState));
}
@Benchmark
@BenchmarkMode(Mode.AverageTime)
public void threadState(ThreadState threadState) throws InterruptedException {
TimeUnit.SECONDS.sleep(1);
String threadName = Thread.currentThread().getName();
String threadGroupName = Thread.currentThread().getThreadGroup().getName();
System.out.println("threadName : %s, threadGroupName %s, threadState : %s".formatted(threadName,
threadGroupName, threadState));
}
public static void main(String[] args) throws RunnerException {
Options opt = new OptionsBuilder()
.include(StatesDemo.class.getSimpleName())
.threads(3)
.threadGroups(2)
.forks(1)
.measurementIterations(1)
.warmupIterations(1)
.build();
Runner runner = new Runner(opt);
runner.run();
}
}
Run the above application, you will observe following messages in the console.
threadName : com.sample.app.StatesDemo.benchmarkState-jmh-worker-1, threadGroupName main, benchmarkState : com.sample.app.jmh_generated.StatesDemo_BenchmarkState_jmhType@148ccf8e threadName : com.sample.app.StatesDemo.benchmarkState-jmh-worker-2, threadGroupName main, benchmarkState : com.sample.app.jmh_generated.StatesDemo_BenchmarkState_jmhType@148ccf8e threadName : com.sample.app.StatesDemo.benchmarkState-jmh-worker-4, threadGroupName main, benchmarkState : com.sample.app.jmh_generated.StatesDemo_BenchmarkState_jmhType@148ccf8e threadName : com.sample.app.StatesDemo.benchmarkState-jmh-worker-2, threadGroupName main, benchmarkState : com.sample.app.jmh_generated.StatesDemo_BenchmarkState_jmhType@148ccf8e
Same benchmark state object ‘com.sample.app.jmh_generated.StatesDemo_BenchmarkState_jmhType@148ccf8e’ is shared across all the method invocations.
threadName : com.sample.app.StatesDemo.threadState-jmh-worker-4, threadGroupName main, threadState : com.sample.app.jmh_generated.StatesDemo_ThreadState_jmhType@148ccf8e threadName : com.sample.app.StatesDemo.threadState-jmh-worker-3, threadGroupName main, threadState : com.sample.app.jmh_generated.StatesDemo_ThreadState_jmhType@32263bb4 threadName : com.sample.app.StatesDemo.threadState-jmh-worker-2, threadGroupName main, threadState : com.sample.app.jmh_generated.StatesDemo_ThreadState_jmhType@3e078932 threadName : com.sample.app.StatesDemo.threadState-jmh-worker-1, threadGroupName main, threadState : com.sample.app.jmh_generated.StatesDemo_ThreadState_jmhType@261a9ff6 threadName : com.sample.app.StatesDemo.threadState-jmh-worker-4, threadGroupName main, threadState : com.sample.app.jmh_generated.StatesDemo_ThreadState_jmhType@148ccf8e threadName : com.sample.app.StatesDemo.threadState-jmh-worker-3, threadGroupName main, threadState : com.sample.app.jmh_generated.StatesDemo_ThreadState_jmhType@32263bb4 threadName : com.sample.app.StatesDemo.threadState-jmh-worker-2, threadGroupName main, threadState : com.sample.app.jmh_generated.StatesDemo_ThreadState_jmhType@3e078932 threadName : com.sample.app.StatesDemo.threadState-jmh-worker-1, threadGroupName main, threadState : com.sample.app.jmh_generated.StatesDemo_ThreadState_jmhType@261a9ff6 threadName : com.sample.app.StatesDemo.threadState-jmh-worker-4, threadGroupName main, threadState : com.sample.app.jmh_generated.StatesDemo_ThreadState_jmhType@148ccf8e
Each worker thread maintains a separate ThreadState object.
Previous Next Home
No comments:
Post a Comment