Chronicle Test Framework

About

Chronicle Test Framework provides test-data generators and API metrics tools for JUnit 4 and JUnit 5. Use the combinators to create permutations, combinations and products, or analyse your packages with the metrics builder.

Permutations

Permutations work like this:

@Test
void print() {
    Permutation.of("A", "B", "C")
            .forEach(System.out::println);
}

This will produce the following output:

[A, B, C]
[A, C, B]
[B, A, C]
[B, C, A]
[C, A, B]
[C, B, A]

Testing using permutations:

@TestFactory
// Exhaustively verifies that any order of setter invocation yields the same result
Stream<DynamicTest> demo() {
    // Operations
    final Consumer<MyBean> setA = myBean -> myBean.setA(1);
    final Consumer<MyBean> setB = myBean -> myBean.setB(2);
    final Consumer<MyBean> setC = myBean -> myBean.setC(3);
    final MyBean expected = new MyBean(1, 2, 3);
    // DynamicTests
    return DynamicTest.stream(Permutation.of(setA, setB, setC),
            Objects::toString,
            operations -> {
                final MyBean actual = new MyBean();
                operations.forEach(oper -> oper.accept(actual));
                assertEquals(expected, actual);
            });
}

API Metrics Builder

The API metrics feature analyses the public and internal surface of your packages. Create the builder via ApiMetrics.builder() then supply the packages to scan, the metrics to apply and the accumulators that gather the results.

ApiMetrics metrics = ApiMetrics.builder()
        .addStandardMetrics()
        .addStandardAccumulators()
        .addPackage("com.example")
        .build();

The builder applies no defaults. If you omit metrics or accumulators the result will be empty. Default metrics and accumulators are only added when you call the relevant addStandard methods. Packages with .internal. in the name, or those ending in .internal, are collected in a separate internal group.

Combinations

Combinations work like this:

@Test
void print() {
    Combination.of("A", "B", "C")
            .forEach(System.out::println);
}

This will produce the following output:

[]
[A]
[B]
[C]
[A, B]
[A, C]
[B, C]
[A, B, C]

Testing using combinations:

@TestFactory // Try all combinations of cosmic ray interference for the Robot state machine
Stream<DynamicTest> demo() {
        return DynamicTest.stream(Combination.<Consumer<FaultTolerantBitSet>>of(
                        FaultTolerantBitSet::cosmicRayBit3,
                        FaultTolerantBitSet::cosmicRayBit23,
                        FaultTolerantBitSet::cosmicRayBit13),
                Objects::toString,
                operations -> {
                    final FaultTolerantBitSet bitSet = new FaultTolerantBitSet();
                    operations.forEach(oper -> oper.accept(bitSet));
                    assertTrue(bitSet.isValid());
                });
    }

Combinations and Permutations

Combinations and Permutations can be combined to create powerful exhaustive test vectors:

    @Test
    void demo() {
        Combination.of("A", "B", "C")
                .flatMap(Permutation::of)
                .forEach(System.out::println);
    }

This will produce the following output:

[]
[A]
[B]
[C]
[A, B]
[B, A]
[A, C]
[C, A]
[B, C]
[C, B]
[A, B, C]
[A, C, B]
[B, A, C]
[B, C, A]
[C, A, B]
[C, B, A]

Products

Products are equivalent to nested loops but are easier to convert to a Stream of DynamicTest objects:

    @Test
    void print() {
        Product.of(Arrays.asList("A", "B", "C"), Arrays.asList(1, 2, 3))
                .forEach(System.out::println);
    }

This will produce the following output:

Product2Impl{first=A, second=1}
Product2Impl{first=A, second=2}
Product2Impl{first=A, second=3}
Product2Impl{first=B, second=1}
Product2Impl{first=B, second=2}
Product2Impl{first=B, second=3}
Product2Impl{first=C, second=1}
Product2Impl{first=C, second=2}
Product2Impl{first=C, second=3}

Products can use built-in tuples like Product2Impl or we can provide custom constructors to use our own. Testing using products:

@TestFactory
// Exhaustively tests if various empty collections invariants holds
Stream<DynamicTest> demo() {
    // Operations
    final List<Collection<Integer>> collections = Arrays.asList(new LinkedList<>(), new ArrayList<>(), new HashSet<>());
    // Operations
    final Consumer<Collection<Integer>> empty =
            c -> assertTrue(c.isEmpty(), c.getClass() + ".empty() was false");
    final Consumer<Collection<Integer>> size =
            c -> assertEquals(0, c.size(), c.getClass() + ".size() != 0");
    final Consumer<Collection<Integer>> streamCount =
            c -> assertEquals(0, c.stream().count(), c.getClass() + ".stream().count() != 0");
    final List<Consumer<Collection<Integer>>> operations = Arrays.asList(empty, size, streamCount);

            // DynamicTests
    return DynamicTest.stream(Product.of(collections, operations),
            Objects::toString,
            tuple -> {
                tuple.second().accept(tuple.first());
            });
}

API Metrics Accumulators

The framework includes a small set of predefined accumulator suppliers used by the API metrics analysis. They help to group metric counts in common ways.

• perMethod() groups by the full method signature so that overloaded methods are reported separately.

• perClassAndMetric() groups first by class and then by metric, giving a table of metric totals for each class.

• PER_METRIC provides totals for each metric across all classes.

• PER_CLASS totals all metrics for each class without distinction.

• PER_PACKAGE reports a single total for each package.

• PER_METHOD_REFERENCE groups by method name only, ignoring parameter types, so all overloads are counted as one.

The convenience factories in Accumulator expose the first two suppliers for general use. The others are mainly for internal analyses but may be reused when custom behaviour is needed.