Improve Constraint Documentation

[aaron868]

Nice library. I've been experimenting with different tests but cannot find documentation about the possible constraints. For example, how do you constrain a String to length 1? Could you please add documentation (along with examples) on the constraints for each of the supported basic types?

Thanks.

[pholser]

@aaron868 Thanks for your interest in junit-quickcheck! Sorry for the late reply.

It's up to individual generators to decide when configuration annotations to admit. You can find these as methods with signature configure(A), where A is an annotation type that is itself marked as GeneratorConfiguration. This tells more about configuration annotations.

@When(satisfies = expr) is another way to constrain generated values. This may help.

For purposes of resolving this issue -- what form would you imagine the additional documentation to take?

[aaron868]

The construct @when(satisfies = expr) would work. I don't see, however, a reference guide for constraint expressions. The link you provided (http://pholser.github.io/junit-quickcheck/site/0.7/usage/constraining.html) does not provide one. I assume that the constraints allowed are the OGNL operators. Could you add a few more examples of using them in the this library?

[pholser]

@aaron868 Sure thing. Any OGNL expression that evaluates to boolean is allowed. #_ is taken to be the annotated parameter.

I'll add a couple more examples of the usage of @When(satisfies = ...). Thanks again for the feedback!

[bjordan2010]

This ocde is a variation on the documentation. Why does this test fail?

@Property(trials = 5)
public void when(@When(satisfies = "#_ > 1000 && #_ < 100000") int num)
{
    System.out.println("When: " + num);
    assertTrue(num > 0);
}

I get:

com.pholser.junit.quickcheck.internal.generator.PropertyParameterGenerationContext$DiscardRatioExceededException: For parameter [junitquickcheck.QuickCheck.when:arg0] with discard ratio [10], 11 unsuccessful values and 0 successes. Stopping.
at com.pholser.junit.quickcheck.internal.generator.PropertyParameterGenerationContext.evaluate(PropertyParameterGenerationContext.java:105)
at com.pholser.junit.quickcheck.internal.generator.PropertyParameterGenerationContext.generate(PropertyParameterGenerationContext.java:83)
at com.pholser.junit.quickcheck.internal.SeededValue.(SeededValue.java:37)

But if I change it to:

 @Property(trials = 5)
public void when(@When(satisfies = "#_ > 1000") int num)
{
    System.out.println("When: " + num);
    assertTrue(num > 0);
}

Then it works. It seems to me like the && is not doing an AND but an OR in the satisfies clause but perhaps I am missing something about how @when works.

[pholser]

@bjordan2010 Thanks for this ... will investigate.

[pholser]

@bjordan2010 I should mention also that if you want to guarantee that the generated values fall within the range you want, you can mark the parameter with @InRange:

    @Property public void inRange(@InRange(min = 1001, max = 9999) int i) {
        // ...
    }

[pholser]

@bjordan2010

@When(satisfies = "#_ > 1000 && #_ < 100000") int i will have junit-quickcheck generate a random int, then subject it to the "when" expression...if it matches, junit-quickcheck verifies the property, and if not, junit-quickcheck discards the value and tries again. If the ratio of discards to matches ever exceeds a certain ratio (10 by default), junit-quickcheck gives up. So @When(satisfies) is sort of a low-fidelity constraint on generation.

When the expr is 1000 < x < 100000, I suspect that very few values actually match; the default generation for integers can be anywhere from MIN_VALUE to MAX_VALUE, so the odds of getting enough hits before you exceed the discard ratio is small. When the expr is x < 1000, you stand a better chance of getting enough hits before that ratio is exceeded.

As noted above, your better bet is to use the @InRange annotation on the property parameter. All the numeric generators in junit-quickcheck-generators support this constraint.

[bjordan2010]

@pholser

Then the example of @when in the documentation 0 <= x <= 9 is also likely to fail since it alludes to looking for "Single Digits" for an integer correct? It makes it seem like those are the only values that will be generated but that is not the case.
That would require a custom generator. A test that will pass for an integer is one that is using @when for -2147483648 <= x <= 2147483647 since that satisfies clause would include all values of an integer. I hope I am understanding the @when clause correctly now.

[pholser]

@bjordan2010 Correct. I wonder if @Assuming would be a more evocative name for @When ... it kind of behaves like JUnit assumptions, with the idea of trying values until trials is met or there are too many non-matches.