[WIP] use RandomTest to define random distributions for tests

[rfourquet]

This is based off #648, so only the last commit is new here.

This uses the RandomTest.jl package to define easily new distributions which can be used for testing.
Starting with an example:

julia> using AbstractAlgebra, RandomTest

julia> randt(ZZ, 10)
10-element Array{BigInt,1}:
 -56650976381335138235754141204422689
                               106353
                                  -20
                                    7
                                    7
                                   -1
                                   31
                                  -56
                                    0
                      -18191886763417

randt(x) is equivalent to rand(test(x)), where test(x) creates a distribution, which ideally generates different "kinds" of values (whatever that means), including corner cases.
Another example:

julia> R, x = ZZ["x"];

julia> randt(R, 5)
5-element Array{AbstractAlgebra.Generic.Poly{BigInt},1}:
 -x^20 - 2*x^19 - 3*x^18 - x^16 - x^15 - 2*x^14 - 3*x^13 - 2*x^12 - x^10 - x^9 - 2*x^8 - x^7 - x^6 + 2*x^5 - x^3 - 2*x^2 - x - 1
 x^20 - 19*x^19 - 2*x^17 - x^16 + 221*x^15 + x^13 - 31*x^12 - x^11 - 2*x^8 - 2*x^7 - x^5 + 22*x^4 + 49*x^2 - 4*x
 -x - 1
 -1
 -x^17 - 2*x^16 - 2*x^15 - 786*x^14 - x^9 - 2*x^7 - x^6 - x^3 - 7*x

One of the benefits of using randt compared to rand is when the parameters to supply to rand are non very meaningful and arbitrary, then randt can generate relatively sensible parameter.

The couple implemented methods here, and modified examples, contain few comments to hopefully help understand how it works (there is of course no documentation yet).

This PR is currently just a preview. It also includes a (even) more speculative example using a @quickcheck macro (very rudimentary implementation), inspired by Haskell's QuickCheck library (I don't have a better link right now).
Unlike in the original QuickCheck, the focus is more on "scaling the size distribution of the generated values" rather than controlling directly the size. For example, to get bigger numbers than above:

julia> rand(scale(5, test(ZZ)), 10)
10-element Array{BigInt,1}:
                                                                                                                         -8664794006
                                                                                                                                -880
                                                                                                                           162545982
                           -10357559959107108727771427394847732419490268649410200387398639185727769360051063349600926020926791431022
 32186247245162422880932587914577229487125664700150545068048814970476825028737043270732181531127430350681341794026507099231043569702
                                                                                                                                  -9
                                                                                                                           -32465504
                                                                                                                                -691
                                                                                                             14715724230660819341839
                                                                                                                                   0

Of course, we can also define test methods which take a parameter to directly influence the size, e.g we could have test(ZZ, 10) or whatnot where 10 suggests or imposes a size.

The RandomTest package is not registered, you need to clone yourself in order to run the examples above.

[rfourquet]

CC @fingolfin

[thofma]

Can we make this a test dependency?

[rfourquet]

Can we make this a test dependency?

Absolutely, I was about to mention it but didn't want to overload the already long OP. Ideally it would be an optional dependency when Pkg implements it (i.e. loaded only when RandomTest is loaded). The advantage of not having this code in tests is that it's then easy to use at the REPL which I find convenient for users.

[fingolfin]

Thanks @rfourquet this looks really promising to me. And sorry for not commenting earlier, I somehow missed it :/.

The following is somewhat orthogonal to your PR and your RandomTest.jl (both of which I still need to study in more detail), but I thought I'd throw it out here anyway while I am here:

Personally, one of the things I'd want/need on the long run is being able to, say, create 10 random invertible Oscar matrices (so not Array) of degree 30 over a basering like ZZ or GF(5) or so.

For the former, I'd imagine that I'd write something like randt(GL(30,ZZ)), as that expresses the purposes precisely (this is also in line with your example randt(ZZ). Of course one wouldn't/shouldn't write it exactly like that in code, more like G = GL(degree,basering) ; .... randt(G) to avoid the overhead of creating that group again and again.

But I am somewhat concerned that some code really, really doesn't want to go to the effort of creating this group object just to create random matrices (unless we tune it very carefully so it can be turned into a zero-cost abstraction). That code then would need a way to specify the desired output in another way. But types don't quite cut it, as you know, as the same type might be shared by several different finite fields (resp. matrices over them), and also the dimension of the matrix is not in the type in general. Also, we still would need a way to convey the information the invertible elements are desired. I guess that could be taken off by suitable MatrixGroupElem types (modulo the specification of dimensions and basering).

Perhaps this is a non-issue and I worry for nothing? (To be clear: even if there is reason to worry, that's not an objection to this PR, it's a wider issue). Perhaps there are good suggestions how to deal with this?

[rfourquet]

Thanks @fingolfin for your comment. So a first remark is that to have randt defined for your object X, you must define test(X) to return a distribution, which typically would invoke a function defined outside of RandomTest, e.g. using make from my other PRs. So your question is not limited to the context of tests.

So, my first thought is that ideally, creating parent objects is cheap, and if some useful precomputed stuff is expensive, it would be computed lazily. For example, creating in AbstractAlgebra a 2x3 ZZ matrix (initialized from an array) is much more expensive than creating the parent object, at least when we don't cache it:

julia> @btime matrix($ZZ, 2, 3, $(rand(1:9, 2, 3)))
  399.070 ns (14 allocations: 400 bytes)
[3  5  9]
[8  9  9]

julia> @btime MatrixSpace($ZZ, 2, 3)
  84.410 ns (0 allocations: 0 bytes)
Matrix Space of 2 rows and 3 columns over Integers

julia> @btime MatrixSpace($ZZ, 2, 3, false) # don't cache
  13.371 ns (1 allocation: 32 bytes)
Matrix Space of 2 rows and 3 columns over Integers

So creating the parent object is relatively not very significant when creating 10 random matrices.
But then, as parent objects are not stored in AA matrices, I agree that it might be beneficial if it was possible to specify a distribution for these matrices which don't involve the parent object. With the other PR, you specify a distribution for matrices via make, e.g.

julia> M = MatrixSpace(ZZ, 2, 3)
Matrix Space of 2 rows and 3 columns over Integers

julia> rand(make(M, 1:9))
[5  1  2]
[7  1  2]

I see two obvious ways to specify the same distribution without creating M: via a dedicated struct, e.g. MakeMatrixElement(ZZ, 2, 3), or via defining another make method (my prefered solution), e.g. make(MatElem, ZZ, elem_distrib, 2, 3)) or make(MatElem{BigInt}, elem_distrib, 2, 3) (elem_distrib is e.g. 1:9 like in the example above). Or shorter, make(MatElem, elem_distrib, 2, 3) where elem_distrib knows that it creates ZZ objects.

Now, for invertible matrices, assuming there is a MatrixGroupElem type as you suggested, we could have similarly make(MatrixGroupElem, elem_distrib, 3).
Then we could define test with something like: test(::Type{MatrixGroupElem}, ZZ, n::Int) = Sized(sz -> make(MatrixGroupElem, 1:sz, n), test(ZZ)) (where Sized is a "combinator" from RandomTest). And why not defining test(::Type{MatrixGroupElem}, ZZ) which chooses n at random.

[thofma]

But I am somewhat concerned that some code really, really doesn't want to go to the effort of creating this group object just to create random matrices (unless we tune it very carefully so it can be turned into a zero-cost abstraction).

Are there situations where the creation is non-negligible compared to the subsequent call to rand(G)? Of course one should not create GF(p) or Z/nZ if one wants efficiently uniform integers in the range 1:p or 1:n.

[fieker]

On Sat, Oct 10, 2020 at 04:10:41AM -0700, thofma wrote: > But I am somewhat concerned that some code really, _really_ doesn't want to go to the effort of creating this group object just to create random matrices (unless we tune it very carefully so it can be turned into a zero-cost abstraction). Are there situations where the creation is non-negligible compared to the subsequent call to `rand(G)`? Of course one should not create `GF(p)` or `Z/nZ` if one wants efficiently uniform integers in the range `1:p` or `1:n`.

rand(GF(p)) SHOULD yield an gfp_elem rand(1:p) an Int rand(1:fmpz(p)) an fmpz even if they are computer science or programming equivalent, in AA they are not

…

-- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: #655 (comment)

[lgoettgens]

RandomTest.jl hasn't had any maintenance since Oct 2020, and isn't registered. Thus I do not see us using this in the near future.