Skip to content

Investigate sparkline tests #1224

@vks

Description

@vks

The sparkline tests are much more likely to fail than the statistical assumptions of the test suggest. This should be better understood by studying the error distribution.

I modified the test by feeding all the numbers in the range 0..100 to the seed_from_u64 and these seeds failed at least one expected error bucket.

1 2 3 8 10 12 14 15 25 38 44 46 48 49 54 63 66 73 81 82 87 93

This means that 22/100 random seeds failed the unit test with error more than 3 standard deviations from expected.
If I allow 4 standard deviations, only a single seed fails 87
I get similar results if I use from_entropy or switch back to the pcg32 implementation.

The other solution would ditch the truly obvious seed of 1 and pick a fair random number like 4 to move on.

Originally posted by @ironhaven in #1203 (comment)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions