Skip to content

Conversation

@yashi
Copy link
Contributor

@yashi yashi commented Feb 25, 2021

rand() and srand() are pseudo-random number generator functions
defined in ISO C. This implementation uses the Linear congruential
generator algorithm with the following parameters, which are the same
as GNU Libc.

Modulus 0x7fffffff
Multiplier 1103515245
Increment 12345

ISO C / POSIX.1-2001 uses bits high 15 bits for Modulus, thus RAND_MAX
32767. ref rand(3).

Signed-off-by: Yasushi SHOJI [email protected]

@yashi yashi requested a review from nashif as a code owner February 25, 2021 12:42
@github-actions github-actions bot added the area: C Library C Standard Library label Feb 25, 2021
@yashi
Copy link
Contributor Author

yashi commented Feb 25, 2021

Or should I call sys_rand32_get(), instead?
https://docs.zephyrproject.org/latest/reference/random/index.html

@github-actions
Copy link

This pull request has been marked as stale because it has been open (more than) 60 days with no activity. Remove the stale label or add a comment saying that you would like to have the label removed otherwise this pull request will automatically be closed in 14 days. Note, that you can always re-open a closed pull request at any time.

@github-actions github-actions bot added the Stale label Apr 27, 2021
@yashi
Copy link
Contributor Author

yashi commented Apr 29, 2021

@enjiamai, @KangJianX would mind to give me some feedback?

@enjiamai
Copy link
Contributor

Hi @yashi , OK, please give me some time to take look at this, I'll feedback to you later. thanks!

@enjiamai
Copy link
Contributor

Hi @yashi , I have no problem with this implementation, it looks good, and I think we need this. I would so appreciate it if some basic test cases for this can also be added to tests/lib/c_lib/src/. How do you think? thank you!

@enjiamai enjiamai requested review from enjiamai and pfalcon April 29, 2021 16:33
Copy link
Contributor

@enjiamai enjiamai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please help to add the test cases for these, thank you!

@github-actions github-actions bot removed the Stale label Apr 30, 2021
@github-actions github-actions bot added the area: Tests Issues related to a particular existing or missing test label Apr 30, 2021
@yashi yashi requested a review from enjiamai April 30, 2021 16:58
Copy link
Contributor

@enjiamai enjiamai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great~ no problem from my side, thanks!

@yashi yashi force-pushed the libc-rand branch 2 times, most recently from 32fe35b to d05db7b Compare May 1, 2021 17:32
Copy link
Contributor

@pfalcon pfalcon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm always +1 for sustainably growing the minimal library. But IMHO there're few cleanups to be done and questions to be answered before this is ready for inclusion.

Another note - we usually try to put tests into a separate commit vs the core changes.

@pfalcon
Copy link
Contributor

pfalcon commented May 5, 2021

I also find the current commit message to be confusing:

            initstate(0, state, sizeof(state));
            printf("%d\n", rand());

            return 0;
    }

See initstate(3p) for more detail.

You don't implement initstate(), so why people should see its man? I however followed the advice and what I see is:

       long int random(void);

       void srandom(unsigned int seed);

       char *initstate(unsigned int seed, char *state, size_t n);

       char *setstate(char *state);

In other words, rand/srand vs random/initstate seems to be two parallel, independent pseudo-random number generation function sets. You here don't implement the latter, so references to initstate() should rather be removed. Likewise, please double-check that rand/srand implementation matches that of glibc. If it is, please update the sample in the commit message to use rand/srand. And if it's not, I'd suggest to just use the sample implementation from POSIX: https://pubs.opengroup.org/onlinepubs/9699919799/functions/rand.html and remove references to glibc.

(And if there's some confusion on my side, please let me know what I'm missing.)

@yashi
Copy link
Contributor Author

yashi commented May 6, 2021

In order to test the Linear congruential generator algorithm, which is refered as TYPE_0 in GLIBC, you have to call initstate() with a state array of size 8 bytes or smaller than 32 byte. Please note that the man page you referred is 3 not 3p, or 3posix, from Linux man-pages project.

I've updated my commit log as below for more clarity.

See initstate(3p) for more detail about how to use LCG in GLIBC.

@yashi yashi requested a review from pfalcon May 6, 2021 07:01
@pfalcon
Copy link
Contributor

pfalcon commented May 6, 2021

@yashi

In order to test the Linear congruential generator algorithm, which is refered as TYPE_0 in GLIBC, you have to call initstate() with a state array of size 8 bytes or smaller than 32 byte.

That's an interesting implementation detail of GLIBC, but I don't see how that relates to the functions srand()/rand() which are being implemented here. Nothing in https://pubs.opengroup.org/onlinepubs/9699919799/functions/rand.html mentions initstate(). Nor you implement initstate(). So, why do you mention it at all?

Let's start from the beginning. Do you want Zephyr's implementation of srand()/rand() to have the same behavior/produce the same values as Glibc's srand()/rand()? If so, I'm generally +1 for that, as Glibc is a well known library and I'm for being consistent with well-know de-facto standard. But then please sure that you exactly follow behavior of srand()/rand() and don't involve unrelated implementation details, like the fact that in Glibc, srand() is implemented in terms of initstats().

And well, if you don't have that specific requirement to follow Glibc behavior, just use the simple algorithm from "Generating the Same Sequence on Different Machines" subheading of https://pubs.opengroup.org/onlinepubs/9699919799/functions/rand.html .

Please note that the man page you referred is 3 not 3p, or 3posix, from Linux man-pages project.

That page still described the functions:

       char *initstate(unsigned seed, char *state, size_t size);
       long random(void);
       char *setstate(char *state);
       void srandom(unsigned seed);

And not srand()/rand().

@yashi
Copy link
Contributor Author

yashi commented May 6, 2021

@pfalcon, thanks for your feedback. The reason why I've chosen GLIBC LCG is that I assumed most of the developers in the Zephyr community prefer to have the same algorithm as their development environment, which is, I also assume, Linux with GLIBC. I don't have any preferences at all. And the reason I put the reference in the commit log is that when someone wants to verify my implementation, it'd be nice to give a point or two for them. I thought it'd be helpful for future references. Nothing else.

I followed GLIBC LCG's behavior in the Zephyr implementation. No detail, nor TYPE_0 thing is in Zephyr implementation. The reference is just for a hint to who wants verification in behavior. If the default algorithm in GLIBC is LCG, I didn't put the reference in the commit log, but it's not. If you just run rand() and srand() on your Linux, you are using a different algorithm and you get different values.

Again, I don't have any preferences in the algorithms. I just want rand() and srand() for compatibility. I can either remove the reference from the commit log or implement the C standard. Please let me know which way you want.

@yashi yashi force-pushed the libc-rand branch 3 times, most recently from a1411ec to 291b848 Compare May 6, 2021 12:47
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this include really needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nop. I'll leave stdlib.h there for checking the rand() and srand() declarations against the definitions.

@yashi
Copy link
Contributor Author

yashi commented Jun 22, 2021

@stephanosio, I saw the MAINTAINERS.yml change and you are now the maintainer of libcs. Would you mind reviewing this PR? I just rebased to the main. Thanks.

@stephanosio stephanosio self-requested a review June 22, 2021 06:57
@cfriedt
Copy link
Member

cfriedt commented Jul 16, 2021

@yashi - can you please address any remaining comments?

@yashi
Copy link
Contributor Author

yashi commented Jul 17, 2021

Yes. Will do it in this weekend.

@yashi yashi self-assigned this Jul 18, 2021
@yashi yashi force-pushed the libc-rand branch 2 times, most recently from d600da5 to aa471fd Compare July 18, 2021 07:38
@yashi yashi requested a review from stephanosio July 18, 2021 08:04
@stephanosio stephanosio requested a review from gmarull July 19, 2021 12:51
Copy link
Member

@stephanosio stephanosio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some additional comments

yashi added 7 commits July 20, 2021 09:30
rand() and srand() are pseudo-random number generator functions
defined in ISO C. This implementation uses the Linear Congruential
Generator (LCG) algorithm with the following parameters, which are the
same as used in GNU Libc "TYPE_0" algorithm.

  Modulus 2^31
  Multiplier 1103515245
  Increment 12345
  Output Bits 30..0

Note that the default algorithm used by GNU Libc is not TYPE_0, and
TYPE_0 should be selected first by an initstate() call as shown below.

All global variables in a C library must be routed to a memory
partition in order to be used by user-mode applications when
CONFIG_USERSPACE is enabled.  Thus, srand_seed is marked as
such. z_libc_partition is originally used by the Newlib C library but
it's generic enough to be used by either the minimal libc or the
newlib.

All other functions in the Minimal C library, however, don't require
global variables/states.  Unconditionally using z_libc_partition with
the minimal libc might be a problem for applications utilizing many
custom memory partitions on platforms with a limited number of MPU
regions (eg. Cortex M0/M3). This commit introduces a kconfig option
CONFIG_MINIMAL_LIBC_RAND so that applications can enable the
functions if needed.  The option is disabled by default.

Because this commit _does_ implement rand() and srand(), our coding
guideline check on GitHub Action finds it as a violation.

    Error: lib/libc/minimal/include/stdlib.h:45:WARNING: Violation to
    rule 21.2 (Should not used a reserved identifier) - srand

But this is false positive.

The following is a simple test program for LCG with GNU Libc.

  #include <stdio.h>
  #include <stdlib.h>

  int main()
  {
          static char state[8];

          /* Switch GLIBC to use LCG/TYPE_0 generator type. */
          initstate(0, state, sizeof(state));

          srand(1);  /* Or any other value. */
          printf("%d\n", rand());
          printf("%d\n", rand());

          return 0;
  }

See initstate(3p) for more detail about how to use LCG in GLIBC.

Signed-off-by: Yasushi SHOJI <[email protected]>
When MBEDTLS_RSA_C is defined, mbedtls define its local version of
rand() function.  Since we already have rand() in our minimal libc, we
can safely remove this.

Signed-off-by: Yasushi SHOJI <[email protected]>
 - simple tests for each function
 - a reproducibility test for rand with the same seed values

Signed-off-by: Yasushi SHOJI <[email protected]>
Under the "OS Abstraction / POSIX Support" section in the "User and
Developer Guides", we somehow had rand() function already listed and
marked supported, but not srand().  Add srand() to the list.

Signed-off-by: Yasushi SHOJI <[email protected]>
Some Kconfig options are left marked as inline literals.  But in
Zephyr document, we use the "kconfig" role provided by Sphinx.

Signed-off-by: Yasushi SHOJI <[email protected]>
Mark variables up if used in plain sentences for easier reading.

Signed-off-by: Yasushi SHOJI <[email protected]>
The global variables in the Minimal C library are now placed in
z_libc_partition, update the document accordingly.

Signed-off-by: Yasushi SHOJI <[email protected]>
@cfriedt
Copy link
Member

cfriedt commented Jul 20, 2021

fixes #6493

Copy link
Member

@stephanosio stephanosio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@cfriedt cfriedt merged commit ab7f07d into zephyrproject-rtos:main Jul 20, 2021
@yashi yashi deleted the libc-rand branch July 21, 2021 06:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: API Changes to public APIs area: C Library C Standard Library area: Documentation area: Kernel area: Tests Issues related to a particular existing or missing test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants