[WIP] Add Stack and Computation Benchmark

[Bhargavasomu]

What was wrong?

As part of the discussion here, a new Stack Benchmark is needed.

How was it fixed?

A TestStack.sol contract was written which would serve as the benchmark.

Cute Animal Picture

[Bhargavasomu]

@pipermerriam I still haven't made this a benchmark as of yet, but this is my first time where I am running a contract on the EVM, and I just had a small doubt.

• For the above TestStack Contract which you mentioned, I ran that code through our EVM, but the final stack values that I am seeing is [2729995958]. As per the above code shouldn't the expected value be [10,000].
• If at all this is stored in Memory (which is in bytes), how could I obtain the final value of v, because I would like to put an assert statement there.

Or is there anything else I am missing? Could you please clarify? Thankyou

[pipermerriam]

Can you rename scripts/stack_benchmark/contract_data to scripts/stack_benchmark/fixtures to better fit with our directory naming conventions elsewhere.

Can you work with @njgheorghita to package up the contract assets as an EthPM package? This would be good dogfooding of our own APIs and should allow us to leverage some higher level APIs for deploying and interacting with the contract and reduce some of the loading and parsing overhead in the test case.

[Bhargavasomu]

Also you can find the above stated values in py36-stack-benchmark

[pipermerriam]

I just realized I was commenting on a WIP so please just ignore anything I commented on that isn't done.

One change I'd ask for if you aren't already planning to is separating the renaming, added docstring, and other unrelated changes to a stand-alone PR -> eth1_hash,

[pipermerriam]

We tend towards using logging instead of print statements for this type of thing. That way we don't have to comment/uncomment them and can control verbosity and output destination using the logging APIs.

[pipermerriam]

Hrm, I'd favor this just being included in the main benchmark run instead of having a separate dedicated CI run.

[Bhargavasomu]

@pipermerriam I am extremely sorry, I think I messed up with git and it is showing some of the previous commits as mine. Will fix it immediately. The Merkle Tree related things aren't supposed to be part of this PR.

[pipermerriam]

For the benchmarking of this I think the final value doesn't matter very much. It's more about determining how many times the stack is pushed/popped from (and maybe even in what quantities).

I only just now realized that #1515 still hasn't been completed as it would provide an ideal API for this. Without that I think you're limited to manually measuring it using something like unittest.mock and then just hard coding the numbers in the benchmark.

Then, suppose the contract code results in 150 stack pushes and 150 stack pops, then you can output some metrics like N pops / second. You should probably run the computation 100 times or something like that to get a sufficiently large sample size.

Then later when #1515 is merged we can adjust the code to dynamically measure the number of push/pops.

[kclowes]

Closing, stale.