-
Notifications
You must be signed in to change notification settings - Fork 0
Suggestions & updates #3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 6 commits
6f8f16c
e68df99
8e7ea25
57237cf
3beb954
54959e4
0cdf76d
acbd3b4
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -4,10 +4,14 @@ | |
| ================================================ | ||
|
|
||
| How do you know if your rules are good enough to catch potential bugs? | ||
| One technique is called "mutation testing" where small faults are injected in to the | ||
| program and it is checked against the same rule. Verification should fail if the rule | ||
| is good at catching the fault. If verification passes, that means your rule has gaps | ||
| that must be addressed. | ||
| One technique for assessing the quality of a specification is called "mutation testing". | ||
| Small modifications (mutations) are intetionally made, one | ||
| at a time, in the contract source code to cause logic faults. The modified code is checked | ||
| using your existing rules. It is important that your rules are passing with the original | ||
srunquist-certora marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| contract code before applying any mutations so that you can safely draw conclusions from | ||
| the mutation tests. With a mutation in the code, verification | ||
| will fail if the coverage of your rule set is good enough to catch the fault. If | ||
| verification still passes, that likely means your rules have a gap that must be addressed. | ||
|
|
||
| We have provided 3 hand-written mutants in | ||
| :clink:`src/certora/mutants <@token-proj/src/certora/mutants>`. | ||
|
|
@@ -19,10 +23,11 @@ Can you detect what the mutation was, for each mutant? | |
| You can see the solution in :file:`solutions/bugs-in-mutants.md`. | ||
|
|
||
|
|
||
| Note that there are other ways to assess the quality of your rule. | ||
| You can mutate the rule to see if it is vacuous, you can check if the rule is a | ||
| tautology, and you can use UNSAT cores to understand what parts of the code were | ||
| covered by the rule. | ||
| .. todo:: The following text needs more explanation / links: | ||
| Note that there are other ways to assess the quality of your rule. | ||
| You can mutate the rule to see if it is vacuous, you can check if the rule is a | ||
| tautology, and you can use UNSAT cores to understand what parts of the code were | ||
srunquist-certora marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| covered by the rule. | ||
|
||
|
|
||
| .. dropdown:: Mutant 1. Solution | ||
|
|
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -22,23 +22,25 @@ You should see an output like so: | |
| Follow your job and see verification results at <LINK> | ||
|
|
||
| Click on that link to see the report page generated by Certora Sunbeam. | ||
| It will show you that a basic :index:`sanity` check has passed. | ||
| You can see the *Source Files* on the report page. | ||
| There is also a *Call Trace* that shows you what sequence of function calls led to this | ||
| outcome. | ||
| It will show you that a basic :index:`sanity` check has passed, indicated | ||
| by a green check mark next to "sanity" in the Rules list. | ||
| You can see the set of *Source Files* that were used by clicking on the | ||
| File Tree icon to the left of the Rules list. | ||
| Now, click on the "satisfy" rule to display related information in the *Call Trace* | ||
| panel that shows you the startng state and sequence of function calls that led to this outcome. | ||
srunquist-certora marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| .. index:: satisfy | ||
|
|
||
| :clink:`src/certora/spec.rs <@token-proj/src/certora/spec.rs>` will show you the | ||
| *sanity* rule we just ran. | ||
| This rule simply calls :rust:`Token::balance()` and checks that the control reaches | ||
| the :cvl:`satisfy` statement that follows. | ||
| You can read more about :cvl:`satisfy` | ||
| the :cvl:`cvlr_satisfy` macro that follows. | ||
| You can read more about the semantics of :cvl:`satisfy` | ||
| `here <https://docs.certora.com/en/latest/docs/cvl/statements.html#satisfy>`_. | ||
|
|
||
| .. important:: | ||
|
|
||
| If you are not able to run certoraRun, see the `Sunbeam Troubleshooting`_ section in | ||
| If you are not able to run ``certoraSorobanProver``, see the `Sunbeam Troubleshooting`_ section in | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. good catch |
||
| the Documentation. | ||
|
|
||
| .. dropdown:: Solution | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,10 +1,7 @@ | ||
| { | ||
| "build_script": "../certora_build.py", | ||
| "process": "emv", | ||
| "rule": [ | ||
| "init_balance" | ||
| ], | ||
| "precise_bitwise_ops": true, | ||
| "prover_version": "master", | ||
| "server": "production" | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Any reason to keep There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. yes no good reason IMO There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. agreed! sorry this was my bad. Thanks for fixing. |
||
| "precise_bitwise_ops": true | ||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,11 +1,8 @@ | ||
| { | ||
| "build_script": "../certora_build.py", | ||
| "process": "emv", | ||
| "rule": [ | ||
| "transfer_is_correct", | ||
| "transfer_no_effect_on_other" | ||
| ], | ||
| "precise_bitwise_ops": true, | ||
| "prover_version": "master", | ||
| "server": "production" | ||
| "precise_bitwise_ops": true | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,10 +1,7 @@ | ||
| { | ||
| "build_script": "../certora_build.py", | ||
| "process": "emv", | ||
| "rule": [ | ||
| "transfer_fails_if_low_balance" | ||
| ], | ||
| "precise_bitwise_ops": true, | ||
| "prover_version": "master", | ||
| "server": "production" | ||
| "precise_bitwise_ops": true | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,13 +1,10 @@ | ||
| { | ||
| "build_script": "../certora_build.py", | ||
| "process": "emv", | ||
| "rule": [ | ||
| "init_balance", | ||
| "transfer_is_correct", | ||
| "transfer_no_effect_on_other", | ||
| "transfer_fails_if_low_balance" | ||
| ], | ||
| "precise_bitwise_ops": true, | ||
| "prover_version": "master", | ||
| "server": "production" | ||
| "precise_bitwise_ops": true | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,10 +1,7 @@ | ||
| { | ||
| "build_script": "../certora_build.py", | ||
| "process": "emv", | ||
| "rule": [ | ||
| "sanity" | ||
| ], | ||
| "precise_bitwise_ops": true, | ||
| "prover_version": "master", | ||
| "server": "production" | ||
| "precise_bitwise_ops": true | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -35,8 +35,6 @@ fn init_balance(e: Env, addr: Address) { | |
| // Exercise 2 | ||
| #[rule] | ||
| fn transfer_is_correct(e: Env, to: Address, from: Address, amount: i64) { | ||
| cvlr_assume!( | ||
| e.storage().persistent().has(&from) && e.storage().persistent().has(&to) && to != from); | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It makes logical sense to require these addresses to exist, but the rule passes without this: There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. you can remove them! I think i was just trying to make something to show the usage of |
||
| let balance_from_before = Token::balance(&e, from.clone()); | ||
| let balance_to_before = Token::balance(&e, to.clone()); | ||
| Token::transfer(&e, from.clone(), to.clone(), amount); | ||
|
|
@@ -61,13 +59,9 @@ fn transfer_no_effect_on_other(e: Env, amount: i64, from: Address, to: Address, | |
| // Exercise 3 | ||
| #[rule] | ||
| fn transfer_fails_if_low_balance(e: Env, to: Address, from: Address, amount: i64) { | ||
| cvlr_assume!( | ||
| e.storage().persistent().has(&from) | ||
| && e.storage().persistent().has(&to) | ||
| && to != from | ||
| && Token::balance(&e, from.clone()) < amount); | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The rule passes without these: So, better to simplify and generalize the rule(?) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. agreed, remove it! |
||
| cvlr_assume!(Token::balance(&e, from.clone()) < amount); | ||
| Token::transfer(&e, from.clone(), to.clone(), amount); | ||
| // should not reach and therefore rule must pass | ||
| // Should not reach the next line due to expected revert in transfer(). Rule fails iff it can be reached. | ||
| cvlr_assert!(false); | ||
| } | ||
|
|
||
|
|
||
Uh oh!
There was an error while loading. Please reload this page.