Skip to content

C++: Annotate field flow tests #3463

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
May 14, 2020

Conversation

MathiasVP
Copy link
Contributor

This PR adds more descriptive annotations to the field flow test directory.

@MathiasVP MathiasVP added the C++ label May 13, 2020
@MathiasVP MathiasVP requested a review from a team as a code owner May 13, 2020 13:17
@MathiasVP MathiasVP changed the title C++: Annotation field flow tests with [IR] and [AST] C++: Annotate field flow tests May 13, 2020
@dbartol
Copy link

dbartol commented May 13, 2020

This test would probably be a good candidate to use TestUtilities.InlineExpectationsTest, which actually looks at the comments in the test source files to determine its expectations. You can even mark lines as false positives or false negatives, and the test framework will let you know when they've been fixed. See cpp/ql/test/library-tests/ir/points_to/points_to.ql for example usage.

@MathiasVP
Copy link
Contributor Author

This test would probably be a good candidate to use TestUtilities.InlineExpectationsTest, which actually looks at the comments in the test source files to determine its expectations. You can even mark lines as false positives or false negatives, and the test framework will let you know when they've been fixed. See cpp/ql/test/library-tests/ir/points_to/points_to.ql for example usage.

That sounds awesome. @jbj did mention something like this, but I didn't manage to find it. I'll take a look at it and update this PR.

@MathiasVP MathiasVP added the WIP This is a work-in-progress, do not merge yet! label May 13, 2020
@MathiasVP MathiasVP removed the WIP This is a work-in-progress, do not merge yet! label May 14, 2020
Copy link
Contributor

@jbj jbj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like how the annotations are now self-maintaining, but I miss how the NOT DETECTED and FALSE POSITIVE annotations stood out visually. The f- and f+ annotations are practically invisible and look very similar. That's of course not an issue specific to this PR, so there's no need to address it here.

sink(b.f.a()); // flow [FALSE POSITIVE through `b2.f.setB` and `b3.f.setB` in AST, NOT DETECTED by IR]
sink(b.f.b()); // flow [FALSE POSITIVE through `b1.f.setA` (AST) and `b3.f.setA` in AST, NOT DETECTED by IR]
sink(b.f.a()); //$ast=flow 55:13 $ast=flow 57:13 $f-:ir=flow
sink(b.f.b()); //$ast=flow 56:13 $ast=flow 58:13 $f-:ir=flow
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was previously annotated as a false positive, but I can see that the comment was written before we increased the access-path approximation by one. So perhaps it's no longer a false positive?

Copy link
Contributor Author

@MathiasVP MathiasVP May 14, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On master there are 4 sources in total for these two sinks (and no false positives). If I add another field lookup (i.e., sink(b.g.f.a())) then we get the additional false positives the comment above mentions.

@MathiasVP
Copy link
Contributor Author

I like how the annotations are now self-maintaining, but I miss how the NOT DETECTED and FALSE POSITIVE annotations stood out visually. The f- and f+ annotations are practically invisible and look very similar. That's of course not an issue specific to this PR, so there's no need to address it here.

I agree. I think we should have an alternative syntax for specifying these cases other than f- and f+.

jbj
jbj previously approved these changes May 14, 2020
Copy link
Contributor

@jbj jbj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but I'll leave it to @dbartol to merge.

@dbartol
Copy link

dbartol commented May 14, 2020

I've opened https://github.com/github/codeql-c-analysis-team/issues/75 to come up with a better syntax. This is the first PR that actually uses the f+/- feature, so I'm glad to get feedback on it.

dbartol
dbartol previously approved these changes May 14, 2020
Copy link

@dbartol dbartol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM as-is, but see my suggestion about removing =flow.

@MathiasVP MathiasVP dismissed stale reviews from dbartol and jbj via 5f9b96c May 14, 2020 13:12
Copy link

@dbartol dbartol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants