Skip to content

Conversation

@andrewrk
Copy link
Member

@andrewrk andrewrk commented Jul 22, 2024

Initial implementation of #20702.

  • Add the -ffuzz and -fno-fuzz CLI arguments.
  • Detect fuzz testing flags from zig cc.
  • Set the correct clang flags when fuzz testing is requested. It can be
    combined with TSAN and UBSAN.
  • Compilation: build fuzzer library when needed which is currently an
    empty zig file.
  • Add optforfuzzing to every function in the llvm backend for modules
    that have requested fuzzing.
  • In ZigLLVMTargetMachineEmitToFile, add the optimization passes for
    sanitizer coverage.
  • std.mem.eql uses a naive implementation optimized for fuzzing when
    builtin.fuzz is true.
  • Add the build system API for enabling fuzzing.
  • Add @disableInstrumentation builtin and use it in start code to prevent using threadlocal variables before threadlocal storage is initialized.
  • Makes the LLVM backend emit sancov calls.

Current status:

$ stage4/bin/zig build-exe ../test/standalone/simple/hello_world/hello.zig -ffuzz 
$ ./hello 
0x105a113: comparison of 0 and 9
0x105a183: switch on value 6 (32 bits) with 1 cases
0x105a113: comparison of 1 and 9
0x105a183: switch on value 1 (32 bits) with 1 cases
0x105a113: comparison of 2 and 9
0x105a183: switch on value 1 (32 bits) with 1 cases
0x105a113: comparison of 3 and 9
0x105a183: switch on value 1 (32 bits) with 1 cases
0x105a113: comparison of 4 and 9
0x105a183: switch on value 1 (32 bits) with 1 cases
0x105a113: comparison of 5 and 9
0x105a183: switch on value 7 (32 bits) with 1 cases
0x105a113: comparison of 6 and 9
0x105a183: switch on value 1685382482 (32 bits) with 1 cases
0x105a113: comparison of 7 and 9
0x105a183: switch on value 1685382480 (32 bits) with 1 cases
0x105a113: comparison of 8 and 9
0x105a183: switch on value 1685382481 (32 bits) with 1 cases
0x105a1d2: comparison of 0 and 0
0x109e4c8: comparison of 18446744073709547520 and 0
0x109e50a: comparison of 0 and 0
0x105afe1: switch on value 0 (16 bits) with 3 cases
0x105a20f: comparison of 0 and 0
0x105a25f: comparison of 16777216 and 8388608
0x109e4c8: comparison of 18446744073709547520 and 0
0x109e50a: comparison of 0 and 0
0x105b1b9: switch on value 0 (16 bits) with 4 cases
0x105a2e4: comparison of 0 and 0
0x109eeac: comparison of 1 and 11
0x109eecf: comparison of 9 and 11
0x109eef2: comparison of 19 and 11
0x109f041: comparison of 0 and 4
0x10b39f8: comparison of 18446744073709547520 and 0
0x10b3a3a: comparison of 0 and 0
0x109f273: comparison of 0 and 0
0x109e4c8: comparison of 18446744073709547520 and 0
0x109e50a: comparison of 0 and 0
0x105bafc: switch on value 0 (16 bits) with 3 cases
0x109ecbc: comparison of 0 and 0
0x109eeac: comparison of 1 and 4
0x109eecf: comparison of 9 and 4
0x109eef2: comparison of 19 and 4
0x109f041: comparison of 0 and 4
0x10b39f8: comparison of 18446744073709547520 and 0
0x10b3a3a: comparison of 0 and 0
0x109f273: comparison of 0 and 0
0x109e4c8: comparison of 18446744073709547520 and 0
0x109e50a: comparison of 0 and 0
0x105bafc: switch on value 0 (16 bits) with 3 cases
0x109ed0d: comparison of 0 and 0
0x109eeac: comparison of 1 and 7
0x109eecf: comparison of 9 and 7
0x109eef2: comparison of 19 and 7
0x109f041: comparison of 0 and 4
0x10b39f8: comparison of 18446744073709547520 and 0
0x10b3a3a: comparison of 0 and 0
0x109f273: comparison of 0 and 0
0x109e4c8: comparison of 18446744073709547520 and 0
0x109e50a: comparison of 0 and 0
0x105bafc: switch on value 0 (16 bits) with 3 cases
0x109ed5e: comparison of 0 and 0
0x109eeac: comparison of 1 and 8
0x109eecf: comparison of 9 and 8
0x109eef2: comparison of 19 and 8
0x109f041: comparison of 0 and 4
0x10b39f8: comparison of 18446744073709547520 and 0
0x10b3a3a: comparison of 0 and 0
0x109f273: comparison of 0 and 0
0x109e4c8: comparison of 18446744073709547520 and 0
0x109e50a: comparison of 0 and 0
0x105bafc: switch on value 0 (16 bits) with 3 cases
0x109edaf: comparison of 0 and 0
0x105b9e8: comparison of 0 and 0
0x109eeac: comparison of 1 and 13
0x109eecf: comparison of 9 and 13
0x109eef2: comparison of 19 and 13
0x109f041: comparison of 0 and 0
0x10b39f8: comparison of 18446744073709547520 and 0
0x10b3a3a: comparison of 0 and 0
0x109f273: comparison of 0 and 0
0x109e4c8: comparison of 18446744073709547520 and 0
0x109e50a: comparison of 0 and 0
0x105bafc: switch on value 0 (16 bits) with 3 cases
0x105a421: comparison of 0 and 0
0x1059c73: comparison of 0 and 14
0x1059ce7: comparison of 0 and 14
0x1059d24: comparison of 14 and 14
0x1059d6a: comparison of 0 and 14
0x109f7c9: comparison of 0 and 14
Hello, World!
0x109e4c8: comparison of 18446744073709547520 and 14
0x109e50a: comparison of 0 and 14
0x109f87b: switch on value 0 (16 bits) with 15 cases
0x105bc4f: comparison of 0 and 0
0x1059dd4: comparison of 0 and 0
0x1059c73: comparison of 14 and 14
0x1059b24: comparison of 0 and 0

I'm putting this up because I think it will already benefit zig-afl-kit mainly because of making std.mem.eql optimize for fuzzing when -ffuzz is selected. Probably also because of the optforfuzzing attribute on all the functions in the LLVM backend.

Closes #5484. See #20702 for follow-up issues.

Next steps:

  • Implement libfuzzer in zig
  • Build system and test runner integration

@andrewrk andrewrk force-pushed the fuzz branch 2 times, most recently from a2fdc16 to 25fc237 Compare July 22, 2024 07:27
andrewrk added 6 commits July 22, 2024 13:07
* Add the `-ffuzz` and `-fno-fuzz` CLI arguments.
* Detect fuzz testing flags from zig cc.
* Set the correct clang flags when fuzz testing is requested. It can be
  combined with TSAN and UBSAN.
* Compilation: build fuzzer library when needed which is currently an
  empty zig file.
* Add optforfuzzing to every function in the llvm backend for modules
  that have requested fuzzing.
* In ZigLLVMTargetMachineEmitToFile, add the optimization passes for
  sanitizer coverage.
* std.mem.eql uses a naive implementation optimized for fuzzing when
  builtin.fuzz is true.

Tracked by #20702
This is needed to ensure that start code does not try to access thread
local storage before it has set up thread local storage.
@ProkopRandacek
Copy link
Contributor

Next steps:

  • Implement libfuzzer in zig
  • Build system and test runner integration

I have committed to implementing this next month this as part of my term project. Is there a space for collaboration? I have to implement this either way and I would love to contribute my implementation upstream.

@andrewrk
Copy link
Member Author

I have committed to implementing this next month this as part of my term project. Is there a space for collaboration? I have to implement this either way and I would love to contribute my implementation upstream.

Sure. I'll have the system wired together shortly, so you can see how it is all supposed to work.

Then it is a matter of improving the genetic algorithm. We can look at afl-fuzz.c for inspiration.

andrewrk added 7 commits July 22, 2024 14:26
This prevents it from trying to access thread local storage before it
has set up thread local storage, particularly when code coverage
instrumentation is enabled.
`-fno-sanitize=function` must come after `-fsanitize=undefined` or it
has no effect.
},
},
.{
"@disableInstrumentation",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add this built-in to the langref?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think it makes sense to leave out of the langref but include in the spec

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add this built-in to the langref?

Yes of course

i think it makes sense to leave out of the langref but include in the spec

This not Meghan's Opinion Tracker. Provide technical arguments, or keep your opinion to yourself.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

-fsanitize=fuzzer support

5 participants