After getting American Fuzzy Lop set up and fuzzing tcpdump at a basic level, we can move onto some more advanced features of afl that allow you to get quite a few speed increases, which is of the utmost importance when fuzzing.
Previously, I introduced afl-gcc as a wrapper for gcc that will instrument the resulting binary as it compiles in order to provide feedback during the fuzzing process. However, this time we will use afl-clang-fast which allows for a nice speedboost.
In order to enable the use of afl-clang-fast, I will need to install clang in the chroot that I created for the first tutorial.
apt-get install clang -y
Inside the afl-1.83b folder is another folder called llvm_mode, which contains the source for the afl-clang-fast clang wrapper. Changing to this directory, we can build these binaries and use them to build tcpdump again in a slightly different way.
cd /root/afl-1.83b/llvm_mode
LLVM_CONFIG=llvm-config-3.4 make
cd ../
make install
which afl-clang-fast
Running the previous commands should build and install afl-clang-fast, then print the location to the new afl-clang-fast binary (in /usr/local/bin). Now I can rebuild tcpdump using the afl-clang-fast binary, which should give me a nice speed boost.
cd /root/tcpdump-4.6.2/
make clean
CC=afl-clang-fast ./configure
make
During compilation, you should see similar printouts to what afl-gcc provided:
afl-clang-fast 1.83b by <lszekeres@google.com>
afl-llvm-pass 1.83b by <lszekeres@google.com>
[+] Instrumented 268 locations (non-hardened mode, ratio 100%).
Once compiled, in order to have clean results, the findings directory used in previous examples needs to be removed and recreated so I can begin fuzzing the new binary with the same afl-fuzz command.
cd /root
rm -rf findings && mkdir findings
afl-fuzz -i testcases/ -o findings/ tcpdump-4.6.2/tcpdump -nr @@
With this, I now get upwards of 1000 executions per second, when using afl-gcc to create the tcpdump binary, I was having trouble maintaining 900 executions per second. That’s about a 10% increase and that adds up over time!
Now, afl-fuzz only takes up a single core, so on most modern systems, a single instance of afl-fuzz leaves a whole lot of system left doing nothing. I can use a master/slave feature of afl that lets one afl-fuzz instance (the master) perform deterministic fuzzing while the slaves perform more traditional random fuzzing.
In order to achieve this, I will remove the findings directory completely and create a new directory called syncdir which all the afl-fuzz instances will use.
cd /root
rm -rf findings && mkdir syncdir
I am also going to use screen this time so that I can start my afl-fuzz processes in the background without worrying about keeping them alive in an open terminal window. The first afl-fuzz instance I will create is the ‘master’ instance whose only job is to perform deterministic fuzzing.
screen
afl-fuzz -i testcases/ -o syncdir/ -M fuzzer1 tcpdump-4.6.2/tcpdump -nr @@
Notice that I am adding a new argument to afl-fuzz. -M means that this process will be a master process, and that the name of the process will be ‘fuzzer1’. My amateurish fuzzing server has 6 cores, so I can easily add 3 slave processes and still have some room left for other processes if need be. If you are still in the screen session for the master fuzzer, hit ctrl-a, then ctrl-d to background the master screen session. I can create the next three afl-fuzz processes in a similar way
screen
afl-fuzz -i testcases/ -o syncdir/ -S fuzzer2 tcpdump-4.6.2/tcpdump -nr @@
Instead of using the -M argument, I am now using -S which means that this process will be a slave process, performing nondeterministic random fuzzing. In order to add 2 more afl-fuzz processes, I run the exact same commands, but naming the new fuzzers fuzzer3 and fuzzer4.
When I list the contents of the syncdir directory, I see the names of my fuzzers that are currently running.
# ls syncdir/
fuzzer1 fuzzer2 fuzzer3 fuzzer4
#
You might be wondering, how can I keep track of where each fuzzer is and how it is doing if each process is running in a screen session in the background? Running a single terminal for each fuzzer takes up a lot of real estate and is cumbersome. Luckily, afl comes with a utility to remedy just this scenario called afl-whatsup. If you point afl-whatsup to the syncdir that all the fuzzer instances are using to store their data, it will give a nice summary of what is currently happening.
# afl-whatsup syncdir/
status check tool for afl-fuzz by <lcamtuf@google.com>
Individual fuzzers
==================
>>> fuzzer4 (0 days, 0 hrs) <<<
cycle 1, lifetime speed 1058 execs/sec, path 73/855 (8%)
pending 392/831, coverage 4.65%, crash count 3 (!)
>>> fuzzer1 (0 days, 0 hrs) <<<
cycle 1, lifetime speed 943 execs/sec, path 83/821 (10%)
pending 381/797, coverage 4.65%, crash count 1 (!)
>>> fuzzer3 (0 days, 0 hrs) <<<
cycle 1, lifetime speed 1040 execs/sec, path 76/859 (8%)
pending 396/833, coverage 4.77%, crash count 1 (!)
>>> fuzzer2 (0 days, 0 hrs) <<<
cycle 1, lifetime speed 1012 execs/sec, path 61/860 (7%)
pending 392/841, coverage 4.61%, crash count 1 (!)
Summary stats
=============
Fuzzers alive : 4
Total run time : 0 days, 0 hours
Total execs : 3 million
Cumulative speed : 4030 execs/sec
Pending paths : 1561 faves, 3302 total
Pending per fuzzer : 390 faves, 825 total (on average)
Crashes found : 6 locally unique
On my fuzzing machine, I actually use the watch command to refresh the output of afl-whatsup every couple of seconds on a passive monitor that I can just glance at. If you notice, cumulatively we are achieving 4000 executions a second, as opposed to the paultry 900-1000 a single fuzzer was giving. Letting this sit for a week will yield some excellent test cases that causes crashes and hangs.