* [PATCH 0/8] Fuzzing misc bugfix and performance improvements
@ 2017-03-20 11:19 Andrew Cooper
2017-03-20 11:19 ` [PATCH 1/8] tools/fuzz: Remove .d files in clean Andrew Cooper
` (8 more replies)
0 siblings, 9 replies; 10+ messages in thread
From: Andrew Cooper @ 2017-03-20 11:19 UTC (permalink / raw
To: Xen-devel; +Cc: Andrew Cooper, Ian Jackson, Wei Liu, Jan Beulich
This series is a couple of bugfixes, but is mainly a set of performance
improvements to using the AFL test harness. This series increases the
performance as reported by afl-fuzz from 6k execs/s to 38k execs/s, or more
than 6 times faster.
Andrew Cooper (8):
tools/fuzz: Remove .d files in clean
tools/fuzz: Use $(CC) for linking the harnesses
tools/fuzz: Include LLVMFuzzerTestOneInput() in the generated .a
tools/insn-fuzz: Use shorter filenames
tools/insn-fuzz: Use getopt() for parsing the command line
tools/insn-fuzz: Accept fuzzing input on stdin
tools/insn-fuzz: Make use of LLVMFuzzerInitialize()
tools/insn-fuzz: Support AFL's afl-clang-fast mode
.gitignore | 2 +-
tools/fuzz/README.afl | 19 +++-
tools/fuzz/libelf/Makefile | 5 +-
tools/fuzz/x86_instruction_emulator/Makefile | 19 ++--
tools/fuzz/x86_instruction_emulator/afl-harness.c | 114 +++++++++++++++++++++
.../afl-x86-insn-emulator-fuzzer.c | 67 ------------
.../{x86-insn-emulator-fuzzer.c => fuzz-emul.c} | 19 ++--
7 files changed, 154 insertions(+), 91 deletions(-)
create mode 100644 tools/fuzz/x86_instruction_emulator/afl-harness.c
delete mode 100644 tools/fuzz/x86_instruction_emulator/afl-x86-insn-emulator-fuzzer.c
rename tools/fuzz/x86_instruction_emulator/{x86-insn-emulator-fuzzer.c => fuzz-emul.c} (99%)
--
2.1.4
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH 1/8] tools/fuzz: Remove .d files in clean
2017-03-20 11:19 [PATCH 0/8] Fuzzing misc bugfix and performance improvements Andrew Cooper
@ 2017-03-20 11:19 ` Andrew Cooper
2017-03-20 11:19 ` [PATCH 2/8] tools/fuzz: Use $(CC) for linking the harnesses Andrew Cooper
` (7 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Andrew Cooper @ 2017-03-20 11:19 UTC (permalink / raw
To: Xen-devel; +Cc: Andrew Cooper, Ian Jackson, Wei Liu, Jan Beulich
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Ian Jackson <Ian.Jackson@eu.citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
---
tools/fuzz/libelf/Makefile | 2 +-
tools/fuzz/x86_instruction_emulator/Makefile | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/tools/fuzz/libelf/Makefile b/tools/fuzz/libelf/Makefile
index 3313601..af44c40 100644
--- a/tools/fuzz/libelf/Makefile
+++ b/tools/fuzz/libelf/Makefile
@@ -30,7 +30,7 @@ distclean: clean
.PHONY: clean
clean:
- rm -f *.o *.a *-libelf-fuzzer
+ rm -f *.o .*.d *.a *-libelf-fuzzer
.PHONY: install
install: all
diff --git a/tools/fuzz/x86_instruction_emulator/Makefile b/tools/fuzz/x86_instruction_emulator/Makefile
index 3b75beb..98fd398 100644
--- a/tools/fuzz/x86_instruction_emulator/Makefile
+++ b/tools/fuzz/x86_instruction_emulator/Makefile
@@ -45,7 +45,7 @@ distclean: clean
.PHONY: clean
clean:
- rm -f *.a *.o *-x86-insn-emulator-fuzzer
+ rm -f *.a *.o .*.d *-x86-insn-emulator-fuzzer
.PHONY: install
install: all
--
2.1.4
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH 2/8] tools/fuzz: Use $(CC) for linking the harnesses
2017-03-20 11:19 [PATCH 0/8] Fuzzing misc bugfix and performance improvements Andrew Cooper
2017-03-20 11:19 ` [PATCH 1/8] tools/fuzz: Remove .d files in clean Andrew Cooper
@ 2017-03-20 11:19 ` Andrew Cooper
2017-03-20 11:19 ` [PATCH 3/8] tools/fuzz: Include LLVMFuzzerTestOneInput() in the generated .a Andrew Cooper
` (6 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Andrew Cooper @ 2017-03-20 11:19 UTC (permalink / raw
To: Xen-devel; +Cc: Andrew Cooper, Ian Jackson, Wei Liu, Jan Beulich
This is necessary to make use of compiler features such as UBSAN.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Ian Jackson <Ian.Jackson@eu.citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
---
tools/fuzz/libelf/Makefile | 1 +
tools/fuzz/x86_instruction_emulator/Makefile | 1 +
2 files changed, 2 insertions(+)
diff --git a/tools/fuzz/libelf/Makefile b/tools/fuzz/libelf/Makefile
index af44c40..c884703 100644
--- a/tools/fuzz/libelf/Makefile
+++ b/tools/fuzz/libelf/Makefile
@@ -20,6 +20,7 @@ libelf.a: $(ELF_LIB_OBJS)
libelf-fuzzer-all: libelf.a libelf-fuzzer.o
afl-libelf-fuzzer: afl-libelf-fuzzer.o libelf-fuzzer.o $(ELF_LIB_OBJS)
+ $(CC) $(CFLAGS) $^ -o $@
# Common targets
.PHONY: all
diff --git a/tools/fuzz/x86_instruction_emulator/Makefile b/tools/fuzz/x86_instruction_emulator/Makefile
index 98fd398..673b5f0 100644
--- a/tools/fuzz/x86_instruction_emulator/Makefile
+++ b/tools/fuzz/x86_instruction_emulator/Makefile
@@ -34,6 +34,7 @@ x86-insn-emulator.a: x86_emulate.o
$(AR) rc $@ $^
afl-x86-insn-emulator-fuzzer: afl-x86-insn-emulator-fuzzer.o x86-insn-emulator-fuzzer.o x86_emulate.o
+ $(CC) $(CFLAGS) $^ -o $@
# Common targets
.PHONY: all
--
2.1.4
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH 3/8] tools/fuzz: Include LLVMFuzzerTestOneInput() in the generated .a
2017-03-20 11:19 [PATCH 0/8] Fuzzing misc bugfix and performance improvements Andrew Cooper
2017-03-20 11:19 ` [PATCH 1/8] tools/fuzz: Remove .d files in clean Andrew Cooper
2017-03-20 11:19 ` [PATCH 2/8] tools/fuzz: Use $(CC) for linking the harnesses Andrew Cooper
@ 2017-03-20 11:19 ` Andrew Cooper
2017-03-20 11:19 ` [PATCH 4/8] tools/insn-fuzz: Use shorter filenames Andrew Cooper
` (5 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Andrew Cooper @ 2017-03-20 11:19 UTC (permalink / raw
To: Xen-devel; +Cc: Andrew Cooper, Ian Jackson, Wei Liu, Jan Beulich
Otherwise they are not suitable for use with libfuzz.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Ian Jackson <Ian.Jackson@eu.citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
---
tools/fuzz/libelf/Makefile | 2 +-
tools/fuzz/x86_instruction_emulator/Makefile | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/tools/fuzz/libelf/Makefile b/tools/fuzz/libelf/Makefile
index c884703..6cdc31e 100644
--- a/tools/fuzz/libelf/Makefile
+++ b/tools/fuzz/libelf/Makefile
@@ -13,7 +13,7 @@ $(ELF_LIB_OBJS): CFLAGS += -DFUZZ_NO_LIBXC $(CFLAGS_xeninclude)
libelf-fuzzer.o: CFLAGS += $(CFLAGS_xeninclude)
-libelf.a: $(ELF_LIB_OBJS)
+libelf.a: libelf-fuzzer.o $(ELF_LIB_OBJS)
$(AR) rc $@ $^
.PHONY: libelf-fuzzer-all
diff --git a/tools/fuzz/x86_instruction_emulator/Makefile b/tools/fuzz/x86_instruction_emulator/Makefile
index 673b5f0..abba186 100644
--- a/tools/fuzz/x86_instruction_emulator/Makefile
+++ b/tools/fuzz/x86_instruction_emulator/Makefile
@@ -30,7 +30,7 @@ x86_emulate.o: x86_emulate.c x86_emulate/x86_emulate.c $(x86_emulate.h)
x86-insn-emulator-fuzzer.o: $(x86_emulate.h)
-x86-insn-emulator.a: x86_emulate.o
+x86-insn-emulator.a: x86-insn-emulator-fuzzer.o x86_emulate.o
$(AR) rc $@ $^
afl-x86-insn-emulator-fuzzer: afl-x86-insn-emulator-fuzzer.o x86-insn-emulator-fuzzer.o x86_emulate.o
--
2.1.4
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH 4/8] tools/insn-fuzz: Use shorter filenames
2017-03-20 11:19 [PATCH 0/8] Fuzzing misc bugfix and performance improvements Andrew Cooper
` (2 preceding siblings ...)
2017-03-20 11:19 ` [PATCH 3/8] tools/fuzz: Include LLVMFuzzerTestOneInput() in the generated .a Andrew Cooper
@ 2017-03-20 11:19 ` Andrew Cooper
2017-03-20 11:19 ` [PATCH 5/8] tools/insn-fuzz: Use getopt() for parsing the command line Andrew Cooper
` (4 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Andrew Cooper @ 2017-03-20 11:19 UTC (permalink / raw
To: Xen-devel; +Cc: Andrew Cooper, Ian Jackson, Wei Liu, Jan Beulich
Amongst other things, these tab complete more easily.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Ian Jackson <Ian.Jackson@eu.citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
---
.gitignore | 2 +-
tools/fuzz/README.afl | 6 +++---
tools/fuzz/x86_instruction_emulator/Makefile | 18 +++++++++---------
.../{afl-x86-insn-emulator-fuzzer.c => afl-harness.c} | 0
.../{x86-insn-emulator-fuzzer.c => fuzz-emul.c} | 0
5 files changed, 13 insertions(+), 13 deletions(-)
rename tools/fuzz/x86_instruction_emulator/{afl-x86-insn-emulator-fuzzer.c => afl-harness.c} (100%)
rename tools/fuzz/x86_instruction_emulator/{x86-insn-emulator-fuzzer.c => fuzz-emul.c} (100%)
diff --git a/.gitignore b/.gitignore
index 2bf815b..c056247 100644
--- a/.gitignore
+++ b/.gitignore
@@ -158,7 +158,7 @@ tools/flask/utils/flask-label-pci
tools/fuzz/libelf/afl-libelf-fuzzer
tools/fuzz/x86_instruction_emulator/asm
tools/fuzz/x86_instruction_emulator/x86_emulate*
-tools/fuzz/x86_instruction_emulator/afl-x86-insn-emulator-fuzzer
+tools/fuzz/x86_instruction_emulator/afl-harness
tools/helpers/_paths.h
tools/helpers/init-xenstore-domain
tools/helpers/xen-init-dom0
diff --git a/tools/fuzz/README.afl b/tools/fuzz/README.afl
index 68e0fa3..357c3c8 100644
--- a/tools/fuzz/README.afl
+++ b/tools/fuzz/README.afl
@@ -18,15 +18,15 @@ Use the x86 instruction emulator fuzzer as an example.
2. run the following commands to build:
$ cd tools/fuzz/x86_instruction_emulator
$ make distclean
- $ make CC=$AFLPATH/afl-gcc afl # produces afl-x86-insn-emulator-fuzzer
+ $ make CC=$AFLPATH/afl-gcc afl # produces afl-harness
3. provide initial test case (fuzzer dependent, see afl-*.c):
$ mkdir testcase_dir
$ dd if=/dev/urandom of=testcase_dir/rand.bin \
- bs=`./afl-x86-insn-emulator-fuzzer --min-input-size` count=1
+ bs=`./afl-harness --min-input-size` count=1
4. run the fuzzer with AFL:
$ $AFLPATH/afl-fuzz -m none -t 1000 -i testcase_dir -o findings_dir -- \
- ./afl-x86-insn-emulator-fuzzer @@
+ ./afl-harness @@
Please see AFL documentation for more information.
diff --git a/tools/fuzz/x86_instruction_emulator/Makefile b/tools/fuzz/x86_instruction_emulator/Makefile
index abba186..6e3fa1e 100644
--- a/tools/fuzz/x86_instruction_emulator/Makefile
+++ b/tools/fuzz/x86_instruction_emulator/Makefile
@@ -1,11 +1,11 @@
XEN_ROOT=$(CURDIR)/../../..
include $(XEN_ROOT)/tools/Rules.mk
-.PHONY: x86-instruction-emulator-fuzzer-all
+.PHONY: x86-insn-fuzz-all
ifeq ($(CONFIG_X86_64),y)
-x86-instruction-emulator-fuzzer-all: x86-insn-emulator.a x86-insn-emulator-fuzzer.o afl
+x86-insn-fuzz-all: x86-insn-fuzzer.a fuzz-emul.o afl
else
-x86-instruction-emulator-fuzzer-all:
+x86-insn-fuzz-all:
endif
x86_emulate:
@@ -28,17 +28,17 @@ x86_emulate.h := x86_emulate.h x86_emulate/x86_emulate.h $(x86.h)
x86_emulate.o: x86_emulate.c x86_emulate/x86_emulate.c $(x86_emulate.h)
-x86-insn-emulator-fuzzer.o: $(x86_emulate.h)
+fuzz-emul.o: $(x86_emulate.h)
-x86-insn-emulator.a: x86-insn-emulator-fuzzer.o x86_emulate.o
+x86-insn-fuzzer.a: fuzz-emul.o x86_emulate.o
$(AR) rc $@ $^
-afl-x86-insn-emulator-fuzzer: afl-x86-insn-emulator-fuzzer.o x86-insn-emulator-fuzzer.o x86_emulate.o
+afl-harness: afl-harness.o fuzz-emul.o x86_emulate.o
$(CC) $(CFLAGS) $^ -o $@
# Common targets
.PHONY: all
-all: x86-instruction-emulator-fuzzer-all
+all: x86-insn-fuzz-all
.PHONY: distclean
distclean: clean
@@ -46,10 +46,10 @@ distclean: clean
.PHONY: clean
clean:
- rm -f *.a *.o .*.d *-x86-insn-emulator-fuzzer
+ rm -f *.a *.o .*.d afl-harness
.PHONY: install
install: all
.PHONY: afl
-afl: afl-x86-insn-emulator-fuzzer
+afl: afl-harness
diff --git a/tools/fuzz/x86_instruction_emulator/afl-x86-insn-emulator-fuzzer.c b/tools/fuzz/x86_instruction_emulator/afl-harness.c
similarity index 100%
rename from tools/fuzz/x86_instruction_emulator/afl-x86-insn-emulator-fuzzer.c
rename to tools/fuzz/x86_instruction_emulator/afl-harness.c
diff --git a/tools/fuzz/x86_instruction_emulator/x86-insn-emulator-fuzzer.c b/tools/fuzz/x86_instruction_emulator/fuzz-emul.c
similarity index 100%
rename from tools/fuzz/x86_instruction_emulator/x86-insn-emulator-fuzzer.c
rename to tools/fuzz/x86_instruction_emulator/fuzz-emul.c
--
2.1.4
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH 5/8] tools/insn-fuzz: Use getopt() for parsing the command line
2017-03-20 11:19 [PATCH 0/8] Fuzzing misc bugfix and performance improvements Andrew Cooper
` (3 preceding siblings ...)
2017-03-20 11:19 ` [PATCH 4/8] tools/insn-fuzz: Use shorter filenames Andrew Cooper
@ 2017-03-20 11:19 ` Andrew Cooper
2017-03-20 11:19 ` [PATCH 6/8] tools/insn-fuzz: Accept fuzzing input on stdin Andrew Cooper
` (3 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Andrew Cooper @ 2017-03-20 11:19 UTC (permalink / raw
To: Xen-devel; +Cc: Andrew Cooper, Ian Jackson, Wei Liu, Jan Beulich
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Ian Jackson <Ian.Jackson@eu.citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
---
tools/fuzz/x86_instruction_emulator/afl-harness.c | 40 +++++++++++++++++++----
1 file changed, 34 insertions(+), 6 deletions(-)
diff --git a/tools/fuzz/x86_instruction_emulator/afl-harness.c b/tools/fuzz/x86_instruction_emulator/afl-harness.c
index 16edbd6..102566c 100644
--- a/tools/fuzz/x86_instruction_emulator/afl-harness.c
+++ b/tools/fuzz/x86_instruction_emulator/afl-harness.c
@@ -3,6 +3,7 @@
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
+#include <getopt.h>
extern int LLVMFuzzerTestOneInput(const uint8_t *data_p, size_t size);
extern unsigned int fuzz_minimal_input_size(void);
@@ -17,16 +18,43 @@ int main(int argc, char **argv)
setbuf(stdout, NULL);
- if ( argc != 2 )
+ while ( 1 )
{
- printf("Expecting only one argument\n");
- exit(-1);
+ enum {
+ OPT_MIN_SIZE,
+ };
+ static const struct option lopts[] = {
+ { "min-input-size", no_argument, NULL, OPT_MIN_SIZE },
+ { 0, 0, 0, 0 }
+ };
+ int c = getopt_long_only(argc, argv, "", lopts, NULL);
+
+ if ( c == -1 )
+ break;
+
+ switch ( c )
+ {
+ case OPT_MIN_SIZE:
+ printf("%u\n", fuzz_minimal_input_size());
+ exit(0);
+ break;
+
+ case '?':
+ printf("Usage: %s $FILE | [--min-input-size]\n", argv[0]);
+ exit(-1);
+ break;
+
+ default:
+ printf("Bad getopt return %d (%c)\n", c, c);
+ exit(-1);
+ break;
+ }
}
- if ( !strcmp(argv[1], "--min-input-size") )
+ if ( optind != (argc - 1) )
{
- printf("%u\n", fuzz_minimal_input_size());
- exit(0);
+ printf("Expecting only one argument\n");
+ exit(-1);
}
fp = fopen(argv[1], "rb");
--
2.1.4
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH 6/8] tools/insn-fuzz: Accept fuzzing input on stdin
2017-03-20 11:19 [PATCH 0/8] Fuzzing misc bugfix and performance improvements Andrew Cooper
` (4 preceding siblings ...)
2017-03-20 11:19 ` [PATCH 5/8] tools/insn-fuzz: Use getopt() for parsing the command line Andrew Cooper
@ 2017-03-20 11:19 ` Andrew Cooper
2017-03-20 11:19 ` [PATCH 7/8] tools/insn-fuzz: Make use of LLVMFuzzerInitialize() Andrew Cooper
` (2 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Andrew Cooper @ 2017-03-20 11:19 UTC (permalink / raw
To: Xen-devel; +Cc: Andrew Cooper, Ian Jackson, Wei Liu, Jan Beulich
This is rather faster for afl-fuzz to arrange than using an explicit file
parameter. Also update the README to recommend using a tmpfs for findings_dir
which reduces disk load and is more performant.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Ian Jackson <Ian.Jackson@eu.citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
---
tools/fuzz/README.afl | 7 ++++--
tools/fuzz/x86_instruction_emulator/afl-harness.c | 29 ++++++++++++++---------
2 files changed, 23 insertions(+), 13 deletions(-)
diff --git a/tools/fuzz/README.afl b/tools/fuzz/README.afl
index 357c3c8..c5f749a 100644
--- a/tools/fuzz/README.afl
+++ b/tools/fuzz/README.afl
@@ -25,8 +25,11 @@ Use the x86 instruction emulator fuzzer as an example.
$ dd if=/dev/urandom of=testcase_dir/rand.bin \
bs=`./afl-harness --min-input-size` count=1
+3a. use a tmpfs for findings_dir (Perf improvement and reduced disk load)
+ $ mkdir findings_dir
+ $ mount -t tmpfs -o size=512M tmpfs findings_dir
+
4. run the fuzzer with AFL:
- $ $AFLPATH/afl-fuzz -m none -t 1000 -i testcase_dir -o findings_dir -- \
- ./afl-harness @@
+ $ $AFLPATH/afl-fuzz -t 1000 -i testcase_dir -o findings_dir -- ./afl-harness
Please see AFL documentation for more information.
diff --git a/tools/fuzz/x86_instruction_emulator/afl-harness.c b/tools/fuzz/x86_instruction_emulator/afl-harness.c
index 102566c..89d8605 100644
--- a/tools/fuzz/x86_instruction_emulator/afl-harness.c
+++ b/tools/fuzz/x86_instruction_emulator/afl-harness.c
@@ -14,7 +14,7 @@ static uint8_t input[INPUT_SIZE];
int main(int argc, char **argv)
{
size_t size;
- FILE *fp;
+ FILE *fp = NULL;
setbuf(stdout, NULL);
@@ -40,6 +40,7 @@ int main(int argc, char **argv)
break;
case '?':
+ usage:
printf("Usage: %s $FILE | [--min-input-size]\n", argv[0]);
exit(-1);
break;
@@ -51,17 +52,19 @@ int main(int argc, char **argv)
}
}
- if ( optind != (argc - 1) )
- {
- printf("Expecting only one argument\n");
- exit(-1);
- }
+ if ( optind == argc ) /* No positional parameters. Use stdin. */
+ fp = stdin;
+ else if ( optind != (argc - 1) )
+ goto usage;
- fp = fopen(argv[1], "rb");
- if ( fp == NULL )
+ if ( fp != stdin ) /* If not using stdin, open the provided file. */
{
- perror("fopen");
- exit(-1);
+ fp = fopen(argv[optind], "rb");
+ if ( fp == NULL )
+ {
+ perror("fopen");
+ exit(-1);
+ }
}
size = fread(input, 1, INPUT_SIZE, fp);
@@ -78,7 +81,11 @@ int main(int argc, char **argv)
exit(-1);
}
- fclose(fp);
+ if ( fp != stdin )
+ {
+ fclose(fp);
+ fp = NULL;
+ }
LLVMFuzzerTestOneInput(input, size);
--
2.1.4
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH 7/8] tools/insn-fuzz: Make use of LLVMFuzzerInitialize()
2017-03-20 11:19 [PATCH 0/8] Fuzzing misc bugfix and performance improvements Andrew Cooper
` (5 preceding siblings ...)
2017-03-20 11:19 ` [PATCH 6/8] tools/insn-fuzz: Accept fuzzing input on stdin Andrew Cooper
@ 2017-03-20 11:19 ` Andrew Cooper
2017-03-20 11:19 ` [PATCH 8/8] tools/insn-fuzz: Support AFL's afl-clang-fast mode Andrew Cooper
2017-03-20 15:55 ` [PATCH 0/8] Fuzzing misc bugfix and performance improvements Wei Liu
8 siblings, 0 replies; 10+ messages in thread
From: Andrew Cooper @ 2017-03-20 11:19 UTC (permalink / raw
To: Xen-devel; +Cc: Andrew Cooper, Ian Jackson, Wei Liu, Jan Beulich
libfuzz can perform one-time initialisation by calling LLVMFuzzerInitialize().
Move emul_test_init() into this, to avoid repeating it on every
LLVMFuzzerTestOneInput() call.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Ian Jackson <Ian.Jackson@eu.citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
---
tools/fuzz/x86_instruction_emulator/afl-harness.c | 4 ++++
tools/fuzz/x86_instruction_emulator/fuzz-emul.c | 19 +++++++++++--------
2 files changed, 15 insertions(+), 8 deletions(-)
diff --git a/tools/fuzz/x86_instruction_emulator/afl-harness.c b/tools/fuzz/x86_instruction_emulator/afl-harness.c
index 89d8605..63aff59 100644
--- a/tools/fuzz/x86_instruction_emulator/afl-harness.c
+++ b/tools/fuzz/x86_instruction_emulator/afl-harness.c
@@ -5,6 +5,7 @@
#include <string.h>
#include <getopt.h>
+extern int LLVMFuzzerInitialize(int *argc, char ***argv);
extern int LLVMFuzzerTestOneInput(const uint8_t *data_p, size_t size);
extern unsigned int fuzz_minimal_input_size(void);
@@ -57,6 +58,9 @@ int main(int argc, char **argv)
else if ( optind != (argc - 1) )
goto usage;
+ if ( LLVMFuzzerInitialize(&argc, &argv) )
+ exit(-1);
+
if ( fp != stdin ) /* If not using stdin, open the provided file. */
{
fp = fopen(argv[optind], "rb");
diff --git a/tools/fuzz/x86_instruction_emulator/fuzz-emul.c b/tools/fuzz/x86_instruction_emulator/fuzz-emul.c
index 4b1e324..3b3041d 100644
--- a/tools/fuzz/x86_instruction_emulator/fuzz-emul.c
+++ b/tools/fuzz/x86_instruction_emulator/fuzz-emul.c
@@ -646,9 +646,19 @@ static void sanitize_input(struct x86_emulate_ctxt *ctxt)
}
}
+int LLVMFuzzerInitialize(int *argc, char ***argv)
+{
+ if ( !emul_test_init() )
+ {
+ printf("Warning: Stack could not be made executable (%d).\n", errno);
+ return 1;
+ }
+
+ return 0;
+}
+
int LLVMFuzzerTestOneInput(const uint8_t *data_p, size_t size)
{
- bool stack_exec;
struct cpu_user_regs regs = {};
struct x86_emulate_ctxt ctxt = {
.regs = ®s,
@@ -657,13 +667,6 @@ int LLVMFuzzerTestOneInput(const uint8_t *data_p, size_t size)
};
int rc;
- stack_exec = emul_test_init();
- if ( !stack_exec )
- {
- printf("Warning: Stack could not be made executable (%d).\n", errno);
- return 1;
- }
-
/* Reset all global state variables */
memset(&input, 0, sizeof(input));
data_index = 0;
--
2.1.4
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH 8/8] tools/insn-fuzz: Support AFL's afl-clang-fast mode
2017-03-20 11:19 [PATCH 0/8] Fuzzing misc bugfix and performance improvements Andrew Cooper
` (6 preceding siblings ...)
2017-03-20 11:19 ` [PATCH 7/8] tools/insn-fuzz: Make use of LLVMFuzzerInitialize() Andrew Cooper
@ 2017-03-20 11:19 ` Andrew Cooper
2017-03-20 15:55 ` [PATCH 0/8] Fuzzing misc bugfix and performance improvements Wei Liu
8 siblings, 0 replies; 10+ messages in thread
From: Andrew Cooper @ 2017-03-20 11:19 UTC (permalink / raw
To: Xen-devel; +Cc: Andrew Cooper, Ian Jackson, Wei Liu, Jan Beulich
AFL has an alternative llvm-base instrumentation mode, which has much lower
overhead than the traditional afl-gcc.
One extra ability is to chose exactly where the master process gets
initialised to, before being forked for testing. This point is chosen after
the call to LLVMFuzzerInitialize(), so the stack isn't being remapped
executable for every test.
Another extra ability is to feed multiple inputs into a single test process,
to reduce the number of fork() calls required overall. Two caveats are that if
stdin is used for data, it must be unbuffered, and if input is passed via a
command line parameter, the underlying file must be opened and closed on each
iteration.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Ian Jackson <Ian.Jackson@eu.citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
This patch is mostly re-indentation, and far easier reviewed using `git diff
--ignore-all-space`
---
tools/fuzz/README.afl | 10 ++++-
tools/fuzz/x86_instruction_emulator/afl-harness.c | 54 +++++++++++++----------
2 files changed, 40 insertions(+), 24 deletions(-)
diff --git a/tools/fuzz/README.afl b/tools/fuzz/README.afl
index c5f749a..4758de2 100644
--- a/tools/fuzz/README.afl
+++ b/tools/fuzz/README.afl
@@ -18,7 +18,15 @@ Use the x86 instruction emulator fuzzer as an example.
2. run the following commands to build:
$ cd tools/fuzz/x86_instruction_emulator
$ make distclean
- $ make CC=$AFLPATH/afl-gcc afl # produces afl-harness
+
+ If you have a new enough version of Clang/LLVM and have configured AFL's
+ llvm_mode, make use of afl-clang-fast:
+
+ $ make CC=$AFLPATH/afl-clang-fast afl # produces afl-harness
+
+ If not, use the default afl-gcc:
+
+ $ make CC=$AFLPATH/afl-gcc afl # produces afl-harness
3. provide initial test case (fuzzer dependent, see afl-*.c):
$ mkdir testcase_dir
diff --git a/tools/fuzz/x86_instruction_emulator/afl-harness.c b/tools/fuzz/x86_instruction_emulator/afl-harness.c
index 63aff59..1548693 100644
--- a/tools/fuzz/x86_instruction_emulator/afl-harness.c
+++ b/tools/fuzz/x86_instruction_emulator/afl-harness.c
@@ -17,6 +17,7 @@ int main(int argc, char **argv)
size_t size;
FILE *fp = NULL;
+ setbuf(stdin, NULL);
setbuf(stdout, NULL);
while ( 1 )
@@ -61,37 +62,44 @@ int main(int argc, char **argv)
if ( LLVMFuzzerInitialize(&argc, &argv) )
exit(-1);
- if ( fp != stdin ) /* If not using stdin, open the provided file. */
+#ifdef __AFL_HAVE_MANUAL_CONTROL
+ __AFL_INIT();
+
+ while ( __AFL_LOOP(1000) )
+#endif
{
- fp = fopen(argv[optind], "rb");
- if ( fp == NULL )
+ if ( fp != stdin ) /* If not using stdin, open the provided file. */
{
- perror("fopen");
- exit(-1);
+ fp = fopen(argv[optind], "rb");
+ if ( fp == NULL )
+ {
+ perror("fopen");
+ exit(-1);
+ }
}
- }
- size = fread(input, 1, INPUT_SIZE, fp);
+ size = fread(input, 1, INPUT_SIZE, fp);
- if ( ferror(fp) )
- {
- perror("fread");
- exit(-1);
- }
+ if ( ferror(fp) )
+ {
+ perror("fread");
+ exit(-1);
+ }
- if ( !feof(fp) )
- {
- printf("Input too large\n");
- exit(-1);
- }
+ if ( !feof(fp) )
+ {
+ printf("Input too large\n");
+ exit(-1);
+ }
- if ( fp != stdin )
- {
- fclose(fp);
- fp = NULL;
- }
+ if ( fp != stdin )
+ {
+ fclose(fp);
+ fp = NULL;
+ }
- LLVMFuzzerTestOneInput(input, size);
+ LLVMFuzzerTestOneInput(input, size);
+ }
return 0;
}
--
2.1.4
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH 0/8] Fuzzing misc bugfix and performance improvements
2017-03-20 11:19 [PATCH 0/8] Fuzzing misc bugfix and performance improvements Andrew Cooper
` (7 preceding siblings ...)
2017-03-20 11:19 ` [PATCH 8/8] tools/insn-fuzz: Support AFL's afl-clang-fast mode Andrew Cooper
@ 2017-03-20 15:55 ` Wei Liu
8 siblings, 0 replies; 10+ messages in thread
From: Wei Liu @ 2017-03-20 15:55 UTC (permalink / raw
To: Andrew Cooper; +Cc: Wei Liu, Ian Jackson, Jan Beulich, Xen-devel
On Mon, Mar 20, 2017 at 11:19:37AM +0000, Andrew Cooper wrote:
> This series is a couple of bugfixes, but is mainly a set of performance
> improvements to using the AFL test harness. This series increases the
> performance as reported by afl-fuzz from 6k execs/s to 38k execs/s, or more
> than 6 times faster.
>
> Andrew Cooper (8):
> tools/fuzz: Remove .d files in clean
> tools/fuzz: Use $(CC) for linking the harnesses
> tools/fuzz: Include LLVMFuzzerTestOneInput() in the generated .a
> tools/insn-fuzz: Use shorter filenames
> tools/insn-fuzz: Use getopt() for parsing the command line
> tools/insn-fuzz: Accept fuzzing input on stdin
> tools/insn-fuzz: Make use of LLVMFuzzerInitialize()
> tools/insn-fuzz: Support AFL's afl-clang-fast mode
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2017-03-20 15:55 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2017-03-20 11:19 [PATCH 0/8] Fuzzing misc bugfix and performance improvements Andrew Cooper
2017-03-20 11:19 ` [PATCH 1/8] tools/fuzz: Remove .d files in clean Andrew Cooper
2017-03-20 11:19 ` [PATCH 2/8] tools/fuzz: Use $(CC) for linking the harnesses Andrew Cooper
2017-03-20 11:19 ` [PATCH 3/8] tools/fuzz: Include LLVMFuzzerTestOneInput() in the generated .a Andrew Cooper
2017-03-20 11:19 ` [PATCH 4/8] tools/insn-fuzz: Use shorter filenames Andrew Cooper
2017-03-20 11:19 ` [PATCH 5/8] tools/insn-fuzz: Use getopt() for parsing the command line Andrew Cooper
2017-03-20 11:19 ` [PATCH 6/8] tools/insn-fuzz: Accept fuzzing input on stdin Andrew Cooper
2017-03-20 11:19 ` [PATCH 7/8] tools/insn-fuzz: Make use of LLVMFuzzerInitialize() Andrew Cooper
2017-03-20 11:19 ` [PATCH 8/8] tools/insn-fuzz: Support AFL's afl-clang-fast mode Andrew Cooper
2017-03-20 15:55 ` [PATCH 0/8] Fuzzing misc bugfix and performance improvements Wei Liu
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.