JavaScript Fuzzing in Mozilla, 2017

• What is fuzzing?
• Why is fuzzing successful?
• Tips for effective fuzzing

the people writing and running fuzzers are by far the most effective QA people at Mozilla.

Nicholas Nethercote, MemShrink & Low Level Tools Team Manager at Mozilla

• Give random input to software
• May find easy-to-detect problems
  • Crashes
  • Assertion failures
    • 1194022
• These can manifest in the real-world

1. Address Sanitizer (ASan)
2. Thread Sanitizer (TSan)
3. Leak Sanitizer (LSan)
4. Valgrind
5. and Undefined Behaviour Sanitizer (UBSan), etc.

• mangleme - Michał Zalewski
• jsfunfuzz - Jesse Ruderman & Gary Kwong
• DOMFuzz - Jesse Ruderman
  • Likely to be replaced

• LangFuzz - Christian Holler
• Peach
• American Fuzzy Lop (AFL) - Michał Zalewski
• Dharma - Christoph Diehl

• Bug bounty programs
  • Mozilla
    • A use-after-free can possibly be worth US$3,000 - US$5,000
  • Google
  • Microsoft
  • Apple (via The Verge)
• Bug counts
  • ~7,000 - jsfunfuzz & DOMFuzz
  • >10% sec-critical

• Black Hat USA 2005 - The Art of File Format Fuzzing
• Black Hat USA 2015
  • list of talks, incl. on fuzzing
• DEF CON 14: The Evolving Art of Fuzzing
• Pwn2Own winners
  • Charlie Miller, 2010
  • Pwn2Own Ars Technica article, 2014

• In an unsafe language?
• Well-sandboxed or not?
• Parsing controllable input?
• Security boundaries?
• Areas other security researchers are working on?

• Isolated modules
  • SpiderMonkey JS shell
  • media shell
• Can be tested on Firefox Desktop

• Bugs
• Tests
• Manpower
• Resistance
• Hard to find serious bugs

• Too many fuzzbugs → resistance
• Value
• Fuzzing team → QA
• Took time to realize

• Fuzzbug reports
  • Concise
  • Popular
  • May be duplicates of real-world bugs
  • Devs sometimes wait for fuzzbugs

Fuzzing is an effective way to find corner cases overlooked when creating a patch and a test for it. A comprehensive fuzz bug report can also provide more information on how to fix it.

[:arai] Mozilla community contributor

• Fix fuzzbugs
• Tests land on trunk (master branch)
• More starting points for fuzzing

• Add assertions & testing functions
• Test components in isolation
• Dynamic analysis tool support
  • AddressSanitizer (ASan)
  • ThreadSanitizer (TSan)
  • Valgrind

• Security bugs → higher-priority
• Fuzzblockers → higher-priority
• Fuzzbugs → generally lower-priority
• The more we discover, the less others find
• More bugs discovered, as we scale

• Automation frameworks
• Devices not powerful enough
  • Mitigation: ARM simulator binary

• Patches first land on trunk (master branch)
• Backout when regressions occur
• Fuzzblockers are fixed
• Tests are rarely disabled

• Input creation (the fuzzer)
• Instrumentation
• Monitoring
• Log bucketing
• Testcase reduction
• Bisection

• funfuzz (Gary Kwong)
  • jsfunfuzz harness
• FuzzManager (Christian Holler) & EC2SpotManager
  • Fuzzing management tools collection
• Laniakea (Christoph Diehl)
  • Tool to manage EC2 instances at AWS

• Lithium (Jesse Schwartzentruber & Gary Kwong)
  • Line-based testcase reducer
• octo (Various)
  • Fuzzing framework in JavaScript


• MozillaSecurity @ GitHub

• Developers
• OSS community
• #fuzzing on irc.mozilla.org
• Photo by Yortw
• Mozilla Style template
• 
  
• gkw@mozilla.com