What is fuzzing and fuzz testing? (original) (raw)

Fuzzing definition

Fuzzing—also known as fuzz testing—is an automated software testing technique that involves inputting random or invalid data into a computer program and observing its behavior and output.

The goal of fuzzing is to reveal bugs and security vulnerabilities in source code you might not find through traditional testing methods. IT teams often use fuzzing to test software that handles user input, such as web apps, network protocols, file formats, and operating systems.

Integrating fuzzing into your team’s software development lifecycle (SDLC) and quality assurance process can help improve the security and performance of your products well ahead of their ship date. Doing so has the potential to save you overall costs and other resources.

How fuzz testing works

Fuzz testing involves using tools and frameworks to automate the process of generating, mutating, and sending input data to systems under test and reporting the results. Fuzzing mimics real-world data input scenarios to identify flaws and weaknesses in the systems being tested.

Setting up and running a fuzzing campaign consists of the following steps:

1. Define test objectives and scope. Identify which software system or component to test. It should handle user input.
2. Select a fuzzing tool. Hundreds of commercial and free fuzzing tools and engines exist. Different tools work with different programming languages, so pick a tool that works with the app or hardware you want to test.
3. Start fuzzing. The fuzzing tool (also known as a fuzzer) will generate or mutate input data—either randomly or based on predetermined heuristics—and feed that data to the system being tested.
4. Monitor the target. The fuzzer will detect crashes, memory leaks, buffer overflows, exceptions, and other deviations from the expected output.
5. Repair anomalies. The fuzzer will also create a report of the anomalies found and the input data that caused those anomalies. Your development teams can analyze these issues and work on viable fixes.

Fuzzing techniques and strategies

Different fuzzing techniques vary in how they generate or alter input data. Here are some of the most common techniques.

• Black-box fuzzing involves randomly generating input data without any knowledge of the internal structure of the system being tested. Although this type of fuzzing can generate vast quantities of input data, it may not be effective at finding complex bugs and vulnerabilities.
• White-box fuzzing involves generating input data based on analysis of the system’s source code. This method can generate more targeted, relevant input data than black-box fuzzing and find more intricate errors and vulnerabilities.
• Grey-box fuzzing is a blend of black-box and white-box fuzzing, balancing efficiency with effectiveness. This type of fuzzing generates input data based on partial knowledge of the system being tested.
• Mutation-based fuzzing tests a system's tolerance or error-handling against minor or common variations of input data.The input data is generated by modifying valid, sample, previous, or other existing input data. The mutation is performed by changing, deleting, inserting, or swapping bits, bytes, or blocks of data.
• Generation-based fuzzing tests a system's compliance or compatibility with different or new formats of input data. Input data is generated from scratch by a random number generator, a model, or a template.
• Coverage-guided fuzzing can efficiently reveal a system’s potential security weaknesses, such as memory corruption bugs or logic errors. This testing technique maximizes code coverage by continuously generating and mutating input data and prioritizing inputs that lead to previously unexplored areas of the code.

Types of fuzzing targets

Fuzzing can be applied to a number of software systems and components that handle user input:

• Apps, libraries, and APIs. Fuzz testing can identify input validation errors, memory leaks, and API misuse in software components. To test, use a tool to input random or malformed data into the interfaces of an app, library, or API, and monitor for crashes and unexpected behaviors.
• Operating systems. Fuzzing can help find privilege escalation, race conditions, and memory leaks. To test, send the data to an operating system’s kernel, system calls, and drivers.
• Network protocols. Use fuzzing to help detect buffer overflows, memory corruption, and packet spoofing. To test, send the inputs to a network’s interfaces, stacks, and apps.
• File formats. Fuzz testing can identify heap overflows, stack overflows, and format string vulnerabilities. To test, send the data to file parsers, viewers, and processors.
• Web apps and browsers. Use fuzzing to detect vulnerabilities that could make a web app, server, or browser susceptible to cross-site scripting (XSS), SQL injections, remote code execution, or denial-of-service (DoS) cyberattacks. To test, use an automated tool to input random data into the fields of a web app or browser, and monitor for crashes or unexpected behaviors.

Benefits of fuzz testing

As mentioned, fuzzing can help improve the quality and security of your software. Specifically fuzzing can:

• Detect bugs and vulnerabilities that may be hard to find with unit testing, integration testing, code review, and other security testing methods.
• Test your system's robustness and resilience against malicious or unexpected input and help prevent potential cyberattacks or exploits.
• Increase the code coverage and test coverage of your system and reveal edge cases and corner cases. In edge cases, the system is pushed to its limits. In corner cases, multiple parameters are pushed to their limits simultaneously.
• Improve the reliability, stability, and performance of your system and reduce the risk of failures or crashes in production.
• Save time and resources by automating the testing process and potentially reducing the need for manual testing or human intervention.

Challenges and limitations of fuzz testing

Fuzzing isn’t always a fast, smooth, or accurate process, especially when testing complex and stateful apps or when using open-source fuzzing tools. Some of the biggest downsides of fuzz testing include:

• Consuming valuable time and resources. Fuzz testing can require a large amount of input data and can take a long time to complete. Plus, finding bugs and vulnerabilities can require significant computing power.
• Generating false positives and false negatives. Fuzzing can yield incorrect or misleading results, including trivial or benign anomalies. Interpreting the results of fuzz testing can require a fair amount of manual assessment, which drains your resources.
• Yielding unpredictable or inconclusive results. The fuzzing process is marked by randomness and varied inputs, feedback, and environments. For this reason, there’s no guarantee that fuzz testing will generate the same results or the same coverage every time.
• Yielding incomplete or insufficient results. Fuzzing may not address all the possible input data, code paths, or scenarios of the system being tested. It also may miss rare or hidden bugs or vulnerabilities.

Best practices for effective fuzzing

Here are some best practices you can use to help better your fuzz testing to be as efficient and effective as possible:

• Pick the right tools and techniques for each scenario. Carefully consider the characteristics of the app or functionality to test, the complexity of input data, and the desired testing objectives.
• Develop a diverse set of input data. Use valid, invalid, common, uncommon, simple, complex, and malformed data.
• Use intelligent mutation strategies. Techniques likemutation-based fuzzing and generation-based fuzzing can help generate effective test cases that improve code coverage and vulnerability detection.
• Triage the results. Assess how severe, frequent, and easy-to-reproduce the discovered anomalies are and whether your fuzzer has detected any security implications. Work closely with your development teams to remediate vulnerabilities. For example, you could implement code fixes, input validation mechanisms, or security controls.
• Continue using other testing techniques. Employ fuzzing in conjunction with other testing methods to ensure you have extensive coverage. For example, manual testing and static analysis.

Conclusion

Building fuzzing into your software development lifecycle can help your organization minimize flaws and vulnerabilities in products and features before you ship them. The earlier in the development lifecycle you detect software bugs and weaknesses, the easier and cheaper they are to address.

Explore more fuzzing resources