diff --git a/.gitignore b/.gitignore index a4d2467..28ebee6 100644 --- a/.gitignore +++ b/.gitignore @@ -9,7 +9,7 @@ /*.egg-info # Don't distribute test results, since they get regenerated -/test-results* +/test-* # We may create tarballs, don't include those /*.tar.gz diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 7fbe255..7b4a8bc 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -55,15 +55,15 @@ An easy way to install pylint is to use pip. Most python installs have pip, but if yours does not (e.g., Cygwin), install pip with: -> python -m ensurepip +`python -m ensurepip` You may want to upgrade pip with: -> pip install --upgrade pip +`pip install --upgrade pip` Finally, you can actually install pylint using: -> pip install pylint +`pip install pylint` ## Code Conventions @@ -80,16 +80,12 @@ patches to improve that are often welcome. The code must run on both Python 2.7 and Python 3. To check that it works on both, run: -~~~~ -make check -~~~~ +`make check` We use "pylint" to check for style and other generic quality problems. To check that the code passes these quality tests, run: -~~~~ -make pylint -~~~~ +`make pylint` We require that the pylint results for contributions be at least 9.5/10 as configured with the provided "pylintrc" file, without any errors ("E"). diff --git a/INSTALL.txt b/INSTALL.md similarity index 76% rename from INSTALL.txt rename to INSTALL.md index e55ab49..95529e7 100644 --- a/INSTALL.txt +++ b/INSTALL.md @@ -1,24 +1,23 @@ -Installing flawfinder +# Installing flawfinder You can install flawfinder a number of different ways. Choose the approach that's most convenient for you! -The options (described below) are (1) pip, -(2) package for Unix-like system, (3) source install, and (4) run directly. +The options (described below) are (1) pip, (2) package for Unix-like system, (3) source install, and (4) run directly. -1. PIP +## 1. PIP For many, the simple approach is to first install Python (2.7 or something in the 3.* series). Then use "pip" to install flawfinder (this will normally download the package): - pip install flawfinder +`pip install flawfinder` One advantage for using pip is that you'll generally get the *current* released version. -2. PACKAGE FOR UNIX-LIKE SYSTEM (including Cygwin): +## 2. PACKAGE FOR UNIX-LIKE SYSTEM (including Cygwin): If you use an RPM-based system (e.g., Red Hat) or deb-based system (e.g., Debian), you can use their respective RPM or debian installation @@ -27,14 +26,17 @@ For a ports-based system where you have a current port, just use that. This will work out-of-the-box; it may not be the most recent version. +One way to accomplish this is: +`sudo apt install flawfinder` -3. TARBALL (SOURCE INSTALL) + +## 3. TARBALL (SOURCE INSTALL) QUICK START: The quick way to install flawfinder from the tarball is to unpack the tarball and type in something like this on the commmand line: - sudo make prefix=/usr install +`sudo make prefix=/usr install` Omit prefix=/usr to install in /usr/local instead. Omit "sudo" if you are already root. @@ -44,12 +46,12 @@ as described below, including prefix= and DESTDIR. Not enough? Here are more detailed step-by-step instructions and options. * Download the "tarball" and uncompress it. - GNU-based systems can run "tar xvzf flawfinder*.tar.gz" to do so, - then "cd" into the directory created. If that doesn't work - (e.g., you have an old tar program), use: - gunzip flawfinder*.tar.gz - tar xvf flawfinder*.tar - cd flawfinder-* + GNU-based systems can run `tar xvzf flawfinder-.tar.gz` to do so, + then move into the newly created directory with `cd flawfinder-` + If that doesn't work (e.g., you have an old tar program), use: + `gunzip flawfinder-.tar.gz` + `tar xvf flawfinder-.tar` + `cd flawfinder-` * Decide where you want to put it. Flawfinder normally installs everything in /usr/local, with the program in /usr/local/bin and the man page in @@ -79,7 +81,7 @@ Not enough? Here are more detailed step-by-step instructions and options. whenever you use make. This will be another make install override. If you'll just install it, do this: - make PYTHONEXT=.py install + `make PYTHONEXT=.py install` If you don't want to pass the "PYTHONEXT" extension each time, you can change the file "makefile" to remember this. Just change @@ -89,31 +91,31 @@ Not enough? Here are more detailed step-by-step instructions and options. * Now install it, giving whatever overrides you need. Currently it really only installs two files, an executable and a man page (documentation). In most cases, you'll need to be root, so run this first: - su + `su` - Then give the "make install" command appropriate for your system. + Then give the `make install` command appropriate for your system. For an all-default installation, which is what you need for most cases: make install - (you need to be root; "make uninstall" reverses it). + (you need to be root; `make uninstall` reverses it). To install in /usr (the program in /usr/bin, the manual in /usr/man): - make prefix=/usr install + `make prefix=/usr install` or alternatively, using the older flawfinder conventions: - make INSTALL_DIR=/usr install + `make INSTALL_DIR=/usr install` To install in /usr on Cygwin: - make prefix=/usr PYTHONEXT=.py install + `make prefix=/usr PYTHONEXT=.py install` To put the binaries in /usr/bin, and the manuals under /usr/local/share/man (common for Red Hat Linux), do: - make prefix=/usr mandir=/usr/local/share/man install + `make prefix=/usr mandir=/usr/local/share/man install` - The installer and uninstaller honor DESTDIR. + The installer and uninstaller honor `DESTDIR`. -4. DIRECT EXECUTION +## 4. DIRECT EXECUTION You can also simply run the program in the directory you've unpacked it into. It's a simple Python program, just type into a command line: - ./flawfinder files_or_directory +`./flawfinder files_or_directory` diff --git a/README b/README.md similarity index 75% rename from README rename to README.md index 54e562d..b233db6 100644 --- a/README +++ b/README.md @@ -1,4 +1,6 @@ -This is "flawfinder" by David A. Wheeler, . +# About + +This is "flawfinder" by [David A. Wheeler](mailto:dwheeler@dwheeler.com). Flawfinder is a simple program that scans C/C++ source code and reports potential security flaws. It can be a useful tool for examining software @@ -7,29 +9,36 @@ static source code analysis tools more generally. It is designed to be easy to install and use. Flawfinder supports the Common Weakness Enumeration (CWE) and is officially CWE-Compatible. -For more information, see: - http://www.dwheeler.com/flawfinder +For more information, see the [project website](http://www.dwheeler.com/flawfinder) + +# Platforms Flawfinder is designed for use on Unix/Linux/POSIX systems (including Cygwin, Linux-based systems, MacOS, and *BSDs) as a command line tool. It requires Python 2.7 or Python 3. +# Installation + If you just want to *use* it, you can install flawfinder with Python's "pip" or with your system's package manager (flawfinder has packages for many systems). It also supports easy installation -following usual "make install" source installation conventions. -The file INSTALL.txt has more detailed installation instructions. +following usual `make install` source installation conventions. +The file [INSTALL.md](INSTALL.md) has more detailed installation instructions. You don't HAVE to install it to run it, but it's easiest that way. +# Usage + To run flawfinder, just give it a list of source files or directories to example. For example, to examine all files in "src/" and down recursively: - flawfinder src/ +`flawfinder src/` The manual page (flawfinder.1 or flawfinder.pdf) describes how to use flawfinder (including its various options) and related information -(such as how it supports CWE). For example, the "--html" option generates -output in HTML format. The "--help" option gives a brief list of options. +(such as how it supports CWE). For example, the `--html` option generates +output in HTML format. The `--help` option gives a brief list of options. + +# Under the hood More technically, flawfinder uses lexical scanning to find tokens (such as function names) that suggest likely vulnerabilities, estimates their @@ -42,8 +51,12 @@ vulnerabilities in programs that cannot be built or cannot be linked. Flawfinder also doesn't get as confused by macro definitions and other oddities that more sophisticated tools have trouble with. +# Contributions + We love contributions! For more information on contributing, see -the file CONTRIBUTING.md. +the file [CONTRIBUTING.md](CONTRIBUTING.md). + +# License Flawfinder is released under the GNU GPL license version 2 or later (GPL-2.0+). -See the COPYING file for license information. +See the [COPYING](COPYING) file for license information. diff --git a/flawfinder b/flawfinder index 975b9ab..b0a5f4b 100755 --- a/flawfinder +++ b/flawfinder @@ -92,7 +92,7 @@ required_regex = None # If non-None, regex that must be met to report required_regex_compiled = None ERROR_ON_DISABLED_VALUE = 999 -error_level = ERROR_ON_DISABLED_VALUE # Level where we're return error code +error_level = ERROR_ON_DISABLED_VALUE # Level where we're return error code error_level_exceeded = False displayed_header = 0 # Have we displayed the header yet? @@ -315,10 +315,12 @@ def htmlize(s): # Take s, and return legal (UTF-8) HTML. return s.replace("&", "&").replace("<", "<").replace(">", ">") + def h(s): # htmlize s if we're generating html, otherwise just return s. return htmlize(s) if output_format else s + def print_multi_line(text): # Print text as multiple indented lines. width = 78 @@ -337,6 +339,7 @@ def print_multi_line(text): print(w, end='') position = position + len(w) + 1 + # This matches references to CWE identifiers, so we can HTMLize them. # We don't refer to CWEs with one digit, so we'll only match on 2+ digits. link_cwe_pattern = re.compile(r'(CWE-([1-9][0-9]+))([,()!/])') @@ -607,6 +610,7 @@ def c_singleton_string(text): "Returns true if text is a C string with 0 or 1 character." return 1 if p_c_singleton_string.search(text) else 0 + # This string defines a C constant. p_c_constant_string = re.compile(r'^\s*L?"([^\\]|\\[^0-6]|\\[0-6]+)*"$') @@ -826,9 +830,11 @@ def cpp_unsafe_stl(hit): if len(hit.parameters) <= 4: add_warning(hit) + def normal(hit): add_warning(hit) + # "c_ruleset": the rules for identifying "hits" in C (potential warnings). # It's a dictionary, where the key is the function name causing the hit, # and the value is a tuple with the following format: @@ -1444,7 +1450,7 @@ def process_c_file(f, patch_infos): linebegin = 1 codeinline = 0 # 1 when we see some code (so increment sloc at newline) - if (patch_infos is not None) and (not f in patch_infos): + if (patch_infos is not None) and (f not in patch_infos): # This file isn't in the patch list, so don't bother analyzing it. if not quiet: if output_format: @@ -1610,7 +1616,7 @@ def expand_ruleset(ruleset): # Note that this "for" loop modifies the ruleset while it's iterating, # so we *must* convert the keys into a list before iterating. for rule in list(ruleset.keys()): - if "|" in rule: # We found a rule to expand. + if "|" in rule: # We found a rule to expand. for newrule in rule.split("|"): if newrule in ruleset: print("Error: Rule %s, when expanded, overlaps %s" % ( @@ -2020,10 +2026,12 @@ def process_files(): process_file_args(files, patch_infos) return True + def hitlist_sort_key(hit): """Sort key for hitlist.""" return (-hit.level, hit.filename, hit.line, hit.column, hit.name) + def show_final_results(): global hitlist global error_level_exceeded @@ -2034,7 +2042,7 @@ def show_final_results(): possible_levels = (0, 1, 2, 3, 4, 5) for i in possible_levels: # Initialize count_per_level count_per_level[i] = 0 - for i in possible_levels: # Initialize count_per_level + for i in possible_levels: # Initialize count_per_level_and_up count_per_level_and_up[i] = 0 if show_immediately or not quiet: # Separate the final results. print() @@ -2055,7 +2063,7 @@ def show_final_results(): if output_format: print("
    ") for hit in hitlist: - if not diffhitlist_filename or not hit in diff_hitlist: + if not diffhitlist_filename or hit not in diff_hitlist: count_per_level[hit.level] = count_per_level[hit.level] + 1 if hit.level >= minimum_level: hit.show() @@ -2183,6 +2191,7 @@ def flawfind(): save_if_desired() return 1 if error_level_exceeded else 0 + if __name__ == '__main__': try: sys.exit(flawfind()) diff --git a/makefile b/makefile index 0d79503..bde1baa 100644 --- a/makefile +++ b/makefile @@ -177,9 +177,9 @@ test_006: flawfinder test.c test_007: setup.py @echo 'test_007 (setup.py sane)' - @test "`$(PYTHON) setup.py --name`" == 'flawfinder' - @test "`$(PYTHON) setup.py --license`" == 'GPL-2.0+' - @test "`$(PYTHON) setup.py --author`" == 'David A. Wheeler' + @test "`$(PYTHON) setup.py --name`" = 'flawfinder' + @test "`$(PYTHON) setup.py --license`" = 'GPL-2.0+' + @test "`$(PYTHON) setup.py --author`" = 'David A. Wheeler' # Run all tests on *one* version of Python; # output shows differences from expected results. diff --git a/release_process.md b/release_process.md index 9a80e0a..12646af 100644 --- a/release_process.md +++ b/release_process.md @@ -15,26 +15,24 @@ setup.py index.html Then run: -make test && make test-is-correct # update version number in tests +`make test && make test-is-correct` # update version number in tests ## Test it -make check # Run tests in Python 2 and 3 +`make check` # Run tests in Python 2 and 3 ## Tag version Once you're sure this is the *real* version, tag it: -git tag VERSION -git push --tags +`git tag VERSION` +`git push --tags` ## Create tarball Run: +`make distribute` -~~~~ -make distribute -~~~~ ## Post tarball @@ -48,12 +46,8 @@ the usual places: Create a PyPi distribution package: -~~~~ -make pypi -~~~~ +`make pypi` And upload it: -~~~~ -make upload-pypi -~~~~ +`make upload-pypi`