walkero
/
flawfinder
1
0
Fork 0
Code Issues Pull Requests Projects Releases 1 Wiki Activity

Compare commits

base: walkero:amigaos
Branches Tags
walkero:amigaos
walkero:master
walkero:2.0.19r1-amiga
walkero:2.0.19
walkero:2.0.19-alpha1
walkero:2.0.18
walkero:2.0.17
walkero:2.0.16
walkero:2.0.15
walkero:2.0.14
walkero:2.0.12
walkero:2.0.11
walkero:2.0.10
walkero:2.0.9
walkero:2.0.8
walkero:2.0.7
walkero:2.0.6
walkero:2.0.5
walkero:2.0.4
walkero:2.0.2
walkero:2.0.0
walkero:1.31
walkero:1.29
walkero:1.28
walkero:v1.27
walkero:1.27
..
compare: walkero:1.31
Branches Tags
walkero:amigaos
walkero:master
walkero:2.0.19r1-amiga
walkero:2.0.19
walkero:2.0.19-alpha1
walkero:2.0.18
walkero:2.0.17
walkero:2.0.16
walkero:2.0.15
walkero:2.0.14
walkero:2.0.12
walkero:2.0.11
walkero:2.0.10
walkero:2.0.9
walkero:2.0.8
walkero:2.0.7
walkero:2.0.6
walkero:2.0.5
walkero:2.0.4
walkero:2.0.2
walkero:2.0.0
walkero:1.31
walkero:1.29
walkero:1.28
walkero:v1.27
walkero:1.27

No commits in common. "amigaos" and "1.31" have entirely different histories.
amigaos ... 1.31

89 changed files with 2373 additions and 14345 deletions
Show Stats Expand all files Collapse all files

27
.github/workflows/main.yml vendored
View File

@ -1,27 +0,0 @@
on: [push]
jobs:
flawfinder:
name: Flawfinder
runs-on: ubuntu-latest
steps:
# To use this repository's private action,
# you must check out the repository
- name: Checkout
uses: actions/checkout@v2
- name: Flawfinder action step
uses: ./ # Uses an action in the root directory
with:
arguments: '--sarif ./setup.py'
output: 'flawfinder_results.sarif'
- name: Upload a Build Artifact
uses: actions/upload-artifact@v2.2.4
with:
path: flawfinder_results.sarif
- name: Upload analysis results to GitHub Security tab
uses: github/codeql-action/upload-sarif@v1
with:
sarif_file: ${{github.workspace}}/flawfinder_results.sarif

15
.gitignore vendored
View File

@ -1,15 +0,0 @@
# Ignore some files
# Compiled python modules.
*.pyc
# Setuptools distribution folder.
/dist/
# Python egg metadata, regenerated from source files by setuptools.
/*.egg-info
# Don't distribute test results, since they get regenerated
/test-*
# We may create tarballs, don't include those
/*.tar.gz

144
CONTRIBUTING.md
View File

@ -1,144 +0,0 @@
# How to contribute to Flawfinder
We love contributions! Here's how to do them in a way that will
make everyone's lives easy.
Flawfinder has long been maintained on SourceForge.
We now support reporting and changes using either SourceForge or GitHub.
## Reporting
For normal problems, bugs, and feature requests, *except* for
vulnerabilities, please file a
[GitHub issue](https://github.com/david-a-wheeler/flawfinder/issues) or
[SourceForge ticket](https://sourceforge.net/p/flawfinder/_list/tickets).
If you find a vulnerability, please separately send a private email to
[David A. Wheeler](https://dwheeler.com/contactme.html).
To maintain confidentiality,
please use an email system that implements STARTTLS hop-by-hop email
encryption on outgoing email (many do, including
Gmail, hotmail.com, live.com, outlook.com, and runbox.com).
For more about STARTTLS, see the
EFF's [STARTTLS Everywhere](https://www.starttls-everywhere.org/) project.
We plan to handle vulnerabilities separately, fixing them and *then*
telling the world. We will gladly provide credit to vulnerability reporters
(unless you don't want the credit). We've never had a vulnerability
report, so this is theoretical at this time.
## Change process
We use "git" to track changes. To propose a change, create a fork
(copy) of the repository, make your changes, and create a
GitHub pull request or SourceForge merge request (they are the same thing).
If you're not familiar with the process, here's some
documentation for
[GitHub](https://help.github.com/articles/about-pull-requests/)
and
[SourceForge](https://sourceforge.net/p/forge/documentation/Git/).
## License and DCO
All proposed changes must be released under at least the project license,
in this case the GNU GPL version 2 or later (GPL-2.0+).
Proposers must agree to the
[Developer's Certificate of Origin](https://developercertificate.org/),
aka DCO.
The DCO basically says that you assert that you're legally allowed to
provide the commit. Please include in your commit a statement of the
form to confirm this ("git commit -s" will do this):
> Signed-off-by: Your-name \<your-email-address\>
You must include the DCO in your first commit proposal.
If you forget occasionally, we'll assume that you just forgot, but
please try to not forget.
## Development environment setup
As always, if you're modifying the software, you'll need to have
your development environment set up. You need:
* make
* python2 (invocable as "python2")
* python3 (invocable as "python3")
* pylint (see below)
An easy way to install pylint is to use pip.
Most python installs have pip, but if yours does not
(e.g., Cygwin), install pip with:
~~~~
python -m ensurepip
~~~~
You may want to upgrade pip with:
~~~~
pip install --upgrade pip
~~~~
Finally, you can actually install pylint using:
~~~~
pip install pylint
~~~~
## Code Conventions
To make the program easy to install everywhere, the main executable
is exactly one self-contained file. That involves some compromises,
but for now, please keep it that way.
We generally use the code conventions of
[PEP 8](https://www.python.org/dev/peps/pep-0008/).
The Python code uses 4-space indents (we used to use 2-space indents).
Do not use tabs. In some cases the code doesn't yet comply;
patches to improve that are often welcome.
The code must run on both Python 2.7 and Python 3.
To check that it works on both, run:
~~~~
make check
~~~~
We use "pylint" to check for style and other generic quality problems.
To check that the code passes these quality tests, run:
~~~~
make pylint
~~~~
We require that the pylint results for contributions be at least 9.5/10 as
configured with the provided "pylintrc" file, without any errors ("E").
Better is better. The current version *does* cause some pylint reports
(patches to fix those are welcome!). Note that we configure pylint
with the included "pylintrc" file.
We intentionally disable some checks as being "less important",
for example, the current code has many lines longer than 80 characters.
That said, patches to make lines fit in 80 characters are welcome.
## Tests
Make *sure* that your code passes the automated tests.
As noted above, invoke tests with
"make check", which tests the code using both Python2 and Python3.
It's our policy that as major new functionality is added to the software
produced by the project, tests of that functionality should be added to
the automated test suite.
## Other
Project documentation tends to be in markdown (.md) format.
We use "~~~~" so that it's easy to cut-and-paste commands if desired.
The main document is a man page, which is then converted to PDF.
Avoid trailing space or tab on a line in source files - those can create
hard-to-understand "differences" on lines.
We have earned a
[CII Best Practices Badge](https://bestpractices.coreinfrastructure.org/projects/323)... make sure we keep it!

163
ChangeLog
View File

@ -1,166 +1,3 @@
2021-08-29 David A. Wheeler
* Version 2.0.19
* Fix so we send error messages to stderr instead of stdout.
Originally we sent some to stdout by mistake, which could
mess up results since the error messages would be mixed up
with the results.
2021-06-24 David A. Wheeler
* Version 2.0.18
* Fix SARIF output. SARIF output is new to flawfinder, and
there was a subtle error in its generation that causes GitHub
to reject the SARIF file.
2021-06-02 David A. Wheeler
* Version 2.0.17
* Fix the distributed tarball, which didn't include the
key source file due to the earlier file restructure.
* Minor code style fix, which simplifies the code slightly.
* Update date in manual page to 2021. That's important because
the documentation now includes information on `--sarif`.
2021-05-31 David A. Wheeler
* Version 2.0.16
* The distributed source file is now flawfinder.py, not flawfinder.
This is part of a change that improves
improve cross-platform ease-of-use by using entry_points.
That said, "make install" will still
install it as "flawfinder" (so those who install it via
"make install" will see no change). Many thanks to Ben Spoor!
* Added support for generating SARIF output, use --sarif.
A big thanks to Yong Yan for this work!
* Track curly brace level to reduce some problems, my thanks to
Greg Myers for the work!
* Improved handling of Git patch format, thanks to
Robin Geffroy.
2021-01-11 David A. Wheeler
* Version 2.0.15
* Fixed some release problems in 2.0.14.
* Improved handling of LoadLibraryEx; flawfinderr no longer complains
about certain constructs that are known to be safe (eliminating
some false positives).
2021-01-09 David A. Wheeler
* Version 2.0.14
* If there are >0 hits, tell users how to ignore them as part of the
tool output.
* Various Windows improvments.
Ignore LoadLibraryEx if its third parameter is
LOAD_LIBRARY_SEARCH_SYSTEM32, as this is safe, and
remove the rule for InitialCriticalSection
(this is no longer a vulnerability on current widely-used versions
of Windows)
* Various C++ improvements. Add .hpp support for C++,
ignore "system::" to reduce false positives,
treat ' as digit separator when file extension is a C++ file
(for C++14).
* I had some release problems; this is identified as 2.0.14
(skipping a few minor numbers) to ensure that the version
number uniquely identifies a specific release.
2020-02-17 David A. Wheeler
* Version 2.0.11
* Provide a much more detailed error report, including recommended
solutions, when character encoding problems hit.
As Python3 has slowly gained in popularity, its failure to provide
useful built-ins to handle real-world character encoding problems
hurts more people. (E.g., many files don't comply with *any*
character set encoding standard, and Python3 can't read them
without enabling options that are wrong for others.)
We can at least provide much more detailed feedback to help
explain the various options available.
2019-06-22 David A. Wheeler
* Version 2.0.10
* Use binary mode when reading a diffhitlist.
My thanks to Michał Górny, who both reported the problem
and provided the patch!
2019-05-19 David A. Wheeler
* Version 2.0.9
* Fixes a serious defect in --diffhitlist
2019-05-17 Labidurie Jerome
* Fixed a serious defect in --diffhitlist option and added a unit test
2019-01-21 David A. Wheeler <dwheeler, at, dwheeler.com>
* Version 2.0.8
* Don't warn if memcpy call includes sizeof(first arg).
Thanks to Michael Clark for this improvement!
* Bugfix (banned function _ftcsat should be _ftcscat).
Thanks to Lucas Ramage for reporting this!
* Documentation tweaks. Make it clear that GitHub issues and
pull requests are supported, and use ~~~~ in markdown
to ease copy-and-paste from documentation.
2018-09-30 David A. Wheeler <dwheeler, at, dwheeler.com>
* Incorporate many small improvements from nickthetait
* Fix a number of bugs reported by philipp
* Update URLs for www.dwheeler.com -> dwheeler.com
2018-04-04 David A. Wheeler <dwheeler, at, dwheeler.com>
* Version 2.0.6
2018-01-26 David A. Wheeler <dwheeler, at, dwheeler.com>
* Small fixes
* Update cwe.mitre.org URLs to use https
2017-11-16 David A. Wheeler <dwheeler, at, dwheeler.com>
* add detection of crypt_r function
* add detection of errant equal, mismatch, and is_permutation
* update CWE, risk, and discussion for C++14 STL functions
* Always report hit counts correctly, even if ignored using -m
* Update www.dwheeler.com URLs to use https
2017-09-02 David A. Wheeler <dwheeler, at, dwheeler.com>
* Version 2.0.4
* Switch from distutils to setuptools
* Directly support "pip" installs
2017-08-26 David A. Wheeler <dwheeler, at, dwheeler.com>
* Version 2.0.2
* Flawfinder can now run on either Python 2.7 or 3
* Added more tests
* Implemented additional code cleanups recommended by Pylint
* Modified documentation in various ways to clarify things
2017-08-13 David A. Wheeler <dwheeler, at, dwheeler.com>
* Version 2.0.1
* Tranform many internal constructs to work on Python 2 or 3,
with the eventual goal of making it run on either.
2017-07-29 David A. Wheeler <dwheeler, at, dwheeler.com>
* Version 2.0.0
* Change version numbers to use Semantic Versioning (x.y.z)
* Add support for generating CSV (comma-separated value) format,
to make this tool easier to integrate into larger toolsuites.
* Fixed a number of issues - and even a few bugs - found by the
Python static analysis tool pylint.
* Document in CONTRIBUTING.md how to contribute to the project.
* Change version number to 2.0.0, because we have a subtle
interface change that won't affect most people but it
*may* affect those who use postprocess
flawfinder data on CWEs. The fundamental issue is that
in some cases a hit corresponds to multiple CWEs. As a result,
in some cases flawfinder will list a sequence of CWEs
in the format "more-general/more-specific", where the CWE actually
being mapped is followed by a "!".
This is always done whenever a flaw is not mapped directly to
a top 25 CWE, but the mapping is related to such a CWE.
So "CWE-119!/CWE-120" means that the vulnerability is mapped
to CWE-119 and that CWE-120 is a subset of CWE-119.
In contrast, "CWE-362/CWE-367!" means that the hit is mapped to
CWE-367, a subset of CWE-362.
Note that this is a subtle syntax change from flawfinder
version 1.31; in flawfinder version 1.31,
the form "more-general:more-specific" meant what is now listed as
"more-general!/more-specific", while
"more-general/more-specific" meant "more-general/more-specific!".
Tools can handle both the version 1.31 and the current format,
if they wish, by noting that the older format did not use "!" at all.
These mapping mechanisms simplify searching for certain CWEs.
2014-08-03 David A. Wheeler <dwheeler, at, dwheeler.com>
* Release version 1.31, a set of small improvements mostly CWE-related.
* Note that flawfinder is officially CWE-compatible.

10
Dockerfile
View File

@ -1,10 +0,0 @@
# Container image that runs your code
FROM python:3
# Copies your code file from your action repository to the filesystem path `/` of the container
COPY entrypoint.sh /entrypoint.sh
RUN pip install flawfinder
# Code file to execute when the docker container starts up (`entrypoint.sh`)
ENTRYPOINT ["/entrypoint.sh"]

106
INSTALL.md → INSTALL.txt
View File

@ -1,65 +1,30 @@
# Installing flawfinder
You can install flawfinder a number of different ways.
Choose the approach that's most convenient for you!
The options (described below) are (1) pip, (2) package for Unix-like system, (3) source install, and (4) run directly.
## 1. PIP
For many, the simple approach is to first install Python
(2.7 or something reasonable in the 3.X series).
Then use `pip` to install flawfinder
(this will normally download the package):
~~~~
pip install flawfinder
~~~~
One advantage for using pip is that you'll generally get the
*current* released version.
## 2. PACKAGE FOR UNIX-LIKE SYSTEM (including Cygwin):
To install flawfinder on a Unix-like system (including Cygwin):
If you use an RPM-based system (e.g., Red Hat) or deb-based system
(e.g., Debian), you can use their respective RPM or debian installation
program and just install it; then ignore the rest of these instructions.
(e.g., Debian), use their respective RPM or debian installation program
and just install it; then ignore the rest of these instructions.
For a ports-based system where you have a current port, just use that.
This will work out-of-the-box; it may not be the most recent version.
One way to accomplish this is:
~~~~
sudo apt install flawfinder
~~~~
## 3. TARBALL (SOURCE INSTALL)
QUICK START:
The quick way to install flawfinder from the tarball is to
unpack the tarball and type in something like this on the command line:
~~~~
sudo make prefix=/usr install
~~~~
unpack the tarball and type in something like this on the commmand line:
sudo make prefix=/usr install
Omit prefix=/usr to install in /usr/local instead.
Omit "sudo" if you are already root.
Note that this installation approach follows the usual install conventions
as described below, including prefix= and DESTDIR.
You don't even have to install it; running it as "./flawfinder ..."
will work just fine if you have Python 2 installed (though it's
probably more convenient to install it).
Not enough? Here are more detailed step-by-step instructions and options.
* Download the "tarball" and uncompress it.
GNU-based systems can run `tar xvzf flawfinder-<version>.tar.gz` to do so,
then move into the newly created directory with `cd flawfinder-<version>`
If that doesn't work (e.g., you have an old tar program), use:
`gunzip flawfinder-<version>.tar.gz`
`tar xvf flawfinder-<version>.tar`
`cd flawfinder-<version>`
GNU-based systems can run "tar xvzf flawfinder*.tar.gz" to do so,
then "cd" into the directory created. If that doesn't work
(e.g., you have an old tar program), use:
gunzip flawfinder*.tar.gz
tar xvf flawfinder*.tar
cd flawfinder-*
* Decide where you want to put it. Flawfinder normally installs everything
in /usr/local, with the program in /usr/local/bin and the man page in
@ -78,18 +43,18 @@ Not enough? Here are more detailed step-by-step instructions and options.
You can also use the older flawfinder makefile variables to control
installation; you can set:
`INSTALL_DIR` = prefix, default $(prefix);
`INSTALL_DIR_BIN` = program location, default `$(bindir)`;
`INSTALL_DIR_MAN` = manual location, default `$(man1dir)`.
Note that the default of `INSTALL_DIR_MAN` has changed; at one time
it was `$(prefix)/man/man1`, but now it is `$(prefix)/share/man/man1`
INSTALL_DIR = prefix, default $(prefix)
INSTALL_DIR_BIN = program location, default $(bindir)
INSTALL_DIR_MAN = manual location, default $(man1dir)
Note that the default of INSTALL_DIR_MAN has changed; at one time
it was $(prefix)/man/man1, but now it is $(prefix)/share/man/man1
* If you're using Cygwin on Windows, you can install it using "make install"
but you need to tell the makefile to use the .py extension
whenever you use make. This will be another make install override.
If you'll just install it, do this:
`make PYTHONEXT=.py install`
make PYTHONEXT=.py install
If you don't want to pass the "PYTHONEXT" extension each time,
you can change the file "makefile" to remember this. Just change
@ -99,33 +64,34 @@ Not enough? Here are more detailed step-by-step instructions and options.
* Now install it, giving whatever overrides you need. Currently it really
only installs two files, an executable and a man page (documentation).
In most cases, you'll need to be root, so run this first:
`su`
su
Then give the `make install` command appropriate for your system.
Then give the "make install" command appropriate for your system.
For an all-default installation, which is what you need for most cases:
make install
(you need to be root; `make uninstall` reverses it).
(you need to be root; "make uninstall" reverses it).
To install in /usr (the program in /usr/bin, the manual in /usr/man):
`make prefix=/usr install`
make prefix=/usr install
or alternatively, using the older flawfinder conventions:
`make INSTALL_DIR=/usr install`
make INSTALL_DIR=/usr install
To install in /usr on Cygwin:
`make prefix=/usr PYTHONEXT=.py install`
make prefix=/usr PYTHONEXT=.py install
To put the binaries in /usr/bin, and the manuals under /usr/local/share/man
(common for Red Hat Linux), do:
`make prefix=/usr mandir=/usr/local/share/man install`
make prefix=/usr mandir=/usr/local/share/man install
The installer and uninstaller honor `DESTDIR`.
The installer and uninstaller honor DESTDIR.
## 4. DIRECT EXECUTION
* Windows systems should be able to run this on the command line (cmd.exe)
directly, but I haven't tried that.
* You can also simply run the program in the directory you've unpacked it
into. It's a simple Python program, just type into a command line:
./flawfinder files_or_directory
You can also simply run the program in the directory you've unpacked it
into. It's a simple Python program, just type into a command line:
~~~~
./flawfinder FILES-OR-DIRECTORY
~~~~

2
MANIFEST.in
View File

@ -1,5 +1,5 @@
include COPYING
include README.md
include README
include announcement
include ChangeLog
include flawfinder.1*

20
Makefile.amiga
View File

@ -1,20 +0,0 @@
#
# Project: flawfinder
# Created by George "walkero" Sokianos
# 2022-07-25
#
release: clean
mkdir -p release/flawfinder
cp -r release_files/* release/flawfinder/
cp flawfinder.py release/flawfinder/flawfinder
protect release/flawfinder/flawfinder srwed
cp -r simplejson release/flawfinder
cp ChangeLog release/flawfinder/
cp README.md release/flawfinder/
cp COPYING release/flawfinder/
lha -aeqr3 a flawfinder.lha release/
clean:
rm -f simplejson/#?.pyc

48
README Normal file
View File

@ -0,0 +1,48 @@
This is "flawfinder" by David A. Wheeler, <dwheeler@dwheeler.com>.
Flawfinder is a simple program that scans C/C++ source code and reports
potential security flaws. It can be a useful tool for examining software
for vulnerabilities, and it can also serve as a simple introduction to
static source code analysis tools more generally. It is designed to
be easy to install and use. Flawfinder supports the Common Weakness
Enumeration (CWE) and is officially CWE-Compatible.
For more information, see:
http://www.dwheeler.com/flawfinder
Flawfinder is designed for use on Unix/Linux/POSIX systems
(including Cygwin, Linux-based systems, MacOS, and *BSDs) as a
command line tool. It requires Python 2 (version 2.5 or later).
You can typically install flawfinder from its source code by doing this:
tar xvzf FILENAME.tar.gz # Uncompress distribution file
cd flawfinder-* # cd into it.
sudo make prefix=/usr install # Install in /usr
This installs the program as "/usr/bin/flawfinder" as well as the man page.
You can omit the "prefix=/usr"; it will then install under "/usr/local".
The file INSTALL.txt has more detailed installation instructions;
flawfinder supports the usual conventions (prefix, DESTDIR, etc.).
You don't HAVE to install it to run it, but it's easiest that way.
To run flawfinder, just give it a list of source files or directories to
example. For example, to examine all files in "src/" and down recursively:
flawfinder src/
The manual page (flawfinder.1 or flawfinder.pdf) describes how to use
flawfinder (including its various options) and related information
(such as how it supports CWE). For example, the "--html" option generates
output in HTML format. The "--help" option gives a brief list of options.
More technically, flawfinder uses lexical scanning to find tokens
(such as function names) that suggest likely vulnerabilities, estimates their
level of risk (e.g., by the text of function calls), and reports the results.
Flawfinder does not use or have access to information about control flow,
data flow, or data types. Thus, flawfinder will necessarily
produce many false positives for vulnerabilities and fail to report
many vulnerabilities. On the other hand, flawfinder can find
vulnerabilities in programs that cannot be built or cannot be linked.
Flawfinder also doesn't get as confused by macro definitions
and other oddities that more sophisticated tools have trouble with.
Flawfinder is released under the GNU GPL license version 2 or later (GPLv2+).
See the COPYING file for license information.

139
README.md
View File

@ -1,139 +0,0 @@
# About
This is "flawfinder" by [David A. Wheeler](mailto:dwheeler@dwheeler.com).
Flawfinder is a simple program that scans C/C++ source code and reports
potential security flaws. It can be a useful tool for examining software
for vulnerabilities, and it can also serve as a simple introduction to
static source code analysis tools more generally. It is designed to
be easy to install and use. Flawfinder supports the Common Weakness
Enumeration (CWE) and is officially CWE-Compatible.
For more information, see the [project website](http://dwheeler.com/flawfinder)
# Platforms
Flawfinder is designed for use on Unix/Linux/POSIX systems
(including Cygwin, Linux-based systems, MacOS, and various BSDs) as a
command line tool. It requires either Python 2.7 or Python 3.
# Installation
If you just want to *use* it, you can install flawfinder with
Python's "pip" or with your system's package manager (flawfinder has
packages for many systems). It also supports easy installation
following usual `make install` source installation conventions.
The file [INSTALL.md](INSTALL.md) has more detailed installation instructions.
You don't HAVE to install it to run it, but it's easiest that way.
# Usage
To run flawfinder, just give it a list of source files or directories to
example. For example, to examine all files in "src/" and down recursively:
~~~~
flawfinder src/
~~~~
To examine all files in the *current* directory and down recursively:
~~~~
flawfinder ./
~~~~
Hits (findings) are given a risk level from 0 (very low risk) to 5 (high risk),
By default, findings of risk level 1 or higher are shown.
You can show only the hits of risk level 4 or higher in the current
directory and down this way:
~~~~
flawfinder --minlevel 4 ./
~~~~
The manual page (flawfinder.1 or flawfinder.pdf) describes how to use
flawfinder (including its various options) and related information
(such as how it supports CWE). For example, the `--html` option generates
output in HTML format. The `--help` option gives a brief list of options.
# Character Encoding Errors
Flawfinder must be able to correctly interpret your source code's
character encoding.
In the vast majority of cases this is not a problem, especially
if the source code is correctly encoded using UTF-8 and your system
is configured to use UTF-8 (the most common situation by far).
However, it's possible for flawfinder to halt if there is a
character encoding problem and you're running Python3.
The usual symptom is error messages like this:
`Error: encoding error in FILENAME 'ENCODING' codec can't decode byte ... in position ...: invalid start byte`
Unfortunately, Python3 fails to provide useful built-ins to deal with this.
Thus, it's non-trivial to deal with this problem without depending on external
libraries (which we're trying to avoid).
If you have this problem, see the flawfinder manual page for a collection
of various solutions.
One of the simplest is to simply convert the source code and system
configuration to UTF-8.
You can convert source code to UTF-8 using tools such as the
system tool `iconv` or the Python program
[`cvt2utf`](https://pypi.org/project/cvt2utf/);
you can install `cvt2utf` using `pip install cvt2utf`.
# Under the hood
More technically, flawfinder uses lexical scanning to find tokens
(such as function names) that suggest likely vulnerabilities, estimates their
level of risk (e.g., by the text of function calls), and reports the results.
Flawfinder does not use or have access to information about control flow,
data flow, or data types. Thus, flawfinder will necessarily
produce many false positives for vulnerabilities and fail to report
many vulnerabilities. On the other hand, flawfinder can find
vulnerabilities in programs that cannot be built or cannot be linked.
Flawfinder also doesn't get as confused by macro definitions
and other oddities that more sophisticated tools have trouble with.
# Flawfinder GitHub Action
There's a GitHub action available for those who use GitHub.
## Usage
See [action.yml](https://github.com/david-a-wheeler/flawfinder/blob/main/action.yml)
Create a .yml file under .github/workflows with the following contents:
### Basic demo:
```yml
- name: flawfinder_scan
uses: david-a-wheeler/flawfinder@2.0.19
with:
arguments: '--sarif ./'
output: 'flawfinder_results.sarif'
```
You can add many other additions to the arguments.
For example, `--error-level=4` will cause an error to be returned if
flawfinder finds a vulnerability of level 4 or higher.
Notice the version number after the `@` symbol; you can select a
different version.
You can find the action name and version string from [Marketplace](https://github.com/marketplace/actions/flawfinder_scan)
by clicking "Use latest/xxx version" button.
### Input options:
- arguments: [Flawfinder command arguments](ttps://github.com/david-a-wheeler/flawfinder/blob/master/README.md#usage)
- output: Flawfinder output file name. Can be uploaded to GitHub.
# Contributions
We love contributions! For more information on contributing, see
the file [CONTRIBUTING.md](CONTRIBUTING.md).
# License
Flawfinder is released under the GNU GPL license version 2 or later (GPL-2.0+).
See the [COPYING](COPYING) file for license information.

17
action.yml
View File

@ -1,17 +0,0 @@
name: 'flawfinder_scan'
description: 'Execute Flawfinder to scan source code for vulnerabilities'
inputs:
arguments:
description: 'Command arguments to be sent to Flawfinder'
required: true
default: ''
output:
description: 'Output file name'
required: true
default: ''
runs:
using: 'docker'
image: 'Dockerfile'
args:
- ${{ inputs.arguments }}
- ${{ inputs.output }}

235
test/correct-results.html → correct-results.html
View File

@ -9,9 +9,9 @@
<body>
<h1>Flawfinder Results</h1>
Here are the security scan results from
<a href="https://dwheeler.com/flawfinder">Flawfinder version 2.0.19</a>,
(C) 2001-2019 <a href="https://dwheeler.com">David A. Wheeler</a>.
Number of rules (primarily dangerous function names) in C/C++ ruleset: 222
<a href="http://www.dwheeler.com/flawfinder">Flawfinder version 1.31</a>,
(C) 2001-2014 <a href="http://www.dwheeler.com">David A. Wheeler</a>.
Number of rules (primarily dangerous function names) in C/C++ ruleset: 169
<p>
Examining test.c <br>
Examining test2.c <br>
@ -20,186 +20,175 @@ Examining test2.c <br>
<ul>
<li>test.c:32: <b> [5] </b> (buffer) <i> gets:
Does not check for buffer overflows (<a
href="https://cwe.mitre.org/data/definitions/120.html">CWE-120</a>, <a
href="https://cwe.mitre.org/data/definitions/20.html">CWE-20</a>). Use
href="http://cwe.mitre.org/data/definitions/120.html">CWE-120</a>, <a
href="http://cwe.mitre.org/data/definitions/20.html">CWE-20</a>). Use
fgets() instead. </i>
<pre>
gets(f);
</pre>
<li>test.c:60: <b> [5] </b> (buffer) <i> strncat:
<li>test.c:56: <b> [5] </b> (buffer) <i> strncat:
Easily used incorrectly (e.g., incorrectly computing the correct maximum
size to add) [MS-banned] (<a
href="https://cwe.mitre.org/data/definitions/120.html">CWE-120</a>).
Consider strcat_s, strlcat, snprintf, or automatically resizing strings.
Risk is high; the length parameter appears to be a constant, instead of
computing the number of characters left. </i>
size to add) (<a
href="http://cwe.mitre.org/data/definitions/120.html">CWE-120</a>).
Consider strcat_s, strlcat, or automatically resizing strings. Risk is
high; the length parameter appears to be a constant, instead of computing
the number of characters left. </i>
<pre>
strncat(d,s,sizeof(d)); /* Misuse - this should be flagged as riskier. */
</pre>
<li>test.c:61: <b> [5] </b> (buffer) <i> _tcsncat:
<li>test.c:57: <b> [5] </b> (buffer) <i> _tcsncat:
Easily used incorrectly (e.g., incorrectly computing the correct maximum
size to add) [MS-banned] (<a
href="https://cwe.mitre.org/data/definitions/120.html">CWE-120</a>).
size to add) (<a
href="http://cwe.mitre.org/data/definitions/120.html">CWE-120</a>).
Consider strcat_s, strlcat, or automatically resizing strings. Risk is
high; the length parameter appears to be a constant, instead of computing
the number of characters left. </i>
<pre>
_tcsncat(d,s,sizeof(d)); /* Misuse - flag as riskier */
</pre>
<li>test.c:64: <b> [5] </b> (buffer) <i> MultiByteToWideChar:
<li>test.c:60: <b> [5] </b> (buffer) <i> MultiByteToWideChar:
Requires maximum length in CHARACTERS, not bytes (<a
href="https://cwe.mitre.org/data/definitions/120.html">CWE-120</a>). Risk
is high, it appears that the size is given as bytes, but the function
requires size as characters. </i>
href="http://cwe.mitre.org/data/definitions/120.html">CWE-120</a>). Risk is
high, it appears that the size is given as bytes, but the function requires
size as characters. </i>
<pre>
MultiByteToWideChar(CP_ACP,0,szName,-1,wszUserName,sizeof(wszUserName));
</pre>
<li>test.c:66: <b> [5] </b> (buffer) <i> MultiByteToWideChar:
<li>test.c:62: <b> [5] </b> (buffer) <i> MultiByteToWideChar:
Requires maximum length in CHARACTERS, not bytes (<a
href="https://cwe.mitre.org/data/definitions/120.html">CWE-120</a>). Risk
is high, it appears that the size is given as bytes, but the function
requires size as characters. </i>
href="http://cwe.mitre.org/data/definitions/120.html">CWE-120</a>). Risk is
high, it appears that the size is given as bytes, but the function requires
size as characters. </i>
<pre>
MultiByteToWideChar(CP_ACP,0,szName,-1,wszUserName,sizeof wszUserName);
</pre>
<li>test.c:77: <b> [5] </b> (misc) <i> SetSecurityDescriptorDacl:
<li>test.c:73: <b> [5] </b> (misc) <i> SetSecurityDescriptorDacl:
Never create NULL ACLs; an attacker can set it to Everyone (Deny All
Access), which would even forbid administrator access (<a
href="https://cwe.mitre.org/data/definitions/732.html">CWE-732</a>). </i>
href="http://cwe.mitre.org/data/definitions/732.html">CWE-732</a>). </i>
<pre>
SetSecurityDescriptorDacl(&amp;sd,TRUE,NULL,FALSE);
</pre>
<li>test.c:77: <b> [5] </b> (misc) <i> SetSecurityDescriptorDacl:
<li>test.c:73: <b> [5] </b> (misc) <i> SetSecurityDescriptorDacl:
Never create NULL ACLs; an attacker can set it to Everyone (Deny All
Access), which would even forbid administrator access (<a
href="https://cwe.mitre.org/data/definitions/732.html">CWE-732</a>). </i>
href="http://cwe.mitre.org/data/definitions/732.html">CWE-732</a>). </i>
<pre>
SetSecurityDescriptorDacl(&amp;sd,TRUE,NULL,FALSE);
</pre>
<li>test.c:17: <b> [4] </b> (buffer) <i> strcpy:
Does not check for buffer overflows when copying to destination [MS-banned]
(<a href="https://cwe.mitre.org/data/definitions/120.html">CWE-120</a>).
Consider using snprintf, strcpy_s, or strlcpy (warning: strncpy easily
Does not check for buffer overflows when copying to destination (<a
href="http://cwe.mitre.org/data/definitions/120.html">CWE-120</a>).
Consider using strcpy_s, strncpy, or strlcpy (warning, strncpy is easily
misused). </i>
<pre>
strcpy(b, a);
</pre>
<li>test.c:20: <b> [4] </b> (buffer) <i> sprintf:
Does not check for buffer overflows (<a
href="https://cwe.mitre.org/data/definitions/120.html">CWE-120</a>). Use
href="http://cwe.mitre.org/data/definitions/120.html">CWE-120</a>). Use
sprintf_s, snprintf, or vsnprintf. </i>
<pre>
sprintf(s, "hello %s", bug);
</pre>
<li>test.c:21: <b> [4] </b> (buffer) <i> sprintf:
Does not check for buffer overflows (<a
href="https://cwe.mitre.org/data/definitions/120.html">CWE-120</a>). Use
href="http://cwe.mitre.org/data/definitions/120.html">CWE-120</a>). Use
sprintf_s, snprintf, or vsnprintf. </i>
<pre>
sprintf(s, gettext("hello %s"), bug);
</pre>
<li>test.c:22: <b> [4] </b> (format) <i> sprintf:
Potential format string problem (<a
href="https://cwe.mitre.org/data/definitions/134.html">CWE-134</a>). Make
href="http://cwe.mitre.org/data/definitions/134.html">CWE-134</a>). Make
format string constant. </i>
<pre>
sprintf(s, unknown, bug);
</pre>
<li>test.c:23: <b> [4] </b> (format) <i> printf:
If format strings can be influenced by an attacker, they can be exploited
(<a href="https://cwe.mitre.org/data/definitions/134.html">CWE-134</a>).
Use a constant for the format specification. </i>
(<a href="http://cwe.mitre.org/data/definitions/134.html">CWE-134</a>). Use
a constant for the format specification. </i>
<pre>
printf(bf, x);
</pre>
<li>test.c:25: <b> [4] </b> (buffer) <i> scanf:
The scanf() family's %s operation, without a limit specification, permits
buffer overflows (<a
href="https://cwe.mitre.org/data/definitions/120.html">CWE-120</a>, <a
href="https://cwe.mitre.org/data/definitions/20.html">CWE-20</a>). Specify
a limit to %s, or use a different input function. </i>
href="http://cwe.mitre.org/data/definitions/120.html">CWE-120</a>, <a
href="http://cwe.mitre.org/data/definitions/20.html">CWE-20</a>). Specify a
limit to %s, or use a different input function. </i>
<pre>
scanf("%s", s);
</pre>
<li>test.c:27: <b> [4] </b> (buffer) <i> scanf:
The scanf() family's %s operation, without a limit specification, permits
buffer overflows (<a
href="https://cwe.mitre.org/data/definitions/120.html">CWE-120</a>, <a
href="https://cwe.mitre.org/data/definitions/20.html">CWE-20</a>). Specify
a limit to %s, or use a different input function. </i>
href="http://cwe.mitre.org/data/definitions/120.html">CWE-120</a>, <a
href="http://cwe.mitre.org/data/definitions/20.html">CWE-20</a>). Specify a
limit to %s, or use a different input function. </i>
<pre>
scanf("%s", s);
</pre>
<li>test.c:38: <b> [4] </b> (format) <i> syslog:
If syslog's format strings can be influenced by an attacker, they can be
exploited (<a
href="https://cwe.mitre.org/data/definitions/134.html">CWE-134</a>). Use a
href="http://cwe.mitre.org/data/definitions/134.html">CWE-134</a>). Use a
constant format string for syslog. </i>
<pre>
syslog(LOG_ERR, attacker_string);
</pre>
<li>test.c:49: <b> [4] </b> (buffer) <i> _mbscpy:
Does not check for buffer overflows when copying to destination [MS-banned]
(<a href="https://cwe.mitre.org/data/definitions/120.html">CWE-120</a>).
Does not check for buffer overflows when copying to destination (<a
href="http://cwe.mitre.org/data/definitions/120.html">CWE-120</a>).
Consider using a function version that stops copying at the end of the
buffer. </i>
<pre>
_mbscpy(d,s); /* like strcpy, this doesn't check for buffer overflow */
</pre>
<li>test.c:56: <b> [4] </b> (buffer) <i> lstrcat:
Does not check for buffer overflows when concatenating to destination
[MS-banned] (<a
href="https://cwe.mitre.org/data/definitions/120.html">CWE-120</a>). </i>
<li>test.c:52: <b> [4] </b> (buffer) <i> lstrcat:
Does not check for buffer overflows when concatenating to destination (<a
href="http://cwe.mitre.org/data/definitions/120.html">CWE-120</a>). </i>
<pre>
lstrcat(d,s);
</pre>
<li>test.c:79: <b> [3] </b> (shell) <i> CreateProcess:
<li>test.c:75: <b> [3] </b> (shell) <i> CreateProcess:
This causes a new process to execute and is difficult to use safely (<a
href="https://cwe.mitre.org/data/definitions/78.html">CWE-78</a>). Specify
href="http://cwe.mitre.org/data/definitions/78.html">CWE-78</a>). Specify
the application path in the first argument, NOT as part of the second, or
embedded spaces could allow an attacker to force a different program to
run. </i>
<pre>
CreateProcess(NULL, "C:\\Program Files\\GoodGuy\\GoodGuy.exe -x", "");
</pre>
<li>test.c:79: <b> [3] </b> (shell) <i> CreateProcess:
<li>test.c:75: <b> [3] </b> (shell) <i> CreateProcess:
This causes a new process to execute and is difficult to use safely (<a
href="https://cwe.mitre.org/data/definitions/78.html">CWE-78</a>). Specify
href="http://cwe.mitre.org/data/definitions/78.html">CWE-78</a>). Specify
the application path in the first argument, NOT as part of the second, or
embedded spaces could allow an attacker to force a different program to
run. </i>
<pre>
CreateProcess(NULL, "C:\\Program Files\\GoodGuy\\GoodGuy.exe -x", "");
</pre>
<li>test.c:81: <b> [3] </b> (misc) <i> LoadLibraryEx:
Ensure that the full path to the library is specified, or current directory
may be used (<a
href="https://cwe.mitre.org/data/definitions/829.html">CWE-829</a>, <a
href="https://cwe.mitre.org/data/definitions/20.html">CWE-20</a>). Use a
flag like LOAD_LIBRARY_SEARCH_SYSTEM32 or
LOAD_LIBRARY_SEARCH_APPLICATION_DIR to search only desired folders. </i>
<pre>
(void) LoadLibraryEx(L"user32.dll", nullptr, LOAD_LIBRARY_AS_DATAFILE);
</pre>
<li>test.c:99: <b> [3] </b> (buffer) <i> getopt_long:
<li>test.c:91: <b> [3] </b> (buffer) <i> getopt_long:
Some older implementations do not protect against internal buffer overflows
(<a href="https://cwe.mitre.org/data/definitions/120.html">CWE-120</a>, <a
href="https://cwe.mitre.org/data/definitions/20.html">CWE-20</a>). Check
(<a href="http://cwe.mitre.org/data/definitions/120.html">CWE-120</a>, <a
href="http://cwe.mitre.org/data/definitions/20.html">CWE-20</a>). Check
implementation on installation, or limit the size of all string inputs. </i>
<pre>
while ((optc = getopt_long (argc, argv, "a",longopts, NULL )) != EOF) {
</pre>
<li>test.c:16: <b> [2] </b> (buffer) <i> strcpy:
Does not check for buffer overflows when copying to destination [MS-banned]
(<a href="https://cwe.mitre.org/data/definitions/120.html">CWE-120</a>).
Consider using snprintf, strcpy_s, or strlcpy (warning: strncpy easily
Does not check for buffer overflows when copying to destination (<a
href="http://cwe.mitre.org/data/definitions/120.html">CWE-120</a>).
Consider using strcpy_s, strncpy, or strlcpy (warning, strncpy is easily
misused). Risk is low because the source is a constant string. </i>
<pre>
strcpy(a, gettext("Hello there")); // Did this work?
</pre>
<li>test.c:19: <b> [2] </b> (buffer) <i> sprintf:
Does not check for buffer overflows (<a
href="https://cwe.mitre.org/data/definitions/120.html">CWE-120</a>). Use
href="http://cwe.mitre.org/data/definitions/120.html">CWE-120</a>). Use
sprintf_s, snprintf, or vsnprintf. Risk is low because the source has a
constant maximum length. </i>
<pre>
@ -207,72 +196,56 @@ Examining test2.c <br>
</pre>
<li>test.c:45: <b> [2] </b> (buffer) <i> char:
Statically-sized arrays can be improperly restricted, leading to potential
overflows or other issues (<a
href="https://cwe.mitre.org/data/definitions/119.html">CWE-119</a>!/<a
href="https://cwe.mitre.org/data/definitions/120.html">CWE-120</a>).
Perform bounds checking, use functions that limit length, or ensure that
the size is larger than the maximum possible length. </i>
overflows or other issues (CWE-119:<a
href="http://cwe.mitre.org/data/definitions/120.html">CWE-120</a>). Perform
bounds checking, use functions that limit length, or ensure that the size
is larger than the maximum possible length. </i>
<pre>
char d[20];
</pre>
<li>test.c:46: <b> [2] </b> (buffer) <i> char:
Statically-sized arrays can be improperly restricted, leading to potential
overflows or other issues (<a
href="https://cwe.mitre.org/data/definitions/119.html">CWE-119</a>!/<a
href="https://cwe.mitre.org/data/definitions/120.html">CWE-120</a>).
Perform bounds checking, use functions that limit length, or ensure that
the size is larger than the maximum possible length. </i>
overflows or other issues (CWE-119:<a
href="http://cwe.mitre.org/data/definitions/120.html">CWE-120</a>). Perform
bounds checking, use functions that limit length, or ensure that the size
is larger than the maximum possible length. </i>
<pre>
char s[20];
</pre>
<li>test.c:50: <b> [2] </b> (buffer) <i> memcpy:
Does not check for buffer overflows when copying to destination (<a
href="https://cwe.mitre.org/data/definitions/120.html">CWE-120</a>). Make
href="http://cwe.mitre.org/data/definitions/120.html">CWE-120</a>). Make
sure destination can always hold the source data. </i>
<pre>
memcpy(d,s); // fail - no size
memcpy(d,s);
</pre>
<li>test.c:53: <b> [2] </b> (buffer) <i> memcpy:
<li>test.c:51: <b> [2] </b> (buffer) <i> CopyMemory:
Does not check for buffer overflows when copying to destination (<a
href="https://cwe.mitre.org/data/definitions/120.html">CWE-120</a>). Make
sure destination can always hold the source data. </i>
<pre>
memcpy(&amp;n,s,sizeof(s)); // fail - sizeof not of destination
</pre>
<li>test.c:54: <b> [2] </b> (buffer) <i> memcpy:
Does not check for buffer overflows when copying to destination (<a
href="https://cwe.mitre.org/data/definitions/120.html">CWE-120</a>). Make
sure destination can always hold the source data. </i>
<pre>
memcpy(d,s,n); // fail - size unguessable
</pre>
<li>test.c:55: <b> [2] </b> (buffer) <i> CopyMemory:
Does not check for buffer overflows when copying to destination (<a
href="https://cwe.mitre.org/data/definitions/120.html">CWE-120</a>). Make
href="http://cwe.mitre.org/data/definitions/120.html">CWE-120</a>). Make
sure destination can always hold the source data. </i>
<pre>
CopyMemory(d,s);
</pre>
<li>test.c:105: <b> [2] </b> (misc) <i> fopen:
<li>test.c:97: <b> [2] </b> (misc) <i> fopen:
Check when opening files - can an attacker redirect it (via symlinks),
force the opening of special file type (e.g., device files), move things
around to create a race condition, control its ancestors, or change its
contents? (<a
href="https://cwe.mitre.org/data/definitions/362.html">CWE-362</a>). </i>
href="http://cwe.mitre.org/data/definitions/362.html">CWE-362</a>). </i>
<pre>
f = fopen("/etc/passwd", "r");
</pre>
<li>test.c:15: <b> [1] </b> (buffer) <i> strcpy:
Does not check for buffer overflows when copying to destination [MS-banned]
(<a href="https://cwe.mitre.org/data/definitions/120.html">CWE-120</a>).
Consider using snprintf, strcpy_s, or strlcpy (warning: strncpy easily
Does not check for buffer overflows when copying to destination (<a
href="http://cwe.mitre.org/data/definitions/120.html">CWE-120</a>).
Consider using strcpy_s, strncpy, or strlcpy (warning, strncpy is easily
misused). Risk is low because the source is a constant character. </i>
<pre>
strcpy(a, "\n"); // Did this work?
</pre>
<li>test.c:18: <b> [1] </b> (buffer) <i> sprintf:
Does not check for buffer overflows (<a
href="https://cwe.mitre.org/data/definitions/120.html">CWE-120</a>). Use
href="http://cwe.mitre.org/data/definitions/120.html">CWE-120</a>). Use
sprintf_s, snprintf, or vsnprintf. Risk is low because the source is a
constant character. </i>
<pre>
@ -280,79 +253,75 @@ Examining test2.c <br>
</pre>
<li>test.c:26: <b> [1] </b> (buffer) <i> scanf:
It's unclear if the %s limit in the format string is small enough (<a
href="https://cwe.mitre.org/data/definitions/120.html">CWE-120</a>). Check
href="http://cwe.mitre.org/data/definitions/120.html">CWE-120</a>). Check
that the limit is sufficiently small, or use a different input function. </i>
<pre>
scanf("%10s", s);
</pre>
<li>test.c:57: <b> [1] </b> (buffer) <i> strncpy:
<li>test.c:53: <b> [1] </b> (buffer) <i> strncpy:
Easily used incorrectly; doesn't always \0-terminate or check for invalid
pointers [MS-banned] (<a
href="https://cwe.mitre.org/data/definitions/120.html">CWE-120</a>). </i>
pointers (<a
href="http://cwe.mitre.org/data/definitions/120.html">CWE-120</a>). </i>
<pre>
strncpy(d,s);
</pre>
<li>test.c:58: <b> [1] </b> (buffer) <i> _tcsncpy:
<li>test.c:54: <b> [1] </b> (buffer) <i> _tcsncpy:
Easily used incorrectly; doesn't always \0-terminate or check for invalid
pointers [MS-banned] (<a
href="https://cwe.mitre.org/data/definitions/120.html">CWE-120</a>). </i>
pointers (<a
href="http://cwe.mitre.org/data/definitions/120.html">CWE-120</a>). </i>
<pre>
_tcsncpy(d,s);
</pre>
<li>test.c:59: <b> [1] </b> (buffer) <i> strncat:
<li>test.c:55: <b> [1] </b> (buffer) <i> strncat:
Easily used incorrectly (e.g., incorrectly computing the correct maximum
size to add) [MS-banned] (<a
href="https://cwe.mitre.org/data/definitions/120.html">CWE-120</a>).
Consider strcat_s, strlcat, snprintf, or automatically resizing strings. </i>
size to add) (<a
href="http://cwe.mitre.org/data/definitions/120.html">CWE-120</a>).
Consider strcat_s, strlcat, or automatically resizing strings. </i>
<pre>
strncat(d,s,10);
</pre>
<li>test.c:62: <b> [1] </b> (buffer) <i> strlen:
<li>test.c:58: <b> [1] </b> (buffer) <i> strlen:
Does not handle strings that are not \0-terminated; if given one it may
perform an over-read (it could cause a crash if unprotected) (<a
href="https://cwe.mitre.org/data/definitions/126.html">CWE-126</a>). </i>
href="http://cwe.mitre.org/data/definitions/126.html">CWE-126</a>). </i>
<pre>
n = strlen(d);
</pre>
<li>test.c:68: <b> [1] </b> (buffer) <i> MultiByteToWideChar:
<li>test.c:64: <b> [1] </b> (buffer) <i> MultiByteToWideChar:
Requires maximum length in CHARACTERS, not bytes (<a
href="https://cwe.mitre.org/data/definitions/120.html">CWE-120</a>). Risk
is very low, the length appears to be in characters not bytes. </i>
href="http://cwe.mitre.org/data/definitions/120.html">CWE-120</a>). Risk is
very low, the length appears to be in characters not bytes. </i>
<pre>
MultiByteToWideChar(CP_ACP,0,szName,-1,wszUserName,sizeof(wszUserName)/sizeof(wszUserName[0]));
</pre>
<li>test.c:70: <b> [1] </b> (buffer) <i> MultiByteToWideChar:
<li>test.c:66: <b> [1] </b> (buffer) <i> MultiByteToWideChar:
Requires maximum length in CHARACTERS, not bytes (<a
href="https://cwe.mitre.org/data/definitions/120.html">CWE-120</a>). Risk
is very low, the length appears to be in characters not bytes. </i>
href="http://cwe.mitre.org/data/definitions/120.html">CWE-120</a>). Risk is
very low, the length appears to be in characters not bytes. </i>
<pre>
MultiByteToWideChar(CP_ACP,0,szName,-1,wszUserName,sizeof wszUserName /sizeof(wszUserName[0]));
</pre>
</ul>
<h2>Analysis Summary</h2>
<p>
Hits = 39
Hits = 36
<br>
Lines analyzed = 126
Lines analyzed = 118
<br>
Physical Source Lines of Code (SLOC) = 86
Physical Source Lines of Code (SLOC) = 80
<br>
Hits@level = [0] 16 [1] 9 [2] 9 [3] 4 [4] 10 [5] 7 <br>
Hits@level+ = [0+] 55 [1+] 39 [2+] 30 [3+] 21 [4+] 17 [5+] 7 <br>
Hits/KSLOC@level+ = [0+] 639.535 [1+] 453.488 [2+] 348.837 [3+] 244.186 [4+] 197.674 [5+] 81.3953 <br>
Hits@level = [0] 0 [1] 9 [2] 7 [3] 3 [4] 10 [5] 7 <br>
Hits@level+ = [0+] 36 [1+] 36 [2+] 27 [3+] 20 [4+] 17 [5+] 7 <br>
Hits/KSLOC@level+ = [0+] 450 [1+] 450 [2+] 337.5 [3+] 250 [4+] 212.5 [5+] 87.5 <br>
Suppressed hits = 2 (use --neverignore to show them)
<br>
Minimum risk level = 1
<br>
Not every hit is necessarily a security vulnerability.
You can inhibit a report by adding a comment in this form:
// flawfinder: ignore
Make *sure* it's a false positive!
You can use the option --neverignore to show these.
<br>
There may be other security vulnerabilities; review your code!
<br>
See '<a href="https://dwheeler.com/secure-programs">Secure Programming HOWTO</a>'
(<a href="https://dwheeler.com/secure-programs">https://dwheeler.com/secure-programs</a>) for more information.
See '<a href="http://www.dwheeler.com/secure-programs">Secure Programming for Linux and Unix HOWTO</a>'
(<a href="http://www.dwheeler.com/secure-programs">http://www.dwheeler.com/secure-programs</a>) for more information.
</body>
</html>

123
test/correct-results.txt → correct-results.txt
View File

@ -1,5 +1,5 @@
Flawfinder version 2.0.19, (C) 2001-2019 David A. Wheeler.
Number of rules (primarily dangerous function names) in C/C++ ruleset: 222
Flawfinder version 1.31, (C) 2001-2014 David A. Wheeler.
Number of rules (primarily dangerous function names) in C/C++ ruleset: 169
Examining test.c
Examining test2.c
@ -7,35 +7,34 @@ FINAL RESULTS:
test.c:32: [5] (buffer) gets:
Does not check for buffer overflows (CWE-120, CWE-20). Use fgets() instead.
test.c:60: [5] (buffer) strncat:
test.c:56: [5] (buffer) strncat:
Easily used incorrectly (e.g., incorrectly computing the correct maximum
size to add) [MS-banned] (CWE-120). Consider strcat_s, strlcat, snprintf,
or automatically resizing strings. Risk is high; the length parameter
appears to be a constant, instead of computing the number of characters
left.
test.c:61: [5] (buffer) _tcsncat:
size to add) (CWE-120). Consider strcat_s, strlcat, or automatically
resizing strings. Risk is high; the length parameter appears to be a
constant, instead of computing the number of characters left.
test.c:57: [5] (buffer) _tcsncat:
Easily used incorrectly (e.g., incorrectly computing the correct maximum
size to add) [MS-banned] (CWE-120). Consider strcat_s, strlcat, or
automatically resizing strings. Risk is high; the length parameter appears
to be a constant, instead of computing the number of characters left.
test.c:64: [5] (buffer) MultiByteToWideChar:
size to add) (CWE-120). Consider strcat_s, strlcat, or automatically
resizing strings. Risk is high; the length parameter appears to be a
constant, instead of computing the number of characters left.
test.c:60: [5] (buffer) MultiByteToWideChar:
Requires maximum length in CHARACTERS, not bytes (CWE-120). Risk is high,
it appears that the size is given as bytes, but the function requires size
as characters.
test.c:66: [5] (buffer) MultiByteToWideChar:
test.c:62: [5] (buffer) MultiByteToWideChar:
Requires maximum length in CHARACTERS, not bytes (CWE-120). Risk is high,
it appears that the size is given as bytes, but the function requires size
as characters.
test.c:77: [5] (misc) SetSecurityDescriptorDacl:
test.c:73: [5] (misc) SetSecurityDescriptorDacl:
Never create NULL ACLs; an attacker can set it to Everyone (Deny All
Access), which would even forbid administrator access (CWE-732).
test.c:77: [5] (misc) SetSecurityDescriptorDacl:
test.c:73: [5] (misc) SetSecurityDescriptorDacl:
Never create NULL ACLs; an attacker can set it to Everyone (Deny All
Access), which would even forbid administrator access (CWE-732).
test.c:17: [4] (buffer) strcpy:
Does not check for buffer overflows when copying to destination [MS-banned]
(CWE-120). Consider using snprintf, strcpy_s, or strlcpy (warning: strncpy
easily misused).
Does not check for buffer overflows when copying to destination (CWE-120).
Consider using strcpy_s, strncpy, or strlcpy (warning, strncpy is easily
misused).
test.c:20: [4] (buffer) sprintf:
Does not check for buffer overflows (CWE-120). Use sprintf_s, snprintf, or
vsnprintf.
@ -59,68 +58,58 @@ test.c:38: [4] (format) syslog:
If syslog's format strings can be influenced by an attacker, they can be
exploited (CWE-134). Use a constant format string for syslog.
test.c:49: [4] (buffer) _mbscpy:
Does not check for buffer overflows when copying to destination [MS-banned]
(CWE-120). Consider using a function version that stops copying at the end
of the buffer.
test.c:56: [4] (buffer) lstrcat:
Does not check for buffer overflows when copying to destination (CWE-120).
Consider using a function version that stops copying at the end of the
buffer.
test.c:52: [4] (buffer) lstrcat:
Does not check for buffer overflows when concatenating to destination
[MS-banned] (CWE-120).
test.c:79: [3] (shell) CreateProcess:
(CWE-120).
test.c:75: [3] (shell) CreateProcess:
This causes a new process to execute and is difficult to use safely
(CWE-78). Specify the application path in the first argument, NOT as part
of the second, or embedded spaces could allow an attacker to force a
different program to run.
test.c:79: [3] (shell) CreateProcess:
test.c:75: [3] (shell) CreateProcess:
This causes a new process to execute and is difficult to use safely
(CWE-78). Specify the application path in the first argument, NOT as part
of the second, or embedded spaces could allow an attacker to force a
different program to run.
test.c:81: [3] (misc) LoadLibraryEx:
Ensure that the full path to the library is specified, or current directory
may be used (CWE-829, CWE-20). Use a flag like LOAD_LIBRARY_SEARCH_SYSTEM32
or LOAD_LIBRARY_SEARCH_APPLICATION_DIR to search only desired folders.
test.c:99: [3] (buffer) getopt_long:
test.c:91: [3] (buffer) getopt_long:
Some older implementations do not protect against internal buffer overflows
(CWE-120, CWE-20). Check implementation on installation, or limit the size
of all string inputs.
test.c:16: [2] (buffer) strcpy:
Does not check for buffer overflows when copying to destination [MS-banned]
(CWE-120). Consider using snprintf, strcpy_s, or strlcpy (warning: strncpy
easily misused). Risk is low because the source is a constant string.
Does not check for buffer overflows when copying to destination (CWE-120).
Consider using strcpy_s, strncpy, or strlcpy (warning, strncpy is easily
misused). Risk is low because the source is a constant string.
test.c:19: [2] (buffer) sprintf:
Does not check for buffer overflows (CWE-120). Use sprintf_s, snprintf, or
vsnprintf. Risk is low because the source has a constant maximum length.
test.c:45: [2] (buffer) char:
Statically-sized arrays can be improperly restricted, leading to potential
overflows or other issues (CWE-119!/CWE-120). Perform bounds checking, use
overflows or other issues (CWE-119:CWE-120). Perform bounds checking, use
functions that limit length, or ensure that the size is larger than the
maximum possible length.
test.c:46: [2] (buffer) char:
Statically-sized arrays can be improperly restricted, leading to potential
overflows or other issues (CWE-119!/CWE-120). Perform bounds checking, use
overflows or other issues (CWE-119:CWE-120). Perform bounds checking, use
functions that limit length, or ensure that the size is larger than the
maximum possible length.
test.c:50: [2] (buffer) memcpy:
Does not check for buffer overflows when copying to destination (CWE-120).
Make sure destination can always hold the source data.
test.c:53: [2] (buffer) memcpy:
test.c:51: [2] (buffer) CopyMemory:
Does not check for buffer overflows when copying to destination (CWE-120).
Make sure destination can always hold the source data.
test.c:54: [2] (buffer) memcpy:
Does not check for buffer overflows when copying to destination (CWE-120).
Make sure destination can always hold the source data.
test.c:55: [2] (buffer) CopyMemory:
Does not check for buffer overflows when copying to destination (CWE-120).
Make sure destination can always hold the source data.
test.c:105: [2] (misc) fopen:
test.c:97: [2] (misc) fopen:
Check when opening files - can an attacker redirect it (via symlinks),
force the opening of special file type (e.g., device files), move things
around to create a race condition, control its ancestors, or change its
contents? (CWE-362).
test.c:15: [1] (buffer) strcpy:
Does not check for buffer overflows when copying to destination [MS-banned]
(CWE-120). Consider using snprintf, strcpy_s, or strlcpy (warning: strncpy
easily misused). Risk is low because the source is a constant character.
Does not check for buffer overflows when copying to destination (CWE-120).
Consider using strcpy_s, strncpy, or strlcpy (warning, strncpy is easily
misused). Risk is low because the source is a constant character.
test.c:18: [1] (buffer) sprintf:
Does not check for buffer overflows (CWE-120). Use sprintf_s, snprintf, or
vsnprintf. Risk is low because the source is a constant character.
@ -128,46 +117,40 @@ test.c:26: [1] (buffer) scanf:
It's unclear if the %s limit in the format string is small enough
(CWE-120). Check that the limit is sufficiently small, or use a different
input function.
test.c:57: [1] (buffer) strncpy:
test.c:53: [1] (buffer) strncpy:
Easily used incorrectly; doesn't always \0-terminate or check for invalid
pointers [MS-banned] (CWE-120).
test.c:58: [1] (buffer) _tcsncpy:
pointers (CWE-120).
test.c:54: [1] (buffer) _tcsncpy:
Easily used incorrectly; doesn't always \0-terminate or check for invalid
pointers [MS-banned] (CWE-120).
test.c:59: [1] (buffer) strncat:
pointers (CWE-120).
test.c:55: [1] (buffer) strncat:
Easily used incorrectly (e.g., incorrectly computing the correct maximum
size to add) [MS-banned] (CWE-120). Consider strcat_s, strlcat, snprintf,
or automatically resizing strings.
test.c:62: [1] (buffer) strlen:
size to add) (CWE-120). Consider strcat_s, strlcat, or automatically
resizing strings.
test.c:58: [1] (buffer) strlen:
Does not handle strings that are not \0-terminated; if given one it may
perform an over-read (it could cause a crash if unprotected) (CWE-126).
test.c:68: [1] (buffer) MultiByteToWideChar:
test.c:64: [1] (buffer) MultiByteToWideChar:
Requires maximum length in CHARACTERS, not bytes (CWE-120). Risk is very
low, the length appears to be in characters not bytes.
test.c:70: [1] (buffer) MultiByteToWideChar:
test.c:66: [1] (buffer) MultiByteToWideChar:
Requires maximum length in CHARACTERS, not bytes (CWE-120). Risk is very
low, the length appears to be in characters not bytes.
ANALYSIS SUMMARY:
Hits = 39
Lines analyzed = 126
Physical Source Lines of Code (SLOC) = 86
Hits@level = [0] 16 [1] 9 [2] 9 [3] 4 [4] 10 [5] 7
Hits@level+ = [0+] 55 [1+] 39 [2+] 30 [3+] 21 [4+] 17 [5+] 7
Hits/KSLOC@level+ = [0+] 639.535 [1+] 453.488 [2+] 348.837 [3+] 244.186 [4+] 197.674 [5+] 81.3953
Hits = 36
Lines analyzed = 118
Physical Source Lines of Code (SLOC) = 80
Hits@level = [0] 0 [1] 9 [2] 7 [3] 3 [4] 10 [5] 7
Hits@level+ = [0+] 36 [1+] 36 [2+] 27 [3+] 20 [4+] 17 [5+] 7
Hits/KSLOC@level+ = [0+] 450 [1+] 450 [2+] 337.5 [3+] 250 [4+] 212.5 [5+] 87.5
Suppressed hits = 2 (use --neverignore to show them)
Minimum risk level = 1
Not every hit is necessarily a security vulnerability.
You can inhibit a report by adding a comment in this form:
// flawfinder: ignore
Make *sure* it's a false positive!
You can use the option --neverignore to show these.
There may be other security vulnerabilities; review your code!
See 'Secure Programming HOWTO'
(https://dwheeler.com/secure-programs) for more information.
See 'Secure Programming for Linux and Unix HOWTO'
(http://www.dwheeler.com/secure-programs) for more information.
Testing for no ending newline:
Lines analyzed = 32

11
entrypoint.sh
View File

@ -1,11 +0,0 @@
#!/bin/sh -l
# $1 whitespace-separated arguments. Some filenames may need to be escaped.
# $2 output filename
output="${2:-flawfinder-output.txt}"
flawfinder $1 > "$output"
result="$?"
cat "$output"
exit "$result"

1973
flawfinder Executable file
View File

File diff suppressed because it is too large Load Diff

334
flawfinder.1
View File

@ -1,5 +1,5 @@
'\"
.\" (C) Copyright 2001-2018 David A. Wheeler (dwheeler@dwheeler.com)
.\" (C) Copyright 2001-2014 David A. Wheeler (dwheeler@dwheeler.com)
.\"
.\" This program is free software; you can redistribute it and/or modify
.\" it under the terms of the GNU General Public License as published by
@ -19,7 +19,7 @@
.\"
.\" Man page created 17 May 2001 by David A. Wheeler (dwheeler@dwheeler.com)
.\"
.TH FLAWFINDER 1 "3 Jun 2021" "Flawfinder" "Flawfinder"
.TH FLAWFINDER 1 "3 Aug 2014" "Flawfinder" "Flawfinder"
.SH NAME
flawfinder \- lexically find potential security flaws ("hits") in source code
.SH SYNOPSIS
@ -48,15 +48,12 @@ flawfinder \- lexically find potential security flaws ("hits") in source code
.\" Selecting Output Format:
.RB [ \-\-context | \-c ]
.RB [ \-\-columns | \-C ]
.RB [ \-\-csv ]
.RB [ \-\-dataonly | \-D ]
.RB [ \-\-html | \-H ]
.RB [ \-\-immediate | -i ]
.RB [ \-\-sarif ]
.RB [ \-\-singleline | \-S ]
.RB [ \-\-omittime ]
.RB [ \-\-quiet | \-Q ]
.RB [ \-\-error-level=\fRLEVEL\fR ]
.br
.\" Managing hit list.
[\fB\-\-loadhitlist=\fR\fIF\fR]
@ -75,15 +72,12 @@ in that directory (and its subdirectories, recursively) will be examined.
Thus, for most projects, simply give flawfinder the name of the source
code's topmost directory (use ``.'' for the current directory),
and flawfinder will examine all of the project's C/C++ source code.
Flawfinder does \fInot\fR require that you be able to build your software,
so it can be used even with incomplete source code.
If you only want to have \fIchanges\fR reviewed, save a unified diff
of those changes (created by GNU "diff -u" or "svn diff" or "git diff")
in a patch file and use the \-\-patch (\-P) option.
.PP
Flawfinder will produce a list of ``hits'' (potential
security flaws, also called findings),
sorted by risk; the riskiest hits are shown first.
security flaws), sorted by risk; the riskiest hits are shown first.
The risk level is shown inside square brackets and
varies from 0, very little risk, to 5, great risk.
This risk level depends not only on the function, but on the values of the
@ -98,7 +92,7 @@ the number of false hits in internationalized programs.
Flawfinder will do the same sort of thing with _T() and _TEXT(),
common Microsoft macros for handling internationalized programs.
.\" For more info, see: http://www.rpi.edu/~pudeyo/articles/unicode.html
Flawfinder correctly ignores text inside comments and strings.
Flawfinder correctly ignores most text inside comments and strings.
Normally flawfinder shows all hits with a risk level of at least 1,
but you can use the \-\-minlevel option
to show only hits with higher risk levels if you wish.
@ -106,12 +100,8 @@ Hit descriptions also note the relevant
Common Weakness Enumeration (CWE) identifier(s) in parentheses,
as discussed below.
Flawfinder is officially CWE-Compatible.
Hit descriptions with "[MS-banned]" indicate functions that are in the
banned list of functions released by Microsoft; see
http://msdn.microsoft.com/en-us/library/bb288454.aspx
for more information about banned functions.
.PP
Not every hit (aka finding) is actually a security vulnerability,
Not every hit is actually a security vulnerability,
and not every security vulnerability is necessarily found.
Nevertheless, flawfinder can be an aid in finding and removing
security vulnerabilities.
@ -190,7 +180,7 @@ While searching directories recursively, flawfinder only opens and
examines regular files that have C/C++ filename extensions.
Flawfinder presumes that files are C/C++ files if they have the extensions
".c", ".h", ".ec", ".ecp", ".pgc", ".C", ".cpp",
".CPP", ".cxx", ".c++", ".cc", ".CC", ".pcc", ".hpp", or ".H".
".CPP", ".cxx", ".cc", ".CC", ".pcc", ".hpp", or ".H".
The filename ``\-'' means the standard input.
To prevent security problems,
special files (such as device special files and named pipes) are
@ -220,12 +210,6 @@ is displayed; by default this is 1 (use \-\-minlevel to change this).
The summary ends with important reminders:
Not every hit is necessarily a security vulnerability, and
there may be other security vulnerabilities not reported by the tool.
.PP
Flawfinder can easily integrate into a continuous integration system.
You might want to check out the \-\-error\-level option to help do that, e.g.,
using \-\-error\-level=4 will cause an error to be returned if flawfinder
finds a vulnerability of level 4 or higher.
.PP
Flawfinder is released under the GNU GPL license version 2 or later (GPLv2+).
.PP
@ -233,7 +217,6 @@ Flawfinder works similarly to another program, ITS4, which is not
fully open source software (as defined in the Open Source Definition)
nor free software (as defined by the Free Software Foundation).
The author of Flawfinder has never seen ITS4's source code.
Flawfinder is similar in many ways to RATS, if you are familiar with RATS.
.SH "BRIEF TUTORIAL"
@ -270,11 +253,11 @@ If you're getting an extraordinary number of false positives because
variable names look like dangerous function names, use the \-F option
to remove reports about them.
If you don't understand the error message, please see documents such as the
.UR "https://dwheeler.com/secure-programs"
.I "Secure Programming HOWTO"
.UR "http://www.dwheeler.com/secure-programs"
.I "Writing Secure Programs for Linux and Unix HOWTO"
.UE
at
https://dwheeler.com/secure-programs
http://www.dwheeler.com/secure-programs
which provides more information on writing secure programs.
.PP
Once you identify the problem and understand it, you can fix it.
@ -304,8 +287,7 @@ integrated development environments; see the examples for
more information.
.PP
Flawfinder includes many other options, including ones to
create HTML versions of the output (useful for prettier displays) and
OASIS Static Analysis Results Interchange Format (SARIF) output.
create HTML versions of the output (useful for prettier displays).
The next section describes those options in more detail.
.SH OPTIONS
@ -316,19 +298,18 @@ select input data,
select which hits to display,
select the output format,
and perform hitlist management.
The commonly-used flawfinder options
support the standard option syntax defined in the
Flawfinder supports the standard syntax defined in the
POSIX (Issue 7, 2013 Edition) section ``Utility Conventions''.
Flawfinder also supports the GNU long options
It also supports the GNU long options
(double-dash options of form \-\-\fIoption\fR)
as defined in the \fIGNU C Library Reference Manual\fR
``Program Argument Syntax Conventions''
and \fIGNU Coding Standards\fR ``Standards for Command Line Interfaces''.
Long option arguments can be provided as ``--name=value'' or ``-name value''.
All options can be accessed using the more
Some options can only be accessed using the more
readable GNU long option conventions;
some less commonly used options can \fIonly\fR be accessed
using long option conventions.
common options are also supported
by the older single-letter option convention.
.SS "Documentation"
@ -382,8 +363,7 @@ Note that for flawfinder version 1.01 and before, this was the default.
Enter directories whose names begin with ".".
Normally such directories are ignored, since they normally
include version control private data (such as .git/ or .svn/),
build metadata (such as .makepp),
configuration information, and so on.
configurations, and so on.
.TP
.BI \-\-nolink
@ -396,7 +376,7 @@ this behavior is now the default.
.TP
\fB\-P\fR \fIpatchfile\fR
Examine the selected files or directories, but only report hits in lines
that are added or modified as described in the given patch file.
that are added or modified by the given patch file.
The patch file must be in a recognized unified diff format
(e.g., the output of GNU "diff -u old new", "svn diff", or "git diff [commit]").
Flawfinder assumes that the patch has already been applied to the files.
@ -415,16 +395,6 @@ if you have a different format, again regenerate it first.
Only hits that occur on resultant changed lines, or immediately
above and below them, are reported.
This option implies \-\-neverignore.
\fBWarning\fR: Do \fInot\fR pass a patch file without the
\fB\-P\fR, because flawfinder will then try to treat the file as a
source file.
This will often work, but the line numbers will be relative
to the beginning of the patch file, not the positions in the
source code.
Note that you \fBmust\fR also provide the actual files to analyze,
and not just the patch file; when using \fB\-P\fR files are only reported
if they are both listed in the patch and also listed (directly or indirectly)
in the list of files to analyze.
.SS "Selecting Hits to Display"
@ -496,30 +466,6 @@ false positives).
Show context, i.e., the line having the "hit"/potential flaw.
By default the line is shown immediately after the warning.
.TP
.BI \-\-csv
Generate output in comma-separated-value (CSV) format.
This is the recommended format for sending to other tools for processing.
It will always generate a header row, followed by 0 or more data rows
(one data row for each hit).
Selecting this option automatically enables \-\-quiet and
\-\-dataonly.
The headers are mostly self-explanatory.
"File" is the filename, "Line" is the line number,
"Column" is the column (starting from 1),
"Level" is the risk level (0-5, 5 is riskiest),
"Category" is the general flawfinder category,
"Name" is the name of the triggering rule,
"Warning" is text explaining why it is a hit (finding),
"Suggestion" is text suggesting how it might be fixed,
"Note" is other explanatory notes,
"CWEs" is the list of one or more CWEs,
"Context" is the source code line triggering the hit,
and "Fingerprint" is the SHA-256 hash of the context once
its leading and trailing whitespace have been removed
(the fingerprint may help detect and eliminate later duplications).
If you use Python3, the hash is of the context when encoded as UTF-8.
.TP
.BI "\-\-dataonly"
.TP
@ -539,35 +485,6 @@ Format the output as HTML instead of as simple text.
.BI -i
Immediately display hits (don't just wait until the end).
.TP
.BI \-\-sarif
Produce output in the OASIS
Static Analysis Results Interchange Format (SARIF) format (a JSON-based format).
The goals of the SARIF format, as explained in
version 2.1.0 (27 March 2020) of its specification, include being able to
"comprehensively capture the range of data produced by commonly
used static analysis tools."
SARIF output identifies the tool name as "Flawfinder".
The flawfinder levels 0 through 5 are mapped to SARIF rank (by dividing by 5),
SARIF level, and the default viewer action as follows:
Flawfinder 0: SARIF rank 0.0, SARIF level note, Does not display by default
Flawfinder 1: SARIF rank 0.2, SARIF level note, Does not display by default
Flawfinder 2: SARIF rank 0.4, SARIF level note, Does not display by default
Flawfinder 3: SARIF rank 0.6, SARIF level warning, Displays by default, does not break build / other processes
Flawfinder 4: SARIF rank 0.8, SARIF level error, Displays by default, breaks build/ other processes
Flawfinder 5: SARIF rank 1.0, SARIF level error, Displays by default, breaks build/ other processes
A big thanks to Yong Yan implementing SARIF output generation for flawfinder!
For more about the SARIF format, see:
https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=sarif
.TP
.BI "\-\-singleline"
.TP
@ -588,15 +505,6 @@ the output doesn't vary depending on how long the analysis takes.
Don't display status information (i.e., which files are being examined)
while the analysis is going on.
.TP
.BI "\-\-error-level=LEVEL"
Return a nonzero (false) error code if there is at least one
hit of LEVEL or higher. If a diffhitlist is provided,
hits noted in it are ignored.
This option can be useful within a continuous integration script,
especially if you mark known-okay lines as "flawfinder: ignore".
Usually you want level to be fairly high, such as 4 or 5.
By default, flawfinder returns 0 (true) on a successful run.
.SS "Hitlist Management"
@ -611,12 +519,6 @@ Save all resulting hits (the "hitlist") to F.
Load the hitlist from F instead of analyzing source programs.
Warning: Do \fInot\fR load hitlists from untrusted sources
(for security reasons).
These are internally implemented using Python's "pickle" facility,
which trusts the input.
Note that stored hitlists often cannot be read when using an older version
of Python, in particular, if savehitlist was used but
flawfinder was run using Python 3,
the hitlist can't be loaded by running flawfinder with Python 2.
.TP
\fB\-\-diffhitlist=\fR\fIF\fR
@ -632,91 +534,6 @@ The difference algorithm is conservative;
hits are only considered the ``same'' if they have the same
filename, line number, column position, function name, and risk level.
.SS "Character Encoding Errors"
Flawfinder uses the character encoding rules set by Python.
Sometimes source code does not perfectly follow some encoding rules.
If you run flawfinder with Python 2
these non-conformities often do not impact processing in practice.
However, if you run flawfinder with Python 3, this can be a problem.
Python 3 developers wants the world to always use encodings perfectly correctly,
everywhere, and in general wants everyone to only use UTF-8.
UTF-8 is a great encoding, and it is very popular, but
the world often doesn't care what the Python 3 developers want.
When running flawfinder using Python 3, the program will crash hard if
\fIany\fR source file has \fIany\fR non-conforming text.
It will do this even if the non-conforming text is in comments or strings
(where it often doesn't matter).
Python 3 fails to provide useful built-ins to deal with
the messiness of the real world, so it's
non-trivial to deal with this problem without depending on external
libraries (which we're trying to avoid).
A symptom of this problem
is if you run flawfinder and you see an error message like this:
\fIError: encoding error in ,1.c\fR
\fI'utf-8' codec can't decode byte 0xff in position 45: invalid start byte\fR
What you are seeing is the result of an internal UnicodeDecodeError.
If this happens to you, there are several options:
Option #1 (special case):
if your system normally uses an encoding other than UTF-8,
is properly set up to use that encoding (using LC_ALL and maybe LC_CTYPE),
and the input files are in that non-UTF-8 encoding,
it may be that Python3 is (incorrectly) ignoring your configuration.
In that case, simply tell Python3 to use your
configuration by setting the environment variable PYTHONUTF8=0, e.g.,
run flawfinder as:
"PYTHONUTF8=0 python3 flawfinder ...".
Option #2 (special case): If you know what the encoding of the files is,
you can force use of that encoding. E.g., if the encoding
is BLAH, run flawfinder as:
"PYTHONUTF8=0 LC_ALL=C.BLAH python3 flawfinder ...".
You can replace "C" after LC_ALL= with your real language locale
(e.g., "en_US").
Option #3: If you don't know what the encoding is, or the encoding is
inconsistent (e.g., the common case of UTF-8 files with some
characters encoded using Windows-1252 instead),
then you can force the system to use the
ISO-8859-1 (Latin-1) encoding in which all bytes are allowed.
If the inconsistencies are only in comments and strings, and the
underlying character set is "close enough" to ASCII, this can get you
going in a hurry.
You can do this by running:
"PYTHONUTF8=0 LC_ALL=C.ISO-8859-1 python3 flawfinder ...".
In some cases you may not need the "PYTHONUTF8=0".
You may be able to replace "C" after LC_ALL= with your real language locale
(e.g., "en_US").
Option #4: Convert the encoding of the files to be analyzed so that it's
a single encoding - it's highly recommended to convert to UTF-8.
For example, the system program "iconv"
or the Python program cvt2utf
can be used to convert encodings.
(You can install cvt2utf with "pip install cvtutf").
This works well if some files have one encoding, and some have another,
but they are consistent within a single file.
If the files have encoding errors, you'll have to fix them.
Option #5: Run flawfinder using Python 2 instead of Python 3.
E.g., "python2 flawfinder ...".
To be clear:
I strongly recommend using the UTF-8 encoding for all source code,
and use continuous integration tests to ensure that the source code
is always valid UTF-8.
If you do that, many problems disappear.
But in the real world this is not always the situation.
Hopefully
this information will help you deal with real-world encoding problems.
.SH EXAMPLES
@ -736,14 +553,6 @@ reporting on all hits found.
By default flawfinder will skip symbolic links and
directories with names that start with a period.
.TP
.B "flawfinder \-\-error-level=4 ."
Examine all the C/C++ files in the current directory
and its subdirectories (recursively);
return an error code if there are vulnerabilities
level 4 and up (the two highest risk levels).
This is a plausible way to use flawfinder in a continuous integration system.
.TP
.B "flawfinder \-\-minlevel=4 ."
Examine all the C/C++ files in the current directory
@ -762,21 +571,12 @@ inputs appropriately).
Examine all the C/C++ files in the directory mydir and its subdirectories,
including even the hits marked for ignoring in the code comments.
.TP
.B "flawfinder \-\-csv ."
Examine the current directory down (recursively), and report all
hits in CSV format.
This is the recommended form if you want to further process
flawfinder output using other tools
(such as data correlation tools).
.TP
.B "flawfinder \-QD mydir"
Examine mydir and report only the actual results
(removing the header and footer of the output).
This form may be useful
if the output will be piped into other tools for further analysis,
though CSV format is probably the better choice in that case.
This form is useful
if the output will be piped into other tools for further analysis.
The \-C (\-\-columns) and \-S (\-\-singleline)
options can also be useful if you're piping the data
into other tools.
@ -942,8 +742,8 @@ that can occur in software's architecture, design, code or implementation
that can lead to exploitable security vulnerabilities...
created to serve as a common language for
describing software security weaknesses''
(https://cwe.mitre.org/about/faq.html).
For more information on CWEs, see https://cwe.mitre.org.
(http://cwe.mitre.org/about/faq.html).
For more information on CWEs, see http://cwe.mitre.org.
.PP
Flawfinder supports the CWE and is officially CWE-Compatible.
Hit descriptions typically include a relevant
@ -958,39 +758,20 @@ The HTML report also includes hypertext links to the CWE definitions
hosted at MITRE.
In this way, flawfinder is designed to meet the CWE-Output requirement.
.PP
In some cases there are CWE mapping and usage challenges; here is how
flawfinder handles them.
If the same entry maps to multiple CWEs simultaneously,
all the CWE mappings are listed as separated by commas.
This often occurs with CWE-20, Improper Input Validation;
thus the report "CWE-676, CWE-120" maps to two CWEs.
In addition, flawfinder provides additional information for those who are
are interested in the CWE/SANS top 25 list 2011 (https://cwe.mitre.org/top25/)
when mappings are not directly to them.
Many people will want to search for specific CWEs in this top 25 list,
such as CWE-120 (classic buffer overflow).
The challenge is that some flawfinder hits map
to a more general CWE that would include a top 25 item, while in some
other cases hits map to a more specific vulnerability that is
only a subset of a top 25 item.
To resolve this, in some cases flawfinder will list a sequence of CWEs
in the format "more-general/more-specific", where the CWE actually
being mapped is followed by a "!".
This is always done whenever a flaw is not mapped directly to
a top 25 CWE, but the mapping is related to such a CWE.
So "CWE-119!/CWE-120" means that the vulnerability is mapped
to CWE-119 and that CWE-120 is a subset of CWE-119.
In contrast, "CWE-362/CWE-367!" means that the hit is mapped to
CWE-367, a subset of CWE-362.
Note that this is a subtle syntax change from flawfinder version 1.31;
in flawfinder version 1.31,
the form "more-general:more-specific" meant what is now listed as
"more-general!/more-specific", while
"more-general/more-specific" meant "more-general/more-specific!".
Tools can handle both the version 1.31 and the current format,
if they wish, by noting that the older format did not use "!" at all
(and thus this is easy to distinguish).
These mapping mechanisms simplify searching for certain CWEs.
Many of the CWEs reported by flawfinder
are identified in the CWE/SANS top 25 list 2011 (http://cwe.mitre.org/top25/).
Many people will want to search for CWEs in this list,
such as CWE-120 (classic buffer overflow),
When flawfinder maps to a CWE that is more general than a top 25 item,
it lists it as more-general:more-specific
(e.g., CWE-119:CWE-120), where more-general is the actual mapping.
If flawfinder maps to a more specific CWE item that is a specific
case of a top 25 item,
it is listed in the form top-25/more-specific (e.g., CWE-362/CWE-367),
where the real mapping is the more specific CWE entry.
If the same entry maps to multiple CWEs, the CWEs are separated by commas
(this often occurs with CWE-20, Improper Input Validation).
This simplifies searching for certain CWEs.
.PP
CWE version 2.7 (released June 23, 2014) was used for the mapping.
The current CWE mappings select the most specific CWE the tool can determine.
@ -1030,7 +811,7 @@ CWE-22: Improper Limitation of a Pathname to a Restricted Directory (``Path Trav
CWE-78: Improper Neutralization of Special Elements used in an OS Command (``OS Command Injection'')*
.IP \(bu
CWE-119: Improper Restriction of Operations within the Bounds of a Memory Buffer
(a parent of CWE-120*, so this is shown as CWE-119!/CWE-120)
(a parent of CWE-120*, so this is shown as CWE-119:CWE-120)
.IP \(bu
CWE-120: Buffer Copy without Checking Size of Input (``Classic Buffer Overflow'')*
.IP \(bu
@ -1126,7 +907,7 @@ can be fixed.
However, developers and reviewers must
know how to develop secure software to use this tool, because otherwise,
\fIa fool with a tool is still a fool\fR.
My book at https://dwheeler.com/secure-programs may help.
My book at http://www.dwheeler.com/secure-programs may help.
.PP
This tool should be, at most, a small part of a larger software
development process designed
@ -1155,7 +936,7 @@ Also, do not use the \-\-allowlink option in such cases;
attackers could create malicious symbolic links to files outside of their
source code area (such as /etc/passwd).
.PP
Source code management systems (like GitHub, SourceForge, and Savannah)
Source code management systems (like SourceForge and Savannah)
definitely fall into this category; if you're maintaining one of those
systems, first copy or extract the files into a separate directory
(that can't be controlled by attackers)
@ -1204,13 +985,13 @@ the hitlist; in that context this restriction is not a problem.
.PP
Flawfinder is based on simple text pattern matching, which is
part of its fundamental design and not easily changed.
This design approach leads to a number of fundamental limitations, e.g.,
This design apporach leads to a number of fundamental limitations, e.g.,
a higher false positive rate, and is the underlying cause of
most of the bugs listed here.
On the positive side, flawfinder doesn't get confused by many
complicated preprocessor sequences that other tools sometimes choke on;
flawfinder can often handle code that cannot link, and sometimes
cannot even compile or build.
cannot even build.
.PP
Flawfinder is currently limited to C/C++.
In addition, when analyzing C++ it focuses primarily on the C subset of C++.
@ -1251,29 +1032,6 @@ Such constructs are bad style, and will confuse many other tools too.
If you must analyze such files, rewrite those lines.
Thankfully, these are quite rare.
.PP
Flawfinder reports vulnerabilities regardless of the parameters of
"#if" or "#ifdef".
A construct "#if VALUE" will often have VALUE of 0 in some cases, and
non-zero in others.
Similarly, "#ifdef VALUE" will have VALUE defined in some cases, and
not defined in others.
Flawfinder reports in all cases, which
means that flawfinder has a chance of reporting vulnerabilities in
all alternatives.
This is not a bug, this is intended behavior.
.PP
Flawfinder will report hits even if they are between
a literal "#if 0" and "#endif".
It would be possible to change this particular situation, but directly
using "#if 0" to comment-out code (other than during debugging) indicates
(1) the removal is very temporary (in which case we should still report it) or
(2) very poor code practices.
If you want to permanently get rid of code, then
delete it instead of using "#if 0", since you can always see what it was
using your version control software.
If you don't use version control software, then that's the bug
you need to fix right now.
.PP
Some complex or unusual constructs can mislead flawfinder.
In particular, if a parameter begins with gettext(" and ends with ),
flawfinder will presume that the parameter of gettext is a constant.
@ -1299,15 +1057,13 @@ analyzing programs that aren't application-layer code
The techniques may still be useful; feel free to replace the database
if your situation is significantly different from normal.
.PP
Flawfinder's default output format (filename:linenumber, followed optionally
Flawfinder's output format (filename:linenumber, followed optionally
by a :columnnumber) can be misunderstood if any source files have
very weird filenames.
Filenames embedding a newline/linefeed character will cause odd breaks,
and filenames including colon (:) are likely to be misunderstood.
This is especially important if flawfinder's output is being used
by other tools, such as filters or text editors.
If you are using flawfinder's output in other tools, consider using its
CSV format instead (which can handle this).
If you're looking at new code, examine the files for such characters.
It's incredibly unwise to have such filenames anyway;
many tools can't handle such filenames at all.
@ -1328,7 +1084,7 @@ to report hits, so that they can be examined further, instead of silently
ignoring them.
Thus, flawfinder prefers to have false positives (reports that
turn out to not be problems) rather than false negatives
(failures to report security vulnerabilities).
(failure to report on a security vulnerability).
But this is a generality; flawfinder uses simplistic heuristics and
simply can't get everything "right".
.PP
@ -1341,11 +1097,11 @@ Still, it can serve as a useful aid for humans, helping to identify
useful places to examine further, and that's the point of this simple tool.
.SH "SEE ALSO"
See the flawfinder website at https://dwheeler.com/flawfinder.
See the flawfinder website at http://www.dwheeler.com/flawfinder.
You should also see the
.I "Secure Programming HOWTO"
.I "Secure Programming for Unix and Linux HOWTO"
at
.IR "https://dwheeler.com/secure-programs" .
.IR "http://www.dwheeler.com/secure-programs" .
.SH AUTHOR
David A. Wheeler (dwheeler@dwheeler.com).

2553
flawfinder.py
View File

File diff suppressed because it is too large Load Diff

8
flawfinder.spec
View File

@ -1,11 +1,11 @@
Name: flawfinder
Summary: Examines C/C++ source code for security flaws
Version: 2.0.19
Version: 1.31
Release: 1%{?dist}
License: GPLv2+
Group: Development/Tools
URL: http://dwheeler.com/flawfinder/
Source: http://dwheeler.com/flawfinder/%{name}-%{version}.tar.gz
URL: http://www.dwheeler.com/flawfinder/
Source: http://www.dwheeler.com/flawfinder/%{name}-%{version}.tar.gz
Requires: python
BuildArch: noarch
BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root-%(%{__id_u} -n)
@ -31,7 +31,7 @@ rm -rf $RPM_BUILD_ROOT
%files
%defattr(-,root,root,-)
%doc README.md ChangeLog COPYING flawfinder.ps
%doc README ChangeLog COPYING flawfinder.ps
%{_bindir}/*
%{_mandir}/man1/*

26
flawtest.c Normal file
View File

@ -0,0 +1,26 @@
/* Test flawfinder. This program won't compile or run; that's not necessary
for this to be a useful test. */
main() {
char d[20];
char s[20];
int n;
_mbscpy(d,s); /* like strcpy, this doesn't check for buffer overflow */
memcpy(d,s);
CopyMemory(d,s);
lstrcat(d,s);
strncpy(d,s);
_tcsncpy(d,s);
strncat(d,s,10);
strncat(d,s,sizeof(d)); /* Misuse - this should be flagged as riskier. */
_tcsncat(d,s,sizeof(d)); /* Misuse - flag as riskier */
n = strlen(d);
/* This is wrong, and should be flagged as risky: */
MultiByteToWideChar(CP_ACP,0,szName,-1,wszUserName,sizeof(wszUserName));
/* This is much better: */
MultiByteToWideChar(CP_ACP,0,szName,-1,wszUserName,sizeof(wszUserName)/sizeof(wszUserName[0]));
}

10
junk.c Normal file
View File

@ -0,0 +1,10 @@
#include <stdio.h>
main() {
char abuf[1000];
FILE *FR = stdin;
fscanf(FR, "%2000s", abuf);
printf("Result = %s\n", abuf);
strcpy(new,old);
}

86
makefile
View File

@ -1,12 +1,15 @@
# Flawfinder.
# Released under the General Public License (GPL) version 2 or later.
# (C) 2001-2021 David A. Wheeler.
# Flawfinder. Released under the General Public License (GPL).
# (C) 2001-2014 David A. Wheeler.
# See "release_process.md" for release process, including
# how to change version numbers.
# To change version number, edit this here, the beginning of the
# "flawfinder" script, flawfinder.spec, setup.py, and index.html.
# Then "make test-is-correct" to get the updated version number.
# To distribute, "make distribute && su && make rpm".
# Then use make my_install to install to website image.
# Eventually switch to using DistUtils to autogenerate.
NAME=flawfinder
VERSION=2.0.19
VERSION=1.31
RPM_VERSION=1
VERSIONEDNAME=$(NAME)-$(VERSION)
ARCH=noarch
@ -46,8 +49,6 @@ RPMBUILD=rpmbuild
DESTDIR=
TESTDIR=test/
all: flawfinder.pdf flawfinder.1.gz
chmod -R a+rX *
@ -66,15 +67,14 @@ INSTALL_DATA=cp -p
# (admittedly rare) problem of bad date/timestamps causing the
# compiled code to override later uncompiled Python code.
install:
-$(MKDIR_P) "$(DESTDIR)$(INSTALL_DIR_BIN)"
$(INSTALL_PROGRAM) flawfinder.py "$(DESTDIR)$(INSTALL_DIR_BIN)/flawfinder$(PYTHONEXT)"
chmod a+x "$(DESTDIR)$(INSTALL_DIR_BIN)/flawfinder$(PYTHONEXT)"
-$(MKDIR_P) "$(DESTDIR)$(INSTALL_DIR_MAN)"
$(INSTALL_DATA) flawfinder.1 "$(DESTDIR)$(INSTALL_DIR_MAN)/flawfinder.1"
-$(MKDIR_P) $(DESTDIR)$(INSTALL_DIR_BIN)
$(INSTALL_PROGRAM) flawfinder$(PYTHONEXT) $(DESTDIR)$(INSTALL_DIR_BIN)/flawfinder$(PYTHONEXT)
-$(MKDIR_P) $(DESTDIR)$(INSTALL_DIR_MAN)
$(INSTALL_DATA) flawfinder.1 $(DESTDIR)$(INSTALL_DIR_MAN)/flawfinder.1
uninstall:
rm -f "$(DESTDIR)$(INSTALL_DIR_BIN)/flawfinder$(PYTHONEXT)"
rm -f "$(DESTDIR)$(INSTALL_DIR_MAN)/flawfinder.1"
rm -f $(DESTDIR)$(INSTALL_DIR_BIN)/flawfinder$(PYTHONEXT)
rm -f $(DESTDIR)$(INSTALL_DIR_MAN)/flawfinder.1
flawfinder.1.gz: flawfinder.1
gzip -c9 < flawfinder.1 > flawfinder.1.gz
@ -83,7 +83,6 @@ flawfinder.ps: flawfinder.1
man -t ./flawfinder.1 > flawfinder.ps
flawfinder.pdf: flawfinder.ps
which ps2pdf # Ensure we have ps2pdf installed (from ghostscript)
ps2pdf flawfinder.ps flawfinder.pdf
# Not built by default, since man2html is not widely available
@ -98,12 +97,13 @@ clean:
rm -f *.tar *.exe ./cwe
distribute: clean flawfinder.pdf flawfinder.ps
rm -fr build dist flawfinder.egg-info ,tempdir
chmod -R a+rX *
mkdir ,tempdir
cp -r -p [a-zA-Z]* ,tempdir
cp -p [a-zA-Z]* ,tempdir
rm -f ,tempdir/*.tar.gz
rm -f ,tempdir/*.rpm
# We don't need both "flawfinder" and "flawfinder.py":
rm -f ,tempdir/flawfinder.py
mv ,tempdir flawfinder-$(VERSION)
# Nobody else needs "update" either.
rm -f ,tempdir/update
@ -112,39 +112,38 @@ distribute: clean flawfinder.pdf flawfinder.ps
# Don't include (out of date) index.html
rm -f ,tempdir/index.html
tar cvfz flawfinder-$(VERSION).tar.gz flawfinder-$(VERSION)
chmod ugo+r flawfinder-$(VERSION).tar.gz
chown --reference=. flawfinder-$(VERSION).tar.gz
rm -fr flawfinder-$(VERSION)
dist: distribute
# This *creates* a PyPi distribution package. Use "upload-pypi" to upload it
pypi:
python setup.py bdist_wheel --universal
# NOTE: Only do this after running "make pypi" & being satisfied with it
# Use "-r pypitest" to upload to pypitest.
upload-pypi:
twine upload dist/*
time:
echo "Timing the program. First, time taken:"
time ./flawfinder.py $(SAMPLE_DIR)/*/*.[ch] > /dev/null
time ./flawfinder $(SAMPLE_DIR)/*/*.[ch] > /dev/null
echo "Lines examined:"
wc -l $(SAMPLE_DIR)/*/*.[ch] | tail -2
test:
cd $(TESTDIR); $(MAKE) test
test: flawfinder test.c test2.c
# Omit time report so that results are always the same textually.
./flawfinder --omittime test.c test2.c > test-results.txt
echo >> test-results.txt
echo "Testing for no ending newline:" >> test-results.txt
./flawfinder --omittime no-ending-newline.c | \
grep 'Lines analyzed' >> test-results.txt
./flawfinder --omittime --html --context test.c test2.c > test-results.html
@echo "Differences from expected results:"
@diff -u correct-results.txt test-results.txt
@diff -u correct-results.html test-results.html
# Usual check routine. Run all tests using *both* python2 and python3.
check:
cd $(TESTDIR); $(MAKE) check
check: test
# Run "make test-is-correct" if the results are as expected.
test-is-correct:
cd $(TESTDIR); $(MAKE) test-is-correct
test-is-correct: test-results.txt
mv test-results.txt correct-results.txt
mv test-results.html correct-results.html
profile:
/usr/lib/python1.5/profile.py ./flawfinder.py > profile-results $(SAMPLE_DIR)/*/*.[ch] > profile-results
/usr/lib/python1.5/profile.py ./flawfinder > profile-results $(SAMPLE_DIR)/*/*.[ch] > profile-results
rpm: distribute
@ -159,7 +158,7 @@ rpm: distribute
# cp -p /usr/src/redhat/RPMS/$(ARCH)/$(VERSIONEDNAME)-$(RPM_VERSION)*.rpm $(VERSIONEDNAME)-$(RPM_VERSION).noarch.rpm
cp -p /usr/src/redhat/RPMS/$(ARCH)/$(VERSIONEDNAME)-$(RPM_VERSION)*.rpm .
cp -p /usr/src/redhat/SRPMS/$(VERSIONEDNAME)-$(RPM_VERSION)*.src.rpm .
chown --reference=README.md *.rpm
chown --reference=README *.rpm
# Install, for testing. Ignore the "not installed" message here,
# unless you already installed it; we're just removing any old copies:
-rpm -e flawfinder
@ -172,7 +171,7 @@ my-install: flawfinder.pdf flawfinder.ps test
cp -p $(VERSIONEDNAME).tar.gz \
flawfinder flawfinder.1 makefile \
flawfinder.pdf flawfinder.ps ChangeLog \
$(TESTDIR)/test.c $(TESTDIR)/test2.c $(TESTDIR)/test-results.txt $(TESTDIR)/test-results.html \
test.c test2.c test-results.txt test-results.html \
/home/dwheeler/dwheeler.com/flawfinder/
# This is intended to be a local capability to list CWEs
@ -183,15 +182,14 @@ cwe: cwe.c
$(CC) -o cwe cwe.c -lfl
show-cwes: cwe
./cwe < flawfinder.py | sort -u -V
./cwe < flawfinder | sort -u -V
pylint:
pylint flawfinder
.PHONY: install clean test check profile test-is-correct rpm \
uninstall distribute my-install show-cwes pylint
.PHONY: install clean test check profile test-is-correct rpm uninstall distribute my-install show-cwes
# When I switch to using "DistUtils", I may need to move the MANIFEST.in
# file into a subdirectory (named flawfinder-versionnumber).
# I can then create all the distribution files by just typing:
# python setup.py bdist_rpm

0
test/no-ending-newline.c → no-ending-newline.c
View File

436
pylintrc
View File

@ -1,436 +0,0 @@
[MASTER]
# A comma-separated list of package or module names from where C extensions may
# be loaded. Extensions are loading into the active Python interpreter and may
# run arbitrary code
extension-pkg-whitelist=
# Add files or directories to the blacklist. They should be base names, not
# paths.
ignore=CVS
# Add files or directories matching the regex patterns to the blacklist. The
# regex matches against base names, not paths.
ignore-patterns=
# Python code to execute, usually for sys.path manipulation such as
# pygtk.require().
#init-hook=
# Use multiple processes to speed up Pylint.
jobs=1
# List of plugins (as comma separated values of python modules names) to load,
# usually to register additional checkers.
load-plugins=
# Pickle collected data for later comparisons.
persistent=yes
# Specify a configuration file.
#rcfile=
# Allow loading of arbitrary C extensions. Extensions are imported into the
# active Python interpreter and may run arbitrary code.
unsafe-load-any-extension=no
[MESSAGES CONTROL]
# Only show warnings with the listed confidence levels. Leave empty to show
# all. Valid levels: HIGH, INFERENCE, INFERENCE_FAILURE, UNDEFINED
confidence=
# Disable the message, report, category or checker with the given id(s). You
# can either give multiple identifiers separated by comma (,) or put this
# option multiple times (only on the command line, not in the configuration
# file where it should appear only once).You can also use "--disable=all" to
# disable everything first and then reenable specific checks. For example, if
# you want to run only the similarities checker, you can use "--disable=all
# --enable=similarities". If you want to run only the classes checker, but have
# no Warning level messages displayed, use"--disable=all --enable=classes
# --disable=W"
# Flawfinder specifics: This disables "invalid-name", because that checker
# has a serious bug: it can't handle global variables and thinks they are
# constants (so it raises a hailstorm of incorrect reports).
# We *use* global-statement, so don't warn about it.
# TODO: perhaps stop disabling some, e.g.,
# missing-docstring, too-many-statements,
# too-many-branches,too-many-locals,too-many-nested-blocks
disable=print-statement,parameter-unpacking,unpacking-in-except,old-raise-syntax,backtick,long-suffix,old-ne-operator,old-octal-literal,import-star-module-level,raw-checker-failed,bad-inline-option,locally-disabled,locally-enabled,file-ignored,suppressed-message,useless-suppression,deprecated-pragma,apply-builtin,basestring-builtin,buffer-builtin,cmp-builtin,coerce-builtin,execfile-builtin,file-builtin,long-builtin,raw_input-builtin,reduce-builtin,standarderror-builtin,unicode-builtin,xrange-builtin,coerce-method,delslice-method,getslice-method,setslice-method,no-absolute-import,old-division,dict-iter-method,dict-view-method,next-method-called,metaclass-assignment,indexing-exception,raising-string,reload-builtin,oct-method,hex-method,nonzero-method,cmp-method,input-builtin,round-builtin,intern-builtin,unichr-builtin,map-builtin-not-iterating,zip-builtin-not-iterating,range-builtin-not-iterating,filter-builtin-not-iterating,using-cmp-argument,eq-without-hash,div-method,idiv-method,rdiv-method,exception-message-attribute,invalid-str-codec,sys-max-int,bad-python3-import,deprecated-string-function,deprecated-str-translate-call,invalid-name,missing-docstring,global-statement,too-many-statements,too-many-branches,too-many-locals,too-many-instance-attributes,too-many-nested-blocks
# Enable the message, report, category or checker with the given id(s). You can
# either give multiple identifier separated by comma (,) or put this option
# multiple time (only on the command line, not in the configuration file where
# it should appear only once). See also the "--disable" option for examples.
enable=
[REPORTS]
# Python expression which should return a note less than 10 (10 is the highest
# note). You have access to the variables errors warning, statement which
# respectively contain the number of errors / warnings messages and the total
# number of statements analyzed. This is used by the global evaluation report
# (RP0004).
evaluation=10.0 - ((float(5 * error + warning + refactor + convention) / statement) * 10)
# Template used to display messages. This is a python new-style format string
# used to format the message information. See doc for all details
#msg-template=
# Set the output format. Available formats are text, parseable, colorized, json
# and msvs (visual studio).You can also give a reporter class, eg
# mypackage.mymodule.MyReporterClass.
output-format=text
# Tells whether to display a full report or only the messages
reports=no
# Activate the evaluation score.
score=yes
[REFACTORING]
# Maximum number of nested blocks for function / method body
max-nested-blocks=5
[BASIC]
# Naming hint for argument names
argument-name-hint=(([a-z][a-z0-9_]{2,30})|(_[a-z0-9_]*))$
# Regular expression matching correct argument names
argument-rgx=(([a-z][a-z0-9_]{2,30})|(_[a-z0-9_]*))$
# Naming hint for attribute names
attr-name-hint=(([a-z][a-z0-9_]{2,30})|(_[a-z0-9_]*))$
# Regular expression matching correct attribute names
attr-rgx=(([a-z][a-z0-9_]{2,30})|(_[a-z0-9_]*))$
# Bad variable names which should always be refused, separated by a comma
bad-names=foo,bar,baz,toto,tutu,tata
# Naming hint for class attribute names
class-attribute-name-hint=([A-Za-z_][A-Za-z0-9_]{2,30}|(__.*__))$
# Regular expression matching correct class attribute names
class-attribute-rgx=([A-Za-z_][A-Za-z0-9_]{2,30}|(__.*__))$
# Naming hint for class names
class-name-hint=[A-Z_][a-zA-Z0-9]+$
# Regular expression matching correct class names
class-rgx=[A-Z_][a-zA-Z0-9]+$
# Naming hint for constant names
const-name-hint=(([A-Z_][A-Z0-9_]*)|(__.*__))$
# Regular expression matching correct constant names
const-rgx=(([A-Z_][A-Z0-9_]*)|(__.*__))$
# Minimum line length for functions/classes that require docstrings, shorter
# ones are exempt.
docstring-min-length=-1
# Naming hint for function names
function-name-hint=(([a-z][a-z0-9_]{2,30})|(_[a-z0-9_]*))$
# Regular expression matching correct function names
function-rgx=(([a-z][a-z0-9_]{2,30})|(_[a-z0-9_]*))$
# Good variable names which should always be accepted, separated by a comma
good-names=i,j,k,ex,Run,_
# Include a hint for the correct naming format with invalid-name
include-naming-hint=no
# Naming hint for inline iteration names
inlinevar-name-hint=[A-Za-z_][A-Za-z0-9_]*$
# Regular expression matching correct inline iteration names
inlinevar-rgx=[A-Za-z_][A-Za-z0-9_]*$
# Naming hint for method names
method-name-hint=(([a-z][a-z0-9_]{2,30})|(_[a-z0-9_]*))$
# Regular expression matching correct method names
method-rgx=(([a-z][a-z0-9_]{2,30})|(_[a-z0-9_]*))$
# Naming hint for module names
module-name-hint=(([a-z_][a-z0-9_]*)|([A-Z][a-zA-Z0-9]+))$
# Regular expression matching correct module names
module-rgx=(([a-z_][a-z0-9_]*)|([A-Z][a-zA-Z0-9]+))$
# Colon-delimited sets of names that determine each other's naming style when
# the name regexes allow several styles.
name-group=
# Regular expression which should only match function or class names that do
# not require a docstring.
no-docstring-rgx=^_
# List of decorators that produce properties, such as abc.abstractproperty. Add
# to this list to register other decorators that produce valid properties.
property-classes=abc.abstractproperty
# Naming hint for variable names
variable-name-hint=(([a-z][a-z0-9_]{2,30})|(_[a-z0-9_]*))$
# Regular expression matching correct variable names
variable-rgx=(([a-z][a-z0-9_]{2,30})|(_[a-z0-9_]*))$
[FORMAT]
# Expected format of line ending, e.g. empty (any line ending), LF or CRLF.
expected-line-ending-format=
# Regexp for a line that is allowed to be longer than the limit.
ignore-long-lines=^\s*(# )?<?https?://\S+>?$
# Number of spaces of indent required inside a hanging or continued line.
indent-after-paren=4
# String used as indentation unit. This is usually " " (4 spaces) or "\t" (1
# tab).
indent-string=' '
# Maximum number of characters on a single line.
# TODO: Flawfinder specific - eventually we'll bring this down to 80
max-line-length=500
# Maximum number of lines in a module
# Flawfinder specific - we want this to be a single file
max-module-lines=5000
# List of optional constructs for which whitespace checking is disabled. `dict-
# separator` is used to allow tabulation in dicts, etc.: {1 : 1,\n222: 2}.
# `trailing-comma` allows a space between comma and closing bracket: (a, ).
# `empty-line` allows space-only lines.
no-space-check=trailing-comma,dict-separator
# Allow the body of a class to be on the same line as the declaration if body
# contains single statement.
single-line-class-stmt=no
# Allow the body of an if to be on the same line as the test if there is no
# else.
single-line-if-stmt=no
[LOGGING]
# Logging modules to check that the string format arguments are in logging
# function parameter format
logging-modules=logging
[MISCELLANEOUS]
# List of note tags to take in consideration, separated by a comma.
# notes=FIXME,XXX,TODO
# Flawfinder specifics: We already note them, no need to report
notes=
[SIMILARITIES]
# Ignore comments when computing similarities.
ignore-comments=yes
# Ignore docstrings when computing similarities.
ignore-docstrings=yes
# Ignore imports when computing similarities.
ignore-imports=no
# Minimum lines number of a similarity.
min-similarity-lines=4
[SPELLING]
# Spelling dictionary name. Available dictionaries: none. To make it working
# install python-enchant package.
spelling-dict=
# List of comma separated words that should not be checked.
spelling-ignore-words=
# A path to a file that contains private dictionary; one word per line.
spelling-private-dict-file=
# Tells whether to store unknown words to indicated private dictionary in
# --spelling-private-dict-file option instead of raising a message.
spelling-store-unknown-words=no
[TYPECHECK]
# List of decorators that produce context managers, such as
# contextlib.contextmanager. Add to this list to register other decorators that
# produce valid context managers.
contextmanager-decorators=contextlib.contextmanager
# List of members which are set dynamically and missed by pylint inference
# system, and so shouldn't trigger E1101 when accessed. Python regular
# expressions are accepted.
generated-members=
# Tells whether missing members accessed in mixin class should be ignored. A
# mixin class is detected if its name ends with "mixin" (case insensitive).
ignore-mixin-members=yes
# This flag controls whether pylint should warn about no-member and similar
# checks whenever an opaque object is returned when inferring. The inference
# can return multiple potential results while evaluating a Python object, but
# some branches might not be evaluated, which results in partial inference. In
# that case, it might be useful to still emit no-member and other checks for
# the rest of the inferred objects.
ignore-on-opaque-inference=yes
# List of class names for which member attributes should not be checked (useful
# for classes with dynamically set attributes). This supports the use of
# qualified names.
ignored-classes=optparse.Values,thread._local,_thread._local
# List of module names for which member attributes should not be checked
# (useful for modules/projects where namespaces are manipulated during runtime
# and thus existing member attributes cannot be deduced by static analysis. It
# supports qualified module names, as well as Unix pattern matching.
ignored-modules=
# Show a hint with possible names when a member name was not found. The aspect
# of finding the hint is based on edit distance.
missing-member-hint=yes
# The minimum edit distance a name should have in order to be considered a
# similar match for a missing member name.
missing-member-hint-distance=1
# The total number of similar names that should be taken in consideration when
# showing a hint for a missing member.
missing-member-max-choices=1
[VARIABLES]
# List of additional names supposed to be defined in builtins. Remember that
# you should avoid to define new builtins when possible.
additional-builtins=
# Tells whether unused global variables should be treated as a violation.
allow-global-unused-variables=yes
# List of strings which can identify a callback function by name. A callback
# name must start or end with one of those strings.
callbacks=cb_,_cb
# A regular expression matching the name of dummy variables (i.e. expectedly
# not used).
dummy-variables-rgx=_+$|(_[a-zA-Z0-9_]*[a-zA-Z0-9]+?$)|dummy|^ignored_|^unused_
# Argument names that match this expression will be ignored. Default to name
# with leading underscore
ignored-argument-names=_.*|^ignored_|^unused_
# Tells whether we should check for unused import in __init__ files.
init-import=no
# List of qualified module names which can have objects that can redefine
# builtins.
redefining-builtins-modules=six.moves,future.builtins
[CLASSES]
# List of method names used to declare (i.e. assign) instance attributes.
defining-attr-methods=__init__,__new__,setUp
# List of member names, which should be excluded from the protected access
# warning.
exclude-protected=_asdict,_fields,_replace,_source,_make
# List of valid names for the first argument in a class method.
valid-classmethod-first-arg=cls
# List of valid names for the first argument in a metaclass class method.
valid-metaclass-classmethod-first-arg=mcs
[DESIGN]
# Maximum number of arguments for function / method
max-args=5
# Maximum number of attributes for a class (see R0902).
max-attributes=7
# Maximum number of boolean expressions in a if statement
max-bool-expr=5
# Maximum number of branch for function / method body
max-branches=12
# Maximum number of locals for function / method body
max-locals=15
# Maximum number of parents for a class (see R0901).
max-parents=7
# Maximum number of public methods for a class (see R0904).
max-public-methods=20
# Maximum number of return / yield for function / method body
max-returns=6
# Maximum number of statements in function / method body
max-statements=50
# Minimum number of public methods for a class (see R0903).
min-public-methods=2
[IMPORTS]
# Allow wildcard imports from modules that define __all__.
allow-wildcard-with-all=no
# Analyse import fallback blocks. This can be used to support both Python 2 and
# 3 compatible code, which means that the block might have code that exists
# only in one or another interpreter, leading to false positives when analysed.
analyse-fallback-blocks=no
# Deprecated modules which should not be used, separated by a comma
deprecated-modules=regsub,TERMIOS,Bastion,rexec
# Create a graph of external dependencies in the given file (report RP0402 must
# not be disabled)
ext-import-graph=
# Create a graph of every (i.e. internal and external) dependencies in the
# given file (report RP0402 must not be disabled)
import-graph=
# Create a graph of internal dependencies in the given file (report RP0402 must
# not be disabled)
int-import-graph=
# Force import order to recognize a module as part of the standard
# compatibility libraries.
known-standard-library=
# Force import order to recognize a module as part of a third party library.
known-third-party=enchant
[EXCEPTIONS]
# Exceptions that will emit a warning when being caught. Defaults to
# "Exception"
overgeneral-exceptions=Exception

BIN
release_files/COPYING.info
View File

Binary file not shown.

BIN
release_files/ChangeLog.info
View File

Binary file not shown.

BIN
release_files/README.md.info
View File

Binary file not shown.

1963
release_files/flawfinder.html
View File

File diff suppressed because it is too large Load Diff

BIN
release_files/flawfinder.html.info
View File

Binary file not shown.

BIN
release_files/flawfinder.info
View File

Binary file not shown.

BIN
release_files/flawfinder.pdf
View File

Binary file not shown.

BIN
release_files/flawfinder.pdf.info
View File

Binary file not shown.

93
release_process.md
View File

@ -1,93 +0,0 @@
# Release process
Here's information on how to release an update to flawfinder.
## Changing version number
Ensure that the version number is the intended final value.
Make sure every release has a unique version number.
To change version number, edit the following files:
makefile
flawfinder.py
flawfinder.spec
setup.py
index.html # in dwheeler.com/flawfinder
Then run several times:
~~~~
make test ; make test-is-correct # update version number in tests
~~~~
## Test it
~~~~
make check # Run tests in Python 2 and 3
~~~~
## Commit it
~~~~
git commit -asv
~~~~
## Tag version
Once you're sure this is the *real* version, tag it:
~~~~
git tag VERSION
git push --tags origin # SourceForge
git push --tags github # GitHub
~~~~
## Create tarball
Run:
~~~~
make distribute
~~~~
## Post tarball
Then post the tarball flawfinder-VERSION.tar.gz to
the usual places:
* SourceForge "files" directory, and set it to be the default download.
* dwheeler.com/flawfinder
Do this *before* creating the PyPi distribution package for pip.
## Post to pip
First, install the programs to create a PyPi distribution package
if they are not already installed. On Cygwin first run:
~~~~
python -m ensurepip
pip install --upgrade pip
pip install wheel
pip install twine
~~~~
Then create a PyPi distribution package (for Python2 and Python3):
~~~~
make pypi
~~~~
Now upload the PyPi distribution package:
~~~~
make upload-pypi
~~~~
## After it's uploaded
Change the version number in the repo NOW, so that there will not
be two different released versions with the same version number.
See the list at the beginning of this document for the list of
files to change.

8
setup.cfg
View File

@ -1,13 +1,7 @@
[metadata]
description-file = README.md
[bdist_wheel]
universal=1
[bdist_rpm]
release = 1
doc_files = ChangeLog
README.md
README
COPYING
flawfinder.ps
flawfinder.pdf

54
setup.py
View File

@ -3,49 +3,39 @@
# This is the setup.py script for "flawfinder" by David A. Wheeler.
# My thanks to Jon Nelson, who created the initial setup.py script.
# Template for creating your own setup.py. See the USAGE file in
# the Distutils source distribution for descriptions of all the
# options shown below. Brief instructions on what to do:
# - set the other metadata: version, description, author, author_email
# and url. All of these except 'description' are required, although
# you may supply 'maintainer' and 'maintainer_email' in place of (or in
# addition to) 'author' and 'author_email' as appropriate.
# - fill in or delete the 'packages', 'package_dir', 'py_modules',
# and 'ext_modules' options as appropriate -- see USAGE for details
# - delete this comment and change '__revision__' to whatever is
# appropriate for your revision control system of choice (just make
# sure it stores the revision number for your distribution's setup.py
# script, *not* the examples/template_setup.py file from Distutils!)
"""Setup script for the flawfinder tool."""
from setuptools import setup # Don't need find_packages
from distutils.core import setup
import commands
setup (# Distribution meta-data
name = "flawfinder",
version = "2.0.19",
# We install a script, not a separate package.
# packages = ["flawfinder"], # Must be same as name
# Do not need: packages=find_packages(),
version = "1.31",
description = "a program that examines source code looking for security weaknesses",
author = "David A. Wheeler",
author_email = "dwheeler@dwheeler.com",
license = 'GPL-2.0+',
license = 'GPL',
long_description = """Flawfinder is a program that can scan
C/C++ source code and identify out potential security flaws,
ranking them by likely severity.
It is released under the GNU GPL license.""",
url = "http://dwheeler.com/flawfinder/",
download_url = "https://sourceforge.net/projects/flawfinder/files/flawfinder-2.0.8.tar.gz/download",
zip_safe = True,
keywords = ['analysis', 'security', 'analyzer'],
classifiers = [
'Development Status :: 5 - Production/Stable',
'Environment :: Console',
'Intended Audience :: Developers',
'License :: OSI Approved :: GNU General Public License v2 or later (GPLv2+)',
'Natural Language :: English',
'Programming Language :: Python :: 2.7',
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.6',
'Operating System :: OS Independent',
'Topic :: Security',
'Topic :: Software Development :: Build Tools',
'Topic :: Software Development :: Quality Assurance',
'Topic :: Software Development :: Testing'
],
python_requires = '>=2.7',
entry_points={
'console_scripts': [
'flawfinder = flawfinder:main',
],
},
url = "http://www.dwheeler.com/flawfinder/",
scripts = [ 'flawfinder' ],
data_files = [ ('share/man/man1', [ 'flawfinder.1.gz' ]) ],
py_modules = ['flawfinder'],
py_modules = [ ],
)

577
simplejson/__init__.py
View File

@ -1,577 +0,0 @@
r"""JSON (JavaScript Object Notation) <http://json.org> is a subset of
JavaScript syntax (ECMA-262 3rd edition) used as a lightweight data
interchange format.
:mod:`simplejson` exposes an API familiar to users of the standard library
:mod:`marshal` and :mod:`pickle` modules. It is the externally maintained
version of the :mod:`json` library contained in Python 2.6, but maintains
compatibility back to Python 2.5 and (currently) has significant performance
advantages, even without using the optional C extension for speedups.
Encoding basic Python object hierarchies::
>>> import simplejson as json
>>> json.dumps(['foo', {'bar': ('baz', None, 1.0, 2)}])
'["foo", {"bar": ["baz", null, 1.0, 2]}]'
>>> print(json.dumps("\"foo\bar"))
"\"foo\bar"
>>> print(json.dumps(u'\u1234'))
"\u1234"
>>> print(json.dumps('\\'))
"\\"
>>> print(json.dumps({"c": 0, "b": 0, "a": 0}, sort_keys=True))
{"a": 0, "b": 0, "c": 0}
>>> from simplejson.compat import StringIO
>>> io = StringIO()
>>> json.dump(['streaming API'], io)
>>> io.getvalue()
'["streaming API"]'
Compact encoding::
>>> import simplejson as json
>>> obj = [1,2,3,{'4': 5, '6': 7}]
>>> json.dumps(obj, separators=(',',':'), sort_keys=True)
'[1,2,3,{"4":5,"6":7}]'
Pretty printing::
>>> import simplejson as json
>>> print(json.dumps({'4': 5, '6': 7}, sort_keys=True, indent=' '))
{
"4": 5,
"6": 7
}
Decoding JSON::
>>> import simplejson as json
>>> obj = [u'foo', {u'bar': [u'baz', None, 1.0, 2]}]
>>> json.loads('["foo", {"bar":["baz", null, 1.0, 2]}]') == obj
True
>>> json.loads('"\\"foo\\bar"') == u'"foo\x08ar'
True
>>> from simplejson.compat import StringIO
>>> io = StringIO('["streaming API"]')
>>> json.load(io)[0] == 'streaming API'
True
Specializing JSON object decoding::
>>> import simplejson as json
>>> def as_complex(dct):
... if '__complex__' in dct:
... return complex(dct['real'], dct['imag'])
... return dct
...
>>> json.loads('{"__complex__": true, "real": 1, "imag": 2}',
... object_hook=as_complex)
(1+2j)
>>> from decimal import Decimal
>>> json.loads('1.1', parse_float=Decimal) == Decimal('1.1')
True
Specializing JSON object encoding::
>>> import simplejson as json
>>> def encode_complex(obj):
... if isinstance(obj, complex):
... return [obj.real, obj.imag]
... raise TypeError('Object of type %s is not JSON serializable' %
... obj.__class__.__name__)
...
>>> json.dumps(2 + 1j, default=encode_complex)
'[2.0, 1.0]'
>>> json.JSONEncoder(default=encode_complex).encode(2 + 1j)
'[2.0, 1.0]'
>>> ''.join(json.JSONEncoder(default=encode_complex).iterencode(2 + 1j))
'[2.0, 1.0]'
Using simplejson.tool from the shell to validate and pretty-print::
$ echo '{"json":"obj"}' | python -m simplejson.tool
{
"json": "obj"
}
$ echo '{ 1.2:3.4}' | python -m simplejson.tool
Expecting property name: line 1 column 3 (char 2)
"""
from __future__ import absolute_import
__version__ = '3.16.0'
__all__ = [
'dump', 'dumps', 'load', 'loads',
'JSONDecoder', 'JSONDecodeError', 'JSONEncoder',
'OrderedDict', 'simple_first', 'RawJSON'
]
__author__ = 'Bob Ippolito <bob@redivi.com>'
from decimal import Decimal
from .errors import JSONDecodeError
from .raw_json import RawJSON
from .decoder import JSONDecoder
from .encoder import JSONEncoder, JSONEncoderForHTML
def _import_OrderedDict():
import collections
try:
return collections.OrderedDict
except AttributeError:
from . import ordered_dict
return ordered_dict.OrderedDict
OrderedDict = _import_OrderedDict()
def _import_c_make_encoder():
try:
from ._speedups import make_encoder
return make_encoder
except ImportError:
return None
_default_encoder = JSONEncoder(
skipkeys=False,
ensure_ascii=True,
check_circular=True,
allow_nan=True,
indent=None,
separators=None,
encoding='utf-8',
default=None,
use_decimal=True,
namedtuple_as_object=True,
tuple_as_array=True,
iterable_as_array=False,
bigint_as_string=False,
item_sort_key=None,
for_json=False,
ignore_nan=False,
int_as_string_bitcount=None,
)
def dump(obj, fp, skipkeys=False, ensure_ascii=True, check_circular=True,
allow_nan=True, cls=None, indent=None, separators=None,
encoding='utf-8', default=None, use_decimal=True,
namedtuple_as_object=True, tuple_as_array=True,
bigint_as_string=False, sort_keys=False, item_sort_key=None,
for_json=False, ignore_nan=False, int_as_string_bitcount=None,
iterable_as_array=False, **kw):
"""Serialize ``obj`` as a JSON formatted stream to ``fp`` (a
``.write()``-supporting file-like object).
If *skipkeys* is true then ``dict`` keys that are not basic types
(``str``, ``unicode``, ``int``, ``long``, ``float``, ``bool``, ``None``)
will be skipped instead of raising a ``TypeError``.
If *ensure_ascii* is false, then the some chunks written to ``fp``
may be ``unicode`` instances, subject to normal Python ``str`` to
``unicode`` coercion rules. Unless ``fp.write()`` explicitly
understands ``unicode`` (as in ``codecs.getwriter()``) this is likely
to cause an error.
If *check_circular* is false, then the circular reference check
for container types will be skipped and a circular reference will
result in an ``OverflowError`` (or worse).
If *allow_nan* is false, then it will be a ``ValueError`` to
serialize out of range ``float`` values (``nan``, ``inf``, ``-inf``)
in strict compliance of the original JSON specification, instead of using
the JavaScript equivalents (``NaN``, ``Infinity``, ``-Infinity``). See
*ignore_nan* for ECMA-262 compliant behavior.
If *indent* is a string, then JSON array elements and object members
will be pretty-printed with a newline followed by that string repeated
for each level of nesting. ``None`` (the default) selects the most compact
representation without any newlines. For backwards compatibility with
versions of simplejson earlier than 2.1.0, an integer is also accepted
and is converted to a string with that many spaces.
If specified, *separators* should be an
``(item_separator, key_separator)`` tuple. The default is ``(', ', ': ')``
if *indent* is ``None`` and ``(',', ': ')`` otherwise. To get the most
compact JSON representation, you should specify ``(',', ':')`` to eliminate
whitespace.
*encoding* is the character encoding for str instances, default is UTF-8.
*default(obj)* is a function that should return a serializable version
of obj or raise ``TypeError``. The default simply raises ``TypeError``.
If *use_decimal* is true (default: ``True``) then decimal.Decimal
will be natively serialized to JSON with full precision.
If *namedtuple_as_object* is true (default: ``True``),
:class:`tuple` subclasses with ``_asdict()`` methods will be encoded
as JSON objects.
If *tuple_as_array* is true (default: ``True``),
:class:`tuple` (and subclasses) will be encoded as JSON arrays.
If *iterable_as_array* is true (default: ``False``),
any object not in the above table that implements ``__iter__()``
will be encoded as a JSON array.
If *bigint_as_string* is true (default: ``False``), ints 2**53 and higher
or lower than -2**53 will be encoded as strings. This is to avoid the
rounding that happens in Javascript otherwise. Note that this is still a
lossy operation that will not round-trip correctly and should be used
sparingly.
If *int_as_string_bitcount* is a positive number (n), then int of size
greater than or equal to 2**n or lower than or equal to -2**n will be
encoded as strings.
If specified, *item_sort_key* is a callable used to sort the items in
each dictionary. This is useful if you want to sort items other than
in alphabetical order by key. This option takes precedence over
*sort_keys*.
If *sort_keys* is true (default: ``False``), the output of dictionaries
will be sorted by item.
If *for_json* is true (default: ``False``), objects with a ``for_json()``
method will use the return value of that method for encoding as JSON
instead of the object.
If *ignore_nan* is true (default: ``False``), then out of range
:class:`float` values (``nan``, ``inf``, ``-inf``) will be serialized as
``null`` in compliance with the ECMA-262 specification. If true, this will
override *allow_nan*.
To use a custom ``JSONEncoder`` subclass (e.g. one that overrides the
``.default()`` method to serialize additional types), specify it with
the ``cls`` kwarg. NOTE: You should use *default* or *for_json* instead
of subclassing whenever possible.
"""
# cached encoder
if (not skipkeys and ensure_ascii and
check_circular and allow_nan and
cls is None and indent is None and separators is None and
encoding == 'utf-8' and default is None and use_decimal
and namedtuple_as_object and tuple_as_array and not iterable_as_array
and not bigint_as_string and not sort_keys
and not item_sort_key and not for_json
and not ignore_nan and int_as_string_bitcount is None
and not kw
):
iterable = _default_encoder.iterencode(obj)
else:
if cls is None:
cls = JSONEncoder
iterable = cls(skipkeys=skipkeys, ensure_ascii=ensure_ascii,
check_circular=check_circular, allow_nan=allow_nan, indent=indent,
separators=separators, encoding=encoding,
default=default, use_decimal=use_decimal,
namedtuple_as_object=namedtuple_as_object,
tuple_as_array=tuple_as_array,
iterable_as_array=iterable_as_array,
bigint_as_string=bigint_as_string,
sort_keys=sort_keys,
item_sort_key=item_sort_key,
for_json=for_json,
ignore_nan=ignore_nan,
int_as_string_bitcount=int_as_string_bitcount,
**kw).iterencode(obj)
# could accelerate with writelines in some versions of Python, at
# a debuggability cost
for chunk in iterable:
fp.write(chunk)
def dumps(obj, skipkeys=False, ensure_ascii=True, check_circular=True,
allow_nan=True, cls=None, indent=None, separators=None,
encoding='utf-8', default=None, use_decimal=True,
namedtuple_as_object=True, tuple_as_array=True,
bigint_as_string=False, sort_keys=False, item_sort_key=None,
for_json=False, ignore_nan=False, int_as_string_bitcount=None,
iterable_as_array=False, **kw):
"""Serialize ``obj`` to a JSON formatted ``str``.
If ``skipkeys`` is false then ``dict`` keys that are not basic types
(``str``, ``unicode``, ``int``, ``long``, ``float``, ``bool``, ``None``)
will be skipped instead of raising a ``TypeError``.
If ``ensure_ascii`` is false, then the return value will be a
``unicode`` instance subject to normal Python ``str`` to ``unicode``
coercion rules instead of being escaped to an ASCII ``str``.
If ``check_circular`` is false, then the circular reference check
for container types will be skipped and a circular reference will
result in an ``OverflowError`` (or worse).
If ``allow_nan`` is false, then it will be a ``ValueError`` to
serialize out of range ``float`` values (``nan``, ``inf``, ``-inf``) in
strict compliance of the JSON specification, instead of using the
JavaScript equivalents (``NaN``, ``Infinity``, ``-Infinity``).
If ``indent`` is a string, then JSON array elements and object members
will be pretty-printed with a newline followed by that string repeated
for each level of nesting. ``None`` (the default) selects the most compact
representation without any newlines. For backwards compatibility with
versions of simplejson earlier than 2.1.0, an integer is also accepted
and is converted to a string with that many spaces.
If specified, ``separators`` should be an
``(item_separator, key_separator)`` tuple. The default is ``(', ', ': ')``
if *indent* is ``None`` and ``(',', ': ')`` otherwise. To get the most
compact JSON representation, you should specify ``(',', ':')`` to eliminate
whitespace.
``encoding`` is the character encoding for str instances, default is UTF-8.
``default(obj)`` is a function that should return a serializable version
of obj or raise TypeError. The default simply raises TypeError.
If *use_decimal* is true (default: ``True``) then decimal.Decimal
will be natively serialized to JSON with full precision.
If *namedtuple_as_object* is true (default: ``True``),
:class:`tuple` subclasses with ``_asdict()`` methods will be encoded
as JSON objects.
If *tuple_as_array* is true (default: ``True``),
:class:`tuple` (and subclasses) will be encoded as JSON arrays.
If *iterable_as_array* is true (default: ``False``),
any object not in the above table that implements ``__iter__()``
will be encoded as a JSON array.
If *bigint_as_string* is true (not the default), ints 2**53 and higher
or lower than -2**53 will be encoded as strings. This is to avoid the
rounding that happens in Javascript otherwise.
If *int_as_string_bitcount* is a positive number (n), then int of size
greater than or equal to 2**n or lower than or equal to -2**n will be
encoded as strings.
If specified, *item_sort_key* is a callable used to sort the items in
each dictionary. This is useful if you want to sort items other than
in alphabetical order by key. This option takes precendence over
*sort_keys*.
If *sort_keys* is true (default: ``False``), the output of dictionaries
will be sorted by item.
If *for_json* is true (default: ``False``), objects with a ``for_json()``
method will use the return value of that method for encoding as JSON
instead of the object.
If *ignore_nan* is true (default: ``False``), then out of range
:class:`float` values (``nan``, ``inf``, ``-inf``) will be serialized as
``null`` in compliance with the ECMA-262 specification. If true, this will
override *allow_nan*.
To use a custom ``JSONEncoder`` subclass (e.g. one that overrides the
``.default()`` method to serialize additional types), specify it with
the ``cls`` kwarg. NOTE: You should use *default* instead of subclassing
whenever possible.
"""
# cached encoder
if (not skipkeys and ensure_ascii and
check_circular and allow_nan and
cls is None and indent is None and separators is None and
encoding == 'utf-8' and default is None and use_decimal
and namedtuple_as_object and tuple_as_array and not iterable_as_array
and not bigint_as_string and not sort_keys
and not item_sort_key and not for_json
and not ignore_nan and int_as_string_bitcount is None
and not kw
):
return _default_encoder.encode(obj)
if cls is None:
cls = JSONEncoder
return cls(
skipkeys=skipkeys, ensure_ascii=ensure_ascii,
check_circular=check_circular, allow_nan=allow_nan, indent=indent,
separators=separators, encoding=encoding, default=default,
use_decimal=use_decimal,
namedtuple_as_object=namedtuple_as_object,
tuple_as_array=tuple_as_array,
iterable_as_array=iterable_as_array,
bigint_as_string=bigint_as_string,
sort_keys=sort_keys,
item_sort_key=item_sort_key,
for_json=for_json,
ignore_nan=ignore_nan,
int_as_string_bitcount=int_as_string_bitcount,
**kw).encode(obj)
_default_decoder = JSONDecoder(encoding=None, object_hook=None,
object_pairs_hook=None)
def load(fp, encoding=None, cls=None, object_hook=None, parse_float=None,
parse_int=None, parse_constant=None, object_pairs_hook=None,
use_decimal=False, namedtuple_as_object=True, tuple_as_array=True,
**kw):
"""Deserialize ``fp`` (a ``.read()``-supporting file-like object containing
a JSON document) to a Python object.
*encoding* determines the encoding used to interpret any
:class:`str` objects decoded by this instance (``'utf-8'`` by
default). It has no effect when decoding :class:`unicode` objects.
Note that currently only encodings that are a superset of ASCII work,
strings of other encodings should be passed in as :class:`unicode`.
*object_hook*, if specified, will be called with the result of every
JSON object decoded and its return value will be used in place of the
given :class:`dict`. This can be used to provide custom
deserializations (e.g. to support JSON-RPC class hinting).
*object_pairs_hook* is an optional function that will be called with
the result of any object literal decode with an ordered list of pairs.
The return value of *object_pairs_hook* will be used instead of the
:class:`dict`. This feature can be used to implement custom decoders
that rely on the order that the key and value pairs are decoded (for
example, :func:`collections.OrderedDict` will remember the order of
insertion). If *object_hook* is also defined, the *object_pairs_hook*
takes priority.
*parse_float*, if specified, will be called with the string of every
JSON float to be decoded. By default, this is equivalent to
``float(num_str)``. This can be used to use another datatype or parser
for JSON floats (e.g. :class:`decimal.Decimal`).
*parse_int*, if specified, will be called with the string of every
JSON int to be decoded. By default, this is equivalent to
``int(num_str)``. This can be used to use another datatype or parser
for JSON integers (e.g. :class:`float`).
*parse_constant*, if specified, will be called with one of the
following strings: ``'-Infinity'``, ``'Infinity'``, ``'NaN'``. This
can be used to raise an exception if invalid JSON numbers are
encountered.
If *use_decimal* is true (default: ``False``) then it implies
parse_float=decimal.Decimal for parity with ``dump``.
To use a custom ``JSONDecoder`` subclass, specify it with the ``cls``
kwarg. NOTE: You should use *object_hook* or *object_pairs_hook* instead
of subclassing whenever possible.
"""
return loads(fp.read(),
encoding=encoding, cls=cls, object_hook=object_hook,
parse_float=parse_float, parse_int=parse_int,
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook,
use_decimal=use_decimal, **kw)
def loads(s, encoding=None, cls=None, object_hook=None, parse_float=None,
parse_int=None, parse_constant=None, object_pairs_hook=None,
use_decimal=False, **kw):
"""Deserialize ``s`` (a ``str`` or ``unicode`` instance containing a JSON
document) to a Python object.
*encoding* determines the encoding used to interpret any
:class:`str` objects decoded by this instance (``'utf-8'`` by
default). It has no effect when decoding :class:`unicode` objects.
Note that currently only encodings that are a superset of ASCII work,
strings of other encodings should be passed in as :class:`unicode`.
*object_hook*, if specified, will be called with the result of every
JSON object decoded and its return value will be used in place of the
given :class:`dict`. This can be used to provide custom
deserializations (e.g. to support JSON-RPC class hinting).
*object_pairs_hook* is an optional function that will be called with
the result of any object literal decode with an ordered list of pairs.
The return value of *object_pairs_hook* will be used instead of the
:class:`dict`. This feature can be used to implement custom decoders
that rely on the order that the key and value pairs are decoded (for
example, :func:`collections.OrderedDict` will remember the order of
insertion). If *object_hook* is also defined, the *object_pairs_hook*
takes priority.
*parse_float*, if specified, will be called with the string of every
JSON float to be decoded. By default, this is equivalent to
``float(num_str)``. This can be used to use another datatype or parser
for JSON floats (e.g. :class:`decimal.Decimal`).
*parse_int*, if specified, will be called with the string of every
JSON int to be decoded. By default, this is equivalent to
``int(num_str)``. This can be used to use another datatype or parser
for JSON integers (e.g. :class:`float`).
*parse_constant*, if specified, will be called with one of the
following strings: ``'-Infinity'``, ``'Infinity'``, ``'NaN'``. This
can be used to raise an exception if invalid JSON numbers are
encountered.
If *use_decimal* is true (default: ``False``) then it implies
parse_float=decimal.Decimal for parity with ``dump``.
To use a custom ``JSONDecoder`` subclass, specify it with the ``cls``
kwarg. NOTE: You should use *object_hook* or *object_pairs_hook* instead
of subclassing whenever possible.
"""
if (cls is None and encoding is None and object_hook is None and
parse_int is None and parse_float is None and
parse_constant is None and object_pairs_hook is None
and not use_decimal and not kw):
return _default_decoder.decode(s)
if cls is None:
cls = JSONDecoder
if object_hook is not None:
kw['object_hook'] = object_hook
if object_pairs_hook is not None:
kw['object_pairs_hook'] = object_pairs_hook
if parse_float is not None:
kw['parse_float'] = parse_float
if parse_int is not None:
kw['parse_int'] = parse_int
if parse_constant is not None:
kw['parse_constant'] = parse_constant
if use_decimal:
if parse_float is not None:
raise TypeError("use_decimal=True implies parse_float=Decimal")
kw['parse_float'] = Decimal
return cls(encoding=encoding, **kw).decode(s)
def _toggle_speedups(enabled):
from . import decoder as dec
from . import encoder as enc
from . import scanner as scan
c_make_encoder = _import_c_make_encoder()
if enabled:
dec.scanstring = dec.c_scanstring or dec.py_scanstring
enc.c_make_encoder = c_make_encoder
enc.encode_basestring_ascii = (enc.c_encode_basestring_ascii or
enc.py_encode_basestring_ascii)
scan.make_scanner = scan.c_make_scanner or scan.py_make_scanner
else:
dec.scanstring = dec.py_scanstring
enc.c_make_encoder = None
enc.encode_basestring_ascii = enc.py_encode_basestring_ascii
scan.make_scanner = scan.py_make_scanner
dec.make_scanner = scan.make_scanner
global _default_decoder
_default_decoder = JSONDecoder(
encoding=None,
object_hook=None,
object_pairs_hook=None,
)
global _default_encoder
_default_encoder = JSONEncoder(
skipkeys=False,
ensure_ascii=True,
check_circular=True,
allow_nan=True,
indent=None,
separators=None,
encoding='utf-8',
default=None,
)
def simple_first(kv):
"""Helper function to pass to item_sort_key to sort simple
elements to the top, then container elements.
"""
return (isinstance(kv[1], (list, dict, tuple)), kv[0])

3384
simplejson/_speedups.c
View File

File diff suppressed because it is too large Load Diff

34
simplejson/compat.py
View File

@ -1,34 +0,0 @@
"""Python 3 compatibility shims
"""
import sys
if sys.version_info[0] < 3:
PY3 = False
def b(s):
return s
try:
from cStringIO import StringIO
except ImportError:
from StringIO import StringIO
BytesIO = StringIO
text_type = unicode
binary_type = str
string_types = (basestring,)
integer_types = (int, long)
unichr = unichr
reload_module = reload
else:
PY3 = True
if sys.version_info[:2] >= (3, 4):
from importlib import reload as reload_module
else:
from imp import reload as reload_module
def b(s):
return bytes(s, 'latin1')
from io import StringIO, BytesIO
text_type = str
binary_type = bytes
string_types = (str,)
integer_types = (int,)
unichr = chr
long_type = integer_types[-1]

400
simplejson/decoder.py
View File

@ -1,400 +0,0 @@
"""Implementation of JSONDecoder
"""
from __future__ import absolute_import
import re
import sys
import struct
from .compat import PY3, unichr
from .scanner import make_scanner, JSONDecodeError
def _import_c_scanstring():
try:
from ._speedups import scanstring
return scanstring
except ImportError:
return None
c_scanstring = _import_c_scanstring()
# NOTE (3.1.0): JSONDecodeError may still be imported from this module for
# compatibility, but it was never in the __all__
__all__ = ['JSONDecoder']
FLAGS = re.VERBOSE | re.MULTILINE | re.DOTALL
def _floatconstants():
if sys.version_info < (2, 6):
_BYTES = '7FF80000000000007FF0000000000000'.decode('hex')
nan, inf = struct.unpack('>dd', _BYTES)
else:
nan = float('nan')
inf = float('inf')
return nan, inf, -inf
NaN, PosInf, NegInf = _floatconstants()
_CONSTANTS = {
'-Infinity': NegInf,
'Infinity': PosInf,
'NaN': NaN,
}
STRINGCHUNK = re.compile(r'(.*?)(["\\\x00-\x1f])', FLAGS)
BACKSLASH = {
'"': u'"', '\\': u'\\', '/': u'/',
'b': u'\b', 'f': u'\f', 'n': u'\n', 'r': u'\r', 't': u'\t',
}
DEFAULT_ENCODING = "utf-8"
def py_scanstring(s, end, encoding=None, strict=True,
_b=BACKSLASH, _m=STRINGCHUNK.match, _join=u''.join,
_PY3=PY3, _maxunicode=sys.maxunicode):
"""Scan the string s for a JSON string. End is the index of the
character in s after the quote that started the JSON string.
Unescapes all valid JSON string escape sequences and raises ValueError
on attempt to decode an invalid string. If strict is False then literal
control characters are allowed in the string.
Returns a tuple of the decoded string and the index of the character in s
after the end quote."""
if encoding is None:
encoding = DEFAULT_ENCODING
chunks = []
_append = chunks.append
begin = end - 1
while 1:
chunk = _m(s, end)
if chunk is None:
raise JSONDecodeError(
"Unterminated string starting at", s, begin)
end = chunk.end()
content, terminator = chunk.groups()
# Content is contains zero or more unescaped string characters
if content:
if not _PY3 and not isinstance(content, unicode):
content = unicode(content, encoding)
_append(content)
# Terminator is the end of string, a literal control character,
# or a backslash denoting that an escape sequence follows
if terminator == '"':
break
elif terminator != '\\':
if strict:
msg = "Invalid control character %r at"
raise JSONDecodeError(msg, s, end)
else:
_append(terminator)
continue
try:
esc = s[end]
except IndexError:
raise JSONDecodeError(
"Unterminated string starting at", s, begin)
# If not a unicode escape sequence, must be in the lookup table
if esc != 'u':
try:
char = _b[esc]
except KeyError:
msg = "Invalid \\X escape sequence %r"
raise JSONDecodeError(msg, s, end)
end += 1
else:
# Unicode escape sequence
msg = "Invalid \\uXXXX escape sequence"
esc = s[end + 1:end + 5]
escX = esc[1:2]
if len(esc) != 4 or escX == 'x' or escX == 'X':
raise JSONDecodeError(msg, s, end - 1)
try:
uni = int(esc, 16)
except ValueError:
raise JSONDecodeError(msg, s, end - 1)
end += 5
# Check for surrogate pair on UCS-4 systems
# Note that this will join high/low surrogate pairs
# but will also pass unpaired surrogates through
if (_maxunicode > 65535 and
uni & 0xfc00 == 0xd800 and
s[end:end + 2] == '\\u'):
esc2 = s[end + 2:end + 6]
escX = esc2[1:2]
if len(esc2) == 4 and not (escX == 'x' or escX == 'X'):
try:
uni2 = int(esc2, 16)
except ValueError:
raise JSONDecodeError(msg, s, end)
if uni2 & 0xfc00 == 0xdc00:
uni = 0x10000 + (((uni - 0xd800) << 10) |
(uni2 - 0xdc00))
end += 6
char = unichr(uni)
# Append the unescaped character
_append(char)
return _join(chunks), end
# Use speedup if available
scanstring = c_scanstring or py_scanstring
WHITESPACE = re.compile(r'[ \t\n\r]*', FLAGS)
WHITESPACE_STR = ' \t\n\r'
def JSONObject(state, encoding, strict, scan_once, object_hook,
object_pairs_hook, memo=None,
_w=WHITESPACE.match, _ws=WHITESPACE_STR):
(s, end) = state
# Backwards compatibility
if memo is None:
memo = {}
memo_get = memo.setdefault
pairs = []
# Use a slice to prevent IndexError from being raised, the following
# check will raise a more specific ValueError if the string is empty
nextchar = s[end:end + 1]
# Normally we expect nextchar == '"'
if nextchar != '"':
if nextchar in _ws:
end = _w(s, end).end()
nextchar = s[end:end + 1]
# Trivial empty object
if nextchar == '}':
if object_pairs_hook is not None:
result = object_pairs_hook(pairs)
return result, end + 1
pairs = {}
if object_hook is not None:
pairs = object_hook(pairs)
return pairs, end + 1
elif nextchar != '"':
raise JSONDecodeError(
"Expecting property name enclosed in double quotes",
s, end)
end += 1
while True:
key, end = scanstring(s, end, encoding, strict)
key = memo_get(key, key)
# To skip some function call overhead we optimize the fast paths where
# the JSON key separator is ": " or just ":".
if s[end:end + 1] != ':':
end = _w(s, end).end()
if s[end:end + 1] != ':':
raise JSONDecodeError("Expecting ':' delimiter", s, end)
end += 1
try:
if s[end] in _ws:
end += 1
if s[end] in _ws:
end = _w(s, end + 1).end()
except IndexError:
pass
value, end = scan_once(s, end)
pairs.append((key, value))
try:
nextchar = s[end]
if nextchar in _ws:
end = _w(s, end + 1).end()
nextchar = s[end]
except IndexError:
nextchar = ''
end += 1
if nextchar == '}':
break
elif nextchar != ',':
raise JSONDecodeError("Expecting ',' delimiter or '}'", s, end - 1)
try:
nextchar = s[end]
if nextchar in _ws:
end += 1
nextchar = s[end]
if nextchar in _ws:
end = _w(s, end + 1).end()
nextchar = s[end]
except IndexError:
nextchar = ''
end += 1
if nextchar != '"':
raise JSONDecodeError(
"Expecting property name enclosed in double quotes",
s, end - 1)
if object_pairs_hook is not None:
result = object_pairs_hook(pairs)
return result, end
pairs = dict(pairs)
if object_hook is not None:
pairs = object_hook(pairs)
return pairs, end
def JSONArray(state, scan_once, _w=WHITESPACE.match, _ws=WHITESPACE_STR):
(s, end) = state
values = []
nextchar = s[end:end + 1]
if nextchar in _ws:
end = _w(s, end + 1).end()
nextchar = s[end:end + 1]
# Look-ahead for trivial empty array
if nextchar == ']':
return values, end + 1
elif nextchar == '':
raise JSONDecodeError("Expecting value or ']'", s, end)
_append = values.append
while True:
value, end = scan_once(s, end)
_append(value)
nextchar = s[end:end + 1]
if nextchar in _ws:
end = _w(s, end + 1).end()
nextchar = s[end:end + 1]
end += 1
if nextchar == ']':
break
elif nextchar != ',':
raise JSONDecodeError("Expecting ',' delimiter or ']'", s, end - 1)
try:
if s[end] in _ws:
end += 1
if s[end] in _ws:
end = _w(s, end + 1).end()
except IndexError:
pass
return values, end
class JSONDecoder(object):
"""Simple JSON <http://json.org> decoder
Performs the following translations in decoding by default:
+---------------+-------------------+
| JSON | Python |
+===============+===================+
| object | dict |
+---------------+-------------------+
| array | list |
+---------------+-------------------+
| string | str, unicode |
+---------------+-------------------+
| number (int) | int, long |
+---------------+-------------------+
| number (real) | float |
+---------------+-------------------+
| true | True |
+---------------+-------------------+
| false | False |
+---------------+-------------------+
| null | None |
+---------------+-------------------+
It also understands ``NaN``, ``Infinity``, and ``-Infinity`` as
their corresponding ``float`` values, which is outside the JSON spec.
"""
def __init__(self, encoding=None, object_hook=None, parse_float=None,
parse_int=None, parse_constant=None, strict=True,
object_pairs_hook=None):
"""
*encoding* determines the encoding used to interpret any
:class:`str` objects decoded by this instance (``'utf-8'`` by
default). It has no effect when decoding :class:`unicode` objects.
Note that currently only encodings that are a superset of ASCII work,
strings of other encodings should be passed in as :class:`unicode`.
*object_hook*, if specified, will be called with the result of every
JSON object decoded and its return value will be used in place of the
given :class:`dict`. This can be used to provide custom
deserializations (e.g. to support JSON-RPC class hinting).
*object_pairs_hook* is an optional function that will be called with
the result of any object literal decode with an ordered list of pairs.
The return value of *object_pairs_hook* will be used instead of the
:class:`dict`. This feature can be used to implement custom decoders
that rely on the order that the key and value pairs are decoded (for
example, :func:`collections.OrderedDict` will remember the order of
insertion). If *object_hook* is also defined, the *object_pairs_hook*
takes priority.
*parse_float*, if specified, will be called with the string of every
JSON float to be decoded. By default, this is equivalent to
``float(num_str)``. This can be used to use another datatype or parser
for JSON floats (e.g. :class:`decimal.Decimal`).
*parse_int*, if specified, will be called with the string of every
JSON int to be decoded. By default, this is equivalent to
``int(num_str)``. This can be used to use another datatype or parser
for JSON integers (e.g. :class:`float`).
*parse_constant*, if specified, will be called with one of the
following strings: ``'-Infinity'``, ``'Infinity'``, ``'NaN'``. This
can be used to raise an exception if invalid JSON numbers are
encountered.
*strict* controls the parser's behavior when it encounters an
invalid control character in a string. The default setting of
``True`` means that unescaped control characters are parse errors, if
``False`` then control characters will be allowed in strings.
"""
if encoding is None:
encoding = DEFAULT_ENCODING
self.encoding = encoding
self.object_hook = object_hook
self.object_pairs_hook = object_pairs_hook
self.parse_float = parse_float or float
self.parse_int = parse_int or int
self.parse_constant = parse_constant or _CONSTANTS.__getitem__
self.strict = strict
self.parse_object = JSONObject
self.parse_array = JSONArray
self.parse_string = scanstring
self.memo = {}
self.scan_once = make_scanner(self)
def decode(self, s, _w=WHITESPACE.match, _PY3=PY3):
"""Return the Python representation of ``s`` (a ``str`` or ``unicode``
instance containing a JSON document)
"""
if _PY3 and isinstance(s, bytes):
s = str(s, self.encoding)
obj, end = self.raw_decode(s)
end = _w(s, end).end()
if end != len(s):
raise JSONDecodeError("Extra data", s, end, len(s))
return obj
def raw_decode(self, s, idx=0, _w=WHITESPACE.match, _PY3=PY3):
"""Decode a JSON document from ``s`` (a ``str`` or ``unicode``
beginning with a JSON document) and return a 2-tuple of the Python
representation and the index in ``s`` where the document ended.
Optionally, ``idx`` can be used to specify an offset in ``s`` where
the JSON document begins.
This can be used to decode a JSON document from a string that may
have extraneous data at the end.
"""
if idx < 0:
# Ensure that raw_decode bails on negative indexes, the regex
# would otherwise mask this behavior. #98
raise JSONDecodeError('Expecting value', s, idx)
if _PY3 and not isinstance(s, str):
raise TypeError("Input string must be text, not bytes")
# strip UTF-8 bom
if len(s) > idx:
ord0 = ord(s[idx])
if ord0 == 0xfeff:
idx += 1
elif ord0 == 0xef and s[idx:idx + 3] == '\xef\xbb\xbf':
idx += 3
return self.scan_once(s, idx=_w(s, idx).end())

722
simplejson/encoder.py
View File

@ -1,722 +0,0 @@
"""Implementation of JSONEncoder
"""
from __future__ import absolute_import
import re
from operator import itemgetter
# Do not import Decimal directly to avoid reload issues
import decimal
from .compat import unichr, binary_type, text_type, string_types, integer_types, PY3
def _import_speedups():
try:
from . import _speedups
return _speedups.encode_basestring_ascii, _speedups.make_encoder
except ImportError:
return None, None
c_encode_basestring_ascii, c_make_encoder = _import_speedups()
from .decoder import PosInf
from .raw_json import RawJSON
ESCAPE = re.compile(r'[\x00-\x1f\\"]')
ESCAPE_ASCII = re.compile(r'([\\"]|[^\ -~])')
HAS_UTF8 = re.compile(r'[\x80-\xff]')
ESCAPE_DCT = {
'\\': '\\\\',
'"': '\\"',
'\b': '\\b',
'\f': '\\f',
'\n': '\\n',
'\r': '\\r',
'\t': '\\t',
}
for i in range(0x20):
#ESCAPE_DCT.setdefault(chr(i), '\\u{0:04x}'.format(i))
ESCAPE_DCT.setdefault(chr(i), '\\u%04x' % (i,))
FLOAT_REPR = repr
def encode_basestring(s, _PY3=PY3, _q=u'"'):
"""Return a JSON representation of a Python string
"""
if _PY3:
if isinstance(s, bytes):
s = str(s, 'utf-8')
elif type(s) is not str:
# convert an str subclass instance to exact str
# raise a TypeError otherwise
s = str.__str__(s)
else:
if isinstance(s, str) and HAS_UTF8.search(s) is not None:
s = unicode(s, 'utf-8')
elif type(s) not in (str, unicode):
# convert an str subclass instance to exact str
# convert a unicode subclass instance to exact unicode
# raise a TypeError otherwise
if isinstance(s, str):
s = str.__str__(s)
else:
s = unicode.__getnewargs__(s)[0]
def replace(match):
return ESCAPE_DCT[match.group(0)]
return _q + ESCAPE.sub(replace, s) + _q
def py_encode_basestring_ascii(s, _PY3=PY3):
"""Return an ASCII-only JSON representation of a Python string
"""
if _PY3:
if isinstance(s, bytes):
s = str(s, 'utf-8')
elif type(s) is not str:
# convert an str subclass instance to exact str
# raise a TypeError otherwise
s = str.__str__(s)
else:
if isinstance(s, str) and HAS_UTF8.search(s) is not None:
s = unicode(s, 'utf-8')
elif type(s) not in (str, unicode):
# convert an str subclass instance to exact str
# convert a unicode subclass instance to exact unicode
# raise a TypeError otherwise
if isinstance(s, str):
s = str.__str__(s)
else:
s = unicode.__getnewargs__(s)[0]
def replace(match):
s = match.group(0)
try:
return ESCAPE_DCT[s]
except KeyError:
n = ord(s)
if n < 0x10000:
#return '\\u{0:04x}'.format(n)
return '\\u%04x' % (n,)
else:
# surrogate pair
n -= 0x10000
s1 = 0xd800 | ((n >> 10) & 0x3ff)
s2 = 0xdc00 | (n & 0x3ff)
#return '\\u{0:04x}\\u{1:04x}'.format(s1, s2)
return '\\u%04x\\u%04x' % (s1, s2)
return '"' + str(ESCAPE_ASCII.sub(replace, s)) + '"'
encode_basestring_ascii = (
c_encode_basestring_ascii or py_encode_basestring_ascii)
class JSONEncoder(object):
"""Extensible JSON <http://json.org> encoder for Python data structures.
Supports the following objects and types by default:
+-------------------+---------------+
| Python | JSON |
+===================+===============+
| dict, namedtuple | object |
+-------------------+---------------+
| list, tuple | array |
+-------------------+---------------+
| str, unicode | string |
+-------------------+---------------+
| int, long, float | number |
+-------------------+---------------+
| True | true |
+-------------------+---------------+
| False | false |
+-------------------+---------------+
| None | null |
+-------------------+---------------+
To extend this to recognize other objects, subclass and implement a
``.default()`` method with another method that returns a serializable
object for ``o`` if possible, otherwise it should call the superclass
implementation (to raise ``TypeError``).
"""
item_separator = ', '
key_separator = ': '
def __init__(self, skipkeys=False, ensure_ascii=True,
check_circular=True, allow_nan=True, sort_keys=False,
indent=None, separators=None, encoding='utf-8', default=None,
use_decimal=True, namedtuple_as_object=True,
tuple_as_array=True, bigint_as_string=False,
item_sort_key=None, for_json=False, ignore_nan=False,
int_as_string_bitcount=None, iterable_as_array=False):
"""Constructor for JSONEncoder, with sensible defaults.
If skipkeys is false, then it is a TypeError to attempt
encoding of keys that are not str, int, long, float or None. If
skipkeys is True, such items are simply skipped.
If ensure_ascii is true, the output is guaranteed to be str
objects with all incoming unicode characters escaped. If
ensure_ascii is false, the output will be unicode object.
If check_circular is true, then lists, dicts, and custom encoded
objects will be checked for circular references during encoding to
prevent an infinite recursion (which would cause an OverflowError).
Otherwise, no such check takes place.
If allow_nan is true, then NaN, Infinity, and -Infinity will be
encoded as such. This behavior is not JSON specification compliant,
but is consistent with most JavaScript based encoders and decoders.
Otherwise, it will be a ValueError to encode such floats.
If sort_keys is true, then the output of dictionaries will be
sorted by key; this is useful for regression tests to ensure
that JSON serializations can be compared on a day-to-day basis.
If indent is a string, then JSON array elements and object members
will be pretty-printed with a newline followed by that string repeated
for each level of nesting. ``None`` (the default) selects the most compact
representation without any newlines. For backwards compatibility with
versions of simplejson earlier than 2.1.0, an integer is also accepted
and is converted to a string with that many spaces.
If specified, separators should be an (item_separator, key_separator)
tuple. The default is (', ', ': ') if *indent* is ``None`` and
(',', ': ') otherwise. To get the most compact JSON representation,
you should specify (',', ':') to eliminate whitespace.
If specified, default is a function that gets called for objects
that can't otherwise be serialized. It should return a JSON encodable
version of the object or raise a ``TypeError``.
If encoding is not None, then all input strings will be
transformed into unicode using that encoding prior to JSON-encoding.
The default is UTF-8.
If use_decimal is true (default: ``True``), ``decimal.Decimal`` will
be supported directly by the encoder. For the inverse, decode JSON
with ``parse_float=decimal.Decimal``.
If namedtuple_as_object is true (the default), objects with
``_asdict()`` methods will be encoded as JSON objects.
If tuple_as_array is true (the default), tuple (and subclasses) will
be encoded as JSON arrays.
If *iterable_as_array* is true (default: ``False``),
any object not in the above table that implements ``__iter__()``
will be encoded as a JSON array.
If bigint_as_string is true (not the default), ints 2**53 and higher
or lower than -2**53 will be encoded as strings. This is to avoid the
rounding that happens in Javascript otherwise.
If int_as_string_bitcount is a positive number (n), then int of size
greater than or equal to 2**n or lower than or equal to -2**n will be
encoded as strings.
If specified, item_sort_key is a callable used to sort the items in
each dictionary. This is useful if you want to sort items other than
in alphabetical order by key.
If for_json is true (not the default), objects with a ``for_json()``
method will use the return value of that method for encoding as JSON
instead of the object.
If *ignore_nan* is true (default: ``False``), then out of range
:class:`float` values (``nan``, ``inf``, ``-inf``) will be serialized
as ``null`` in compliance with the ECMA-262 specification. If true,
this will override *allow_nan*.
"""
self.skipkeys = skipkeys
self.ensure_ascii = ensure_ascii
self.check_circular = check_circular
self.allow_nan = allow_nan
self.sort_keys = sort_keys
self.use_decimal = use_decimal
self.namedtuple_as_object = namedtuple_as_object
self.tuple_as_array = tuple_as_array
self.iterable_as_array = iterable_as_array
self.bigint_as_string = bigint_as_string
self.item_sort_key = item_sort_key
self.for_json = for_json
self.ignore_nan = ignore_nan
self.int_as_string_bitcount = int_as_string_bitcount
if indent is not None and not isinstance(indent, string_types):
indent = indent * ' '
self.indent = indent
if separators is not None:
self.item_separator, self.key_separator = separators
elif indent is not None:
self.item_separator = ','
if default is not None:
self.default = default
self.encoding = encoding
def default(self, o):
"""Implement this method in a subclass such that it returns
a serializable object for ``o``, or calls the base implementation
(to raise a ``TypeError``).
For example, to support arbitrary iterators, you could
implement default like this::
def default(self, o):
try:
iterable = iter(o)
except TypeError:
pass
else:
return list(iterable)
return JSONEncoder.default(self, o)
"""
raise TypeError('Object of type %s is not JSON serializable' %
o.__class__.__name__)
def encode(self, o):
"""Return a JSON string representation of a Python data structure.
>>> from simplejson import JSONEncoder
>>> JSONEncoder().encode({"foo": ["bar", "baz"]})
'{"foo": ["bar", "baz"]}'
"""
# This is for extremely simple cases and benchmarks.
if isinstance(o, binary_type):
_encoding = self.encoding
if (_encoding is not None and not (_encoding == 'utf-8')):
o = text_type(o, _encoding)
if isinstance(o, string_types):
if self.ensure_ascii:
return encode_basestring_ascii(o)
else:
return encode_basestring(o)
# This doesn't pass the iterator directly to ''.join() because the
# exceptions aren't as detailed. The list call should be roughly
# equivalent to the PySequence_Fast that ''.join() would do.
chunks = self.iterencode(o, _one_shot=True)
if not isinstance(chunks, (list, tuple)):
chunks = list(chunks)
if self.ensure_ascii:
return ''.join(chunks)
else:
return u''.join(chunks)
def iterencode(self, o, _one_shot=False):
"""Encode the given object and yield each string
representation as available.
For example::
for chunk in JSONEncoder().iterencode(bigobject):
mysocket.write(chunk)
"""
if self.check_circular:
markers = {}
else:
markers = None
if self.ensure_ascii:
_encoder = encode_basestring_ascii
else:
_encoder = encode_basestring
if self.encoding != 'utf-8' and self.encoding is not None:
def _encoder(o, _orig_encoder=_encoder, _encoding=self.encoding):
if isinstance(o, binary_type):
o = text_type(o, _encoding)
return _orig_encoder(o)
def floatstr(o, allow_nan=self.allow_nan, ignore_nan=self.ignore_nan,
_repr=FLOAT_REPR, _inf=PosInf, _neginf=-PosInf):
# Check for specials. Note that this type of test is processor
# and/or platform-specific, so do tests which don't depend on
# the internals.
if o != o:
text = 'NaN'
elif o == _inf:
text = 'Infinity'
elif o == _neginf:
text = '-Infinity'
else:
if type(o) != float:
# See #118, do not trust custom str/repr
o = float(o)
return _repr(o)
if ignore_nan:
text = 'null'
elif not allow_nan:
raise ValueError(
"Out of range float values are not JSON compliant: " +
repr(o))
return text
key_memo = {}
int_as_string_bitcount = (
53 if self.bigint_as_string else self.int_as_string_bitcount)
if (_one_shot and c_make_encoder is not None
and self.indent is None):
_iterencode = c_make_encoder(
markers, self.default, _encoder, self.indent,
self.key_separator, self.item_separator, self.sort_keys,
self.skipkeys, self.allow_nan, key_memo, self.use_decimal,
self.namedtuple_as_object, self.tuple_as_array,
int_as_string_bitcount,
self.item_sort_key, self.encoding, self.for_json,
self.ignore_nan, decimal.Decimal, self.iterable_as_array)
else:
_iterencode = _make_iterencode(
markers, self.default, _encoder, self.indent, floatstr,
self.key_separator, self.item_separator, self.sort_keys,
self.skipkeys, _one_shot, self.use_decimal,
self.namedtuple_as_object, self.tuple_as_array,
int_as_string_bitcount,
self.item_sort_key, self.encoding, self.for_json,
self.iterable_as_array, Decimal=decimal.Decimal)
try:
return _iterencode(o, 0)
finally:
key_memo.clear()
class JSONEncoderForHTML(JSONEncoder):
"""An encoder that produces JSON safe to embed in HTML.
To embed JSON content in, say, a script tag on a web page, the
characters &, < and > should be escaped. They cannot be escaped
with the usual entities (e.g. &amp;) because they are not expanded
within <script> tags.
This class also escapes the line separator and paragraph separator
characters U+2028 and U+2029, irrespective of the ensure_ascii setting,
as these characters are not valid in JavaScript strings (see
http://timelessrepo.com/json-isnt-a-javascript-subset).
"""
def encode(self, o):
# Override JSONEncoder.encode because it has hacks for
# performance that make things more complicated.
chunks = self.iterencode(o, True)
if self.ensure_ascii:
return ''.join(chunks)
else:
return u''.join(chunks)
def iterencode(self, o, _one_shot=False):
chunks = super(JSONEncoderForHTML, self).iterencode(o, _one_shot)
for chunk in chunks:
chunk = chunk.replace('&', '\\u0026')
chunk = chunk.replace('<', '\\u003c')
chunk = chunk.replace('>', '\\u003e')
if not self.ensure_ascii:
chunk = chunk.replace(u'\u2028', '\\u2028')
chunk = chunk.replace(u'\u2029', '\\u2029')
yield chunk
def _make_iterencode(markers, _default, _encoder, _indent, _floatstr,
_key_separator, _item_separator, _sort_keys, _skipkeys, _one_shot,
_use_decimal, _namedtuple_as_object, _tuple_as_array,
_int_as_string_bitcount, _item_sort_key,
_encoding,_for_json,
_iterable_as_array,
## HACK: hand-optimized bytecode; turn globals into locals
_PY3=PY3,
ValueError=ValueError,
string_types=string_types,
Decimal=None,
dict=dict,
float=float,
id=id,
integer_types=integer_types,
isinstance=isinstance,
list=list,
str=str,
tuple=tuple,
iter=iter,
):
if _use_decimal and Decimal is None:
Decimal = decimal.Decimal
if _item_sort_key and not callable(_item_sort_key):
raise TypeError("item_sort_key must be None or callable")
elif _sort_keys and not _item_sort_key:
_item_sort_key = itemgetter(0)
if (_int_as_string_bitcount is not None and
(_int_as_string_bitcount <= 0 or
not isinstance(_int_as_string_bitcount, integer_types))):
raise TypeError("int_as_string_bitcount must be a positive integer")
def _encode_int(value):
skip_quoting = (
_int_as_string_bitcount is None
or
_int_as_string_bitcount < 1
)
if type(value) not in integer_types:
# See #118, do not trust custom str/repr
value = int(value)
if (
skip_quoting or
(-1 << _int_as_string_bitcount)
< value <
(1 << _int_as_string_bitcount)
):
return str(value)
return '"' + str(value) + '"'
def _iterencode_list(lst, _current_indent_level):
if not lst:
yield '[]'
return
if markers is not None:
markerid = id(lst)
if markerid in markers:
raise ValueError("Circular reference detected")
markers[markerid] = lst
buf = '['
if _indent is not None:
_current_indent_level += 1
newline_indent = '\n' + (_indent * _current_indent_level)
separator = _item_separator + newline_indent
buf += newline_indent
else:
newline_indent = None
separator = _item_separator
first = True
for value in lst:
if first:
first = False
else:
buf = separator
if isinstance(value, string_types):
yield buf + _encoder(value)
elif _PY3 and isinstance(value, bytes) and _encoding is not None:
yield buf + _encoder(value)
elif isinstance(value, RawJSON):
yield buf + value.encoded_json
elif value is None:
yield buf + 'null'
elif value is True:
yield buf + 'true'
elif value is False:
yield buf + 'false'
elif isinstance(value, integer_types):
yield buf + _encode_int(value)
elif isinstance(value, float):
yield buf + _floatstr(value)
elif _use_decimal and isinstance(value, Decimal):
yield buf + str(value)
else:
yield buf
for_json = _for_json and getattr(value, 'for_json', None)
if for_json and callable(for_json):
chunks = _iterencode(for_json(), _current_indent_level)
elif isinstance(value, list):
chunks = _iterencode_list(value, _current_indent_level)
else:
_asdict = _namedtuple_as_object and getattr(value, '_asdict', None)
if _asdict and callable(_asdict):
chunks = _iterencode_dict(_asdict(),
_current_indent_level)
elif _tuple_as_array and isinstance(value, tuple):
chunks = _iterencode_list(value, _current_indent_level)
elif isinstance(value, dict):
chunks = _iterencode_dict(value, _current_indent_level)
else:
chunks = _iterencode(value, _current_indent_level)
for chunk in chunks:
yield chunk
if first:
# iterable_as_array misses the fast path at the top
yield '[]'
else:
if newline_indent is not None:
_current_indent_level -= 1
yield '\n' + (_indent * _current_indent_level)
yield ']'
if markers is not None:
del markers[markerid]
def _stringify_key(key):
if isinstance(key, string_types): # pragma: no cover
pass
elif _PY3 and isinstance(key, bytes) and _encoding is not None:
key = str(key, _encoding)
elif isinstance(key, float):
key = _floatstr(key)
elif key is True:
key = 'true'
elif key is False:
key = 'false'
elif key is None:
key = 'null'
elif isinstance(key, integer_types):
if type(key) not in integer_types:
# See #118, do not trust custom str/repr
key = int(key)
key = str(key)
elif _use_decimal and isinstance(key, Decimal):
key = str(key)
elif _skipkeys:
key = None
else:
raise TypeError('keys must be str, int, float, bool or None, '
'not %s' % key.__class__.__name__)
return key
def _iterencode_dict(dct, _current_indent_level):
if not dct:
yield '{}'
return
if markers is not None:
markerid = id(dct)
if markerid in markers:
raise ValueError("Circular reference detected")
markers[markerid] = dct
yield '{'
if _indent is not None:
_current_indent_level += 1
newline_indent = '\n' + (_indent * _current_indent_level)
item_separator = _item_separator + newline_indent
yield newline_indent
else:
newline_indent = None
item_separator = _item_separator
first = True
if _PY3:
iteritems = dct.items()
else:
iteritems = dct.iteritems()
if _item_sort_key:
items = []
for k, v in dct.items():
if not isinstance(k, string_types):
k = _stringify_key(k)
if k is None:
continue
items.append((k, v))
items.sort(key=_item_sort_key)
else:
items = iteritems
for key, value in items:
if not (_item_sort_key or isinstance(key, string_types)):
key = _stringify_key(key)
if key is None:
# _skipkeys must be True
continue
if first:
first = False
else:
yield item_separator
yield _encoder(key)
yield _key_separator
if isinstance(value, string_types):
yield _encoder(value)
elif _PY3 and isinstance(value, bytes) and _encoding is not None:
yield _encoder(value)
elif isinstance(value, RawJSON):
yield value.encoded_json
elif value is None:
yield 'null'
elif value is True:
yield 'true'
elif value is False:
yield 'false'
elif isinstance(value, integer_types):
yield _encode_int(value)
elif isinstance(value, float):
yield _floatstr(value)
elif _use_decimal and isinstance(value, Decimal):
yield str(value)
else:
for_json = _for_json and getattr(value, 'for_json', None)
if for_json and callable(for_json):
chunks = _iterencode(for_json(), _current_indent_level)
elif isinstance(value, list):
chunks = _iterencode_list(value, _current_indent_level)
else:
_asdict = _namedtuple_as_object and getattr(value, '_asdict', None)
if _asdict and callable(_asdict):
chunks = _iterencode_dict(_asdict(),
_current_indent_level)
elif _tuple_as_array and isinstance(value, tuple):
chunks = _iterencode_list(value, _current_indent_level)
elif isinstance(value, dict):
chunks = _iterencode_dict(value, _current_indent_level)
else:
chunks = _iterencode(value, _current_indent_level)
for chunk in chunks:
yield chunk
if newline_indent is not None:
_current_indent_level -= 1
yield '\n' + (_indent * _current_indent_level)
yield '}'
if markers is not None:
del markers[markerid]
def _iterencode(o, _current_indent_level):
if isinstance(o, string_types):
yield _encoder(o)
elif _PY3 and isinstance(o, bytes) and _encoding is not None:
yield _encoder(o)
elif isinstance(o, RawJSON):
yield o.encoded_json
elif o is None:
yield 'null'
elif o is True:
yield 'true'
elif o is False:
yield 'false'
elif isinstance(o, integer_types):
yield _encode_int(o)
elif isinstance(o, float):
yield _floatstr(o)
else:
for_json = _for_json and getattr(o, 'for_json', None)
if for_json and callable(for_json):
for chunk in _iterencode(for_json(), _current_indent_level):
yield chunk
elif isinstance(o, list):
for chunk in _iterencode_list(o, _current_indent_level):
yield chunk
else:
_asdict = _namedtuple_as_object and getattr(o, '_asdict', None)
if _asdict and callable(_asdict):
for chunk in _iterencode_dict(_asdict(),
_current_indent_level):
yield chunk
elif (_tuple_as_array and isinstance(o, tuple)):
for chunk in _iterencode_list(o, _current_indent_level):
yield chunk
elif isinstance(o, dict):
for chunk in _iterencode_dict(o, _current_indent_level):
yield chunk
elif _use_decimal and isinstance(o, Decimal):
yield str(o)
else:
while _iterable_as_array:
# Markers are not checked here because it is valid for
# an iterable to return self.
try:
o = iter(o)
except TypeError:
break
for chunk in _iterencode_list(o, _current_indent_level):
yield chunk
return
if markers is not None:
markerid = id(o)
if markerid in markers:
raise ValueError("Circular reference detected")
markers[markerid] = o
o = _default(o)
for chunk in _iterencode(o, _current_indent_level):
yield chunk
if markers is not None:
del markers[markerid]
return _iterencode

53
simplejson/errors.py
View File

@ -1,53 +0,0 @@
"""Error classes used by simplejson
"""
__all__ = ['JSONDecodeError']
def linecol(doc, pos):
lineno = doc.count('\n', 0, pos) + 1
if lineno == 1:
colno = pos + 1
else:
colno = pos - doc.rindex('\n', 0, pos)
return lineno, colno
def errmsg(msg, doc, pos, end=None):
lineno, colno = linecol(doc, pos)
msg = msg.replace('%r', repr(doc[pos:pos + 1]))
if end is None:
fmt = '%s: line %d column %d (char %d)'
return fmt % (msg, lineno, colno, pos)
endlineno, endcolno = linecol(doc, end)
fmt = '%s: line %d column %d - line %d column %d (char %d - %d)'
return fmt % (msg, lineno, colno, endlineno, endcolno, pos, end)
class JSONDecodeError(ValueError):
"""Subclass of ValueError with the following additional properties:
msg: The unformatted error message
doc: The JSON document being parsed
pos: The start index of doc where parsing failed
end: The end index of doc where parsing failed (may be None)
lineno: The line corresponding to pos
colno: The column corresponding to pos
endlineno: The line corresponding to end (may be None)
endcolno: The column corresponding to end (may be None)
"""
# Note that this exception is used from _speedups
def __init__(self, msg, doc, pos, end=None):
ValueError.__init__(self, errmsg(msg, doc, pos, end=end))
self.msg = msg
self.doc = doc
self.pos = pos
self.end = end
self.lineno, self.colno = linecol(doc, pos)
if end is not None:
self.endlineno, self.endcolno = linecol(doc, end)
else:
self.endlineno, self.endcolno = None, None
def __reduce__(self):
return self.__class__, (self.msg, self.doc, self.pos, self.end)

103
simplejson/ordered_dict.py
View File

@ -1,103 +0,0 @@
"""Drop-in replacement for collections.OrderedDict by Raymond Hettinger
http://code.activestate.com/recipes/576693/
"""
from UserDict import DictMixin
class OrderedDict(dict, DictMixin):
def __init__(self, *args, **kwds):
if len(args) > 1:
raise TypeError('expected at most 1 arguments, got %d' % len(args))
try:
self.__end
except AttributeError:
self.clear()
self.update(*args, **kwds)
def clear(self):
self.__end = end = []
end += [None, end, end] # sentinel node for doubly linked list
self.__map = {} # key --> [key, prev, next]
dict.clear(self)
def __setitem__(self, key, value):
if key not in self:
end = self.__end
curr = end[1]
curr[2] = end[1] = self.__map[key] = [key, curr, end]
dict.__setitem__(self, key, value)
def __delitem__(self, key):
dict.__delitem__(self, key)
key, prev, next = self.__map.pop(key)
prev[2] = next
next[1] = prev
def __iter__(self):
end = self.__end
curr = end[2]
while curr is not end:
yield curr[0]
curr = curr[2]
def __reversed__(self):
end = self.__end
curr = end[1]
while curr is not end:
yield curr[0]
curr = curr[1]
def popitem(self, last=True):
if not self:
raise KeyError('dictionary is empty')
key = reversed(self).next() if last else iter(self).next()
value = self.pop(key)
return key, value
def __reduce__(self):
items = [[k, self[k]] for k in self]
tmp = self.__map, self.__end
del self.__map, self.__end
inst_dict = vars(self).copy()
self.__map, self.__end = tmp
if inst_dict:
return (self.__class__, (items,), inst_dict)
return self.__class__, (items,)
def keys(self):
return list(self)
setdefault = DictMixin.setdefault
update = DictMixin.update
pop = DictMixin.pop
values = DictMixin.values
items = DictMixin.items
iterkeys = DictMixin.iterkeys
itervalues = DictMixin.itervalues
iteritems = DictMixin.iteritems
def __repr__(self):
if not self:
return '%s()' % (self.__class__.__name__,)
return '%s(%r)' % (self.__class__.__name__, self.items())
def copy(self):
return self.__class__(self)
@classmethod
def fromkeys(cls, iterable, value=None):
d = cls()
for key in iterable:
d[key] = value
return d
def __eq__(self, other):
if isinstance(other, OrderedDict):
return len(self)==len(other) and \
all(p==q for p, q in zip(self.items(), other.items()))
return dict.__eq__(self, other)
def __ne__(self, other):
return not self == other

9
simplejson/raw_json.py
View File

@ -1,9 +0,0 @@
"""Implementation of RawJSON
"""
class RawJSON(object):
"""Wrap an encoded JSON document for direct embedding in the output
"""
def __init__(self, encoded_json):
self.encoded_json = encoded_json

85
simplejson/scanner.py
View File

@ -1,85 +0,0 @@
"""JSON token scanner
"""
import re
from .errors import JSONDecodeError
def _import_c_make_scanner():
try:
from ._speedups import make_scanner
return make_scanner
except ImportError:
return None
c_make_scanner = _import_c_make_scanner()
__all__ = ['make_scanner', 'JSONDecodeError']
NUMBER_RE = re.compile(
r'(-?(?:0|[1-9]\d*))(\.\d+)?([eE][-+]?\d+)?',
(re.VERBOSE | re.MULTILINE | re.DOTALL))
def py_make_scanner(context):
parse_object = context.parse_object
parse_array = context.parse_array
parse_string = context.parse_string
match_number = NUMBER_RE.match
encoding = context.encoding
strict = context.strict
parse_float = context.parse_float
parse_int = context.parse_int
parse_constant = context.parse_constant
object_hook = context.object_hook
object_pairs_hook = context.object_pairs_hook
memo = context.memo
def _scan_once(string, idx):
errmsg = 'Expecting value'
try:
nextchar = string[idx]
except IndexError:
raise JSONDecodeError(errmsg, string, idx)
if nextchar == '"':
return parse_string(string, idx + 1, encoding, strict)
elif nextchar == '{':
return parse_object((string, idx + 1), encoding, strict,
_scan_once, object_hook, object_pairs_hook, memo)
elif nextchar == '[':
return parse_array((string, idx + 1), _scan_once)
elif nextchar == 'n' and string[idx:idx + 4] == 'null':
return None, idx + 4
elif nextchar == 't' and string[idx:idx + 4] == 'true':
return True, idx + 4
elif nextchar == 'f' and string[idx:idx + 5] == 'false':
return False, idx + 5
m = match_number(string, idx)
if m is not None:
integer, frac, exp = m.groups()
if frac or exp:
res = parse_float(integer + (frac or '') + (exp or ''))
else:
res = parse_int(integer)
return res, m.end()
elif nextchar == 'N' and string[idx:idx + 3] == 'NaN':
return parse_constant('NaN'), idx + 3
elif nextchar == 'I' and string[idx:idx + 8] == 'Infinity':
return parse_constant('Infinity'), idx + 8
elif nextchar == '-' and string[idx:idx + 9] == '-Infinity':
return parse_constant('-Infinity'), idx + 9
else:
raise JSONDecodeError(errmsg, string, idx)
def scan_once(string, idx):
if idx < 0:
# Ensure the same behavior as the C speedup, otherwise
# this would work for *some* negative string indices due
# to the behavior of __getitem__ for strings. #98
raise JSONDecodeError('Expecting value', string, idx)
try:
return _scan_once(string, idx)
finally:
memo.clear()
return scan_once
make_scanner = c_make_scanner or py_make_scanner

74
simplejson/tests/__init__.py
View File

@ -1,74 +0,0 @@
from __future__ import absolute_import
import unittest
import sys
import os
class NoExtensionTestSuite(unittest.TestSuite):
def run(self, result):
import simplejson
simplejson._toggle_speedups(False)
result = unittest.TestSuite.run(self, result)
simplejson._toggle_speedups(True)
return result
class TestMissingSpeedups(unittest.TestCase):
def runTest(self):
if hasattr(sys, 'pypy_translation_info'):
"PyPy doesn't need speedups! :)"
elif hasattr(self, 'skipTest'):
self.skipTest('_speedups.so is missing!')
def additional_tests(suite=None):
import simplejson
import simplejson.encoder
import simplejson.decoder
if suite is None:
suite = unittest.TestSuite()
try:
import doctest
except ImportError:
if sys.version_info < (2, 7):
# doctests in 2.6 depends on cStringIO
return suite
raise
for mod in (simplejson, simplejson.encoder, simplejson.decoder):
suite.addTest(doctest.DocTestSuite(mod))
suite.addTest(doctest.DocFileSuite('../../index.rst'))
return suite
def all_tests_suite():
def get_suite():
suite_names = [
'simplejson.tests.%s' % (os.path.splitext(f)[0],)
for f in os.listdir(os.path.dirname(__file__))
if f.startswith('test_') and f.endswith('.py')
]
return additional_tests(
unittest.TestLoader().loadTestsFromNames(suite_names))
suite = get_suite()
import simplejson
if simplejson._import_c_make_encoder() is None:
suite.addTest(TestMissingSpeedups())
else:
suite = unittest.TestSuite([
suite,
NoExtensionTestSuite([get_suite()]),
])
return suite
def main():
runner = unittest.TextTestRunner(verbosity=1 + sys.argv.count('-v'))
suite = all_tests_suite()
raise SystemExit(not runner.run(suite).wasSuccessful())
if __name__ == '__main__':
import os
import sys
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))))
main()

67
simplejson/tests/test_bigint_as_string.py
View File

@ -1,67 +0,0 @@
from unittest import TestCase
import simplejson as json
class TestBigintAsString(TestCase):
# Python 2.5, at least the one that ships on Mac OS X, calculates
# 2 ** 53 as 0! It manages to calculate 1 << 53 correctly.
values = [(200, 200),
((1 << 53) - 1, 9007199254740991),
((1 << 53), '9007199254740992'),
((1 << 53) + 1, '9007199254740993'),
(-100, -100),
((-1 << 53), '-9007199254740992'),
((-1 << 53) - 1, '-9007199254740993'),
((-1 << 53) + 1, -9007199254740991)]
options = (
{"bigint_as_string": True},
{"int_as_string_bitcount": 53}
)
def test_ints(self):
for opts in self.options:
for val, expect in self.values:
self.assertEqual(
val,
json.loads(json.dumps(val)))
self.assertEqual(
expect,
json.loads(json.dumps(val, **opts)))
def test_lists(self):
for opts in self.options:
for val, expect in self.values:
val = [val, val]
expect = [expect, expect]
self.assertEqual(
val,
json.loads(json.dumps(val)))
self.assertEqual(
expect,
json.loads(json.dumps(val, **opts)))
def test_dicts(self):
for opts in self.options:
for val, expect in self.values:
val = {'k': val}
expect = {'k': expect}
self.assertEqual(
val,
json.loads(json.dumps(val)))
self.assertEqual(
expect,
json.loads(json.dumps(val, **opts)))
def test_dict_keys(self):
for opts in self.options:
for val, _ in self.values:
expect = {str(val): 'value'}
val = {val: 'value'}
self.assertEqual(
expect,
json.loads(json.dumps(val)))
self.assertEqual(
expect,
json.loads(json.dumps(val, **opts)))

73
simplejson/tests/test_bitsize_int_as_string.py
View File

@ -1,73 +0,0 @@
from unittest import TestCase
import simplejson as json
class TestBitSizeIntAsString(TestCase):
# Python 2.5, at least the one that ships on Mac OS X, calculates
# 2 ** 31 as 0! It manages to calculate 1 << 31 correctly.
values = [
(200, 200),
((1 << 31) - 1, (1 << 31) - 1),
((1 << 31), str(1 << 31)),
((1 << 31) + 1, str((1 << 31) + 1)),
(-100, -100),
((-1 << 31), str(-1 << 31)),
((-1 << 31) - 1, str((-1 << 31) - 1)),
((-1 << 31) + 1, (-1 << 31) + 1),
]
def test_invalid_counts(self):
for n in ['foo', -1, 0, 1.0]:
self.assertRaises(
TypeError,
json.dumps, 0, int_as_string_bitcount=n)
def test_ints_outside_range_fails(self):
self.assertNotEqual(
str(1 << 15),
json.loads(json.dumps(1 << 15, int_as_string_bitcount=16)),
)
def test_ints(self):
for val, expect in self.values:
self.assertEqual(
val,
json.loads(json.dumps(val)))
self.assertEqual(
expect,
json.loads(json.dumps(val, int_as_string_bitcount=31)),
)
def test_lists(self):
for val, expect in self.values:
val = [val, val]
expect = [expect, expect]
self.assertEqual(
val,
json.loads(json.dumps(val)))
self.assertEqual(
expect,
json.loads(json.dumps(val, int_as_string_bitcount=31)))
def test_dicts(self):
for val, expect in self.values:
val = {'k': val}
expect = {'k': expect}
self.assertEqual(
val,
json.loads(json.dumps(val)))
self.assertEqual(
expect,
json.loads(json.dumps(val, int_as_string_bitcount=31)))
def test_dict_keys(self):
for val, _ in self.values:
expect = {str(val): 'value'}
val = {val: 'value'}
self.assertEqual(
expect,
json.loads(json.dumps(val)))
self.assertEqual(
expect,
json.loads(json.dumps(val, int_as_string_bitcount=31)))

30
simplejson/tests/test_check_circular.py
View File

@ -1,30 +0,0 @@
from unittest import TestCase
import simplejson as json
def default_iterable(obj):
return list(obj)
class TestCheckCircular(TestCase):
def test_circular_dict(self):
dct = {}
dct['a'] = dct
self.assertRaises(ValueError, json.dumps, dct)
def test_circular_list(self):
lst = []
lst.append(lst)
self.assertRaises(ValueError, json.dumps, lst)
def test_circular_composite(self):
dct2 = {}
dct2['a'] = []
dct2['a'].append(dct2)
self.assertRaises(ValueError, json.dumps, dct2)
def test_circular_default(self):
json.dumps([set()], default=default_iterable)
self.assertRaises(TypeError, json.dumps, [set()])
def test_circular_off_default(self):
json.dumps([set()], default=default_iterable, check_circular=False)
self.assertRaises(TypeError, json.dumps, [set()], check_circular=False)

71
simplejson/tests/test_decimal.py
View File

@ -1,71 +0,0 @@
import decimal
from decimal import Decimal
from unittest import TestCase
from simplejson.compat import StringIO, reload_module
import simplejson as json
class TestDecimal(TestCase):
NUMS = "1.0", "10.00", "1.1", "1234567890.1234567890", "500"
def dumps(self, obj, **kw):
sio = StringIO()
json.dump(obj, sio, **kw)
res = json.dumps(obj, **kw)
self.assertEqual(res, sio.getvalue())
return res
def loads(self, s, **kw):
sio = StringIO(s)
res = json.loads(s, **kw)
self.assertEqual(res, json.load(sio, **kw))
return res
def test_decimal_encode(self):
for d in map(Decimal, self.NUMS):
self.assertEqual(self.dumps(d, use_decimal=True), str(d))
def test_decimal_decode(self):
for s in self.NUMS:
self.assertEqual(self.loads(s, parse_float=Decimal), Decimal(s))
def test_stringify_key(self):
for d in map(Decimal, self.NUMS):
v = {d: d}
self.assertEqual(
self.loads(
self.dumps(v, use_decimal=True), parse_float=Decimal),
{str(d): d})
def test_decimal_roundtrip(self):
for d in map(Decimal, self.NUMS):
# The type might not be the same (int and Decimal) but they
# should still compare equal.
for v in [d, [d], {'': d}]:
self.assertEqual(
self.loads(
self.dumps(v, use_decimal=True), parse_float=Decimal),
v)
def test_decimal_defaults(self):
d = Decimal('1.1')
# use_decimal=True is the default
self.assertRaises(TypeError, json.dumps, d, use_decimal=False)
self.assertEqual('1.1', json.dumps(d))
self.assertEqual('1.1', json.dumps(d, use_decimal=True))
self.assertRaises(TypeError, json.dump, d, StringIO(),
use_decimal=False)
sio = StringIO()
json.dump(d, sio)
self.assertEqual('1.1', sio.getvalue())
sio = StringIO()
json.dump(d, sio, use_decimal=True)
self.assertEqual('1.1', sio.getvalue())
def test_decimal_reload(self):
# Simulate a subinterpreter that reloads the Python modules but not
# the C code https://github.com/simplejson/simplejson/issues/34
global Decimal
Decimal = reload_module(decimal).Decimal
import simplejson.encoder
simplejson.encoder.Decimal = Decimal
self.test_decimal_roundtrip()

119
simplejson/tests/test_decode.py
View File

@ -1,119 +0,0 @@
from __future__ import absolute_import
import decimal
from unittest import TestCase
import simplejson as json
from simplejson.compat import StringIO, b, binary_type
from simplejson import OrderedDict
class MisbehavingBytesSubtype(binary_type):
def decode(self, encoding=None):
return "bad decode"
def __str__(self):
return "bad __str__"
def __bytes__(self):
return b("bad __bytes__")
class TestDecode(TestCase):
if not hasattr(TestCase, 'assertIs'):
def assertIs(self, a, b):
self.assertTrue(a is b, '%r is %r' % (a, b))
def test_decimal(self):
rval = json.loads('1.1', parse_float=decimal.Decimal)
self.assertTrue(isinstance(rval, decimal.Decimal))
self.assertEqual(rval, decimal.Decimal('1.1'))
def test_float(self):
rval = json.loads('1', parse_int=float)
self.assertTrue(isinstance(rval, float))
self.assertEqual(rval, 1.0)
def test_decoder_optimizations(self):
# Several optimizations were made that skip over calls to
# the whitespace regex, so this test is designed to try and
# exercise the uncommon cases. The array cases are already covered.
rval = json.loads('{ "key" : "value" , "k":"v" }')
self.assertEqual(rval, {"key":"value", "k":"v"})
def test_empty_objects(self):
s = '{}'
self.assertEqual(json.loads(s), eval(s))
s = '[]'
self.assertEqual(json.loads(s), eval(s))
s = '""'
self.assertEqual(json.loads(s), eval(s))
def test_object_pairs_hook(self):
s = '{"xkd":1, "kcw":2, "art":3, "hxm":4, "qrt":5, "pad":6, "hoy":7}'
p = [("xkd", 1), ("kcw", 2), ("art", 3), ("hxm", 4),
("qrt", 5), ("pad", 6), ("hoy", 7)]
self.assertEqual(json.loads(s), eval(s))
self.assertEqual(json.loads(s, object_pairs_hook=lambda x: x), p)
self.assertEqual(json.load(StringIO(s),
object_pairs_hook=lambda x: x), p)
od = json.loads(s, object_pairs_hook=OrderedDict)
self.assertEqual(od, OrderedDict(p))
self.assertEqual(type(od), OrderedDict)
# the object_pairs_hook takes priority over the object_hook
self.assertEqual(json.loads(s,
object_pairs_hook=OrderedDict,
object_hook=lambda x: None),
OrderedDict(p))
def check_keys_reuse(self, source, loads):
rval = loads(source)
(a, b), (c, d) = sorted(rval[0]), sorted(rval[1])
self.assertIs(a, c)
self.assertIs(b, d)
def test_keys_reuse_str(self):
s = u'[{"a_key": 1, "b_\xe9": 2}, {"a_key": 3, "b_\xe9": 4}]'.encode('utf8')
self.check_keys_reuse(s, json.loads)
def test_keys_reuse_unicode(self):
s = u'[{"a_key": 1, "b_\xe9": 2}, {"a_key": 3, "b_\xe9": 4}]'
self.check_keys_reuse(s, json.loads)
def test_empty_strings(self):
self.assertEqual(json.loads('""'), "")
self.assertEqual(json.loads(u'""'), u"")
self.assertEqual(json.loads('[""]'), [""])
self.assertEqual(json.loads(u'[""]'), [u""])
def test_raw_decode(self):
cls = json.decoder.JSONDecoder
self.assertEqual(
({'a': {}}, 9),
cls().raw_decode("{\"a\": {}}"))
# http://code.google.com/p/simplejson/issues/detail?id=85
self.assertEqual(
({'a': {}}, 9),
cls(object_pairs_hook=dict).raw_decode("{\"a\": {}}"))
# https://github.com/simplejson/simplejson/pull/38
self.assertEqual(
({'a': {}}, 11),
cls().raw_decode(" \n{\"a\": {}}"))
def test_bytes_decode(self):
cls = json.decoder.JSONDecoder
data = b('"\xe2\x82\xac"')
self.assertEqual(cls().decode(data), u'\u20ac')
self.assertEqual(cls(encoding='latin1').decode(data), u'\xe2\x82\xac')
self.assertEqual(cls(encoding=None).decode(data), u'\u20ac')
data = MisbehavingBytesSubtype(b('"\xe2\x82\xac"'))
self.assertEqual(cls().decode(data), u'\u20ac')
self.assertEqual(cls(encoding='latin1').decode(data), u'\xe2\x82\xac')
self.assertEqual(cls(encoding=None).decode(data), u'\u20ac')
def test_bounds_checking(self):
# https://github.com/simplejson/simplejson/issues/98
j = json.decoder.JSONDecoder()
for i in [4, 5, 6, -1, -2, -3, -4, -5, -6]:
self.assertRaises(ValueError, j.scan_once, '1234', i)
self.assertRaises(ValueError, j.raw_decode, '1234', i)
x, y = sorted(['128931233', '472389423'], key=id)
diff = id(x) - id(y)
self.assertRaises(ValueError, j.scan_once, y, diff)
self.assertRaises(ValueError, j.raw_decode, y, i)

9
simplejson/tests/test_default.py
View File

@ -1,9 +0,0 @@
from unittest import TestCase
import simplejson as json
class TestDefault(TestCase):
def test_default(self):
self.assertEqual(
json.dumps(type, default=repr),
json.dumps(repr(type)))

249
simplejson/tests/test_dump.py
View File

@ -1,249 +0,0 @@
from unittest import TestCase
from simplejson.compat import StringIO, long_type, b, binary_type, text_type, PY3
import simplejson as json
class MisbehavingTextSubtype(text_type):
def __str__(self):
return "FAIL!"
class MisbehavingBytesSubtype(binary_type):
def decode(self, encoding=None):
return "bad decode"
def __str__(self):
return "bad __str__"
def __bytes__(self):
return b("bad __bytes__")
def as_text_type(s):
if PY3 and isinstance(s, bytes):
return s.decode('ascii')
return s
def decode_iso_8859_15(b):
return b.decode('iso-8859-15')
class TestDump(TestCase):
def test_dump(self):
sio = StringIO()
json.dump({}, sio)
self.assertEqual(sio.getvalue(), '{}')
def test_constants(self):
for c in [None, True, False]:
self.assertTrue(json.loads(json.dumps(c)) is c)
self.assertTrue(json.loads(json.dumps([c]))[0] is c)
self.assertTrue(json.loads(json.dumps({'a': c}))['a'] is c)
def test_stringify_key(self):
items = [(b('bytes'), 'bytes'),
(1.0, '1.0'),
(10, '10'),
(True, 'true'),
(False, 'false'),
(None, 'null'),
(long_type(100), '100')]
for k, expect in items:
self.assertEqual(
json.loads(json.dumps({k: expect})),
{expect: expect})
self.assertEqual(
json.loads(json.dumps({k: expect}, sort_keys=True)),
{expect: expect})
self.assertRaises(TypeError, json.dumps, {json: 1})
for v in [{}, {'other': 1}, {b('derp'): 1, 'herp': 2}]:
for sort_keys in [False, True]:
v0 = dict(v)
v0[json] = 1
v1 = dict((as_text_type(key), val) for (key, val) in v.items())
self.assertEqual(
json.loads(json.dumps(v0, skipkeys=True, sort_keys=sort_keys)),
v1)
self.assertEqual(
json.loads(json.dumps({'': v0}, skipkeys=True, sort_keys=sort_keys)),
{'': v1})
self.assertEqual(
json.loads(json.dumps([v0], skipkeys=True, sort_keys=sort_keys)),
[v1])
def test_dumps(self):
self.assertEqual(json.dumps({}), '{}')
def test_encode_truefalse(self):
self.assertEqual(json.dumps(
{True: False, False: True}, sort_keys=True),
'{"false": true, "true": false}')
self.assertEqual(
json.dumps(
{2: 3.0,
4.0: long_type(5),
False: 1,
long_type(6): True,
"7": 0},
sort_keys=True),
'{"2": 3.0, "4.0": 5, "6": true, "7": 0, "false": 1}')
def test_ordered_dict(self):
# http://bugs.python.org/issue6105
items = [('one', 1), ('two', 2), ('three', 3), ('four', 4), ('five', 5)]
s = json.dumps(json.OrderedDict(items))
self.assertEqual(
s,
'{"one": 1, "two": 2, "three": 3, "four": 4, "five": 5}')
def test_indent_unknown_type_acceptance(self):
"""
A test against the regression mentioned at `github issue 29`_.
The indent parameter should accept any type which pretends to be
an instance of int or long when it comes to being multiplied by
strings, even if it is not actually an int or long, for
backwards compatibility.
.. _github issue 29:
http://github.com/simplejson/simplejson/issue/29
"""
class AwesomeInt(object):
"""An awesome reimplementation of integers"""
def __init__(self, *args, **kwargs):
if len(args) > 0:
# [construct from literals, objects, etc.]
# ...
# Finally, if args[0] is an integer, store it
if isinstance(args[0], int):
self._int = args[0]
# [various methods]
def __mul__(self, other):
# [various ways to multiply AwesomeInt objects]
# ... finally, if the right-hand operand is not awesome enough,
# try to do a normal integer multiplication
if hasattr(self, '_int'):
return self._int * other
else:
raise NotImplementedError("To do non-awesome things with"
" this object, please construct it from an integer!")
s = json.dumps([0, 1, 2], indent=AwesomeInt(3))
self.assertEqual(s, '[\n 0,\n 1,\n 2\n]')
def test_accumulator(self):
# the C API uses an accumulator that collects after 100,000 appends
lst = [0] * 100000
self.assertEqual(json.loads(json.dumps(lst)), lst)
def test_sort_keys(self):
# https://github.com/simplejson/simplejson/issues/106
for num_keys in range(2, 32):
p = dict((str(x), x) for x in range(num_keys))
sio = StringIO()
json.dump(p, sio, sort_keys=True)
self.assertEqual(sio.getvalue(), json.dumps(p, sort_keys=True))
self.assertEqual(json.loads(sio.getvalue()), p)
def test_misbehaving_text_subtype(self):
# https://github.com/simplejson/simplejson/issues/185
text = "this is some text"
self.assertEqual(
json.dumps(MisbehavingTextSubtype(text)),
json.dumps(text)
)
self.assertEqual(
json.dumps([MisbehavingTextSubtype(text)]),
json.dumps([text])
)
self.assertEqual(
json.dumps({MisbehavingTextSubtype(text): 42}),
json.dumps({text: 42})
)
def test_misbehaving_bytes_subtype(self):
data = b("this is some data \xe2\x82\xac")
self.assertEqual(
json.dumps(MisbehavingBytesSubtype(data)),
json.dumps(data)
)
self.assertEqual(
json.dumps([MisbehavingBytesSubtype(data)]),
json.dumps([data])
)
self.assertEqual(
json.dumps({MisbehavingBytesSubtype(data): 42}),
json.dumps({data: 42})
)
def test_bytes_toplevel(self):
self.assertEqual(json.dumps(b('\xe2\x82\xac')), r'"\u20ac"')
self.assertRaises(UnicodeDecodeError, json.dumps, b('\xa4'))
self.assertEqual(json.dumps(b('\xa4'), encoding='iso-8859-1'),
r'"\u00a4"')
self.assertEqual(json.dumps(b('\xa4'), encoding='iso-8859-15'),
r'"\u20ac"')
if PY3:
self.assertRaises(TypeError, json.dumps, b('\xe2\x82\xac'),
encoding=None)
self.assertRaises(TypeError, json.dumps, b('\xa4'),
encoding=None)
self.assertEqual(json.dumps(b('\xa4'), encoding=None,
default=decode_iso_8859_15),
r'"\u20ac"')
else:
self.assertEqual(json.dumps(b('\xe2\x82\xac'), encoding=None),
r'"\u20ac"')
self.assertRaises(UnicodeDecodeError, json.dumps, b('\xa4'),
encoding=None)
self.assertRaises(UnicodeDecodeError, json.dumps, b('\xa4'),
encoding=None, default=decode_iso_8859_15)
def test_bytes_nested(self):
self.assertEqual(json.dumps([b('\xe2\x82\xac')]), r'["\u20ac"]')
self.assertRaises(UnicodeDecodeError, json.dumps, [b('\xa4')])
self.assertEqual(json.dumps([b('\xa4')], encoding='iso-8859-1'),
r'["\u00a4"]')
self.assertEqual(json.dumps([b('\xa4')], encoding='iso-8859-15'),
r'["\u20ac"]')
if PY3:
self.assertRaises(TypeError, json.dumps, [b('\xe2\x82\xac')],
encoding=None)
self.assertRaises(TypeError, json.dumps, [b('\xa4')],
encoding=None)
self.assertEqual(json.dumps([b('\xa4')], encoding=None,
default=decode_iso_8859_15),
r'["\u20ac"]')
else:
self.assertEqual(json.dumps([b('\xe2\x82\xac')], encoding=None),
r'["\u20ac"]')
self.assertRaises(UnicodeDecodeError, json.dumps, [b('\xa4')],
encoding=None)
self.assertRaises(UnicodeDecodeError, json.dumps, [b('\xa4')],
encoding=None, default=decode_iso_8859_15)
def test_bytes_key(self):
self.assertEqual(json.dumps({b('\xe2\x82\xac'): 42}), r'{"\u20ac": 42}')
self.assertRaises(UnicodeDecodeError, json.dumps, {b('\xa4'): 42})
self.assertEqual(json.dumps({b('\xa4'): 42}, encoding='iso-8859-1'),
r'{"\u00a4": 42}')
self.assertEqual(json.dumps({b('\xa4'): 42}, encoding='iso-8859-15'),
r'{"\u20ac": 42}')
if PY3:
self.assertRaises(TypeError, json.dumps, {b('\xe2\x82\xac'): 42},
encoding=None)
self.assertRaises(TypeError, json.dumps, {b('\xa4'): 42},
encoding=None)
self.assertRaises(TypeError, json.dumps, {b('\xa4'): 42},
encoding=None, default=decode_iso_8859_15)
self.assertEqual(json.dumps({b('\xa4'): 42}, encoding=None,
skipkeys=True),
r'{}')
else:
self.assertEqual(json.dumps({b('\xe2\x82\xac'): 42}, encoding=None),
r'{"\u20ac": 42}')
self.assertRaises(UnicodeDecodeError, json.dumps, {b('\xa4'): 42},
encoding=None)
self.assertRaises(UnicodeDecodeError, json.dumps, {b('\xa4'): 42},
encoding=None, default=decode_iso_8859_15)
self.assertRaises(UnicodeDecodeError, json.dumps, {b('\xa4'): 42},
encoding=None, skipkeys=True)

47
simplejson/tests/test_encode_basestring_ascii.py
View File

@ -1,47 +0,0 @@
from unittest import TestCase
import simplejson.encoder
from simplejson.compat import b
CASES = [
(u'/\\"\ucafe\ubabe\uab98\ufcde\ubcda\uef4a\x08\x0c\n\r\t`1~!@#$%^&*()_+-=[]{}|;:\',./<>?', '"/\\\\\\"\\ucafe\\ubabe\\uab98\\ufcde\\ubcda\\uef4a\\b\\f\\n\\r\\t`1~!@#$%^&*()_+-=[]{}|;:\',./<>?"'),
(u'\u0123\u4567\u89ab\ucdef\uabcd\uef4a', '"\\u0123\\u4567\\u89ab\\ucdef\\uabcd\\uef4a"'),
(u'controls', '"controls"'),
(u'\x08\x0c\n\r\t', '"\\b\\f\\n\\r\\t"'),
(u'{"object with 1 member":["array with 1 element"]}', '"{\\"object with 1 member\\":[\\"array with 1 element\\"]}"'),
(u' s p a c e d ', '" s p a c e d "'),
(u'\U0001d120', '"\\ud834\\udd20"'),
(u'\u03b1\u03a9', '"\\u03b1\\u03a9"'),
(b('\xce\xb1\xce\xa9'), '"\\u03b1\\u03a9"'),
(u'\u03b1\u03a9', '"\\u03b1\\u03a9"'),
(b('\xce\xb1\xce\xa9'), '"\\u03b1\\u03a9"'),
(u'\u03b1\u03a9', '"\\u03b1\\u03a9"'),
(u'\u03b1\u03a9', '"\\u03b1\\u03a9"'),
(u"`1~!@#$%^&*()_+-={':[,]}|;.</>?", '"`1~!@#$%^&*()_+-={\':[,]}|;.</>?"'),
(u'\x08\x0c\n\r\t', '"\\b\\f\\n\\r\\t"'),
(u'\u0123\u4567\u89ab\ucdef\uabcd\uef4a', '"\\u0123\\u4567\\u89ab\\ucdef\\uabcd\\uef4a"'),
]
class TestEncodeBaseStringAscii(TestCase):
def test_py_encode_basestring_ascii(self):
self._test_encode_basestring_ascii(simplejson.encoder.py_encode_basestring_ascii)
def test_c_encode_basestring_ascii(self):
if not simplejson.encoder.c_encode_basestring_ascii:
return
self._test_encode_basestring_ascii(simplejson.encoder.c_encode_basestring_ascii)
def _test_encode_basestring_ascii(self, encode_basestring_ascii):
fname = encode_basestring_ascii.__name__
for input_string, expect in CASES:
result = encode_basestring_ascii(input_string)
#self.assertEqual(result, expect,
# '{0!r} != {1!r} for {2}({3!r})'.format(
# result, expect, fname, input_string))
self.assertEqual(result, expect,
'%r != %r for %s(%r)' % (result, expect, fname, input_string))
def test_sorted_dict(self):
items = [('one', 1), ('two', 2), ('three', 3), ('four', 4), ('five', 5)]
s = simplejson.dumps(dict(items), sort_keys=True)
self.assertEqual(s, '{"five": 5, "four": 4, "one": 1, "three": 3, "two": 2}')

38
simplejson/tests/test_encode_for_html.py
View File

@ -1,38 +0,0 @@
import unittest
import simplejson as json
class TestEncodeForHTML(unittest.TestCase):
def setUp(self):
self.decoder = json.JSONDecoder()
self.encoder = json.JSONEncoderForHTML()
self.non_ascii_encoder = json.JSONEncoderForHTML(ensure_ascii=False)
def test_basic_encode(self):
self.assertEqual(r'"\u0026"', self.encoder.encode('&'))
self.assertEqual(r'"\u003c"', self.encoder.encode('<'))
self.assertEqual(r'"\u003e"', self.encoder.encode('>'))
self.assertEqual(r'"\u2028"', self.encoder.encode(u'\u2028'))
def test_non_ascii_basic_encode(self):
self.assertEqual(r'"\u0026"', self.non_ascii_encoder.encode('&'))
self.assertEqual(r'"\u003c"', self.non_ascii_encoder.encode('<'))
self.assertEqual(r'"\u003e"', self.non_ascii_encoder.encode('>'))
self.assertEqual(r'"\u2028"', self.non_ascii_encoder.encode(u'\u2028'))
def test_basic_roundtrip(self):
for char in '&<>':
self.assertEqual(
char, self.decoder.decode(
self.encoder.encode(char)))
def test_prevent_script_breakout(self):
bad_string = '</script><script>alert("gotcha")</script>'
self.assertEqual(
r'"\u003c/script\u003e\u003cscript\u003e'
r'alert(\"gotcha\")\u003c/script\u003e"',
self.encoder.encode(bad_string))
self.assertEqual(
bad_string, self.decoder.decode(
self.encoder.encode(bad_string)))

68
simplejson/tests/test_errors.py
View File

@ -1,68 +0,0 @@
import sys, pickle
from unittest import TestCase
import simplejson as json
from simplejson.compat import text_type, b
class TestErrors(TestCase):
def test_string_keys_error(self):
data = [{'a': 'A', 'b': (2, 4), 'c': 3.0, ('d',): 'D tuple'}]
try:
json.dumps(data)
except TypeError:
err = sys.exc_info()[1]
else:
self.fail('Expected TypeError')
self.assertEqual(str(err),
'keys must be str, int, float, bool or None, not tuple')
def test_not_serializable(self):
try:
json.dumps(json)
except TypeError:
err = sys.exc_info()[1]
else:
self.fail('Expected TypeError')
self.assertEqual(str(err),
'Object of type module is not JSON serializable')
def test_decode_error(self):
err = None
try:
json.loads('{}\na\nb')
except json.JSONDecodeError:
err = sys.exc_info()[1]
else:
self.fail('Expected JSONDecodeError')
self.assertEqual(err.lineno, 2)
self.assertEqual(err.colno, 1)
self.assertEqual(err.endlineno, 3)
self.assertEqual(err.endcolno, 2)
def test_scan_error(self):
err = None
for t in (text_type, b):
try:
json.loads(t('{"asdf": "'))
except json.JSONDecodeError:
err = sys.exc_info()[1]
else:
self.fail('Expected JSONDecodeError')
self.assertEqual(err.lineno, 1)
self.assertEqual(err.colno, 10)
def test_error_is_pickable(self):
err = None
try:
json.loads('{}\na\nb')
except json.JSONDecodeError:
err = sys.exc_info()[1]
else:
self.fail('Expected JSONDecodeError')
s = pickle.dumps(err)
e = pickle.loads(s)
self.assertEqual(err.msg, e.msg)
self.assertEqual(err.doc, e.doc)
self.assertEqual(err.pos, e.pos)
self.assertEqual(err.end, e.end)

176
simplejson/tests/test_fail.py
View File

@ -1,176 +0,0 @@
import sys
from unittest import TestCase
import simplejson as json
# 2007-10-05
JSONDOCS = [
# http://json.org/JSON_checker/test/fail1.json
'"A JSON payload should be an object or array, not a string."',
# http://json.org/JSON_checker/test/fail2.json
'["Unclosed array"',
# http://json.org/JSON_checker/test/fail3.json
'{unquoted_key: "keys must be quoted"}',
# http://json.org/JSON_checker/test/fail4.json
'["extra comma",]',
# http://json.org/JSON_checker/test/fail5.json
'["double extra comma",,]',
# http://json.org/JSON_checker/test/fail6.json
'[ , "<-- missing value"]',
# http://json.org/JSON_checker/test/fail7.json
'["Comma after the close"],',
# http://json.org/JSON_checker/test/fail8.json
'["Extra close"]]',
# http://json.org/JSON_checker/test/fail9.json
'{"Extra comma": true,}',
# http://json.org/JSON_checker/test/fail10.json
'{"Extra value after close": true} "misplaced quoted value"',
# http://json.org/JSON_checker/test/fail11.json
'{"Illegal expression": 1 + 2}',
# http://json.org/JSON_checker/test/fail12.json
'{"Illegal invocation": alert()}',
# http://json.org/JSON_checker/test/fail13.json
'{"Numbers cannot have leading zeroes": 013}',
# http://json.org/JSON_checker/test/fail14.json
'{"Numbers cannot be hex": 0x14}',
# http://json.org/JSON_checker/test/fail15.json
'["Illegal backslash escape: \\x15"]',
# http://json.org/JSON_checker/test/fail16.json
'[\\naked]',
# http://json.org/JSON_checker/test/fail17.json
'["Illegal backslash escape: \\017"]',
# http://json.org/JSON_checker/test/fail18.json
'[[[[[[[[[[[[[[[[[[[["Too deep"]]]]]]]]]]]]]]]]]]]]',
# http://json.org/JSON_checker/test/fail19.json
'{"Missing colon" null}',
# http://json.org/JSON_checker/test/fail20.json
'{"Double colon":: null}',
# http://json.org/JSON_checker/test/fail21.json
'{"Comma instead of colon", null}',
# http://json.org/JSON_checker/test/fail22.json
'["Colon instead of comma": false]',
# http://json.org/JSON_checker/test/fail23.json
'["Bad value", truth]',
# http://json.org/JSON_checker/test/fail24.json
"['single quote']",
# http://json.org/JSON_checker/test/fail25.json
'["\ttab\tcharacter\tin\tstring\t"]',
# http://json.org/JSON_checker/test/fail26.json
'["tab\\ character\\ in\\ string\\ "]',
# http://json.org/JSON_checker/test/fail27.json
'["line\nbreak"]',
# http://json.org/JSON_checker/test/fail28.json
'["line\\\nbreak"]',
# http://json.org/JSON_checker/test/fail29.json
'[0e]',
# http://json.org/JSON_checker/test/fail30.json
'[0e+]',
# http://json.org/JSON_checker/test/fail31.json
'[0e+-1]',
# http://json.org/JSON_checker/test/fail32.json
'{"Comma instead if closing brace": true,',
# http://json.org/JSON_checker/test/fail33.json
'["mismatch"}',
# http://code.google.com/p/simplejson/issues/detail?id=3
u'["A\u001FZ control characters in string"]',
# misc based on coverage
'{',
'{]',
'{"foo": "bar"]',
'{"foo": "bar"',
'nul',
'nulx',
'-',
'-x',
'-e',
'-e0',
'-Infinite',
'-Inf',
'Infinit',
'Infinite',
'NaM',
'NuN',
'falsy',
'fal',
'trug',
'tru',
'1e',
'1ex',
'1e-',
'1e-x',
]
SKIPS = {
1: "why not have a string payload?",
18: "spec doesn't specify any nesting limitations",
}
class TestFail(TestCase):
def test_failures(self):
for idx, doc in enumerate(JSONDOCS):
idx = idx + 1
if idx in SKIPS:
json.loads(doc)
continue
try:
json.loads(doc)
except json.JSONDecodeError:
pass
else:
self.fail("Expected failure for fail%d.json: %r" % (idx, doc))
def test_array_decoder_issue46(self):
# http://code.google.com/p/simplejson/issues/detail?id=46
for doc in [u'[,]', '[,]']:
try:
json.loads(doc)
except json.JSONDecodeError:
e = sys.exc_info()[1]
self.assertEqual(e.pos, 1)
self.assertEqual(e.lineno, 1)
self.assertEqual(e.colno, 2)
except Exception:
e = sys.exc_info()[1]
self.fail("Unexpected exception raised %r %s" % (e, e))
else:
self.fail("Unexpected success parsing '[,]'")
def test_truncated_input(self):
test_cases = [
('', 'Expecting value', 0),
('[', "Expecting value or ']'", 1),
('[42', "Expecting ',' delimiter", 3),
('[42,', 'Expecting value', 4),
('["', 'Unterminated string starting at', 1),
('["spam', 'Unterminated string starting at', 1),
('["spam"', "Expecting ',' delimiter", 7),
('["spam",', 'Expecting value', 8),
('{', 'Expecting property name enclosed in double quotes', 1),
('{"', 'Unterminated string starting at', 1),
('{"spam', 'Unterminated string starting at', 1),
('{"spam"', "Expecting ':' delimiter", 7),
('{"spam":', 'Expecting value', 8),
('{"spam":42', "Expecting ',' delimiter", 10),
('{"spam":42,', 'Expecting property name enclosed in double quotes',
11),
('"', 'Unterminated string starting at', 0),
('"spam', 'Unterminated string starting at', 0),
('[,', "Expecting value", 1),
]
for data, msg, idx in test_cases:
try:
json.loads(data)
except json.JSONDecodeError:
e = sys.exc_info()[1]
self.assertEqual(
e.msg[:len(msg)],
msg,
"%r doesn't start with %r for %r" % (e.msg, msg, data))
self.assertEqual(
e.pos, idx,
"pos %r != %r for %r" % (e.pos, idx, data))
except Exception:
e = sys.exc_info()[1]
self.fail("Unexpected exception raised %r %s" % (e, e))
else:
self.fail("Unexpected success parsing '%r'" % (data,))

35
simplejson/tests/test_float.py
View File

@ -1,35 +0,0 @@
import math
from unittest import TestCase
from simplejson.compat import long_type, text_type
import simplejson as json
from simplejson.decoder import NaN, PosInf, NegInf
class TestFloat(TestCase):
def test_degenerates_allow(self):
for inf in (PosInf, NegInf):
self.assertEqual(json.loads(json.dumps(inf)), inf)
# Python 2.5 doesn't have math.isnan
nan = json.loads(json.dumps(NaN))
self.assertTrue((0 + nan) != nan)
def test_degenerates_ignore(self):
for f in (PosInf, NegInf, NaN):
self.assertEqual(json.loads(json.dumps(f, ignore_nan=True)), None)
def test_degenerates_deny(self):
for f in (PosInf, NegInf, NaN):
self.assertRaises(ValueError, json.dumps, f, allow_nan=False)
def test_floats(self):
for num in [1617161771.7650001, math.pi, math.pi**100,
math.pi**-100, 3.1]:
self.assertEqual(float(json.dumps(num)), num)
self.assertEqual(json.loads(json.dumps(num)), num)
self.assertEqual(json.loads(text_type(json.dumps(num))), num)
def test_ints(self):
for num in [1, long_type(1), 1<<32, 1<<64]:
self.assertEqual(json.dumps(num), str(num))
self.assertEqual(int(json.dumps(num)), num)
self.assertEqual(json.loads(json.dumps(num)), num)
self.assertEqual(json.loads(text_type(json.dumps(num))), num)

97
simplejson/tests/test_for_json.py
View File

@ -1,97 +0,0 @@
import unittest
import simplejson as json
class ForJson(object):
def for_json(self):
return {'for_json': 1}
class NestedForJson(object):
def for_json(self):
return {'nested': ForJson()}
class ForJsonList(object):
def for_json(self):
return ['list']
class DictForJson(dict):
def for_json(self):
return {'alpha': 1}
class ListForJson(list):
def for_json(self):
return ['list']
class TestForJson(unittest.TestCase):
def assertRoundTrip(self, obj, other, for_json=True):
if for_json is None:
# None will use the default
s = json.dumps(obj)
else:
s = json.dumps(obj, for_json=for_json)
self.assertEqual(
json.loads(s),
other)
def test_for_json_encodes_stand_alone_object(self):
self.assertRoundTrip(
ForJson(),
ForJson().for_json())
def test_for_json_encodes_object_nested_in_dict(self):
self.assertRoundTrip(
{'hooray': ForJson()},
{'hooray': ForJson().for_json()})
def test_for_json_encodes_object_nested_in_list_within_dict(self):
self.assertRoundTrip(
{'list': [0, ForJson(), 2, 3]},
{'list': [0, ForJson().for_json(), 2, 3]})
def test_for_json_encodes_object_nested_within_object(self):
self.assertRoundTrip(
NestedForJson(),
{'nested': {'for_json': 1}})
def test_for_json_encodes_list(self):
self.assertRoundTrip(
ForJsonList(),
ForJsonList().for_json())
def test_for_json_encodes_list_within_object(self):
self.assertRoundTrip(
{'nested': ForJsonList()},
{'nested': ForJsonList().for_json()})
def test_for_json_encodes_dict_subclass(self):
self.assertRoundTrip(
DictForJson(a=1),
DictForJson(a=1).for_json())
def test_for_json_encodes_list_subclass(self):
self.assertRoundTrip(
ListForJson(['l']),
ListForJson(['l']).for_json())
def test_for_json_ignored_if_not_true_with_dict_subclass(self):
for for_json in (None, False):
self.assertRoundTrip(
DictForJson(a=1),
{'a': 1},
for_json=for_json)
def test_for_json_ignored_if_not_true_with_list_subclass(self):
for for_json in (None, False):
self.assertRoundTrip(
ListForJson(['l']),
['l'],
for_json=for_json)
def test_raises_typeerror_if_for_json_not_true_with_object(self):
self.assertRaises(TypeError, json.dumps, ForJson())
self.assertRaises(TypeError, json.dumps, ForJson(), for_json=False)

86
simplejson/tests/test_indent.py
View File

@ -1,86 +0,0 @@
from unittest import TestCase
import textwrap
import simplejson as json
from simplejson.compat import StringIO
class TestIndent(TestCase):
def test_indent(self):
h = [['blorpie'], ['whoops'], [], 'd-shtaeou', 'd-nthiouh',
'i-vhbjkhnth',
{'nifty': 87}, {'field': 'yes', 'morefield': False} ]
expect = textwrap.dedent("""\
[
\t[
\t\t"blorpie"
\t],
\t[
\t\t"whoops"
\t],
\t[],
\t"d-shtaeou",
\t"d-nthiouh",
\t"i-vhbjkhnth",
\t{
\t\t"nifty": 87
\t},
\t{
\t\t"field": "yes",
\t\t"morefield": false
\t}
]""")
d1 = json.dumps(h)
d2 = json.dumps(h, indent='\t', sort_keys=True, separators=(',', ': '))
d3 = json.dumps(h, indent=' ', sort_keys=True, separators=(',', ': '))
d4 = json.dumps(h, indent=2, sort_keys=True, separators=(',', ': '))
h1 = json.loads(d1)
h2 = json.loads(d2)
h3 = json.loads(d3)
h4 = json.loads(d4)
self.assertEqual(h1, h)
self.assertEqual(h2, h)
self.assertEqual(h3, h)
self.assertEqual(h4, h)
self.assertEqual(d3, expect.replace('\t', ' '))
self.assertEqual(d4, expect.replace('\t', ' '))
# NOTE: Python 2.4 textwrap.dedent converts tabs to spaces,
# so the following is expected to fail. Python 2.4 is not a
# supported platform in simplejson 2.1.0+.
self.assertEqual(d2, expect)
def test_indent0(self):
h = {3: 1}
def check(indent, expected):
d1 = json.dumps(h, indent=indent)
self.assertEqual(d1, expected)
sio = StringIO()
json.dump(h, sio, indent=indent)
self.assertEqual(sio.getvalue(), expected)
# indent=0 should emit newlines
check(0, '{\n"3": 1\n}')
# indent=None is more compact
check(None, '{"3": 1}')
def test_separators(self):
lst = [1,2,3,4]
expect = '[\n1,\n2,\n3,\n4\n]'
expect_spaces = '[\n1, \n2, \n3, \n4\n]'
# Ensure that separators still works
self.assertEqual(
expect_spaces,
json.dumps(lst, indent=0, separators=(', ', ': ')))
# Force the new defaults
self.assertEqual(
expect,
json.dumps(lst, indent=0, separators=(',', ': ')))
# Added in 2.1.4
self.assertEqual(
expect,
json.dumps(lst, indent=0))

27
simplejson/tests/test_item_sort_key.py
View File

@ -1,27 +0,0 @@
from unittest import TestCase
import simplejson as json
from operator import itemgetter
class TestItemSortKey(TestCase):
def test_simple_first(self):
a = {'a': 1, 'c': 5, 'jack': 'jill', 'pick': 'axe', 'array': [1, 5, 6, 9], 'tuple': (83, 12, 3), 'crate': 'dog', 'zeak': 'oh'}
self.assertEqual(
'{"a": 1, "c": 5, "crate": "dog", "jack": "jill", "pick": "axe", "zeak": "oh", "array": [1, 5, 6, 9], "tuple": [83, 12, 3]}',
json.dumps(a, item_sort_key=json.simple_first))
def test_case(self):
a = {'a': 1, 'c': 5, 'Jack': 'jill', 'pick': 'axe', 'Array': [1, 5, 6, 9], 'tuple': (83, 12, 3), 'crate': 'dog', 'zeak': 'oh'}
self.assertEqual(
'{"Array": [1, 5, 6, 9], "Jack": "jill", "a": 1, "c": 5, "crate": "dog", "pick": "axe", "tuple": [83, 12, 3], "zeak": "oh"}',
json.dumps(a, item_sort_key=itemgetter(0)))
self.assertEqual(
'{"a": 1, "Array": [1, 5, 6, 9], "c": 5, "crate": "dog", "Jack": "jill", "pick": "axe", "tuple": [83, 12, 3], "zeak": "oh"}',
json.dumps(a, item_sort_key=lambda kv: kv[0].lower()))
def test_item_sort_key_value(self):
# https://github.com/simplejson/simplejson/issues/173
a = {'a': 1, 'b': 0}
self.assertEqual(
'{"b": 0, "a": 1}',
json.dumps(a, item_sort_key=lambda kv: kv[1]))

31
simplejson/tests/test_iterable.py
View File

@ -1,31 +0,0 @@
import unittest
from simplejson.compat import StringIO
import simplejson as json
def iter_dumps(obj, **kw):
return ''.join(json.JSONEncoder(**kw).iterencode(obj))
def sio_dump(obj, **kw):
sio = StringIO()
json.dumps(obj, **kw)
return sio.getvalue()
class TestIterable(unittest.TestCase):
def test_iterable(self):
for l in ([], [1], [1, 2], [1, 2, 3]):
for opts in [{}, {'indent': 2}]:
for dumps in (json.dumps, iter_dumps, sio_dump):
expect = dumps(l, **opts)
default_expect = dumps(sum(l), **opts)
# Default is False
self.assertRaises(TypeError, dumps, iter(l), **opts)
self.assertRaises(TypeError, dumps, iter(l), iterable_as_array=False, **opts)
self.assertEqual(expect, dumps(iter(l), iterable_as_array=True, **opts))
# Ensure that the "default" gets called
self.assertEqual(default_expect, dumps(iter(l), default=sum, **opts))
self.assertEqual(default_expect, dumps(iter(l), iterable_as_array=False, default=sum, **opts))
# Ensure that the "default" does not get called
self.assertEqual(
expect,
dumps(iter(l), iterable_as_array=True, default=sum, **opts))

122
simplejson/tests/test_namedtuple.py
View File

@ -1,122 +0,0 @@
from __future__ import absolute_import
import unittest
import simplejson as json
from simplejson.compat import StringIO
try:
from collections import namedtuple
except ImportError:
class Value(tuple):
def __new__(cls, *args):
return tuple.__new__(cls, args)
def _asdict(self):
return {'value': self[0]}
class Point(tuple):
def __new__(cls, *args):
return tuple.__new__(cls, args)
def _asdict(self):
return {'x': self[0], 'y': self[1]}
else:
Value = namedtuple('Value', ['value'])
Point = namedtuple('Point', ['x', 'y'])
class DuckValue(object):
def __init__(self, *args):
self.value = Value(*args)
def _asdict(self):
return self.value._asdict()
class DuckPoint(object):
def __init__(self, *args):
self.point = Point(*args)
def _asdict(self):
return self.point._asdict()
class DeadDuck(object):
_asdict = None
class DeadDict(dict):
_asdict = None
CONSTRUCTORS = [
lambda v: v,
lambda v: [v],
lambda v: [{'key': v}],
]
class TestNamedTuple(unittest.TestCase):
def test_namedtuple_dumps(self):
for v in [Value(1), Point(1, 2), DuckValue(1), DuckPoint(1, 2)]:
d = v._asdict()
self.assertEqual(d, json.loads(json.dumps(v)))
self.assertEqual(
d,
json.loads(json.dumps(v, namedtuple_as_object=True)))
self.assertEqual(d, json.loads(json.dumps(v, tuple_as_array=False)))
self.assertEqual(
d,
json.loads(json.dumps(v, namedtuple_as_object=True,
tuple_as_array=False)))
def test_namedtuple_dumps_false(self):
for v in [Value(1), Point(1, 2)]:
l = list(v)
self.assertEqual(
l,
json.loads(json.dumps(v, namedtuple_as_object=False)))
self.assertRaises(TypeError, json.dumps, v,
tuple_as_array=False, namedtuple_as_object=False)
def test_namedtuple_dump(self):
for v in [Value(1), Point(1, 2), DuckValue(1), DuckPoint(1, 2)]:
d = v._asdict()
sio = StringIO()
json.dump(v, sio)
self.assertEqual(d, json.loads(sio.getvalue()))
sio = StringIO()
json.dump(v, sio, namedtuple_as_object=True)
self.assertEqual(
d,
json.loads(sio.getvalue()))
sio = StringIO()
json.dump(v, sio, tuple_as_array=False)
self.assertEqual(d, json.loads(sio.getvalue()))
sio = StringIO()
json.dump(v, sio, namedtuple_as_object=True,
tuple_as_array=False)
self.assertEqual(
d,
json.loads(sio.getvalue()))
def test_namedtuple_dump_false(self):
for v in [Value(1), Point(1, 2)]:
l = list(v)
sio = StringIO()
json.dump(v, sio, namedtuple_as_object=False)
self.assertEqual(
l,
json.loads(sio.getvalue()))
self.assertRaises(TypeError, json.dump, v, StringIO(),
tuple_as_array=False, namedtuple_as_object=False)
def test_asdict_not_callable_dump(self):
for f in CONSTRUCTORS:
self.assertRaises(TypeError,
json.dump, f(DeadDuck()), StringIO(), namedtuple_as_object=True)
sio = StringIO()
json.dump(f(DeadDict()), sio, namedtuple_as_object=True)
self.assertEqual(
json.dumps(f({})),
sio.getvalue())
def test_asdict_not_callable_dumps(self):
for f in CONSTRUCTORS:
self.assertRaises(TypeError,
json.dumps, f(DeadDuck()), namedtuple_as_object=True)
self.assertEqual(
json.dumps(f({})),
json.dumps(f(DeadDict()), namedtuple_as_object=True))

71
simplejson/tests/test_pass1.py
View File

@ -1,71 +0,0 @@
from unittest import TestCase
import simplejson as json
# from http://json.org/JSON_checker/test/pass1.json
JSON = r'''
[
"JSON Test Pattern pass1",
{"object with 1 member":["array with 1 element"]},
{},
[],
-42,
true,
false,
null,
{
"integer": 1234567890,
"real": -9876.543210,
"e": 0.123456789e-12,
"E": 1.234567890E+34,
"": 23456789012E66,
"zero": 0,
"one": 1,
"space": " ",
"quote": "\"",
"backslash": "\\",
"controls": "\b\f\n\r\t",
"slash": "/ & \/",
"alpha": "abcdefghijklmnopqrstuvwyz",
"ALPHA": "ABCDEFGHIJKLMNOPQRSTUVWYZ",
"digit": "0123456789",
"special": "`1~!@#$%^&*()_+-={':[,]}|;.</>?",
"hex": "\u0123\u4567\u89AB\uCDEF\uabcd\uef4A",
"true": true,
"false": false,
"null": null,
"array":[ ],
"object":{ },
"address": "50 St. James Street",
"url": "http://www.JSON.org/",
"comment": "// /* <!-- --",
"# -- --> */": " ",
" s p a c e d " :[1,2 , 3
,
4 , 5 , 6 ,7 ],"compact": [1,2,3,4,5,6,7],
"jsontext": "{\"object with 1 member\":[\"array with 1 element\"]}",
"quotes": "&#34; \u0022 %22 0x22 034 &#x22;",
"\/\\\"\uCAFE\uBABE\uAB98\uFCDE\ubcda\uef4A\b\f\n\r\t`1~!@#$%^&*()_+-=[]{}|;:',./<>?"
: "A key can be any string"
},
0.5 ,98.6
,
99.44
,
1066,
1e1,
0.1e1,
1e-1,
1e00,2e+00,2e-00
,"rosebud"]
'''
class TestPass1(TestCase):
def test_parse(self):
# test in/out equivalence and parsing
res = json.loads(JSON)
out = json.dumps(res)
self.assertEqual(res, json.loads(out))

14
simplejson/tests/test_pass2.py
View File

@ -1,14 +0,0 @@
from unittest import TestCase
import simplejson as json
# from http://json.org/JSON_checker/test/pass2.json
JSON = r'''
[[[[[[[[[[[[[[[[[[["Not too deep"]]]]]]]]]]]]]]]]]]]
'''
class TestPass2(TestCase):
def test_parse(self):
# test in/out equivalence and parsing
res = json.loads(JSON)
out = json.dumps(res)
self.assertEqual(res, json.loads(out))

20
simplejson/tests/test_pass3.py
View File

@ -1,20 +0,0 @@
from unittest import TestCase
import simplejson as json
# from http://json.org/JSON_checker/test/pass3.json
JSON = r'''
{
"JSON Test Pattern pass3": {
"The outermost value": "must be an object or array.",
"In this test": "It is an object."
}
}
'''
class TestPass3(TestCase):
def test_parse(self):
# test in/out equivalence and parsing
res = json.loads(JSON)
out = json.dumps(res)
self.assertEqual(res, json.loads(out))

47
simplejson/tests/test_raw_json.py
View File

@ -1,47 +0,0 @@
import unittest
import simplejson as json
dct1 = {
'key1': 'value1'
}
dct2 = {
'key2': 'value2',
'd1': dct1
}
dct3 = {
'key2': 'value2',
'd1': json.dumps(dct1)
}
dct4 = {
'key2': 'value2',
'd1': json.RawJSON(json.dumps(dct1))
}
class TestRawJson(unittest.TestCase):
def test_normal_str(self):
self.assertNotEqual(json.dumps(dct2), json.dumps(dct3))
def test_raw_json_str(self):
self.assertEqual(json.dumps(dct2), json.dumps(dct4))
self.assertEqual(dct2, json.loads(json.dumps(dct4)))
def test_list(self):
self.assertEqual(
json.dumps([dct2]),
json.dumps([json.RawJSON(json.dumps(dct2))]))
self.assertEqual(
[dct2],
json.loads(json.dumps([json.RawJSON(json.dumps(dct2))])))
def test_direct(self):
self.assertEqual(
json.dumps(dct2),
json.dumps(json.RawJSON(json.dumps(dct2))))
self.assertEqual(
dct2,
json.loads(json.dumps(json.RawJSON(json.dumps(dct2)))))

67
simplejson/tests/test_recursion.py
View File

@ -1,67 +0,0 @@
from unittest import TestCase
import simplejson as json
class JSONTestObject:
pass
class RecursiveJSONEncoder(json.JSONEncoder):
recurse = False
def default(self, o):
if o is JSONTestObject:
if self.recurse:
return [JSONTestObject]
else:
return 'JSONTestObject'
return json.JSONEncoder.default(o)
class TestRecursion(TestCase):
def test_listrecursion(self):
x = []
x.append(x)
try:
json.dumps(x)
except ValueError:
pass
else:
self.fail("didn't raise ValueError on list recursion")
x = []
y = [x]
x.append(y)
try:
json.dumps(x)
except ValueError:
pass
else:
self.fail("didn't raise ValueError on alternating list recursion")
y = []
x = [y, y]
# ensure that the marker is cleared
json.dumps(x)
def test_dictrecursion(self):
x = {}
x["test"] = x
try:
json.dumps(x)
except ValueError:
pass
else:
self.fail("didn't raise ValueError on dict recursion")
x = {}
y = {"a": x, "b": x}
# ensure that the marker is cleared
json.dumps(y)
def test_defaultrecursion(self):
enc = RecursiveJSONEncoder()
self.assertEqual(enc.encode(JSONTestObject), '"JSONTestObject"')
enc.recurse = True
try:
enc.encode(JSONTestObject)
except ValueError:
pass
else:
self.fail("didn't raise ValueError on default recursion")

196
simplejson/tests/test_scanstring.py
View File

@ -1,196 +0,0 @@
import sys
from unittest import TestCase
import simplejson as json
import simplejson.decoder
from simplejson.compat import b, PY3
class TestScanString(TestCase):
# The bytes type is intentionally not used in most of these tests
# under Python 3 because the decoder immediately coerces to str before
# calling scanstring. In Python 2 we are testing the code paths
# for both unicode and str.
#
# The reason this is done is because Python 3 would require
# entirely different code paths for parsing bytes and str.
#
def test_py_scanstring(self):
self._test_scanstring(simplejson.decoder.py_scanstring)
def test_c_scanstring(self):
if not simplejson.decoder.c_scanstring:
return
self._test_scanstring(simplejson.decoder.c_scanstring)
self.assertTrue(isinstance(simplejson.decoder.c_scanstring('""', 0)[0], str))
def _test_scanstring(self, scanstring):
if sys.maxunicode == 65535:
self.assertEqual(
scanstring(u'"z\U0001d120x"', 1, None, True),
(u'z\U0001d120x', 6))
else:
self.assertEqual(
scanstring(u'"z\U0001d120x"', 1, None, True),
(u'z\U0001d120x', 5))
self.assertEqual(
scanstring('"\\u007b"', 1, None, True),
(u'{', 8))
self.assertEqual(
scanstring('"A JSON payload should be an object or array, not a string."', 1, None, True),
(u'A JSON payload should be an object or array, not a string.', 60))
self.assertEqual(
scanstring('["Unclosed array"', 2, None, True),
(u'Unclosed array', 17))
self.assertEqual(
scanstring('["extra comma",]', 2, None, True),
(u'extra comma', 14))
self.assertEqual(
scanstring('["double extra comma",,]', 2, None, True),
(u'double extra comma', 21))
self.assertEqual(
scanstring('["Comma after the close"],', 2, None, True),
(u'Comma after the close', 24))
self.assertEqual(
scanstring('["Extra close"]]', 2, None, True),
(u'Extra close', 14))
self.assertEqual(
scanstring('{"Extra comma": true,}', 2, None, True),
(u'Extra comma', 14))
self.assertEqual(
scanstring('{"Extra value after close": true} "misplaced quoted value"', 2, None, True),
(u'Extra value after close', 26))
self.assertEqual(
scanstring('{"Illegal expression": 1 + 2}', 2, None, True),
(u'Illegal expression', 21))
self.assertEqual(
scanstring('{"Illegal invocation": alert()}', 2, None, True),
(u'Illegal invocation', 21))
self.assertEqual(
scanstring('{"Numbers cannot have leading zeroes": 013}', 2, None, True),
(u'Numbers cannot have leading zeroes', 37))
self.assertEqual(
scanstring('{"Numbers cannot be hex": 0x14}', 2, None, True),
(u'Numbers cannot be hex', 24))
self.assertEqual(
scanstring('[[[[[[[[[[[[[[[[[[[["Too deep"]]]]]]]]]]]]]]]]]]]]', 21, None, True),
(u'Too deep', 30))
self.assertEqual(
scanstring('{"Missing colon" null}', 2, None, True),
(u'Missing colon', 16))
self.assertEqual(
scanstring('{"Double colon":: null}', 2, None, True),
(u'Double colon', 15))
self.assertEqual(
scanstring('{"Comma instead of colon", null}', 2, None, True),
(u'Comma instead of colon', 25))
self.assertEqual(
scanstring('["Colon instead of comma": false]', 2, None, True),
(u'Colon instead of comma', 25))
self.assertEqual(
scanstring('["Bad value", truth]', 2, None, True),
(u'Bad value', 12))
for c in map(chr, range(0x00, 0x1f)):
self.assertEqual(
scanstring(c + '"', 0, None, False),
(c, 2))
self.assertRaises(
ValueError,
scanstring, c + '"', 0, None, True)
self.assertRaises(ValueError, scanstring, '', 0, None, True)
self.assertRaises(ValueError, scanstring, 'a', 0, None, True)
self.assertRaises(ValueError, scanstring, '\\', 0, None, True)
self.assertRaises(ValueError, scanstring, '\\u', 0, None, True)
self.assertRaises(ValueError, scanstring, '\\u0', 0, None, True)
self.assertRaises(ValueError, scanstring, '\\u01', 0, None, True)
self.assertRaises(ValueError, scanstring, '\\u012', 0, None, True)
self.assertRaises(ValueError, scanstring, '\\u0123', 0, None, True)
if sys.maxunicode > 65535:
self.assertRaises(ValueError,
scanstring, '\\ud834\\u"', 0, None, True)
self.assertRaises(ValueError,
scanstring, '\\ud834\\x0123"', 0, None, True)
def test_issue3623(self):
self.assertRaises(ValueError, json.decoder.scanstring, "xxx", 1,
"xxx")
self.assertRaises(UnicodeDecodeError,
json.encoder.encode_basestring_ascii, b("xx\xff"))
def test_overflow(self):
# Python 2.5 does not have maxsize, Python 3 does not have maxint
maxsize = getattr(sys, 'maxsize', getattr(sys, 'maxint', None))
assert maxsize is not None
self.assertRaises(OverflowError, json.decoder.scanstring, "xxx",
maxsize + 1)
def test_surrogates(self):
scanstring = json.decoder.scanstring
def assertScan(given, expect, test_utf8=True):
givens = [given]
if not PY3 and test_utf8:
givens.append(given.encode('utf8'))
for given in givens:
(res, count) = scanstring(given, 1, None, True)
self.assertEqual(len(given), count)
self.assertEqual(res, expect)
assertScan(
u'"z\\ud834\\u0079x"',
u'z\ud834yx')
assertScan(
u'"z\\ud834\\udd20x"',
u'z\U0001d120x')
assertScan(
u'"z\\ud834\\ud834\\udd20x"',
u'z\ud834\U0001d120x')
assertScan(
u'"z\\ud834x"',
u'z\ud834x')
assertScan(
u'"z\\udd20x"',
u'z\udd20x')
assertScan(
u'"z\ud834x"',
u'z\ud834x')
# It may look strange to join strings together, but Python is drunk.
# https://gist.github.com/etrepum/5538443
assertScan(
u'"z\\ud834\udd20x12345"',
u''.join([u'z\ud834', u'\udd20x12345']))
assertScan(
u'"z\ud834\\udd20x"',
u''.join([u'z\ud834', u'\udd20x']))
# these have different behavior given UTF8 input, because the surrogate
# pair may be joined (in maxunicode > 65535 builds)
assertScan(
u''.join([u'"z\ud834', u'\udd20x"']),
u''.join([u'z\ud834', u'\udd20x']),
test_utf8=False)
self.assertRaises(ValueError,
scanstring, u'"z\\ud83x"', 1, None, True)
self.assertRaises(ValueError,
scanstring, u'"z\\ud834\\udd2x"', 1, None, True)

42
simplejson/tests/test_separators.py
View File

@ -1,42 +0,0 @@
import textwrap
from unittest import TestCase
import simplejson as json
class TestSeparators(TestCase):
def test_separators(self):
h = [['blorpie'], ['whoops'], [], 'd-shtaeou', 'd-nthiouh', 'i-vhbjkhnth',
{'nifty': 87}, {'field': 'yes', 'morefield': False} ]
expect = textwrap.dedent("""\
[
[
"blorpie"
] ,
[
"whoops"
] ,
[] ,
"d-shtaeou" ,
"d-nthiouh" ,
"i-vhbjkhnth" ,
{
"nifty" : 87
} ,
{
"field" : "yes" ,
"morefield" : false
}
]""")
d1 = json.dumps(h)
d2 = json.dumps(h, indent=' ', sort_keys=True, separators=(' ,', ' : '))
h1 = json.loads(d1)
h2 = json.loads(d2)
self.assertEqual(h1, h)
self.assertEqual(h2, h)
self.assertEqual(d2, expect)

114
simplejson/tests/test_speedups.py
View File

@ -1,114 +0,0 @@
from __future__ import with_statement
import sys
import unittest
from unittest import TestCase
import simplejson
from simplejson import encoder, decoder, scanner
from simplejson.compat import PY3, long_type, b
def has_speedups():
return encoder.c_make_encoder is not None
def skip_if_speedups_missing(func):
def wrapper(*args, **kwargs):
if not has_speedups():
if hasattr(unittest, 'SkipTest'):
raise unittest.SkipTest("C Extension not available")
else:
sys.stdout.write("C Extension not available")
return
return func(*args, **kwargs)
return wrapper
class BadBool:
def __bool__(self):
1/0
__nonzero__ = __bool__
class TestDecode(TestCase):
@skip_if_speedups_missing
def test_make_scanner(self):
self.assertRaises(AttributeError, scanner.c_make_scanner, 1)
@skip_if_speedups_missing
def test_bad_bool_args(self):
def test(value):
decoder.JSONDecoder(strict=BadBool()).decode(value)
self.assertRaises(ZeroDivisionError, test, '""')
self.assertRaises(ZeroDivisionError, test, '{}')
if not PY3:
self.assertRaises(ZeroDivisionError, test, u'""')
self.assertRaises(ZeroDivisionError, test, u'{}')
class TestEncode(TestCase):
@skip_if_speedups_missing
def test_make_encoder(self):
self.assertRaises(
TypeError,
encoder.c_make_encoder,
None,
("\xCD\x7D\x3D\x4E\x12\x4C\xF9\x79\xD7"
"\x52\xBA\x82\xF2\x27\x4A\x7D\xA0\xCA\x75"),
None
)
@skip_if_speedups_missing
def test_bad_str_encoder(self):
# Issue #31505: There shouldn't be an assertion failure in case
# c_make_encoder() receives a bad encoder() argument.
import decimal
def bad_encoder1(*args):
return None
enc = encoder.c_make_encoder(
None, lambda obj: str(obj),
bad_encoder1, None, ': ', ', ',
False, False, False, {}, False, False, False,
None, None, 'utf-8', False, False, decimal.Decimal, False)
self.assertRaises(TypeError, enc, 'spam', 4)
self.assertRaises(TypeError, enc, {'spam': 42}, 4)
def bad_encoder2(*args):
1/0
enc = encoder.c_make_encoder(
None, lambda obj: str(obj),
bad_encoder2, None, ': ', ', ',
False, False, False, {}, False, False, False,
None, None, 'utf-8', False, False, decimal.Decimal, False)
self.assertRaises(ZeroDivisionError, enc, 'spam', 4)
@skip_if_speedups_missing
def test_bad_bool_args(self):
def test(name):
encoder.JSONEncoder(**{name: BadBool()}).encode({})
self.assertRaises(ZeroDivisionError, test, 'skipkeys')
self.assertRaises(ZeroDivisionError, test, 'ensure_ascii')
self.assertRaises(ZeroDivisionError, test, 'check_circular')
self.assertRaises(ZeroDivisionError, test, 'allow_nan')
self.assertRaises(ZeroDivisionError, test, 'sort_keys')
self.assertRaises(ZeroDivisionError, test, 'use_decimal')
self.assertRaises(ZeroDivisionError, test, 'namedtuple_as_object')
self.assertRaises(ZeroDivisionError, test, 'tuple_as_array')
self.assertRaises(ZeroDivisionError, test, 'bigint_as_string')
self.assertRaises(ZeroDivisionError, test, 'for_json')
self.assertRaises(ZeroDivisionError, test, 'ignore_nan')
self.assertRaises(ZeroDivisionError, test, 'iterable_as_array')
@skip_if_speedups_missing
def test_int_as_string_bitcount_overflow(self):
long_count = long_type(2)**32+31
def test():
encoder.JSONEncoder(int_as_string_bitcount=long_count).encode(0)
self.assertRaises((TypeError, OverflowError), test)
if PY3:
@skip_if_speedups_missing
def test_bad_encoding(self):
with self.assertRaises(UnicodeEncodeError):
encoder.JSONEncoder(encoding='\udcff').encode({b('key'): 123})

21
simplejson/tests/test_str_subclass.py
View File

@ -1,21 +0,0 @@
from unittest import TestCase
import simplejson
from simplejson.compat import text_type
# Tests for issue demonstrated in https://github.com/simplejson/simplejson/issues/144
class WonkyTextSubclass(text_type):
def __getslice__(self, start, end):
return self.__class__('not what you wanted!')
class TestStrSubclass(TestCase):
def test_dump_load(self):
for s in ['', '"hello"', 'text', u'\u005c']:
self.assertEqual(
s,
simplejson.loads(simplejson.dumps(WonkyTextSubclass(s))))
self.assertEqual(
s,
simplejson.loads(simplejson.dumps(WonkyTextSubclass(s),
ensure_ascii=False)))

37
simplejson/tests/test_subclass.py
View File

@ -1,37 +0,0 @@
from unittest import TestCase
import simplejson as json
from decimal import Decimal
class AlternateInt(int):
def __repr__(self):
return 'invalid json'
__str__ = __repr__
class AlternateFloat(float):
def __repr__(self):
return 'invalid json'
__str__ = __repr__
# class AlternateDecimal(Decimal):
# def __repr__(self):
# return 'invalid json'
class TestSubclass(TestCase):
def test_int(self):
self.assertEqual(json.dumps(AlternateInt(1)), '1')
self.assertEqual(json.dumps(AlternateInt(-1)), '-1')
self.assertEqual(json.loads(json.dumps({AlternateInt(1): 1})), {'1': 1})
def test_float(self):
self.assertEqual(json.dumps(AlternateFloat(1.0)), '1.0')
self.assertEqual(json.dumps(AlternateFloat(-1.0)), '-1.0')
self.assertEqual(json.loads(json.dumps({AlternateFloat(1.0): 1})), {'1.0': 1})
# NOTE: Decimal subclasses are not supported as-is
# def test_decimal(self):
# self.assertEqual(json.dumps(AlternateDecimal('1.0')), '1.0')
# self.assertEqual(json.dumps(AlternateDecimal('-1.0')), '-1.0')

114
simplejson/tests/test_tool.py
View File

@ -1,114 +0,0 @@
from __future__ import with_statement
import os
import sys
import textwrap
import unittest
import subprocess
import tempfile
try:
# Python 3.x
from test.support import strip_python_stderr
except ImportError:
# Python 2.6+
try:
from test.test_support import strip_python_stderr
except ImportError:
# Python 2.5
import re
def strip_python_stderr(stderr):
return re.sub(
r"\[\d+ refs\]\r?\n?$".encode(),
"".encode(),
stderr).strip()
def open_temp_file():
if sys.version_info >= (2, 6):
file = tempfile.NamedTemporaryFile(delete=False)
filename = file.name
else:
fd, filename = tempfile.mkstemp()
file = os.fdopen(fd, 'w+b')
return file, filename
class TestTool(unittest.TestCase):
data = """
[["blorpie"],[ "whoops" ] , [
],\t"d-shtaeou",\r"d-nthiouh",
"i-vhbjkhnth", {"nifty":87}, {"morefield" :\tfalse,"field"
:"yes"} ]
"""
expect = textwrap.dedent("""\
[
[
"blorpie"
],
[
"whoops"
],
[],
"d-shtaeou",
"d-nthiouh",
"i-vhbjkhnth",
{
"nifty": 87
},
{
"field": "yes",
"morefield": false
}
]
""")
def runTool(self, args=None, data=None):
argv = [sys.executable, '-m', 'simplejson.tool']
if args:
argv.extend(args)
proc = subprocess.Popen(argv,
stdin=subprocess.PIPE,
stderr=subprocess.PIPE,
stdout=subprocess.PIPE)
out, err = proc.communicate(data)
self.assertEqual(strip_python_stderr(err), ''.encode())
self.assertEqual(proc.returncode, 0)
return out.decode('utf8').splitlines()
def test_stdin_stdout(self):
self.assertEqual(
self.runTool(data=self.data.encode()),
self.expect.splitlines())
def test_infile_stdout(self):
infile, infile_name = open_temp_file()
try:
infile.write(self.data.encode())
infile.close()
self.assertEqual(
self.runTool(args=[infile_name]),
self.expect.splitlines())
finally:
os.unlink(infile_name)
def test_infile_outfile(self):
infile, infile_name = open_temp_file()
try:
infile.write(self.data.encode())
infile.close()
# outfile will get overwritten by tool, so the delete
# may not work on some platforms. Do it manually.
outfile, outfile_name = open_temp_file()
try:
outfile.close()
self.assertEqual(
self.runTool(args=[infile_name, outfile_name]),
[])
with open(outfile_name, 'rb') as f:
self.assertEqual(
f.read().decode('utf8').splitlines(),
self.expect.splitlines()
)
finally:
os.unlink(outfile_name)
finally:
os.unlink(infile_name)

47
simplejson/tests/test_tuple.py
View File

@ -1,47 +0,0 @@
import unittest
from simplejson.compat import StringIO
import simplejson as json
class TestTuples(unittest.TestCase):
def test_tuple_array_dumps(self):
t = (1, 2, 3)
expect = json.dumps(list(t))
# Default is True
self.assertEqual(expect, json.dumps(t))
self.assertEqual(expect, json.dumps(t, tuple_as_array=True))
self.assertRaises(TypeError, json.dumps, t, tuple_as_array=False)
# Ensure that the "default" does not get called
self.assertEqual(expect, json.dumps(t, default=repr))
self.assertEqual(expect, json.dumps(t, tuple_as_array=True,
default=repr))
# Ensure that the "default" gets called
self.assertEqual(
json.dumps(repr(t)),
json.dumps(t, tuple_as_array=False, default=repr))
def test_tuple_array_dump(self):
t = (1, 2, 3)
expect = json.dumps(list(t))
# Default is True
sio = StringIO()
json.dump(t, sio)
self.assertEqual(expect, sio.getvalue())
sio = StringIO()
json.dump(t, sio, tuple_as_array=True)
self.assertEqual(expect, sio.getvalue())
self.assertRaises(TypeError, json.dump, t, StringIO(),
tuple_as_array=False)
# Ensure that the "default" does not get called
sio = StringIO()
json.dump(t, sio, default=repr)
self.assertEqual(expect, sio.getvalue())
sio = StringIO()
json.dump(t, sio, tuple_as_array=True, default=repr)
self.assertEqual(expect, sio.getvalue())
# Ensure that the "default" gets called
sio = StringIO()
json.dump(t, sio, tuple_as_array=False, default=repr)
self.assertEqual(
json.dumps(repr(t)),
sio.getvalue())

154
simplejson/tests/test_unicode.py
View File

@ -1,154 +0,0 @@
import sys
import codecs
from unittest import TestCase
import simplejson as json
from simplejson.compat import unichr, text_type, b, BytesIO
class TestUnicode(TestCase):
def test_encoding1(self):
encoder = json.JSONEncoder(encoding='utf-8')
u = u'\N{GREEK SMALL LETTER ALPHA}\N{GREEK CAPITAL LETTER OMEGA}'
s = u.encode('utf-8')
ju = encoder.encode(u)
js = encoder.encode(s)
self.assertEqual(ju, js)
def test_encoding2(self):
u = u'\N{GREEK SMALL LETTER ALPHA}\N{GREEK CAPITAL LETTER OMEGA}'
s = u.encode('utf-8')
ju = json.dumps(u, encoding='utf-8')
js = json.dumps(s, encoding='utf-8')
self.assertEqual(ju, js)
def test_encoding3(self):
u = u'\N{GREEK SMALL LETTER ALPHA}\N{GREEK CAPITAL LETTER OMEGA}'
j = json.dumps(u)
self.assertEqual(j, '"\\u03b1\\u03a9"')
def test_encoding4(self):
u = u'\N{GREEK SMALL LETTER ALPHA}\N{GREEK CAPITAL LETTER OMEGA}'
j = json.dumps([u])
self.assertEqual(j, '["\\u03b1\\u03a9"]')
def test_encoding5(self):
u = u'\N{GREEK SMALL LETTER ALPHA}\N{GREEK CAPITAL LETTER OMEGA}'
j = json.dumps(u, ensure_ascii=False)
self.assertEqual(j, u'"' + u + u'"')
def test_encoding6(self):
u = u'\N{GREEK SMALL LETTER ALPHA}\N{GREEK CAPITAL LETTER OMEGA}'
j = json.dumps([u], ensure_ascii=False)
self.assertEqual(j, u'["' + u + u'"]')
def test_big_unicode_encode(self):
u = u'\U0001d120'
self.assertEqual(json.dumps(u), '"\\ud834\\udd20"')
self.assertEqual(json.dumps(u, ensure_ascii=False), u'"\U0001d120"')
def test_big_unicode_decode(self):
u = u'z\U0001d120x'
self.assertEqual(json.loads('"' + u + '"'), u)
self.assertEqual(json.loads('"z\\ud834\\udd20x"'), u)
def test_unicode_decode(self):
for i in range(0, 0xd7ff):
u = unichr(i)
#s = '"\\u{0:04x}"'.format(i)
s = '"\\u%04x"' % (i,)
self.assertEqual(json.loads(s), u)
def test_object_pairs_hook_with_unicode(self):
s = u'{"xkd":1, "kcw":2, "art":3, "hxm":4, "qrt":5, "pad":6, "hoy":7}'
p = [(u"xkd", 1), (u"kcw", 2), (u"art", 3), (u"hxm", 4),
(u"qrt", 5), (u"pad", 6), (u"hoy", 7)]
self.assertEqual(json.loads(s), eval(s))
self.assertEqual(json.loads(s, object_pairs_hook=lambda x: x), p)
od = json.loads(s, object_pairs_hook=json.OrderedDict)
self.assertEqual(od, json.OrderedDict(p))
self.assertEqual(type(od), json.OrderedDict)
# the object_pairs_hook takes priority over the object_hook
self.assertEqual(json.loads(s,
object_pairs_hook=json.OrderedDict,
object_hook=lambda x: None),
json.OrderedDict(p))
def test_default_encoding(self):
self.assertEqual(json.loads(u'{"a": "\xe9"}'.encode('utf-8')),
{'a': u'\xe9'})
def test_unicode_preservation(self):
self.assertEqual(type(json.loads(u'""')), text_type)
self.assertEqual(type(json.loads(u'"a"')), text_type)
self.assertEqual(type(json.loads(u'["a"]')[0]), text_type)
def test_ensure_ascii_false_returns_unicode(self):
# http://code.google.com/p/simplejson/issues/detail?id=48
self.assertEqual(type(json.dumps([], ensure_ascii=False)), text_type)
self.assertEqual(type(json.dumps(0, ensure_ascii=False)), text_type)
self.assertEqual(type(json.dumps({}, ensure_ascii=False)), text_type)
self.assertEqual(type(json.dumps("", ensure_ascii=False)), text_type)
def test_ensure_ascii_false_bytestring_encoding(self):
# http://code.google.com/p/simplejson/issues/detail?id=48
doc1 = {u'quux': b('Arr\xc3\xaat sur images')}
doc2 = {u'quux': u'Arr\xeat sur images'}
doc_ascii = '{"quux": "Arr\\u00eat sur images"}'
doc_unicode = u'{"quux": "Arr\xeat sur images"}'
self.assertEqual(json.dumps(doc1), doc_ascii)
self.assertEqual(json.dumps(doc2), doc_ascii)
self.assertEqual(json.dumps(doc1, ensure_ascii=False), doc_unicode)
self.assertEqual(json.dumps(doc2, ensure_ascii=False), doc_unicode)
def test_ensure_ascii_linebreak_encoding(self):
# http://timelessrepo.com/json-isnt-a-javascript-subset
s1 = u'\u2029\u2028'
s2 = s1.encode('utf8')
expect = '"\\u2029\\u2028"'
expect_non_ascii = u'"\u2029\u2028"'
self.assertEqual(json.dumps(s1), expect)
self.assertEqual(json.dumps(s2), expect)
self.assertEqual(json.dumps(s1, ensure_ascii=False), expect_non_ascii)
self.assertEqual(json.dumps(s2, ensure_ascii=False), expect_non_ascii)
def test_invalid_escape_sequences(self):
# incomplete escape sequence
self.assertRaises(json.JSONDecodeError, json.loads, '"\\u')
self.assertRaises(json.JSONDecodeError, json.loads, '"\\u1')
self.assertRaises(json.JSONDecodeError, json.loads, '"\\u12')
self.assertRaises(json.JSONDecodeError, json.loads, '"\\u123')
self.assertRaises(json.JSONDecodeError, json.loads, '"\\u1234')
# invalid escape sequence
self.assertRaises(json.JSONDecodeError, json.loads, '"\\u123x"')
self.assertRaises(json.JSONDecodeError, json.loads, '"\\u12x4"')
self.assertRaises(json.JSONDecodeError, json.loads, '"\\u1x34"')
self.assertRaises(json.JSONDecodeError, json.loads, '"\\ux234"')
if sys.maxunicode > 65535:
# invalid escape sequence for low surrogate
self.assertRaises(json.JSONDecodeError, json.loads, '"\\ud800\\u"')
self.assertRaises(json.JSONDecodeError, json.loads, '"\\ud800\\u0"')
self.assertRaises(json.JSONDecodeError, json.loads, '"\\ud800\\u00"')
self.assertRaises(json.JSONDecodeError, json.loads, '"\\ud800\\u000"')
self.assertRaises(json.JSONDecodeError, json.loads, '"\\ud800\\u000x"')
self.assertRaises(json.JSONDecodeError, json.loads, '"\\ud800\\u00x0"')
self.assertRaises(json.JSONDecodeError, json.loads, '"\\ud800\\u0x00"')
self.assertRaises(json.JSONDecodeError, json.loads, '"\\ud800\\ux000"')
def test_ensure_ascii_still_works(self):
# in the ascii range, ensure that everything is the same
for c in map(unichr, range(0, 127)):
self.assertEqual(
json.dumps(c, ensure_ascii=False),
json.dumps(c))
snowman = u'\N{SNOWMAN}'
self.assertEqual(
json.dumps(c, ensure_ascii=False),
'"' + c + '"')
def test_strip_bom(self):
content = u"\u3053\u3093\u306b\u3061\u308f"
json_doc = codecs.BOM_UTF8 + b(json.dumps(content))
self.assertEqual(json.load(BytesIO(json_doc)), content)
for doc in json_doc, json_doc.decode('utf8'):
self.assertEqual(json.loads(doc), content)

42
simplejson/tool.py
View File

@ -1,42 +0,0 @@
r"""Command-line tool to validate and pretty-print JSON
Usage::
$ echo '{"json":"obj"}' | python -m simplejson.tool
{
"json": "obj"
}
$ echo '{ 1.2:3.4}' | python -m simplejson.tool
Expecting property name: line 1 column 2 (char 2)
"""
from __future__ import with_statement
import sys
import simplejson as json
def main():
if len(sys.argv) == 1:
infile = sys.stdin
outfile = sys.stdout
elif len(sys.argv) == 2:
infile = open(sys.argv[1], 'r')
outfile = sys.stdout
elif len(sys.argv) == 3:
infile = open(sys.argv[1], 'r')
outfile = open(sys.argv[2], 'w')
else:
raise SystemExit(sys.argv[0] + " [infile [outfile]]")
with infile:
try:
obj = json.load(infile,
object_pairs_hook=json.OrderedDict,
use_decimal=True)
except ValueError:
raise SystemExit(sys.exc_info()[1])
with outfile:
json.dump(obj, outfile, sort_keys=True, indent=' ', use_decimal=True)
outfile.write('\n')
if __name__ == '__main__':
main()

9
sloctest.c Normal file
View File

@ -0,0 +1,9 @@
/* This is a test. Should produce 6 SLOC.
*/
#include <stdio.h>
#define HI 10
main() {
a = 1; /* hi */
"hi"
}

10
test/test.c → test.c
View File

@ -47,11 +47,7 @@ demo2() {
int n;
_mbscpy(d,s); /* like strcpy, this doesn't check for buffer overflow */
memcpy(d,s); // fail - no size
memcpy(d, s, sizeof(d)); // pass
memcpy(& n, s, sizeof( n )); // pass
memcpy(&n,s,sizeof(s)); // fail - sizeof not of destination
memcpy(d,s,n); // fail - size unguessable
memcpy(d,s);
CopyMemory(d,s);
lstrcat(d,s);
strncpy(d,s);
@ -77,10 +73,6 @@ demo2() {
SetSecurityDescriptorDacl(&sd,TRUE,NULL,FALSE);
/* This one is a bad idea - first param shouldn't be NULL */
CreateProcess(NULL, "C:\\Program Files\\GoodGuy\\GoodGuy.exe -x", "");
/* Bad, may load from current directory */
(void) LoadLibraryEx(L"user32.dll", nullptr, LOAD_LIBRARY_AS_DATAFILE);
/* This should be ignored, since it's loading only from System32 */
(void) LoadLibraryEx(L"user32.dll", nullptr, LOAD_LIBRARY_SEARCH_SYSTEM32 | LOAD_LIBRARY_REQUIRE_SIGNED_TARGET);
/* Test interaction of quote characters */
printf("%c\n", 'x');
printf("%c\n", '"');

7
test/correct-results-004.txt
View File

@ -1,7 +0,0 @@
test.c:32:2: [5] (buffer) gets:Does not check for buffer overflows (CWE-120, CWE-20). Use fgets() instead.
test.c:60:3: [5] (buffer) strncat:Easily used incorrectly (e.g., incorrectly computing the correct maximum size to add) [MS-banned] (CWE-120). Consider strcat_s, strlcat, snprintf, or automatically resizing strings. Risk is high; the length parameter appears to be a constant, instead of computing the number of characters left.
test.c:61:3: [5] (buffer) _tcsncat:Easily used incorrectly (e.g., incorrectly computing the correct maximum size to add) [MS-banned] (CWE-120). Consider strcat_s, strlcat, or automatically resizing strings. Risk is high; the length parameter appears to be a constant, instead of computing the number of characters left.
test.c:64:3: [5] (buffer) MultiByteToWideChar:Requires maximum length in CHARACTERS, not bytes (CWE-120). Risk is high, it appears that the size is given as bytes, but the function requires size as characters.
test.c:66:3: [5] (buffer) MultiByteToWideChar:Requires maximum length in CHARACTERS, not bytes (CWE-120). Risk is high, it appears that the size is given as bytes, but the function requires size as characters.
test.c:77:3: [5] (misc) SetSecurityDescriptorDacl:Never create NULL ACLs; an attacker can set it to Everyone (Deny All Access), which would even forbid administrator access (CWE-732).
test.c:77:3: [5] (misc) SetSecurityDescriptorDacl:Never create NULL ACLs; an attacker can set it to Everyone (Deny All Access), which would even forbid administrator access (CWE-732).

1
test/correct-results-005.txt
View File

@ -1 +0,0 @@
test-patched.c:13:2: [4] (buffer) strcpy:Does not check for buffer overflows when copying to destination [MS-banned] (CWE-120). Consider using snprintf, strcpy_s, or strlcpy (warning: strncpy easily misused).

7
test/correct-results-006.txt
View File

@ -1,7 +0,0 @@
test.c:32:2: [5] (buffer) gets:Does not check for buffer overflows (CWE-120, CWE-20). Use fgets() instead.
test.c:60:3: [5] (buffer) strncat:Easily used incorrectly (e.g., incorrectly computing the correct maximum size to add) [MS-banned] (CWE-120). Consider strcat_s, strlcat, snprintf, or automatically resizing strings. Risk is high; the length parameter appears to be a constant, instead of computing the number of characters left.
test.c:61:3: [5] (buffer) _tcsncat:Easily used incorrectly (e.g., incorrectly computing the correct maximum size to add) [MS-banned] (CWE-120). Consider strcat_s, strlcat, or automatically resizing strings. Risk is high; the length parameter appears to be a constant, instead of computing the number of characters left.
test.c:64:3: [5] (buffer) MultiByteToWideChar:Requires maximum length in CHARACTERS, not bytes (CWE-120). Risk is high, it appears that the size is given as bytes, but the function requires size as characters.
test.c:66:3: [5] (buffer) MultiByteToWideChar:Requires maximum length in CHARACTERS, not bytes (CWE-120). Risk is high, it appears that the size is given as bytes, but the function requires size as characters.
test.c:77:3: [5] (misc) SetSecurityDescriptorDacl:Never create NULL ACLs; an attacker can set it to Everyone (Deny All Access), which would even forbid administrator access (CWE-732).
test.c:77:3: [5] (misc) SetSecurityDescriptorDacl:Never create NULL ACLs; an attacker can set it to Everyone (Deny All Access), which would even forbid administrator access (CWE-732).

18
test/correct-results-008.txt
View File

@ -1,18 +0,0 @@
Flawfinder version 2.0.19, (C) 2001-2019 David A. Wheeler.
Showing hits not in test-saved-hitlist-008.txt
Number of rules (primarily dangerous function names) in C/C++ ruleset: 222
ANALYSIS SUMMARY:
No hits found.
Lines analyzed = 125
Physical Source Lines of Code (SLOC) = 86
Hits@level = [0] 0 [1] 0 [2] 0 [3] 0 [4] 0 [5] 0
Hits@level+ = [0+] 0 [1+] 0 [2+] 0 [3+] 0 [4+] 0 [5+] 0
Hits/KSLOC@level+ = [0+] 0 [1+] 0 [2+] 0 [3+] 0 [4+] 0 [5+] 0
Suppressed hits = 2 (use --neverignore to show them)
Minimum risk level = 1
There may be other security vulnerabilities; review your code!
See 'Secure Programming HOWTO'
(https://dwheeler.com/secure-programs) for more information.

40
test/correct-results.csv
View File

@ -1,40 +0,0 @@
File,Line,Column,DefaultLevel,Level,Category,Name,Warning,Suggestion,Note,CWEs,Context,Fingerprint,ToolVersion,RuleId,HelpUri
test.c,32,2,5,5,buffer,gets,"Does not check for buffer overflows (CWE-120, CWE-20).",Use fgets() instead.,,"CWE-120, CWE-20", gets(f);,6a5bb383fb44030b0d9428b17359e94ba3979bc1ce702be450427f85592c649a,2.0.19,FF1014,https://cwe.mitre.org/data/definitions/120.html
test.c,60,3,1,5,buffer,strncat,"Easily used incorrectly (e.g., incorrectly computing the correct maximum size to add) [MS-banned] (CWE-120).","Consider strcat_s, strlcat, snprintf, or automatically resizing strings.","Risk is high; the length parameter appears to be a constant, instead of computing the number of characters left.",CWE-120," strncat(d,s,sizeof(d)); /* Misuse - this should be flagged as riskier. */",cbd19c308547e79af13436d8f7dbcf6c62e49e4f62ba9aee38fbef29e0772f74,2.0.19,FF1010,https://cwe.mitre.org/data/definitions/120.html
test.c,61,3,1,5,buffer,_tcsncat,"Easily used incorrectly (e.g., incorrectly computing the correct maximum size to add) [MS-banned] (CWE-120).","Consider strcat_s, strlcat, or automatically resizing strings.","Risk is high; the length parameter appears to be a constant, instead of computing the number of characters left.",CWE-120," _tcsncat(d,s,sizeof(d)); /* Misuse - flag as riskier */",c3f6ba2c710efc878e66df4578894fd408452cb7cdec7ae6f492a3b1796f8c42,2.0.19,FF1011,https://cwe.mitre.org/data/definitions/120.html
test.c,64,3,2,5,buffer,MultiByteToWideChar,"Requires maximum length in CHARACTERS, not bytes (CWE-120).",,"Risk is high, it appears that the size is given as bytes, but the function requires size as characters.",CWE-120," MultiByteToWideChar(CP_ACP,0,szName,-1,wszUserName,sizeof(wszUserName));",4f5b73ff337a54d6e1d9a369659ca0ddb4f80e6b7e38a17e5b112f6d3e266e69,2.0.19,FF1023,https://cwe.mitre.org/data/definitions/120.html
test.c,66,3,2,5,buffer,MultiByteToWideChar,"Requires maximum length in CHARACTERS, not bytes (CWE-120).",,"Risk is high, it appears that the size is given as bytes, but the function requires size as characters.",CWE-120," MultiByteToWideChar(CP_ACP,0,szName,-1,wszUserName,sizeof wszUserName);",9ecdc1e903acc16a646bf7909a630ae22a7593b70952c39ce6bd9c5a23fad0fd,2.0.19,FF1023,https://cwe.mitre.org/data/definitions/120.html
test.c,77,3,5,5,misc,SetSecurityDescriptorDacl,"Never create NULL ACLs; an attacker can set it to Everyone (Deny All Access), which would even forbid administrator access (CWE-732).",,,CWE-732," SetSecurityDescriptorDacl(&sd,TRUE,NULL,FALSE);",5fed1e135b593b4c943e66e89a26ff131eba18b83a32a8af37d1c0bd7b01aadb,2.0.19,FF1060,https://cwe.mitre.org/data/definitions/732.html
test.c,77,3,5,5,misc,SetSecurityDescriptorDacl,"Never create NULL ACLs; an attacker can set it to Everyone (Deny All Access), which would even forbid administrator access (CWE-732).",,,CWE-732," SetSecurityDescriptorDacl(&sd,TRUE,NULL,FALSE);",5fed1e135b593b4c943e66e89a26ff131eba18b83a32a8af37d1c0bd7b01aadb,2.0.19,FF1060,https://cwe.mitre.org/data/definitions/732.html
test.c,17,2,4,4,buffer,strcpy,Does not check for buffer overflows when copying to destination [MS-banned] (CWE-120).,"Consider using snprintf, strcpy_s, or strlcpy (warning: strncpy easily misused).",,CWE-120," strcpy(b, a);",c01c8472bb53022e912da4da2faebc67d537855da324020c44bfd5e608a79b77,2.0.19,FF1001,https://cwe.mitre.org/data/definitions/120.html
test.c,20,2,4,4,buffer,sprintf,Does not check for buffer overflows (CWE-120).,"Use sprintf_s, snprintf, or vsnprintf.",,CWE-120," sprintf(s, ""hello %s"", bug);",814237858ab012010f3355a49480dd6fa0a2cb8cf8356a98ac1c17c9febf6521,2.0.19,FF1015,https://cwe.mitre.org/data/definitions/120.html
test.c,21,2,4,4,buffer,sprintf,Does not check for buffer overflows (CWE-120).,"Use sprintf_s, snprintf, or vsnprintf.",,CWE-120," sprintf(s, gettext(""hello %s""), bug);",b793f18f143fb2297c49e0639384ad73db86eb01a44377aa4d5d09b44b03d747,2.0.19,FF1015,https://cwe.mitre.org/data/definitions/120.html
test.c,22,2,4,4,format,sprintf,Potential format string problem (CWE-134).,Make format string constant.,,CWE-134," sprintf(s, unknown, bug);",16ebc2ff96ee4bab2695783709e97b597ca9c8b8cc149e33aed859f0fafd3431,2.0.19,FF1015,https://cwe.mitre.org/data/definitions/134.html
test.c,23,2,4,4,format,printf,"If format strings can be influenced by an attacker, they can be exploited (CWE-134).",Use a constant for the format specification.,,CWE-134," printf(bf, x);",46f42896019245d2dffc4caf4fe018b073ce2a58203676eaa28b6374558a5b5d,2.0.19,FF1016,https://cwe.mitre.org/data/definitions/134.html
test.c,25,2,4,4,buffer,scanf,"The scanf() family's %s operation, without a limit specification, permits buffer overflows (CWE-120, CWE-20).","Specify a limit to %s, or use a different input function.",,"CWE-120, CWE-20"," scanf(""%s"", s);",3f169dd9fe508f70438f818770a3cb8b0f228e4245ea11a929a5fb0a7839fd5f,2.0.19,FF1020,https://cwe.mitre.org/data/definitions/120.html
test.c,27,2,4,4,buffer,scanf,"The scanf() family's %s operation, without a limit specification, permits buffer overflows (CWE-120, CWE-20).","Specify a limit to %s, or use a different input function.",,"CWE-120, CWE-20"," scanf(""%s"", s);",3f169dd9fe508f70438f818770a3cb8b0f228e4245ea11a929a5fb0a7839fd5f,2.0.19,FF1020,https://cwe.mitre.org/data/definitions/120.html
test.c,38,2,4,4,format,syslog,"If syslog's format strings can be influenced by an attacker, they can be exploited (CWE-134).",Use a constant format string for syslog.,,CWE-134," syslog(LOG_ERR, attacker_string);",22e98963d5af7b197a090bd522d2d39b8d8ee7bdf08453fd2008939c92cd9677,2.0.19,FF1018,https://cwe.mitre.org/data/definitions/134.html
test.c,49,3,4,4,buffer,_mbscpy,Does not check for buffer overflows when copying to destination [MS-banned] (CWE-120).,Consider using a function version that stops copying at the end of the buffer.,,CWE-120," _mbscpy(d,s); /* like strcpy, this doesn't check for buffer overflow */",e00a4a1a0a3603db98a23fcff3c9cdfd9012f5a81826814d9508e0f22089b993,2.0.19,FF1003,https://cwe.mitre.org/data/definitions/120.html
test.c,56,3,4,4,buffer,lstrcat,Does not check for buffer overflows when concatenating to destination [MS-banned] (CWE-120).,,,CWE-120," lstrcat(d,s);",364b4c512862fdccbca27d2fa7737995b5d24b637a760976c940ae636218d340,2.0.19,FF1006,https://cwe.mitre.org/data/definitions/120.html
test.c,79,3,3,3,shell,CreateProcess,This causes a new process to execute and is difficult to use safely (CWE-78).,"Specify the application path in the first argument, NOT as part of the second, or embedded spaces could allow an attacker to force a different program to run.",,CWE-78," CreateProcess(NULL, ""C:\\Program Files\\GoodGuy\\GoodGuy.exe -x"", """");",3c712b38d0857bde3832d85ad35ac9859be55c5f5f1c20af659a577dd4d0acbf,2.0.19,FF1046,https://cwe.mitre.org/data/definitions/78.html
test.c,79,3,3,3,shell,CreateProcess,This causes a new process to execute and is difficult to use safely (CWE-78).,"Specify the application path in the first argument, NOT as part of the second, or embedded spaces could allow an attacker to force a different program to run.",,CWE-78," CreateProcess(NULL, ""C:\\Program Files\\GoodGuy\\GoodGuy.exe -x"", """");",3c712b38d0857bde3832d85ad35ac9859be55c5f5f1c20af659a577dd4d0acbf,2.0.19,FF1046,https://cwe.mitre.org/data/definitions/78.html
test.c,81,10,3,3,misc,LoadLibraryEx,"Ensure that the full path to the library is specified, or current directory may be used (CWE-829, CWE-20).",Use a flag like LOAD_LIBRARY_SEARCH_SYSTEM32 or LOAD_LIBRARY_SEARCH_APPLICATION_DIR to search only desired folders.,,"CWE-829, CWE-20"," (void) LoadLibraryEx(L""user32.dll"", nullptr, LOAD_LIBRARY_AS_DATAFILE);",b1f99ecaa31e682487d795afbf03282fd56ad9f2aa630d0196219b277d2a68c9,2.0.19,FF1059,https://cwe.mitre.org/data/definitions/829.html
test.c,99,20,3,3,buffer,getopt_long,"Some older implementations do not protect against internal buffer overflows (CWE-120, CWE-20).","Check implementation on installation, or limit the size of all string inputs.",,"CWE-120, CWE-20"," while ((optc = getopt_long (argc, argv, ""a"",longopts, NULL )) != EOF) {",5bedf6e5bccf596008ef191ec4c5d4cc51a32cff0c05ef62d5f10fab93d0cc24,2.0.19,FF1027,https://cwe.mitre.org/data/definitions/120.html
test.c,16,2,4,2,buffer,strcpy,Does not check for buffer overflows when copying to destination [MS-banned] (CWE-120).,"Consider using snprintf, strcpy_s, or strlcpy (warning: strncpy easily misused).",Risk is low because the source is a constant string.,CWE-120," strcpy(a, gettext(""Hello there"")); // Did this work?",d64070fb93ff0bb797fb926f4dddc7212d42f77e288d5ceb0cd30ed2979fa28d,2.0.19,FF1001,https://cwe.mitre.org/data/definitions/120.html
test.c,19,2,4,2,buffer,sprintf,Does not check for buffer overflows (CWE-120).,"Use sprintf_s, snprintf, or vsnprintf.",Risk is low because the source has a constant maximum length.,CWE-120," sprintf(s, ""hello"");",907b46be1c3ea7b38f90a4d1b0f43b7751cd8cbe38fae840930ff006b702157d,2.0.19,FF1015,https://cwe.mitre.org/data/definitions/120.html
test.c,45,3,2,2,buffer,char,"Statically-sized arrays can be improperly restricted, leading to potential overflows or other issues (CWE-119!/CWE-120).","Perform bounds checking, use functions that limit length, or ensure that the size is larger than the maximum possible length.",,CWE-119!/CWE-120, char d[20];,36c87517700337a59cc3ad3218cfdde56cad37d69cdeccee5a55ab232d5c7946,2.0.19,FF1013,https://cwe.mitre.org/data/definitions/119.html
test.c,46,3,2,2,buffer,char,"Statically-sized arrays can be improperly restricted, leading to potential overflows or other issues (CWE-119!/CWE-120).","Perform bounds checking, use functions that limit length, or ensure that the size is larger than the maximum possible length.",,CWE-119!/CWE-120, char s[20];,213de8e8815fc84c423b55fd845fea541f25744718e486234364bb457863b597,2.0.19,FF1013,https://cwe.mitre.org/data/definitions/119.html
test.c,50,3,2,2,buffer,memcpy,Does not check for buffer overflows when copying to destination (CWE-120).,Make sure destination can always hold the source data.,,CWE-120," memcpy(d,s); // fail - no size",e667b352fb0748c67b607b11577b11bad87545779c39923e61839dd04056055f,2.0.19,FF1004,https://cwe.mitre.org/data/definitions/120.html
test.c,53,3,2,2,buffer,memcpy,Does not check for buffer overflows when copying to destination (CWE-120).,Make sure destination can always hold the source data.,,CWE-120," memcpy(&n,s,sizeof(s)); // fail - sizeof not of destination",01bcc2c8ba2d928ac3315b4dcc6593042ea05e62888a10a6d2cf16797a65ed32,2.0.19,FF1004,https://cwe.mitre.org/data/definitions/120.html
test.c,54,3,2,2,buffer,memcpy,Does not check for buffer overflows when copying to destination (CWE-120).,Make sure destination can always hold the source data.,,CWE-120," memcpy(d,s,n); // fail - size unguessable",2517a2fb5981193a6017cca660d16e85aab133706cbec302df97aaa623fc77ef,2.0.19,FF1004,https://cwe.mitre.org/data/definitions/120.html
test.c,55,3,2,2,buffer,CopyMemory,Does not check for buffer overflows when copying to destination (CWE-120).,Make sure destination can always hold the source data.,,CWE-120," CopyMemory(d,s);",977f8c805ddd76ff32e0f7aea08701ba97d9ce6955136e98b308ed4f70eb2e11,2.0.19,FF1004,https://cwe.mitre.org/data/definitions/120.html
test.c,105,7,2,2,misc,fopen,"Check when opening files - can an attacker redirect it (via symlinks), force the opening of special file type (e.g., device files), move things around to create a race condition, control its ancestors, or change its contents? (CWE-362).",,,CWE-362," f = fopen(""/etc/passwd"", ""r""); ",2ec6928c77a8b54caa61d0459f367c4394ee1f5e6f488753f587bfa9c780bad8,2.0.19,FF1040,https://cwe.mitre.org/data/definitions/362.html
test.c,15,2,4,1,buffer,strcpy,Does not check for buffer overflows when copying to destination [MS-banned] (CWE-120).,"Consider using snprintf, strcpy_s, or strlcpy (warning: strncpy easily misused).",Risk is low because the source is a constant character.,CWE-120," strcpy(a, ""\n""); // Did this work?",0badc5f4c500d17b42794feaca54ee0f49e607a32510af3ed749579001017edb,2.0.19,FF1001,https://cwe.mitre.org/data/definitions/120.html
test.c,18,2,4,1,buffer,sprintf,Does not check for buffer overflows (CWE-120).,"Use sprintf_s, snprintf, or vsnprintf.",Risk is low because the source is a constant character.,CWE-120," sprintf(s, ""\n"");",c65fbd60851f3c8ace22332805966606488c0d242c1823493c582e267609b1a7,2.0.19,FF1015,https://cwe.mitre.org/data/definitions/120.html
test.c,26,2,4,1,buffer,scanf,It's unclear if the %s limit in the format string is small enough (CWE-120).,"Check that the limit is sufficiently small, or use a different input function.",,CWE-120," scanf(""%10s"", s);",e24c4c801f10acfa93098b2bef58524efe4f88237f2dd8b58be9afa838616afe,2.0.19,FF1020,https://cwe.mitre.org/data/definitions/120.html
test.c,57,3,1,1,buffer,strncpy,Easily used incorrectly; doesn't always \0-terminate or check for invalid pointers [MS-banned] (CWE-120).,,,CWE-120," strncpy(d,s);",8fa14bf72393a00f667ffcc06b7b7e5f0b6d2f16d8d67444db06b0deb35b5f5e,2.0.19,FF1008,https://cwe.mitre.org/data/definitions/120.html
test.c,58,3,1,1,buffer,_tcsncpy,Easily used incorrectly; doesn't always \0-terminate or check for invalid pointers [MS-banned] (CWE-120).,,,CWE-120," _tcsncpy(d,s);",691fabd4ca960a00e4c538eee0187ee0fdf59bd43dd71e792c14175150369b8b,2.0.19,FF1009,https://cwe.mitre.org/data/definitions/120.html
test.c,59,3,1,1,buffer,strncat,"Easily used incorrectly (e.g., incorrectly computing the correct maximum size to add) [MS-banned] (CWE-120).","Consider strcat_s, strlcat, snprintf, or automatically resizing strings.",,CWE-120," strncat(d,s,10);",dd92f996a554bfbc038bea27640ba25dcf298383140a8330dca7cdacf493a701,2.0.19,FF1010,https://cwe.mitre.org/data/definitions/120.html
test.c,62,7,1,1,buffer,strlen,Does not handle strings that are not \0-terminated; if given one it may perform an over-read (it could cause a crash if unprotected) (CWE-126).,,,CWE-126, n = strlen(d);,db7201c7df7f543ea76febb060bda167e414e71e3d18095fe1def69f8c47a4f6,2.0.19,FF1022,https://cwe.mitre.org/data/definitions/126.html
test.c,68,3,2,1,buffer,MultiByteToWideChar,"Requires maximum length in CHARACTERS, not bytes (CWE-120).",,"Risk is very low, the length appears to be in characters not bytes.",CWE-120," MultiByteToWideChar(CP_ACP,0,szName,-1,wszUserName,sizeof(wszUserName)/sizeof(wszUserName[0]));",1813fc329227b38abae867d8023a9e29c7517d679fe55c86f8300dde681b6470,2.0.19,FF1023,https://cwe.mitre.org/data/definitions/120.html
test.c,70,3,2,1,buffer,MultiByteToWideChar,"Requires maximum length in CHARACTERS, not bytes (CWE-120).",,"Risk is very low, the length appears to be in characters not bytes.",CWE-120," MultiByteToWideChar(CP_ACP,0,szName,-1,wszUserName,sizeof wszUserName /sizeof(wszUserName[0]));",7c6cdcb10ad3a16b8bfd56e3dac84829f9bc3e39d4dde74a2be9bbe000102fc5,2.0.19,FF1023,https://cwe.mitre.org/data/definitions/120.html
1 File Line Column DefaultLevel Level Category Name Warning Suggestion Note CWEs Context Fingerprint ToolVersion RuleId HelpUri
2 test.c 32 2 5 5 buffer gets Does not check for buffer overflows (CWE-120, CWE-20). Use fgets() instead. CWE-120, CWE-20 gets(f); 6a5bb383fb44030b0d9428b17359e94ba3979bc1ce702be450427f85592c649a 2.0.19 FF1014 https://cwe.mitre.org/data/definitions/120.html
3 test.c 60 3 1 5 buffer strncat Easily used incorrectly (e.g., incorrectly computing the correct maximum size to add) [MS-banned] (CWE-120). Consider strcat_s, strlcat, snprintf, or automatically resizing strings. Risk is high; the length parameter appears to be a constant, instead of computing the number of characters left. CWE-120 strncat(d,s,sizeof(d)); /* Misuse - this should be flagged as riskier. */ cbd19c308547e79af13436d8f7dbcf6c62e49e4f62ba9aee38fbef29e0772f74 2.0.19 FF1010 https://cwe.mitre.org/data/definitions/120.html
4 test.c 61 3 1 5 buffer _tcsncat Easily used incorrectly (e.g., incorrectly computing the correct maximum size to add) [MS-banned] (CWE-120). Consider strcat_s, strlcat, or automatically resizing strings. Risk is high; the length parameter appears to be a constant, instead of computing the number of characters left. CWE-120 _tcsncat(d,s,sizeof(d)); /* Misuse - flag as riskier */ c3f6ba2c710efc878e66df4578894fd408452cb7cdec7ae6f492a3b1796f8c42 2.0.19 FF1011 https://cwe.mitre.org/data/definitions/120.html
5 test.c 64 3 2 5 buffer MultiByteToWideChar Requires maximum length in CHARACTERS, not bytes (CWE-120). Risk is high, it appears that the size is given as bytes, but the function requires size as characters. CWE-120 MultiByteToWideChar(CP_ACP,0,szName,-1,wszUserName,sizeof(wszUserName)); 4f5b73ff337a54d6e1d9a369659ca0ddb4f80e6b7e38a17e5b112f6d3e266e69 2.0.19 FF1023 https://cwe.mitre.org/data/definitions/120.html
6 test.c 66 3 2 5 buffer MultiByteToWideChar Requires maximum length in CHARACTERS, not bytes (CWE-120). Risk is high, it appears that the size is given as bytes, but the function requires size as characters. CWE-120 MultiByteToWideChar(CP_ACP,0,szName,-1,wszUserName,sizeof wszUserName); 9ecdc1e903acc16a646bf7909a630ae22a7593b70952c39ce6bd9c5a23fad0fd 2.0.19 FF1023 https://cwe.mitre.org/data/definitions/120.html
7 test.c 77 3 5 5 misc SetSecurityDescriptorDacl Never create NULL ACLs; an attacker can set it to Everyone (Deny All Access), which would even forbid administrator access (CWE-732). CWE-732 SetSecurityDescriptorDacl(&sd,TRUE,NULL,FALSE); 5fed1e135b593b4c943e66e89a26ff131eba18b83a32a8af37d1c0bd7b01aadb 2.0.19 FF1060 https://cwe.mitre.org/data/definitions/732.html
8 test.c 77 3 5 5 misc SetSecurityDescriptorDacl Never create NULL ACLs; an attacker can set it to Everyone (Deny All Access), which would even forbid administrator access (CWE-732). CWE-732 SetSecurityDescriptorDacl(&sd,TRUE,NULL,FALSE); 5fed1e135b593b4c943e66e89a26ff131eba18b83a32a8af37d1c0bd7b01aadb 2.0.19 FF1060 https://cwe.mitre.org/data/definitions/732.html
9 test.c 17 2 4 4 buffer strcpy Does not check for buffer overflows when copying to destination [MS-banned] (CWE-120). Consider using snprintf, strcpy_s, or strlcpy (warning: strncpy easily misused). CWE-120 strcpy(b, a); c01c8472bb53022e912da4da2faebc67d537855da324020c44bfd5e608a79b77 2.0.19 FF1001 https://cwe.mitre.org/data/definitions/120.html
10 test.c 20 2 4 4 buffer sprintf Does not check for buffer overflows (CWE-120). Use sprintf_s, snprintf, or vsnprintf. CWE-120 sprintf(s, "hello %s", bug); 814237858ab012010f3355a49480dd6fa0a2cb8cf8356a98ac1c17c9febf6521 2.0.19 FF1015 https://cwe.mitre.org/data/definitions/120.html
11 test.c 21 2 4 4 buffer sprintf Does not check for buffer overflows (CWE-120). Use sprintf_s, snprintf, or vsnprintf. CWE-120 sprintf(s, gettext("hello %s"), bug); b793f18f143fb2297c49e0639384ad73db86eb01a44377aa4d5d09b44b03d747 2.0.19 FF1015 https://cwe.mitre.org/data/definitions/120.html
12 test.c 22 2 4 4 format sprintf Potential format string problem (CWE-134). Make format string constant. CWE-134 sprintf(s, unknown, bug); 16ebc2ff96ee4bab2695783709e97b597ca9c8b8cc149e33aed859f0fafd3431 2.0.19 FF1015 https://cwe.mitre.org/data/definitions/134.html
13 test.c 23 2 4 4 format printf If format strings can be influenced by an attacker, they can be exploited (CWE-134). Use a constant for the format specification. CWE-134 printf(bf, x); 46f42896019245d2dffc4caf4fe018b073ce2a58203676eaa28b6374558a5b5d 2.0.19 FF1016 https://cwe.mitre.org/data/definitions/134.html
14 test.c 25 2 4 4 buffer scanf The scanf() family's %s operation, without a limit specification, permits buffer overflows (CWE-120, CWE-20). Specify a limit to %s, or use a different input function. CWE-120, CWE-20 scanf("%s", s); 3f169dd9fe508f70438f818770a3cb8b0f228e4245ea11a929a5fb0a7839fd5f 2.0.19 FF1020 https://cwe.mitre.org/data/definitions/120.html
15 test.c 27 2 4 4 buffer scanf The scanf() family's %s operation, without a limit specification, permits buffer overflows (CWE-120, CWE-20). Specify a limit to %s, or use a different input function. CWE-120, CWE-20 scanf("%s", s); 3f169dd9fe508f70438f818770a3cb8b0f228e4245ea11a929a5fb0a7839fd5f 2.0.19 FF1020 https://cwe.mitre.org/data/definitions/120.html
16 test.c 38 2 4 4 format syslog If syslog's format strings can be influenced by an attacker, they can be exploited (CWE-134). Use a constant format string for syslog. CWE-134 syslog(LOG_ERR, attacker_string); 22e98963d5af7b197a090bd522d2d39b8d8ee7bdf08453fd2008939c92cd9677 2.0.19 FF1018 https://cwe.mitre.org/data/definitions/134.html
17 test.c 49 3 4 4 buffer _mbscpy Does not check for buffer overflows when copying to destination [MS-banned] (CWE-120). Consider using a function version that stops copying at the end of the buffer. CWE-120 _mbscpy(d,s); /* like strcpy, this doesn't check for buffer overflow */ e00a4a1a0a3603db98a23fcff3c9cdfd9012f5a81826814d9508e0f22089b993 2.0.19 FF1003 https://cwe.mitre.org/data/definitions/120.html
18 test.c 56 3 4 4 buffer lstrcat Does not check for buffer overflows when concatenating to destination [MS-banned] (CWE-120). CWE-120 lstrcat(d,s); 364b4c512862fdccbca27d2fa7737995b5d24b637a760976c940ae636218d340 2.0.19 FF1006 https://cwe.mitre.org/data/definitions/120.html
19 test.c 79 3 3 3 shell CreateProcess This causes a new process to execute and is difficult to use safely (CWE-78). Specify the application path in the first argument, NOT as part of the second, or embedded spaces could allow an attacker to force a different program to run. CWE-78 CreateProcess(NULL, "C:\\Program Files\\GoodGuy\\GoodGuy.exe -x", ""); 3c712b38d0857bde3832d85ad35ac9859be55c5f5f1c20af659a577dd4d0acbf 2.0.19 FF1046 https://cwe.mitre.org/data/definitions/78.html
20 test.c 79 3 3 3 shell CreateProcess This causes a new process to execute and is difficult to use safely (CWE-78). Specify the application path in the first argument, NOT as part of the second, or embedded spaces could allow an attacker to force a different program to run. CWE-78 CreateProcess(NULL, "C:\\Program Files\\GoodGuy\\GoodGuy.exe -x", ""); 3c712b38d0857bde3832d85ad35ac9859be55c5f5f1c20af659a577dd4d0acbf 2.0.19 FF1046 https://cwe.mitre.org/data/definitions/78.html
21 test.c 81 10 3 3 misc LoadLibraryEx Ensure that the full path to the library is specified, or current directory may be used (CWE-829, CWE-20). Use a flag like LOAD_LIBRARY_SEARCH_SYSTEM32 or LOAD_LIBRARY_SEARCH_APPLICATION_DIR to search only desired folders. CWE-829, CWE-20 (void) LoadLibraryEx(L"user32.dll", nullptr, LOAD_LIBRARY_AS_DATAFILE); b1f99ecaa31e682487d795afbf03282fd56ad9f2aa630d0196219b277d2a68c9 2.0.19 FF1059 https://cwe.mitre.org/data/definitions/829.html
22 test.c 99 20 3 3 buffer getopt_long Some older implementations do not protect against internal buffer overflows (CWE-120, CWE-20). Check implementation on installation, or limit the size of all string inputs. CWE-120, CWE-20 while ((optc = getopt_long (argc, argv, "a",longopts, NULL )) != EOF) { 5bedf6e5bccf596008ef191ec4c5d4cc51a32cff0c05ef62d5f10fab93d0cc24 2.0.19 FF1027 https://cwe.mitre.org/data/definitions/120.html
23 test.c 16 2 4 2 buffer strcpy Does not check for buffer overflows when copying to destination [MS-banned] (CWE-120). Consider using snprintf, strcpy_s, or strlcpy (warning: strncpy easily misused). Risk is low because the source is a constant string. CWE-120 strcpy(a, gettext("Hello there")); // Did this work? d64070fb93ff0bb797fb926f4dddc7212d42f77e288d5ceb0cd30ed2979fa28d 2.0.19 FF1001 https://cwe.mitre.org/data/definitions/120.html
24 test.c 19 2 4 2 buffer sprintf Does not check for buffer overflows (CWE-120). Use sprintf_s, snprintf, or vsnprintf. Risk is low because the source has a constant maximum length. CWE-120 sprintf(s, "hello"); 907b46be1c3ea7b38f90a4d1b0f43b7751cd8cbe38fae840930ff006b702157d 2.0.19 FF1015 https://cwe.mitre.org/data/definitions/120.html
25 test.c 45 3 2 2 buffer char Statically-sized arrays can be improperly restricted, leading to potential overflows or other issues (CWE-119!/CWE-120). Perform bounds checking, use functions that limit length, or ensure that the size is larger than the maximum possible length. CWE-119!/CWE-120 char d[20]; 36c87517700337a59cc3ad3218cfdde56cad37d69cdeccee5a55ab232d5c7946 2.0.19 FF1013 https://cwe.mitre.org/data/definitions/119.html
26 test.c 46 3 2 2 buffer char Statically-sized arrays can be improperly restricted, leading to potential overflows or other issues (CWE-119!/CWE-120). Perform bounds checking, use functions that limit length, or ensure that the size is larger than the maximum possible length. CWE-119!/CWE-120 char s[20]; 213de8e8815fc84c423b55fd845fea541f25744718e486234364bb457863b597 2.0.19 FF1013 https://cwe.mitre.org/data/definitions/119.html
27 test.c 50 3 2 2 buffer memcpy Does not check for buffer overflows when copying to destination (CWE-120). Make sure destination can always hold the source data. CWE-120 memcpy(d,s); // fail - no size e667b352fb0748c67b607b11577b11bad87545779c39923e61839dd04056055f 2.0.19 FF1004 https://cwe.mitre.org/data/definitions/120.html
28 test.c 53 3 2 2 buffer memcpy Does not check for buffer overflows when copying to destination (CWE-120). Make sure destination can always hold the source data. CWE-120 memcpy(&n,s,sizeof(s)); // fail - sizeof not of destination 01bcc2c8ba2d928ac3315b4dcc6593042ea05e62888a10a6d2cf16797a65ed32 2.0.19 FF1004 https://cwe.mitre.org/data/definitions/120.html
29 test.c 54 3 2 2 buffer memcpy Does not check for buffer overflows when copying to destination (CWE-120). Make sure destination can always hold the source data. CWE-120 memcpy(d,s,n); // fail - size unguessable 2517a2fb5981193a6017cca660d16e85aab133706cbec302df97aaa623fc77ef 2.0.19 FF1004 https://cwe.mitre.org/data/definitions/120.html
30 test.c 55 3 2 2 buffer CopyMemory Does not check for buffer overflows when copying to destination (CWE-120). Make sure destination can always hold the source data. CWE-120 CopyMemory(d,s); 977f8c805ddd76ff32e0f7aea08701ba97d9ce6955136e98b308ed4f70eb2e11 2.0.19 FF1004 https://cwe.mitre.org/data/definitions/120.html
31 test.c 105 7 2 2 misc fopen Check when opening files - can an attacker redirect it (via symlinks), force the opening of special file type (e.g., device files), move things around to create a race condition, control its ancestors, or change its contents? (CWE-362). CWE-362 f = fopen("/etc/passwd", "r"); 2ec6928c77a8b54caa61d0459f367c4394ee1f5e6f488753f587bfa9c780bad8 2.0.19 FF1040 https://cwe.mitre.org/data/definitions/362.html
32 test.c 15 2 4 1 buffer strcpy Does not check for buffer overflows when copying to destination [MS-banned] (CWE-120). Consider using snprintf, strcpy_s, or strlcpy (warning: strncpy easily misused). Risk is low because the source is a constant character. CWE-120 strcpy(a, "\n"); // Did this work? 0badc5f4c500d17b42794feaca54ee0f49e607a32510af3ed749579001017edb 2.0.19 FF1001 https://cwe.mitre.org/data/definitions/120.html
33 test.c 18 2 4 1 buffer sprintf Does not check for buffer overflows (CWE-120). Use sprintf_s, snprintf, or vsnprintf. Risk is low because the source is a constant character. CWE-120 sprintf(s, "\n"); c65fbd60851f3c8ace22332805966606488c0d242c1823493c582e267609b1a7 2.0.19 FF1015 https://cwe.mitre.org/data/definitions/120.html
34 test.c 26 2 4 1 buffer scanf It's unclear if the %s limit in the format string is small enough (CWE-120). Check that the limit is sufficiently small, or use a different input function. CWE-120 scanf("%10s", s); e24c4c801f10acfa93098b2bef58524efe4f88237f2dd8b58be9afa838616afe 2.0.19 FF1020 https://cwe.mitre.org/data/definitions/120.html
35 test.c 57 3 1 1 buffer strncpy Easily used incorrectly; doesn't always \0-terminate or check for invalid pointers [MS-banned] (CWE-120). CWE-120 strncpy(d,s); 8fa14bf72393a00f667ffcc06b7b7e5f0b6d2f16d8d67444db06b0deb35b5f5e 2.0.19 FF1008 https://cwe.mitre.org/data/definitions/120.html
36 test.c 58 3 1 1 buffer _tcsncpy Easily used incorrectly; doesn't always \0-terminate or check for invalid pointers [MS-banned] (CWE-120). CWE-120 _tcsncpy(d,s); 691fabd4ca960a00e4c538eee0187ee0fdf59bd43dd71e792c14175150369b8b 2.0.19 FF1009 https://cwe.mitre.org/data/definitions/120.html
37 test.c 59 3 1 1 buffer strncat Easily used incorrectly (e.g., incorrectly computing the correct maximum size to add) [MS-banned] (CWE-120). Consider strcat_s, strlcat, snprintf, or automatically resizing strings. CWE-120 strncat(d,s,10); dd92f996a554bfbc038bea27640ba25dcf298383140a8330dca7cdacf493a701 2.0.19 FF1010 https://cwe.mitre.org/data/definitions/120.html
38 test.c 62 7 1 1 buffer strlen Does not handle strings that are not \0-terminated; if given one it may perform an over-read (it could cause a crash if unprotected) (CWE-126). CWE-126 n = strlen(d); db7201c7df7f543ea76febb060bda167e414e71e3d18095fe1def69f8c47a4f6 2.0.19 FF1022 https://cwe.mitre.org/data/definitions/126.html
39 test.c 68 3 2 1 buffer MultiByteToWideChar Requires maximum length in CHARACTERS, not bytes (CWE-120). Risk is very low, the length appears to be in characters not bytes. CWE-120 MultiByteToWideChar(CP_ACP,0,szName,-1,wszUserName,sizeof(wszUserName)/sizeof(wszUserName[0])); 1813fc329227b38abae867d8023a9e29c7517d679fe55c86f8300dde681b6470 2.0.19 FF1023 https://cwe.mitre.org/data/definitions/120.html
40 test.c 70 3 2 1 buffer MultiByteToWideChar Requires maximum length in CHARACTERS, not bytes (CWE-120). Risk is very low, the length appears to be in characters not bytes. CWE-120 MultiByteToWideChar(CP_ACP,0,szName,-1,wszUserName,sizeof wszUserName /sizeof(wszUserName[0])); 7c6cdcb10ad3a16b8bfd56e3dac84829f9bc3e39d4dde74a2be9bbe000102fc5 2.0.19 FF1023 https://cwe.mitre.org/data/definitions/120.html

110
test/makefile
View File

@ -1,110 +0,0 @@
# Flawfinder.
# Released under the General Public License (GPL) version 2 or later.
# (C) 2001-2017 David A. Wheeler.
PYTHON=python
PYTHON2=python2
PYTHON3=python3
FLAWFINDER=../flawfinder.py
SETUPPY=../setup.py
test_001: $(FLAWFINDER) test.c test2.c
@echo 'test_001 (text output)'
@# Omit time report so that results are always the same textually.
@$(PYTHON) $(FLAWFINDER) --omittime test.c test2.c > test-results.txt
@echo >> test-results.txt
@echo "Testing for no ending newline:" >> test-results.txt
@$(PYTHON) $(FLAWFINDER) --omittime no-ending-newline.c | \
grep 'Lines analyzed' >> test-results.txt
@diff -u --ignore-all-space correct-results.txt test-results.txt
test_002: $(FLAWFINDER) test.c test2.c
@echo 'test_002 (HTML output)'
@$(PYTHON) $(FLAWFINDER) --omittime --html --context test.c test2.c > test-results.html
@diff -u --ignore-all-space correct-results.html test-results.html
test_003: $(FLAWFINDER) test.c test2.c
@echo 'test_003 (CSV output)'
@$(PYTHON) $(FLAWFINDER) --csv test.c test2.c > test-results.csv
@diff -u --ignore-all-space correct-results.csv test-results.csv
test_004: $(FLAWFINDER) test.c
@echo 'test_004 (single-line)'
@$(PYTHON) $(FLAWFINDER) -m 5 -S -DC --quiet test.c > \
test-results-004.txt
@diff -u --ignore-all-space correct-results-004.txt test-results-004.txt
test_005: $(FLAWFINDER) test-diff-005.patch test-patched.c
@echo 'test_005 (diff)'
@$(PYTHON) $(FLAWFINDER) -SQDC -P test-diff-005.patch \
test-patched.c > test-results-005.txt
@diff -u --ignore-all-space correct-results-005.txt test-results-005.txt
test_006: $(FLAWFINDER) test.c
@echo 'test_006 (save/load hitlist)'
@$(PYTHON) $(FLAWFINDER) -S -DC --quiet \
--savehitlist test-saved-hitlist-006.txt \
test.c > test-junk-006.txt
@$(PYTHON) $(FLAWFINDER) -SQDC -m 5 \
--loadhitlist test-saved-hitlist-006.txt > \
test-results-006.txt
@diff -u --ignore-all-space correct-results-006.txt test-results-006.txt
test_007: $(SETUPPY)
@echo 'test_007 (setup.py sane)'
@test "`$(PYTHON) $(SETUPPY) --name`" = 'flawfinder'
@test "`$(PYTHON) $(SETUPPY) --license`" = 'GPL-2.0+'
@test "`$(PYTHON) $(SETUPPY) --author`" = 'David A. Wheeler'
test_008: $(FLAWFINDER) test.c
@echo 'test_008 (diff hitlist)'
@$(PYTHON) $(FLAWFINDER) -S -DC --quiet \
--savehitlist test-saved-hitlist-008.txt \
test.c > test-junk-008.txt
@$(PYTHON) $(FLAWFINDER) -S -C --quiet --omittime \
--diffhitlist test-saved-hitlist-008.txt test.c > \
test-results-008.txt
@diff -u --ignore-all-space correct-results-008.txt test-results-008.txt
test_009: $(FLAWFINDER) test-cpp-digit-separator.cpp
@echo 'test_009 (C++ digit separator)'
# C++ file should have no problem recognizing ' as digit separator
@$(PYTHON) $(FLAWFINDER) test-cpp-digit-separator.cpp > /dev/null
# C file should fail with unterminated char literal error
@$(PYTHON) $(FLAWFINDER) test-cpp-digit-separator.c 2>&1 \
| grep 'File ended while in string.' \
> /dev/null
test_010: $(FLAWFINDER) test-boost-system.hpp
@echo 'test_010 (system:: ignored)'
@$(PYTHON) $(FLAWFINDER) --error-level 2 test-boost-system.hpp \
> /dev/null
# Run all tests on *one* version of Python;
# output shows differences from expected results.
# If everything works as expected, it just prints test numbers.
# Set PYTHON as needed, including to ""
test: test_001 test_002 test_003 test_004 test_005 test_006 test_007 test_008 \
test_009 test_010
@echo 'All tests pass!'
# Usual check routine. Run all tests using *both* python2 and python3.
check:
@echo "Testing with $(PYTHON2)"
@PYTHON="$(PYTHON2)" $(MAKE) test
@echo
@echo "Testing with $(PYTHON3)"
@PYTHON="$(PYTHON3)" $(MAKE) test
# Run "make test-is-correct" if the results are as expected.
test-is-correct: test-results.txt
cp -p test-results.txt correct-results.txt
cp -p test-results.html correct-results.html
cp -p test-results.csv correct-results.csv
cp -p test-results-004.txt correct-results-004.txt
cp -p test-results-005.txt correct-results-005.txt
cp -p test-results-006.txt correct-results-006.txt
cp -p test-results-008.txt correct-results-008.txt
.PHONY: test check test-is-correct

4
test/test-boost-system.hpp
View File

@ -1,4 +0,0 @@
// Ensure reference to boost::system is ignored
void HandleWrite(const boost::system::error_code &error);

1
test/test-cpp-digit-separator.c
View File

@ -1 +0,0 @@
int main() { return 0'000; }

1
test/test-cpp-digit-separator.cpp
View File

@ -1 +0,0 @@
int main() { return 0'000; }

13
test/test-diff-005.patch
View File

@ -1,13 +0,0 @@
--- test.c 2017-08-26 15:33:59.480235200 -0400
+++ test-patched.c 2017-08-23 22:20:22.458331500 -0400
@@ -9,6 +9,10 @@
printf("hello\n");
}
+int bad(char *a, char *b) {
+ strcpy(b, a);
+}
+
/* This is a strcpy test. */
int demo(char *a, char *b) {

121
test/test-patched.c
View File

@ -1,121 +0,0 @@
/* Test flawfinder. This program won't compile or run; that's not necessary
for this to be a useful test. */
#include <stdio.h>
#define hello(x) goodbye(x)
#define WOKKA "stuff"
main() {
printf("hello\n");
}
int bad(char *a, char *b) {
strcpy(b, a);
}
/* This is a strcpy test. */
int demo(char *a, char *b) {
strcpy(a, "\n"); // Did this work?
strcpy(a, gettext("Hello there")); // Did this work?
strcpy(b, a);
sprintf(s, "\n");
sprintf(s, "hello");
sprintf(s, "hello %s", bug);
sprintf(s, gettext("hello %s"), bug);
sprintf(s, unknown, bug);
printf(bf, x);
scanf("%d", &x);
scanf("%s", s);
scanf("%10s", s);
scanf("%s", s);
gets(f); // Flawfinder: ignore
printf("\\");
/* Flawfinder: ignore */
gets(f);
gets(f);
/* These are okay, but flawfinder version < 0.20 incorrectly used
the first parameter as the parameter for the format string */
syslog(LOG_ERR,"cannot open config file (%s): %s",filename,strerror(errno))
syslog(LOG_CRIT,"malloc() failed");
/* But this one SHOULD trigger a warning. */
syslog(LOG_ERR, attacker_string);
}
demo2() {
char d[20];
char s[20];
int n;
_mbscpy(d,s); /* like strcpy, this doesn't check for buffer overflow */
memcpy(d,s);
CopyMemory(d,s);
lstrcat(d,s);
strncpy(d,s);
_tcsncpy(d,s);
strncat(d,s,10);
strncat(d,s,sizeof(d)); /* Misuse - this should be flagged as riskier. */
_tcsncat(d,s,sizeof(d)); /* Misuse - flag as riskier */
n = strlen(d);
/* This is wrong, and should be flagged as risky: */
MultiByteToWideChar(CP_ACP,0,szName,-1,wszUserName,sizeof(wszUserName));
/* This is also wrong, and should be flagged as risky: */
MultiByteToWideChar(CP_ACP,0,szName,-1,wszUserName,sizeof wszUserName);
/* This is much better: */
MultiByteToWideChar(CP_ACP,0,szName,-1,wszUserName,sizeof(wszUserName)/sizeof(wszUserName[0]));
/* This is much better: */
MultiByteToWideChar(CP_ACP,0,szName,-1,wszUserName,sizeof wszUserName /sizeof(wszUserName[0]));
/* This is an example of bad code - the third paramer is NULL, so it creates
a NULL ACL. Note that Flawfinder can't detect when a
SECURITY_DESCRIPTOR structure is manually created with a NULL value
as the ACL; doing so would require a tool that handles C/C++
and knows about types more that flawfinder currently does.
Anyway, this needs to be detected: */
SetSecurityDescriptorDacl(&sd,TRUE,NULL,FALSE);
/* This one is a bad idea - first param shouldn't be NULL */
CreateProcess(NULL, "C:\\Program Files\\GoodGuy\\GoodGuy.exe -x", "");
/* Test interaction of quote characters */
printf("%c\n", 'x');
printf("%c\n", '"');
printf("%c\n", '\"');
printf("%c\n", '\'');
printf("%c\n", '\177');
printf("%c\n", '\xfe');
printf("%c\n", '\xd');
printf("%c\n", '\n');
printf("%c\n", '\\');
printf("%c\n", "'");
}
int getopt_example(int argc,char *argv[]) {
while ((optc = getopt_long (argc, argv, "a",longopts, NULL )) != EOF) {
}
}
int testfile() {
FILE *f;
f = fopen("/etc/passwd", "r");
fclose(f);
}
/* Regression test: handle \\\n after end of string */
#define assert(x) {\
if (!(x)) {\
fprintf(stderr,"Assertion failed.\n"\
"File: %s\nLine: %d\n"\
"Assertion: %s\n\n"\
,__FILE__,__LINE__,#x);\
exit(1);\
};\
}
int accesstest() {
int access = 0; /* Not a function call. Should be caught by the
false positive test, and NOT labelled as a problem. */
}

0
test/test2.c → test2.c
View File