* addons: Reduce memory consumption
Parse dump files incrementaly using ElementTree.iterparse. Clean unused
resources during parsing. This method is explained in following
article: https://www.ibm.com/developerworks/xml/library/x-hiperfparse/
Memory consumption was reduced about 30% (measured with mprof),
execution time increased about 5% (measured with time utility).
More description available in PR.
* Switch to lxml and update iterparse routines
Use lxml module instead default xml.etree. Lxml provides convenient
wrappers around iterparse method that accepts `tag` argument. That
easer incremental parsing routines to select specific tags from roottree
like `dump` and `dumps`.
Element.clear() method was replaced by `lxml_clean` because lxml
keeps additional information to nodes that should be removed.
Added note about large consumption RAM on large dump files.
This commit doesn't solve this problem completely, but provides a way
to improve current parser to add incremental Configuration serialization
later.
* Working on iterative parser
* Added iterative Configurations parser
* fix
* Fix varlist iteration
* make sure that standards node was loaded
This commit will add feature to detect essential type categories
for operators of ternary operation.
This fixes issues with rule 10.1 and close the following ticket:
https://trac.cppcheck.net/ticket/9543
* misra.py: Allow executeCheck take multiple args
* misra.py: Fix R12.3 FN in variables declaration
Make rule 12.3 check detect commas in variables declaration code.
* Fix excluded lines
* Add support for global variables
* Fix FN when linenr from other file was excluded
* Add a few more tests
* Handle more cases
Handle additional cases to check commans in variables declaration
including:
* multiline variables declaration
* functions and structures initialized in the same line
* expanding macroses in initialization
* Fix FP in global struct initialization
* Add another test
This commit fixes __repr__ methods introduced in d538315268. Fields that recursively links to other cppcheckdata objects was removed to avoid max recursion depth crash on printing.
* misra.py: R14.2: Verify for counter modification
Add additional check to detect modification of loop counter in loop
body. Related issue: https://trac.cppcheck.net/ticket/9490
Add small fix that treat all assignment operators defined in N1750
6.5.16 as has side affects. This will affects rules 13.1, 13.3, 13.5
and allow to catch some false negatives.
* Add tests for fixed FPs for R13.{1,5,6}
* fix
* use isAssignmentOp from cppcheck data
* remove unused set
* handle case with empty body or syntax error
* add test with outer variable
* Fix FP in nested loops, add tests
* Fix FP on outer variables
* Fixup false positives for not loop counters
* y2038 addon: Fix that check can never return True, add tests
At the beginning of `check_y2038_safe()` the variable `y2038safe` should
be initialized with `True` and only be set to `False` if there are any
issues. Otherwise it could never become `True`.
In the unit tests the return value of `check_y2038_safe()` is now
verified. But it does not yet work for the "good" example. The "good"
example also returns `False` since it finds warnings in the include
file. So this verification is marked with a "FIXME" comment.
* y2038 tests: Add "good" test file that does not use time functionality
The test file y2038-test-5-good-no-time-used.c does not use any time
functionality so the y2038 addon is not allowed to issue any warnings
and the check must return with `True` (code is safe).
Fixed usage of hasSideEffectsRecursive which was causes crashes
of rules 12.4 and 13.1.
Add more tests for these rules.
See trac ticket: https://trac.cppcheck.net/ticket/9487
Add some more tests for rule 10.1. They are given as examples in the
MISRA document. The 10.1 test that causes a false positive for 10.6 can be
found under the 10.6 tests.
Track ticket for false positive:
https://trac.cppcheck.net/ticket/9488
Trac ticket for the false negative:
https://trac.cppcheck.net/ticket/9489
* cert.py: Add ENV33-C-EX1 exception
ENV33-C-EX1: It is permissible to call system() with a null pointer
argument to determine the presence of a command processor for the
system. [1]
[1]: https://wiki.sei.cmu.edu/confluence/pages/viewpage.action?pageId=87152177
* Handle different null-pointer constants
* cppcheck.cpp: Check for JSON error when parsing addon .json files
This fixes that errors in JSON files given via `--addon=*.json` are
silently ignored and maybe only a part of the JSON file is used.
Now the error message which picojson can return is checked and a
corresponding error message is returned again by getAddonInfo().
* naming.json: Fix missing comma
* CLI: Fix naming violations detected by addon naming.py via naming.json
* Addon naming: Add argument for validating names of constants
* LIB: Rename functions/variables so they are valid, loosen naming rules
* GUI: Fix naming violations
The unused argument causes the new check for rule 2.7 to output an error
here.
I verified that the regression test still works without the unused
argument (misra.py would still crash).
Travis should be happy again.
* Add check for MISRA-C 2012 Rule 2.7 and test cases.
* Cast unused function parameters of some test patterns to "void".
* Fix function parameter void cast in "misra_17_6". Add comment to "misra_11_7_extra" to make all function parameter void casts look alike.
Use Python 3 instead of Python 2 if addons are executed directly.
Running cert.py and misra.py against test/cfg/std.c.dump shows that
Python 3 needs only half the time compared to Python 2. I have tested
it repeatedly and the results are always the same. This is no surprise
at all. The memory footprint is very likely also significantly better
but i have not tested it.
* addons: Add '--recursive' arg. Clean up and clarify errors messages.
This commit introduce '--recursive' option for cppcheck addons.
Iff this option is set addon will recursively traverse directories in
given input files to find files with '.dump' suffix that would be
checked. Otherwise it will treat input directory as error (current
behaviour).
Add additional error handling with more clear error messages, clean up
the code.
* Add regex specifier
* Roll back --recursive option
* Update addons section in manual
Rule 21.1 now doesn't report errors on usage of library functions
started from underscore (e.g. `_exit') and valid errno assignment.
See added tests for examples.
Fix R16.3 state machine which doesn't report errors in last case/default
statemenet. See added tests for examples.
Issue on trac: https://trac.cppcheck.net/ticket/8548
* misra.py: Fix large memory usage with suppressions
Don't save whole lxml locations when collecting rules suppressions
statistics. This fixes the problem with large memory usage when
some rules are suppressed.
See issue description:
https://sourceforge.net/p/cppcheck/discussion/development/thread/51fc053626/
* don't override 'file' builtin
These changes will improve misra.py execution time in the
case where we use globally suppressed checks. For example, when using
single MISRA rule and suppress others, we could get about twice the
execution speed, since most of `MisraChecker' class methods will not
be started at all.
Some measurements and discussion are available at:
https://github.com/danmar/cppcheck/pull/2301
Version 1.88 changed the parsing of the MISRA rules file adding a
severity setting. This caused a regression in rule parsing.
In particular the following format used to parse cleanly and produce
rule output that would show the severity as part of the rule text.
Rule 1.2
Advisory
Rule text goes here.
Rule 1.3
Required
More rule text goes here.
As of 1.88 a file structured like above would parse as having no rules.
The problem is the use of blank lines as a rule delimiter. The
modified rule parser wants to see a rules formatted like below:
Rule 3.1 Required
R3.1 text.
Rule 4.1 Required
R4.1 text.
or:
Rule 1.1
Add this rule and parse to next, skipping empty lines.
Rule 1.2
Rule text.
Any rule text that did not fall into one of the above formats would
result in incomplete rule text parsing.
Change the parsing of the rule text file so that blank lines are ignored
instead of treating them as a delimiter between rules. Instead use the
start of the next rule as a delimiter for the end of the previous rule.
This allows both of the newer formats but also supports the behavior of
pre-1.88 versions.
Change units tests that were specifically forbidding the use of blank
lines to ones that allow blank lines.
* make ellipsis ... a single token
Using cppcheck -E to preprocess code with ellipsis produces output that
can't be compiled because ... is split into 3 tokens.
* try to fix addon
* cert.py: Add '--quiet' option and test
Main sentinel also was required.
* travis.yml: Install pytest with --user
To avoid possible conflicts with system packages later.
* Quickfix
* Update addons structure. Make Y2038 addon works.
All addons are now located in same directory, where cppcheck-gui is
looking for them.
Y2038 addon has been updated to latest cppcheck version. Output is same
as 303622f01c commit as it described in
README: 303622f01c/addons/y2038/README.
* Clean up .travis.yml
* Update travis.yml paths.
* misra.py: Bring back -P argument for backward compatibility
* Fix paths
* Normalize dumpfile paths tpo cppcheck format
* Fixup 3 test.
* Add arguments regression test.
* Fixing travis build
Use renamed pylintrc file that is only meant for Travis checks.
Check all Python scripts in 'addons', 'htmlreport' and 'tools'
Errors for `_socketobject` class are disabled, see
https://stackoverflow.com/questions/10300082/how-to-prevent-python-pylint-complaining-about-socket-class-sendall-method
Install imported modules `unittest2` and `pexpect` via pip.
Add "./addons" to search-path for modules because
"tools/compare-ast-clang-and-cppcheck.py" imports cppcheckdata.py from
addons. Pylint does not seem to evaluate
`sys.path.insert(0, '../addons')` in the script. So an `init-hook` is
necessary in pylintrc_travis.
* Run pylint (using a pylintrc file) on the addon Python scripts. It serves as a build breaker for severe issues.
* Skip y2038 addon for now until it's fixed.
* MISRA: Fix suppressed rules line numbers
Line numbers represented as strings in lxml ETree, but we use it in
integer comparison later.
* MISRA: Use standard library function instead file_prefix.
* MISRA: Use pytest's capsys for capturing, add suppressions tests.
* travis.yml: Update pytest version
I am not sure but it looks like "self." is missing here.
And the attribute "inconclusive" is initialized as a "bool", so i guess
"True" should be assigned here?
* misra.py: Fixup load rules parser.
* misra.py: Report when rule text is missing in rule-texts file
* misra.py: Allow to skip misra checks not specified in rule-texts.
* misra.py: Remove top-level control flow.
Create separate class that stores settings, instead of global variables.
This is required to perform imports from misra.py for testing purposes.
* misra.py: Add simple pytest test for load rules.
* misra.py: Add document structure tests.
* misra.py: Exit after show rules table.
* misra.py: Add document structure tests.
* misra.py: Fixup import pitfall with python2
* misra.py: Minor fixes
* Add cmd parameter for choosing between C90 and C99
Misra specifies different requirements to the uniqueness of
macros/enums/variables depending on what C standard
that's being used.
* Add standards configuration to each dump file
Read standards config from misra addon to decide what rules to use.
* Posix as standard setting should be deprecated. Don't include this in the xml
* Rewritten to use a switch
* Modifiy check for MISRA rule 4.1 to avoid skipping constant character literal enclosed by single quotes.
Add test for MISRA rule 4.1 which contains non-compliant cases defined by MISRA C 2012 document.
* Moved additional test cases for MISRA rule 4.1 to appropriate test case section.
Without the `for scope in data.scopes:` loop, scope is not assigned anything.
From the context a loop over `data.scopes` could be intended. But other things like indentation would be possibly correct too. Not sure how this code should be.
We try this fix.
https://trac.cppcheck.net/ticket/8946
Add tests to travis script for verifying rule text loading.
Add dummy rule text files.
misra.py: Try to find a suitable codec for rule texts file.
* Added script to check ROS naming style
* Added header to script
* Enhanced namingng.py and added ROS_naming.json
* Correction of style bugs
* Removed trailing whitespace
* Removed trailing whitespace
* Removed path
* Remove path from file name
* Check if the token belongs to the current file or is included
* Reverted
* Fixed msg errors
* Added a new naming check addon. Also verifies variable and function prefixes
* Verification added to code
* added naming checks ng selftest to travis file
* Ensure zero exit value for tests if tests succeed
* Expected values adjusted
* Fixed copy and paste error
* MISRA: Allow printing of the suppressed rules to the console
--show-suppressed-rules will print rules in the suppression rule list to
the console sorted by rule number.
* MISRA: Correct rule suppression for entire file scope
The entire file scope suppression check was checking for the rule item
list to be None instead of looking for None as an entry into the list.
Correct this check and modify the documentation to explicitly state that
an entry of None in the rule item list will set the scope for that
suppression to be the entire file.
* MISRA: Tests for checking per-file rule suppressions
To run:
../../cppcheck --suppressions-list=suppressions.txt --dump misra-suppressions*-test.c
python ../misra.py misra-suppressions*-test.c.dump
There should be no violations reported
* MISRA: Allow ignoring a prefix from file paths when suppression matching
For environments that run cppcheck from the build system cppcheck may be
passed a filename that is a complete path.
Often this path will include a portion that is specific to the developer
or to the environment where the project is located.
The per-file suppression rules do filename matching based on the
filename passed to cppcheck. To match any path information also has to
be included into the suppressions file provided to cppcheck via the
--suppressions-list= option.
This limits the usefulness of the per-file based suppressions because
it requires the suppression to be customized on a per instance basis.
Add a option "--file-prefix" that allows a prefix to be excluded from
the file path when doing the suppression filename matching.
Example.
Given the following structure:
/test/path1/misra-suppressions1-test.c
/test/path1/misra-suppressions2-test.c
specifying --file-prefix /test/path1 will allow the use of
misra-suppressions1-test.c and misra-suppressions2-test.c as filenames
in the suppressions file without leading patch information but still
match the suppression rule.
* MISRA: Tests for --file-prefix option
To run:
../../cppcheck --suppressions-list=suppressions.txt \
--dump misra-suppressions*-test.c \
path1/misra-suppressions*-test.c
python ../misra.py misra-suppressions*-test.c.dump \
path1/misra-suppressions*-test.c
There should be no violations reported
Functions with variadic arguments trip an exception in the MISRA checker
because some of the token is None and does not have some of the members
the code is expecting.
Prevent this by checking to see if the token is None and skipping the
code that tries to use that value.
* MISRA: Refactor many top level functions into a class
All the checker operations were implemented as individual functions. In
order to share data globals were used.
By refactoring all these into class methods data can be shared between
them without resorting to globals.
This change is scope only. No functional change for any of the methods.
Data is still shared via globals.
* MISRA: Refactor non-option globals into MisraChecker class
- Move all non-option global variables into the MisraChecker class
- Allows data to be shared among the class methods without needing
globals.
- Move global VERIFY_EXPECTED to class variable verify_expected
- Move global VERIFY_ACTUAL to class variable verify_actual
- Move global VIOLATIONS to class variable violations
- Move global suppressRules to class variable suppressedRules
- Move global suppressions to class variable dumpfileSuppressions
This refactoring is in anticipation of parsing and using the
suppressions added into the dump file by cppcheck.
Only variable naming and scope changed. No functional change for any of the
methods.
* MISRA: Restore original summary behavior
Version 1.84 introduced a regression in the behavior of the rule summary
output due to changes in the way multiple input files were handled.
The intended behavior of the summary was to output the total number of
violations after all files have been processed.
Commit aa831ce972 restored the input file
handling behavior but left summary behavior such that a summary output
was produced for each file that caused a violation instead of the total
number of violations after all files were processed.
Move the -verify logic up into the main loop so that the exit calls are
in the top level and restore the original behavior of the summary
output.
* MISRA: Support per file rule suppressions
Parse the suppressions list from cppcheck and extract MISRA rule strings from
the suppressions class by matching for errorId strings that begin with
'MISRA' or 'misra'. Extract the MISRA rule from those strings by
looking for a '_' or a '.' to separate rule numbers.
Store the rule number, filename, line number, and symbol name from the
suppression entry into a structure that allows for dictionary lookups
by the rule number and then the filename. All the line number
and symbol entries for that filename are are stored in list of tuples of
(line number, symbol name). A rule entry that has a value of None for
the filename is treated as a global suppression for all files. A
filename entry that has None for the rule items list is treated as a
suppression for the entire file. If the rule item list exist then it is
searched for matching line numbers.
Although symbol names are parsed and added int the list of rule items
they are not used for rule matching. Symbol names can include regular
expressions. Adding support for symbol names and regular expressions is
left as a future feature.
The existing global suppression method provided by the --suppress-rules
option is supported. Those rules are added into the suppressions
structure as if they were provided by the suppressions list as global
suppressions. ie A rule with a None for the filename value.
Add function isStandardFunction() that checks if the given function is a standard function.
Only when this function returns true for the currently checked rand() tokens it is reported as a violation.
Tests added for C and C++.
* MISRA addon: Support multiple files with argparse
* MISRA addon fix exit code
Fix to run all dump files then exit with error code if any error in any file found.
* added CLion project folder to .gitignore
* adjusted project name in CMakeLists.txt
* avoid warning when compiling "Debug" with Visual Studio via CMake
There was a GCC-style compiler flag in the common flags in compileroptions.cmake which caused the following warning:
cl : Command line warning D9002 : ignoring unknown option '-O0'
* compileroptions.cmake: restored original formatting
* daca2.py: added missing import
* misra.py: removed unnecessary escaping from regular expression
* added CLion project folder to .gitignore
* adjusted project name in CMakeLists.txt
* avoid warning when compiling "Debug" with Visual Studio via CMake
There was a GCC-style compiler flag in the common flags in compileroptions.cmake which caused the following warning:
cl : Command line warning D9002 : ignoring unknown option '-O0'
* compileroptions.cmake: restored original formatting
* some small *.py script cleanups
WARNING: breaking change - now default misra.py report style will be the
same as cppcheck itself, but previosly misra.py default template was a
little different
* Added ignore list
- Added ignore list
- Use argparse to parse arguments
- Source formatting
* misra.py changed argument name
- Changed ` --ignore-rules` to ` --suppress-rules`
* Resolved VERIFY option in misra.py
Resolved broken broken logic in commit to add argparse.
* edit suppression list
Modified names
Hide some arguments
* command line help formatting
* Revert removal of --no-summary argument
* missing text rules no longer reported
* VERIFY scope modified to global
Output will still be given for style violations.
This allows the build process output to be smaller and cleaner,
making it easier to spot and address problems.
- Show a summary of the of the number of violations. This can be
disabled with the --no-summary option.
- Return an error code if -verify flag not used and violation exists.
Use python's __file__ variable to figure out where the checker script is
located.
This allows a customized per project scripts to be used instead of the
one provided in the cppcheck release.
* Added rule 5.2
* updated 5.2
request-checks: true
* Added rule 5.3
* Changed rule 5.4, 5.5
* Updated test suite for Rule 5.2
* Changes in Rule 5.4 and 5.5
* Change in function name in test suite and removed type from class token in cppcheck
* Changed the name of function in misra-test.c
* Modified rule 5.3
* Modified misra-test.c for rule 5.3
* Added rule 5.2
* updated 5.2
request-checks: true
* Added rule 5.3
* Changed rule 5.4, 5.5
* Updated test suite for Rule 5.2
* Changes in Rule 5.4 and 5.5
* Change in function name in test suite and removed type from class token in cppcheck
* Changed the name of function in misra-test.c
The method “match” was used as a module-level function in for loops
of implementations for MISRA check functions so far.
Use compiled regular expression objects instead.
Link: https://trac.cppcheck.net/ticket/8547
Signed-off-by: Markus Elfring <elfring@users.sourceforge.net>
* Added parsing suppressions from dump xml.
* Added code to dump suppressions to an xml file
* Added declaration for dump function
* Suppressions will now be written to the xml file when a dump is requested
* Fixed syntax error
* Removed excess whitespace
* Fixed indentation to be consistent
* Fixed indentation to be consistent
* Fixed indentation to be consistent
* Added missing include for ErrorLogger::toXml
* Fixed suggestions from pull request #1166
Switched to using ranged for loop to iterate through suppressions.
Made the line number attribute optional, rather than 0 if not specified. This means when Python deserialises it it will be None, which is more pythonic.
* Implemented checking suppressions in reportError
This modification expects suppressions and a function to be called to write a line of output to be passed in. The function checks if any of the suppressions match the warning (with the new Suppression.isMatch function) and if so returns None. This change maintains the old behaviour of returning the warning text, but adds the possibility of returning None if the warning was suppressed.
* Fixed code quality warnings
* Removed more extraneous whitespace
Georgy Komarov
Daniel Marjamäki
Francesc Elies
Sebastian
Mathias Schmid
Armin Müller
Conrado Gouvea
Richard A. Smith
IOBYTE
Conrado Gouvea
fuzzelhjb
amai2012
eivindt
Servando
practicalswift
Thorsten Sick
Kyle Chisholm
Oliver Stöneberg
Konrad Grochowski
orbitcowboy
Swasti Shrivastava
Matthias Krüger
Markus Elfring
rebnridgway
amai