Commit Graph

402 Commits

Author SHA1 Message Date
Georgy Komarov d977761e76 addons: Reduce memory consumption (#2395)
* addons: Reduce memory consumption

Parse dump files incrementaly using ElementTree.iterparse. Clean unused
resources during parsing.  This method is explained in following
article: https://www.ibm.com/developerworks/xml/library/x-hiperfparse/

Memory consumption was reduced about 30% (measured with mprof),
execution time increased about 5% (measured with time utility).
More description available in PR.

* Switch to lxml and update iterparse routines

Use lxml module instead default xml.etree. Lxml provides convenient
wrappers around iterparse method that accepts `tag` argument. That
easer incremental parsing routines to select specific tags from roottree
like `dump` and `dumps`.

Element.clear() method was replaced by `lxml_clean` because lxml
keeps additional information to nodes that should be removed.

Added note about large consumption RAM on large dump files.
This commit doesn't solve this problem completely, but provides a way
to improve current parser to add incremental Configuration serialization
later.

* Working on iterative parser

* Added iterative Configurations parser

* fix

* Fix varlist iteration

* make sure that standards node was loaded
2019-12-27 08:50:56 +01:00
Georgy Komarov 730ea4163b misra.py: Handle essential type categories for ternary operations (#2455)
This commit will add feature to detect essential type categories
for operators of ternary operation.

This fixes issues with rule 10.1 and close the following ticket:
https://trac.cppcheck.net/ticket/9543
2019-12-21 07:40:15 +01:00
Georgy Komarov aadb35888f misra.py: Fix 12.3 FP (#2454)
This will close https://trac.cppcheck.net/ticket/9544
2019-12-19 15:55:51 +01:00
Georgy Komarov d0dd200444 misra.py: Fix R12.3 false negative in variables declaration (#2453)
Make rule 12.3 check detect commas in the variables declaration code.
2019-12-19 08:36:10 +01:00
Daniel Marjamäki 7977bbf456 Revert "misra.py: Fix R12.3 false negative in variables declaration (#2431)"
This reverts commit 7ce8ae408a.
2019-12-18 13:31:52 +01:00
Georgy Komarov 7ce8ae408a misra.py: Fix R12.3 false negative in variables declaration (#2431)
* misra.py: Allow executeCheck take multiple args

* misra.py: Fix R12.3 FN in variables declaration

Make rule 12.3 check detect commas in variables declaration code.

* Fix excluded lines

* Add support for global variables

* Fix FN when linenr from other file was excluded

* Add a few more tests

* Handle more cases

Handle additional cases to check commans in variables declaration
including:
* multiline variables declaration
* functions and structures initialized in the same line
* expanding macroses in initialization

* Fix FP in global struct initialization

* Add another test
2019-12-18 11:05:57 +01:00
Georgy Komarov 88a3b4c685 addons: Fix __repr__ methods (#2448)
This commit fixes __repr__ methods introduced in d538315268. Fields that recursively links to other cppcheckdata objects was removed to avoid max recursion depth crash on printing.
2019-12-15 19:46:54 +01:00
Georgy Komarov c46e44e39e misra.py: R14.2: Verify for loop counter modification (#2409)
* misra.py: R14.2: Verify for counter modification

Add additional check to detect modification of loop counter in loop
body. Related issue: https://trac.cppcheck.net/ticket/9490

Add small fix that treat all assignment operators defined in N1750
6.5.16 as has side affects. This will affects rules 13.1, 13.3, 13.5
and allow to catch some false negatives.

* Add tests for fixed FPs for R13.{1,5,6}

* fix

* use isAssignmentOp from cppcheck data

* remove unused set

* handle case with empty body or syntax error

* add test with outer variable

* Fix FP in nested loops, add tests

* Fix FP on outer variables

* Fixup false positives for not loop counters
2019-12-15 18:23:12 +01:00
Georgy Komarov f87cd9818c misra.py: Fix false negative for rule 10.1 (#2449)
This will close Trac 9489.
2019-12-15 18:10:22 +01:00
Daniel Marjamäki 5e07528af5 Fixed #9505 (MISRA 20.7 check suggests code change that leads to invalid code) 2019-12-14 16:31:41 +01:00
Francesc Elies 40aefa1ba4 [windows] cannot run test-misra (binary not found) (#2441)
set PYTHONPATH=addons
    python -m pytest addons/test/test-misra.py
2019-12-10 18:32:41 +01:00
Francesc Elies d538315268 [python] classes are missing __repr__ methods (#2437)
* [python] classes don't have a __repr__ method

* [python] removes unused import
2019-12-09 19:08:45 +01:00
Daniel Marjamäki 3d3e6f384c Fix #9505 (MISRA 20.7 check suggests code change that leads to invalid code) 2019-12-08 20:07:52 +01:00
Daniel Marjamäki 61c2c5a5f0 Misra: rule 5.3 is implemented in Cppcheck 2019-12-08 19:06:34 +01:00
Daniel Marjamäki 4a4b6fe39b MISRA: Remove rule 5.3, Cppcheck core has these warnings 2019-12-08 19:01:51 +01:00
Daniel Marjamäki 55afaebdac Fixed #9506 (false positive: MISRA 12.4) 2019-12-08 18:52:52 +01:00
Sebastian fb96e5433a
y2038 addon: Fix that check can never return True, add tests (#2417)
* y2038 addon: Fix that check can never return True, add tests

At the beginning of `check_y2038_safe()` the variable `y2038safe` should
be initialized with `True` and only be set to `False` if there are any
issues. Otherwise it could never become `True`.
In the unit tests the return value of `check_y2038_safe()` is now
verified. But it does not yet work for the "good" example. The "good"
example also returns `False` since it finds warnings in the include
file. So this verification is marked with a "FIXME" comment.

* y2038 tests: Add "good" test file that does not use time functionality

The test file y2038-test-5-good-no-time-used.c does not use any time
functionality so the y2038 addon is not allowed to issue any warnings
and the check must return with `True` (code is safe).
2019-12-03 18:49:11 +01:00
Georgy Komarov 6bacc1987c misra.py: Fix R12.4 and R13.1 crashes on large initializer lists (#2410)
Fixed usage of hasSideEffectsRecursive which was causes crashes
of rules 12.4 and 13.1.

Add more tests for these rules.

See trac ticket: https://trac.cppcheck.net/ticket/9487
2019-12-01 15:03:33 +01:00
Georgy Komarov c1770414ba misra.py: Fix crash on c++ code (#2414) 2019-12-01 14:51:12 +01:00
Sebastian 8d114a40e4
misra-test.c: Add more tests for rule 14.2 (including FP and FN) (#2399)
Adding more tests for rule 14.2 revealed a false negative when the
loop counter is changed inside the loop.
Corresponding line in the example suite:
https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/blob/master/R_14_02.c#L39
Also a false positive has been revealed when the loop counter is
initialized in a function that is called in the first `for` clause.
Corresponding line in the example suite:
https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/blob/master/R_14_02.c#L43
2019-11-30 11:27:31 +01:00
Georgy Komarov bd6f236481 misra.py: Fix 5.3 FP (#2405)
* misra.py: Fix 5.3 FP

This will close https://trac.cppcheck.net/ticket/9497

* fix names conflict

* fix naming
2019-11-30 10:05:57 +01:00
Sebastian e46191e6e8
misra-test.c: Add more tests for rule 10.1 (including FP and FN) (#2398)
Add some more tests for rule 10.1. They are given as examples in the
MISRA document. The 10.1 test that causes a false positive for 10.6 can be
found under the 10.6 tests.
Track ticket for false positive:
https://trac.cppcheck.net/ticket/9488
Trac ticket for the false negative:
https://trac.cppcheck.net/ticket/9489
2019-11-27 09:09:17 +01:00
Georgy Komarov 2c28440f15 cert.py: Add ENV33-C-EX1 exception (#2396)
* cert.py: Add ENV33-C-EX1 exception

ENV33-C-EX1: It is permissible to call system() with a null pointer
argument to determine the presence of a command processor for the
system. [1]

[1]: https://wiki.sei.cmu.edu/confluence/pages/viewpage.action?pageId=87152177

* Handle different null-pointer constants
2019-11-24 10:31:08 +01:00
Georgy Komarov 14469aced8 cert.py: Fix EXP05C crash (#2392)
* cert.py: Fix EXP05 crash

* Fix arguments check
2019-11-23 17:41:47 +01:00
Georgy Komarov b9bc5d6d78 cert.py: Add ENV33-C (#2377) 2019-11-21 06:46:46 +01:00
Sebastian c990d10ffa
Check for JSON error when parsing addon .json files + fixes (#2374)
* cppcheck.cpp: Check for JSON error when parsing addon .json files

This fixes that errors in JSON files given via `--addon=*.json` are
silently ignored and maybe only a part of the JSON file is used.
Now the error message which picojson can return is checked and a
corresponding error message is returned again by getAddonInfo().

* naming.json: Fix missing comma

* CLI: Fix naming violations detected by addon naming.py via naming.json

* Addon naming: Add argument for validating names of constants

* LIB: Rename functions/variables so they are valid, loosen naming rules

* GUI: Fix naming violations
2019-11-20 15:37:09 +01:00
Sebastian 4f1db90367
naming.py: Fix FP for constructors/destructors (#2375) 2019-11-18 18:41:57 +01:00
versat 0dd180d5a2 misra-test.cpp: Remove redundant unused function argument
The unused argument causes the new check for rule 2.7 to output an error
here.
I verified that the regression test still works without the unused
argument (misra.py would still crash).
Travis should be happy again.
2019-11-18 15:57:53 +01:00
Mathias Schmid 8c22b2a316 Add check for MISRA-C 2012 Rule 2.7 and test cases. (#2340)
* Add check for MISRA-C 2012 Rule 2.7 and test cases.

* Cast unused function parameters of some test patterns to "void".

* Fix function parameter void cast in "misra_17_6". Add comment to "misra_11_7_extra" to make all function parameter void casts look alike.
2019-11-18 06:56:30 +01:00
Sebastian 8a32a210f8
addons: Change shebang to use Python 3 instead of Python 2 (#2361)
Use Python 3 instead of Python 2 if addons are executed directly.
Running cert.py and misra.py against test/cfg/std.c.dump shows that
Python 3 needs only half the time compared to Python 2. I have tested
it repeatedly and the results are always the same. This is no surprise
at all. The memory footprint is very likely also significantly better
but i have not tested it.
2019-11-15 21:38:20 +01:00
Georgy Komarov 3bd126486f addons: Clean up and clarify addons usage. (#2359)
* addons: Add '--recursive' arg. Clean up and clarify errors messages.

This commit introduce '--recursive' option for cppcheck addons.
Iff this option is set addon will recursively traverse directories in
given input files to find files with '.dump' suffix that would be
checked. Otherwise it will treat input directory as error (current
behaviour).

Add additional error handling with more clear error messages, clean up
the code.

* Add regex specifier

* Roll back --recursive option

* Update addons section in manual
2019-11-15 20:14:30 +01:00
Georgy Komarov da163ae0e8 misra.py: Add regression test for R21.1 crash (#2356) 2019-11-13 20:36:31 +01:00
Sebastian 2b1a1a2a38
misra.py: Fix mutable default argument in "misra_17_2()" (#2355)
PyCharm inspection warns for argument `visited=set()` with
"Default argument is mutable".
According to different sources one should nearly never use mutable
default arguments:
https://docs.python-guide.org/writing/gotchas/#mutable-default-arguments
https://dev.to/florimondmanca/python-mutable-defaults-are-the-source-of-all-evil-6kk
2019-11-13 14:45:34 +01:00
Georgy Komarov dbe23f878b misra.py: Fix R21.1 crash on C++ code (#2354)
Variables from cppcheckdata.Variable have no nameToken if they are
objects of user-defined classes.
2019-11-13 07:58:26 +01:00
Sebastian 3e0ecc8bed
misra.py: Fix typos, PEP 8 and Inspection warnings (#2352) 2019-11-12 15:32:05 +01:00
Sebastian 4398cf0dec
addons/namingng.py: Fix some PEP 8 and Inspections warnings (#2351) 2019-11-12 09:47:48 +01:00
Sebastian bd5d82c9bd
cppcheckdata.py: Fix PEP 8 and Inspection warnings (#2350) 2019-11-12 09:30:43 +01:00
Sebastian 1daf076d63
pylintrc_travis: Add check for bad-indentation, fix issues in misra.py (#2349) 2019-11-11 13:53:19 +01:00
Georgy Komarov 16e8016259 misra.py: Fix R21.1 FalsePositive on standard functions and errno (#2339)
Rule 21.1 now doesn't report errors on usage of library functions
started from underscore (e.g. `_exit') and valid errno assignment.

See added tests for examples.
2019-11-08 17:20:37 +01:00
Georgy Komarov ba217815cb misra.py: Fix R16.3 FN in last case/default statement (#2325)
Fix R16.3 state machine which doesn't report errors in last case/default
statemenet. See added tests for examples.

Issue on trac: https://trac.cppcheck.net/ticket/8548
2019-11-08 08:06:10 +01:00
Sebastian 828a3fda1f
pylintrc_travis: Enable more checks, fix anomalous-backslash-in-string issues (#2334) 2019-11-06 17:49:37 +01:00
Sebastian 2ce6167894 pylintrc_travis: Enable check for unnecessary-semicolon and fix issue (#2333) 2019-11-06 13:40:15 +01:00
Sebastian 0a38b1b511 pylintrc_travis: Enable more messages and fix issues (#2332)
The plan is to enable more and more useful messages and fix the issues
step by step to improve the code quality.
2019-11-05 21:05:43 +01:00
Georgy Komarov c207828a11 misra.py: Fix large memory usage with suppressions (#2321)
* misra.py: Fix large memory usage with suppressions

Don't save whole lxml locations when collecting rules suppressions
statistics. This fixes the problem with large memory usage when
some rules are suppressed.

See issue description:
https://sourceforge.net/p/cppcheck/discussion/development/thread/51fc053626/

* don't override 'file' builtin
2019-11-04 06:58:33 +01:00
Georgy Komarov ff61d8e370 misra.py: Skip execution for globally suppressed checks (#2301)
These changes will improve misra.py execution time in the
case where we use globally suppressed checks. For example, when using
single MISRA rule and suppress others, we could get about twice the
execution speed, since most of `MisraChecker' class methods will not
be started at all.

Some measurements and discussion are available at:
https://github.com/danmar/cppcheck/pull/2301
2019-11-04 06:57:41 +01:00
Armin Müller bdd41151ed Typos found by running "codespell" (#2314) 2019-10-31 15:03:18 +01:00
Georgy Komarov b35091b827 misra.py: Fix 18.8 crash on checking undefined id (#2299) 2019-10-26 17:29:38 +02:00
Georgy Komarov 72f07c8a33 Add MISRA checks for rules 21.1 and 21.12 (#2284)
* Add MISRA 21.1 check

This also required add static field for functions in symboldatabase.

* Add MISRA R21.12

* Use newer ASSERT macroses in tests
2019-10-26 08:32:46 +02:00
Georgy Komarov d1fce6b7ad addons: fixup image in README (#2277) 2019-10-17 07:08:10 +02:00
Mathias Schmid 387132389a Add check for MISRA-C 2012 rule 3.2 and test cases. (#2269)
* Add check for MISRA-C 2012 rule 3.2 and test cases.

* Fix formatting of MISRA rule 3.2 example.
2019-10-16 11:31:42 +02:00