Commit Graph

45 Commits

Author SHA1 Message Date
Georgy Komarov d977761e76 addons: Reduce memory consumption (#2395)
* addons: Reduce memory consumption

Parse dump files incrementaly using ElementTree.iterparse. Clean unused
resources during parsing.  This method is explained in following
article: https://www.ibm.com/developerworks/xml/library/x-hiperfparse/

Memory consumption was reduced about 30% (measured with mprof),
execution time increased about 5% (measured with time utility).
More description available in PR.

* Switch to lxml and update iterparse routines

Use lxml module instead default xml.etree. Lxml provides convenient
wrappers around iterparse method that accepts `tag` argument. That
easer incremental parsing routines to select specific tags from roottree
like `dump` and `dumps`.

Element.clear() method was replaced by `lxml_clean` because lxml
keeps additional information to nodes that should be removed.

Added note about large consumption RAM on large dump files.
This commit doesn't solve this problem completely, but provides a way
to improve current parser to add incremental Configuration serialization
later.

* Working on iterative parser

* Added iterative Configurations parser

* fix

* Fix varlist iteration

* make sure that standards node was loaded
2019-12-27 08:50:56 +01:00
Georgy Komarov 2c28440f15 cert.py: Add ENV33-C-EX1 exception (#2396)
* cert.py: Add ENV33-C-EX1 exception

ENV33-C-EX1: It is permissible to call system() with a null pointer
argument to determine the presence of a command processor for the
system. [1]

[1]: https://wiki.sei.cmu.edu/confluence/pages/viewpage.action?pageId=87152177

* Handle different null-pointer constants
2019-11-24 10:31:08 +01:00
Georgy Komarov 14469aced8 cert.py: Fix EXP05C crash (#2392)
* cert.py: Fix EXP05 crash

* Fix arguments check
2019-11-23 17:41:47 +01:00
Georgy Komarov b9bc5d6d78 cert.py: Add ENV33-C (#2377) 2019-11-21 06:46:46 +01:00
Sebastian 8a32a210f8
addons: Change shebang to use Python 3 instead of Python 2 (#2361)
Use Python 3 instead of Python 2 if addons are executed directly.
Running cert.py and misra.py against test/cfg/std.c.dump shows that
Python 3 needs only half the time compared to Python 2. I have tested
it repeatedly and the results are always the same. This is no surprise
at all. The memory footprint is very likely also significantly better
but i have not tested it.
2019-11-15 21:38:20 +01:00
Georgy Komarov 3bd126486f addons: Clean up and clarify addons usage. (#2359)
* addons: Add '--recursive' arg. Clean up and clarify errors messages.

This commit introduce '--recursive' option for cppcheck addons.
Iff this option is set addon will recursively traverse directories in
given input files to find files with '.dump' suffix that would be
checked. Otherwise it will treat input directory as error (current
behaviour).

Add additional error handling with more clear error messages, clean up
the code.

* Add regex specifier

* Roll back --recursive option

* Update addons section in manual
2019-11-15 20:14:30 +01:00
Sebastian 0a38b1b511 pylintrc_travis: Enable more messages and fix issues (#2332)
The plan is to enable more and more useful messages and fix the issues
step by step to improve the code quality.
2019-11-05 21:05:43 +01:00
Armin Müller 785cc69644 Typos found by running "codespell" (#2072) 2019-08-08 21:05:28 +02:00
Georgy Komarov fe2885e430 cert.py: Refactor arguments handling (#2044)
* cert.py: Add '--quiet' option and test

Main sentinel also was required.

* travis.yml: Install pytest with --user

To avoid possible conflicts with system packages later.

* Quickfix
2019-07-26 15:40:25 +02:00
fuzzelhjb cc63728d6a add support for CERT-API01-C check (#1980)
* add support for CERT-API01-C check

* remove extra ident and simplify check
2019-07-24 10:39:31 +02:00
fuzzelhjb 999aa407f4 add CERT MSC24-C check (#1903)
* add CERT MSC24-C check

* make check a bit more robust

* fix cert tester

* finetune MSC24 check
2019-07-04 12:19:52 +02:00
fuzzelhjb 1887bd3cf0 add CERT STR11-C check (#1902)
* add CERT STR11-C check

* fix some logical checks

* fix merge issue

* update str11 check
2019-07-02 11:44:24 +02:00
fuzzelhjb 9a7ecd622c add CERT exp15-c check (#1897)
* add CERT exp15-c check

* fix false positive

* remove useless check

* fine tune check and test
2019-06-26 18:49:47 +02:00
fuzzelhjb f36d671bc5 add CERT STR03-C check (#1898)
* add CERT STR03-C check

* fix cert test
2019-06-24 18:41:43 +02:00
Daniel Marjamäki bd59999fd0 cert.py: Use cppcheck.getArguments instead of local implementation 2019-06-23 14:05:24 +02:00
fuzzelhjb 8fcfe0ceb0 add CERT STR07-C check 2019-06-23 13:59:24 +02:00
fuzzelhjb 5e3da9e82a add CERT STR05-C check (#1899)
* add CERT STR05-C check

* update check
2019-06-17 20:42:23 +02:00
Daniel Marjamäki 97157046f7 addons: /reportErrorCli/reportError/ 2019-04-14 08:54:53 +02:00
Daniel Marjamäki bf5ad56dba cert.py: Removed CLI 2019-04-10 18:09:27 +02:00
Daniel Marjamäki f2889d5c82 Try to unify addon output 2019-04-09 20:36:24 +02:00
Daniel Marjamäki bf9006737a Addons: Adding support in Cppcheck CLI to execute addons 2019-04-07 17:01:59 +02:00
Daniel Marjamäki 25a3aeef86 cert.py: another fix for valueflow 2018-10-29 01:39:52 +01:00
Daniel Marjamäki b5241ecb80 cert.py: more robust handling of values 2018-10-28 20:55:15 +01:00
Sebastian dce3281766
addons/cert.py: Add MSC30-C: Do not use rand() (#1348)
Add function isStandardFunction() that checks if the given function is a standard function.
Only when this function returns true for the currently checked rand() tokens it is reported as a violation.
Tests added for C and C++.
2018-08-24 15:05:50 +02:00
Oliver Stöneberg 13cf982b77 some small *.py script cleanup (#1328)
* added CLion project folder to .gitignore

* adjusted project name in CMakeLists.txt

* avoid warning when compiling "Debug" with Visual Studio via CMake

There was a GCC-style compiler flag in the common flags in compileroptions.cmake which caused the following warning:
cl : Command line warning D9002 : ignoring unknown option '-O0'

* compileroptions.cmake: restored original formatting

* some small *.py script cleanups
2018-08-05 20:36:21 +02:00
Daniel Marjamäki 2f18fbabaa Addons: Renamed classStart/classEnd to match SymbolDatabase 2018-04-28 23:06:54 +02:00
Daniel Marjamäki 8f21ba91e3 cert.py: Fix FP 2018-04-12 20:23:50 +02:00
Daniel Marjamäki 97b17d50aa cert.py: fixed fp in exp05 for non-pointer casts 2018-04-03 15:12:01 +02:00
Daniel Marjamäki 2db7c6e1a3 cert.py: Add rule EXP-05: do not cast away const 2018-04-03 13:35:19 +02:00
amai ed25e21929 Set executable bit for addons/ python scripts 2018-03-20 20:54:59 +01:00
amai2012 449dcc15e8 Add links to external references 2018-03-16 08:12:39 +01:00
Daniel Marjamäki 8eae9cdd60 cert.py: Added testcases for EXP42 and fixed the code a little 2018-01-23 17:18:47 +01:00
Daniel Marjamäki 438b1ccb91 cert.py: update cert-INT31-C checker 2017-10-16 13:35:07 +02:00
Daniel Marjamäki 8a8339274b cert.py: add testfile 2017-10-15 16:18:29 +02:00
Daniel Marjamäki 56ceb5a58c cert.py: Added INT-31-C check, ensure that integer conversions do not result in lost or misinterpreted data 2017-10-14 22:24:24 +02:00
Daniel Marjamäki b7af13ce63 cert.py: reformat output again 2017-08-15 21:44:43 +02:00
Daniel Marjamäki 256adfbcb5 cert.py: Reformatting output 2017-08-15 20:34:54 +02:00
Ayaz Salikhov 3dc4188292 Improve Python code 2017-06-05 13:23:00 +02:00
Ayaz Salikhov 2dd6168258 Improve Python code 2017-06-04 22:51:48 +02:00
Albert ARIBAUD (3ADEV) e7fdb1c825 Fix multiple config dumps
With multiple configurations, option --dump only dumps
the last configuration.

Fix it to dump every configuration.

Also update all Python addons so that they can handle
multiple-configuration dumps.

Additionally run autopep8 on addons/*.py.

The results of 'make test' before and after applying
this commit are identical.
2015-12-14 18:20:35 +01:00
Daniel Marjamäki 4464d309fa cert.py: remove messages for reading struct padding data. in the EXP-42 warning describe how it can be fixed 2015-08-29 08:25:48 +02:00
Daniel Marjamäki 498c920180 addons/cert.py: better handling of structs. read pragmas in the file. 2015-08-28 18:07:12 +02:00
XhmikosR 7d0075357e PEP8 fixes.
[ci skip]
2015-08-21 11:59:52 +03:00
Daniel Marjamäki 71eb96b8d3 Addons: Added simple addon checking for CERT rule EXP42-C. Don't compare struct padding data. 2015-08-19 10:21:25 +02:00
Daniel Marjamäki abbe11f3b7 Addons: Added cert.py 2015-08-18 16:14:53 +02:00