Commit Graph

124 Commits

Author SHA1 Message Date
Georgy Komarov d977761e76 addons: Reduce memory consumption (#2395)
* addons: Reduce memory consumption

Parse dump files incrementaly using ElementTree.iterparse. Clean unused
resources during parsing.  This method is explained in following
article: https://www.ibm.com/developerworks/xml/library/x-hiperfparse/

Memory consumption was reduced about 30% (measured with mprof),
execution time increased about 5% (measured with time utility).
More description available in PR.

* Switch to lxml and update iterparse routines

Use lxml module instead default xml.etree. Lxml provides convenient
wrappers around iterparse method that accepts `tag` argument. That
easer incremental parsing routines to select specific tags from roottree
like `dump` and `dumps`.

Element.clear() method was replaced by `lxml_clean` because lxml
keeps additional information to nodes that should be removed.

Added note about large consumption RAM on large dump files.
This commit doesn't solve this problem completely, but provides a way
to improve current parser to add incremental Configuration serialization
later.

* Working on iterative parser

* Added iterative Configurations parser

* fix

* Fix varlist iteration

* make sure that standards node was loaded
2019-12-27 08:50:56 +01:00
Georgy Komarov 88a3b4c685 addons: Fix __repr__ methods (#2448)
This commit fixes __repr__ methods introduced in d538315268. Fields that recursively links to other cppcheckdata objects was removed to avoid max recursion depth crash on printing.
2019-12-15 19:46:54 +01:00
Francesc Elies d538315268 [python] classes are missing __repr__ methods (#2437)
* [python] classes don't have a __repr__ method

* [python] removes unused import
2019-12-09 19:08:45 +01:00
Georgy Komarov 3bd126486f addons: Clean up and clarify addons usage. (#2359)
* addons: Add '--recursive' arg. Clean up and clarify errors messages.

This commit introduce '--recursive' option for cppcheck addons.
Iff this option is set addon will recursively traverse directories in
given input files to find files with '.dump' suffix that would be
checked. Otherwise it will treat input directory as error (current
behaviour).

Add additional error handling with more clear error messages, clean up
the code.

* Add regex specifier

* Roll back --recursive option

* Update addons section in manual
2019-11-15 20:14:30 +01:00
Sebastian bd5d82c9bd
cppcheckdata.py: Fix PEP 8 and Inspection warnings (#2350) 2019-11-12 09:30:43 +01:00
Georgy Komarov 72f07c8a33 Add MISRA checks for rules 21.1 and 21.12 (#2284)
* Add MISRA 21.1 check

This also required add static field for functions in symboldatabase.

* Add MISRA R21.12

* Use newer ASSERT macroses in tests
2019-10-26 08:32:46 +02:00
Daniel Marjamäki 2d9a131817 Refactoring: Rename variables. Do not use leading _. Renamed 'col' to 'column' 2019-08-18 12:19:05 +02:00
Daniel Marjamäki f33a8a417f Put 'isExpandedMacro' info in the dump files 2019-08-15 14:24:56 +02:00
Daniel Marjamäki f524bf79ad Misra: Added 17.2 checker 2019-07-13 15:10:07 +02:00
Sebastian d1386112a9
addons/cppcheckdata.py: Fix missing self, and wrong assignment. (#1919)
I am not sure but it looks like "self." is missing here.
And the attribute "inconclusive" is initialized as a "bool", so i guess
"True" should be assigned here?
2019-06-25 19:16:57 +02:00
Daniel Marjamäki c97dc79815 --addon: Use json for addon output 2019-06-24 19:29:43 +02:00
Daniel Marjamäki 63bd29d644 addons/cppcheckdata.py: Add 'cppcheckdata.getArguments' function 2019-06-23 13:54:33 +02:00
eivindt 44670005ea Record C/CPP/Posix Standard used for cppcheck in dump file / use this for misra checking (#1782)
* Add cmd parameter for choosing between C90 and C99

Misra specifies different requirements to the uniqueness of
macros/enums/variables depending on what C standard
that's being used.

* Add standards configuration to each dump file

Read standards config from misra addon to decide what rules to use.

* Posix as standard setting should be deprecated.  Don't include this in the xml

* Rewritten to use a switch
2019-06-14 12:06:57 +02:00
Daniel Marjamäki 1584e62367 cppcheckdata: Added 'simpleMatch' function 2019-06-06 15:38:15 +02:00
Daniel Marjamäki d039c388ec Fixed #8582 (Regression MISRA Rule 5.1) 2019-05-26 18:46:35 +02:00
Daniel Marjamäki 54be0bc3cf Misra: Added checker for misra rule 17.7 2019-05-22 21:39:33 +02:00
Daniel Marjamäki e0da316759 cppcheckdata.py: fix sys.errout 2019-04-16 08:26:58 +02:00
Daniel Marjamäki 97157046f7 addons: /reportErrorCli/reportError/ 2019-04-14 08:54:53 +02:00
Daniel Marjamäki b1b5b27b4e addons: write column in error message(s) 2019-04-13 10:22:13 +02:00
Daniel Marjamäki 4509b25f97 replace tabs with spaces in cppcheckdata.py 2019-04-10 06:48:16 +02:00
Daniel Marjamäki 4415992a62 Addons: Remove the xml output. You can get xml output from the GUI/command line clients. 2019-04-09 20:42:17 +02:00
Daniel Marjamäki f2889d5c82 Try to unify addon output 2019-04-09 20:36:24 +02:00
eivindt 0debd8d520 Misra xml output (#1787)
* Add (very) simple XML output support

* Removed xml top and tail, better handled outside
2019-04-05 12:36:58 +02:00
Daniel Marjamäki 2730b16326 cppcheckdata.py: read more data from dumps 2018-10-28 20:47:44 +01:00
Oliver Stöneberg 13cf982b77 some small *.py script cleanup (#1328)
* added CLion project folder to .gitignore

* adjusted project name in CMakeLists.txt

* avoid warning when compiling "Debug" with Visual Studio via CMake

There was a GCC-style compiler flag in the common flags in compileroptions.cmake which caused the following warning:
cl : Command line warning D9002 : ignoring unknown option '-O0'

* compileroptions.cmake: restored original formatting

* some small *.py script cleanups
2018-08-05 20:36:21 +02:00
Daniel Marjamäki 9edcae97fc Fixed #8578 (Argument scope as global) 2018-06-19 08:50:32 +02:00
Daniel Marjamäki 2f23757c74 addons/naming.py: Allow regular expression for private member variable names 2018-06-17 14:46:59 +02:00
Konrad Grochowski c30f21ed2a cppcheckdata.py - support for missing file name in suppression (#1286) 2018-06-13 22:32:27 +02:00
Konrad Grochowski 0c6d60d202 MISRA - support for per file/line error suppression (#1275)
* cppcheckdata.py: fixed Suppression.isMatch method

* cppcheckdata.py: fixed parsing <suppressions> tag

* misra.py: now uses cppcheckdata.reportError and supports suppressions
2018-06-04 21:50:21 +02:00
Swasti Shrivastava c511b3de20 Modified rule 11.4 (#1252)
* Modified rule 11.4

* Made changes in isEnum()

* Made suggested changes in isEnum()
2018-05-22 09:07:48 +02:00
Daniel Marjamäki 8be593db8f misra: fixed false negatives 2018-05-04 22:18:22 +02:00
Daniel Marjamäki 2a9ee563c2 misra: avoid fp for essentially boolean conditions when bitfield member is used 2018-05-02 20:56:03 +02:00
Daniel Marjamäki fce7a0a128 dump: add Function::type 2018-04-30 16:52:51 +02:00
Daniel Marjamäki 2f18fbabaa Addons: Renamed classStart/classEnd to match SymbolDatabase 2018-04-28 23:06:54 +02:00
rebnridgway 995b496ddf Add suppressions to the XML dump (#1166)
* Added parsing suppressions from dump xml.

* Added code to dump suppressions to an xml file

* Added declaration for dump function

* Suppressions will now be written to the xml file when a dump is requested

* Fixed syntax error

* Removed excess whitespace

* Fixed indentation to be consistent

* Fixed indentation to be consistent

* Fixed indentation to be consistent

* Added missing include for ErrorLogger::toXml

* Fixed suggestions from pull request #1166

Switched to using ranged for loop to iterate through suppressions.
Made the line number attribute optional, rather than 0 if not specified.  This means when Python deserialises it it will be None, which is more pythonic.

* Implemented checking suppressions in reportError

This modification expects suppressions and a function to be called to write a line of output to be passed in.  The function checks if any of the suppressions match the warning (with the new Suppression.isMatch function) and if so returns None.  This change maintains the old behaviour of returning the warning text, but adds the possibility of returning None if the warning was suppressed.

* Fixed code quality warnings

* Removed more extraneous whitespace
2018-04-24 22:19:24 +02:00
Daniel Marjamäki 97c86ff0e6 Dumps: Added isVirtual/isImplicitlyVirtual info 2018-04-13 21:15:43 +02:00
Daniel Marjamäki 8f21ba91e3 cert.py: Fix FP 2018-04-12 20:23:50 +02:00
Daniel Marjamäki 1767fe525b Addons: Fixed handling of noname arguments 2018-04-10 11:10:10 +02:00
Daniel Marjamäki 1270ae2ad8 cppcheckdata: fixed loading of function data 2018-04-03 13:20:21 +02:00
amai ed25e21929 Set executable bit for addons/ python scripts 2018-03-20 20:54:59 +01:00
Daniel Marjamäki b8d0da31de threadsafety.py: warn for local static non-class variables also 2018-03-14 11:55:01 +01:00
Daniel Marjamäki 773cc07d44 cppcheckdata.py: parsing of NULL pointers from VS 32-bit and 64-bit 2017-10-07 22:44:35 +02:00
Ayaz Salikhov 2e6a22e882 Improve Python code 2017-07-22 11:05:50 +02:00
Ayaz Salikhov 3dc4188292 Improve Python code 2017-06-05 13:23:00 +02:00
Ayaz Salikhov 2dd6168258 Improve Python code 2017-06-04 22:51:48 +02:00
Daniel Marjamäki 4f64e67298 Misra: Add rule 8.11 2017-04-17 07:45:27 +02:00
Daniel Marjamäki 85a6e9ce59 Misra: Add rule 11.3 2017-04-16 12:13:30 +02:00
Daniel Marjamäki 1045ece946 dump: refactor valueType dump 2017-04-16 09:11:20 +02:00
Daniel Marjamäki b59d46091e Misra: Add rule 11.8 2017-04-15 21:55:07 +02:00
Daniel Marjamäki dd8b96f4c8 dump: Read platform info from dump file 2017-04-15 19:15:48 +02:00
Daniel Marjamäki 41e07c0614 Misra: Added rule 17.1 2017-04-14 14:13:37 +02:00
Daniel Marjamäki 6e0af5d01f MISRA 19 and 28 2017-04-09 10:11:54 +02:00
Daniel Marjamäki 8755023c1c Fix MISRA 58 2017-04-08 19:33:26 +02:00
Daniel Marjamäki 2ca85a1c40 dump: add isUnsigned/isSigned 2016-12-09 22:31:47 +01:00
John-Paul Ore ffa8af69f2 Update cppcheckdata.py
fixes problem reading from dump files.  One character typo in Scope class.
2016-09-07 10:38:27 -05:00
XhmikosR f232f342e7 addons/*.py: formatting.
[ci skip]
2016-07-28 11:11:08 +03:00
Daniel Marjamäki 8c8ad96fe5 cppcheckdata.py: reformat documentation 2016-07-25 13:47:26 +02:00
XhmikosR 07b43c6929 Ran autopep8. 2016-07-25 13:16:55 +03:00
Daniel Marjamäki 85c7456546 http://cppcheck.sourceforge.net => http://cppcheck.net 2016-06-05 18:17:47 +02:00
Albert ARIBAUD (3ADEV) 38e70dfb74 Preprocessor directives for addons
This patch augments the XML dumps with a 'directivelist'
subnode which lists all raw preprocessor directives met
while reading the source code in each configuration.

Also, the addons/cppcheckdata.py file has been extended
to give easy access to the list of directives and to
provide Python support for the --template (or short -t)
option.

Finally, an new addon, addons/y2038/y2038.py, is created
to detect when a glibc symbol might be Y2038-sensitive,
based on whether and how _TIME_BITS and _USE_TIME_BITS64
are defined when meeting the symbol.
2016-01-15 12:36:35 +01:00
Daniel Marjamäki 7d4c37430b minor tweak of comment 2015-12-14 20:30:13 +01:00
Daniel Marjamäki 8f27cec991 Revert "minor tweak of comment"
This reverts commit b1d1869f22.
2015-12-14 20:29:29 +01:00
Daniel Marjamäki b1d1869f22 minor tweak of comment 2015-12-14 20:03:40 +01:00
Albert ARIBAUD (3ADEV) e7fdb1c825 Fix multiple config dumps
With multiple configurations, option --dump only dumps
the last configuration.

Fix it to dump every configuration.

Also update all Python addons so that they can handle
multiple-configuration dumps.

Additionally run autopep8 on addons/*.py.

The results of 'make test' before and after applying
this commit are identical.
2015-12-14 18:20:35 +01:00
Daniel Marjamäki 498c920180 addons/cert.py: better handling of structs. read pragmas in the file. 2015-08-28 18:07:12 +02:00
Daniel Marjamäki eb3b3de81f cppcheckdata: Added doxygen comments 2015-08-26 18:10:33 +02:00
XhmikosR 7d0075357e PEP8 fixes.
[ci skip]
2015-08-21 11:59:52 +03:00
Daniel Marjamäki 502eebfc9c addons: fix initialization in CppcheckData when multiple files are parsed 2015-08-19 12:16:13 +02:00
Daniel Marjamäki 248f468c67 Addons: Use builtin xml library instead of lxml 2015-08-18 10:59:57 +02:00
Daniel Marjamäki 0e82730ee4 addons/naming.py: fixed error report for function name 2015-07-28 14:34:37 +02:00
Daniel Marjamäki 5a57e4030a dump: Add Function name attribute 2015-07-28 14:18:58 +02:00
Daniel Marjamäki d828cd29cb cppcheckdata.py: Added comments 2015-06-21 13:26:32 +02:00
Daniel Marjamäki 1df0204bbe addons: Use bool instead of string for cppcheckdata properties is... 2015-06-21 09:54:07 +02:00
Daniel Marjamäki d0e6b77fd1 cppcheckdata.py: moved file from tools to addons 2015-06-21 09:47:52 +02:00