Commit Graph

128 Commits

Author SHA1 Message Date
Sebastian ec521fba36
donate-cpu-server.py: Upgrade to work with Python 3 (drop Python 2 support) (#2292)
* donate-cpu-server.py: Use tools to prepare code to work with Python 3

The following commands were used for these changes:
futurize -1 -w donate-cpu-server.py
2to3 -w donate-cpu-server.py

* Make the server work under Python 3

Manually fixed the Unicode issues. Received data is decoded, sent data
is encoded.

* Add backward compatible type hints (in comments)

This enables better static analysis and suggestions in an IDE.

* Fix Pylint warning "Comparison to literal"

* .travis.yml: Fix/enhance pylint verification and Python compilation

donate-cpu-server.py is only Python 3 compatible, so it must be ignored
for pylint verification under Python 2.
All Python scripts that were verified with pylint under Python 2 are
now also verified with pylint under Python 3.

* donate-cpu-server.py: Add shebang and mark script as executable

* start_donate_cpu_server_test_local.sh: Directly execute server

Since the server script is executable now and has a shebang it can
be directly executed.

* Use Python 3.0 function annotations instead of comment type hints

Reference: https://www.python.org/dev/peps/pep-3107/
2019-10-26 21:10:21 +02:00
Maksim Derbasov a7a568d75a donate cpu server: added 2 commands (#2283) 2019-10-19 10:52:04 +02:00
Sebastian f3f00d1780
donate-cpu-server.py: Use "/var/daca@home" as work path by default. (#2281)
Fall back to "~/daca@home" if "/var/daca@home" does not exist.
Print the used work path when the script starts.
This way we do not have to change the server script before uploading
it to the server while being fully backwards compatible.
2019-10-18 20:04:50 +02:00
Sebastian c8e1ca7127
donate-cpu-server.py: Fix PEP8 issues (#2280) 2019-10-18 12:57:51 +02:00
Rikard Falkeborn 6e1c7e48b6 Donate cpu server crash locations (#2276)
donate-cpu-server: Print grouped stack traces in crash report

Example output:

	Packages: psi xenomai
	Token::hasKnownValue (this=0x0) at lib/token.h:988
	988	        return mImpl->mValues && std::any_of(mImpl->mValues->begin(), mImpl->mValues->end(), std::mem_fn(&ValueFlow::Value::isKnown));
	#0 Token::hasKnownValue (...) at lib/token.h:988
	#1 valueFlowReverse (...) at build/valueflow.cpp:3775
	#2 valueFlowBeforeCondition (...) at /usr/include/c++/8/bits/stl_list.h:301
	#3 ValueFlow::setValues (...) at build/valueflow.cpp:8403
	#4 Tokenizer::simplifyTokens1 (...) at build/tokenize.cpp:11095
	#5 CppCheck::checkFile (...) at build/cppcheck.cpp:513
	#6 CppCheck::check (...) at /usr/include/c++/8/bits/basic_string.h:936
	#7 CppCheckExecutor::check_internal (...) at cli/cppcheckexecutor.cpp:884
	#8 CppCheckExecutor::check (...) at cli/cppcheckexecutor.cpp:198
	#9 main (...) at cli/main.cpp:95

	Packages: broker
	valueFlowReverse (tokenlist=tokenlist@entry=0x7fffffffbce0, tok=tok@entry=0x555555cc9930, varToken=varToken@entry=0x555555cc9b70, val=..., val2=..., errorLogger=errorLogger@entry=0x7fffffffcb20, settings=0x7fffffffcd40) at build/valueflow.cpp:3775
	3775	                if (!assignTok->hasKnownValue()) {
	#0 valueFlowReverse (...) at build/valueflow.cpp:3775
	#1 valueFlowBeforeCondition (...) at build/valueflow.cpp:4092
	#2 ValueFlow::setValues (...) at build/valueflow.cpp:8406
	#3 Tokenizer::simplifyTokens1 (...) at build/tokenize.cpp:11095
	#4 CppCheck::checkFile (...) at build/cppcheck.cpp:513
	#5 CppCheck::check (...) at build/cppcheck.cpp:197
	#6 CppCheckExecutor::check_internal (...) at cli/cppcheckexecutor.cpp:884
	#7 CppCheckExecutor::check (...) at cli/cppcheckexecutor.cpp:198
	#8 main (...) at cli/main.cpp:95
2019-10-18 09:56:15 +02:00
Sebastian 5a05e8d242
donate-cpu-server.py: Get rid of hard-coded paths (#2159) 2019-09-10 07:58:37 +02:00
versat 8b949c33de donate-cpu-server.py: Fix crash during diff report generation
If there are *.diff files with old version numbers the server script
crashed because it always expects a key with the current OLD_VERSION.
This fix ignores entries in *.diff files that are not made against the
current OLD_VERSION.
2019-09-02 13:46:57 +02:00
Daniel Marjamäki 8865fb544d createrelease: clarify daca@home update 2019-09-01 15:34:06 +02:00
Sebastian a309095ec3
donate-cpu-server.py: Fix rare version verification issue (#2008)
The loop should be exited as soon as the line with the Cppcheck version
information has been found. Not only when the verification failed.
2019-07-17 14:29:07 +02:00
Sebastian db3284e3dd
donate-cpu-server.py: Fix #9192: Old results are shown in the reports (#1941)
Trac ticket: https://trac.cppcheck.net/ticket/9192
This commit also fixes that negative values of the elapsed time are
used for calculating total times. These crashes and errors are now
ignored in the time report since there is no useful timing information
in that case.

Tested with a local daca@home server with old and new results.
2019-07-01 14:55:38 +02:00
Daniel Marjamäki 8be4af33d3 tools/donate-cpu-server: speed up the diff reports using the new diff files 2019-06-30 23:16:04 +02:00
Daniel Marjamäki 098144fbe8 donate-cpu-server.py: Fixed bug 2019-06-30 14:14:02 +02:00
Daniel Marjamäki c9db395d43 donate-cpu-server: Refactor diff handling. Statistics for a package is generated when the package is uploaded. 2019-06-30 14:02:51 +02:00
Daniel Marjamäki 090fa255e8 daca@home: use Cppcheck-1.88 as base version 2019-06-29 11:01:42 +02:00
versat 20814d37d0 donate-cpu-server.py: Fix wrong detection of invalid messageIds.
Packages now can contain something like:
```
    head results:
    Checking temp/openvdb/Platform.cc: __GNUC__=1...
    [New Thread 7892.0x91c]
```
"New Thread 7892.0x91c" was wrongly identified as messageId in the HEAD
report.
This commit adds code to skip lines that start with `[` or where the
messageId contains at least one space.
2019-04-03 11:46:45 +02:00
versat a05cdadbf4 donate-cpu-server.py: Remove obsolete experimental "fast" code.
Since the directory for the results does no longer exist on the server,
the server currently crashes every time older clients try to upload
experimental fast results via "write-fast" command.
Now this command is just ignored so the server is instantly ready
again after a "write-fast" command.
2019-03-29 12:52:27 +01:00
Oliver Stöneberg 36950d7d0d donate-cpu-server.py: added stale report / cleanups (#1767)
* donate-cpu.server.py: increased "Package" column width for latest report and small cleanup

* donate-cpu.server.py: added date and time to crash report

* donate-cpu.server.py: simplified strDateTime()

* donate-cpu.server.py: add stale report to show results which are older than 30 days

* donate-cpu-server.py: added version and some logging
2019-03-29 12:12:16 +01:00
versat e64fc279f0 donate-cpu-server.py: Format total times like the package times.
Without formatting it could happen sometimes that the time is printed
like this: 245.099999999, now only one digit after the decimal point
is shown.
2019-02-28 13:46:40 +01:00
Sebastian c10ddaef4a
donate-cpu-server.py: Replace syntax that is deprecated in Python 3. (#1712)
The function `iteritems()` of `dict`s is deprecated. The recommended
alternative is to use `items()`, this function also works with Python 2.

The next issue is that lambdas can no longer unpack tuple parameters
in Python 3. It would be possible to use some workaround and still use
a lambda, but using `operator.itemgetter(1)` instead is faster and the
recommended method in such a case.

The syntax is now compatible with Python 2 and 3 but the server script
still does not work with Python 3. For example `socket.recv()` returns
`bytes` in Python 3 and `str` in Python 2. Currently `str` is expected
so it does not work with Python 3.
2019-02-28 13:34:23 +01:00
Sebastian 4e90922e5b Donate CPU: Also use and accept .tar.bz2 package files. (#1707)
In my tests there were about 1500 additional packages
available as bz2 on the server.
For some packages a newer version is now used if it is
only available as .tar.bz2 archive.
The donate-cpu.py client is tested to work with .tar.bz2
files under Python 2.7.15 and 3.6.8.
2019-02-28 08:40:07 +01:00
Sebastian 8a9f97b5c8
donate-cpu-server.py: Verify compared Cppcheck version (#1673)
Ignore normal results (not fast or info) where the diff was made against the wrong OLD_VERSION. This avoids unwanted results when some client still analyzes an old package but the OLD_VERSION in the server script is changed for example.
Results missing the Cppcheck version info completely are also ignored.
Tested locally with correct and wrong version numbers.
2019-02-18 10:13:18 +01:00
versat a41e663cd1 Donate CPU: Optimize `list` and `dict` initialization
Initialization with `[]` and `{}` is faster than with `list()` and `dict()`.
Details:
https://stackoverflow.com/questions/30216000/why-is-faster-than-list
2019-02-12 09:47:57 +01:00
versat a1d0509d61 donate-cpu-server.py: Optimize checkLibrary* report generation
Use lists that are joined in the end instead of slower string concatenations with `+`.
Tested locally
2019-02-12 08:53:03 +01:00
Daniel Marjamäki 78ea6d71ac donate-cpu: run cppcheck with --experimental-fast also 2019-02-09 22:05:12 +01:00
versat 6a592d18ab donate-cpu-server.py: Set OLD_VERSION to 1.87 2019-02-09 15:18:17 +01:00
Sebastian 36e174f8f3
donate-cpu-server.py: Add logging of uncaught exceptions (to file and console) (#1646)
Two logging handler are added. One just prints all output with at least INFO severity to the console. The other only prints ERROR severity and above to a rotating file. The file size is limited to 100 kB. Since one backup file is used that results in a maximum of 200 kB disk usage.
The log file is saved in the directory where the server script is.
Hopefully this way some issues can be found more easily.
Tested locally.
2019-02-09 12:41:02 +01:00
Sebastian 92d492e35d
donate-cpu-server.py: Limit number of functions shown in checkLibrary reports (#1641)
The checkLibraryFunction report is currently about 15 MB in size, lists more than 140000 functions and takes a long time to be generated.
Limiting the functions that are shown should save bandwidth, time and other resources on the server (and client that downloads the report).
Tested locally.
2019-02-05 10:39:40 +01:00
Sebastian 598cd468f1
donate-cpu-server.py: Add checkLibraryUseIgnore report (#1642) 2019-02-04 15:53:51 +01:00
versat 00eb6185fa donate-cpu-server.py: Clarify "Total time" output in time report. 2019-02-04 12:39:22 +01:00
Sebastian bcfd7c0919
donate-cpu-server.py: Fix that results of package "crash" can not be … (#1620)
* donate-cpu-server.py: Fix that results of package "crash" can not be viewed

Rename crash report page from "crash" to "crash.html" so the url ending with "crash" shows the results of package "crash".

* Also rename "diff" and "head" reports to *.html

In case there would be such packages in the future this avoids conflicts.
2019-01-24 12:02:45 +01:00
Sebastian a410cea59a
Donate CPU: Collect information messages and provide checkLibrary reports (#1610)
Related trac ticket: https://trac.cppcheck.net/ticket/8947
Enable information messages and "--check-library" in the Cppcheck
parameters.
Store the information messages and the rest of the messages in different
variables and upload them separately.
The server stores the information messages in a sub-directory similarly
to the normal issue messages in one file per package.
Reports for "checkLibraryFunction" and "checkLibraryNoReturn" message ids
are generated by the server now.
2019-01-22 15:27:13 +01:00
Sebastian 7281a886e2
donate-cpu-server.py: Enhance formatting of latest.html output (#1605)
Column width can now be easily changed via the `column_width` array.
Increased column width for the packages column so it is more likely that the date is not dropped.
Increased the column width for the number of results since there are packages which have about 100000 results.
2019-01-15 11:39:47 +01:00
versat 0c93e0c8cb donate-cpu-server.py: Increase max accepted result size to 2 MB 2019-01-11 14:58:27 +01:00
Sebastian e8fb3c0546
donate-cpu-server.py: Time report: Further improvements and fixes (#1586)
Changes:
- Rename "time" page to "time.html" to avoid conflict with "time" package
- Output content as HTML instead of plain text
- Format head row of preformatted table bold
- Show factor for total time now too

Tested locally
2019-01-11 10:05:19 +01:00
Sebastian fd9ca16e51
donate-cpu-server.py: Time report: Format output, add showing factor (#1582)
Format the time output so the columns have the same width (as long as a text is not too long).
Add showing the factor of the time difference so very suspicious differences can be seen instantly.
Made the function body more PEP 8 conform by using lowercase variable names with underscores as delimiters.
Tested locally.
2019-01-10 19:10:56 +01:00
Sebastian 1dc5ff022e
donate-cpu-server.py: Fix PEP 8 and other warnings (#1578)
There should be no functional changes.
Tested locally.
2019-01-09 18:39:42 +01:00
Sebastian 44135ef926
donate-cpu-server.py: Provide HEAD report (similar to diff report) (#1572)
Tested on local server.
2019-01-09 10:47:58 +01:00
Sebastian 30d53a911e
donate-cpu-server.py: Fix package names in time output. (#1570)
The package names contained the whole path, so the leading "resultPath" is now removed to only show the package names.
Tested on local server.
2019-01-07 19:31:34 +01:00
Sebastian 164a46f8cd donate-cpu-server.py: Fix date/time in report is not found. (#1562)
Instead of searching for a specific year (which could change) the script searches now for the current and last year to find the date and time information in the results.
2019-01-04 11:51:32 +01:00
Sebastian a4a3373838 donate-cpu-server.py: Fix wrong number of "+" diffs in latest results. (#1517)
This should fix the issue that the number after the "+" in the "Diff" column is always one too large.
2018-12-10 20:05:34 +01:00
Daniel Marjamäki 49e500d6bf donate-cpu: use 1.86 as base version 2018-12-08 11:39:44 +01:00
Daniel Marjamäki f0826ed9d9 donate-cpu-server: restore handling of crashes. do not recheck. 2018-12-02 07:10:23 +01:00
Daniel Marjamäki 5532d1f877 donate-cpu: Add GetCppcheckVersions command 2018-11-29 21:19:45 +01:00
Daniel Marjamäki e11afb7b0c donate-cpu-server: refactor 2018-11-29 06:23:29 +01:00
Daniel Marjamäki 6b4f5026f7 donate-cpu-server: show crash history 2018-11-28 20:36:19 +01:00
Daniel Marjamäki bbcadb0f9b donate-cpu-server: try to track crashes better 2018-11-28 14:03:54 +01:00
IOBYTE 8e6013c8af Allow daca to display latest results of packages with a plus in their name. (#1496) 2018-11-28 06:09:29 +01:00
Daniel Marjamäki af60b6f125 donate-cpu-server: save crash history 2018-11-27 19:08:52 +01:00
Daniel Marjamäki 347c46491a donate-cpu-server: try to handle uploaded crash results better 2018-11-19 10:57:04 +01:00
Daniel Marjamäki e8ac45a5a7 donate-cpu-server.py: Recheck crashes more frequently 2018-11-17 19:32:10 +01:00
Daniel Marjamäki f72847530e Donate CPU: make server more robust when latest results are missing 2018-10-15 11:01:51 +02:00
Daniel Marjamäki 441d50df25 donate-cpu: use 1.85 as base version 2018-10-14 16:31:13 +02:00
Daniel Marjamäki 4a4d4fd2be Donate CPU: Improve diff report so changes from today can be seen separately 2018-09-06 17:31:07 +02:00
Daniel Marjamäki ee104303b7 Donate CPU: create diff report for todays results 2018-09-06 06:53:40 +02:00
Sebastian 220c06e356 Donate CPU Server: Fix broken links in crash report. (#1364) 2018-09-03 13:07:47 +02:00
Daniel Marjamäki f88326c51f Donate CPU: let the server remember latest packages when restarting 2018-09-02 14:08:09 +02:00
Daniel Marjamäki c6369e6519 Donate CPU: Added a simple time report 2018-09-02 09:35:38 +02:00
Daniel Marjamäki 215bc3b303 Donate CPU:
* timeout after 30 seconds
 * added content for http://cppcheck.osuosl.org:8000/
2018-09-02 07:28:25 +02:00
Daniel Marjamäki 5767a0ed6f Donate CPU: Added crash report 2018-08-31 14:28:01 +02:00
Daniel Marjamäki ec5a13d351 Donate CPU: Fix old path 2018-08-27 18:44:17 +02:00
Daniel Marjamäki 4002fcd3e3 Donate CPU: Improved the server, now you can see diff 2018-08-27 18:21:16 +02:00
Daniel Marjamäki bbebdf0ab4 Donate CPU: Updated the 'latest.html' report 2018-08-26 16:57:09 +02:00
Daniel Marjamäki 629639f6ae Donate CPU: Updated scripts 2018-08-26 16:47:20 +02:00
Daniel Marjamäki 08837b2232 Donate CPU: Use CRLF in HTTP responses 2018-08-26 11:17:18 +02:00
Daniel Marjamäki c65d79c0ec Donate CPU: Speedup server. Keep packageIndex in package-index.txt 2018-08-25 20:58:31 +02:00
Daniel Marjamäki 829a373416 Python cleanup 2018-08-25 20:43:20 +02:00
Daniel Marjamäki 5da38e33c0 Donate CPU: Make server more robust 2018-08-25 20:00:04 +02:00
Daniel Marjamäki 7049249460 Donate CPU: Tried to make scripts more robust 2018-08-25 18:38:51 +02:00
Daniel Marjamäki 18343d1122 Donate CPU: get package 2018-08-25 11:08:41 +02:00
Daniel Marjamäki 4ece582bed Donate CPU: looking at packages 2018-08-25 10:59:49 +02:00
Daniel Marjamäki 33b93cdd3a Donate CPU: send data in chunks 2018-08-25 10:25:05 +02:00
Daniel Marjamäki 48f5792ca1 Donate CPU: Speedup the latest report 2018-08-25 09:06:15 +02:00
Daniel Marjamäki efe40950c9 Donate CPU: Added report http://cppcheck.osuosl.org:8000/latest.html 2018-08-25 08:49:40 +02:00
Daniel Marjamäki 045ef9a715 Donate CPU: More updates. Run both 1.84 and head. 2018-08-24 21:07:50 +02:00
Daniel Marjamäki 8c88f65fb2 Donate CPU: minor updates in server 2018-08-24 13:04:25 +02:00
Daniel Marjamäki b049203be7 Donate CPU: updated server 2018-08-23 22:13:53 +02:00
Daniel Marjamäki 1e919e696b Donate CPU: show info on server 2018-08-23 21:54:46 +02:00
Daniel Marjamäki 8459380a79 Donate CPU: Added simple client/server scripts 2018-08-23 21:31:02 +02:00