Commit Graph

902 Commits

Author SHA1 Message Date
versat 4e051ef865 donate-cpu.py: Improve Python library detection.
The official documentation recommends to include the Python C API via
`#include "Python.h"`:
https://docs.python.org/3/c-api/intro.html
And many projects do it exactly this way, that is why the client script
often does not detect the usage of the Python C API.
2019-03-15 16:16:38 +01:00
versat 49f2fd6d4a donate-cpu.py: Detect and use ruby library. 2019-03-15 15:34:06 +01:00
Sebastian 1230357146
Library: Add initial configuration with tests for SQLite3 library. (#1737) 2019-03-13 13:57:40 +01:00
Daniel Marjamäki 35bbe5617a Revert "Add semver import for sorting ls-lR list (#1735)"
This reverts commit e61343f213.
2019-03-12 06:25:11 +01:00
Bo Rydberg e61343f213 Add semver import for sorting ls-lR list (#1735) 2019-03-12 06:14:31 +01:00
Daniel Marjamäki 5915fd50a5 triage: add options 'show 100 random results' and 'version' 2019-03-09 11:01:22 +01:00
orbitcowboy b9623eb4a6 donate-cpu.py: Do not used windows.cfg in daca@home for now. 2019-03-08 17:30:39 +01:00
orbitcowboy 267074964d donate-cpu.py: Auto detect <windows.h> and activate windows.cfg. 2019-03-07 00:59:57 +01:00
versat e64c631dea start_donate_cpu_client_productive.sh: Update client script regularly.
Use the new option `--max-packages=` to update the client script (and
the repository) regularly.
2019-03-06 08:41:16 +01:00
orbitcowboy fee5639fc8 donate-cpu.py: Enable microsoft_sal library in case <sal.h> is found by scanPackage(). 2019-03-05 18:18:24 +01:00
versat 0498b9ec34 opgengl.cfg: Introduce OpenGL library configuration file.
Prepare donate-cpu.py to detect OpenGL library usage but comment it
out for now.
2019-03-04 14:10:06 +01:00
Daniel Marjamäki d84dbf6b5f reduce.py: Improved help text 2019-03-03 20:18:54 +01:00
orbitcowboy d4511d3a22 donate-cpu.py: Auto detect cppunit configuration after next release. 2019-03-03 17:05:40 +01:00
orbitcowboy d1b42434f3 donate-cpu.py: Auto detect cppunit configuration after next release. 2019-03-03 17:04:03 +01:00
Daniel Marjamäki cb575576c7 tools/readme.md: the tools/reduce.cpp has been removed and tools/reduce.py should be used instead 2019-03-02 18:48:44 +01:00
Daniel Marjamäki 5e88258005 Makefile: Remove reduce target 2019-03-02 12:11:04 +01:00
Daniel Marjamäki 71a1290ef0 Remove reduce.cpp, let's use reduce.py instead 2019-03-02 08:13:21 +01:00
Sebastian 1877476f61
donate-cpu.py: Add new argument `--max-packages=N`. (#1718)
The client script will exit after the specified number of packages
have been processed. 0 means infinitely.
Useful for example to regularly quit the script, check for updates to
the client and start it again. Or as an alternative to the `--stop-time`
argument.
2019-03-01 22:34:50 +01:00
orbitcowboy 2a8f42d0f4 donate-cpu.py: Also extract 'h++'-files. 2019-02-28 22:41:21 +01:00
versat e64fc279f0 donate-cpu-server.py: Format total times like the package times.
Without formatting it could happen sometimes that the time is printed
like this: 245.099999999, now only one digit after the decimal point
is shown.
2019-02-28 13:46:40 +01:00
Sebastian c10ddaef4a
donate-cpu-server.py: Replace syntax that is deprecated in Python 3. (#1712)
The function `iteritems()` of `dict`s is deprecated. The recommended
alternative is to use `items()`, this function also works with Python 2.

The next issue is that lambdas can no longer unpack tuple parameters
in Python 3. It would be possible to use some workaround and still use
a lambda, but using `operator.itemgetter(1)` instead is faster and the
recommended method in such a case.

The syntax is now compatible with Python 2 and 3 but the server script
still does not work with Python 3. For example `socket.recv()` returns
`bytes` in Python 3 and `str` in Python 2. Currently `str` is expected
so it does not work with Python 3.
2019-02-28 13:34:23 +01:00
Sebastian 4e90922e5b Donate CPU: Also use and accept .tar.bz2 package files. (#1707)
In my tests there were about 1500 additional packages
available as bz2 on the server.
For some packages a newer version is now used if it is
only available as .tar.bz2 archive.
The donate-cpu.py client is tested to work with .tar.bz2
files under Python 2.7.15 and 3.6.8.
2019-02-28 08:40:07 +01:00
Sebastian 8c32cfe853
donate-cpu.py: Fix crash during decoding under Python 3. (#1710)
Python 3 directly decodes the text when it is read(). If there is any
invalid UTF-8 character in the text an exception is thrown (IIRC it is
UnicodeDecodeError). Opening the file with `error='ignore'` avoids
throwing an exception and just ignores the invalid character. Since
this is only possible since Python version 3 there must be extra code
for older versions.
The test script has been enhanced. It now also uses a package which
contains a file with at least one invalid UTF-8 character.
2019-02-27 20:01:38 +01:00
Sebastian 117eed2255
donate-cpu.py: Also extract .qml files. (#1708)
.qml files are used when the Qt library is loaded.
2019-02-27 14:59:19 +01:00
versat bcd7830fcb donate-cpu.py: Detect and use googletest library 2019-02-23 18:47:19 +01:00
Sebastian 9379266a52
donate-cpu.py: Improve regex to find more includes. (#1688)
Now also found:
- Includes directly at the beginning of a file
- Indented includes
- Includes where there is no white-space between
  "include" and header name
2019-02-23 17:13:35 +01:00
orbitcowboy 52c7a1b1c2 donate-cpu.py: Auto detect TinyXML2 cfg. 2019-02-22 17:51:47 +01:00
Sebastian 70ea6bfb9b
libcerror.cfg: Add libcerror configuration file. (#1684)
The function libcerror_error_set() is currently the function for which
daca@home most often reports a missing configuration (more than 80000
times).
Official repository of libcerror: https://github.com/libyal/libcerror
The library configuration has been tested with the library libvhdi:
ftp://ftp.se.debian.org/debian/pool/main/libv/libvhdi/libvhdi_20181227.orig.tar.gz
2019-02-22 12:52:07 +01:00
Sebastian e27a44a0ea
donate-cpu.py: Improve header detection by using regex. (#1679)
This detects more includes / headers. For example includes like
"# include <gtk/gtk.h>" with a space before "include" as it is used in
the package http://cppcheck.osuosl.org:8000/gbatnav are now also
detected.
The regex search also searches all includes for one library in one go
instead of one include per loop.
Tested with several packages to make sure libraries that were detected
before are still detected.
2019-02-21 08:14:47 +01:00
versat eeedb44e92 donate-cpu.py: Fix regression: Also upload results if only crashes happened
If "head" and "OLD_VERSION" both crash there are no messages and the
variable `results_exist" is set to False. But still the results must be uploaded
for the crashes to be saved also.
Tested with the package http://cppcheck.osuosl.org:8000/double-conversion
2019-02-19 13:29:32 +01:00
rikardfalkeborn 1c1778e4df donate_cpu: Refactor library checks (#1675) 2019-02-19 11:50:39 +01:00
Sebastian 8a9f97b5c8
donate-cpu-server.py: Verify compared Cppcheck version (#1673)
Ignore normal results (not fast or info) where the diff was made against the wrong OLD_VERSION. This avoids unwanted results when some client still analyzes an old package but the OLD_VERSION in the server script is changed for example.
Results missing the Cppcheck version info completely are also ignored.
Tested locally with correct and wrong version numbers.
2019-02-18 10:13:18 +01:00
Sebastian ff16995b26
triage: Implement Clang-Tidy and Clazy hints (#1674)
These are mainly readability and optimization changes.
2019-02-16 17:52:36 +01:00
versat a41e663cd1 Donate CPU: Optimize `list` and `dict` initialization
Initialization with `[]` and `{}` is faster than with `list()` and `dict()`.
Details:
https://stackoverflow.com/questions/30216000/why-is-faster-than-list
2019-02-12 09:47:57 +01:00
versat a1d0509d61 donate-cpu-server.py: Optimize checkLibrary* report generation
Use lists that are joined in the end instead of slower string concatenations with `+`.
Tested locally
2019-02-12 08:53:03 +01:00
versat cf00e0383d donate-cpu.py: Improve Qt library detection
Tested locally
2019-02-11 15:38:28 +01:00
Sebastian 110248c8d8
donate-cpu.py: Optimize string concatenation (use list) (#1657)
Using a list and join it in the end is really much faster when there are several hundred or thousands of strings.
2019-02-10 08:57:35 +01:00
Daniel Marjamäki 78ea6d71ac donate-cpu: run cppcheck with --experimental-fast also 2019-02-09 22:05:12 +01:00
versat bdfc20a482 donate-cpu.py: Enable "python" and "boost" library detection
Only enable the library option for those configurations if the
corresponding .cfg file exists to not crash Cppcheck if an
older version than 1.87 is used as the "old" version.
2019-02-09 18:11:09 +01:00
versat 6a592d18ab donate-cpu-server.py: Set OLD_VERSION to 1.87 2019-02-09 15:18:17 +01:00
Sebastian 36e174f8f3
donate-cpu-server.py: Add logging of uncaught exceptions (to file and console) (#1646)
Two logging handler are added. One just prints all output with at least INFO severity to the console. The other only prints ERROR severity and above to a rotating file. The file size is limited to 100 kB. Since one backup file is used that results in a maximum of 200 kB disk usage.
The log file is saved in the directory where the server script is.
Hopefully this way some issues can be found more easily.
Tested locally.
2019-02-09 12:41:02 +01:00
Sebastian a997d0b387
donate-cpu.py: Introduce version number for client script (#1654)
The version number is added to the result output so issues could be more easily reproduced and in the future if it is necessary to handle any specific version of the client by the server in a special way (for example because it is erroneous) this is possible now.
2019-02-09 09:27:54 +01:00
Daniel Marjamäki bd7790fd8c Update copyright year 2019-02-09 07:24:06 +01:00
versat ed4038b1df donate-cpu.py: Minor cleanup: Move `--library=gnu` to `libraries` variable 2019-02-08 11:19:51 +01:00
versat 06d16622e9 donate-cpu.py: Use SDL library configuration when applicable
Tested locally.
Searching for `#include <SDL.h>` does not always work. Sometimes `#include "SDL.h"` is used, but for now lets see for how many packages it works this way.
2019-02-05 13:38:42 +01:00
Sebastian 92d492e35d
donate-cpu-server.py: Limit number of functions shown in checkLibrary reports (#1641)
The checkLibraryFunction report is currently about 15 MB in size, lists more than 140000 functions and takes a long time to be generated.
Limiting the functions that are shown should save bandwidth, time and other resources on the server (and client that downloads the report).
Tested locally.
2019-02-05 10:39:40 +01:00
Sebastian 598cd468f1
donate-cpu-server.py: Add checkLibraryUseIgnore report (#1642) 2019-02-04 15:53:51 +01:00
amai2012 d24c27c9f8 donate-cpu: Postpone usage of python.cfg after release 1.87 2019-02-04 13:07:57 +01:00
versat 00eb6185fa donate-cpu-server.py: Clarify "Total time" output in time report. 2019-02-04 12:39:22 +01:00
versat dcff026da0 donate-cpu.py: Simplify check if line is not empty
In Python an empty string returns False in a conditional check. No need to calculate lenght or compare it with an empty string.
2019-02-04 11:25:29 +01:00