This improves the performance of the templatesimplefier by caching the template name position. I am not sure if the works entirely correctly but all the tests do pass with this change. Running this with gtest headers without removing unused template headers the time went from 48s to 5s, almost a 10x improvement.
* template simplifier: fix instantiation of variadic template with no arguments
* fix white space change
* add support for <class...>
* add variadic template flag
* Fix issue 8890: AST broken calling member function from templated base class
* Format
* Check for double bracket
* Add test to createLinks2
* Remove extra test
* Reduce test case for links
* build: remove -Wabi and add -Wundef
gcc >= 8 throws a warning about -Wabi (without a specific ABI version)
being ignored, while -Wundef seems more useful (as shown by the change
in config.h, which was probably an unfortunate typo)
travis.yaml should probably be updated soon, but was left out from this
change as the current images don't yet need it
* lib: unused function in valueflow
refactored out since 8c03be3212
lib/valueflow.cpp:3124:21: warning: unused function 'endTemplateArgument' [-Wunused-function]
* readme: include picojson
* make: also clean exe
This will warn for cases where searching in an associative container happens before insertion, like this:
```cpp
void f1(std::set<unsigned>& s, unsigned x) {
if (s.find(x) == s.end()) {
s.insert(x);
}
}
void f2(std::map<unsigned, unsigned>& m, unsigned x) {
if (m.find(x) == m.end()) {
m.emplace(x, 1);
} else {
m[x] = 1;
}
}
```
In the case of the map it could be written as `m[x] = 1` as it will create the key if it doesnt exist, so the extra search is not necessary.
I have this marked as `performance` as it is mostly concerning performance, but there could be a copy-paste error possibly, although I dont think thats common.
A common pattern is to have a function like similar to this:
bool isFlagSet(uint32_t f) {
return f & 0x4;
}
Warning that the function returns a non-boolean in this case is too
noisy, it would be better suited for a Misra check, so remove the
warnings in the most obvious cases.
Change the astStringVerbose() recursion to extend a string instead of
returning one. This has the benefit that for tokens where the recursion
runs deep (typically large arrays), the time savings can be substantial
(see comments on benchmarks further down).
The reason is that previously, for each token, the astString of its
operands was constructed, and then appended to this tokens astString.
This led to a lot of unnecessary string copying (and with that
allocations). Instead, by passing the string by reference, the number
of temporary strings is greatly reduced.
Another way of seeing it is that previously, the string was constructed
from end to beginning, but now it is constructed from the beginning to
end. There was no notable speedup by preallocating the entire string
using string::reserve() (at least not on Linux).
To benchmark, the changes and master were tested on Linux using the
commands:
make
time cppcheck --debug --verbose $file >/dev/null
i.e., the cppcheck binary was compiled with the settings in the
Makefile. Printing the output to screen or file will of course take
longer time.
In Trac ticket #8355 which triggered this change, an example file from the
Wine repository was attached. Running the above cppcheck on master took
24 minutes and with the changes in this commmit, took 22 seconds.
Another test made was on lib/tokenlist.cpp in the cppcheck repo, which is
more "normal" file. On that file there was no measurable time difference.
A synthetic benchmark was generated to illustrate the effects on dumping
the ast for arrays of different sizes. The generate code looked as
follows:
const int array[] = {...};
with different number of elements. The results are as follows (times are
in seconds):
N master optimized
10 0.1 0.1
100 0.1 0.1
1000 2.8 0.7
2000 19 1.8
3000 53 3.8
5000 350 10
10000 3215 38
As we can see, for small arrays, there is no time difference, but for
large arrays the time savings are substantial.
Before this fix, the code:
```
class A {
A(int, int x=3){
x;
}
};
```
Was considered OK.
But explicit keyword is still needed
I'm still new to open-source contributions, so I will gladly take advice.
This fixes simplifyUsing to remove 'typename' and 'template' from type
aliases of the form: using T3 = typename T1::template T3<T2>;
This lets the template simplifier instantiate the type alias which will
then remove the using type alias.
The crash will still happen if there is no instantiation because the
type alias will not be removed. The type alias is what cppcheck is
crashing on after the template simplifier and that still needs fixing.
* Fixed#8889 (varid on function when using trailing return type.)
Don't set varid for trailing return type.
* Add a test for #9066 (Tokenizer::setVarId: varid set for trailing return type)
* Handle 'arguments' sections in compile_commands.json
Previous code assumes 'commands' exists and ill assert if t does not.
* Correct typo checking for "arguments" rather than "commands"
* Use ostringstring rather than stringstream
* Add test deominstrating graceful degradation
* Add test for parsing "arguments" rather than "commands"
This is trying to fix the issue by fixing the ast and symbol database. First, the ast nodes will be created for the init list and the symbol database will not mark it as a scope. I am not sure if this is the correct approach as I dont really understand how the AST part works.
It did change the AST for `try {} catch (...) {}` but that is because it incorrectly treats `try {}` as an initializer list.
Improve the internal check for redundant null pointer check before
calling Token::Match() (and friends). Now, warn about code snippets like
if (a && tok && Token::Match(tok, "foo"))
Also, extend the check for the inverted case.
There is still no warning for
if (tok && a && Token::Match(tok, "foo"))
since that would require checking if a is independent of tok.
* teststring.cpp: Fix ternary syntax in tests
* stringLiteralWrite: Add tests wide character and utf16 strings
* suspiciousStringCompare: Add test with wide character string
* strPlusChar: Handle wide characters
* incorrectStringCompare: Add test with wide string
* Suspicious string compare: suggest wcscmp for wide strings
* deadStrcmp: Extend to handle wide strings
* sprintfOverlappingData: Print name of strcmp function
* Conversion of char literal to boolean, add wide character tests
* Conversion of char literal to boolean, fix ternary
Fix some crashes caused by the template simplifier generating bad code
for some instantiations.
Sorry but there are no tests because I was unable to get C-Reduce to
create examples that were not garbage code.
This only fixes the crash. It does not fix the underlying problem of
template using with templates of templates causing the use of deleted
instantiations.
temp.bufferSizeArg2 was not initialized when only bufferSizeArg1
was specified or the value was out of range. But in valueflow.cpp in
valueFlowDynamicBufferSize() it was used as if it is always initialized
and has a sane value (greater than 0).
* Add defines set by compiler options when using compilation database
sets __cplusplus and __STDC_VERSION__ based on -std and the defines for -municode, -fpie, -fPIE, -fpic and -fPIC
* Fixed merge
This limits the recursion depth as a last line of defense to avoid stack
overflows when there are really huge arrays.
See https://trac.cppcheck.net/ticket/8922
This fixes issue 8996 by improving the alias checking by using lifetime analysis. It also extends the lifetime checker to handle constructors and initializer lists for containers and arrays.
Some POSIX and Windows functions require buffers of at least some
specific size. This is now possible to configure via for example this
minsize configuration: `<minsize type="value" value="26"/>`.
The range for valid buffer size values is 1 to LLONG_MAX
(9223372036854775807)
- CLI: Save the libraries that should be loaded to a list and load them
after the std.cfg has been loaded.
- GUI: Load std.cfg (and windows.cfg / posix.cfg when applicable) before
setting other options and loading the other libraries.
In the project-file-dialog the std.cfg is searched first. If some
other library fails to load is is retried with first loading std.cfg.
- boost.cfg: Enable containers that depend on std containers.
There are important TODOs still; for instance adding CTU support using our CTU infrastructure, add handling of pointers (maybe I'll use FwdAnalysis for this), add handling of multidimensional arrays, etc..
This handles concatenated strings and characters from simplecpp.
Previously, L'c' would be preprocessed to the tokens "L" and "'c'".
cppcheck would then remove the "L" token and set "'c'" to be a wide
character literal. Now, it needs to remove the prefix instead.
When doing this, add handling of utf32 encoded literals (U) and UTF-8
encoded literals (u8).
CheckUninitVar::isMemberVariableAssignment uses argument direction "out"
now also to check for assignment when the member variable is handed over
to a function by reference.
testuninitvar.cpp: Improve tests, use a test library configuration.
CheckUninitVar::isMemberVariableAssignment uses argument direction
to check for assignment when the member variable is handed over to a
function by reference. Currently implemented for "in" direction. "out"
will be added with another commit.
lib/settings.cpp:53:7: warning: field 'removeUnusedIncludedTemplates' will be
initialized after field 'removeUnusedTemplates' [-Wreorder]
removeUnusedIncludedTemplates(false),
^
lib/settings.cpp:54:7: warning: field 'removeUnusedTemplates' will be
initialized after field 'checkConfiguration' [-Wreorder]
removeUnusedTemplates(false),
^
* std.cfg: Add further argument directions (in, out, inout).
* testlibrary.cpp: Add test for function argument direction configuration.
* std.cfg: runastyle and add some more direction configurations.
* library.h: Add documentation for function argument direction enum.
* Do not use "direction" library information for pointer arguments.
Also fix further unmatched uninitvar messages in std configuration
tests.
* std.cfg: Add more argument direction configurations.
* test/cfg/std.c: Add test for argument direction configuration.
* astutils.cpp: Only ignore pointer arguments for out/inout arguments.
* library.h: Use suggested documentation for argument direction enum.
This enhances the library configuration so the direction of function
arguments can be specified (in, out, inout).
isVariableChangedByFunctionCall() uses this information now to avoid
guessing.
If no 'alternatives' argument was specified and the `<warn/>` element
did not contain any text Cppcheck crashed because of a null pointer
access.
If there is no 'reason' and no 'alternatives argument and also no text loadFunction() returns with an error.
* template simplifier: make sure all instantiations are found and expanded in #5097
* template simplifier: check output on another test
* template simplifier: add output to another test
* template simplifier: instantiate template class when something inside class instantiated.
* template simplifier: add output to another test that now works
This uses the lifetime analysis to check when comparing pointer that point to different objects:
```cpp
int main(void)
{
int foo[10];
int bar[10];
int diff;
if(foo > bar) // Undefined Behavior
{
diff = 1;
}
return 0;
}
```
This will now warn for cases like this:
```cpp
auto& f() {
std::vector<int> x;
return x[0];
}
```
It also improves the handling of address of operator, so it can now warn across some function calls, like this:
```cpp
int& f(int& a) {
return a;
}
int* hello() {
int x = 0;
return &f(x);
}
```
Even if `ptr` is a local variable, the object `ptr->item` might be not.
So taking address of `ptr->item` is definitely not unsafe in general.
This commit fixes false positives triggered by commit
1.85-249-gf42648fe2 on the following code of sssd:
https://github.com/SSSD/sssd/blob/d409df33/src/sbus/request/sbus_request.c#L359
This reworks constStatement to find more issues. It catches issue [8827](https://trac.cppcheck.net/ticket/8827):
```cpp
extern void foo(int,const char*,int);
void f(int value)
{
foo(42,"test",42),(value&42);
}
```
It also catches from issue [8451](https://trac.cppcheck.net/ticket/8451):
```cpp
void f1(int x) {
1;
(1);
(char)1;
((char)1);
!x;
(!x);
~x;
}
```
And also:
```cpp
void f(int x) {
x;
}
```
The other examples are not caught due to incomplete AST.
Add a call to simplifyPlatformTypes() in
SymbolDatabase::setValueTypeInTokenList() to simplify return types of
library configured functions. This fixes the FN in #8141. Regression
tests are added, both for the original issue and another FN in the comments.
In order to do that, move simplifyPlatformTypes() to TokenList from Tokenizer.
This is a pure refactoring and does not change any behaviour. The code was
literally copy-pasted from one file to another and in two places
'list.front()' was changed to 'front()'.
When adding the call to simplifyPlatformTypes(), the original type of
v.size() where v is a container is changed from 'size_t' to 'std::size_t'.
Tests are updated accordingly. It can be noted that if v is declared as
'class fred : public std::vector<int> {} v', the original type of 'v.size()'
is still 'size_t' and not 'std::size_t'.
* Fixed#8962 ("(debug) Unknown type 'T'" with template typename parameter)
Only simple one parameter template functions with one function parameter
are supported.
* Added TODO test case for FIXME.
otherwise showing (with Apple LLVM version 10.0.0):
lib/settings.cpp:34:7: warning: field 'jointSuppressionReport' will be
initialized after field 'maxCtuDepth' [-Wreorder]
jointSuppressionReport(false),
* Fixed#8971 ("(debug) Unknown type 'x'." using alias in class members)
* template simplifier: partial fix for #8972
Add support for multi-token default template parameters.
* template simplifier: fix for #8971
Remove typename outside of templates.
* Fixed#8960 ("(debug) Unknown type 'x'." with alias in template class alias)
This commit adds non-template type alias support to the template
simplifier. Only relatively simple type aliases are supported at this
time. More complex types will be added later.
--debug-warnings will show unsupported type aliases.
Type alias support will be removed from the symbol database in the
future. Type alias tests have been removed from the symbol database
tests.
* Add the changes.
* Fix codacy warning.
* Fix travis warnings.
* template simplifier: fix crash on windows
Use right token when searching for template type alias to delete.
* template simplifier: fix a cppcheck warning
This has basic handling of GUI projects. But further work will be needed to handle addons etc, the plan is that we will be able to run addons from the command line soon.
The unsigned less than zero checker looked for patterns like "<= 0".
Switching to use valueflow improves the checker in a few aspects.
First, it removes false positives where instead of 0, the code is using
0L, 0U, etc. Instead of having to hard code the different variants of 0,
valueflow handles this automatically. This fixes FPs on the form
uint32_t value = 0xFUL;
void f() {
if (value < 0u)
{
value = 0u;
}
}
where 0u was previously not recognized by the checker. This fixes#8836.
Morover, it makes it possible to handle templates properly. In commit
fa076598ad, all warnings inside templates
were made inconclusive, since the checker had no idea if "0" came from
a template parameter or not.
This makes it possible to not warn for the following case which was
reported as a FP in #3233
template<int n> void foo(unsigned int x) {
if (x <= n);
}
foo<0>();
but give a warning for the following case
template<int n> void foo(unsigned int x) {
if (x <= 0);
}
Previously, both these cases gave inconclusive warnings.
Finally, it makes it possible to give warnings for the following code:
void f(unsigned x) {
int y = 0;
if (x <= y) {}
}
Also, previously, the checker for unsigned variables larger than 0, the
checker used the string of the astoperand. This meant that for code like
the following:
void f(unsigned x, unsigned y) {
if (x -y >= 0) {}
}
cppcheck would output
[unsigned-expression-positive.c] (style) Unsigned variable '-' can't be negative so it is unnecessary to test it.
using expressionString() instead gives a better error message
[unsigned-expression-positive.c] (style) Unsigned expression 'x-z' can't be negative so it is unnecessary to test it.
This will use the lifetime checker for dangling references. It will find these cases for indirectly assigned reference:
```cpp
int &foo()
{
int s = 0;
int& x = s;
return x;
}
```
This will also fix issue 510 as well:
```cpp
int &f( int k )
{
static int &r = k;
return r;
}
```
In case the XML code of a library configuration is invalid Cppcheck now additionally prints out some helpful error description like this:
"Error=XML_ERROR_MISMATCHED_ELEMENT ErrorID=16 (0x10) Line number=304: XMLElement name=noreturn"
* out of line member functions are a namespace
* template<...> and *_cast<> can't be instantiations
* refactor code to use less function parameters
* fix instantiation scopes
* use full name with namespace when available
* fallback to just matching names when full name doesn't match
* fix for CMake compile_commands.json input - director does not include trailing / which makes include directories wrong - so add it if it doesnt exist
* fix the bugfix for trailing / in the directory name of CMAKE JSON file, add also new test case to see if it works in both cases (with and without trailing /)
* revert adding accidental new line
due to equal arguments...
* iterators1 (`CheckStl::iteratorsError(const Token*, const std::string&, const std::string&)`) and
* iterators2 (`CheckStl::iteratorsError(const Token*, const Token*, const std::string&, const std::string&)`)
... produced equal messages. Equal messages were filtered-out `CppCheck::reportErr(const ErrorLogger::ErrorMessage&)`.
So the error iterators2 disapeared from the error list.
This fixes valueflow to have a value for `||` operator here:
```cpp
bool f()
{
bool a = (4 == 3); // <-- 0
bool b = (3 == 3); // <-- 1
return a || b; // <-- 1
}
```
When comparing if the shift is large enough to make the result zero, use
an unsigned long long to make sure the result fits. Also, a check that
avoids setting the value if the shift is equal to or larger than the
number of bits in the operand (this is undefined behaviour). Finally,
add a check to make sure the calculated value is not too large to store.
Add test cases to cover this.
This was detected by an MSVC warning.
valueflow.cpp(1350): warning C4334: '<<' : result of 32-bit shift implicitly
converted to 64 bits (was 64-bit shift intended?)
* use already cached name token rather than recalculating it
multiple times
* cache end of template parameters token and use it rather than
recalculating it multiple times
* remove unnecessary end of template token and name token checks
* remove function parameter that is already contained in another
parameter
* valueflow: remove unused variable known
since e4677ae640 will trigger :
lib/valueflow.cpp:506:20: warning: unused variable 'known' [-Wunused-variable]
const bool known = (parent->astOperand1()->hasKnownValue() ||
* templatesimplifier: cleanup
since 48c960f56c showing:
lib/templatesimplifier.h:279:16: warning: private field 'mTokenizer' is not used
[-Wunused-private-field]
Tokenizer *mTokenizer;
* split CheckNullPointer::arithmeticError() into
* CheckNullPointer::pointerArithmeticError() and
* CheckNullPointer::redundantConditionWarning()
* Additional errorlist entry:
```XML
<error id="nullPointerArithmeticRedundantCheck"
severity="warning"
msg="Either the condition is redundant or there is pointer arithmetic with NULL pointer."
verbose="Either the condition is redundant or there is pointer arithmetic with NULL pointer." cwe="682"/>
```
This fixes issue in:
```cpp
void f()
{
char stack[512];
RGNDATA *data;
if (data_size > sizeof (stack))
data = malloc (data_size);
else
data = (RGNDATA *)stack;
if ((char *)data != stack)
free (data); // <- data is not stack
}
```
It seems the `ProgramMemory` can't handle two known values(such as int and tok) together. So instead `ValueFlowAfterAssign` runs `ValueFlowForward` with tok values and then runs it with the other values.
* Code changes for Token::mImpl optimisation
* Added new TokenImpl optimisation
Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%.
* Several optimisations
Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it.
Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations.
Moved template removal to the end of the list as this reduces redundant iteration and saves time.
* Added tok argument to simplifyCalculations
This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list).
* Removed mention of member from another change
* Re-added and optimised some code deleted in error
Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators.
* Changed mTemplateInstantiations to a vector
This is an optimisation that makes repeated linear searches of this collection significantly faster.
Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized.
* A cleaner optimisation to removing template tokens
This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified.
* Reverted vector to list
Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members.
* Added member for template simplifier pointer
This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches.
* Turned loop inside out
This means we only have to iterate through the std::list once. std::list is very expensive to iterate through.
* Latest code from danmar and fixed optimisations
In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases.
* Added changes from danmar
Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently.
* Tokens can have many templateSimplifierPointers
* templateSimplifierPointers must be kept in sync
This makes arrays non-null in valueflow, so it can catch comparisons against null that is always true:
```cpp
void f(void) {
int buf[42];
if( buf != 0) {;} // << always true
}
```
Add a check for function arguments that can be constant:
```cpp
extern void bar(int);
void f(int x) {
bar((x & 0x01) >> 7); // function 'bar' is always called with a '0'-argument
}
```
* Implement hexadecimal float conversion in MathLib::toDoubleNumber independent of C99/C++17
* Refactor MathLib::isFloatHex and cure some false results
orbitcowboy
Paul Fultz II
IOBYTE
Daniel Marjamäki
Rikard Falkeborn
Carlo Marcelo Arenas Belón
adamjrichter
Nicodemes
Frank Zingsheim
Gary Leutheuser
amai2012
bbennetts
Márton Csordás
Oliver Stöneberg
Sebastian
Frank Zingsheim
Pavol Misik
amai2012
Carlo Marcelo Arenas Belón
PKEuS
ivangalkin
Kamil Dudka
Simon Martin
Lauri Nurmi
Robert Reif
juremenart
practicalswift
Oliver Schode
rebnridgway
PKEuS
Igor
Colomban Wendling