Commit Graph

365 Commits

Author SHA1 Message Date
Rikard Falkeborn 5c061c1c12 Set correct type and size of string and char literals (#2275)
* Set correct type and size of string and char literals

Use that string and char literal tokens store the prefix. This makes
it possible to distinghuish between different type of string literals
(i.e., utf8 encoded strings, utf16, wide strings, etc) which have
different type.

When the tokens holding the string and character values have the correct
type, it is possible to improve Token::getStrSize() to give the correct
result for all string types. Previously, it would return the number of
characters in the string, i.e., it would give the wrong size unless
the type of the string was char*.

Since strings now can have different size (in number of bytes) and
length (in number of elements), add a new helper function that returns
the number of characters. Checkers have been updated to use the correct
functions.

Having the size makes it possible to find more problems with prefixed
strings, and to reduce false positives, for example in the buffer
overflow checker.

Also, improve the stringLiteralWrite error message to also print the
prefix of the string (if there is one).

* Add comment and update string length
2019-10-20 07:11:57 +02:00
Rikard Falkeborn 297360920a Keep prefix in string and char literals (#2272)
Keeping the prefix in the token allows cppcheck to print the correct
string and char literals in debug and error messages.

To achieve this, move some of the helper functions from token.cpp to
utils.h so that checks that look at string and char literals can reuse
them. This is a large part of this commit.

Note that the only user visible change is that when string and char
literals are printed in error messages, the prefix is now included.

For example:

	int f() {
		return test.substr( 0 , 4 ) == U"Hello" ? 0 : 1 ;
	};

now prints U"Hello" instead of "Hello" in the error message.
2019-10-16 11:41:33 +02:00
IOBYTE 3e17c24dd8 fix syntax error on template operator (#2225) 2019-10-03 12:26:45 +02:00
Oliver Stöneberg eac040a00b Various clang-tidy fixes (#2192)
* use range loops

* removed redundant string initializations

* use nullptr

* use proper boolean false

* removed unnecessary continue from end of loop

* removed unnecessary c_str() usage

* use emplace_back()

* removed redundant void arguments
2019-09-25 15:25:19 +02:00
Paul Fultz II ad8abdb0c3 Add impossible values to ValueFlow (#2186)
* Add impossible category

* Replace values

* Try to adjust known values

* Add ! for impossible values

* Add impossible with possible values

* Remove contradictions

* Add values when the branch is not dead

* Only copy possible values

* Dont bail on while loops

* Load std lib in valueflow

* Check for function calls

* Fix stl errors

* Fix incorrect impossible check

* Fix heap-after-use error

* Remove impossible values when they are lowered

* Show the bound and remove overlaps

* Infer conditions

* Dont push pointer values through dynamic_cast

* Add test for dynamic_cast issue

* Add shifttoomanybits test

* Add test for div by zero

* Add a test for issue 9315

* Dont make impossible value inconclusive

* Fix FP with shift operator

* Improve handleKnownValuesInLoop for impossible values

* Fix cppcheck warning

* Fix impossible values for ctu

* Bailout for streams

* Check equality conditions

* Fix overflows

* Add regression test for 9332

* Remove duplicate conditions

* Skip impossible values for invalid value

* Check for null

* Rename bound to range

* Formatting
2019-09-20 15:06:37 +02:00
IOBYTE 8c5cf8c029 Fixed #9046 syntaxError (#2180)
* Fixed #9046 syntaxError

* fix another syntax error

* fix some more syntax errors
2019-09-20 12:35:01 +02:00
Oliver Stöneberg de9f489b08 use range loops / constness (#2181)
* use range loops / constness

* platform.cpp: avoid shadowed variable
2019-09-19 20:29:33 +02:00
Paul Fultz II dc0b3527ad Fix issue 9311: False positive duplicateCondition "same if condition" with pointer inside array of struct (#2166)
* Check for typeOf through an array

* Handle array constructors

* Format

* Fix compile error on gcc 4.8
2019-09-10 19:41:35 +02:00
IOBYTE e5220bdf0c make ellipsis ... a single token (#2143)
* make ellipsis ... a single token

Using cppcheck -E to preprocess code with ellipsis produces output that
can't be compiled because ... is split into 3 tokens.

* try to fix addon
2019-09-04 08:07:30 +02:00
Daniel Marjamäki 9d26be8380 Fixed #5259 (Improve check: Uninitialized variable not reported when used in array initialization) 2019-08-24 14:43:35 +02:00
Paul Fultz II 5c488b9519 Fix issue 9190: FP uninitvar for struct member (#2112)
* Fix issue 9190: FP uninitvar for struct member

* Add more test cases

* Fix false negative
2019-08-24 11:27:47 +02:00
Daniel Marjamäki 2d9a131817 Refactoring: Rename variables. Do not use leading _. Renamed 'col' to 'column' 2019-08-18 12:19:05 +02:00
IOBYTE 5fc5e3728d template simplifier: refactor TemplateSimplifier::TokenAndName into a… (#2073)
* template simplifier: refactor TemplateSimplifier::TokenAndName into a class

assert when more than one family flag is set

* fix function parameter names
2019-08-10 08:42:12 +02:00
IOBYTE 7ebc9d1b5f Fix #9249 (Syntax error on valid C++) (#2062) 2019-08-04 10:24:44 +02:00
rebnridgway d2b9c1f15a Fix some bugs in new Scope Calculation code (#2060)
* Fix for too much information in scope name

When the scope calculation encounters code such as 
"friend class X::Y;"
or
"template<> class X<void> {"
it will now reset the additional name component of the scope that is about to be opened.

* Made sure new scope name is reset after being used
2019-08-02 19:55:08 +02:00
orbitcowboy a9bdf99e07 std.cfg: Added *experimental* support for math constants liken M_PI. These are *NOT* standard, but they are supported by GCC/Clang and VS. 2019-07-31 11:11:01 +02:00
rebnridgway e629f9a90f Calculate token scopes in advance rather than as the tokenlist is iterated (#2038)
* Removed redundant scope calculation

* Add scope propagation code to insertToken

* Add relevant scope code to Token class

* Add code to calculate the scope of Tokens

* Add calculateScopes method to class

* Add missing include for shared_ptr
2019-07-31 09:19:27 +02:00
Daniel Marjamäki fb7f18ddea ValueFlow: fix false detection of duplicate values 2019-07-24 12:30:33 +02:00
Daniel Marjamäki 0be78bbde6 Rename fileInd to fileIndex 2019-07-17 16:28:47 +02:00
Daniel Marjamäki b30ef3d050 Fix wrong 'nonneg' 2019-07-14 17:31:26 +02:00
Daniel Marjamäki 0014fe880e Mark members and arguments with 'nonneg' 2019-07-14 12:22:33 +02:00
Daniel Marjamäki 316537a837 Avoid 'unsigned' for int variables 2019-07-13 16:06:24 +02:00
Daniel Marjamäki cff462c6ef Token deleteNext/deletePrevious parameters 2019-07-13 15:47:53 +02:00
Daniel Marjamäki 68cc7516a1 Annotations: Add annotation __cppcheck_in_range__(low,high) 2019-07-12 11:09:54 +02:00
versat 57d44f1362 astyle formatting
[ci skip]
2019-07-05 14:00:59 +02:00
Paul Fultz II e0ced1c415 Parse lambdas as functions (#1955)
* Parse lambdas as functions

* Fix issue with missing paren

* Fix error when parsing non-existent args

* Remove unused function variable
2019-07-05 12:30:42 +02:00
Daniel Marjamäki d9a9743b8a Revert "Calculate token scopes in advance rather than as the tokenlist is iterated (#1882)"
This reverts commit 0d7836f3a0.
2019-06-30 09:12:18 +02:00
rebnridgway 0d7836f3a0 Calculate token scopes in advance rather than as the tokenlist is iterated (#1882)
* Added scopeinfo member to token class

Moved ScopeInfo2 declaration here as well because that's where it needs to be now.

* Added scopeinfo accessors and declaration to class

* Add new method for calculating scopes

This replaces the methods in the TemplateSimplifier which calculate the current scope as the token list is iterated.  The old method required checking if the scope had changed for every token multiple times (for multiple iterations), which was surprisingly costly.  Calculating scopes in advance like this decreases runtime on a worst-case file by around thirty percent.
ScopeInfo objects are disposed of when the TemplateSimplification is done as they are not used later.

* Add calculateScopes method to header

* Removed code that calculated current scope

This has been replaced by code that calculates the scopes up front and stores them with each token, which is much faster.

* Fixed compile errors from extra parentheses

* Added missing code to fix memory leak

* Added code to actually clean up ScopeInfo structs

* Tidy up a dodgy for loop

* Convert argument to const ref

* Calculate missing scopes

As the templatesimplificator expands templates and does multiple passes it needs to make sure all scopes are calculated.

* Remove copying the scope to the next token

This is now done properly when scopes are calculated.

* Remove call to calculateScopes

This is now done by the TemplateSimplifier.

* Recalculate scopes for every pass of simplifyTemplates

* Add code to calculate extra scopes as they are added

I thought that this might be useful for calculating scopes when Tokens are created, but as there are several ways of creating Tokens that don't guarantee that they are placed in a list it is easier to just calculate scopes when you know you have a list and when you know you're adding to a list.

* Fix several bugs and poorly designed code

Remove the global scopes collection, and clean them up instead by iterating through the tokenlist to find them.  This means scopes can be calculated by functions in the Token class as well as in the Tokenizer class without leaking the scope object.
Fix a couple of bugs in the calculateScopes method and make it more efficient.

* Remove unnecessary calls to calculateScopes

* Move brace to correct position

Calculating scopes during insertToken only needs to happen if we created a new Token.

* Handle 'using namespace' declarations separately

This fixes a bug caused by a statement matching 'struct B < 0 > ;'

* Fix argument name mismatch

* Actually use newScopeInfo when inserting Token

* Switch to using shared_ptr to hold scopeInfos

This means ScopeInfo2 objects get properly cleaned up when they are no longer needed.

* Change ScopeInfo member to be a shared_ptr

* Update code to use shared_ptr

* Add missing include for shared_ptr

* Remove unnecessary cleanup code

This has been replaced by shared_ptr for ScopeInfo2 objects
2019-06-29 14:34:22 +02:00
Daniel Marjamäki beea7fa8d2 Token::index(): Created Token member that indicates the Token position in the token list. It can be used to quickly check if tok1 precedes tok2. 2019-06-21 22:16:23 +02:00
orbitcowboy 821903e2e1 Running astyle [ci skip] 2019-06-02 13:29:20 +02:00
Paul Fultz II 6ae7be0f53 Fix FP with lifetime containers (#1865) 2019-06-02 10:14:48 +02:00
adamjrichter 47ce998e6e lib: Separate statements of the form "assert(a && b)" into "assert(a)" and "assert(b)", for more precise diagnostics. (#1825) 2019-05-07 10:28:31 +02:00
Paul Fultz II 091f4bcf8d Add check for unnecessary search before insertion
This will warn for cases where searching in an associative container happens before insertion, like this:

```cpp
void f1(std::set<unsigned>& s, unsigned x) {
    if (s.find(x) == s.end()) {
        s.insert(x);
    }
}

void f2(std::map<unsigned, unsigned>& m, unsigned x) {
    if (m.find(x) == m.end()) {
        m.emplace(x, 1);
    } else {
        m[x] = 1;
    }
}
```

In the case of the map it could be written as `m[x] = 1` as it will create the key if it doesnt exist, so the extra search is not necessary.

I have this marked as `performance` as it is mostly concerning performance, but there could be a copy-paste error possibly, although I dont think thats common.
2019-05-02 11:04:23 +02:00
Rikard Falkeborn c7d7f8738c Optimize astStringVerbose() for large arrays (#1815)
Change the astStringVerbose() recursion to extend a string instead of
returning one. This has the benefit that for tokens where the recursion
runs deep (typically large arrays), the time savings can be substantial
(see comments on benchmarks further down).

The reason is that previously, for each token, the astString of its
operands was constructed, and then appended to this tokens astString.
This led to a lot of unnecessary string copying (and with that
allocations). Instead, by passing the string by reference, the number
of temporary strings is greatly reduced.

Another way of seeing it is that previously, the string was constructed
from end to beginning, but now it is constructed from the beginning to
end. There was no notable speedup by preallocating the entire string
using string::reserve() (at least not on Linux).

To benchmark, the changes and master were tested on Linux using the
commands:

	make
	time cppcheck --debug --verbose $file >/dev/null

i.e., the cppcheck binary was compiled with the settings in the
Makefile. Printing the output to screen or file will of course take
longer time.

In Trac ticket #8355 which triggered this change, an example file from the
Wine repository was attached. Running the above cppcheck on master took
24 minutes and with the changes in this commmit, took 22 seconds.

Another test made was on lib/tokenlist.cpp in the cppcheck repo, which is
more "normal" file. On that file there was no measurable time difference.

A synthetic benchmark was generated to illustrate the effects on dumping
the ast for arrays of different sizes. The generate code looked as
follows:

	const int array[] = {...};

with different number of elements. The results are as follows (times are
in seconds):

	N	master optimized
	10	0.1    0.1
	100	0.1    0.1
	1000	2.8    0.7
	2000	19     1.8
	3000	53     3.8
	5000	350    10
	10000	3215   38

As we can see, for small arrays, there is no time difference, but for
large arrays the time savings are substantial.
2019-04-30 13:35:48 +02:00
Daniel Marjamäki b1b5b27b4e addons: write column in error message(s) 2019-04-13 10:22:13 +02:00
Rikard Falkeborn 794f65bac1 Handle prefixed strings and characters in Token (#1742)
This makes it possible to call getStrLength() and similar functions
before the tokenizer is called.
2019-03-18 06:18:25 +01:00
Daniel Marjamäki 92f4113b59 Array index: Checking array index out of bounds for dynamic buffers 2019-03-17 13:09:15 +01:00
Daniel Marjamäki df8cfe2fc6 Travis: Fix Cppcheck warning 2019-03-10 12:24:41 +01:00
Rikard Falkeborn 6a3dd9a185 Handle concatenated string and char literals
This handles concatenated strings and characters from simplecpp.
Previously, L'c' would be preprocessed to the tokens  "L" and "'c'".
cppcheck would then remove the "L" token and set "'c'" to be a wide
character literal. Now, it needs to remove the prefix instead.

When doing this, add handling of utf32 encoded literals (U) and UTF-8
encoded literals (u8).
2019-03-10 10:38:50 +01:00
Daniel Marjamäki bd7790fd8c Update copyright year 2019-02-09 07:24:06 +01:00
Daniel Marjamäki a26287f491 Fix Token::findExpressionStartEndTokens() regression 2019-01-20 13:23:19 +01:00
Daniel Marjamäki 8da4e31c42 Fixed #8941 (False Positive: Variable 'f' is assigned a value that is never used.) 2019-01-20 13:20:23 +01:00
Daniel Marjamäki b97b3b7ef8 astyle formatting
[ci skip]
2018-12-21 13:54:59 +01:00
rebnridgway 431d068339 Several fairly significant optimisations (#1518)
* Code changes for Token::mImpl optimisation

* Added new TokenImpl optimisation

Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation.  In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%.

* Several optimisations

Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it.
Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations.
Moved template removal to the end of the list as this reduces redundant iteration and saves time.

* Added tok argument to simplifyCalculations

This means callers can avoid unnecessary work if they know which tokens have already been simplified.  Passing nullptr indicates the original behaviour (starting from the front of the list).

* Removed mention of member from another change

* Re-added and optimised some code deleted in error

Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches.  Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators.

* Changed mTemplateInstantiations to a vector

This is an optimisation that makes repeated linear searches of this collection significantly faster. 
Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized.

* A cleaner optimisation to removing template tokens

This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified.

* Reverted vector to list

Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members.

* Added member for template simplifier pointer

This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches.

* Turned loop inside out

This means we only have to iterate through the std::list once.  std::list is very expensive to iterate through.

* Latest code from danmar and fixed optimisations

In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list.  Previous optimisations to this weren't very effective and broke some edge cases.

* Added changes from danmar

Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently.

* Tokens can have many templateSimplifierPointers

* templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
Daniel Marjamäki 37416010ef Unused value: Fix false positive (ast, {}) 2018-12-14 18:56:09 +01:00
Daniel Marjamäki 39c04b650d astyle formatting
[ci skip]
2018-11-14 19:10:52 +01:00
Paul Fultz II 54453c5802 Fix FP when copying pointer to string (#1479) 2018-11-14 06:59:25 +01:00
Paul Fultz II e839001f3d Fix warnings (#1478) 2018-11-14 06:14:04 +01:00
IOBYTE 17a8a4898d Fix another template simplifier namespace bug. (#1467)
* Fix another template simplifier namespace bug.

* Add missing forward declarations caused by token bug.
2018-11-07 21:25:42 +01:00
Daniel Marjamäki 58b21e3071 Refactoring: Reuse Token::expressionString() logic for finding right-most leaf in tree 2018-10-18 12:09:55 +02:00