Keeping the prefix in the token allows cppcheck to print the correct
string and char literals in debug and error messages.
To achieve this, move some of the helper functions from token.cpp to
utils.h so that checks that look at string and char literals can reuse
them. This is a large part of this commit.
Note that the only user visible change is that when string and char
literals are printed in error messages, the prefix is now included.
For example:
int f() {
return test.substr( 0 , 4 ) == U"Hello" ? 0 : 1 ;
};
now prints U"Hello" instead of "Hello" in the error message.
`__typeof__` is just an alternative keyword for `typeof`, see
https://gcc.gnu.org/onlinedocs/gcc/Typeof.html
Since `typeof` is handled in several checkers it makes sense to define
`__typeof__` as `typeof`.
Tokenizer::simplifyTypedef(): Use `typeof` instead of `__typeof__` to
be consistent with the rest of the code.
* Better handle const/noexcept methods/conversion operator
const or noexcept in a method / (conversion) operator definition were
badly parsed, ending in a bad ast.
This patch tries to make it better, at least making the ast less bad,
so as to avoid errors in later checks.
* Fix parsing of some operator
It is still very broken, but at least, it does not fail.
Here is the previous error:
```
TestSimplifyTypedef::simplifyTypedef129
terminate called after throwing an instance of 'InternalError'
Program received signal SIGABRT, Aborted.
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
51 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1 0x00007ffff612a801 in __GI_abort () at abort.c:79
#2 0x00007ffff6b1d957 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#3 0x00007ffff6b23ab6 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#4 0x00007ffff6b23af1 in std::terminate() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#5 0x00007ffff6b23d24 in __cxa_throw () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#6 0x0000555556366bf8 in Tokenizer::cppcheckError (this=0x7fffffffc2d0, tok=0x607000006760) at ../lib/tokenize.cpp:8721
#7 0x000055555636a4bb in Tokenizer::validate (this=0x7fffffffc2d0) at ../lib/tokenize.cpp:9154
#8 0x000055555633e3aa in Tokenizer::simplifyTokenList1 (this=0x7fffffffc2d0, FileName=0x603000002d50 "test.cpp") at ../lib/tokenize.cpp:4477
#9 0x00005555563223ca in Tokenizer::simplifyTokens1 (this=0x7fffffffc2d0, configuration="") at ../lib/tokenize.cpp:2286
#10 0x00005555563235c8 in Tokenizer::tokenize (this=0x7fffffffc2d0, code=..., FileName=0x555556fda9a0 "test.cpp", configuration="") at ../lib/tokenize.cpp:2345
#11 0x00005555569410ea in TestSimplifyTypedef::tok[abi:cxx11](char const*, bool, cppcheck::Platform::PlatformType, bool) (this=0x555557728580 <(anonymous namespace)::instance_TestSimplifyTypedef>,
code=0x7fffffffcb70 "class c {\n typedef char foo[4];\n foo _a;\n constexpr operator foo &() const noexcept { return _a; }\n};", simplify=false, type=cppcheck::Platform::Native, debugwarnings=true) at ../test/testsimplifytypedef.cpp:192
#12 0x000055555697239e in TestSimplifyTypedef::simplifyTypedef129 (this=0x555557728580 <(anonymous namespace)::instance_TestSimplifyTypedef>) at ../test/testsimplifytypedef.cpp:2599
#13 0x000055555694092c in TestSimplifyTypedef::run (this=0x555557728580 <(anonymous namespace)::instance_TestSimplifyTypedef>) at ../test/testsimplifytypedef.cpp:167
#14 0x00005555569cab84 in TestFixture::run (this=0x555557728580 <(anonymous namespace)::instance_TestSimplifyTypedef>, str="simplifyTypedef129") at ../test/testsuite.cpp:306
#15 0x00005555569cb445 in TestFixture::runTests (args=...) at ../test/testsuite.cpp:329
#16 0x000055555687bdfb in main (argc=2, argv=0x7fffffffd988) at ../test/testrunner.cpp:44
```
* Replace some ASSERT_EQUALS with TODO_ASSERT_EQUALS when the actual result is still wrong
* Remove invalid code from simplifyTypedef129
* Properly skip parentheses
Previously, cppcheck discarded the `extern "C"` specifier. This patch modifies cppcheck to parse each as a Scope in the symbol database, then uses that scope to avoid false positives when making recommendations about changing a function argument to be a reference (since variable references is a C++ feature, unavailable in C, and thus unavailable in `extern "C"`).
I fixed the AST enough to pass testrunner but I don't believe it is
correct.
This code:
void Foo4(int&&b);
has this AST:
( 'void'
|-Foo4
`-&& 'bool'
|-int
`-b 'signed int'
but I don't believe && should have `bool`.
* use range loops
* removed redundant string initializations
* use nullptr
* use proper boolean false
* removed unnecessary continue from end of loop
* removed unnecessary c_str() usage
* use emplace_back()
* removed redundant void arguments
* make ellipsis ... a single token
Using cppcheck -E to preprocess code with ellipsis produces output that
can't be compiled because ... is split into 3 tokens.
* try to fix addon
* Add missing Qt macros
Add two Qt macros that were missing
* Fix issue with __declspec and final
This change is a bit naive but it fixes the issues I was having when combining __declspec(dllexport) and final classes. Without the fix I get errors along the line of "The code 'class x final :' is not handled. You can use -I or --include to add handling of this code. "
* Fix for too much information in scope name
When the scope calculation encounters code such as
"friend class X::Y;"
or
"template<> class X<void> {"
it will now reset the additional name component of the scope that is about to be opened.
* Made sure new scope name is reset after being used
* Removed redundant scope calculation
* Add scope propagation code to insertToken
* Add relevant scope code to Token class
* Add code to calculate the scope of Tokens
* Add calculateScopes method to class
* Add missing include for shared_ptr
* Fix#9225 (Crash on valid C++14 code)
This only fixes the crash. Specialization of nested templates is still
broken.
* fix cppcheck warnings
* fixed another cppcheck warning
* Fix adding unescaped slash token when splitting gcc case range.
Construction like case '!'...'~' converted to a list of separate case
tokens. When slas '\' symbol appears as a part of this list it was added
"as is", but it should be escaped like '\\' to be valid c++ code.
* Add test for switch-case range with slash
* Refactor Tokenizer::simplifyUsing to use continue to reduce indentation
added function findTemplateDeclarationEnd to skip template declarations
to reduce duplicate code
* fix travis build
* Partial fix for #9191 (simplifyTypedef: Problem when namespace is used)
This fixes simplifyUsing which has the same problem as simplifyTypedef.
simplifyUsing was designed to support using namespace but it was never
implemented. The changes are minor to add it.
simplifyTypedef requires much more work to support using namespace.
* reduce scope of variable
* make idx const
* Added scopeinfo member to token class
Moved ScopeInfo2 declaration here as well because that's where it needs to be now.
* Added scopeinfo accessors and declaration to class
* Add new method for calculating scopes
This replaces the methods in the TemplateSimplifier which calculate the current scope as the token list is iterated. The old method required checking if the scope had changed for every token multiple times (for multiple iterations), which was surprisingly costly. Calculating scopes in advance like this decreases runtime on a worst-case file by around thirty percent.
ScopeInfo objects are disposed of when the TemplateSimplification is done as they are not used later.
* Add calculateScopes method to header
* Removed code that calculated current scope
This has been replaced by code that calculates the scopes up front and stores them with each token, which is much faster.
* Fixed compile errors from extra parentheses
* Added missing code to fix memory leak
* Added code to actually clean up ScopeInfo structs
* Tidy up a dodgy for loop
* Convert argument to const ref
* Calculate missing scopes
As the templatesimplificator expands templates and does multiple passes it needs to make sure all scopes are calculated.
* Remove copying the scope to the next token
This is now done properly when scopes are calculated.
* Remove call to calculateScopes
This is now done by the TemplateSimplifier.
* Recalculate scopes for every pass of simplifyTemplates
* Add code to calculate extra scopes as they are added
I thought that this might be useful for calculating scopes when Tokens are created, but as there are several ways of creating Tokens that don't guarantee that they are placed in a list it is easier to just calculate scopes when you know you have a list and when you know you're adding to a list.
* Fix several bugs and poorly designed code
Remove the global scopes collection, and clean them up instead by iterating through the tokenlist to find them. This means scopes can be calculated by functions in the Token class as well as in the Tokenizer class without leaking the scope object.
Fix a couple of bugs in the calculateScopes method and make it more efficient.
* Remove unnecessary calls to calculateScopes
* Move brace to correct position
Calculating scopes during insertToken only needs to happen if we created a new Token.
* Handle 'using namespace' declarations separately
This fixes a bug caused by a statement matching 'struct B < 0 > ;'
* Fix argument name mismatch
* Actually use newScopeInfo when inserting Token
* Switch to using shared_ptr to hold scopeInfos
This means ScopeInfo2 objects get properly cleaned up when they are no longer needed.
* Change ScopeInfo member to be a shared_ptr
* Update code to use shared_ptr
* Add missing include for shared_ptr
* Remove unnecessary cleanup code
This has been replaced by shared_ptr for ScopeInfo2 objects
* fix adding instantiation of first argument to an instantiation
* add support for function pointer template variables
* fix more cases where templates ending in ">>" are changed to end in "> >"
* fix travis build
* standard types can't be a template parameter name
* remove redundant level == 0 checks
* fix lambda in template variable
* fix a test
* Fix issue 8890: AST broken calling member function from templated base class
* Format
* Check for double bracket
* Add test to createLinks2
* Remove extra test
* Reduce test case for links
This fixes simplifyUsing to remove 'typename' and 'template' from type
aliases of the form: using T3 = typename T1::template T3<T2>;
This lets the template simplifier instantiate the type alias which will
then remove the using type alias.
The crash will still happen if there is no instantiation because the
type alias will not be removed. The type alias is what cppcheck is
crashing on after the template simplifier and that still needs fixing.
* Fixed#8889 (varid on function when using trailing return type.)
Don't set varid for trailing return type.
* Add a test for #9066 (Tokenizer::setVarId: varid set for trailing return type)
This handles concatenated strings and characters from simplecpp.
Previously, L'c' would be preprocessed to the tokens "L" and "'c'".
cppcheck would then remove the "L" token and set "'c'" to be a wide
character literal. Now, it needs to remove the prefix instead.
When doing this, add handling of utf32 encoded literals (U) and UTF-8
encoded literals (u8).
Add a call to simplifyPlatformTypes() in
SymbolDatabase::setValueTypeInTokenList() to simplify return types of
library configured functions. This fixes the FN in #8141. Regression
tests are added, both for the original issue and another FN in the comments.
In order to do that, move simplifyPlatformTypes() to TokenList from Tokenizer.
This is a pure refactoring and does not change any behaviour. The code was
literally copy-pasted from one file to another and in two places
'list.front()' was changed to 'front()'.
When adding the call to simplifyPlatformTypes(), the original type of
v.size() where v is a container is changed from 'size_t' to 'std::size_t'.
Tests are updated accordingly. It can be noted that if v is declared as
'class fred : public std::vector<int> {} v', the original type of 'v.size()'
is still 'size_t' and not 'std::size_t'.
* Fixed#8960 ("(debug) Unknown type 'x'." with alias in template class alias)
This commit adds non-template type alias support to the template
simplifier. Only relatively simple type aliases are supported at this
time. More complex types will be added later.
--debug-warnings will show unsupported type aliases.
Type alias support will be removed from the symbol database in the
future. Type alias tests have been removed from the symbol database
tests.
* Add the changes.
* Fix codacy warning.
* Fix travis warnings.
* Fixed#8683 (Using deleted token with multiple template instantiations.)
* Fixed#8321 (heap use after free: templatesimplifier)
* Add a flag to Token indicating that it has a pointer to it.
* Run dmake
* Fix one source of list pointers to deleted tokens.
Refactor TemplateSimplifier class to get access to template lists.
Remove many function parameters now that they are class variables.
Fix one source of list pointers to deleted tokens.
Add tests with no output to catch crashes.
* Run dmake again.
* Make 2 more functions private.
* Make requested changes.
* Missed one change request.
* Use TokenList rather than Tokenizer.
* Move TokenAndName constructor to cpp file so token.h is not needed in header file.
* Allow syntax "typedef 'typename' unsigned 'typename' (functon_name)()" for some old libraries
https://trac.cppcheck.net/ticket/7792
* Unnecessary check of "tokOffset", match-function already checks if it is null
* add testcase ticket #7792: simplifyTypedef124
* Added declaration for deletePrevious function
* Added definition for deletePrevious function
* Fixed crash from deleteThis invalidating pointers
The crash was caused by deleteThis() invalidating the pointer to a constant variable usage. This happened when a usage followed an assignment. This fixes bug #8579.
* Added tokensFront to match tokensBack
This means deletePrevious can set the list's front if necessary.
* Initialised tokensFront in appropriate places
* Switched to using default Token constructor
* Switched to using Token default constructor
* Switched to using default constructor for Token
* Added missing argument to Token constructor
* Changed to use default constructor for Tokens
* Switched to using default constructor for Tokens
* Switched to using default constructor for Token
* Added new test for deleting front Token
Also made sure to use the correct constructor for Token in other tests.
* Syntax error
* Replaced tokensFront and tokensBack with a struct
This decreases the size of the Token class for performance purposes.
* Replaced tokensFront and tokensBack with a struct
* Added tokensFrontBack to destructor
* Reworked to use TokensBackFront struct
Also ran astyle.
* Reworked to use TokenList's TokensFrontBack member
* Reworked to use TokensFrontBack struct
* Reworked to use TokensFrontBack struct
* Reworked to work with TokensFrontBack struct
* Removed unnecessary scope operator
* Added missing parentheses
* Fixed syntax error
* Removed unnecessary constructor
* Default constructor now 0-initialises everything
This is safer for not using a temporary TokensFrontBack object, and doesn't use delegating constructors which aren't supported yet.
* Fixed unsafe null check
* Added missing explicit keyword
* Fixing stylistic nits
Removed default constructor as it has been superseded by the single-argument constructor with a default argument value.
Renamed listEnds to tokensFrontBack.
Fixed if statement that was supposed to be adding safety but would actually cause a crash if tokensFrontBack was null.
* Fixing stylistic nits
Removed default constructor and replaced it with a single-argument constructor with a default value.
* Fixing stylistic nits
Renamed _listEnds to _tokensFrontBack.
* Fixing stylistic nits
Renamed _listEnds to _tokensFrontBack.
* Remove duplicate namespace aliases so they don't produce syntax errors.
DACA2 results showed new SymbolDatabase syntax errors when duplicate
namespace aliases were simplified improperly. The solution is to remove
them in the tokenizer when found.
* Add tests for deleting namespace aliases at end of token list.
* Use eraseTokens to delete multiple tokens at once.
* Optimised simplifyKnownVariables
Changed the check for references to constant variables so instead of iterating through all tokens looking for references (which is very slow for large files), usages of each known variable are recorded so each usage can be checked for whether or not it is a reference. Just checking known usages is a lot quicker than checking through all tokens.
* Fixed test error caused by incorrect code
* Fixes to constant variable simplification optimisation
Indexing variables by varId is easier than by Token pointer, but means we also have to store a reference to the constant variable Token in another collection.
Switched to using unordered_map over map as it has slightly better find performance.
* Fixed incorrect simplification behaviour
This should remove constant variables correctly and efficiently. Requires additional functions, Token::deleteThisInPlace() and TokenList::front(Token*).
* Added setter for TokenList::first
This allows code that adds and removes Tokens from the list to do so without copying nodes into other nodes, which sometimes create difficulties.
* Added deleteThisInPlace function
This allows a token to delete itself without invalidating pointers to the node after it. If this token is the first or last in a list the calling code will have to remember to change the list's front or back.
* Added declaration for deleteThisInPlace()
* Removed premature MatchCompiler optimisation
* Added and removed some functions
Added declaration for deleteToken(Token*) which deletes a single token from the list
Removed public front(Token*) because it broke encapsulation
* Implemented deleteToken(Token*) function
* Removed 'delete this' from deleteThisInPlace
* Switched to using safer function
TokenList::deleteToken is better than calling straight into Token::deleteThisInPlace because it doesn't call delete this and doesn't break the TokenList's encapsulation.
* Replace constant variables in reverse order
This fixes the problem where you have two constant value assignment statements in a row. Replacing and deleting them in reverse order means we avoid the problem of deleteThis() potentially invalidating the pointer to the start of the next assignment statement
* Removed unneeded and unsafe deleteThisInPlace
* Removed unneeded and unsafe deleteThisInPlace
* Removed unneeded deleteToken
* Removed unneeded deleteToken
* Removed extra whitespace
* Don't remove the volatile keyword so we can properly overload functions.
I fixed all the checks that had tests that use volatile. There will
probably be more changes needed due to lack of test coverage for
volatile in some checks.
* Fix unused private function warning.
* Add support for namespace aliases and C++17 nested namespaces.
These are implemented as tokenizer simplifications so changes are not
needed to the tokenizer and symbol database.
* Fix codacy warning.
In 9cea2d6df, simplifications were removed for a number of functions
which should instead be handled with configurations. The commit did
however not update the description of the function, do this now.
Also sin() and sinh() and their float and long double versions were
missing from the comment so add these as well.
Remove TEXT() macro from windows.cfg and handle it internally where it
can be correctly simplified (Ansi vs. Unicode).
Also add handling of _TEXT() macro which is just a synonym for _T().
Add tests to verify correct function and macro simplification.
* Fix#8382 (Syntax error when scanning code with template and attribute)
This commit only addresses #8382. There are issues concerning which
versions of C++ should be supported and also generic C++ 14 attribute
support which can be revisited later.
* Remove all C++ style attributes.
Remove all C++ style attributes when C++ version is 11 or greater.
Rename simplify function to simplifyCPPAttributes.
Handle more cases of roreturn function attribute.
The following snippet triggerd the error:
template<typename DerivedT>
template<typename T>
auto ComposableParserImpl<DerivedT>::operator|( T const &other ) const -> Parser {
return Parser() | static_cast<DerivedT const &>( *this ) | other;
}
Whenever simplifyFunctionParameters was called on a templated class'
templated member function (and probably any nested template), the
tokenizer would recognise it as a syntax error, assuming that return
type *must* come after a template<> token.
* Fixes issue with case inside switch that is not a compound statement was treated as garbage
This fixes an issue with the check for case keywords outside of switch
detection that would treat a case statement inside a switch that is not
a compound statement as garbage, but this is perfectly valid C++. This
construct is used in several libraries, i.e. Google Test.
* Tweak check and handle missing semicolon
Tweaks the check with feedback from danmar.
Handle the case where there is no semicolon and document it with a unit
test.
* Add support for simple c++ 11 type ailases like: using INT = int;
Only types supported by ValueType are supported. Complex types like
function pointers are not supported. Template type aliases are not
supported.
* Fix crash when type in using type alias is simplified away.
This fixes a crash when size_t is replaced with unsigned long in: using
size_t = unsigned long; by the tokenizer.
This does not fix the problem where Tokenizer::simplifyPlatformTypes()
simplifies away size_t in other cases. This is only a problem when the
new type is different from the platform type.
* fix#8284: False positive: "Label 'class' is not used." for anonymous C++ class
Add support for annonymous derived structures and classes.
* Fix travis build (use findsimplematch).
* Fix bug in simplifyLabelsCaseDefault which was inserting ; in wrong place.
* fix for correct parsing of embedded PL/SQL blocks (Oracle Pro*C)
* enforce SQL block end at the end of nearest outer C block, when
appropriate terminator is not found
* added check for ; at the end of END-EXEC and made SQL block detection
more readable
All issues were found with PVS-Studio:
V560 A part of conditional expression is always true: tok. astutils.cpp 407
V560 A part of conditional expression is always true: size > 0. checkbufferoverrun.cpp 709
V547 Expression 'secondTrue' is always true. checkcondition.cpp 1013
V547 Expression 'firstTrue' is always true. checkcondition.cpp 1020
V560 A part of conditional expression is always true: !scan. checkio.cpp 1036
V560 A part of conditional expression is always true: scope->function. checknullpointer.cpp 395
V560 A part of conditional expression is always true: tok2. checkstl.cpp 268
V560 A part of conditional expression is always true: par. tokenize.cpp 9440
V547 Expression '!erased' is always true. symboldatabase.cpp 3990
* Taking care of incorrect template syntax (missing close)
Forward the incorrectness via throw, as there is no direct access to the
syntaxError method, without converting the function to a member function
IOBYTE
Rikard Falkeborn
Daniel Marjamäki
Sebastian
KenPatrickLehrmann
Steve Mokris
Oliver Stöneberg
Thomas Niederberger
Paul Fultz II
rebnridgway
orbitcowboy
Denis
amai2012
Frank Zingsheim
Simon Martin
PKEuS
Colomban Wendling
orbitcowboy
tam do thanh
PKEuS
jrp2014
Lauri Nurmi
Matthias Krüger
Jørgen Kvalsvik
David Hallas
Alexey Eryomenko
Dmitry-Me
Simon Martin
Oleksandr Redko
Ayaz Salikhov
Ayaz Salikhov
Martin Güthle
orbitcowboy
uburuntu
Frank Zingsheim