The template simplifier works well enough now so cleanupAfterSimplify is
no longer necessary. In fact cleanupAfterSimplify was introducing a bug
which improperly simplified C++ style casts.
Bugs should be exposed and fixed properly rather than just hiding them.
Co-authored-by: Robert Reif <reif@FX6840>
* Tokenize: Set varId for variables in enum
Set varIds in enum values. It was previously disabled in 5119ae84b8
to avoid issues with enums named the same as global variables. Take care
to only set varids to variables used to set the value of an enumerator,
not the enumerator itself. This is somewhat complicated by the fact that
at the time this happens, astOperand1(), astOperand2(), astParent() etc
are not set. The current implementation is not perfect, for example in
the code below, y will not have a varid set, but x and z will. This is
deemed sufficient for now.
int x, y, z;
enum E { a = f(x, y, z); };
* Fix#9647: Value of enums with variables as init values
C++ allows enum values to be set using constexprs, which cppcheck did
not handle before. To solve this, add a new pass to valueflow to update
enum values after global consts have been processed. In order to do so,
I moved all settings of enum values to valueflow. After setting the enum
values, we need another call to valueFlowNumber() to actually set users
of the enums.
There is still room for improvements, since each pass of
valueFlowGlobalConstVar() and valueFlowEnumValue() only sets variables
that are possible to set directly, and not if setting the value of a
variable allows us to set the value of another. For example
constexpr int a = 5;
constexpr int b = a + 5;
enum E { X = a };
constexpr E e = X;
Here both b and e will not have their values set, even though cppcheck
should be possible to figure out their values. That's for another PR
though.
This was tested by running test-my-pr.py with 500 packages. The only
difference was one error message in fairy-stockfish_11.1, where cppcheck
now printed the correct size of an array instead of 2147483648 which I
assume is some kind of default value. In that package, using a constexpr
when setting enum values is common, but as mentioned, there was no
change in the number of warnings.
The template parameter is confusing simplifyUsing: it does not compute
properly the scope, and we end up replace "type" in "to_string" with
"void", then later "void" is removed and we have an internal error.
* Add rudimentary support for [[maybe_unused]]
* Add more test cases. use the symboldatabase rather than reparsing. Fix travis error.
* test review actions
* change var to usage._var
* cppcheck.cpp: Check for JSON error when parsing addon .json files
This fixes that errors in JSON files given via `--addon=*.json` are
silently ignored and maybe only a part of the JSON file is used.
Now the error message which picojson can return is checked and a
corresponding error message is returned again by getAddonInfo().
* naming.json: Fix missing comma
* CLI: Fix naming violations detected by addon naming.py via naming.json
* Addon naming: Add argument for validating names of constants
* LIB: Rename functions/variables so they are valid, loosen naming rules
* GUI: Fix naming violations
* fix syntax error for conversion operator for type with global namespace
* fix syntax error when taking address of operator function
* fix syntax error for using ::operator "" _a;
* fix syntax error for template<> void operator "" _h<'a', 'b', 'c'>() {}
* fix syntax error for operator in parentheses
There are probably a lot more valid code patterns that generates syntax
errors so I added "operator" to the error message to make it easier to
find them.
* Set correct type and size of string and char literals
Use that string and char literal tokens store the prefix. This makes
it possible to distinghuish between different type of string literals
(i.e., utf8 encoded strings, utf16, wide strings, etc) which have
different type.
When the tokens holding the string and character values have the correct
type, it is possible to improve Token::getStrSize() to give the correct
result for all string types. Previously, it would return the number of
characters in the string, i.e., it would give the wrong size unless
the type of the string was char*.
Since strings now can have different size (in number of bytes) and
length (in number of elements), add a new helper function that returns
the number of characters. Checkers have been updated to use the correct
functions.
Having the size makes it possible to find more problems with prefixed
strings, and to reduce false positives, for example in the buffer
overflow checker.
Also, improve the stringLiteralWrite error message to also print the
prefix of the string (if there is one).
* Add comment and update string length
Keeping the prefix in the token allows cppcheck to print the correct
string and char literals in debug and error messages.
To achieve this, move some of the helper functions from token.cpp to
utils.h so that checks that look at string and char literals can reuse
them. This is a large part of this commit.
Note that the only user visible change is that when string and char
literals are printed in error messages, the prefix is now included.
For example:
int f() {
return test.substr( 0 , 4 ) == U"Hello" ? 0 : 1 ;
};
now prints U"Hello" instead of "Hello" in the error message.
`__typeof__` is just an alternative keyword for `typeof`, see
https://gcc.gnu.org/onlinedocs/gcc/Typeof.html
Since `typeof` is handled in several checkers it makes sense to define
`__typeof__` as `typeof`.
Tokenizer::simplifyTypedef(): Use `typeof` instead of `__typeof__` to
be consistent with the rest of the code.
* Better handle const/noexcept methods/conversion operator
const or noexcept in a method / (conversion) operator definition were
badly parsed, ending in a bad ast.
This patch tries to make it better, at least making the ast less bad,
so as to avoid errors in later checks.
* Fix parsing of some operator
It is still very broken, but at least, it does not fail.
Here is the previous error:
```
TestSimplifyTypedef::simplifyTypedef129
terminate called after throwing an instance of 'InternalError'
Program received signal SIGABRT, Aborted.
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
51 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1 0x00007ffff612a801 in __GI_abort () at abort.c:79
#2 0x00007ffff6b1d957 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#3 0x00007ffff6b23ab6 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#4 0x00007ffff6b23af1 in std::terminate() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#5 0x00007ffff6b23d24 in __cxa_throw () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#6 0x0000555556366bf8 in Tokenizer::cppcheckError (this=0x7fffffffc2d0, tok=0x607000006760) at ../lib/tokenize.cpp:8721
#7 0x000055555636a4bb in Tokenizer::validate (this=0x7fffffffc2d0) at ../lib/tokenize.cpp:9154
#8 0x000055555633e3aa in Tokenizer::simplifyTokenList1 (this=0x7fffffffc2d0, FileName=0x603000002d50 "test.cpp") at ../lib/tokenize.cpp:4477
#9 0x00005555563223ca in Tokenizer::simplifyTokens1 (this=0x7fffffffc2d0, configuration="") at ../lib/tokenize.cpp:2286
#10 0x00005555563235c8 in Tokenizer::tokenize (this=0x7fffffffc2d0, code=..., FileName=0x555556fda9a0 "test.cpp", configuration="") at ../lib/tokenize.cpp:2345
#11 0x00005555569410ea in TestSimplifyTypedef::tok[abi:cxx11](char const*, bool, cppcheck::Platform::PlatformType, bool) (this=0x555557728580 <(anonymous namespace)::instance_TestSimplifyTypedef>,
code=0x7fffffffcb70 "class c {\n typedef char foo[4];\n foo _a;\n constexpr operator foo &() const noexcept { return _a; }\n};", simplify=false, type=cppcheck::Platform::Native, debugwarnings=true) at ../test/testsimplifytypedef.cpp:192
#12 0x000055555697239e in TestSimplifyTypedef::simplifyTypedef129 (this=0x555557728580 <(anonymous namespace)::instance_TestSimplifyTypedef>) at ../test/testsimplifytypedef.cpp:2599
#13 0x000055555694092c in TestSimplifyTypedef::run (this=0x555557728580 <(anonymous namespace)::instance_TestSimplifyTypedef>) at ../test/testsimplifytypedef.cpp:167
#14 0x00005555569cab84 in TestFixture::run (this=0x555557728580 <(anonymous namespace)::instance_TestSimplifyTypedef>, str="simplifyTypedef129") at ../test/testsuite.cpp:306
#15 0x00005555569cb445 in TestFixture::runTests (args=...) at ../test/testsuite.cpp:329
#16 0x000055555687bdfb in main (argc=2, argv=0x7fffffffd988) at ../test/testrunner.cpp:44
```
* Replace some ASSERT_EQUALS with TODO_ASSERT_EQUALS when the actual result is still wrong
* Remove invalid code from simplifyTypedef129
* Properly skip parentheses
Previously, cppcheck discarded the `extern "C"` specifier. This patch modifies cppcheck to parse each as a Scope in the symbol database, then uses that scope to avoid false positives when making recommendations about changing a function argument to be a reference (since variable references is a C++ feature, unavailable in C, and thus unavailable in `extern "C"`).
I fixed the AST enough to pass testrunner but I don't believe it is
correct.
This code:
void Foo4(int&&b);
has this AST:
( 'void'
|-Foo4
`-&& 'bool'
|-int
`-b 'signed int'
but I don't believe && should have `bool`.
* use range loops
* removed redundant string initializations
* use nullptr
* use proper boolean false
* removed unnecessary continue from end of loop
* removed unnecessary c_str() usage
* use emplace_back()
* removed redundant void arguments
* make ellipsis ... a single token
Using cppcheck -E to preprocess code with ellipsis produces output that
can't be compiled because ... is split into 3 tokens.
* try to fix addon
* Add missing Qt macros
Add two Qt macros that were missing
* Fix issue with __declspec and final
This change is a bit naive but it fixes the issues I was having when combining __declspec(dllexport) and final classes. Without the fix I get errors along the line of "The code 'class x final :' is not handled. You can use -I or --include to add handling of this code. "
* Fix for too much information in scope name
When the scope calculation encounters code such as
"friend class X::Y;"
or
"template<> class X<void> {"
it will now reset the additional name component of the scope that is about to be opened.
* Made sure new scope name is reset after being used
* Removed redundant scope calculation
* Add scope propagation code to insertToken
* Add relevant scope code to Token class
* Add code to calculate the scope of Tokens
* Add calculateScopes method to class
* Add missing include for shared_ptr
* Fix#9225 (Crash on valid C++14 code)
This only fixes the crash. Specialization of nested templates is still
broken.
* fix cppcheck warnings
* fixed another cppcheck warning
* Fix adding unescaped slash token when splitting gcc case range.
Construction like case '!'...'~' converted to a list of separate case
tokens. When slas '\' symbol appears as a part of this list it was added
"as is", but it should be escaped like '\\' to be valid c++ code.
* Add test for switch-case range with slash
* Refactor Tokenizer::simplifyUsing to use continue to reduce indentation
added function findTemplateDeclarationEnd to skip template declarations
to reduce duplicate code
* fix travis build
* Partial fix for #9191 (simplifyTypedef: Problem when namespace is used)
This fixes simplifyUsing which has the same problem as simplifyTypedef.
simplifyUsing was designed to support using namespace but it was never
implemented. The changes are minor to add it.
simplifyTypedef requires much more work to support using namespace.
* reduce scope of variable
* make idx const
* Added scopeinfo member to token class
Moved ScopeInfo2 declaration here as well because that's where it needs to be now.
* Added scopeinfo accessors and declaration to class
* Add new method for calculating scopes
This replaces the methods in the TemplateSimplifier which calculate the current scope as the token list is iterated. The old method required checking if the scope had changed for every token multiple times (for multiple iterations), which was surprisingly costly. Calculating scopes in advance like this decreases runtime on a worst-case file by around thirty percent.
ScopeInfo objects are disposed of when the TemplateSimplification is done as they are not used later.
* Add calculateScopes method to header
* Removed code that calculated current scope
This has been replaced by code that calculates the scopes up front and stores them with each token, which is much faster.
* Fixed compile errors from extra parentheses
* Added missing code to fix memory leak
* Added code to actually clean up ScopeInfo structs
* Tidy up a dodgy for loop
* Convert argument to const ref
* Calculate missing scopes
As the templatesimplificator expands templates and does multiple passes it needs to make sure all scopes are calculated.
* Remove copying the scope to the next token
This is now done properly when scopes are calculated.
* Remove call to calculateScopes
This is now done by the TemplateSimplifier.
* Recalculate scopes for every pass of simplifyTemplates
* Add code to calculate extra scopes as they are added
I thought that this might be useful for calculating scopes when Tokens are created, but as there are several ways of creating Tokens that don't guarantee that they are placed in a list it is easier to just calculate scopes when you know you have a list and when you know you're adding to a list.
* Fix several bugs and poorly designed code
Remove the global scopes collection, and clean them up instead by iterating through the tokenlist to find them. This means scopes can be calculated by functions in the Token class as well as in the Tokenizer class without leaking the scope object.
Fix a couple of bugs in the calculateScopes method and make it more efficient.
* Remove unnecessary calls to calculateScopes
* Move brace to correct position
Calculating scopes during insertToken only needs to happen if we created a new Token.
* Handle 'using namespace' declarations separately
This fixes a bug caused by a statement matching 'struct B < 0 > ;'
* Fix argument name mismatch
* Actually use newScopeInfo when inserting Token
* Switch to using shared_ptr to hold scopeInfos
This means ScopeInfo2 objects get properly cleaned up when they are no longer needed.
* Change ScopeInfo member to be a shared_ptr
* Update code to use shared_ptr
* Add missing include for shared_ptr
* Remove unnecessary cleanup code
This has been replaced by shared_ptr for ScopeInfo2 objects
* fix adding instantiation of first argument to an instantiation
* add support for function pointer template variables
* fix more cases where templates ending in ">>" are changed to end in "> >"
* fix travis build
* standard types can't be a template parameter name
* remove redundant level == 0 checks
* fix lambda in template variable
* fix a test
* Fix issue 8890: AST broken calling member function from templated base class
* Format
* Check for double bracket
* Add test to createLinks2
* Remove extra test
* Reduce test case for links
This fixes simplifyUsing to remove 'typename' and 'template' from type
aliases of the form: using T3 = typename T1::template T3<T2>;
This lets the template simplifier instantiate the type alias which will
then remove the using type alias.
The crash will still happen if there is no instantiation because the
type alias will not be removed. The type alias is what cppcheck is
crashing on after the template simplifier and that still needs fixing.
* Fixed#8889 (varid on function when using trailing return type.)
Don't set varid for trailing return type.
* Add a test for #9066 (Tokenizer::setVarId: varid set for trailing return type)
This handles concatenated strings and characters from simplecpp.
Previously, L'c' would be preprocessed to the tokens "L" and "'c'".
cppcheck would then remove the "L" token and set "'c'" to be a wide
character literal. Now, it needs to remove the prefix instead.
When doing this, add handling of utf32 encoded literals (U) and UTF-8
encoded literals (u8).
Add a call to simplifyPlatformTypes() in
SymbolDatabase::setValueTypeInTokenList() to simplify return types of
library configured functions. This fixes the FN in #8141. Regression
tests are added, both for the original issue and another FN in the comments.
In order to do that, move simplifyPlatformTypes() to TokenList from Tokenizer.
This is a pure refactoring and does not change any behaviour. The code was
literally copy-pasted from one file to another and in two places
'list.front()' was changed to 'front()'.
When adding the call to simplifyPlatformTypes(), the original type of
v.size() where v is a container is changed from 'size_t' to 'std::size_t'.
Tests are updated accordingly. It can be noted that if v is declared as
'class fred : public std::vector<int> {} v', the original type of 'v.size()'
is still 'size_t' and not 'std::size_t'.
* Fixed#8960 ("(debug) Unknown type 'x'." with alias in template class alias)
This commit adds non-template type alias support to the template
simplifier. Only relatively simple type aliases are supported at this
time. More complex types will be added later.
--debug-warnings will show unsupported type aliases.
Type alias support will be removed from the symbol database in the
future. Type alias tests have been removed from the symbol database
tests.
* Add the changes.
* Fix codacy warning.
* Fix travis warnings.
* Fixed#8683 (Using deleted token with multiple template instantiations.)
* Fixed#8321 (heap use after free: templatesimplifier)
* Add a flag to Token indicating that it has a pointer to it.
* Run dmake
* Fix one source of list pointers to deleted tokens.
Refactor TemplateSimplifier class to get access to template lists.
Remove many function parameters now that they are class variables.
Fix one source of list pointers to deleted tokens.
Add tests with no output to catch crashes.
* Run dmake again.
* Make 2 more functions private.
* Make requested changes.
* Missed one change request.
* Use TokenList rather than Tokenizer.
* Move TokenAndName constructor to cpp file so token.h is not needed in header file.
* Allow syntax "typedef 'typename' unsigned 'typename' (functon_name)()" for some old libraries
https://trac.cppcheck.net/ticket/7792
* Unnecessary check of "tokOffset", match-function already checks if it is null
* add testcase ticket #7792: simplifyTypedef124
* Added declaration for deletePrevious function
* Added definition for deletePrevious function
* Fixed crash from deleteThis invalidating pointers
The crash was caused by deleteThis() invalidating the pointer to a constant variable usage. This happened when a usage followed an assignment. This fixes bug #8579.
* Added tokensFront to match tokensBack
This means deletePrevious can set the list's front if necessary.
* Initialised tokensFront in appropriate places
* Switched to using default Token constructor
* Switched to using Token default constructor
* Switched to using default constructor for Token
* Added missing argument to Token constructor
* Changed to use default constructor for Tokens
* Switched to using default constructor for Tokens
* Switched to using default constructor for Token
* Added new test for deleting front Token
Also made sure to use the correct constructor for Token in other tests.
* Syntax error
* Replaced tokensFront and tokensBack with a struct
This decreases the size of the Token class for performance purposes.
* Replaced tokensFront and tokensBack with a struct
* Added tokensFrontBack to destructor
* Reworked to use TokensBackFront struct
Also ran astyle.
* Reworked to use TokenList's TokensFrontBack member
* Reworked to use TokensFrontBack struct
* Reworked to use TokensFrontBack struct
* Reworked to work with TokensFrontBack struct
* Removed unnecessary scope operator
* Added missing parentheses
* Fixed syntax error
* Removed unnecessary constructor
* Default constructor now 0-initialises everything
This is safer for not using a temporary TokensFrontBack object, and doesn't use delegating constructors which aren't supported yet.
* Fixed unsafe null check
* Added missing explicit keyword
* Fixing stylistic nits
Removed default constructor as it has been superseded by the single-argument constructor with a default argument value.
Renamed listEnds to tokensFrontBack.
Fixed if statement that was supposed to be adding safety but would actually cause a crash if tokensFrontBack was null.
* Fixing stylistic nits
Removed default constructor and replaced it with a single-argument constructor with a default value.
* Fixing stylistic nits
Renamed _listEnds to _tokensFrontBack.
* Fixing stylistic nits
Renamed _listEnds to _tokensFrontBack.
* Remove duplicate namespace aliases so they don't produce syntax errors.
DACA2 results showed new SymbolDatabase syntax errors when duplicate
namespace aliases were simplified improperly. The solution is to remove
them in the tokenizer when found.
* Add tests for deleting namespace aliases at end of token list.
* Use eraseTokens to delete multiple tokens at once.
* Optimised simplifyKnownVariables
Changed the check for references to constant variables so instead of iterating through all tokens looking for references (which is very slow for large files), usages of each known variable are recorded so each usage can be checked for whether or not it is a reference. Just checking known usages is a lot quicker than checking through all tokens.
* Fixed test error caused by incorrect code
* Fixes to constant variable simplification optimisation
Indexing variables by varId is easier than by Token pointer, but means we also have to store a reference to the constant variable Token in another collection.
Switched to using unordered_map over map as it has slightly better find performance.
* Fixed incorrect simplification behaviour
This should remove constant variables correctly and efficiently. Requires additional functions, Token::deleteThisInPlace() and TokenList::front(Token*).
* Added setter for TokenList::first
This allows code that adds and removes Tokens from the list to do so without copying nodes into other nodes, which sometimes create difficulties.
* Added deleteThisInPlace function
This allows a token to delete itself without invalidating pointers to the node after it. If this token is the first or last in a list the calling code will have to remember to change the list's front or back.
* Added declaration for deleteThisInPlace()
* Removed premature MatchCompiler optimisation
* Added and removed some functions
Added declaration for deleteToken(Token*) which deletes a single token from the list
Removed public front(Token*) because it broke encapsulation
* Implemented deleteToken(Token*) function
* Removed 'delete this' from deleteThisInPlace
* Switched to using safer function
TokenList::deleteToken is better than calling straight into Token::deleteThisInPlace because it doesn't call delete this and doesn't break the TokenList's encapsulation.
* Replace constant variables in reverse order
This fixes the problem where you have two constant value assignment statements in a row. Replacing and deleting them in reverse order means we avoid the problem of deleteThis() potentially invalidating the pointer to the start of the next assignment statement
* Removed unneeded and unsafe deleteThisInPlace
* Removed unneeded and unsafe deleteThisInPlace
* Removed unneeded deleteToken
* Removed unneeded deleteToken
* Removed extra whitespace
* Don't remove the volatile keyword so we can properly overload functions.
I fixed all the checks that had tests that use volatile. There will
probably be more changes needed due to lack of test coverage for
volatile in some checks.
* Fix unused private function warning.
* Add support for namespace aliases and C++17 nested namespaces.
These are implemented as tokenizer simplifications so changes are not
needed to the tokenizer and symbol database.
* Fix codacy warning.
In 9cea2d6df, simplifications were removed for a number of functions
which should instead be handled with configurations. The commit did
however not update the description of the function, do this now.
Also sin() and sinh() and their float and long double versions were
missing from the comment so add these as well.
Remove TEXT() macro from windows.cfg and handle it internally where it
can be correctly simplified (Ansi vs. Unicode).
Also add handling of _TEXT() macro which is just a synonym for _T().
Add tests to verify correct function and macro simplification.
* Fix#8382 (Syntax error when scanning code with template and attribute)
This commit only addresses #8382. There are issues concerning which
versions of C++ should be supported and also generic C++ 14 attribute
support which can be revisited later.
* Remove all C++ style attributes.
Remove all C++ style attributes when C++ version is 11 or greater.
Rename simplify function to simplifyCPPAttributes.
Handle more cases of roreturn function attribute.
The following snippet triggerd the error:
template<typename DerivedT>
template<typename T>
auto ComposableParserImpl<DerivedT>::operator|( T const &other ) const -> Parser {
return Parser() | static_cast<DerivedT const &>( *this ) | other;
}
Whenever simplifyFunctionParameters was called on a templated class'
templated member function (and probably any nested template), the
tokenizer would recognise it as a syntax error, assuming that return
type *must* come after a template<> token.
* Fixes issue with case inside switch that is not a compound statement was treated as garbage
This fixes an issue with the check for case keywords outside of switch
detection that would treat a case statement inside a switch that is not
a compound statement as garbage, but this is perfectly valid C++. This
construct is used in several libraries, i.e. Google Test.
* Tweak check and handle missing semicolon
Tweaks the check with feedback from danmar.
Handle the case where there is no semicolon and document it with a unit
test.
* Add support for simple c++ 11 type ailases like: using INT = int;
Only types supported by ValueType are supported. Complex types like
function pointers are not supported. Template type aliases are not
supported.
* Fix crash when type in using type alias is simplified away.
This fixes a crash when size_t is replaced with unsigned long in: using
size_t = unsigned long; by the tokenizer.
This does not fix the problem where Tokenizer::simplifyPlatformTypes()
simplifies away size_t in other cases. This is only a problem when the
new type is different from the platform type.
IOBYTE
Daniel Marjamäki
uhziel
Rikard Falkeborn
Armin Müller
Oliver Stöneberg
Georgy Komarov
Simon Martin
Ken-Patrick Lehrmann
Paul Fultz II
PKEuS
Zorgovskiy
shaneasd
RobkeBaer
Sebastian
Daniel Marjamäki
KenPatrickLehrmann
Steve Mokris
Thomas Niederberger
rebnridgway
orbitcowboy
Denis
amai2012
Frank Zingsheim
PKEuS
Colomban Wendling
orbitcowboy
tam do thanh
jrp2014
Lauri Nurmi
Matthias Krüger
Jørgen Kvalsvik
David Hallas
Alexey Eryomenko
Dmitry-Me