Commit Graph

2385 Commits

Author SHA1 Message Date
Daniel Marjamäki f9b132e831 Tokenizer::simplifyTypedef: Fix bug with arrays 2018-10-21 21:15:34 +02:00
Daniel Marjamäki 14afc3fd3a Fixed #8796 (Tokenizer::simplifyCompoundAssignment: Wrong simplification of return) 2018-10-20 21:33:43 +02:00
Daniel Marjamäki 29feaa5a51 Tokenizer: Avoid constant folding 2018-10-20 10:51:50 +02:00
Paul Fultz II 9cf092657c Skip some ast errors to avoid regressions (#1422)
* Skip some ast errors to avoid regressions

* Use simpleMatch

* Skip operator functions

* Add a test for issue 8788
2018-10-13 18:38:44 +02:00
Daniel Marjamäki 7eb5ebe17e Fixed #7887 (User function 'search' is wrongly mixed with std function) 2018-10-10 17:35:53 +02:00
Daniel Marjamäki 25cbfe27cf Partial fix for #7887, Improved tokenizer simplification of 'using namespace std;' - do not simplify user functions 2018-10-10 14:28:53 +02:00
Daniel Marjamäki c3e6e7c338 Fixed #8664 (setVarId: Wrongly set varid for lambda argument) 2018-10-07 20:40:59 +02:00
orbitcowboy d08b39c915
Improved const correctness of local variables. There are no functional changes intended. (#1392) 2018-09-23 20:24:51 +02:00
Simon Martin 6bde2445a6 Ticket #8632: Parenthesize ternary operator operands containing < to avoid wrongly thinking a template instantiation is met. (#1389) 2018-09-23 10:27:38 +02:00
Daniel Marjamäki 523a9c1c4a Fixed #8746 (Syntax error, AST broken (using a::operator=)) 2018-09-09 21:11:45 +02:00
Daniel Marjamäki 1bb1c4cc8e Use early continue 2018-09-09 20:58:30 +02:00
Daniel Marjamäki 404eb6c746 Fixed #8745 (Syntax error: AST broken (or)) 2018-09-09 16:41:06 +02:00
Simon Martin 1d85a78874 Ticket #8654: Properly setup links for variadic template bases. (#1357) 2018-09-05 14:10:56 +02:00
Daniel Marjamäki 5afb755a3c SymbolDatabase: Fix variable matching 2018-09-04 06:39:02 +02:00
IOBYTE 508e8c234b Fixed #8683 (Using deleted token with multiple template instantiations.) (#1353)
* Fixed #8683 (Using deleted token with multiple template instantiations.)

* Fixed #8321 (heap use after free: templatesimplifier)

* Add a flag to Token indicating that it has a pointer to it.

* Run dmake

* Fix one source of list pointers to deleted tokens.

Refactor TemplateSimplifier class to get access to template lists.
Remove many function parameters now that they are class variables.
Fix one source of list pointers to deleted tokens.
Add tests with no output to catch crashes.

* Run dmake again.

* Make 2 more functions private.

* Make requested changes.

* Missed one change request.

* Use TokenList rather than Tokenizer.

* Move TokenAndName constructor to cpp file so token.h is not needed in header file.
2018-09-01 11:26:10 +02:00
Simon Martin acb0b9f07e Ticket #8679: Add support for C++11 thread_local and GCC's (among others) __thread extension. (#1351) 2018-08-26 19:46:36 +02:00
Daniel Marjamäki e442bc47b1 astyle formatting
[ci skip]
2018-08-17 19:55:21 +02:00
tam do thanh f33c09f1a7 Allow syntax "typedef 'typename' unsigned 'typename' (functon_name)()" (#1334)
* Allow syntax "typedef 'typename' unsigned 'typename' (functon_name)()" for some old libraries
https://trac.cppcheck.net/ticket/7792

* Unnecessary check of "tokOffset", match-function already checks if it is null

* add testcase ticket #7792: simplifyTypedef124
2018-08-16 14:30:28 +02:00
Daniel Marjamäki ddbe4b89b5 Renamed --debug to --debug-simplified 2018-08-05 11:19:20 +02:00
Daniel Marjamäki 60485df762 Refactoring; Use range for loops 2018-07-14 22:49:31 +02:00
Daniel Marjamäki 13cf93a322 Rename private variable maxtime 2018-06-18 09:49:00 +02:00
Daniel Marjamäki 66785c238a Renamed _timerResults 2018-06-16 22:14:59 +02:00
Daniel Marjamäki 63e6d35cb6 Renamed _unnamedCount to mUnnamedCount 2018-06-16 22:09:18 +02:00
Daniel Marjamäki ef534c2d0d Renamed _codeWithTemplates 2018-06-16 22:03:04 +02:00
Daniel Marjamäki 5dc29a9a16 Renamed _configuration to mConfiguration 2018-06-16 21:57:48 +02:00
Daniel Marjamäki 598a984071 Renamed _typeSize to mTypeSize 2018-06-16 21:56:05 +02:00
Daniel Marjamäki 5e43366126 Rename members 2018-06-16 21:52:29 +02:00
Daniel Marjamäki 78974e9267 Rename _type to mType 2018-06-16 20:25:54 +02:00
Daniel Marjamäki 8e2b18a75a Rename _varId to mVarId 2018-06-16 16:38:50 +02:00
Daniel Marjamäki d81fe83ca6 renamed _symbolDatabase to mSymbolDatabase 2018-06-16 16:25:27 +02:00
Daniel Marjamäki 79ffe1d4fc Rename _tokenizer, _settings, _errorLogger 2018-06-16 16:10:28 +02:00
Daniel Marjamäki 7163210f72 Fixed #8605 (Segmentation fault below TemplateSimplifier::simplifyCalculations) 2018-06-01 23:03:53 +02:00
IOBYTE 3982c81394 Add support for C++ attribute nodiscard for functions. (#1269) 2018-05-29 21:43:56 +02:00
Daniel Marjamäki 14e60210c9 Optimize Tokenizer::setVarIdPass1 2018-05-29 18:41:05 +02:00
Daniel Marjamäki 02c8a1d61c Tokenizer::setVarIdPass1 optimization 2018-05-28 11:32:47 +02:00
rebnridgway 42a65c5160 Fix crash bug #8579 (#1238)
* Added declaration for deletePrevious function

* Added definition for deletePrevious function

* Fixed crash from deleteThis invalidating pointers

The crash was caused by deleteThis() invalidating the pointer to a constant variable usage.  This happened when a usage followed an assignment.  This fixes bug #8579.

* Added tokensFront to match tokensBack

This means deletePrevious can set the list's front if necessary.

* Initialised tokensFront in appropriate places

* Switched to using default Token constructor

* Switched to using Token default constructor

* Switched to using default constructor for Token

* Added missing argument to Token constructor

* Changed to use default constructor for Tokens

* Switched to using default constructor for Tokens

* Switched to using default constructor for Token

* Added new test for deleting front Token

Also made sure to use the correct constructor for Token in other tests.

* Syntax error

* Replaced tokensFront and tokensBack with a struct

This decreases the size of the Token class for performance purposes.

* Replaced tokensFront and tokensBack with a struct

* Added tokensFrontBack to destructor

* Reworked to use TokensBackFront struct

Also ran astyle.

* Reworked to use TokenList's TokensFrontBack member

* Reworked to use TokensFrontBack struct

* Reworked to use TokensFrontBack struct

* Reworked to work with TokensFrontBack struct

* Removed unnecessary scope operator

* Added missing parentheses

* Fixed syntax error

* Removed unnecessary constructor

* Default constructor now 0-initialises everything

This is safer for not using a temporary TokensFrontBack object, and doesn't use delegating constructors which aren't supported yet.

* Fixed unsafe null check

* Added missing explicit keyword

* Fixing stylistic nits

Removed default constructor as it has been superseded by the single-argument constructor with a default argument value.
Renamed listEnds to tokensFrontBack.
Fixed if statement that was supposed to be adding safety but would actually cause a crash if tokensFrontBack was null.

* Fixing stylistic nits

Removed default constructor and replaced it with a single-argument constructor with a default value.

* Fixing stylistic nits

Renamed _listEnds to _tokensFrontBack.

* Fixing stylistic nits

Renamed _listEnds to _tokensFrontBack.
2018-05-25 07:15:05 +02:00
IOBYTE fc1d62fd45 Fixed #7406 (Tokenizer::simplifyTypedef: array typedef used as template parameter) (#1257) 2018-05-22 17:31:58 +02:00
PKEuS e8bc955023 Optimized Tokenizer::simplifyFunctionReturn(): Replaced nested loops by two consecutive loops; execute second one only if necessary. 2018-05-14 20:51:09 +02:00
PKEuS 513cdc144c Optimization: Save up to 2 of 3 token list iterations in Tokenizer::simplifyKeyword() 2018-05-13 10:12:50 +02:00
IOBYTE 9ee6068e20 Remove duplicate namespace aliases so they don't produce syntax errors. (#1222)
* Remove duplicate namespace aliases so they don't produce syntax errors.

DACA2 results showed new SymbolDatabase syntax errors when duplicate
namespace aliases were simplified improperly. The solution is to remove
them in the tokenizer when found.

* Add tests for deleting namespace aliases at end of token list.

* Use eraseTokens to delete multiple tokens at once.
2018-05-13 08:29:40 +02:00
PKEuS 723bacbf09 Fixed hang introduced in previous commit and further optimized Tokenizer::simplifyStaticConst(): Avoid redundant Token swapping 2018-05-12 21:35:54 +02:00
PKEuS a16e99c710 Optimization: Save 2 of 3 token list iterations in Tokenizer::simplifyStaticConst() 2018-05-12 19:19:48 +02:00
Simon Martin 16e1e1d8f9 Ticket #8550: Properly simplify "typedef class A B;". (#1224) 2018-05-12 10:20:33 +02:00
Daniel Marjamäki a3f1d28fa5 astyle formatting
[ci skip]
2018-05-11 21:25:41 +02:00
rebnridgway cf6e72b150 Optimised simplifyKnownVariables (#1205)
* Optimised simplifyKnownVariables

Changed the check for references to constant variables so instead of iterating through all tokens looking for references (which is very slow for large files), usages of each known variable are recorded so each usage can be checked for whether or not it is a reference.  Just checking known usages is a lot quicker than checking through all tokens.

* Fixed test error caused by incorrect code

* Fixes to constant variable simplification optimisation

Indexing variables by varId is easier than by Token pointer, but means we also have to store a reference to the constant variable Token in another collection.
Switched to using unordered_map over map as it has slightly better find performance.

* Fixed incorrect simplification behaviour

This should remove constant variables correctly and efficiently.  Requires additional functions, Token::deleteThisInPlace() and TokenList::front(Token*).

* Added setter for TokenList::first

This allows code that adds and removes Tokens from the list to do so without copying nodes into other nodes, which sometimes create difficulties.

* Added deleteThisInPlace function

This allows a token to delete itself without invalidating pointers to the node after it.  If this token is the first or last in a list the calling code will have to remember to change the list's front or back.

* Added declaration for deleteThisInPlace()

* Removed premature MatchCompiler optimisation

* Added and removed some functions

Added declaration for deleteToken(Token*) which deletes a single token from the list
Removed public front(Token*) because it broke encapsulation

* Implemented deleteToken(Token*) function

* Removed 'delete this' from deleteThisInPlace

* Switched to using safer function

TokenList::deleteToken is better than calling straight into Token::deleteThisInPlace because it doesn't call delete this and doesn't break the TokenList's encapsulation.

* Replace constant variables in reverse order

This fixes the problem where you have two constant value assignment statements in a row.  Replacing and deleting them in reverse order means we avoid the problem of deleteThis() potentially invalidating the pointer to the start of the next assignment statement

* Removed unneeded and unsafe deleteThisInPlace

* Removed unneeded and unsafe deleteThisInPlace

* Removed unneeded deleteToken

* Removed unneeded deleteToken

* Removed extra whitespace
2018-05-11 21:14:04 +02:00
IOBYTE 184537884f Don't remove the volatile keyword so we can properly overload functions. (#1218)
* Don't remove the volatile keyword so we can properly overload functions.

I fixed all the checks that had tests that use volatile.  There will
probably be more changes needed due to lack of test coverage for
volatile in some checks.

* Fix unused private function warning.
2018-05-10 07:40:01 +02:00
Daniel Marjamäki ecccce0608 Tokenizer::setVarId: Made it ~5% faster 2018-05-09 10:25:29 +02:00
IOBYTE 8b0b659965 Add support for namespace aliases and C++17 nested namespaces. (#1210)
* Add support for namespace aliases and C++17 nested namespaces.

These are implemented as tokenizer simplifications so changes are not
needed to the tokenizer and symbol database.

* Fix codacy warning.
2018-05-08 06:35:51 +02:00
Daniel Marjamäki 59cc479855 Save bitfield bit counts 2018-05-02 20:55:11 +02:00
Daniel Marjamäki 45c4456c24 Tokenizer: Code cleanup 2018-04-29 15:05:13 +02:00
Daniel Marjamäki ca8e19c96d SymbolDatabase: Refactor SymbolDatabase: variable list 2018-04-28 09:38:33 +02:00
Daniel Marjamäki f336c2efe7 Refactoring; Renamed Scope::classStart and Scope::classEnd 2018-04-27 22:36:30 +02:00
Daniel Marjamäki e6a37ec0b7 Fixed #8531 (false positive: (style) The function 'foo' overrides a function in a base class but is not marked with a 'override' specifier.) 2018-04-27 21:49:18 +02:00
Daniel Marjamäki 25599a76a7 Handle 'final' specifier better. 2018-04-27 14:57:43 +02:00
Daniel Marjamäki 8304290f06 astyle formatting
[ci skip]
2018-04-27 10:29:27 +02:00
Simon Martin 9fade65dbb Ticket #8281, #8417: Properly detect the end of "switch" statements to accept all legitimate uses of "case". (#1112) 2018-04-26 22:26:26 +02:00
Daniel Marjamäki 6d86ad78ba Refactoring, use early continue 2018-04-23 16:23:22 +02:00
rikardfalkeborn 57019c0128 Fix comment in Tokenizer::simplifyMathFunctions (#1179)
In 9cea2d6df, simplifications were removed for a number of functions
which should instead be handled with configurations. The commit did
however not update the description of the function, do this now.

Also sin() and sinh() and their float and long double versions were
missing from the comment so add these as well.
2018-04-19 06:14:16 +02:00
Daniel Marjamäki e95ff8c7b6 Avoiding emplace 2018-04-14 19:24:35 +02:00
Sebastian c39a3e3f1c
windows string macros: Handle _T, _TEXT and TEXT internally, add tests (#1163)
Remove TEXT() macro from windows.cfg and handle it internally where it
can be correctly simplified (Ansi vs. Unicode).
Also add handling of _TEXT() macro which is just a synonym for _T().
Add tests to verify correct function and macro simplification.
2018-04-12 08:52:31 +02:00
PKEuS d2146844dd Refactorizations:
- Replace several push_back-calls by emplace_back
- Replace some x = x.substr(0, y) calls by x.erase(y)
2018-04-11 09:44:35 +02:00
PKEuS 17b4721bd2 C++17: Support "if constexpr" (by simplifying it to plain if() statement)
This might lead to complaints about constant expressions as if() statement, but should fix syntax errors.
2018-04-09 11:42:59 +02:00
PKEuS e2002db78d Replaced make_container by C++11 initializer lists 2018-04-08 23:03:44 +02:00
IOBYTE 90983303f0 Fix #8477 (False positive caused by anonymous enum in method body) (#1154) 2018-04-08 08:00:12 +02:00
jrp2014 214322d501 Refactor lib/tokenize 2018-04-05 08:02:59 +02:00
jrp2014 b6504c70ca Improve constness 2018-04-04 21:51:31 +02:00
Daniel Marjamäki c0272fc2ef Fixed #8259 (Don't combine &= for anonymous reference parameters) 2018-04-01 10:27:16 +02:00
Daniel Marjamäki 5a444f764b Fixed one more syntax error FP related to #8390 2018-03-22 15:20:37 +01:00
Daniel Marjamäki 3d4b773fd1 Fixed #8390 (Syntax error from an MSVC internal header) 2018-03-22 09:07:58 +01:00
Daniel Marjamäki 2c3cd402ba Thread safety: changed local static variable 'count' to member variable 2018-03-14 09:41:27 +01:00
Daniel Marjamäki 5c44580528 Refactoring, use early continue and make code a bit more specific. 2018-02-16 22:59:38 +01:00
IOBYTE fcde1d80e9 Fix #8382 (Syntax error when scanning code with template and attribute) (#1089)
* Fix #8382 (Syntax error when scanning code with template and attribute)

This commit only addresses #8382. There are issues concerning which
versions of C++ should be supported and also generic C++ 14 attribute
support which can be revisited later.

* Remove all C++ style attributes.

Remove all C++ style attributes when C++ version is 11 or greater.
Rename simplify function to simplifyCPPAttributes.
Handle more cases of roreturn function attribute.
2018-02-16 22:25:51 +01:00
Lauri Nurmi e0e664f996 Fix permissions of certain non-executable files (#1083)
mode 0755 => 0644
2018-02-09 19:46:38 +01:00
Daniel Marjamäki 13daaac264 astyle formatting
[ci skip]
2018-02-06 08:10:28 +01:00
Alexander Mai 68eb6c4e6f Refactoring: Add Tokenizer::findGarbageCode to time report. The method now calls syntaxError instead of returning an invalid token 2018-02-05 21:47:33 +01:00
Matthias Krüger f009cfc845 fix some typos found by codespell. 2018-02-04 20:53:43 +01:00
Jørgen Kvalsvik a61f21d1b6 Accept nested templates in tokenizer-simplify (#1070)
The following snippet triggerd the error:

template<typename DerivedT>
template<typename T>
auto ComposableParserImpl<DerivedT>::operator|( T const &other ) const -> Parser {
    return Parser() | static_cast<DerivedT const &>( *this ) | other;
}

Whenever simplifyFunctionParameters was called on a templated class'
templated member function (and probably any nested template), the
tokenizer would recognise it as a syntax error, assuming that return
type *must* come after a template<> token.
2018-02-04 09:48:37 +01:00
IOBYTE d721c6aca5 Fixed #8386 (syntax error not found (segmentation fault)) (#1081) 2018-02-03 15:50:05 +01:00
Daniel Marjamäki 21a35de0d2 Tokenizer: Refactoring garbage check 2018-01-27 22:26:43 +01:00
David Hallas 4d18d3948f Fixes issue with case inside switch that is not a compound statement (#1031)
* Fixes issue with case inside switch that is not a compound statement was treated as garbage

This fixes an issue with the check for case keywords outside of switch
detection that would treat a case statement inside a switch that is not
a compound statement as garbage, but this is perfectly valid C++. This
construct is used in several libraries, i.e. Google Test.

* Tweak check and handle missing semicolon

Tweaks the check with feedback from danmar.
Handle the case where there is no semicolon and document it with a unit
test.
2018-01-27 22:21:26 +01:00
Daniel Marjamäki c110770481 Fixed #8325 (False negative: address of auto variable being returned when assigned to another variable first) 2018-01-27 14:48:45 +01:00
Simon Martin 71ba513bdb Ticket #8361: Fix false positive in Tokenizer::findGarbageCode. (#1061) 2018-01-26 22:06:07 +01:00
Daniel Marjamäki 3ebedcd7f5 Fixed VS crash for torture test 2018-01-26 16:05:43 +01:00
Daniel Marjamäki a5f202360a Fixed crash for garbage code, found by fuzzing 2018-01-26 09:34:27 +01:00
Daniel Marjamäki 1b3248b0fc Dont write syntax error for lambda functions 2018-01-25 17:47:27 +01:00
Daniel Marjamäki c04557eb73 Fix crashes detected with fuzzing 2018-01-25 15:53:58 +01:00
IOBYTE 8ce496a127 Fix possible null pointer dereference cppcheck warning. (#1056) 2018-01-24 21:19:52 +01:00
Daniel Marjamäki 96e387a486 Fixed #8351 (segmentation fault on objective C code) 2018-01-24 18:06:11 +01:00
IOBYTE 558e0757c2 Fix simplifyTypedef crash on lambda. (#1054) 2018-01-24 15:04:33 +01:00
Daniel Marjamäki 0f561d0ed6 Fix crashes for garbage code, found by fuzzing 2018-01-24 13:53:03 +01:00
IOBYTE 4710d80a40 Fix #5766 (FP: typedef array throws off parser) (#1052)
* Fix #5766 (FP: typedef array throws off parser)

* Fix travis build.
2018-01-24 09:51:22 +01:00
IOBYTE e6d285d3ca Fixed #8357 (crash: cmake Tests/CMakeLib/testUTF8.cxx --debug --verbose) (#1046) 2018-01-22 08:06:56 +01:00
Martin Ettl 17a54681b8 tokenize: simplify empty string creation string("")->string(). 2018-01-21 16:36:57 +01:00
Alexey Eryomenko 913fdf44b6 member access operators are allowed inside the embedded SQL block when (#1043)
passing arguments for a query
2018-01-21 15:58:12 +01:00
Daniel Marjamäki af26f00e04 gui: update copyright message in about dialog 2018-01-14 15:46:20 +01:00
Daniel Marjamäki c4caee6b18 Updated copyright year 2018-01-14 15:37:52 +01:00
IOBYTE cefb2131c7 Add support for simple c++ 11 type ailases like: using INT = int; (#1024)
* Add support for simple c++ 11 type ailases like: using INT = int;

Only types supported by ValueType are supported. Complex types like
function pointers are not supported. Template type aliases are not
supported.

* Fix crash when type in using type alias is simplified away.

This fixes a crash when size_t is replaced with unsigned long in: using
size_t = unsigned long; by the tokenizer.

This does not fix the problem where Tokenizer::simplifyPlatformTypes()
simplifies away size_t in other cases.  This is only a problem when the
new type is different from the platform type.
2018-01-10 22:16:18 +01:00
Daniel Marjamäki 8c33a95b49 Refactoring: moved method from Tokenizer to TokenList 2018-01-07 14:07:34 +01:00
Daniel Marjamäki 1af69bd0d4 Tokenizer::createLinks2: fix link for >> 2018-01-01 15:20:21 +01:00
Daniel Marjamäki fa42a08a71 Make code less strict that looks for garbage template code. Before a template there might be unknown macros. 2017-12-31 16:25:41 +01:00