walkero
/
cppcheck
1
0
Fork 0
Code Issues Pull Requests Projects Releases 1 Wiki Activity
cppcheck/lib/token.cpp

2536 lines
81 KiB
C++
Raw Normal View History

Formatting: uniformize end of lines.
2008-12-18 22:28:57 +01:00
/*
Fixed Ticket #40, Check copyright texts in files, now that we have new developers.
2009-01-21 21:04:20 +01:00
* Cppcheck - A tool for static C/C++ code analysis
Update Copyright year
2021-03-21 20:58:32 +01:00
* Copyright (C) 2007-2021 Cppcheck team.
Formatting: uniformize end of lines.
2008-12-18 22:28:57 +01:00
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
Fix GPL comments in all files. ">." was missing from the end.
2009-09-27 17:08:31 +02:00
* along with this program. If not, see <http://www.gnu.org/licenses/>.
Formatting: uniformize end of lines.
2008-12-18 22:28:57 +01:00
*/
#include "token.h"
iwyu - include what you use
2017-05-27 04:33:47 +02:00
Fixed #8941 (False Positive: Variable 'f' is assigned a value that is never used.)
2019-01-20 13:20:23 +01:00
#include "astutils.h"
iwyu - include what you use
2017-05-27 04:33:47 +02:00
#include "library.h"
ValueFlow: Added utility functions getValueLE and getValueGE to simplify usage
2014-04-02 06:49:28 +02:00
#include "settings.h"
Dead pointer: Added checking for dead pointer usage when pointer alias local variable that has gone out of scope.
2014-08-05 06:24:23 +02:00
#include "symboldatabase.h"
Fixed compilation: Added missing include
2020-05-19 14:32:50 +02:00
#include "tokenlist.h"
Small refactorizations: - #include cleanup - Use std::array instead of std::vector - Do not create a stringstream to concatenate 4 strings - Use std::cout instead of printf
2015-11-29 10:49:10 +01:00
#include "utils.h"
allow token iteration in range for
2021-06-14 07:39:01 +02:00
#include "tokenrange.h"
Fix issue 9979: false positive: containerOutOfBounds with conditional resize (#3136)
2021-03-30 14:02:28 +02:00
#include "valueflow.h"
iwyu - include what you use
2017-05-27 04:33:47 +02:00
Refactorization: Moved code from header to source - from utils.h to new utils.cpp - from token.h to token.cpp - from valueflow.h to valueflow.cpp - from errorlogger.h to errorlogger.cpp
2020-05-10 20:32:59 +02:00
#include <algorithm>
Token::createMutualLinks(): introduce and use. No functional change.
2009-08-22 16:22:50 +02:00
#include <cassert>
iwyu - include what you use
2017-05-27 04:33:47 +02:00
#include <cctype>
Formatting: uniformize end of lines.
2008-12-18 22:28:57 +01:00
#include <cstring>
Fix issue 9979: false positive: containerOutOfBounds with conditional resize (#3136)
2021-03-30 14:02:28 +02:00
#include <functional>
Formatting: uniformize end of lines.
2008-12-18 22:28:57 +01:00
#include <iostream>
Fix ticket Add/Change #125 (the printout of the token list is wrong)
2009-03-03 21:17:23 +01:00
#include <map>
iwyu - include what you use
2017-05-27 04:33:47 +02:00
#include <set>
clarify calculation: Fixed FP for 'a = (*p ? 1 : 2)'
2013-12-28 11:02:39 +01:00
#include <stack>
iwyu - include what you use
2017-05-27 04:33:47 +02:00
#include <utility>
Ticket #5373: Keep track whether tokens come from a C or C++ file to properly handle %type% in Token::Match with "delete"
2014-03-08 20:56:39 +01:00
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
const std::list<ValueFlow::Value> TokenImpl::mEmptyValueList;
threadsafety: Fix static data in Token::values()
2017-08-13 14:15:24 +02:00
Fix crash bug #8579 (#1238) * Added declaration for deletePrevious function * Added definition for deletePrevious function * Fixed crash from deleteThis invalidating pointers The crash was caused by deleteThis() invalidating the pointer to a constant variable usage. This happened when a usage followed an assignment. This fixes bug #8579. * Added tokensFront to match tokensBack This means deletePrevious can set the list's front if necessary. * Initialised tokensFront in appropriate places * Switched to using default Token constructor * Switched to using Token default constructor * Switched to using default constructor for Token * Added missing argument to Token constructor * Changed to use default constructor for Tokens * Switched to using default constructor for Tokens * Switched to using default constructor for Token * Added new test for deleting front Token Also made sure to use the correct constructor for Token in other tests. * Syntax error * Replaced tokensFront and tokensBack with a struct This decreases the size of the Token class for performance purposes. * Replaced tokensFront and tokensBack with a struct * Added tokensFrontBack to destructor * Reworked to use TokensBackFront struct Also ran astyle. * Reworked to use TokenList's TokensFrontBack member * Reworked to use TokensFrontBack struct * Reworked to use TokensFrontBack struct * Reworked to work with TokensFrontBack struct * Removed unnecessary scope operator * Added missing parentheses * Fixed syntax error * Removed unnecessary constructor * Default constructor now 0-initialises everything This is safer for not using a temporary TokensFrontBack object, and doesn't use delegating constructors which aren't supported yet. * Fixed unsafe null check * Added missing explicit keyword * Fixing stylistic nits Removed default constructor as it has been superseded by the single-argument constructor with a default argument value. Renamed listEnds to tokensFrontBack. Fixed if statement that was supposed to be adding safety but would actually cause a crash if tokensFrontBack was null. * Fixing stylistic nits Removed default constructor and replaced it with a single-argument constructor with a default value. * Fixing stylistic nits Renamed _listEnds to _tokensFrontBack. * Fixing stylistic nits Renamed _listEnds to _tokensFrontBack.
2018-05-25 07:15:05 +02:00
Token::Token(TokensFrontBack *tokensFrontBack) :
renamed _tokensFrontBack to mTokensFrontBack
2018-06-16 16:22:35 +02:00
mTokensFrontBack(tokensFrontBack),
rename _previous and _next
2018-06-16 16:16:55 +02:00
mNext(nullptr),
mPrevious(nullptr),
Rename _link to mLink
2018-06-16 20:30:09 +02:00
mLink(nullptr),
Renamed _tokType to mTokType
2018-06-16 16:40:02 +02:00
mTokType(eNone),
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
mFlags(0)
Formatting: uniformize end of lines.
2008-12-18 22:28:57 +01:00
{
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
mImpl = new TokenImpl();
Formatting: uniformize end of lines.
2008-12-18 22:28:57 +01:00
}
Refactoring: Rename class "TOKEN" to "Token"
2009-01-03 21:29:20 +01:00
Token::~Token()
Formatting: uniformize end of lines.
2008-12-18 22:28:57 +01:00
{
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
delete mImpl;
Formatting: uniformize end of lines.
2008-12-18 22:28:57 +01:00
}
allow token iteration in range for
2021-06-14 07:39:01 +02:00
/*
Format with uncrustify (#3388)
2021-08-07 20:51:18 +02:00
* Get a TokenRange which starts at this token and contains every token following it in order up to but not including 't'
* e.g. for the sequence of tokens A B C D E, C.until(E) would yield the Range C D
* note t can be nullptr to iterate all the way to the end.
*/
allow token iteration in range for
2021-06-14 07:39:01 +02:00
ConstTokenRange Token::until(const Token* t) const
{
return ConstTokenRange(this, t);
}
Use std::unordered_* containers for faster lookups (#3052)
2021-01-16 13:52:09 +01:00
static const std::unordered_set<std::string> controlFlowKeywords = {
Replaced make_container by C++11 initializer lists
2018-04-08 22:54:10 +02:00
"goto",
"do",
"if",
"else",
"for",
"while",
"switch",
"case",
"break",
"continue",
"return"
};
Fixed #8263 (check-library incorrectly reports missing configuration for case when value is in parentheses)
2017-11-17 22:10:39 +01:00
Bugfix: Update token properties on string changes
2011-10-23 20:38:03 +02:00
void Token::update_property_info()
Formatting: uniformize end of lines.
2008-12-18 22:28:57 +01:00
{
Renamed Token::_str
2018-06-16 23:03:15 +02:00
setFlag(fIsControlFlowKeyword, controlFlowKeywords.find(mStr) != controlFlowKeywords.end());
Fixed #8263 (check-library incorrectly reports missing configuration for case when value is in parentheses)
2017-11-17 22:10:39 +01:00
Renamed Token::_str
2018-06-16 23:03:15 +02:00
if (!mStr.empty()) {
if (mStr == "true" || mStr == "false")
Memoize token nature computation
2017-10-15 01:27:47 +02:00
tokType(eBoolean);
Keep prefix in string and char literals (#2272) Keeping the prefix in the token allows cppcheck to print the correct string and char literals in debug and error messages. To achieve this, move some of the helper functions from token.cpp to utils.h so that checks that look at string and char literals can reuse them. This is a large part of this commit. Note that the only user visible change is that when string and char literals are printed in error messages, the prefix is now included. For example: int f() { return test.substr( 0 , 4 ) == U"Hello" ? 0 : 1 ; }; now prints U"Hello" instead of "Hello" in the error message.
2019-10-16 11:41:33 +02:00
else if (isStringLiteral(mStr))
Handle concatenated string and char literals This handles concatenated strings and characters from simplecpp. Previously, L'c' would be preprocessed to the tokens "L" and "'c'". cppcheck would then remove the "L" token and set "'c'" to be a wide character literal. Now, it needs to remove the prefix instead. When doing this, add handling of utf32 encoded literals (U) and UTF-8 encoded literals (u8).
2019-03-10 10:38:50 +01:00
tokType(eString);
Keep prefix in string and char literals (#2272) Keeping the prefix in the token allows cppcheck to print the correct string and char literals in debug and error messages. To achieve this, move some of the helper functions from token.cpp to utils.h so that checks that look at string and char literals can reuse them. This is a large part of this commit. Note that the only user visible change is that when string and char literals are printed in error messages, the prefix is now included. For example: int f() { return test.substr( 0 , 4 ) == U"Hello" ? 0 : 1 ; }; now prints U"Hello" instead of "Hello" in the error message.
2019-10-16 11:41:33 +02:00
else if (isCharLiteral(mStr))
Handle concatenated string and char literals This handles concatenated strings and characters from simplecpp. Previously, L'c' would be preprocessed to the tokens "L" and "'c'". cppcheck would then remove the "L" token and set "'c'" to be a wide character literal. Now, it needs to remove the prefix instead. When doing this, add handling of utf32 encoded literals (U) and UTF-8 encoded literals (u8).
2019-03-10 10:38:50 +01:00
tokType(eChar);
Renamed Token::_str
2018-06-16 23:03:15 +02:00
else if (std::isalpha((unsigned char)mStr[0]) || mStr[0] == '_' || mStr[0] == '$') { // Name
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
if (mImpl->mVarId)
Memoize token nature computation
2017-10-15 01:27:47 +02:00
tokType(eVariable);
Fixed Token::isKeyword: Actually set this flag to a reasonable value in all code paths
2020-05-17 17:25:33 +02:00
else if (mTokensFrontBack && mTokensFrontBack->list && mTokensFrontBack->list->isKeyword(mStr))
tokType(eKeyword);
Renamed _tokType to mTokType
2018-06-16 16:40:02 +02:00
else if (mTokType != eVariable && mTokType != eFunction && mTokType != eType && mTokType != eKeyword)
Memoize token nature computation
2017-10-15 01:27:47 +02:00
tokType(eName);
Renamed Token::_str
2018-06-16 23:03:15 +02:00
} else if (std::isdigit((unsigned char)mStr[0]) || (mStr.length() > 1 && mStr[0] == '-' && std::isdigit((unsigned char)mStr[1])))
Memoize token nature computation
2017-10-15 01:27:47 +02:00
tokType(eNumber);
Renamed Token::_str
2018-06-16 23:03:15 +02:00
else if (mStr == "=" || mStr == "<<=" || mStr == ">>=" ||
(mStr.size() == 2U && mStr[1] == '=' && std::strchr("+-*/%&^|", mStr[0])))
Memoize token nature computation
2017-10-15 01:27:47 +02:00
tokType(eAssignmentOp);
Renamed Token::_str
2018-06-16 23:03:15 +02:00
else if (mStr.size() == 1 && mStr.find_first_of(",[]()?:") != std::string::npos)
Memoize token nature computation
2017-10-15 01:27:47 +02:00
tokType(eExtendedOp);
Renamed Token::_str
2018-06-16 23:03:15 +02:00
else if (mStr=="<<" || mStr==">>" || (mStr.size()==1 && mStr.find_first_of("+-*/%") != std::string::npos))
Memoize token nature computation
2017-10-15 01:27:47 +02:00
tokType(eArithmeticalOp);
Renamed Token::_str
2018-06-16 23:03:15 +02:00
else if (mStr.size() == 1 && mStr.find_first_of("&|^~") != std::string::npos)
Memoize token nature computation
2017-10-15 01:27:47 +02:00
tokType(eBitOp);
Renamed Token::_str
2018-06-16 23:03:15 +02:00
else if (mStr.size() <= 2 &&
(mStr == "&&" ||
mStr == "||" ||
mStr == "!"))
Memoize token nature computation
2017-10-15 01:27:47 +02:00
tokType(eLogicalOp);
Renamed Token::_str
2018-06-16 23:03:15 +02:00
else if (mStr.size() <= 2 && !mLink &&
(mStr == "==" ||
mStr == "!=" ||
Format with uncrustify (#3388)
2021-08-07 20:51:18 +02:00
mStr == "<" ||
Renamed Token::_str
2018-06-16 23:03:15 +02:00
mStr == "<=" ||
Format with uncrustify (#3388)
2021-08-07 20:51:18 +02:00
mStr == ">" ||
Renamed Token::_str
2018-06-16 23:03:15 +02:00
mStr == ">="))
Memoize token nature computation
2017-10-15 01:27:47 +02:00
tokType(eComparisonOp);
Handle c++20 spaceship operator
2021-04-22 19:15:22 +02:00
else if (mStr == "<=>")
tokType(eComparisonOp);
Renamed Token::_str
2018-06-16 23:03:15 +02:00
else if (mStr.size() == 2 &&
(mStr == "++" ||
mStr == "--"))
Memoize token nature computation
2017-10-15 01:27:47 +02:00
tokType(eIncDecOp);
Renamed Token::_str
2018-06-16 23:03:15 +02:00
else if (mStr.size() == 1 && (mStr.find_first_of("{}") != std::string::npos || (mLink && mStr.find_first_of("<>") != std::string::npos)))
Memoize token nature computation
2017-10-15 01:27:47 +02:00
tokType(eBracket);
make ellipsis ... a single token (#2143) * make ellipsis ... a single token Using cppcheck -E to preprocess code with ellipsis produces output that can't be compiled because ... is split into 3 tokens. * try to fix addon
2019-09-04 08:07:30 +02:00
else if (mStr == "...")
tokType(eEllipsis);
Fixed #2676 (setVarId: variables with global scope qualification :: don't get varids)
2011-04-18 06:56:39 +02:00
else
Memoize token nature computation
2017-10-15 01:27:47 +02:00
tokType(eOther);
Bugfix: Update token properties on string changes
2011-10-23 20:38:03 +02:00
} else {
Memoize token nature computation
2017-10-15 01:27:47 +02:00
tokType(eNone);
Fixed #2676 (setVarId: variables with global scope qualification :: don't get varids)
2011-04-18 06:56:39 +02:00
}
Cache result of tok->isStandardType() Also contains test order improvements from Johan Samuelson (#3116). Run time went down from 15.15s to 14.95s for the case below. Benchmarked using these settings: - CXXFLAGS="-O2 -g -NDEBUG" - time -p cppcheck.git -q --std=posix --enable=all --inconclusive -Dfoo . on the "rpm" codebase.
2011-11-09 21:45:59 +01:00
Handle prefixed strings and characters in Token (#1742) This makes it possible to call getStrLength() and similar functions before the tokenizer is called.
2019-03-18 06:18:25 +01:00
update_property_char_string_literal();
Cache result of tok->isStandardType() Also contains test order improvements from Johan Samuelson (#3116). Run time went down from 15.15s to 14.95s for the case below. Benchmarked using these settings: - CXXFLAGS="-O2 -g -NDEBUG" - time -p cppcheck.git -q --std=posix --enable=all --inconclusive -Dfoo . on the "rpm" codebase.
2011-11-09 21:45:59 +01:00
update_property_isStandardType();
}
Use std::unordered_* containers for faster lookups (#3052)
2021-01-16 13:52:09 +01:00
static const std::unordered_set<std::string> stdTypes = { "bool"
Running astlye [ci skip]
2021-01-22 21:47:24 +01:00
, "_Bool"
, "char"
, "double"
, "float"
, "int"
, "long"
, "short"
, "size_t"
, "void"
, "wchar_t"
Format with uncrustify (#3388)
2021-08-07 20:51:18 +02:00
};
Fix (potential) multi-threading issues by moving static local vars (non-POD-type) to file scope
2015-06-10 21:14:17 +02:00
Cache result of tok->isStandardType() Also contains test order improvements from Johan Samuelson (#3116). Run time went down from 15.15s to 14.95s for the case below. Benchmarked using these settings: - CXXFLAGS="-O2 -g -NDEBUG" - time -p cppcheck.git -q --std=posix --enable=all --inconclusive -Dfoo . on the "rpm" codebase.
2011-11-09 21:45:59 +01:00
void Token::update_property_isStandardType()
{
Fixed #5764 (Store Token flags in a single variable)
2014-05-06 06:35:48 +02:00
isStandardType(false);
Cache result of tok->isStandardType() Also contains test order improvements from Johan Samuelson (#3116). Run time went down from 15.15s to 14.95s for the case below. Benchmarked using these settings: - CXXFLAGS="-O2 -g -NDEBUG" - time -p cppcheck.git -q --std=posix --enable=all --inconclusive -Dfoo . on the "rpm" codebase.
2011-11-09 21:45:59 +01:00
Renamed Token::_str
2018-06-16 23:03:15 +02:00
if (mStr.size() < 3)
Cache result of tok->isStandardType() Also contains test order improvements from Johan Samuelson (#3116). Run time went down from 15.15s to 14.95s for the case below. Benchmarked using these settings: - CXXFLAGS="-O2 -g -NDEBUG" - time -p cppcheck.git -q --std=posix --enable=all --inconclusive -Dfoo . on the "rpm" codebase.
2011-11-09 21:45:59 +01:00
return;
Renamed Token::_str
2018-06-16 23:03:15 +02:00
if (stdTypes.find(mStr)!=stdTypes.end()) {
Fixed #5764 (Store Token flags in a single variable)
2014-05-06 06:35:48 +02:00
isStandardType(true);
Memoize token nature computation
2017-10-15 01:27:47 +02:00
tokType(eType);
Cache result of tok->isStandardType() Also contains test order improvements from Johan Samuelson (#3116). Run time went down from 15.15s to 14.95s for the case below. Benchmarked using these settings: - CXXFLAGS="-O2 -g -NDEBUG" - time -p cppcheck.git -q --std=posix --enable=all --inconclusive -Dfoo . on the "rpm" codebase.
2011-11-09 21:45:59 +01:00
}
Bugfix: Update token properties on string changes
2011-10-23 20:38:03 +02:00
}
Improving simplifyKnownVariables() to simplify bool variables used inside if() like int values are simplified.
2008-12-21 14:58:56 +01:00
Handle prefixed strings and characters in Token (#1742) This makes it possible to call getStrLength() and similar functions before the tokenizer is called.
2019-03-18 06:18:25 +01:00
void Token::update_property_char_string_literal()
{
Keep prefix in string and char literals (#2272) Keeping the prefix in the token allows cppcheck to print the correct string and char literals in debug and error messages. To achieve this, move some of the helper functions from token.cpp to utils.h so that checks that look at string and char literals can reuse them. This is a large part of this commit. Note that the only user visible change is that when string and char literals are printed in error messages, the prefix is now included. For example: int f() { return test.substr( 0 , 4 ) == U"Hello" ? 0 : 1 ; }; now prints U"Hello" instead of "Hello" in the error message.
2019-10-16 11:41:33 +02:00
if (mTokType != Token::eString && mTokType != Token::eChar)
Handle prefixed strings and characters in Token (#1742) This makes it possible to call getStrLength() and similar functions before the tokenizer is called.
2019-03-18 06:18:25 +01:00
return;
Keep prefix in string and char literals (#2272) Keeping the prefix in the token allows cppcheck to print the correct string and char literals in debug and error messages. To achieve this, move some of the helper functions from token.cpp to utils.h so that checks that look at string and char literals can reuse them. This is a large part of this commit. Note that the only user visible change is that when string and char literals are printed in error messages, the prefix is now included. For example: int f() { return test.substr( 0 , 4 ) == U"Hello" ? 0 : 1 ; }; now prints U"Hello" instead of "Hello" in the error message.
2019-10-16 11:41:33 +02:00
isLong(((mTokType == Token::eString) && isPrefixStringCharLiteral(mStr, '"', "L")) ||
((mTokType == Token::eChar) && isPrefixStringCharLiteral(mStr, '\'', "L")));
Handle prefixed strings and characters in Token (#1742) This makes it possible to call getStrLength() and similar functions before the tokenizer is called.
2019-03-18 06:18:25 +01:00
}
Cache result of tok->isStandardType() Also contains test order improvements from Johan Samuelson (#3116). Run time went down from 15.15s to 14.95s for the case below. Benchmarked using these settings: - CXXFLAGS="-O2 -g -NDEBUG" - time -p cppcheck.git -q --std=posix --enable=all --inconclusive -Dfoo . on the "rpm" codebase.
2011-11-09 21:45:59 +01:00
Refactoring: Renamed CheckNullPointer::isPointer to Token::isUpperCaseName
2012-06-21 19:00:53 +02:00
bool Token::isUpperCaseName() const
{
if (!isName())
return false;
use range loops / constness (#2181) * use range loops / constness * platform.cpp: avoid shadowed variable
2019-09-19 20:29:33 +02:00
for (char i : mStr) {
if (std::islower(i))
Refactoring: Renamed CheckNullPointer::isPointer to Token::isUpperCaseName
2012-06-21 19:00:53 +02:00
return false;
}
return true;
}
performance enhancement (patch submitted by davidmiller in ticket 231)
2009-03-28 20:33:55 +01:00
void Token::concatStr(std::string const& b)
{
Refactorization: Use std::string::pop_back Merged from LCppC.
2021-02-17 22:39:29 +01:00
mStr.pop_back();
Keep prefix in string and char literals (#2272) Keeping the prefix in the token allows cppcheck to print the correct string and char literals in debug and error messages. To achieve this, move some of the helper functions from token.cpp to utils.h so that checks that look at string and char literals can reuse them. This is a large part of this commit. Note that the only user visible change is that when string and char literals are printed in error messages, the prefix is now included. For example: int f() { return test.substr( 0 , 4 ) == U"Hello" ? 0 : 1 ; }; now prints U"Hello" instead of "Hello" in the error message.
2019-10-16 11:41:33 +02:00
mStr.append(getStringLiteral(b) + "\"");
Bugfix: Update token properties on string changes
2011-10-23 20:38:03 +02:00
Token::ConcatStr(): Handle mixed string literals (#2337) Improve handling of adjacent string literals of different types. Example of adjacent string literals: "ab" L"cd". In C89, C++98 and C++03, this is undefined. As of C99 and C++11, this is well defined and the two string literals are concatenated to L"abcd". C11 and C++11 introduces the utf16, utf32 and (C++ only) utf8 string types. Concatenating any of these with a regular c-string works exactely as the wide string example above. The result of having two adjacent string literals with different prefix is implementation defined, unless one is an UTF-8 string literal and the other is a wide string literal. In this case the behaviour is undefined. Ignore the undefined and ill-formed programs (this behaviour is unchanged) and make sure that concatenating a plain c string literal with a prefixed one works correct (in C99 and C++11 and later versions). It also makes the behaviour consistent since previously, "ab" L"cd" would result in "abcd" while L"ab" "cd" would result in L"abcd". It also means the somewhat awkward updatePropertiesConcatStr() test can be removed since the added tests would not work if update_properties() was not called in concatStr(). Since the prefix is stored in the token, testing the type of the string is not relevant in TestSimplifyTokens. It is tested extensively in TestToken::stringTypes().
2019-11-08 08:03:45 +01:00
if (isCChar() && isStringLiteral(b) && b[0] != '"') {
mStr.insert(0, b.substr(0, b.find('"')));
}
Bugfix: Update token properties on string changes
2011-10-23 20:38:03 +02:00
update_property_info();
performance enhancement (patch submitted by davidmiller in ticket 231)
2009-03-28 20:33:55 +01:00
}
Token::strValue(): added const modifier. Correction for 241f585d34cb6f645d112c5e0cc54fe852c21c3b commit. No functional change.
2009-09-13 10:02:23 +02:00
std::string Token::strValue() const
Fix #594 (division by zero not detected when using atol or atof) http://sourceforge.net/apps/trac/cppcheck/ticket/594 Simplify atol("0") into 0 (and other atol() calls also)
2009-09-12 22:54:47 +02:00
{
Renamed _tokType to mTokType
2018-06-16 16:40:02 +02:00
assert(mTokType == eString);
Keep prefix in string and char literals (#2272) Keeping the prefix in the token allows cppcheck to print the correct string and char literals in debug and error messages. To achieve this, move some of the helper functions from token.cpp to utils.h so that checks that look at string and char literals can reuse them. This is a large part of this commit. Note that the only user visible change is that when string and char literals are printed in error messages, the prefix is now included. For example: int f() { return test.substr( 0 , 4 ) == U"Hello" ? 0 : 1 ; }; now prints U"Hello" instead of "Hello" in the error message.
2019-10-16 11:41:33 +02:00
std::string ret(getStringLiteral(mStr));
Token::strValue: fixed handling of backslash
2015-06-07 11:25:33 +02:00
std::string::size_type pos = 0U;
Added pointer to Type to Token (similar to Token::Variable() and Token::function()): - Accessible via Token::type() - Renamed former Token::type() to Token::tokType() - Removed SymbolDatabase::isClassOrStruct()
2015-08-14 20:46:13 +02:00
while ((pos = ret.find('\\', pos)) != std::string::npos) {
Token::strValue: fixed handling of backslash
2015-06-07 11:25:33 +02:00
ret.erase(pos,1U);
if (ret[pos] >= 'a') {
if (ret[pos] == 'n')
ret[pos] = '\n';
else if (ret[pos] == 'r')
ret[pos] = '\r';
else if (ret[pos] == 't')
ret[pos] = '\t';
}
if (ret[pos] == '0')
return ret.substr(0,pos);
pos++;
}
return ret;
Fix #594 (division by zero not detected when using atol or atof) http://sourceforge.net/apps/trac/cppcheck/ticket/594 Simplify atol("0") into 0 (and other atol() calls also)
2009-09-12 22:54:47 +02:00
}
Mark members and arguments with 'nonneg'
2019-07-14 12:22:33 +02:00
void Token::deleteNext(nonneg int count)
Formatting: uniformize end of lines.
2008-12-18 22:28:57 +01:00
{
Token deleteNext/deletePrevious parameters
2019-07-13 15:47:53 +02:00
while (mNext && count > 0) {
rename _previous and _next
2018-06-16 16:16:55 +02:00
Token *n = mNext;
Fixed #8154 (heap use after free in tokenlist)
2017-08-25 17:17:19 +02:00
// #8154 we are about to be unknown -> destroy the link to us
Rename _link to mLink
2018-06-16 20:30:09 +02:00
if (n->mLink && n->mLink->mLink == n)
n->mLink->link(nullptr);
Fixed #8154 (heap use after free in tokenlist)
2017-08-25 17:17:19 +02:00
rename _previous and _next
2018-06-16 16:16:55 +02:00
mNext = n->next();
Extend 'Token::deleteNext' by introducing a new parameter which determines how many tokens should be deleted. It's still not used, though.
2011-12-07 23:36:11 +01:00
delete n;
Token deleteNext/deletePrevious parameters
2019-07-13 15:47:53 +02:00
--count;
Extend 'Token::deleteNext' by introducing a new parameter which determines how many tokens should be deleted. It's still not used, though.
2011-12-07 23:36:11 +01:00
}
rename _previous and _next
2018-06-16 16:16:55 +02:00
if (mNext)
mNext->previous(this);
renamed _tokensFrontBack to mTokensFrontBack
2018-06-16 16:22:35 +02:00
else if (mTokensFrontBack)
mTokensFrontBack->back = this;
Fix crash bug #8579 (#1238) * Added declaration for deletePrevious function * Added definition for deletePrevious function * Fixed crash from deleteThis invalidating pointers The crash was caused by deleteThis() invalidating the pointer to a constant variable usage. This happened when a usage followed an assignment. This fixes bug #8579. * Added tokensFront to match tokensBack This means deletePrevious can set the list's front if necessary. * Initialised tokensFront in appropriate places * Switched to using default Token constructor * Switched to using Token default constructor * Switched to using default constructor for Token * Added missing argument to Token constructor * Changed to use default constructor for Tokens * Switched to using default constructor for Tokens * Switched to using default constructor for Token * Added new test for deleting front Token Also made sure to use the correct constructor for Token in other tests. * Syntax error * Replaced tokensFront and tokensBack with a struct This decreases the size of the Token class for performance purposes. * Replaced tokensFront and tokensBack with a struct * Added tokensFrontBack to destructor * Reworked to use TokensBackFront struct Also ran astyle. * Reworked to use TokenList's TokensFrontBack member * Reworked to use TokensFrontBack struct * Reworked to use TokensFrontBack struct * Reworked to work with TokensFrontBack struct * Removed unnecessary scope operator * Added missing parentheses * Fixed syntax error * Removed unnecessary constructor * Default constructor now 0-initialises everything This is safer for not using a temporary TokensFrontBack object, and doesn't use delegating constructors which aren't supported yet. * Fixed unsafe null check * Added missing explicit keyword * Fixing stylistic nits Removed default constructor as it has been superseded by the single-argument constructor with a default argument value. Renamed listEnds to tokensFrontBack. Fixed if statement that was supposed to be adding safety but would actually cause a crash if tokensFrontBack was null. * Fixing stylistic nits Removed default constructor and replaced it with a single-argument constructor with a default value. * Fixing stylistic nits Renamed _listEnds to _tokensFrontBack. * Fixing stylistic nits Renamed _listEnds to _tokensFrontBack.
2018-05-25 07:15:05 +02:00
}
Mark members and arguments with 'nonneg'
2019-07-14 12:22:33 +02:00
void Token::deletePrevious(nonneg int count)
Fix crash bug #8579 (#1238) * Added declaration for deletePrevious function * Added definition for deletePrevious function * Fixed crash from deleteThis invalidating pointers The crash was caused by deleteThis() invalidating the pointer to a constant variable usage. This happened when a usage followed an assignment. This fixes bug #8579. * Added tokensFront to match tokensBack This means deletePrevious can set the list's front if necessary. * Initialised tokensFront in appropriate places * Switched to using default Token constructor * Switched to using Token default constructor * Switched to using default constructor for Token * Added missing argument to Token constructor * Changed to use default constructor for Tokens * Switched to using default constructor for Tokens * Switched to using default constructor for Token * Added new test for deleting front Token Also made sure to use the correct constructor for Token in other tests. * Syntax error * Replaced tokensFront and tokensBack with a struct This decreases the size of the Token class for performance purposes. * Replaced tokensFront and tokensBack with a struct * Added tokensFrontBack to destructor * Reworked to use TokensBackFront struct Also ran astyle. * Reworked to use TokenList's TokensFrontBack member * Reworked to use TokensFrontBack struct * Reworked to use TokensFrontBack struct * Reworked to work with TokensFrontBack struct * Removed unnecessary scope operator * Added missing parentheses * Fixed syntax error * Removed unnecessary constructor * Default constructor now 0-initialises everything This is safer for not using a temporary TokensFrontBack object, and doesn't use delegating constructors which aren't supported yet. * Fixed unsafe null check * Added missing explicit keyword * Fixing stylistic nits Removed default constructor as it has been superseded by the single-argument constructor with a default argument value. Renamed listEnds to tokensFrontBack. Fixed if statement that was supposed to be adding safety but would actually cause a crash if tokensFrontBack was null. * Fixing stylistic nits Removed default constructor and replaced it with a single-argument constructor with a default value. * Fixing stylistic nits Renamed _listEnds to _tokensFrontBack. * Fixing stylistic nits Renamed _listEnds to _tokensFrontBack.
2018-05-25 07:15:05 +02:00
{
Token deleteNext/deletePrevious parameters
2019-07-13 15:47:53 +02:00
while (mPrevious && count > 0) {
rename _previous and _next
2018-06-16 16:16:55 +02:00
Token *p = mPrevious;
Fix crash bug #8579 (#1238) * Added declaration for deletePrevious function * Added definition for deletePrevious function * Fixed crash from deleteThis invalidating pointers The crash was caused by deleteThis() invalidating the pointer to a constant variable usage. This happened when a usage followed an assignment. This fixes bug #8579. * Added tokensFront to match tokensBack This means deletePrevious can set the list's front if necessary. * Initialised tokensFront in appropriate places * Switched to using default Token constructor * Switched to using Token default constructor * Switched to using default constructor for Token * Added missing argument to Token constructor * Changed to use default constructor for Tokens * Switched to using default constructor for Tokens * Switched to using default constructor for Token * Added new test for deleting front Token Also made sure to use the correct constructor for Token in other tests. * Syntax error * Replaced tokensFront and tokensBack with a struct This decreases the size of the Token class for performance purposes. * Replaced tokensFront and tokensBack with a struct * Added tokensFrontBack to destructor * Reworked to use TokensBackFront struct Also ran astyle. * Reworked to use TokenList's TokensFrontBack member * Reworked to use TokensFrontBack struct * Reworked to use TokensFrontBack struct * Reworked to work with TokensFrontBack struct * Removed unnecessary scope operator * Added missing parentheses * Fixed syntax error * Removed unnecessary constructor * Default constructor now 0-initialises everything This is safer for not using a temporary TokensFrontBack object, and doesn't use delegating constructors which aren't supported yet. * Fixed unsafe null check * Added missing explicit keyword * Fixing stylistic nits Removed default constructor as it has been superseded by the single-argument constructor with a default argument value. Renamed listEnds to tokensFrontBack. Fixed if statement that was supposed to be adding safety but would actually cause a crash if tokensFrontBack was null. * Fixing stylistic nits Removed default constructor and replaced it with a single-argument constructor with a default value. * Fixing stylistic nits Renamed _listEnds to _tokensFrontBack. * Fixing stylistic nits Renamed _listEnds to _tokensFrontBack.
2018-05-25 07:15:05 +02:00
// #8154 we are about to be unknown -> destroy the link to us
Rename _link to mLink
2018-06-16 20:30:09 +02:00
if (p->mLink && p->mLink->mLink == p)
p->mLink->link(nullptr);
Fix crash bug #8579 (#1238) * Added declaration for deletePrevious function * Added definition for deletePrevious function * Fixed crash from deleteThis invalidating pointers The crash was caused by deleteThis() invalidating the pointer to a constant variable usage. This happened when a usage followed an assignment. This fixes bug #8579. * Added tokensFront to match tokensBack This means deletePrevious can set the list's front if necessary. * Initialised tokensFront in appropriate places * Switched to using default Token constructor * Switched to using Token default constructor * Switched to using default constructor for Token * Added missing argument to Token constructor * Changed to use default constructor for Tokens * Switched to using default constructor for Tokens * Switched to using default constructor for Token * Added new test for deleting front Token Also made sure to use the correct constructor for Token in other tests. * Syntax error * Replaced tokensFront and tokensBack with a struct This decreases the size of the Token class for performance purposes. * Replaced tokensFront and tokensBack with a struct * Added tokensFrontBack to destructor * Reworked to use TokensBackFront struct Also ran astyle. * Reworked to use TokenList's TokensFrontBack member * Reworked to use TokensFrontBack struct * Reworked to use TokensFrontBack struct * Reworked to work with TokensFrontBack struct * Removed unnecessary scope operator * Added missing parentheses * Fixed syntax error * Removed unnecessary constructor * Default constructor now 0-initialises everything This is safer for not using a temporary TokensFrontBack object, and doesn't use delegating constructors which aren't supported yet. * Fixed unsafe null check * Added missing explicit keyword * Fixing stylistic nits Removed default constructor as it has been superseded by the single-argument constructor with a default argument value. Renamed listEnds to tokensFrontBack. Fixed if statement that was supposed to be adding safety but would actually cause a crash if tokensFrontBack was null. * Fixing stylistic nits Removed default constructor and replaced it with a single-argument constructor with a default value. * Fixing stylistic nits Renamed _listEnds to _tokensFrontBack. * Fixing stylistic nits Renamed _listEnds to _tokensFrontBack.
2018-05-25 07:15:05 +02:00
rename _previous and _next
2018-06-16 16:16:55 +02:00
mPrevious = p->previous();
Fix crash bug #8579 (#1238) * Added declaration for deletePrevious function * Added definition for deletePrevious function * Fixed crash from deleteThis invalidating pointers The crash was caused by deleteThis() invalidating the pointer to a constant variable usage. This happened when a usage followed an assignment. This fixes bug #8579. * Added tokensFront to match tokensBack This means deletePrevious can set the list's front if necessary. * Initialised tokensFront in appropriate places * Switched to using default Token constructor * Switched to using Token default constructor * Switched to using default constructor for Token * Added missing argument to Token constructor * Changed to use default constructor for Tokens * Switched to using default constructor for Tokens * Switched to using default constructor for Token * Added new test for deleting front Token Also made sure to use the correct constructor for Token in other tests. * Syntax error * Replaced tokensFront and tokensBack with a struct This decreases the size of the Token class for performance purposes. * Replaced tokensFront and tokensBack with a struct * Added tokensFrontBack to destructor * Reworked to use TokensBackFront struct Also ran astyle. * Reworked to use TokenList's TokensFrontBack member * Reworked to use TokensFrontBack struct * Reworked to use TokensFrontBack struct * Reworked to work with TokensFrontBack struct * Removed unnecessary scope operator * Added missing parentheses * Fixed syntax error * Removed unnecessary constructor * Default constructor now 0-initialises everything This is safer for not using a temporary TokensFrontBack object, and doesn't use delegating constructors which aren't supported yet. * Fixed unsafe null check * Added missing explicit keyword * Fixing stylistic nits Removed default constructor as it has been superseded by the single-argument constructor with a default argument value. Renamed listEnds to tokensFrontBack. Fixed if statement that was supposed to be adding safety but would actually cause a crash if tokensFrontBack was null. * Fixing stylistic nits Removed default constructor and replaced it with a single-argument constructor with a default value. * Fixing stylistic nits Renamed _listEnds to _tokensFrontBack. * Fixing stylistic nits Renamed _listEnds to _tokensFrontBack.
2018-05-25 07:15:05 +02:00
delete p;
Token deleteNext/deletePrevious parameters
2019-07-13 15:47:53 +02:00
--count;
Fix crash bug #8579 (#1238) * Added declaration for deletePrevious function * Added definition for deletePrevious function * Fixed crash from deleteThis invalidating pointers The crash was caused by deleteThis() invalidating the pointer to a constant variable usage. This happened when a usage followed an assignment. This fixes bug #8579. * Added tokensFront to match tokensBack This means deletePrevious can set the list's front if necessary. * Initialised tokensFront in appropriate places * Switched to using default Token constructor * Switched to using Token default constructor * Switched to using default constructor for Token * Added missing argument to Token constructor * Changed to use default constructor for Tokens * Switched to using default constructor for Tokens * Switched to using default constructor for Token * Added new test for deleting front Token Also made sure to use the correct constructor for Token in other tests. * Syntax error * Replaced tokensFront and tokensBack with a struct This decreases the size of the Token class for performance purposes. * Replaced tokensFront and tokensBack with a struct * Added tokensFrontBack to destructor * Reworked to use TokensBackFront struct Also ran astyle. * Reworked to use TokenList's TokensFrontBack member * Reworked to use TokensFrontBack struct * Reworked to use TokensFrontBack struct * Reworked to work with TokensFrontBack struct * Removed unnecessary scope operator * Added missing parentheses * Fixed syntax error * Removed unnecessary constructor * Default constructor now 0-initialises everything This is safer for not using a temporary TokensFrontBack object, and doesn't use delegating constructors which aren't supported yet. * Fixed unsafe null check * Added missing explicit keyword * Fixing stylistic nits Removed default constructor as it has been superseded by the single-argument constructor with a default argument value. Renamed listEnds to tokensFrontBack. Fixed if statement that was supposed to be adding safety but would actually cause a crash if tokensFrontBack was null. * Fixing stylistic nits Removed default constructor and replaced it with a single-argument constructor with a default value. * Fixing stylistic nits Renamed _listEnds to _tokensFrontBack. * Fixing stylistic nits Renamed _listEnds to _tokensFrontBack.
2018-05-25 07:15:05 +02:00
}
rename _previous and _next
2018-06-16 16:16:55 +02:00
if (mPrevious)
mPrevious->next(this);
renamed _tokensFrontBack to mTokensFrontBack
2018-06-16 16:22:35 +02:00
else if (mTokensFrontBack)
mTokensFrontBack->front = this;
Formatting: uniformize end of lines.
2008-12-18 22:28:57 +01:00
}
Fixed #6321: Implemented function Token::swapWithNext().
2014-12-27 10:53:26 +01:00
void Token::swapWithNext()
{
rename _previous and _next
2018-06-16 16:16:55 +02:00
if (mNext) {
Renamed Token::_str
2018-06-16 23:03:15 +02:00
std::swap(mStr, mNext->mStr);
Renamed _tokType to mTokType
2018-06-16 16:40:02 +02:00
std::swap(mTokType, mNext->mTokType);
rename _previous and _next
2018-06-16 16:16:55 +02:00
std::swap(mFlags, mNext->mFlags);
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
std::swap(mImpl, mNext->mImpl);
Optimization: Reduced memory consumption of token list - allocate memory for mTemplateSimplifierPointers only if necessary - reordered member variables in TokenImpl
2020-05-15 09:02:59 +02:00
if (mImpl->mTemplateSimplifierPointers)
for (auto *templateSimplifierPointer : *mImpl->mTemplateSimplifierPointers) {
templateSimplifierPointer->token(this);
}
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
Optimization: Reduced memory consumption of token list - allocate memory for mTemplateSimplifierPointers only if necessary - reordered member variables in TokenImpl
2020-05-15 09:02:59 +02:00
if (mNext->mImpl->mTemplateSimplifierPointers)
for (auto *templateSimplifierPointer : *mNext->mImpl->mTemplateSimplifierPointers) {
templateSimplifierPointer->token(mNext);
}
Rename _link to mLink
2018-06-16 20:30:09 +02:00
if (mNext->mLink)
mNext->mLink->mLink = this;
if (this->mLink)
this->mLink->mLink = mNext;
std::swap(mLink, mNext->mLink);
Fixed #6321: Implemented function Token::swapWithNext().
2014-12-27 10:53:26 +01:00
}
}
Refactor Token::deleteThis()
2017-08-25 23:30:04 +02:00
void Token::takeData(Token *fromToken)
{
Renamed Token::_str
2018-06-16 23:03:15 +02:00
mStr = fromToken->mStr;
Renamed _tokType to mTokType
2018-06-16 16:40:02 +02:00
tokType(fromToken->mTokType);
rename _flags to mFlags
2018-06-16 16:14:34 +02:00
mFlags = fromToken->mFlags;
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
delete mImpl;
mImpl = fromToken->mImpl;
fromToken->mImpl = nullptr;
Optimization: Reduced memory consumption of token list - allocate memory for mTemplateSimplifierPointers only if necessary - reordered member variables in TokenImpl
2020-05-15 09:02:59 +02:00
if (mImpl->mTemplateSimplifierPointers)
for (auto *templateSimplifierPointer : *mImpl->mTemplateSimplifierPointers) {
templateSimplifierPointer->token(this);
}
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
mLink = fromToken->mLink;
Rename _link to mLink
2018-06-16 20:30:09 +02:00
if (mLink)
mLink->link(this);
Refactor Token::deleteThis()
2017-08-25 23:30:04 +02:00
}
Fix ticket #151 (Handling of namespaces) http://apps.sourceforge.net/trac/cppcheck/ticket/151
2009-03-13 00:07:05 +01:00
void Token::deleteThis()
{
rename _previous and _next
2018-06-16 16:16:55 +02:00
if (mNext) { // Copy next to this and delete next
takeData(mNext);
mNext->link(nullptr); // mark as unlinked
Fix ticket #151 (Handling of namespaces) http://apps.sourceforge.net/trac/cppcheck/ticket/151
2009-03-13 00:07:05 +01:00
deleteNext();
Fixed #9860 (unused template not removed properly by default)
2020-11-22 16:43:36 +01:00
} else if (mPrevious) { // Copy previous to this and delete previous
rename _previous and _next
2018-06-16 16:16:55 +02:00
takeData(mPrevious);
Fixed #9860 (unused template not removed properly by default)
2020-11-22 16:43:36 +01:00
mPrevious->link(nullptr);
deletePrevious();
changed the astyle formatting flags
2011-10-13 20:53:06 +02:00
} else {
Fix ticket #151 (Handling of namespaces) http://apps.sourceforge.net/trac/cppcheck/ticket/151
2009-03-13 00:07:05 +01:00
// We are the last token in the list, we can't delete
Improved Tokens and Tokenizer: - Better handling of deleteThis() as last element of token list - Code simplified
2012-02-13 17:44:08 +01:00
// ourselves, so just make us empty
Fixed #9860 (unused template not removed properly by default)
2020-11-22 16:43:36 +01:00
str(";");
Fix ticket #151 (Handling of namespaces) http://apps.sourceforge.net/trac/cppcheck/ticket/151
2009-03-13 00:07:05 +01:00
}
}
Fix ticket #25 (simplify "void f(x) int x; {" into "void f(int x) {")
2009-01-26 23:26:50 +01:00
void Token::replace(Token *replaceThis, Token *start, Token *end)
{
// Fix the whole in the old location of start and end
Reverted 'astyle fix'. Those changes are not in sync with my astyle configuration/setup.
2010-04-02 07:30:58 +02:00
if (start->previous())
Fix ticket #334 (segmentation fault on boost 1.39.0) http://apps.sourceforge.net/trac/cppcheck/ticket/334
2009-05-26 22:22:00 +02:00
start->previous()->next(end->next());
Reverted 'astyle fix'. Those changes are not in sync with my astyle configuration/setup.
2010-04-02 07:30:58 +02:00
if (end->next())
Fix ticket #334 (segmentation fault on boost 1.39.0) http://apps.sourceforge.net/trac/cppcheck/ticket/334
2009-05-26 22:22:00 +02:00
end->next()->previous(start->previous());
Fix ticket #25 (simplify "void f(x) int x; {" into "void f(int x) {")
2009-01-26 23:26:50 +01:00
// Move start and end to their new location
Reverted 'astyle fix'. Those changes are not in sync with my astyle configuration/setup.
2010-04-02 07:30:58 +02:00
if (replaceThis->previous())
Fix ticket #334 (segmentation fault on boost 1.39.0) http://apps.sourceforge.net/trac/cppcheck/ticket/334
2009-05-26 22:22:00 +02:00
replaceThis->previous()->next(start);
Reverted 'astyle fix'. Those changes are not in sync with my astyle configuration/setup.
2010-04-02 07:30:58 +02:00
if (replaceThis->next())
Fix ticket #334 (segmentation fault on boost 1.39.0) http://apps.sourceforge.net/trac/cppcheck/ticket/334
2009-05-26 22:22:00 +02:00
replaceThis->next()->previous(end);
Fix ticket #25 (simplify "void f(x) int x; {" into "void f(int x) {")
2009-01-26 23:26:50 +01:00
start->previous(replaceThis->previous());
end->next(replaceThis->next());
renamed _tokensFrontBack to mTokensFrontBack
2018-06-16 16:22:35 +02:00
if (end->mTokensFrontBack && end->mTokensFrontBack->back == end) {
Reverted 'astyle fix'. Those changes are not in sync with my astyle configuration/setup.
2010-04-02 07:30:58 +02:00
while (end->next())
Ticket #1228 : Handle tokensBack in the Token class. When adding&removing tokens the Token class can make sure that this pointer is updated accordingly. It is very important that the tokensBack has the same scope as the token list, otherwise there will be a dead pointer problem.
2010-01-06 20:19:27 +01:00
end = end->next();
renamed _tokensFrontBack to mTokensFrontBack
2018-06-16 16:22:35 +02:00
end->mTokensFrontBack->back = end;
Ticket #1228 : Handle tokensBack in the Token class. When adding&removing tokens the Token class can make sure that this pointer is updated accordingly. It is very important that the tokensBack has the same scope as the token list, otherwise there will be a dead pointer problem.
2010-01-06 20:19:27 +01:00
}
Renamed _progressValue to mProgressValue
2018-06-16 16:41:25 +02:00
// Update mProgressValue, fileIndex and linenr
Token::replace: Little style change.
2012-01-22 00:02:55 +01:00
for (Token *tok = start; tok != end->next(); tok = tok->next())
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
tok->mImpl->mProgressValue = replaceThis->mImpl->mProgressValue;
Token::move|replace: update progressValues correctly.
2012-01-21 21:05:41 +01:00
Fix ticket #25 (simplify "void f(x) int x; {" into "void f(int x) {")
2009-01-26 23:26:50 +01:00
// Delete old token, which is replaced
delete replaceThis;
}
Refactoring: Rename class "TOKEN" to "Token"
2009-01-03 21:29:20 +01:00
const Token *Token::tokAt(int index) const
Formatting: uniformize end of lines.
2008-12-18 22:28:57 +01:00
{
Refactoring: Rename class "TOKEN" to "Token"
2009-01-03 21:29:20 +01:00
const Token *tok = this;
Refactorized token.cpp: - Removed redundant overload of Token::insertToken() - Fixed leak in Token::deleteThis() - Moved condition out of loop in Token::tokAt()
2015-12-14 22:04:26 +01:00
while (index > 0 && tok) {
tok = tok->next();
--index;
}
while (index < 0 && tok) {
tok = tok->previous();
++index;
Formatting: uniformize end of lines.
2008-12-18 22:28:57 +01:00
}
return tok;
}
1)Add 'Token' function 'linkAt', which simplifies the writing 'tokAt(i)->link()' to 'linkAt(i)'. It's still unused, though. 2)Fix one grammatical mistake inside token.h 3)Style change to token.cpp: remove the whitespace between '!' and 'begin'.
2011-11-11 21:55:37 +01:00
const Token *Token::linkAt(int index) const
{
const Token *tok = this->tokAt(index);
Add a sanity check and an error message to linkAt when the index is outside the tokens range.
2011-11-19 13:34:36 +01:00
if (!tok) {
Fixed #3999 (Error incorrectly written in XML format)
2012-07-25 06:43:54 +02:00
throw InternalError(this, "Internal error. Token::linkAt called with index outside the tokens range.");
Add a sanity check and an error message to linkAt when the index is outside the tokens range.
2011-11-19 13:34:36 +01:00
}
Removed redundant nullpointer checks in token.cpp
2012-09-09 14:34:07 +02:00
return tok->link();
1)Add 'Token' function 'linkAt', which simplifies the writing 'tokAt(i)->link()' to 'linkAt(i)'. It's still unused, though. 2)Fix one grammatical mistake inside token.h 3)Style change to token.cpp: remove the whitespace between '!' and 'begin'.
2011-11-11 21:55:37 +01:00
}
Adapt Token::strAt() to return a const reference like Token::str() does Saves us a complete conversion from string.c_str() back to std::string.
2011-11-14 09:16:47 +01:00
const std::string &Token::strAt(int index) const
Formatting: uniformize end of lines.
2008-12-18 22:28:57 +01:00
{
Refactoring: Rename class "TOKEN" to "Token"
2009-01-03 21:29:20 +01:00
const Token *tok = this->tokAt(index);
Renamed Token::_str
2018-06-16 23:03:15 +02:00
return tok ? tok->mStr : emptyString;
Formatting: uniformize end of lines.
2008-12-18 22:28:57 +01:00
}
Mark members and arguments with 'nonneg'
2019-07-14 12:22:33 +02:00
static int multiComparePercent(const Token *tok, const char*& haystack, nonneg int varid)
Fixed #3294 (Token::Match multi compare false negative)
2011-11-05 19:24:21 +01:00
{
Reduced Token::Match complexity, better support for complex patterns
2014-06-26 18:17:05 +02:00
++haystack;
// Compare only the first character of the string for optimization reasons
switch (haystack[0]) {
case '\0':
case ' ':
case '|':
//simple '%' character
haystack += 1;
if (tok->isArithmeticalOp() && tok->str() == "%")
return 1;
break;
case 'v':
if (haystack[3] == '%') { // %var%
haystack += 4;
Refactorization: Renamed Token::Match pattern %var% to %name%, implement new pattern %var% which is true if varId > 0.
2015-01-31 10:50:39 +01:00
if (tok->varId() != 0)
Reduced Token::Match complexity, better support for complex patterns
2014-06-26 18:17:05 +02:00
return 1;
} else { // %varid%
if (varid == 0) {
throw InternalError(tok, "Internal error. Token::Match called with varid 0. Please report this to Cppcheck developers");
}
Fixed #3294 (Token::Match multi compare false negative)
2011-11-05 19:24:21 +01:00
Reduced Token::Match complexity, better support for complex patterns
2014-06-26 18:17:05 +02:00
haystack += 6;
if (tok->varId() == varid)
return 1;
}
break;
case 't':
// Type (%type%)
{
haystack += 5;
Tokenizer: Fixed unwanted unknownMacro warning for decltype
2020-10-27 09:08:13 +01:00
if (tok->isName() && tok->varId() == 0 && (tok->str() != "delete" || !tok->isKeyword())) // HACK: this is legacy behaviour, it should return false for all keywords, except types
Reduced Token::Match complexity, better support for complex patterns
2014-06-26 18:17:05 +02:00
return 1;
}
break;
case 'a':
Fixed #7230 (Confusing code snippet in error message)
2015-12-31 01:15:49 +01:00
// Accept any token (%any%) or assign (%assign%)
Reduced Token::Match complexity, better support for complex patterns
2014-06-26 18:17:05 +02:00
{
Fixed #7230 (Confusing code snippet in error message)
2015-12-31 01:15:49 +01:00
if (haystack[3] == '%') { // %any%
haystack += 4;
return 1;
} else { // %assign%
haystack += 7;
if (tok->isAssignmentOp())
return 1;
}
Reduced Token::Match complexity, better support for complex patterns
2014-06-26 18:17:05 +02:00
}
Fixed #7230 (Confusing code snippet in error message)
2015-12-31 01:15:49 +01:00
break;
Reduced Token::Match complexity, better support for complex patterns
2014-06-26 18:17:05 +02:00
case 'n':
Refactorization: Renamed Token::Match pattern %var% to %name%, implement new pattern %var% which is true if varId > 0.
2015-01-31 10:50:39 +01:00
// Number (%num%) or name (%name%)
Reduced Token::Match complexity, better support for complex patterns
2014-06-26 18:17:05 +02:00
{
Refactorization: Renamed Token::Match pattern %var% to %name%, implement new pattern %var% which is true if varId > 0.
2015-01-31 10:50:39 +01:00
if (haystack[4] == '%') { // %name%
haystack += 5;
if (tok->isName())
return 1;
} else {
haystack += 4;
if (tok->isNumber())
return 1;
}
Reduced Token::Match complexity, better support for complex patterns
2014-06-26 18:17:05 +02:00
}
break;
case 'c': {
haystack += 1;
// Character (%char%)
if (haystack[0] == 'h') {
haystack += 4;
Added pointer to Type to Token (similar to Token::Variable() and Token::function()): - Accessible via Token::type() - Renamed former Token::type() to Token::tokType() - Removed SymbolDatabase::isClassOrStruct()
2015-08-14 20:46:13 +02:00
if (tok->tokType() == Token::eChar)
Fixed #3294 (Token::Match multi compare false negative)
2011-11-05 19:24:21 +01:00
return 1;
Reduced Token::Match complexity, better support for complex patterns
2014-06-26 18:17:05 +02:00
}
// Const operator (%cop%)
else if (haystack[1] == 'p') {
haystack += 3;
Changed behaviour of %op% pattern accordingly to changes to Token::isOp(). Added %cop% as replacement for old %op%
2013-03-01 11:43:59 +01:00
if (tok->isConstOp())
return 1;
Reduced Token::Match complexity, better support for complex patterns
2014-06-26 18:17:05 +02:00
}
// Comparison operator (%comp%)
else {
haystack += 4;
if (tok->isComparisonOp())
Fixed #3294 (Token::Match multi compare false negative)
2011-11-05 19:24:21 +01:00
return 1;
Reduced Token::Match complexity, better support for complex patterns
2014-06-26 18:17:05 +02:00
}
}
break;
case 's':
// String (%str%)
{
haystack += 4;
Added pointer to Type to Token (similar to Token::Variable() and Token::function()): - Accessible via Token::type() - Renamed former Token::type() to Token::tokType() - Removed SymbolDatabase::isClassOrStruct()
2015-08-14 20:46:13 +02:00
if (tok->tokType() == Token::eString)
Reduced Token::Match complexity, better support for complex patterns
2014-06-26 18:17:05 +02:00
return 1;
}
break;
case 'b':
// Bool (%bool%)
{
haystack += 5;
if (tok->isBoolean())
return 1;
}
break;
case 'o': {
++haystack;
if (haystack[1] == '%') {
// Op (%op%)
if (haystack[0] == 'p') {
haystack += 2;
if (tok->isOp())
return 1;
}
// Or (%or%)
else {
haystack += 2;
Added pointer to Type to Token (similar to Token::Variable() and Token::function()): - Accessible via Token::type() - Renamed former Token::type() to Token::tokType() - Removed SymbolDatabase::isClassOrStruct()
2015-08-14 20:46:13 +02:00
if (tok->tokType() == Token::eBitOp && tok->str() == "|")
Reduced Token::Match complexity, better support for complex patterns
2014-06-26 18:17:05 +02:00
return 1;
}
}
// Oror (%oror%)
else {
haystack += 4;
Added pointer to Type to Token (similar to Token::Variable() and Token::function()): - Accessible via Token::type() - Renamed former Token::type() to Token::tokType() - Removed SymbolDatabase::isClassOrStruct()
2015-08-14 20:46:13 +02:00
if (tok->tokType() == Token::eLogicalOp && tok->str() == "||")
Fixed #3294 (Token::Match multi compare false negative)
2011-11-05 19:24:21 +01:00
return 1;
}
Reduced Token::Match complexity, better support for complex patterns
2014-06-26 18:17:05 +02:00
}
break;
default:
//unknown %cmd%, abort
astyle formatting
2015-03-29 21:05:18 +02:00
throw InternalError(tok, "Unexpected command");
Fixed #3294 (Token::Match multi compare false negative)
2011-11-05 19:24:21 +01:00
}
Reduced Token::Match complexity, better support for complex patterns
2014-06-26 18:17:05 +02:00
if (*haystack == '|')
haystack += 1;
else
return -1;
Fixed #3294 (Token::Match multi compare false negative)
2011-11-05 19:24:21 +01:00
return 0xFFFF;
}
Fix wrong 'nonneg'
2019-07-14 17:31:26 +02:00
int Token::multiCompare(const Token *tok, const char *haystack, nonneg int varid)
Formatting: uniformize end of lines.
2008-12-18 22:28:57 +01:00
{
Fixed and cleaned up Token::Match engine
2014-06-26 17:32:24 +02:00
const char *needle = tok->str().c_str();
Fixed bug in multiCompare, which fixes ticket #66 ([False positive] "Buffer overrun" with "--all")
2009-01-27 20:30:01 +01:00
const char *needlePointer = needle;
astyle formatting
2012-03-25 11:51:59 +02:00
for (;;) {
Reduced Token::Match complexity, better support for complex patterns
2014-06-26 18:17:05 +02:00
if (needlePointer == needle && haystack[0] == '%' && haystack[1] != '|' && haystack[1] != '\0' && haystack[1] != ' ') {
Improve constness
2018-04-04 21:51:31 +02:00
const int ret = multiComparePercent(tok, haystack, varid);
Reduced Token::Match complexity, better support for complex patterns
2014-06-26 18:17:05 +02:00
if (ret < 2)
return ret;
changed the astyle formatting flags
2011-10-13 20:53:06 +02:00
} else if (*haystack == '|') {
if (*needlePointer == 0) {
Refactorings: Optimised the Token::multiCompare function
2010-09-19 15:14:13 +02:00
// If needle is at the end, we have a match.
Formatting: uniformize end of lines.
2008-12-18 22:28:57 +01:00
return 1;
Fixed bug in multiCompare, which fixes ticket #66 ([False positive] "Buffer overrun" with "--all")
2009-01-27 20:30:01 +01:00
}
Formatting: uniformize end of lines.
2008-12-18 22:28:57 +01:00
Fixed bug in multiCompare, which fixes ticket #66 ([False positive] "Buffer overrun" with "--all")
2009-01-27 20:30:01 +01:00
needlePointer = needle;
Refactorings: Optimised the Token::multiCompare function
2010-09-19 15:14:13 +02:00
++haystack;
Reduced Token::Match complexity, better support for complex patterns
2014-06-26 18:17:05 +02:00
} else if (*needlePointer == *haystack) {
if (*needlePointer == '\0')
return 1;
++needlePointer;
++haystack;
changed the astyle formatting flags
2011-10-13 20:53:06 +02:00
} else if (*haystack == ' ' || *haystack == '\0') {
Refactorings: Optimised the Token::multiCompare function
2010-09-19 15:14:13 +02:00
if (needlePointer == needle)
return 0;
break;
Formatting: uniformize end of lines.
2008-12-18 22:28:57 +01:00
}
Fixed bug in multiCompare, which fixes ticket #66 ([False positive] "Buffer overrun" with "--all")
2009-01-27 20:30:01 +01:00
// If haystack and needle don't share the same character,
// find next '|' character.
changed the astyle formatting flags
2011-10-13 20:53:06 +02:00
else {
Refactorings: Optimised the Token::multiCompare function
2010-09-19 15:14:13 +02:00
needlePointer = needle;
Formatting: uniformize end of lines.
2008-12-18 22:28:57 +01:00
changed the astyle formatting flags
2011-10-13 20:53:06 +02:00
do {
Refactorings: Optimised the Token::multiCompare function
2010-09-19 15:14:13 +02:00
++haystack;
changed the astyle formatting flags
2011-10-13 20:53:06 +02:00
} while (*haystack != ' ' && *haystack != '|' && *haystack);
Formatting: uniformize end of lines.
2008-12-18 22:28:57 +01:00
changed the astyle formatting flags
2011-10-13 20:53:06 +02:00
if (*haystack == ' ' || *haystack == '\0') {
Removed support for patterns like |a|b and a||b (equal to a|b|) -> Improved performance by 1,3% (non-matchcompiled build)
2016-02-01 09:55:16 +01:00
return -1;
Refactorings: Optimised the Token::multiCompare function
2010-09-19 15:14:13 +02:00
}
Formatting: uniformize end of lines.
2008-12-18 22:28:57 +01:00
Refactorings: Optimised the Token::multiCompare function
2010-09-19 15:14:13 +02:00
++haystack;
Fixed bug in multiCompare, which fixes ticket #66 ([False positive] "Buffer overrun" with "--all")
2009-01-27 20:30:01 +01:00
}
}
Refactorings: Optimised the Token::multiCompare function
2010-09-19 15:14:13 +02:00
if (*needlePointer == '\0')
return 1;
Formatting: uniformize end of lines.
2008-12-18 22:28:57 +01:00
return -1;
}
optimized non-matchcompiled Token::simpleMatch() a bit (#2640)
2020-05-26 20:13:56 +02:00
bool Token::simpleMatch(const Token *tok, const char pattern[], size_t pattern_len)
Implement simpleMatch that should be use when pattern contains no flags
2008-12-22 00:28:09 +01:00
{
Fix some doxygen comments and add shortcut for tok==NULL to Token::simpleMatch()
2014-04-27 09:32:02 +02:00
if (!tok)
return false; // shortcut
optimized non-matchcompiled Token::simpleMatch() a bit (#2640)
2020-05-26 20:13:56 +02:00
const char *current = pattern;
const char *end = pattern + pattern_len;
const char *next = (const char*)std::memchr(pattern, ' ', pattern_len);
Reverted 'astyle fix'. Those changes are not in sync with my astyle configuration/setup.
2010-04-02 07:30:58 +02:00
if (!next)
optimized non-matchcompiled Token::simpleMatch() a bit (#2640)
2020-05-26 20:13:56 +02:00
next = end;
Formatting: demux simpleMatch for readability
2008-12-23 22:45:47 +01:00
changed the astyle formatting flags
2011-10-13 20:53:06 +02:00
while (*current) {
Improve constness
2018-04-04 21:51:31 +02:00
const std::size_t length = next - current;
Formatting: demux simpleMatch for readability
2008-12-23 22:45:47 +01:00
Renamed Token::_str
2018-06-16 23:03:15 +02:00
if (!tok || length != tok->mStr.length() || std::strncmp(current, tok->mStr.c_str(), length))
Implement simpleMatch that should be use when pattern contains no flags
2008-12-22 00:28:09 +01:00
return false;
Formatting: demux simpleMatch for readability
2008-12-23 22:45:47 +01:00
current = next;
changed the astyle formatting flags
2011-10-13 20:53:06 +02:00
if (*next) {
Support for Sun Studio C++ compiler
2012-12-27 11:51:12 +01:00
next = std::strchr(++current, ' ');
Reverted 'astyle fix'. Those changes are not in sync with my astyle configuration/setup.
2010-04-02 07:30:58 +02:00
if (!next)
optimized non-matchcompiled Token::simpleMatch() a bit (#2640)
2020-05-26 20:13:56 +02:00
next = end;
Formatting: demux simpleMatch for readability
2008-12-23 22:45:47 +01:00
}
tok = tok->next();
Implement simpleMatch that should be use when pattern contains no flags
2008-12-22 00:28:09 +01:00
}
Formatting: demux simpleMatch for readability
2008-12-23 22:45:47 +01:00
Implement simpleMatch that should be use when pattern contains no flags
2008-12-22 00:28:09 +01:00
return true;
}
Refactorizations in class Token: - Added support for < and > in nextArgument() - Made isStandardType inline by writing its implementation to token.h (like other bool is*() functions) - Made firstWordEquals return bool instead of integer (only 0 and 1 were returned) - Use std::string::empty() instead of operator==(""); Use character literals instead of string literals for single characters printed.
2012-04-16 15:28:38 +02:00
bool Token::firstWordEquals(const char *str, const char *word)
Fix ticket #543 (Possible buffer overrun in Token::Match()) http://sourceforge.net/apps/trac/cppcheck/ticket/543 Get rid of string copying in Token::Match(), also improves speed a little. 8.0s -> 7.7s
2009-08-16 22:28:17 +02:00
{
changed the astyle formatting flags
2011-10-13 20:53:06 +02:00
for (;;) {
if (*str != *word) {
Fix bug in class Token that causes recently added modulo-check to fail.
2012-04-26 16:39:16 +02:00
return (*str == ' ' && *word == 0);
changed the astyle formatting flags
2011-10-13 20:53:06 +02:00
} else if (*str == 0)
Fix ticket #543 (Possible buffer overrun in Token::Match()) http://sourceforge.net/apps/trac/cppcheck/ticket/543 Get rid of string copying in Token::Match(), also improves speed a little. 8.0s -> 7.7s
2009-08-16 22:28:17 +02:00
break;
++str;
++word;
}
Fix bug in class Token that causes recently added modulo-check to fail.
2012-04-26 16:39:16 +02:00
return true;
Fix ticket #543 (Possible buffer overrun in Token::Match()) http://sourceforge.net/apps/trac/cppcheck/ticket/543 Get rid of string copying in Token::Match(), also improves speed a little. 8.0s -> 7.7s
2009-08-16 22:28:17 +02:00
}
const char *Token::chrInFirstWord(const char *str, char c)
{
changed the astyle formatting flags
2011-10-13 20:53:06 +02:00
for (;;) {
Reverted 'astyle fix'. Those changes are not in sync with my astyle configuration/setup.
2010-04-02 07:30:58 +02:00
if (*str == ' ' || *str == 0)
Use nullptr instead of 0 or NULL (#936)
2017-08-09 20:00:26 +02:00
return nullptr;
Fix ticket #543 (Possible buffer overrun in Token::Match()) http://sourceforge.net/apps/trac/cppcheck/ticket/543 Get rid of string copying in Token::Match(), also improves speed a little. 8.0s -> 7.7s
2009-08-16 22:28:17 +02:00
Reverted 'astyle fix'. Those changes are not in sync with my astyle configuration/setup.
2010-04-02 07:30:58 +02:00
if (*str == c)
Fix ticket #543 (Possible buffer overrun in Token::Match()) http://sourceforge.net/apps/trac/cppcheck/ticket/543 Get rid of string copying in Token::Match(), also improves speed a little. 8.0s -> 7.7s
2009-08-16 22:28:17 +02:00
return str;
++str;
}
}
Mark members and arguments with 'nonneg'
2019-07-14 12:22:33 +02:00
bool Token::Match(const Token *tok, const char pattern[], nonneg int varid)
Formatting: uniformize end of lines.
2008-12-18 22:28:57 +01:00
{
const char *p = pattern;
changed the astyle formatting flags
2011-10-13 20:53:06 +02:00
while (*p) {
Formatting: uniformize end of lines.
2008-12-18 22:28:57 +01:00
// Skip spaces in pattern..
Reverted 'astyle fix'. Those changes are not in sync with my astyle configuration/setup.
2010-04-02 07:30:58 +02:00
while (*p == ' ')
Refactoring: Prefer ++i; over i++;
2009-01-01 23:22:28 +01:00
++p;
Formatting: uniformize end of lines.
2008-12-18 22:28:57 +01:00
// No token => Success!
Token::Match: add a new pattern: '%comp%'. Token::Match returns true if the token is a comparison operator. It's equivalent to matching '<|<=|==|!=|>=|>' and the tests show that the new pattern speeds up the program execution. Added the new pattern in CheckInternal (and also reordered the %cmd% lists) and in TestToken.
2012-12-01 00:47:07 +01:00
if (*p == '\0')
break;
Formatting: uniformize end of lines.
2008-12-18 22:28:57 +01:00
changed the astyle formatting flags
2011-10-13 20:53:06 +02:00
if (!tok) {
token: when Token::Match reached the end of input, it returned true if the next pattern was !!. It now returns true only if all remaining patterns are !!.
2009-01-10 01:33:48 +01:00
// If we have no tokens, pattern "!!else" should return true
Little tweaks to Token::Match code.
2012-11-25 15:13:41 +01:00
if (p[0] == '!' && p[1] == '!' && p[2] != '\0') {
Optimisation: optimised Token::Match
2010-09-20 20:15:07 +02:00
while (*p && *p != ' ')
++p;
token: when Token::Match reached the end of input, it returned true if the next pattern was !!. It now returns true only if all remaining patterns are !!.
2009-01-10 01:33:48 +01:00
continue;
changed the astyle formatting flags
2011-10-13 20:53:06 +02:00
} else
token: when Token::Match reached the end of input, it returned true if the next pattern was !!. It now returns true only if all remaining patterns are !!.
2009-01-10 01:33:48 +01:00
return false;
}
Formatting: uniformize end of lines.
2008-12-18 22:28:57 +01:00
// [.. => search for a one-character token..
Reduced Token::Match complexity, better support for complex patterns
2014-06-26 18:17:05 +02:00
if (p[0] == '[' && chrInFirstWord(p, ']')) {
Enhance style and code to Token::Match function.
2012-11-20 00:16:53 +01:00
if (tok->str().length() != 1)
Fixed #2882 (One-character token can be mistaken for a multiCompare)
2011-07-15 19:01:36 +02:00
return false;
astyle formatting
2011-07-15 19:02:16 +02:00
Enhance style and code to Token::Match function.
2012-11-20 00:16:53 +01:00
const char *temp = p+1;
Fix ticket #543 (Possible buffer overrun in Token::Match()) http://sourceforge.net/apps/trac/cppcheck/ticket/543 Get rid of string copying in Token::Match(), also improves speed a little. 8.0s -> 7.7s
2009-08-16 22:28:17 +02:00
bool chrFound = false;
Avoid 'unsigned' for int variables
2019-07-13 16:06:24 +02:00
int count = 0;
changed the astyle formatting flags
2011-10-13 20:53:06 +02:00
while (*temp && *temp != ' ') {
if (*temp == ']') {
Fix ticket #543 (Possible buffer overrun in Token::Match()) http://sourceforge.net/apps/trac/cppcheck/ticket/543 Get rid of string copying in Token::Match(), also improves speed a little. 8.0s -> 7.7s
2009-08-16 22:28:17 +02:00
++count;
}
Enhance style and code to Token::Match function.
2012-11-20 00:16:53 +01:00
else if (*temp == tok->str()[0]) {
Fix ticket #543 (Possible buffer overrun in Token::Match()) http://sourceforge.net/apps/trac/cppcheck/ticket/543 Get rid of string copying in Token::Match(), also improves speed a little. 8.0s -> 7.7s
2009-08-16 22:28:17 +02:00
chrFound = true;
break;
}
++temp;
}
Enhance style and code to Token::Match function.
2012-11-20 00:16:53 +01:00
if (count > 1 && tok->str()[0] == ']')
chrFound = true;
Fix ticket #543 (Possible buffer overrun in Token::Match()) http://sourceforge.net/apps/trac/cppcheck/ticket/543 Get rid of string copying in Token::Match(), also improves speed a little. 8.0s -> 7.7s
2009-08-16 22:28:17 +02:00
Reverted 'astyle fix'. Those changes are not in sync with my astyle configuration/setup.
2010-04-02 07:30:58 +02:00
if (!chrFound)
Formatting: uniformize end of lines.
2008-12-18 22:28:57 +01:00
return false;
Enhance style and code to Token::Match function.
2012-11-20 00:16:53 +01:00
p = temp;
while (*p && *p != ' ')
++p;
Formatting: uniformize end of lines.
2008-12-18 22:28:57 +01:00
}
Reduced Token::Match complexity, better support for complex patterns
2014-06-26 18:17:05 +02:00
// Parse "not" options. Token can be anything except the given one
else if (p[0] == '!' && p[1] == '!' && p[2] != '\0') {
p += 2;
if (firstWordEquals(p, tok->str().c_str()))
return false;
while (*p && *p != ' ')
++p;
}
Formatting: uniformize end of lines.
2008-12-18 22:28:57 +01:00
// Parse multi options, such as void|int|char (accept token which is one of these 3)
Reduced Token::Match complexity, better support for complex patterns
2014-06-26 18:17:05 +02:00
else {
Improve constness
2018-04-04 21:51:31 +02:00
const int res = multiCompare(tok, p, varid);
changed the astyle formatting flags
2011-10-13 20:53:06 +02:00
if (res == 0) {
Formatting: uniformize end of lines.
2008-12-18 22:28:57 +01:00
// Empty alternative matches, use the same token on next round
Optimisation: optimised Token::Match
2010-09-20 20:15:07 +02:00
while (*p && *p != ' ')
++p;
Formatting: uniformize end of lines.
2008-12-18 22:28:57 +01:00
continue;
changed the astyle formatting flags
2011-10-13 20:53:06 +02:00
} else if (res == -1) {
Formatting: uniformize end of lines.
2008-12-18 22:28:57 +01:00
// No match
return false;
}
}
Optimisation: optimised Token::Match
2010-09-20 20:15:07 +02:00
while (*p && *p != ' ')
++p;
Formatting: uniformize end of lines.
2008-12-18 22:28:57 +01:00
tok = tok->next();
}
// The end of the pattern has been reached and nothing wrong has been found
return true;
}
Mark members and arguments with 'nonneg'
2019-07-14 12:22:33 +02:00
nonneg int Token::getStrLength(const Token *tok)
Token::getStrLength(): introduce new static method. No functional change.
2009-08-30 13:07:10 +02:00
{
use nullptr in lib/token.cpp
2014-02-15 08:05:54 +01:00
assert(tok != nullptr);
Renamed _tokType to mTokType
2018-06-16 16:40:02 +02:00
assert(tok->mTokType == eString);
Token::getStrLength(): introduce new static method. No functional change.
2009-08-30 13:07:10 +02:00
Avoid 'unsigned' for int variables
2019-07-13 16:06:24 +02:00
int len = 0;
Keep prefix in string and char literals (#2272) Keeping the prefix in the token allows cppcheck to print the correct string and char literals in debug and error messages. To achieve this, move some of the helper functions from token.cpp to utils.h so that checks that look at string and char literals can reuse them. This is a large part of this commit. Note that the only user visible change is that when string and char literals are printed in error messages, the prefix is now included. For example: int f() { return test.substr( 0 , 4 ) == U"Hello" ? 0 : 1 ; }; now prints U"Hello" instead of "Hello" in the error message.
2019-10-16 11:41:33 +02:00
const std::string str(getStringLiteral(tok->str()));
std::string::const_iterator it = str.begin();
const std::string::const_iterator end = str.end();
Token::getStrLength(): introduce new static method. No functional change.
2009-08-30 13:07:10 +02:00
Token::strValue: fixed handling of backslash
2015-06-07 11:25:33 +02:00
while (it != end) {
if (*it == '\\') {
++it;
Fixed #731 (False positive, strcpy copying a buffer with a null character) http://sourceforge.net/apps/trac/cppcheck/ticket/731
2009-09-26 17:58:14 +02:00
// string ends at '\0'
Token::strValue: fixed handling of backslash
2015-06-07 11:25:33 +02:00
if (*it == '0')
return len;
Fixed #731 (False positive, strcpy copying a buffer with a null character) http://sourceforge.net/apps/trac/cppcheck/ticket/731
2009-09-26 17:58:14 +02:00
}
Token::strValue: fixed handling of backslash
2015-06-07 11:25:33 +02:00
if (*it == '\0')
return len;
++it;
Token::getStrLength(): introduce new static method. No functional change.
2009-08-30 13:07:10 +02:00
++len;
}
return len;
}
Set correct type and size of string and char literals (#2275) * Set correct type and size of string and char literals Use that string and char literal tokens store the prefix. This makes it possible to distinghuish between different type of string literals (i.e., utf8 encoded strings, utf16, wide strings, etc) which have different type. When the tokens holding the string and character values have the correct type, it is possible to improve Token::getStrSize() to give the correct result for all string types. Previously, it would return the number of characters in the string, i.e., it would give the wrong size unless the type of the string was char*. Since strings now can have different size (in number of bytes) and length (in number of elements), add a new helper function that returns the number of characters. Checkers have been updated to use the correct functions. Having the size makes it possible to find more problems with prefixed strings, and to reduce false positives, for example in the buffer overflow checker. Also, improve the stringLiteralWrite error message to also print the prefix of the string (if there is one). * Add comment and update string length
2019-10-20 07:11:57 +02:00
nonneg int Token::getStrArraySize(const Token *tok)
Token: Added utility function getStrSize as a complement to getStrLength
2014-08-01 13:12:18 +02:00
{
lib: Separate statements of the form "assert(a && b)" into "assert(a)" and "assert(b)", for more precise diagnostics. (#1825)
2019-05-07 10:28:31 +02:00
assert(tok != nullptr);
assert(tok->tokType() == eString);
Keep prefix in string and char literals (#2272) Keeping the prefix in the token allows cppcheck to print the correct string and char literals in debug and error messages. To achieve this, move some of the helper functions from token.cpp to utils.h so that checks that look at string and char literals can reuse them. This is a large part of this commit. Note that the only user visible change is that when string and char literals are printed in error messages, the prefix is now included. For example: int f() { return test.substr( 0 , 4 ) == U"Hello" ? 0 : 1 ; }; now prints U"Hello" instead of "Hello" in the error message.
2019-10-16 11:41:33 +02:00
const std::string str(getStringLiteral(tok->str()));
Avoid 'unsigned' for int variables
2019-07-13 16:06:24 +02:00
int sizeofstring = 1;
Keep prefix in string and char literals (#2272) Keeping the prefix in the token allows cppcheck to print the correct string and char literals in debug and error messages. To achieve this, move some of the helper functions from token.cpp to utils.h so that checks that look at string and char literals can reuse them. This is a large part of this commit. Note that the only user visible change is that when string and char literals are printed in error messages, the prefix is now included. For example: int f() { return test.substr( 0 , 4 ) == U"Hello" ? 0 : 1 ; }; now prints U"Hello" instead of "Hello" in the error message.
2019-10-16 11:41:33 +02:00
for (int i = 0; i < (int)str.size(); i++) {
Token: Added utility function getStrSize as a complement to getStrLength
2014-08-01 13:12:18 +02:00
if (str[i] == '\\')
++i;
++sizeofstring;
}
return sizeofstring;
}
Set correct type and size of string and char literals (#2275) * Set correct type and size of string and char literals Use that string and char literal tokens store the prefix. This makes it possible to distinghuish between different type of string literals (i.e., utf8 encoded strings, utf16, wide strings, etc) which have different type. When the tokens holding the string and character values have the correct type, it is possible to improve Token::getStrSize() to give the correct result for all string types. Previously, it would return the number of characters in the string, i.e., it would give the wrong size unless the type of the string was char*. Since strings now can have different size (in number of bytes) and length (in number of elements), add a new helper function that returns the number of characters. Checkers have been updated to use the correct functions. Having the size makes it possible to find more problems with prefixed strings, and to reduce false positives, for example in the buffer overflow checker. Also, improve the stringLiteralWrite error message to also print the prefix of the string (if there is one). * Add comment and update string length
2019-10-20 07:11:57 +02:00
nonneg int Token::getStrSize(const Token *tok, const Settings *settings)
{
assert(tok != nullptr && tok->tokType() == eString);
nonneg int sizeofType = 1;
if (tok->valueType()) {
ValueType vt(*tok->valueType());
vt.pointer = 0;
sizeofType = ValueFlow::getSizeOf(vt, settings);
}
return getStrArraySize(tok) * sizeofType;
}
Avoid 'unsigned' for int variables
2019-07-13 16:06:24 +02:00
std::string Token::getCharAt(const Token *tok, MathLib::bigint index)
Fixed #4481 (Simplify %str% [ %num% ])
2013-01-13 20:52:38 +01:00
{
use nullptr in lib/token.cpp
2014-02-15 08:05:54 +01:00
assert(tok != nullptr);
Keep prefix in string and char literals (#2272) Keeping the prefix in the token allows cppcheck to print the correct string and char literals in debug and error messages. To achieve this, move some of the helper functions from token.cpp to utils.h so that checks that look at string and char literals can reuse them. This is a large part of this commit. Note that the only user visible change is that when string and char literals are printed in error messages, the prefix is now included. For example: int f() { return test.substr( 0 , 4 ) == U"Hello" ? 0 : 1 ; }; now prints U"Hello" instead of "Hello" in the error message.
2019-10-16 11:41:33 +02:00
std::string str(getStringLiteral(tok->str()));
std::string::const_iterator it = str.begin();
const std::string::const_iterator end = str.end();
Fixed #4481 (Simplify %str% [ %num% ])
2013-01-13 20:52:38 +01:00
Fixed broken function Token::getCharAt() (#7073)
2015-10-26 11:51:05 +01:00
while (it != end) {
Fixed #4481 (Simplify %str% [ %num% ])
2013-01-13 20:52:38 +01:00
if (index == 0) {
Fixed broken function Token::getCharAt() (#7073)
2015-10-26 11:51:05 +01:00
if (*it == '\0')
return "\\0";
std::string ret(1, *it);
if (*it == '\\') {
++it;
ret += *it;
Fixed #4481 (Simplify %str% [ %num% ])
2013-01-13 20:52:38 +01:00
}
return ret;
}
Fixed broken function Token::getCharAt() (#7073)
2015-10-26 11:51:05 +01:00
if (*it == '\\')
++it;
++it;
Fixed #4481 (Simplify %str% [ %num% ])
2013-01-13 20:52:38 +01:00
--index;
}
assert(index == 0);
return "\\0";
}
Fix #1018 (strcat(strcat()) causes assertion in Tokenizer::validate) Test case provided by povaddict http://sourceforge.net/apps/trac/cppcheck/ticket/1018
2009-11-27 22:21:13 +01:00
void Token::move(Token *srcStart, Token *srcEnd, Token *newLocation)
{
/**[newLocation] -> b -> c -> [srcStart] -> [srcEnd] -> f */
// Fix the gap, which tokens to be moved will leave
srcStart->previous()->next(srcEnd->next());
srcEnd->next()->previous(srcStart->previous());
// Fix the tokens to be moved
srcEnd->next(newLocation->next());
srcStart->previous(newLocation);
// Fix the tokens at newLocation
newLocation->next()->previous(srcEnd);
newLocation->next(srcStart);
Added ErrorLogger::reportProgress and removed ErrorLogger::ReportProgress. This will make it easier for GUI and other clients to display progress information.
2010-08-03 16:36:21 +02:00
// Update _progressValue
Token::move|replace: update progressValues correctly.
2012-01-21 21:05:41 +01:00
for (Token *tok = srcStart; tok != srcEnd->next(); tok = tok->next())
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
tok->mImpl->mProgressValue = newLocation->mImpl->mProgressValue;
Fix #1018 (strcat(strcat()) causes assertion in Tokenizer::validate) Test case provided by povaddict http://sourceforge.net/apps/trac/cppcheck/ticket/1018
2009-11-27 22:21:13 +01:00
}
Tokenizer: Refactorings and improvements. Better handling of []
2011-12-07 21:15:00 +01:00
Token* Token::nextArgument() const
Added Token::nextArgument()
2011-10-23 11:23:48 +02:00
{
for (const Token* tok = this; tok; tok = tok->next()) {
if (tok->str() == ",")
Improve nextArgument by returning 0 if there's an unexpected ';'.
2011-10-28 18:57:10 +02:00
return tok->next();
Refactoring. Use Token::Match instead of hardcoded patterns to increase readability.
2014-09-14 11:26:16 +02:00
else if (tok->link() && Token::Match(tok, "(|{|[|<"))
Refactorizations in class Token: - Added support for < and > in nextArgument() - Made isStandardType inline by writing its implementation to token.h (like other bool is*() functions) - Made firstWordEquals return bool instead of integer (only 0 and 1 were returned) - Use std::string::empty() instead of operator==(""); Use character literals instead of string literals for single characters printed.
2012-04-16 15:28:38 +02:00
tok = tok->link();
Refactoring. Use Token::Match instead of hardcoded patterns to increase readability.
2014-09-14 11:26:16 +02:00
else if (Token::Match(tok, ")|;"))
#6269 false positives in case of overloaded standard library functions. Detect memset() with proper argument cound, using new function numberOfArguments()
2015-12-06 12:50:05 +01:00
return nullptr;
Added Token::nextArgument()
2011-10-23 11:23:48 +02:00
}
#6269 false positives in case of overloaded standard library functions. Detect memset() with proper argument cound, using new function numberOfArguments()
2015-12-06 12:50:05 +01:00
return nullptr;
Added Token::nextArgument()
2011-10-23 11:23:48 +02:00
}
Fixed more false positives of #6056: - Implemented nextArgument() for usages before < and > are linked - slightly optimized nextArgument()
2014-08-20 14:57:00 +02:00
Token* Token::nextArgumentBeforeCreateLinks2() const
{
for (const Token* tok = this; tok; tok = tok->next()) {
if (tok->str() == ",")
return tok->next();
Refactoring. Use Token::Match instead of hardcoded patterns to increase readability.
2014-09-14 11:26:16 +02:00
else if (tok->link() && Token::Match(tok, "(|{|["))
Fixed more false positives of #6056: - Implemented nextArgument() for usages before < and > are linked - slightly optimized nextArgument()
2014-08-20 14:57:00 +02:00
tok = tok->link();
else if (tok->str() == "<") {
const Token* temp = tok->findClosingBracket();
if (temp)
tok = temp;
Refactoring. Use Token::Match instead of hardcoded patterns to increase readability.
2014-09-14 11:26:16 +02:00
} else if (Token::Match(tok, ")|;"))
#6269 false positives in case of overloaded standard library functions. Detect memset() with proper argument cound, using new function numberOfArguments()
2015-12-06 12:50:05 +01:00
return nullptr;
Fixed more false positives of #6056: - Implemented nextArgument() for usages before < and > are linked - slightly optimized nextArgument()
2014-08-20 14:57:00 +02:00
}
#6269 false positives in case of overloaded standard library functions. Detect memset() with proper argument cound, using new function numberOfArguments()
2015-12-06 12:50:05 +01:00
return nullptr;
Fixed more false positives of #6056: - Implemented nextArgument() for usages before < and > are linked - slightly optimized nextArgument()
2014-08-20 14:57:00 +02:00
}
Library: Add <container> tag to Libraries, provide configuration for std::vector, std::deque, std::array and STL strings Token: Added function to jump to the next template argument
2015-01-03 20:35:33 +01:00
Token* Token::nextTemplateArgument() const
{
for (const Token* tok = this; tok; tok = tok->next()) {
if (tok->str() == ",")
return tok->next();
else if (tok->link() && Token::Match(tok, "(|{|[|<"))
tok = tok->link();
else if (Token::Match(tok, ">|;"))
#6269 false positives in case of overloaded standard library functions. Detect memset() with proper argument cound, using new function numberOfArguments()
2015-12-06 12:50:05 +01:00
return nullptr;
Library: Add <container> tag to Libraries, provide configuration for std::vector, std::deque, std::array and STL strings Token: Added function to jump to the next template argument
2015-01-03 20:35:33 +01:00
}
#6269 false positives in case of overloaded standard library functions. Detect memset() with proper argument cound, using new function numberOfArguments()
2015-12-06 12:50:05 +01:00
return nullptr;
Library: Add <container> tag to Libraries, provide configuration for std::vector, std::deque, std::array and STL strings Token: Added function to jump to the next template argument
2015-01-03 20:35:33 +01:00
}
fix syntax error on template operator (#2225)
2019-10-03 12:26:45 +02:00
static bool isOperator(const Token *tok)
{
if (tok->link())
tok = tok->link();
// TODO handle multi token operators
return tok->strAt(-1) == "operator";
}
Refactoring Token::findClosingBracket.
2013-07-31 10:30:20 +02:00
const Token * Token::findClosingBracket() const
Implemented generic mechanism for '<'-'>' "linkage" before link() works (Taken from Scope::findClosingBracket, but C++11 right angle bracket support added): Token::findClosingBracket -> Replaced several indendation-counting mechanisms in tokenize.cpp Fixed build failure in checkclass.cpp
2012-04-18 16:02:03 +02:00
{
Renamed Token::_str
2018-06-16 23:03:15 +02:00
if (mStr != "<")
Refactor Token::findClosingBracket()
2018-01-01 12:22:04 +01:00
return nullptr;
More robust Token::findClosingBracket
2020-11-30 19:26:15 +01:00
if (!mPrevious)
return nullptr;
if (!(mPrevious->isName() ||
Token::Match(mPrevious->previous(), "operator %op% <") ||
Token::Match(mPrevious->tokAt(-2), "operator [([] [)]] <")))
return nullptr;
Change some more 0 literals into nullptr.
2014-02-16 11:47:52 +01:00
const Token *closing = nullptr;
Fix #9249 (Syntax error on valid C++) (#2062)
2019-08-04 10:24:44 +02:00
const bool templateParameter(strAt(-1) == "template");
std::set<std::string> templateParameters;
Refactoring Token::findClosingBracket.
2013-07-31 10:30:20 +02:00
Fixed bug in fixAngleBrackets
2020-11-29 12:56:13 +01:00
bool isDecl = true;
for (const Token *prev = previous(); prev; prev = prev->previous()) {
if (prev->str() == "=")
isDecl = false;
if (Token::simpleMatch(prev, "template <"))
isDecl = true;
if (Token::Match(prev, "[;{}]"))
break;
}
Refactor Token::findClosingBracket()
2018-01-01 12:22:04 +01:00
unsigned int depth = 0;
for (closing = this; closing != nullptr; closing = closing->next()) {
if (Token::Match(closing, "{|[|(")) {
closing = closing->link();
if (!closing)
return nullptr; // #6803
} else if (Token::Match(closing, "}|]|)|;"))
return nullptr;
Fix #9249 (Syntax error on valid C++) (#2062)
2019-08-04 10:24:44 +02:00
// we can make some guesses for template parameters
fix syntax error on template operator (#2225)
2019-10-03 12:26:45 +02:00
else if (closing->str() == "<" && closing->previous() &&
(closing->previous()->isName() || isOperator(closing->previous())) &&
Fixed #9046 syntaxError (#2180) * Fixed #9046 syntaxError * fix another syntax error * fix some more syntax errors
2019-09-20 12:35:01 +02:00
(templateParameter ? templateParameters.find(closing->strAt(-1)) == templateParameters.end() : true))
Refactor Token::findClosingBracket()
2018-01-01 12:22:04 +01:00
++depth;
else if (closing->str() == ">") {
if (--depth == 0)
return closing;
Test for syntax error in nested templates (#2746)
2020-08-25 07:14:44 +02:00
} else if (closing->str() == ">>" || closing->str() == ">>=") {
Fixed bug in fixAngleBrackets
2020-11-29 12:56:13 +01:00
if (!isDecl && depth == 1)
continue;
Refactor Token::findClosingBracket()
2018-01-01 12:22:04 +01:00
if (depth <= 2)
return closing;
depth -= 2;
Implemented generic mechanism for '<'-'>' "linkage" before link() works (Taken from Scope::findClosingBracket, but C++11 right angle bracket support added): Token::findClosingBracket -> Replaced several indendation-counting mechanisms in tokenize.cpp Fixed build failure in checkclass.cpp
2012-04-18 16:02:03 +02:00
}
Fix #9249 (Syntax error on valid C++) (#2062)
2019-08-04 10:24:44 +02:00
// save named template parameter
else if (templateParameter && depth == 1 && closing->str() == "," &&
closing->previous()->isName() && !Match(closing->previous(), "class|typename|."))
templateParameters.insert(closing->strAt(-1));
Implemented generic mechanism for '<'-'>' "linkage" before link() works (Taken from Scope::findClosingBracket, but C++11 right angle bracket support added): Token::findClosingBracket -> Replaced several indendation-counting mechanisms in tokenize.cpp Fixed build failure in checkclass.cpp
2012-04-18 16:02:03 +02:00
}
Refactoring Token::findClosingBracket.
2013-07-31 10:30:20 +02:00
return closing;
}
Token * Token::findClosingBracket()
{
// return value of const function
return const_cast<Token*>(const_cast<const Token*>(this)->findClosingBracket());
Implemented generic mechanism for '<'-'>' "linkage" before link() works (Taken from Scope::findClosingBracket, but C++11 right angle bracket support added): Token::findClosingBracket -> Replaced several indendation-counting mechanisms in tokenize.cpp Fixed build failure in checkclass.cpp
2012-04-18 16:02:03 +02:00
}
Fixed #8693 (Template specialization: Constructor detected as normal … (#1418) * Fixed #8693 (Template specialization: Constructor detected as normal function (functionStatic error)) Refactor template simplifier to remove the existing full specialization function expandSpecialized and allow full specializations to use the existing function expandTemplate. The function expandTemplate was modified to either expand the template like it originally did by copying it or to modify the template in place. Both instantiated and uninstantiated full specializations are modified in place. This also fixes #8692 and probably other related tickets as well. The function simplifyTemplates now tries twice to simplify templates so more templates can be simplified. We should try as many times as necessary to find all possible templates. We can't do that now because uninstantiated templates are left unchanged. It is relatively straight forward to have the new code also expand in place uninstantiated templates with their symbolic types but namespaces are not handled properly (ticket #8671) and it would introduce regressions. * Fix travis warnings.
2018-10-14 16:57:07 +02:00
const Token * Token::findOpeningBracket() const
{
if (mStr != ">")
return nullptr;
const Token *opening = nullptr;
unsigned int depth = 0;
for (opening = this; opening != nullptr; opening = opening->previous()) {
if (Token::Match(opening, "}|]|)")) {
opening = opening->link();
if (!opening)
return nullptr;
} else if (Token::Match(opening, "{|{|(|;"))
return nullptr;
else if (opening->str() == ">")
++depth;
else if (opening->str() == "<") {
if (--depth == 0)
return opening;
}
}
return opening;
}
Token * Token::findOpeningBracket()
{
// return value of const function
return const_cast<Token*>(const_cast<const Token*>(this)->findOpeningBracket());
}
Formatting: uniformize end of lines.
2008-12-18 22:28:57 +01:00
//---------------------------------------------------------------------------
optimized non-matchcompiled Token::simpleMatch() a bit (#2640)
2020-05-26 20:13:56 +02:00
const Token *Token::findsimplematch(const Token * const startTok, const char pattern[], size_t pattern_len)
Implement Token::findsimplematch und use it for simple patterns
2011-10-27 10:54:50 +02:00
{
Use const_cast to add const, better variable name
2015-08-27 16:14:33 +02:00
for (const Token* tok = startTok; tok; tok = tok->next()) {
optimized non-matchcompiled Token::simpleMatch() a bit (#2640)
2020-05-26 20:13:56 +02:00
if (Token::simpleMatch(tok, pattern, pattern_len))
Implement Token::findsimplematch und use it for simple patterns
2011-10-27 10:54:50 +02:00
return tok;
}
Use nullptr instead of 0 or NULL (#936)
2017-08-09 20:00:26 +02:00
return nullptr;
Implement Token::findsimplematch und use it for simple patterns
2011-10-27 10:54:50 +02:00
}
optimized non-matchcompiled Token::simpleMatch() a bit (#2640)
2020-05-26 20:13:56 +02:00
const Token *Token::findsimplematch(const Token * const startTok, const char pattern[], size_t pattern_len, const Token * const end)
Implement Token::findsimplematch und use it for simple patterns
2011-10-27 10:54:50 +02:00
{
Use const_cast to add const, better variable name
2015-08-27 16:14:33 +02:00
for (const Token* tok = startTok; tok && tok != end; tok = tok->next()) {
optimized non-matchcompiled Token::simpleMatch() a bit (#2640)
2020-05-26 20:13:56 +02:00
if (Token::simpleMatch(tok, pattern, pattern_len))
Implement Token::findsimplematch und use it for simple patterns
2011-10-27 10:54:50 +02:00
return tok;
}
#6269 false positives in case of overloaded standard library functions. Detect memset() with proper argument cound, using new function numberOfArguments()
2015-12-06 12:50:05 +01:00
return nullptr;
Implement Token::findsimplematch und use it for simple patterns
2011-10-27 10:54:50 +02:00
}
Mark members and arguments with 'nonneg'
2019-07-14 12:22:33 +02:00
const Token *Token::findmatch(const Token * const startTok, const char pattern[], const nonneg int varId)
Refactoring: findMatch() that supports varId added. %var1% -> %varid% changed
2009-01-04 20:55:12 +01:00
{
Use const_cast to add const, better variable name
2015-08-27 16:14:33 +02:00
for (const Token* tok = startTok; tok; tok = tok->next()) {
Reverted 'astyle fix'. Those changes are not in sync with my astyle configuration/setup.
2010-04-02 07:30:58 +02:00
if (Token::Match(tok, pattern, varId))
Refactoring: findMatch() that supports varId added. %var1% -> %varid% changed
2009-01-04 20:55:12 +01:00
return tok;
}
#6269 false positives in case of overloaded standard library functions. Detect memset() with proper argument cound, using new function numberOfArguments()
2015-12-06 12:50:05 +01:00
return nullptr;
Refactoring: findMatch() that supports varId added. %var1% -> %varid% changed
2009-01-04 20:55:12 +01:00
}
Mark members and arguments with 'nonneg'
2019-07-14 12:22:33 +02:00
const Token *Token::findmatch(const Token * const startTok, const char pattern[], const Token * const end, const nonneg int varId)
Create a symbol database
2010-07-26 16:46:37 +02:00
{
Use const_cast to add const, better variable name
2015-08-27 16:14:33 +02:00
for (const Token* tok = startTok; tok && tok != end; tok = tok->next()) {
Create a symbol database
2010-07-26 16:46:37 +02:00
if (Token::Match(tok, pattern, varId))
return tok;
}
#6269 false positives in case of overloaded standard library functions. Detect memset() with proper argument cound, using new function numberOfArguments()
2015-12-06 12:50:05 +01:00
return nullptr;
Create a symbol database
2010-07-26 16:46:37 +02:00
}
astyle formatting [ci skip]
2019-07-05 14:00:59 +02:00
void Token::function(const Function *f)
{
Parse lambdas as functions (#1955) * Parse lambdas as functions * Fix issue with missing paren * Fix error when parsing non-existent args * Remove unused function variable
2019-07-05 12:30:42 +02:00
mImpl->mFunction = f;
if (f) {
if (f->isLambda())
tokType(eLambda);
else
tokType(eFunction);
} else if (mTokType == eFunction)
tokType(eName);
}
CheckIO: fixed some more false negatives. Ticket: #4964
2013-09-24 06:43:03 +02:00
void Token::insertToken(const std::string &tokenStr, const std::string &originalNameStr, bool prepend)
{
Fix compiler warning (and minor cleanup).
2014-07-05 12:10:23 +02:00
Token *newToken;
Renamed Token::_str
2018-06-16 23:03:15 +02:00
if (mStr.empty())
CheckIO: fixed some more false negatives. Ticket: #4964
2013-09-24 06:43:03 +02:00
newToken = this;
else
renamed _tokensFrontBack to mTokensFrontBack
2018-06-16 16:22:35 +02:00
newToken = new Token(mTokensFrontBack);
CheckIO: fixed some more false negatives. Ticket: #4964
2013-09-24 06:43:03 +02:00
newToken->str(tokenStr);
Store Token::_originalName on heap, since it is only used for a small amount of Tokens (reduce memory usage of TokenList by ~12% (x64))
2014-06-26 10:57:39 +02:00
if (!originalNameStr.empty())
newToken->originalName(originalNameStr);
CheckIO: fixed some more false negatives. Ticket: #4964
2013-09-24 06:43:03 +02:00
Improved Tokens and Tokenizer: - Better handling of deleteThis() as last element of token list - Code simplified
2012-02-13 17:44:08 +01:00
if (newToken != this) {
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
newToken->mImpl->mLineNumber = mImpl->mLineNumber;
newToken->mImpl->mFileIndex = mImpl->mFileIndex;
newToken->mImpl->mProgressValue = mImpl->mProgressValue;
Adjustment for implementation of Token::insertToken() (#1194) Three attributes are reset in this member function even if their values are stored in the same token object. Move a corresponding sanity check so that less assignments could be performed eventually. Link: https://trac.cppcheck.net/ticket/8532 Signed-off-by: Markus Elfring <elfring@users.sourceforge.net>
2018-04-28 17:17:40 +02:00
Add a 'prepend' bool option to Token::InsertToken. If it's true, add a new token before this. Except when this is the first one in the list, because there's not a way to update the list.front (I noticed some Token functions can potentially delete the list.front(), I don't know which effects does it cause).
2012-11-20 02:58:19 +01:00
if (prepend) {
Fix another template simplifier namespace bug. (#1467) * Fix another template simplifier namespace bug. * Add missing forward declarations caused by token bug.
2018-11-07 21:25:42 +01:00
if (this->previous()) {
Add a 'prepend' bool option to Token::InsertToken. If it's true, add a new token before this. Except when this is the first one in the list, because there's not a way to update the list.front (I noticed some Token functions can potentially delete the list.front(), I don't know which effects does it cause).
2012-11-20 02:58:19 +01:00
newToken->previous(this->previous());
newToken->previous()->next(newToken);
Fix another template simplifier namespace bug. (#1467) * Fix another template simplifier namespace bug. * Add missing forward declarations caused by token bug.
2018-11-07 21:25:42 +01:00
} else if (mTokensFrontBack) {
mTokensFrontBack->front = newToken;
}
Add a 'prepend' bool option to Token::InsertToken. If it's true, add a new token before this. Except when this is the first one in the list, because there's not a way to update the list.front (I noticed some Token functions can potentially delete the list.front(), I don't know which effects does it cause).
2012-11-20 02:58:19 +01:00
this->previous(newToken);
newToken->next(this);
} else {
if (this->next()) {
newToken->next(this->next());
newToken->next()->previous(newToken);
renamed _tokensFrontBack to mTokensFrontBack
2018-06-16 16:22:35 +02:00
} else if (mTokensFrontBack) {
mTokensFrontBack->back = newToken;
Add a 'prepend' bool option to Token::InsertToken. If it's true, add a new token before this. Except when this is the first one in the list, because there's not a way to update the list.front (I noticed some Token functions can potentially delete the list.front(), I don't know which effects does it cause).
2012-11-20 02:58:19 +01:00
}
this->next(newToken);
newToken->previous(this);
Improved Tokens and Tokenizer: - Better handling of deleteThis() as last element of token list - Code simplified
2012-02-13 17:44:08 +01:00
}
Calculate token scopes in advance rather than as the tokenlist is iterated (#2038) * Removed redundant scope calculation * Add scope propagation code to insertToken * Add relevant scope code to Token class * Add code to calculate the scope of Tokens * Add calculateScopes method to class * Add missing include for shared_ptr
2019-07-31 09:19:27 +02:00
if (mImpl->mScopeInfo) {
// If the brace is immediately closed there is no point opening a new scope for it
Fixed Token::isKeyword: Actually set this flag to a reasonable value in all code paths
2020-05-17 17:25:33 +02:00
if (newToken->str() == "{") {
Various clang-tidy fixes (#2192) * use range loops * removed redundant string initializations * use nullptr * use proper boolean false * removed unnecessary continue from end of loop * removed unnecessary c_str() usage * use emplace_back() * removed redundant void arguments
2019-09-25 15:25:19 +02:00
std::string nextScopeNameAddition;
Calculate token scopes in advance rather than as the tokenlist is iterated (#2038) * Removed redundant scope calculation * Add scope propagation code to insertToken * Add relevant scope code to Token class * Add code to calculate the scope of Tokens * Add calculateScopes method to class * Add missing include for shared_ptr
2019-07-31 09:19:27 +02:00
// This might be the opening of a member function
Token *tok1 = newToken;
while (Token::Match(tok1->previous(), "const|volatile|final|override|&|&&|noexcept"))
tok1 = tok1->previous();
Fixed several coverity messages
2020-05-20 18:03:03 +02:00
if (tok1->previous() && tok1->strAt(-1) == ")") {
Calculate token scopes in advance rather than as the tokenlist is iterated (#2038) * Removed redundant scope calculation * Add scope propagation code to insertToken * Add relevant scope code to Token class * Add code to calculate the scope of Tokens * Add calculateScopes method to class * Add missing include for shared_ptr
2019-07-31 09:19:27 +02:00
tok1 = tok1->linkAt(-1);
if (Token::Match(tok1->previous(), "throw|noexcept")) {
tok1 = tok1->previous();
while (Token::Match(tok1->previous(), "const|volatile|final|override|&|&&|noexcept"))
tok1 = tok1->previous();
if (tok1->strAt(-1) != ")")
return;
} else if (Token::Match(newToken->tokAt(-2), ":|, %name%")) {
tok1 = tok1->tokAt(-2);
if (tok1->strAt(-1) != ")")
return;
}
if (tok1->strAt(-1) == ">")
tok1 = tok1->previous()->findOpeningBracket();
if (tok1 && Token::Match(tok1->tokAt(-3), "%name% :: %name%")) {
tok1 = tok1->tokAt(-2);
std::string scope = tok1->strAt(-1);
while (Token::Match(tok1->tokAt(-2), ":: %name%")) {
scope = tok1->strAt(-3) + " :: " + scope;
tok1 = tok1->tokAt(-2);
}
nextScopeNameAddition += scope;
}
}
// Or it might be a namespace/class/struct
if (Token::Match(newToken->previous(), "%name%|>")) {
Token* nameTok = newToken->previous();
while (nameTok && !Token::Match(nameTok, "namespace|class|struct|union %name% {|::|:|<")) {
nameTok = nameTok->previous();
}
if (nameTok) {
for (nameTok = nameTok->next(); nameTok && !Token::Match(nameTok, "{|:|<"); nameTok = nameTok->next()) {
nextScopeNameAddition.append(nameTok->str());
nextScopeNameAddition.append(" ");
}
if (nextScopeNameAddition.length() > 0) nextScopeNameAddition = nextScopeNameAddition.substr(0, nextScopeNameAddition.length() - 1);
}
}
// New scope is opening, record it here
std::shared_ptr<ScopeInfo2> newScopeInfo = std::make_shared<ScopeInfo2>(mImpl->mScopeInfo->name, nullptr, mImpl->mScopeInfo->usingNamespaces);
if (!newScopeInfo->name.empty() && !nextScopeNameAddition.empty()) newScopeInfo->name.append(" :: ");
newScopeInfo->name.append(nextScopeNameAddition);
Fix some bugs in new Scope Calculation code (#2060) * Fix for too much information in scope name When the scope calculation encounters code such as "friend class X::Y;" or "template<> class X<void> {" it will now reset the additional name component of the scope that is about to be opened. * Made sure new scope name is reset after being used
2019-08-02 19:55:08 +02:00
nextScopeNameAddition = "";
Calculate token scopes in advance rather than as the tokenlist is iterated (#2038) * Removed redundant scope calculation * Add scope propagation code to insertToken * Add relevant scope code to Token class * Add code to calculate the scope of Tokens * Add calculateScopes method to class * Add missing include for shared_ptr
2019-07-31 09:19:27 +02:00
newToken->scopeInfo(newScopeInfo);
Fixed Token::isKeyword: Actually set this flag to a reasonable value in all code paths
2020-05-17 17:25:33 +02:00
} else if (newToken->str() == "}") {
Calculate token scopes in advance rather than as the tokenlist is iterated (#2038) * Removed redundant scope calculation * Add scope propagation code to insertToken * Add relevant scope code to Token class * Add code to calculate the scope of Tokens * Add calculateScopes method to class * Add missing include for shared_ptr
2019-07-31 09:19:27 +02:00
Token* matchingTok = newToken->previous();
int depth = 0;
while (matchingTok && (depth != 0 || !Token::simpleMatch(matchingTok, "{"))) {
if (Token::simpleMatch(matchingTok, "}")) depth++;
if (Token::simpleMatch(matchingTok, "{")) depth--;
matchingTok = matchingTok->previous();
}
if (matchingTok && matchingTok->previous()) {
newToken->mImpl->mScopeInfo = matchingTok->previous()->scopeInfo();
}
} else {
if (prepend && newToken->previous()) {
newToken->mImpl->mScopeInfo = newToken->previous()->scopeInfo();
std.cfg: Added *experimental* support for math constants liken M_PI. These are *NOT* standard, but they are supported by GCC/Clang and VS.
2019-07-31 11:11:01 +02:00
} else {
Calculate token scopes in advance rather than as the tokenlist is iterated (#2038) * Removed redundant scope calculation * Add scope propagation code to insertToken * Add relevant scope code to Token class * Add code to calculate the scope of Tokens * Add calculateScopes method to class * Add missing include for shared_ptr
2019-07-31 09:19:27 +02:00
newToken->mImpl->mScopeInfo = mImpl->mScopeInfo;
}
Fixed Token::isKeyword: Actually set this flag to a reasonable value in all code paths
2020-05-17 17:25:33 +02:00
if (newToken->str() == ";") {
Calculate token scopes in advance rather than as the tokenlist is iterated (#2038) * Removed redundant scope calculation * Add scope propagation code to insertToken * Add relevant scope code to Token class * Add code to calculate the scope of Tokens * Add calculateScopes method to class * Add missing include for shared_ptr
2019-07-31 09:19:27 +02:00
const Token* statementStart;
for (statementStart = newToken; statementStart->previous() && !Token::Match(statementStart->previous(), ";|{"); statementStart = statementStart->previous());
if (Token::Match(statementStart, "using namespace %name% ::|;")) {
const Token * tok1 = statementStart->tokAt(2);
std::string nameSpace;
while (tok1 && tok1->str() != ";") {
if (!nameSpace.empty())
nameSpace += " ";
nameSpace += tok1->str();
tok1 = tok1->next();
}
mImpl->mScopeInfo->usingNamespaces.insert(nameSpace);
}
}
}
}
Improved Tokens and Tokenizer: - Better handling of deleteThis() as last element of token list - Code simplified
2012-02-13 17:44:08 +01:00
}
Formatting: uniformize end of lines.
2008-12-18 22:28:57 +01:00
}
Style: Updated the coding style with "astyle"
2009-01-05 16:49:57 +01:00
void Token::eraseTokens(Token *begin, const Token *end)
Formatting: uniformize end of lines.
2008-12-18 22:28:57 +01:00
{
Improved Tokens and Tokenizer: - Better handling of deleteThis() as last element of token list - Code simplified
2012-02-13 17:44:08 +01:00
if (!begin || begin == end)
Formatting: uniformize end of lines.
2008-12-18 22:28:57 +01:00
return;
changed the astyle formatting flags
2011-10-13 20:53:06 +02:00
while (begin->next() && begin->next() != end) {
Formatting: uniformize end of lines.
2008-12-18 22:28:57 +01:00
begin->deleteNext();
}
}
Token::createMutualLinks(): introduce and use. No functional change.
2009-08-22 16:22:50 +02:00
void Token::createMutualLinks(Token *begin, Token *end)
{
use nullptr in lib/token.cpp
2014-02-15 08:05:54 +01:00
assert(begin != nullptr);
assert(end != nullptr);
Token::createMutualLinks(): introduce and use. No functional change.
2009-08-22 16:22:50 +02:00
assert(begin != end);
begin->link(end);
end->link(begin);
}
Style: Updated the coding style with "astyle"
2009-01-05 16:49:57 +01:00
void Token::printOut(const char *title) const
Formatting: uniformize end of lines.
2008-12-18 22:28:57 +01:00
{
Token::printOut(): don't write title if it's empty
2013-08-18 09:01:03 +02:00
if (title && title[0])
Refactorized stringification of tokens: - Function that stringifies one token: Token::stringify() - Functions that stringify a list of tokens: Token::stringifyList() -- Single and powerful "base" function, used by several "light" functions Refactorized - testtokenize.cpp and testsimplifytokens.cpp: Use improved stringification functions instead of several local implementations - Avoided redundand creation of std::string when using TestTokenizer::tokenizeAndStringify and in cmdlineparser.cpp
2012-04-16 19:51:07 +02:00
std::cout << "\n### " << title << " ###\n";
Distinguish exprid and varid in --debug output
2020-11-28 15:41:07 +01:00
std::cout << stringifyList(stringifyOptions::forPrintOut(), nullptr, nullptr) << std::endl;
Token debug function printOut() can now print out filename instead of index
2009-11-27 23:04:04 +01:00
}
void Token::printOut(const char *title, const std::vector<std::string> &fileNames) const
{
Token::printOut(): don't write title if it's empty
2013-08-18 09:01:03 +02:00
if (title && title[0])
Refactorized stringification of tokens: - Function that stringifies one token: Token::stringify() - Functions that stringify a list of tokens: Token::stringifyList() -- Single and powerful "base" function, used by several "light" functions Refactorized - testtokenize.cpp and testsimplifytokens.cpp: Use improved stringification functions instead of several local implementations - Avoided redundand creation of std::string when using TestTokenizer::tokenizeAndStringify and in cmdlineparser.cpp
2012-04-16 19:51:07 +02:00
std::cout << "\n### " << title << " ###\n";
Distinguish exprid and varid in --debug output
2020-11-28 15:41:07 +01:00
std::cout << stringifyList(stringifyOptions::forPrintOut(), &fileNames, nullptr) << std::endl;
Formatting: uniformize end of lines.
2008-12-18 22:28:57 +01:00
}
Token::stringifyList: Added function that stringifies a token list
2009-02-13 07:25:29 +01:00
ExprEngine; Slow processing
2020-05-23 11:30:54 +02:00
void Token::printLines(int lines) const
{
const Token *end = this;
while (end && end->linenr() < lines + linenr())
end = end->next();
Add exprId to tokens (#2744)
2020-08-20 18:21:29 +02:00
std::cout << stringifyList(stringifyOptions::forDebugExprId(), nullptr, end) << std::endl;
ExprEngine; Slow processing
2020-05-23 11:30:54 +02:00
}
Token::stringify; refactor in/out parameter to return value
2021-02-21 17:21:33 +01:00
std::string Token::stringify(const stringifyOptions& options) const
Fixed #3407 (False positive: (inconclusive) Found duplicate branches for if and else. (inline assembler))
2011-12-13 21:42:38 +01:00
{
Token::stringify; refactor in/out parameter to return value
2021-02-21 17:21:33 +01:00
std::string ret;
Add exprId to tokens (#2744)
2020-08-20 18:21:29 +02:00
if (options.attributes) {
Refactorized stringification of tokens: - Function that stringifies one token: Token::stringify() - Functions that stringify a list of tokens: Token::stringifyList() -- Single and powerful "base" function, used by several "light" functions Refactorized - testtokenize.cpp and testsimplifytokens.cpp: Use improved stringification functions instead of several local implementations - Avoided redundand creation of std::string when using TestTokenizer::tokenizeAndStringify and in cmdlineparser.cpp
2012-04-16 19:51:07 +02:00
if (isUnsigned())
Token::stringify; refactor in/out parameter to return value
2021-02-21 17:21:33 +01:00
ret += "unsigned ";
Refactorized stringification of tokens: - Function that stringifies one token: Token::stringify() - Functions that stringify a list of tokens: Token::stringifyList() -- Single and powerful "base" function, used by several "light" functions Refactorized - testtokenize.cpp and testsimplifytokens.cpp: Use improved stringification functions instead of several local implementations - Avoided redundand creation of std::string when using TestTokenizer::tokenizeAndStringify and in cmdlineparser.cpp
2012-04-16 19:51:07 +02:00
else if (isSigned())
Token::stringify; refactor in/out parameter to return value
2021-02-21 17:21:33 +01:00
ret += "signed ";
Rewritten handling of declaration specifiers: - Removed simplifyConst() because it did the opposite of the (superior) simplifyStaticConst() - Execute simplifyStaticConst() in simplifyTokenList1() - there is no reason to defer it, and it is required to properly parse declarations like "unsigned static int i;" - Fixed simplifyStaticConst() to handle more patterns. It did not work at the beginning of the token list and for function arguments - Reimplemented Tokenizer::simplifyStdType() -> properly support all possible ways to declare integers as requested by the standard, instead of only a few common permutations of "unsigned|signed", "short|char|long|int" -> Fixed parsing of _Complex/complex types
2015-08-27 14:34:00 +02:00
if (isComplex())
Token::stringify; refactor in/out parameter to return value
2021-02-21 17:21:33 +01:00
ret += "_Complex ";
CheckIO: improved type info - keep track of original typename.
2013-08-31 06:26:39 +02:00
if (isLong()) {
Keep prefix in string and char literals (#2272) Keeping the prefix in the token allows cppcheck to print the correct string and char literals in debug and error messages. To achieve this, move some of the helper functions from token.cpp to utils.h so that checks that look at string and char literals can reuse them. This is a large part of this commit. Note that the only user visible change is that when string and char literals are printed in error messages, the prefix is now included. For example: int f() { return test.substr( 0 , 4 ) == U"Hello" ? 0 : 1 ; }; now prints U"Hello" instead of "Hello" in the error message.
2019-10-16 11:41:33 +02:00
if (!(mTokType == eString || mTokType == eChar))
Token::stringify; refactor in/out parameter to return value
2021-02-21 17:21:33 +01:00
ret += "long ";
CheckIO: improved type info - keep track of original typename.
2013-08-31 06:26:39 +02:00
}
Fixed #3407 (False positive: (inconclusive) Found duplicate branches for if and else. (inline assembler))
2011-12-13 21:42:38 +01:00
}
Add exprId to tokens (#2744)
2020-08-20 18:21:29 +02:00
if (options.macro && isExpandedMacro())
Token::stringify; refactor in/out parameter to return value
2021-02-21 17:21:33 +01:00
ret += '$';
Renamed Token::_str
2018-06-16 23:03:15 +02:00
if (isName() && mStr.find(' ') != std::string::npos) {
use range loops / constness (#2181) * use range loops / constness * platform.cpp: avoid shadowed variable
2019-09-19 20:29:33 +02:00
for (char i : mStr) {
if (i != ' ')
Token::stringify; refactor in/out parameter to return value
2021-02-21 17:21:33 +01:00
ret += i;
update debug token list output for templates.
2017-12-25 23:16:51 +01:00
}
Renamed Token::_str
2018-06-16 23:03:15 +02:00
} else if (mStr[0] != '\"' || mStr.find('\0') == std::string::npos)
Token::stringify; refactor in/out parameter to return value
2021-02-21 17:21:33 +01:00
ret += mStr;
Tokenizer: Improved handling of '\0' in strings
2012-12-25 08:56:12 +01:00
else {
use range loops / constness (#2181) * use range loops / constness * platform.cpp: avoid shadowed variable
2019-09-19 20:29:33 +02:00
for (char i : mStr) {
if (i == '\0')
Token::stringify; refactor in/out parameter to return value
2021-02-21 17:21:33 +01:00
ret += "\\0";
Tokenizer: Improved handling of '\0' in strings
2012-12-25 08:56:12 +01:00
else
Token::stringify; refactor in/out parameter to return value
2021-02-21 17:21:33 +01:00
ret += i;
Tokenizer: Improved handling of '\0' in strings
2012-12-25 08:56:12 +01:00
}
}
refs issue #9089: avoid usage of expensive std::stringstream (#2996)
2021-01-16 19:03:28 +01:00
if (options.varid && mImpl->mVarId != 0) {
Token::stringify; refactor in/out parameter to return value
2021-02-21 17:21:33 +01:00
ret += '@';
ret += (options.idtype ? "var" : "");
ret += std::to_string(mImpl->mVarId);
refs issue #9089: avoid usage of expensive std::stringstream (#2996)
2021-01-16 19:03:28 +01:00
} else if (options.exprid && mImpl->mExprId != 0) {
Token::stringify; refactor in/out parameter to return value
2021-02-21 17:21:33 +01:00
ret += '@';
ret += (options.idtype ? "expr" : "");
ret += std::to_string(mImpl->mExprId);
refs issue #9089: avoid usage of expensive std::stringstream (#2996)
2021-01-16 19:03:28 +01:00
}
Token::stringify; refactor in/out parameter to return value
2021-02-21 17:21:33 +01:00
return ret;
Fixed #3407 (False positive: (inconclusive) Found duplicate branches for if and else. (inline assembler))
2011-12-13 21:42:38 +01:00
}
Token::stringify; refactor in/out parameter to return value
2021-02-21 17:21:33 +01:00
std::string Token::stringify(bool varid, bool attributes, bool macro) const
Add exprId to tokens (#2744)
2020-08-20 18:21:29 +02:00
{
stringifyOptions options;
options.varid = varid;
options.attributes = attributes;
options.macro = macro;
Token::stringify; refactor in/out parameter to return value
2021-02-21 17:21:33 +01:00
return stringify(options);
Add exprId to tokens (#2744)
2020-08-20 18:21:29 +02:00
}
std::string Token::stringifyList(const stringifyOptions& options, const std::vector<std::string>* fileNames, const Token* end) const
Token debug function printOut() can now print out filename instead of index
2009-11-27 23:04:04 +01:00
{
Refactorized stringification of tokens: - Function that stringifies one token: Token::stringify() - Functions that stringify a list of tokens: Token::stringifyList() -- Single and powerful "base" function, used by several "light" functions Refactorized - testtokenize.cpp and testsimplifytokens.cpp: Use improved stringification functions instead of several local implementations - Avoided redundand creation of std::string when using TestTokenizer::tokenizeAndStringify and in cmdlineparser.cpp
2012-04-16 19:51:07 +02:00
if (this == end)
return "";
Token debug function printOut() can now print out filename instead of index
2009-11-27 23:04:04 +01:00
refs issue #9089: avoid usage of expensive std::stringstream (#2996)
2021-01-16 19:03:28 +01:00
std::string ret;
Refactoring: Combined printOut() and stringifyList() in Token class
2009-03-03 20:45:58 +01:00
Add exprId to tokens (#2744)
2020-08-20 18:21:29 +02:00
unsigned int lineNumber = mImpl->mLineNumber - (options.linenumbers ? 1U : 0U);
unsigned int fileIndex = options.files ? ~0U : mImpl->mFileIndex;
Fixed #1487 (fix gcc compiler warnings)
2010-08-06 21:02:43 +02:00
std::map<int, unsigned int> lineNumbers;
Refactorized stringification of tokens: - Function that stringifies one token: Token::stringify() - Functions that stringify a list of tokens: Token::stringifyList() -- Single and powerful "base" function, used by several "light" functions Refactorized - testtokenize.cpp and testsimplifytokens.cpp: Use improved stringification functions instead of several local implementations - Avoided redundand creation of std::string when using TestTokenizer::tokenizeAndStringify and in cmdlineparser.cpp
2012-04-16 19:51:07 +02:00
for (const Token *tok = this; tok != end; tok = tok->next()) {
Fix ticket Add/Change #125 (the printout of the token list is wrong)
2009-03-03 21:17:23 +01:00
bool fileChange = false;
Rename fileInd to fileIndex
2019-07-17 16:28:47 +02:00
if (tok->mImpl->mFileIndex != fileIndex) {
if (fileIndex != ~0U) {
lineNumbers[fileIndex] = tok->mImpl->mFileIndex;
Fix ticket Add/Change #125 (the printout of the token list is wrong)
2009-03-03 21:17:23 +01:00
}
Rename fileInd to fileIndex
2019-07-17 16:28:47 +02:00
fileIndex = tok->mImpl->mFileIndex;
Add exprId to tokens (#2744)
2020-08-20 18:21:29 +02:00
if (options.files) {
refs issue #9089: avoid usage of expensive std::stringstream (#2996)
2021-01-16 19:03:28 +01:00
ret += "\n\n##file ";
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
if (fileNames && fileNames->size() > tok->mImpl->mFileIndex)
refs issue #9089: avoid usage of expensive std::stringstream (#2996)
2021-01-16 19:03:28 +01:00
ret += fileNames->at(tok->mImpl->mFileIndex);
Refactorized stringification of tokens: - Function that stringifies one token: Token::stringify() - Functions that stringify a list of tokens: Token::stringifyList() -- Single and powerful "base" function, used by several "light" functions Refactorized - testtokenize.cpp and testsimplifytokens.cpp: Use improved stringification functions instead of several local implementations - Avoided redundand creation of std::string when using TestTokenizer::tokenizeAndStringify and in cmdlineparser.cpp
2012-04-16 19:51:07 +02:00
else
refs issue #9089: avoid usage of expensive std::stringstream (#2996)
2021-01-16 19:03:28 +01:00
ret += std::to_string(fileIndex);
ret += '\n';
Refactorized stringification of tokens: - Function that stringifies one token: Token::stringify() - Functions that stringify a list of tokens: Token::stringifyList() -- Single and powerful "base" function, used by several "light" functions Refactorized - testtokenize.cpp and testsimplifytokens.cpp: Use improved stringification functions instead of several local implementations - Avoided redundand creation of std::string when using TestTokenizer::tokenizeAndStringify and in cmdlineparser.cpp
2012-04-16 19:51:07 +02:00
}
Fix ticket Add/Change #125 (the printout of the token list is wrong)
2009-03-03 21:17:23 +01:00
Rename fileInd to fileIndex
2019-07-17 16:28:47 +02:00
lineNumber = lineNumbers[fileIndex];
Fix ticket Add/Change #125 (the printout of the token list is wrong)
2009-03-03 21:17:23 +01:00
fileChange = true;
}
Add exprId to tokens (#2744)
2020-08-20 18:21:29 +02:00
if (options.linebreaks && (lineNumber != tok->linenr() || fileChange)) {
Rename fileInd to fileIndex
2019-07-17 16:28:47 +02:00
if (lineNumber+4 < tok->linenr() && fileIndex == tok->mImpl->mFileIndex) {
refs issue #9089: avoid usage of expensive std::stringstream (#2996)
2021-01-16 19:03:28 +01:00
ret += '\n';
ret += std::to_string(lineNumber+1);
ret += ":\n|\n";
ret += std::to_string(tok->linenr()-1);
ret += ":\n";
ret += std::to_string(tok->linenr());
ret += ": ";
Add exprId to tokens (#2744)
2020-08-20 18:21:29 +02:00
} else if (this == tok && options.linenumbers) {
refs issue #9089: avoid usage of expensive std::stringstream (#2996)
2021-01-16 19:03:28 +01:00
ret += std::to_string(tok->linenr());
ret += ": ";
fix template debug output line numbers (#2938)
2020-12-08 10:35:13 +01:00
} else if (lineNumber > tok->linenr()) {
lineNumber = tok->linenr();
refs issue #9089: avoid usage of expensive std::stringstream (#2996)
2021-01-16 19:03:28 +01:00
ret += '\n';
fix template debug output line numbers (#2938)
2020-12-08 10:35:13 +01:00
if (options.linenumbers) {
refs issue #9089: avoid usage of expensive std::stringstream (#2996)
2021-01-16 19:03:28 +01:00
ret += std::to_string(lineNumber);
ret += ':';
Add symbolic values to ValueFlow (#3367)
2021-07-30 21:29:35 +02:00
ret += ' ';
fix template debug output line numbers (#2938)
2020-12-08 10:35:13 +01:00
}
Improvements in token.cpp: - Shortened stringify output - 4 or more subsequent empty lines are abbreviated - Fixed detection of comparision operators - Make use of _type: Replaced three asserts by one
2012-08-02 20:36:54 +02:00
} else {
while (lineNumber < tok->linenr()) {
++lineNumber;
refs issue #9089: avoid usage of expensive std::stringstream (#2996)
2021-01-16 19:03:28 +01:00
ret += '\n';
Add exprId to tokens (#2744)
2020-08-20 18:21:29 +02:00
if (options.linenumbers) {
refs issue #9089: avoid usage of expensive std::stringstream (#2996)
2021-01-16 19:03:28 +01:00
ret += std::to_string(lineNumber);
ret += ':';
Improvements in token.cpp: - Shortened stringify output - 4 or more subsequent empty lines are abbreviated - Fixed detection of comparision operators - Make use of _type: Replaced three asserts by one
2012-08-02 20:36:54 +02:00
if (lineNumber == tok->linenr())
refs issue #9089: avoid usage of expensive std::stringstream (#2996)
2021-01-16 19:03:28 +01:00
ret += ' ';
Improvements in token.cpp: - Shortened stringify output - 4 or more subsequent empty lines are abbreviated - Fixed detection of comparision operators - Make use of _type: Replaced three asserts by one
2012-08-02 20:36:54 +02:00
}
Refactorized stringification of tokens: - Function that stringifies one token: Token::stringify() - Functions that stringify a list of tokens: Token::stringifyList() -- Single and powerful "base" function, used by several "light" functions Refactorized - testtokenize.cpp and testsimplifytokens.cpp: Use improved stringification functions instead of several local implementations - Avoided redundand creation of std::string when using TestTokenizer::tokenizeAndStringify and in cmdlineparser.cpp
2012-04-16 19:51:07 +02:00
}
Fix ticket Add/Change #125 (the printout of the token list is wrong)
2009-03-03 21:17:23 +01:00
}
Fix more -Wshadow compile warnings
2010-04-09 21:40:37 +02:00
lineNumber = tok->linenr();
Token::stringifyList: Added function that stringifies a token list
2009-02-13 07:25:29 +01:00
}
Fix ticket Add/Change #125 (the printout of the token list is wrong)
2009-03-03 21:17:23 +01:00
Token::stringify; refactor in/out parameter to return value
2021-02-21 17:21:33 +01:00
ret += tok->stringify(options); // print token
fix template debug output line numbers (#2938)
2020-12-08 10:35:13 +01:00
if (tok->next() != end && (!options.linebreaks || (tok->next()->linenr() == tok->linenr() && tok->next()->fileIndex() == tok->fileIndex())))
refs issue #9089: avoid usage of expensive std::stringstream (#2996)
2021-01-16 19:03:28 +01:00
ret += ' ';
Token::stringifyList: Added function that stringifies a token list
2009-02-13 07:25:29 +01:00
}
Add exprId to tokens (#2744)
2020-08-20 18:21:29 +02:00
if (options.linebreaks && (options.files || options.linenumbers))
refs issue #9089: avoid usage of expensive std::stringstream (#2996)
2021-01-16 19:03:28 +01:00
ret += '\n';
return ret;
Token::stringifyList: Added function that stringifies a token list
2009-02-13 07:25:29 +01:00
}
Add exprId to tokens (#2744)
2020-08-20 18:21:29 +02:00
std::string Token::stringifyList(bool varid, bool attributes, bool linenumbers, bool linebreaks, bool files, const std::vector<std::string>* fileNames, const Token* end) const
{
stringifyOptions options;
options.varid = varid;
options.attributes = attributes;
options.macro = attributes;
options.linenumbers = linenumbers;
options.linebreaks = linebreaks;
options.files = files;
return stringifyList(options, fileNames, end);
}
Token::stringifyList: Added function that stringifies a token list
2009-02-13 07:25:29 +01:00
Refactorized stringification of tokens: - Function that stringifies one token: Token::stringify() - Functions that stringify a list of tokens: Token::stringifyList() -- Single and powerful "base" function, used by several "light" functions Refactorized - testtokenize.cpp and testsimplifytokens.cpp: Use improved stringification functions instead of several local implementations - Avoided redundand creation of std::string when using TestTokenizer::tokenizeAndStringify and in cmdlineparser.cpp
2012-04-16 19:51:07 +02:00
std::string Token::stringifyList(const Token* end, bool attributes) const
{
Use nullptr instead of 0 or NULL (#936)
2017-08-09 20:00:26 +02:00
return stringifyList(false, attributes, false, false, false, nullptr, end);
Refactorized stringification of tokens: - Function that stringifies one token: Token::stringify() - Functions that stringify a list of tokens: Token::stringifyList() -- Single and powerful "base" function, used by several "light" functions Refactorized - testtokenize.cpp and testsimplifytokens.cpp: Use improved stringification functions instead of several local implementations - Avoided redundand creation of std::string when using TestTokenizer::tokenizeAndStringify and in cmdlineparser.cpp
2012-04-16 19:51:07 +02:00
}
std::string Token::stringifyList(bool varid) const
{
Use nullptr instead of 0 or NULL (#936)
2017-08-09 20:00:26 +02:00
return stringifyList(varid, false, true, true, true, nullptr, nullptr);
Refactorized stringification of tokens: - Function that stringifies one token: Token::stringify() - Functions that stringify a list of tokens: Token::stringifyList() -- Single and powerful "base" function, used by several "light" functions Refactorized - testtokenize.cpp and testsimplifytokens.cpp: Use improved stringification functions instead of several local implementations - Avoided redundand creation of std::string when using TestTokenizer::tokenizeAndStringify and in cmdlineparser.cpp
2012-04-16 19:51:07 +02:00
}
AST: Simple AST. Not used in the clients. Only part of the testing for now.
2012-12-15 20:21:09 +01:00
Improve AST cyclic detection performance (#3330)
2021-07-08 21:13:51 +02:00
void Token::astParent(Token* tok)
Check for cycles in AST when adding nodes to prevent hangs from cyclic ASTs (#3329)
2021-07-07 08:21:35 +02:00
{
Improve AST cyclic detection performance (#3330)
2021-07-08 21:13:51 +02:00
const Token* tok2 = tok;
while (tok2) {
if (this == tok2)
throw InternalError(this, "Internal error. AST cyclic dependency.");
tok2 = tok2->astParent();
Check for cycles in AST when adding nodes to prevent hangs from cyclic ASTs (#3329)
2021-07-07 08:21:35 +02:00
}
Improve AST cyclic detection performance (#3330)
2021-07-08 21:13:51 +02:00
// Clear children to avoid nodes referenced twice
if (this->astParent()) {
Token* parent = this->astParent();
if (parent->astOperand1() == this)
parent->mImpl->mAstOperand1 = nullptr;
if (parent->astOperand2() == this)
parent->mImpl->mAstOperand2 = nullptr;
}
mImpl->mAstParent = tok;
Check for cycles in AST when adding nodes to prevent hangs from cyclic ASTs (#3329)
2021-07-07 08:21:35 +02:00
}
AST: Simple AST. Not used in the clients. Only part of the testing for now.
2012-12-15 20:21:09 +01:00
void Token::astOperand1(Token *tok)
{
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
if (mImpl->mAstOperand1)
Improve AST cyclic detection performance (#3330)
2021-07-08 21:13:51 +02:00
mImpl->mAstOperand1->astParent(nullptr);
AST: generate tree from bottom and upwards
2012-12-16 11:48:19 +01:00
// goto parent operator
Fixed two bugs in AST compilation (#5830): - Correctly parse operators .* and ->* - Support empty branch in ternary expression (a?:c)
2014-05-19 21:54:59 +02:00
if (tok) {
Improve AST cyclic detection performance (#3330)
2021-07-08 21:13:51 +02:00
tok = tok->astTop();
tok->astParent(this);
Fixed two bugs in AST compilation (#5830): - Correctly parse operators .* and ->* - Support empty branch in ternary expression (a?:c)
2014-05-19 21:54:59 +02:00
}
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
mImpl->mAstOperand1 = tok;
AST: Simple AST. Not used in the clients. Only part of the testing for now.
2012-12-15 20:21:09 +01:00
}
void Token::astOperand2(Token *tok)
{
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
if (mImpl->mAstOperand2)
Improve AST cyclic detection performance (#3330)
2021-07-08 21:13:51 +02:00
mImpl->mAstOperand2->astParent(nullptr);
AST: generate tree from bottom and upwards
2012-12-16 11:48:19 +01:00
// goto parent operator
Fixed two bugs in AST compilation (#5830): - Correctly parse operators .* and ->* - Support empty branch in ternary expression (a?:c)
2014-05-19 21:54:59 +02:00
if (tok) {
Improve AST cyclic detection performance (#3330)
2021-07-08 21:13:51 +02:00
tok = tok->astTop();
tok->astParent(this);
Fixed two bugs in AST compilation (#5830): - Correctly parse operators .* and ->* - Support empty branch in ternary expression (a?:c)
2014-05-19 21:54:59 +02:00
}
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
mImpl->mAstOperand2 = tok;
AST: Simple AST. Not used in the clients. Only part of the testing for now.
2012-12-15 20:21:09 +01:00
}
Refactoring: Reuse Token::expressionString() logic for finding right-most leaf in tree
2018-10-18 12:09:55 +02:00
static const Token* goToLeftParenthesis(const Token* start, const Token* end)
{
// move start to lpar in such expression: '(*it).x'
int par = 0;
for (const Token *tok = start; tok && tok != end; tok = tok->next()) {
if (tok->str() == "(")
++par;
else if (tok->str() == ")") {
if (par == 0)
start = tok->link();
else
--par;
}
}
return start;
}
static const Token* goToRightParenthesis(const Token* start, const Token* end)
{
// move end to rpar in such expression: '2>(x+1)'
int par = 0;
for (const Token *tok = end; tok && tok != start; tok = tok->previous()) {
if (tok->str() == ")")
++par;
else if (tok->str() == "(") {
if (par == 0)
end = tok->link();
else
--par;
}
}
return end;
}
std::pair<const Token *, const Token *> Token::findExpressionStartEndTokens() const
{
const Token * const top = this;
Fixed #8941 (False Positive: Variable 'f' is assigned a value that is never used.)
2019-01-20 13:20:23 +01:00
// find start node in AST tree
Refactoring: Reuse Token::expressionString() logic for finding right-most leaf in tree
2018-10-18 12:09:55 +02:00
const Token *start = top;
Fixed #9737 (False positive: unreadVariable)
2020-10-25 20:32:45 +01:00
while (start->astOperand1() && precedes(start->astOperand1(), start))
Refactoring: Reuse Token::expressionString() logic for finding right-most leaf in tree
2018-10-18 12:09:55 +02:00
start = start->astOperand1();
Fixed #8941 (False Positive: Variable 'f' is assigned a value that is never used.)
2019-01-20 13:20:23 +01:00
// find end node in AST tree
Refactoring: Reuse Token::expressionString() logic for finding right-most leaf in tree
2018-10-18 12:09:55 +02:00
const Token *end = top;
while (end->astOperand1() && (end->astOperand2() || end->isUnaryPreOp())) {
Fixed #8941 (False Positive: Variable 'f' is assigned a value that is never used.)
2019-01-20 13:20:23 +01:00
// lambda..
if (end->str() == "[") {
const Token *lambdaEnd = findLambdaEndToken(end);
if (lambdaEnd) {
end = lambdaEnd;
break;
}
Fix Token::findExpressionStartEndTokens() regression
2019-01-20 13:23:19 +01:00
}
AST: Better handling of '(type){..}'
2020-03-08 16:45:51 +01:00
if (Token::Match(end,"(|[|{") &&
Fix Token::findExpressionStartEndTokens() regression
2019-01-20 13:23:19 +01:00
!(Token::Match(end, "( %type%") && !end->astOperand2())) {
Refactoring: Reuse Token::expressionString() logic for finding right-most leaf in tree
2018-10-18 12:09:55 +02:00
end = end->link();
break;
}
end = end->astOperand2() ? end->astOperand2() : end->astOperand1();
}
Fixed #8941 (False Positive: Variable 'f' is assigned a value that is never used.)
2019-01-20 13:20:23 +01:00
// skip parentheses
Refactoring: Reuse Token::expressionString() logic for finding right-most leaf in tree
2018-10-18 12:09:55 +02:00
start = goToLeftParenthesis(start, end);
end = goToRightParenthesis(start, end);
Unused value: Fix false positive (ast, {})
2018-12-14 18:56:09 +01:00
if (Token::simpleMatch(end, "{"))
end = end->link();
Refactoring: Reuse Token::expressionString() logic for finding right-most leaf in tree
2018-10-18 12:09:55 +02:00
return std::pair<const Token *, const Token *>(start,end);
}
Revert "Fixed false negative #5815" This reverts commit dc6c278d83527b711124b4856cff641b5f36d0e6.
2015-09-09 14:46:47 +02:00
bool Token::isCalculation() const
clarify calculation: Fixed FP for 'a = (*p ? 1 : 2)'
2013-12-28 11:02:39 +01:00
{
if (!Token::Match(this, "%cop%|++|--"))
return false;
if (Token::Match(this, "*|&")) {
// dereference or address-of?
if (!this->astOperand2())
return false;
if (this->astOperand2()->str() == "[")
return false;
// type specification?
std::stack<const Token *> operands;
operands.push(this);
while (!operands.empty()) {
const Token *op = operands.top();
operands.pop();
if (op->isNumber() || op->varId() > 0)
return true;
if (op->astOperand1())
operands.push(op->astOperand1());
if (op->astOperand2())
operands.push(op->astOperand2());
else if (Token::Match(op, "*|&"))
return false;
}
// type specification => return false
return false;
}
return true;
}
Fixed #6560 (ValueFlow: handling ternary operator better in valueFlowSubFunction)
2015-05-24 17:02:00 +02:00
bool Token::isUnaryPreOp() const
Suspicious string comparison: Refactoring using AST. Fixed FP in Lac.
2014-07-28 14:27:35 +02:00
{
Fixed #6560 (ValueFlow: handling ternary operator better in valueFlowSubFunction)
2015-05-24 17:02:00 +02:00
if (!astOperand1() || astOperand2())
Suspicious string comparison: Refactoring using AST. Fixed FP in Lac.
2014-07-28 14:27:35 +02:00
return false;
Fixed #6560 (ValueFlow: handling ternary operator better in valueFlowSubFunction)
2015-05-24 17:02:00 +02:00
if (!Token::Match(this, "++|--"))
Suspicious string comparison: Refactoring using AST. Fixed FP in Lac.
2014-07-28 14:27:35 +02:00
return true;
rename _previous and _next
2018-06-16 16:16:55 +02:00
const Token *tokbefore = mPrevious;
const Token *tokafter = mNext;
Fixed crash on garbage code (#7074)
2015-10-26 13:29:47 +01:00
for (int distance = 1; distance < 10 && tokbefore; distance++) {
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
if (tokbefore == mImpl->mAstOperand1)
Suspicious string comparison: Refactoring using AST. Fixed FP in Lac.
2014-07-28 14:27:35 +02:00
return false;
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
if (tokafter == mImpl->mAstOperand1)
Suspicious string comparison: Refactoring using AST. Fixed FP in Lac.
2014-07-28 14:27:35 +02:00
return true;
rename _previous and _next
2018-06-16 16:16:55 +02:00
tokbefore = tokbefore->mPrevious;
tokafter = tokafter->mPrevious;
Suspicious string comparison: Refactoring using AST. Fixed FP in Lac.
2014-07-28 14:27:35 +02:00
}
return false; // <- guess
}
Show struct member in unsignedLessThanZeroError warning Before: [/tmp/test.c:8]: (style) Checking if unsigned variable '.' is less than zero. [/tmp/test.c:12]: (style) Checking if unsigned variable '.' is less than zero. After: [/tmp/test.c:8]: (style) Checking if unsigned variable 'd.n' is less than zero. [/tmp/test.c:12]: (style) Checking if unsigned variable 'd.n' is less than zero.
2016-09-19 06:13:09 +02:00
static std::string stringFromTokenRange(const Token* start, const Token* end)
{
refs issue #9089: avoid usage of expensive std::stringstream (#2996)
2021-01-16 19:03:28 +01:00
std::string ret;
Fixed #8085 (Token::expressionString: unsigned long long)
2017-07-21 09:16:42 +02:00
if (end)
astyle formatting [ci skip]
2017-07-21 09:17:25 +02:00
end = end->next();
Refactoring: Moved astStringify to the Token class and renamed it to expressionString. This can often be useful when reporting error messages that contains source code.
2014-01-17 18:37:49 +01:00
for (const Token *tok = start; tok && tok != end; tok = tok->next()) {
astyle formatting [ci skip]
2017-07-21 09:17:25 +02:00
if (tok->isUnsigned())
refs issue #9089: avoid usage of expensive std::stringstream (#2996)
2021-01-16 19:03:28 +01:00
ret += "unsigned ";
Keep prefix in string and char literals (#2272) Keeping the prefix in the token allows cppcheck to print the correct string and char literals in debug and error messages. To achieve this, move some of the helper functions from token.cpp to utils.h so that checks that look at string and char literals can reuse them. This is a large part of this commit. Note that the only user visible change is that when string and char literals are printed in error messages, the prefix is now included. For example: int f() { return test.substr( 0 , 4 ) == U"Hello" ? 0 : 1 ; }; now prints U"Hello" instead of "Hello" in the error message.
2019-10-16 11:41:33 +02:00
if (tok->isLong() && !tok->isLiteral())
refs issue #9089: avoid usage of expensive std::stringstream (#2996)
2021-01-16 19:03:28 +01:00
ret += "long ";
Fixed #8506 (CPPCheck printing invalid characters in output)
2020-09-04 20:43:54 +02:00
if (tok->tokType() == Token::eString) {
for (unsigned char c: tok->str()) {
if (c == '\n')
refs issue #9089: avoid usage of expensive std::stringstream (#2996)
2021-01-16 19:03:28 +01:00
ret += "\\n";
Fixed #8506 (CPPCheck printing invalid characters in output)
2020-09-04 20:43:54 +02:00
else if (c == '\r')
refs issue #9089: avoid usage of expensive std::stringstream (#2996)
2021-01-16 19:03:28 +01:00
ret += "\\r";
Fixed #8506 (CPPCheck printing invalid characters in output)
2020-09-04 20:43:54 +02:00
else if (c == '\t')
refs issue #9089: avoid usage of expensive std::stringstream (#2996)
2021-01-16 19:03:28 +01:00
ret += "\\t";
Fixed #8506 (CPPCheck printing invalid characters in output)
2020-09-04 20:43:54 +02:00
else if (c >= ' ' && c <= 126)
refs issue #9089: avoid usage of expensive std::stringstream (#2996)
2021-01-16 19:03:28 +01:00
ret += c;
Fixed #8506 (CPPCheck printing invalid characters in output)
2020-09-04 20:43:54 +02:00
else {
char str[10];
sprintf(str, "\\x%02x", c);
refs issue #9089: avoid usage of expensive std::stringstream (#2996)
2021-01-16 19:03:28 +01:00
ret += str;
Fixed #8506 (CPPCheck printing invalid characters in output)
2020-09-04 20:43:54 +02:00
}
}
} else if (tok->originalName().empty() || tok->isUnsigned() || tok->isLong()) {
refs issue #9089: avoid usage of expensive std::stringstream (#2996)
2021-01-16 19:03:28 +01:00
ret += tok->str();
Fixed #8085 (Token::expressionString: unsigned long long)
2017-07-21 09:16:42 +02:00
} else
refs issue #9089: avoid usage of expensive std::stringstream (#2996)
2021-01-16 19:03:28 +01:00
ret += tok->originalName();
Refactorization: Renamed Token::Match pattern %var% to %name%, implement new pattern %var% which is true if varId > 0.
2015-01-31 10:50:39 +01:00
if (Token::Match(tok, "%name%|%num% %name%|%num%"))
refs issue #9089: avoid usage of expensive std::stringstream (#2996)
2021-01-16 19:03:28 +01:00
ret += ' ';
Refactoring: Moved astStringify to the Token class and renamed it to expressionString. This can often be useful when reporting error messages that contains source code.
2014-01-17 18:37:49 +01:00
}
refs issue #9089: avoid usage of expensive std::stringstream (#2996)
2021-01-16 19:03:28 +01:00
return ret;
Refactoring: Moved astStringify to the Token class and renamed it to expressionString. This can often be useful when reporting error messages that contains source code.
2014-01-17 18:37:49 +01:00
}
Show struct member in unsignedLessThanZeroError warning Before: [/tmp/test.c:8]: (style) Checking if unsigned variable '.' is less than zero. [/tmp/test.c:12]: (style) Checking if unsigned variable '.' is less than zero. After: [/tmp/test.c:8]: (style) Checking if unsigned variable 'd.n' is less than zero. [/tmp/test.c:12]: (style) Checking if unsigned variable 'd.n' is less than zero.
2016-09-19 06:13:09 +02:00
std::string Token::expressionString() const
{
Refactoring: Reuse Token::expressionString() logic for finding right-most leaf in tree
2018-10-18 12:09:55 +02:00
const auto tokens = findExpressionStartEndTokens();
return stringFromTokenRange(tokens.first, tokens.second);
Show struct member in unsignedLessThanZeroError warning Before: [/tmp/test.c:8]: (style) Checking if unsigned variable '.' is less than zero. [/tmp/test.c:12]: (style) Checking if unsigned variable '.' is less than zero. After: [/tmp/test.c:8]: (style) Checking if unsigned variable 'd.n' is less than zero. [/tmp/test.c:12]: (style) Checking if unsigned variable 'd.n' is less than zero.
2016-09-19 06:13:09 +02:00
}
Mark members and arguments with 'nonneg'
2019-07-14 12:22:33 +02:00
static void astStringXml(const Token *tok, nonneg int indent, std::ostream &out)
Add simple xml debug output. When both --xml and --debug is used there will be xml debug output.
2014-07-13 17:21:45 +02:00
{
const std::string strindent(indent, ' ');
Fixed #5982 (Add xml dump)
2014-07-14 15:51:45 +02:00
out << strindent << "<token str=\"" << tok->str() << '\"';
use nonneg int for varid and exprid (#3085)
2021-01-27 19:49:13 +01:00
if (tok->varId())
Fixed #5982 (Add xml dump)
2014-07-14 15:51:45 +02:00
out << " varId=\"" << MathLib::toString(tok->varId()) << '\"';
if (tok->variable())
out << " variable=\"" << tok->variable() << '\"';
if (tok->function())
out << " function=\"" << tok->function() << '\"';
Refactorization: Allocate Token::_values (ValueFlow information) dynamically, reducing size of each token by around 10%
2017-03-27 18:48:34 +02:00
if (!tok->values().empty())
out << " values=\"" << &tok->values() << '\"';
Fixed #5982 (Add xml dump)
2014-07-14 15:51:45 +02:00
Add simple xml debug output. When both --xml and --debug is used there will be xml debug output.
2014-07-13 17:21:45 +02:00
if (!tok->astOperand1() && !tok->astOperand2()) {
Fixed #5982 (Add xml dump)
2014-07-14 15:51:45 +02:00
out << "/>" << std::endl;
Add simple xml debug output. When both --xml and --debug is used there will be xml debug output.
2014-07-13 17:21:45 +02:00
}
Fixed #5982 (Add xml dump)
2014-07-14 15:51:45 +02:00
else {
out << '>' << std::endl;
if (tok->astOperand1())
astStringXml(tok->astOperand1(), indent+2U, out);
if (tok->astOperand2())
astStringXml(tok->astOperand2(), indent+2U, out);
out << strindent << "</token>" << std::endl;
}
Add simple xml debug output. When both --xml and --debug is used there will be xml debug output.
2014-07-13 17:21:45 +02:00
}
AST: Tweaked debug output
2020-10-31 18:57:48 +01:00
void Token::printAst(bool verbose, bool xml, const std::vector<std::string> &fileNames, std::ostream &out) const
Make it possible to create AST, by using the --ast flag
2013-11-02 18:37:35 +01:00
{
Adjust cppcheck AST debug output a little bit. Write location for each AST tree
2020-10-28 21:41:21 +01:00
if (!xml)
out << "\n\n##AST" << std::endl;
AST: Updated printAst
2015-07-21 11:54:11 +02:00
std::set<const Token *> printed;
Make it possible to create AST, by using the --ast flag
2013-11-02 18:37:35 +01:00
for (const Token *tok = this; tok; tok = tok->next()) {
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
if (!tok->mImpl->mAstParent && tok->mImpl->mAstOperand1) {
Adjust cppcheck AST debug output a little bit. Write location for each AST tree
2020-10-28 21:41:21 +01:00
if (printed.find(tok) != printed.end())
AST: Updated printAst
2015-07-21 11:54:11 +02:00
continue;
printed.insert(tok);
Add simple xml debug output. When both --xml and --debug is used there will be xml debug output.
2014-07-13 17:21:45 +02:00
if (xml) {
addons: write column in error message(s)
2019-04-13 10:22:13 +02:00
out << "<ast scope=\"" << tok->scope() << "\" fileIndex=\"" << tok->fileIndex() << "\" linenr=\"" << tok->linenr()
Refactoring: Rename variables. Do not use leading _. Renamed 'col' to 'column'
2019-08-18 12:19:05 +02:00
<< "\" column=\"" << tok->column() << "\">" << std::endl;
AST: Updated printAst
2015-07-21 11:54:11 +02:00
astStringXml(tok, 2U, out);
Fixed #5982 (Add xml dump)
2014-07-14 15:51:45 +02:00
out << "</ast>" << std::endl;
Add simple xml debug output. When both --xml and --debug is used there will be xml debug output.
2014-07-13 17:21:45 +02:00
} else if (verbose)
AST: Tweaked debug output
2020-10-31 18:57:48 +01:00
out << "[" << fileNames[tok->fileIndex()] << ":" << tok->linenr() << "]" << std::endl << tok->astStringVerbose() << std::endl;
AST: improved AST printout when --verbose is used
2014-02-24 17:22:29 +01:00
else
AST: Updated printAst
2015-07-21 11:54:11 +02:00
out << tok->astString(" ") << std::endl;
added debug output for the value flow
2014-01-18 09:58:32 +01:00
if (tok->str() == "(")
tok = tok->link();
Make it possible to create AST, by using the --ast flag
2013-11-02 18:37:35 +01:00
}
}
}
added debug output for the value flow
2014-01-18 09:58:32 +01:00
Mark members and arguments with 'nonneg'
2019-07-14 12:22:33 +02:00
static void indent(std::string &str, const nonneg int indent1, const nonneg int indent2)
AST: improved AST printout when --verbose is used
2014-02-24 17:22:29 +01:00
{
Avoid 'unsigned' for int variables
2019-07-13 16:06:24 +02:00
for (int i = 0; i < indent1; ++i)
Optimize astStringVerbose() for large arrays (#1815) Change the astStringVerbose() recursion to extend a string instead of returning one. This has the benefit that for tokens where the recursion runs deep (typically large arrays), the time savings can be substantial (see comments on benchmarks further down). The reason is that previously, for each token, the astString of its operands was constructed, and then appended to this tokens astString. This led to a lot of unnecessary string copying (and with that allocations). Instead, by passing the string by reference, the number of temporary strings is greatly reduced. Another way of seeing it is that previously, the string was constructed from end to beginning, but now it is constructed from the beginning to end. There was no notable speedup by preallocating the entire string using string::reserve() (at least not on Linux). To benchmark, the changes and master were tested on Linux using the commands: make time cppcheck --debug --verbose $file >/dev/null i.e., the cppcheck binary was compiled with the settings in the Makefile. Printing the output to screen or file will of course take longer time. In Trac ticket #8355 which triggered this change, an example file from the Wine repository was attached. Running the above cppcheck on master took 24 minutes and with the changes in this commmit, took 22 seconds. Another test made was on lib/tokenlist.cpp in the cppcheck repo, which is more "normal" file. On that file there was no measurable time difference. A synthetic benchmark was generated to illustrate the effects on dumping the ast for arrays of different sizes. The generate code looked as follows: const int array[] = {...}; with different number of elements. The results are as follows (times are in seconds): N master optimized 10 0.1 0.1 100 0.1 0.1 1000 2.8 0.7 2000 19 1.8 3000 53 3.8 5000 350 10 10000 3215 38 As we can see, for small arrays, there is no time difference, but for large arrays the time savings are substantial.
2019-04-30 13:35:48 +02:00
str += ' ';
Avoid 'unsigned' for int variables
2019-07-13 16:06:24 +02:00
for (int i = indent1; i < indent2; i += 2)
Optimize astStringVerbose() for large arrays (#1815) Change the astStringVerbose() recursion to extend a string instead of returning one. This has the benefit that for tokens where the recursion runs deep (typically large arrays), the time savings can be substantial (see comments on benchmarks further down). The reason is that previously, for each token, the astString of its operands was constructed, and then appended to this tokens astString. This led to a lot of unnecessary string copying (and with that allocations). Instead, by passing the string by reference, the number of temporary strings is greatly reduced. Another way of seeing it is that previously, the string was constructed from end to beginning, but now it is constructed from the beginning to end. There was no notable speedup by preallocating the entire string using string::reserve() (at least not on Linux). To benchmark, the changes and master were tested on Linux using the commands: make time cppcheck --debug --verbose $file >/dev/null i.e., the cppcheck binary was compiled with the settings in the Makefile. Printing the output to screen or file will of course take longer time. In Trac ticket #8355 which triggered this change, an example file from the Wine repository was attached. Running the above cppcheck on master took 24 minutes and with the changes in this commmit, took 22 seconds. Another test made was on lib/tokenlist.cpp in the cppcheck repo, which is more "normal" file. On that file there was no measurable time difference. A synthetic benchmark was generated to illustrate the effects on dumping the ast for arrays of different sizes. The generate code looked as follows: const int array[] = {...}; with different number of elements. The results are as follows (times are in seconds): N master optimized 10 0.1 0.1 100 0.1 0.1 1000 2.8 0.7 2000 19 1.8 3000 53 3.8 5000 350 10 10000 3215 38 As we can see, for small arrays, there is no time difference, but for large arrays the time savings are substantial.
2019-04-30 13:35:48 +02:00
str += "| ";
AST: improved AST printout when --verbose is used
2014-02-24 17:22:29 +01:00
}
Mark members and arguments with 'nonneg'
2019-07-14 12:22:33 +02:00
void Token::astStringVerboseRecursive(std::string& ret, const nonneg int indent1, const nonneg int indent2) const
AST: improved AST printout when --verbose is used
2014-02-24 17:22:29 +01:00
{
Verbose AST dump: Show if token is expanded from macro I also added the token variable id, but that decreased readability of the AST tree too much.
2015-01-17 23:11:22 +01:00
if (isExpandedMacro())
Print type information in AST dump
2016-01-01 13:54:07 +01:00
ret += '$';
Renamed Token::_str
2018-06-16 23:03:15 +02:00
ret += mStr;
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
if (mImpl->mValueType)
ret += " \'" + mImpl->mValueType->str() + '\'';
Token: Write function() in ast debug output
2020-01-12 15:04:25 +01:00
if (function()) {
std::ostringstream ostr;
Clang import; fixed ast for function call with multiple arguments
2020-10-09 07:54:16 +02:00
ostr << std::hex << function();
Token: Write function() in ast debug output
2020-01-12 15:04:25 +01:00
ret += " f:" + ostr.str();
}
Print type information in AST dump
2016-01-01 13:54:07 +01:00
ret += '\n';
Verbose AST dump: Show if token is expanded from macro I also added the token variable id, but that decreased readability of the AST tree too much.
2015-01-17 23:11:22 +01:00
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
if (mImpl->mAstOperand1) {
Avoid 'unsigned' for int variables
2019-07-13 16:06:24 +02:00
int i1 = indent1, i2 = indent2 + 2;
Optimize astStringVerbose() for large arrays (#1815) Change the astStringVerbose() recursion to extend a string instead of returning one. This has the benefit that for tokens where the recursion runs deep (typically large arrays), the time savings can be substantial (see comments on benchmarks further down). The reason is that previously, for each token, the astString of its operands was constructed, and then appended to this tokens astString. This led to a lot of unnecessary string copying (and with that allocations). Instead, by passing the string by reference, the number of temporary strings is greatly reduced. Another way of seeing it is that previously, the string was constructed from end to beginning, but now it is constructed from the beginning to end. There was no notable speedup by preallocating the entire string using string::reserve() (at least not on Linux). To benchmark, the changes and master were tested on Linux using the commands: make time cppcheck --debug --verbose $file >/dev/null i.e., the cppcheck binary was compiled with the settings in the Makefile. Printing the output to screen or file will of course take longer time. In Trac ticket #8355 which triggered this change, an example file from the Wine repository was attached. Running the above cppcheck on master took 24 minutes and with the changes in this commmit, took 22 seconds. Another test made was on lib/tokenlist.cpp in the cppcheck repo, which is more "normal" file. On that file there was no measurable time difference. A synthetic benchmark was generated to illustrate the effects on dumping the ast for arrays of different sizes. The generate code looked as follows: const int array[] = {...}; with different number of elements. The results are as follows (times are in seconds): N master optimized 10 0.1 0.1 100 0.1 0.1 1000 2.8 0.7 2000 19 1.8 3000 53 3.8 5000 350 10 10000 3215 38 As we can see, for small arrays, there is no time difference, but for large arrays the time savings are substantial.
2019-04-30 13:35:48 +02:00
if (indent1 == indent2 && !mImpl->mAstOperand2)
AST: improved AST printout when --verbose is used
2014-02-24 17:22:29 +01:00
i1 += 2;
Optimize astStringVerbose() for large arrays (#1815) Change the astStringVerbose() recursion to extend a string instead of returning one. This has the benefit that for tokens where the recursion runs deep (typically large arrays), the time savings can be substantial (see comments on benchmarks further down). The reason is that previously, for each token, the astString of its operands was constructed, and then appended to this tokens astString. This led to a lot of unnecessary string copying (and with that allocations). Instead, by passing the string by reference, the number of temporary strings is greatly reduced. Another way of seeing it is that previously, the string was constructed from end to beginning, but now it is constructed from the beginning to end. There was no notable speedup by preallocating the entire string using string::reserve() (at least not on Linux). To benchmark, the changes and master were tested on Linux using the commands: make time cppcheck --debug --verbose $file >/dev/null i.e., the cppcheck binary was compiled with the settings in the Makefile. Printing the output to screen or file will of course take longer time. In Trac ticket #8355 which triggered this change, an example file from the Wine repository was attached. Running the above cppcheck on master took 24 minutes and with the changes in this commmit, took 22 seconds. Another test made was on lib/tokenlist.cpp in the cppcheck repo, which is more "normal" file. On that file there was no measurable time difference. A synthetic benchmark was generated to illustrate the effects on dumping the ast for arrays of different sizes. The generate code looked as follows: const int array[] = {...}; with different number of elements. The results are as follows (times are in seconds): N master optimized 10 0.1 0.1 100 0.1 0.1 1000 2.8 0.7 2000 19 1.8 3000 53 3.8 5000 350 10 10000 3215 38 As we can see, for small arrays, there is no time difference, but for large arrays the time savings are substantial.
2019-04-30 13:35:48 +02:00
indent(ret, indent1, indent2);
ret += mImpl->mAstOperand2 ? "|-" : "`-";
mImpl->mAstOperand1->astStringVerboseRecursive(ret, i1, i2);
AST: improved AST printout when --verbose is used
2014-02-24 17:22:29 +01:00
}
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
if (mImpl->mAstOperand2) {
Avoid 'unsigned' for int variables
2019-07-13 16:06:24 +02:00
int i1 = indent1, i2 = indent2 + 2;
Optimize astStringVerbose() for large arrays (#1815) Change the astStringVerbose() recursion to extend a string instead of returning one. This has the benefit that for tokens where the recursion runs deep (typically large arrays), the time savings can be substantial (see comments on benchmarks further down). The reason is that previously, for each token, the astString of its operands was constructed, and then appended to this tokens astString. This led to a lot of unnecessary string copying (and with that allocations). Instead, by passing the string by reference, the number of temporary strings is greatly reduced. Another way of seeing it is that previously, the string was constructed from end to beginning, but now it is constructed from the beginning to end. There was no notable speedup by preallocating the entire string using string::reserve() (at least not on Linux). To benchmark, the changes and master were tested on Linux using the commands: make time cppcheck --debug --verbose $file >/dev/null i.e., the cppcheck binary was compiled with the settings in the Makefile. Printing the output to screen or file will of course take longer time. In Trac ticket #8355 which triggered this change, an example file from the Wine repository was attached. Running the above cppcheck on master took 24 minutes and with the changes in this commmit, took 22 seconds. Another test made was on lib/tokenlist.cpp in the cppcheck repo, which is more "normal" file. On that file there was no measurable time difference. A synthetic benchmark was generated to illustrate the effects on dumping the ast for arrays of different sizes. The generate code looked as follows: const int array[] = {...}; with different number of elements. The results are as follows (times are in seconds): N master optimized 10 0.1 0.1 100 0.1 0.1 1000 2.8 0.7 2000 19 1.8 3000 53 3.8 5000 350 10 10000 3215 38 As we can see, for small arrays, there is no time difference, but for large arrays the time savings are substantial.
2019-04-30 13:35:48 +02:00
if (indent1 == indent2)
AST: improved AST printout when --verbose is used
2014-02-24 17:22:29 +01:00
i1 += 2;
Optimize astStringVerbose() for large arrays (#1815) Change the astStringVerbose() recursion to extend a string instead of returning one. This has the benefit that for tokens where the recursion runs deep (typically large arrays), the time savings can be substantial (see comments on benchmarks further down). The reason is that previously, for each token, the astString of its operands was constructed, and then appended to this tokens astString. This led to a lot of unnecessary string copying (and with that allocations). Instead, by passing the string by reference, the number of temporary strings is greatly reduced. Another way of seeing it is that previously, the string was constructed from end to beginning, but now it is constructed from the beginning to end. There was no notable speedup by preallocating the entire string using string::reserve() (at least not on Linux). To benchmark, the changes and master were tested on Linux using the commands: make time cppcheck --debug --verbose $file >/dev/null i.e., the cppcheck binary was compiled with the settings in the Makefile. Printing the output to screen or file will of course take longer time. In Trac ticket #8355 which triggered this change, an example file from the Wine repository was attached. Running the above cppcheck on master took 24 minutes and with the changes in this commmit, took 22 seconds. Another test made was on lib/tokenlist.cpp in the cppcheck repo, which is more "normal" file. On that file there was no measurable time difference. A synthetic benchmark was generated to illustrate the effects on dumping the ast for arrays of different sizes. The generate code looked as follows: const int array[] = {...}; with different number of elements. The results are as follows (times are in seconds): N master optimized 10 0.1 0.1 100 0.1 0.1 1000 2.8 0.7 2000 19 1.8 3000 53 3.8 5000 350 10 10000 3215 38 As we can see, for small arrays, there is no time difference, but for large arrays the time savings are substantial.
2019-04-30 13:35:48 +02:00
indent(ret, indent1, indent2);
ret += "`-";
mImpl->mAstOperand2->astStringVerboseRecursive(ret, i1, i2);
AST: improved AST printout when --verbose is used
2014-02-24 17:22:29 +01:00
}
Optimize astStringVerbose() for large arrays (#1815) Change the astStringVerbose() recursion to extend a string instead of returning one. This has the benefit that for tokens where the recursion runs deep (typically large arrays), the time savings can be substantial (see comments on benchmarks further down). The reason is that previously, for each token, the astString of its operands was constructed, and then appended to this tokens astString. This led to a lot of unnecessary string copying (and with that allocations). Instead, by passing the string by reference, the number of temporary strings is greatly reduced. Another way of seeing it is that previously, the string was constructed from end to beginning, but now it is constructed from the beginning to end. There was no notable speedup by preallocating the entire string using string::reserve() (at least not on Linux). To benchmark, the changes and master were tested on Linux using the commands: make time cppcheck --debug --verbose $file >/dev/null i.e., the cppcheck binary was compiled with the settings in the Makefile. Printing the output to screen or file will of course take longer time. In Trac ticket #8355 which triggered this change, an example file from the Wine repository was attached. Running the above cppcheck on master took 24 minutes and with the changes in this commmit, took 22 seconds. Another test made was on lib/tokenlist.cpp in the cppcheck repo, which is more "normal" file. On that file there was no measurable time difference. A synthetic benchmark was generated to illustrate the effects on dumping the ast for arrays of different sizes. The generate code looked as follows: const int array[] = {...}; with different number of elements. The results are as follows (times are in seconds): N master optimized 10 0.1 0.1 100 0.1 0.1 1000 2.8 0.7 2000 19 1.8 3000 53 3.8 5000 350 10 10000 3215 38 As we can see, for small arrays, there is no time difference, but for large arrays the time savings are substantial.
2019-04-30 13:35:48 +02:00
}
std::string Token::astStringVerbose() const
{
std::string ret;
astStringVerboseRecursive(ret);
AST: improved AST printout when --verbose is used
2014-02-24 17:22:29 +01:00
return ret;
}
astyle formatting [ci skip]
2020-05-18 19:53:35 +02:00
std::string Token::astStringZ3() const
{
Fix #9700 duplicateBranch false positive from missing scope operator :: in ast (#2646)
2020-05-18 19:31:13 +02:00
if (!astOperand1())
return str();
if (!astOperand2())
return "(" + str() + " " + astOperand1()->astStringZ3() + ")";
return "(" + str() + " " + astOperand1()->astStringZ3() + " " + astOperand2()->astStringZ3() + ")";
}
AST: improved AST printout when --verbose is used
2014-02-24 17:22:29 +01:00
Fixed #5982 (Add xml dump)
2014-07-14 15:51:45 +02:00
void Token::printValueFlow(bool xml, std::ostream &out) const
added debug output for the value flow
2014-01-18 09:58:32 +01:00
{
Avoid 'unsigned' for int variables
2019-07-13 16:06:24 +02:00
int line = 0;
Fixed #5982 (Add xml dump)
2014-07-14 15:51:45 +02:00
if (xml)
out << " <valueflow>" << std::endl;
else
out << "\n\n##Value flow" << std::endl;
added debug output for the value flow
2014-01-18 09:58:32 +01:00
for (const Token *tok = this; tok; tok = tok->next()) {
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
if (!tok->mImpl->mValues)
added debug output for the value flow
2014-01-18 09:58:32 +01:00
continue;
Fixed crash when printing out ValueFlow debug output
2020-01-20 08:10:39 +01:00
if (tok->mImpl->mValues->empty()) // Values might be removed by removeContradictions
continue;
Fixed #5982 (Add xml dump)
2014-07-14 15:51:45 +02:00
if (xml)
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
out << " <values id=\"" << tok->mImpl->mValues << "\">" << std::endl;
Fixed #5982 (Add xml dump)
2014-07-14 15:51:45 +02:00
else if (line != tok->linenr())
out << "Line " << tok->linenr() << std::endl;
added debug output for the value flow
2014-01-18 09:58:32 +01:00
line = tok->linenr();
ValueFlow: Changed --debug output so we can see if a variable is known or possible
2015-07-26 15:36:09 +02:00
if (!xml) {
ValueFlow: Try to clarify debug output for impossible values
2020-05-08 16:13:55 +02:00
ValueFlow::Value::ValueKind valueKind = tok->mImpl->mValues->front().valueKind;
bool same = true;
for (const ValueFlow::Value &value : *tok->mImpl->mValues) {
if (value.valueKind != valueKind) {
same = false;
break;
}
}
out << " " << tok->str() << " ";
if (same) {
switch (valueKind) {
case ValueFlow::Value::ValueKind::Impossible:
case ValueFlow::Value::ValueKind::Known:
out << "always ";
break;
case ValueFlow::Value::ValueKind::Inconclusive:
Improvements to debug output (#3091)
2021-01-28 22:19:37 +01:00
out << "inconclusive ";
break;
ValueFlow: Try to clarify debug output for impossible values
2020-05-08 16:13:55 +02:00
case ValueFlow::Value::ValueKind::Possible:
out << "possible ";
break;
cleaned up includes based on include-what-you-use (#2632) * cleaned up includes based on include-what-you-use * token.cpp: fixed -Wextra-semi-stmt warning
2020-05-10 16:45:45 +02:00
}
ValueFlow: Try to clarify debug output for impossible values
2020-05-08 16:13:55 +02:00
}
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
if (tok->mImpl->mValues->size() > 1U)
ValueFlow: Changed --debug output so we can see if a variable is known or possible
2015-07-26 15:36:09 +02:00
out << '{';
}
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
for (const ValueFlow::Value &value : *tok->mImpl->mValues) {
Fixed #5982 (Add xml dump)
2014-07-14 15:51:45 +02:00
if (xml) {
ValueFlow: start adding valueflow handling of strings and pointer aliases
2014-08-03 20:11:22 +02:00
out << " <value ";
Fixed #8681 (ValueFlow: Container size)
2018-08-10 11:29:16 +02:00
switch (value.valueType) {
Refactoring: Convert ValueType to enum class (#3005)
2021-01-02 09:30:00 +01:00
case ValueFlow::Value::ValueType::INT:
Fixed #7701 (ValueFlow: handle 64-bit unsigned values )
2017-07-22 12:19:46 +02:00
if (tok->valueType() && tok->valueType()->sign == ValueType::UNSIGNED)
Fixed #8681 (ValueFlow: Container size)
2018-08-10 11:29:16 +02:00
out << "intvalue=\"" << (MathLib::biguint)value.intvalue << '\"';
Fixed #7701 (ValueFlow: handle 64-bit unsigned values )
2017-07-22 12:19:46 +02:00
else
Fixed #8681 (ValueFlow: Container size)
2018-08-10 11:29:16 +02:00
out << "intvalue=\"" << value.intvalue << '\"';
ValueFlow: Added float type
2016-11-13 22:59:56 +01:00
break;
Refactoring: Convert ValueType to enum class (#3005)
2021-01-02 09:30:00 +01:00
case ValueFlow::Value::ValueType::TOK:
Fixed #8681 (ValueFlow: Container size)
2018-08-10 11:29:16 +02:00
out << "tokvalue=\"" << value.tokvalue << '\"';
ValueFlow: Added float type
2016-11-13 22:59:56 +01:00
break;
Refactoring: Convert ValueType to enum class (#3005)
2021-01-02 09:30:00 +01:00
case ValueFlow::Value::ValueType::FLOAT:
Fixed #8681 (ValueFlow: Container size)
2018-08-10 11:29:16 +02:00
out << "floatvalue=\"" << value.floatValue << '\"';
ValueFlow: Added float type
2016-11-13 22:59:56 +01:00
break;
Refactoring: Convert ValueType to enum class (#3005)
2021-01-02 09:30:00 +01:00
case ValueFlow::Value::ValueType::MOVED:
Fixed #8681 (ValueFlow: Container size)
2018-08-10 11:29:16 +02:00
out << "movedvalue=\"" << ValueFlow::Value::toString(value.moveKind) << '\"';
Fixed #6180 (Usage of variable after std::move or std::forward)
2016-11-20 15:14:49 +01:00
break;
Refactoring: Convert ValueType to enum class (#3005)
2021-01-02 09:30:00 +01:00
case ValueFlow::Value::ValueType::UNINIT:
fixed #7998 (uninitialized variable is not found when used with switch/case)
2017-04-23 18:05:14 +02:00
out << "uninit=\"1\"";
break;
Refactoring: Convert ValueType to enum class (#3005)
2021-01-02 09:30:00 +01:00
case ValueFlow::Value::ValueType::BUFFER_SIZE:
Array index: Checking array index out of bounds for dynamic buffers
2019-03-17 13:09:15 +01:00
out << "buffer-size=\"" << value.intvalue << "\"";
break;
Refactoring: Convert ValueType to enum class (#3005)
2021-01-02 09:30:00 +01:00
case ValueFlow::Value::ValueType::CONTAINER_SIZE:
Fixed #8681 (ValueFlow: Container size)
2018-08-10 11:29:16 +02:00
out << "container-size=\"" << value.intvalue << '\"';
break;
Refactoring: Convert ValueType to enum class (#3005)
2021-01-02 09:30:00 +01:00
case ValueFlow::Value::ValueType::ITERATOR_START:
Fix issue 737: new check: Dereference end iterator
2020-08-17 23:36:45 +02:00
out << "iterator-start=\"" << value.intvalue << '\"';
break;
Refactoring: Convert ValueType to enum class (#3005)
2021-01-02 09:30:00 +01:00
case ValueFlow::Value::ValueType::ITERATOR_END:
Fix issue 737: new check: Dereference end iterator
2020-08-17 23:36:45 +02:00
out << "iterator-end=\"" << value.intvalue << '\"';
break;
Refactoring: Convert ValueType to enum class (#3005)
2021-01-02 09:30:00 +01:00
case ValueFlow::Value::ValueType::LIFETIME:
Fix warnings (#1478)
2018-11-14 06:14:04 +01:00
out << "lifetime=\"" << value.tokvalue << '\"';
Add more ValueFlow info in the xml dumps (#3402)
2021-08-15 08:02:31 +02:00
out << " lifetime-scope=\"" << ValueFlow::Value::toString(value.lifetimeScope) << "\"";
out << " lifetime-kind=\"" << ValueFlow::Value::toString(value.lifetimeKind) << "\"";
Fix warnings (#1478)
2018-11-14 06:14:04 +01:00
break;
Add symbolic values to ValueFlow (#3367)
2021-07-30 21:29:35 +02:00
case ValueFlow::Value::ValueType::SYMBOLIC:
Add more ValueFlow info in the xml dumps (#3402)
2021-08-15 08:02:31 +02:00
out << "symbolic=\"" << value.tokvalue << '\"';
out << " symbolic-delta=\"" << value.intvalue << '\"';
Add symbolic values to ValueFlow (#3367)
2021-07-30 21:29:35 +02:00
break;
ValueFlow: Added float type
2016-11-13 22:59:56 +01:00
}
Add more ValueFlow info in the xml dumps (#3402)
2021-08-15 08:02:31 +02:00
out << " bound=\"" << ValueFlow::Value::toString(value.bound) << "\"";
Fixed #8681 (ValueFlow: Container size)
2018-08-10 11:29:16 +02:00
if (value.condition)
out << " condition-line=\"" << value.condition->linenr() << '\"';
if (value.isKnown())
ValueFlow: Changed --debug output so we can see if a variable is known or possible
2015-07-26 15:36:09 +02:00
out << " known=\"true\"";
Fixed #8681 (ValueFlow: Container size)
2018-08-10 11:29:16 +02:00
else if (value.isPossible())
ValueFlow: Changed --debug output so we can see if a variable is known or possible
2015-07-26 15:36:09 +02:00
out << " possible=\"true\"";
Add impossible values to ValueFlow (#2186) * Add impossible category * Replace values * Try to adjust known values * Add ! for impossible values * Add impossible with possible values * Remove contradictions * Add values when the branch is not dead * Only copy possible values * Dont bail on while loops * Load std lib in valueflow * Check for function calls * Fix stl errors * Fix incorrect impossible check * Fix heap-after-use error * Remove impossible values when they are lowered * Show the bound and remove overlaps * Infer conditions * Dont push pointer values through dynamic_cast * Add test for dynamic_cast issue * Add shifttoomanybits test * Add test for div by zero * Add a test for issue 9315 * Dont make impossible value inconclusive * Fix FP with shift operator * Improve handleKnownValuesInLoop for impossible values * Fix cppcheck warning * Fix impossible values for ctu * Bailout for streams * Check equality conditions * Fix overflows * Add regression test for 9332 * Remove duplicate conditions * Skip impossible values for invalid value * Check for null * Rename bound to range * Formatting
2019-09-20 15:06:37 +02:00
else if (value.isImpossible())
out << " impossible=\"true\"";
Fixed #8681 (ValueFlow: Container size)
2018-08-10 11:29:16 +02:00
else if (value.isInconclusive())
ValueFlow: Put 'inconclusive' state in the ValueKind. A value can't be both known and inconclusive.
2017-09-20 22:41:36 +02:00
out << " inconclusive=\"true\"";
Add more ValueFlow info in the xml dumps (#3402)
2021-08-15 08:02:31 +02:00
out << " path=\"" << value.path << "\"";
Fixed #5982 (Add xml dump)
2014-07-14 15:51:45 +02:00
out << "/>" << std::endl;
}
else {
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
if (&value != &tok->mImpl->mValues->front())
ValueFlow: start adding valueflow handling of strings and pointer aliases
2014-08-03 20:11:22 +02:00
out << ",";
Add impossible values to ValueFlow (#2186) * Add impossible category * Replace values * Try to adjust known values * Add ! for impossible values * Add impossible with possible values * Remove contradictions * Add values when the branch is not dead * Only copy possible values * Dont bail on while loops * Load std lib in valueflow * Check for function calls * Fix stl errors * Fix incorrect impossible check * Fix heap-after-use error * Remove impossible values when they are lowered * Show the bound and remove overlaps * Infer conditions * Dont push pointer values through dynamic_cast * Add test for dynamic_cast issue * Add shifttoomanybits test * Add test for div by zero * Add a test for issue 9315 * Dont make impossible value inconclusive * Fix FP with shift operator * Improve handleKnownValuesInLoop for impossible values * Fix cppcheck warning * Fix impossible values for ctu * Bailout for streams * Check equality conditions * Fix overflows * Add regression test for 9332 * Remove duplicate conditions * Skip impossible values for invalid value * Check for null * Rename bound to range * Formatting
2019-09-20 15:06:37 +02:00
if (value.isImpossible())
out << "!";
if (value.bound == ValueFlow::Value::Bound::Lower)
ValueFlow: Improved debug output.
2021-01-01 18:08:03 +01:00
out << ">=";
Add impossible values to ValueFlow (#2186) * Add impossible category * Replace values * Try to adjust known values * Add ! for impossible values * Add impossible with possible values * Remove contradictions * Add values when the branch is not dead * Only copy possible values * Dont bail on while loops * Load std lib in valueflow * Check for function calls * Fix stl errors * Fix incorrect impossible check * Fix heap-after-use error * Remove impossible values when they are lowered * Show the bound and remove overlaps * Infer conditions * Dont push pointer values through dynamic_cast * Add test for dynamic_cast issue * Add shifttoomanybits test * Add test for div by zero * Add a test for issue 9315 * Dont make impossible value inconclusive * Fix FP with shift operator * Improve handleKnownValuesInLoop for impossible values * Fix cppcheck warning * Fix impossible values for ctu * Bailout for streams * Check equality conditions * Fix overflows * Add regression test for 9332 * Remove duplicate conditions * Skip impossible values for invalid value * Check for null * Rename bound to range * Formatting
2019-09-20 15:06:37 +02:00
if (value.bound == ValueFlow::Value::Bound::Upper)
ValueFlow: Improved debug output.
2021-01-01 18:08:03 +01:00
out << "<=";
Fixed #8681 (ValueFlow: Container size)
2018-08-10 11:29:16 +02:00
switch (value.valueType) {
Refactoring: Convert ValueType to enum class (#3005)
2021-01-02 09:30:00 +01:00
case ValueFlow::Value::ValueType::INT:
Fix issue 10225: false positive: knownConditionTrueFalse (#3196)
2021-04-05 10:20:14 +02:00
out << value.intvalue;
ValueFlow: Added float type
2016-11-13 22:59:56 +01:00
break;
Refactoring: Convert ValueType to enum class (#3005)
2021-01-02 09:30:00 +01:00
case ValueFlow::Value::ValueType::TOK:
Fixed #8681 (ValueFlow: Container size)
2018-08-10 11:29:16 +02:00
out << value.tokvalue->str();
ValueFlow: Added float type
2016-11-13 22:59:56 +01:00
break;
Refactoring: Convert ValueType to enum class (#3005)
2021-01-02 09:30:00 +01:00
case ValueFlow::Value::ValueType::FLOAT:
Fixed #8681 (ValueFlow: Container size)
2018-08-10 11:29:16 +02:00
out << value.floatValue;
ValueFlow: Added float type
2016-11-13 22:59:56 +01:00
break;
Refactoring: Convert ValueType to enum class (#3005)
2021-01-02 09:30:00 +01:00
case ValueFlow::Value::ValueType::MOVED:
Fixed #8681 (ValueFlow: Container size)
2018-08-10 11:29:16 +02:00
out << ValueFlow::Value::toString(value.moveKind);
Fixed #6180 (Usage of variable after std::move or std::forward)
2016-11-20 15:14:49 +01:00
break;
Refactoring: Convert ValueType to enum class (#3005)
2021-01-02 09:30:00 +01:00
case ValueFlow::Value::ValueType::UNINIT:
fixed #7998 (uninitialized variable is not found when used with switch/case)
2017-04-23 18:05:14 +02:00
out << "Uninit";
break;
Refactoring: Convert ValueType to enum class (#3005)
2021-01-02 09:30:00 +01:00
case ValueFlow::Value::ValueType::BUFFER_SIZE:
case ValueFlow::Value::ValueType::CONTAINER_SIZE:
Fixed #8681 (ValueFlow: Container size)
2018-08-10 11:29:16 +02:00
out << "size=" << value.intvalue;
break;
Refactoring: Convert ValueType to enum class (#3005)
2021-01-02 09:30:00 +01:00
case ValueFlow::Value::ValueType::ITERATOR_START:
Fix issue 737: new check: Dereference end iterator
2020-08-17 23:36:45 +02:00
out << "start=" << value.intvalue;
break;
Refactoring: Convert ValueType to enum class (#3005)
2021-01-02 09:30:00 +01:00
case ValueFlow::Value::ValueType::ITERATOR_END:
Fix issue 737: new check: Dereference end iterator
2020-08-17 23:36:45 +02:00
out << "end=" << value.intvalue;
break;
Refactoring: Convert ValueType to enum class (#3005)
2021-01-02 09:30:00 +01:00
case ValueFlow::Value::ValueType::LIFETIME:
Show lifetime kind in output (#3285)
2021-06-04 21:41:30 +02:00
out << "lifetime[" << ValueFlow::Value::toString(value.lifetimeKind) << "]=("
<< value.tokvalue->expressionString() << ")";
Fix warnings (#1478)
2018-11-14 06:14:04 +01:00
break;
Add symbolic values to ValueFlow (#3367)
2021-07-30 21:29:35 +02:00
case ValueFlow::Value::ValueType::SYMBOLIC:
out << "symbolic=(" << value.tokvalue->expressionString();
if (value.intvalue > 0)
out << "+" << value.intvalue;
else if (value.intvalue < 0)
out << "-" << -value.intvalue;
out << ")";
break;
ValueFlow: Added float type
2016-11-13 22:59:56 +01:00
}
Fix issue 9190: FP uninitvar for struct member (#2112) * Fix issue 9190: FP uninitvar for struct member * Add more test cases * Fix false negative
2019-08-24 11:27:47 +02:00
if (value.indirect > 0)
Fixed #5259 (Improve check: Uninitialized variable not reported when used in array initialization)
2019-08-24 14:43:35 +02:00
for (int i=0; i<value.indirect; i++)
Fix issue 9190: FP uninitvar for struct member (#2112) * Fix issue 9190: FP uninitvar for struct member * Add more test cases * Fix false negative
2019-08-24 11:27:47 +02:00
out << "*";
Fix issue 9360: False positive: arrayIndexOutOfBounds when function is called with different array sizes (#2541)
2020-02-17 10:31:08 +01:00
if (value.path > 0)
out << "@" << value.path;
Fixed #5982 (Add xml dump)
2014-07-14 15:51:45 +02:00
}
added debug output for the value flow
2014-01-18 09:58:32 +01:00
}
Fixed #5982 (Add xml dump)
2014-07-14 15:51:45 +02:00
if (xml)
out << " </values>" << std::endl;
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
else if (tok->mImpl->mValues->size() > 1U)
ValueFlow: Changed --debug output so we can see if a variable is known or possible
2015-07-26 15:36:09 +02:00
out << '}' << std::endl;
Fixed #5982 (Add xml dump)
2014-07-14 15:51:45 +02:00
else
ValueFlow: Changed --debug output so we can see if a variable is known or possible
2015-07-26 15:36:09 +02:00
out << std::endl;
added debug output for the value flow
2014-01-18 09:58:32 +01:00
}
Fixed #5982 (Add xml dump)
2014-07-14 15:51:45 +02:00
if (xml)
out << " </valueflow>" << std::endl;
added debug output for the value flow
2014-01-18 09:58:32 +01:00
}
ValueFlow: Added utility functions getValueLE and getValueGE to simplify usage
2014-04-02 06:49:28 +02:00
const ValueFlow::Value * Token::getValueLE(const MathLib::bigint val, const Settings *settings) const
{
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
if (!mImpl->mValues)
Refactorization: Allocate Token::_values (ValueFlow information) dynamically, reducing size of each token by around 10%
2017-03-27 18:48:34 +02:00
return nullptr;
Fix 10298: ValueFlow: Wrong known value, 'x == -1' implicit unsigned cast for rhs (#3277)
2021-06-04 17:17:41 +02:00
return ValueFlow::findValue(*mImpl->mValues, settings, [&](const ValueFlow::Value& v) {
return !v.isImpossible() && v.isIntValue() && v.intvalue <= val;
});
ValueFlow: Added utility functions getValueLE and getValueGE to simplify usage
2014-04-02 06:49:28 +02:00
}
const ValueFlow::Value * Token::getValueGE(const MathLib::bigint val, const Settings *settings) const
{
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
if (!mImpl->mValues)
Refactorization: Allocate Token::_values (ValueFlow information) dynamically, reducing size of each token by around 10%
2017-03-27 18:48:34 +02:00
return nullptr;
Fix 10298: ValueFlow: Wrong known value, 'x == -1' implicit unsigned cast for rhs (#3277)
2021-06-04 17:17:41 +02:00
return ValueFlow::findValue(*mImpl->mValues, settings, [&](const ValueFlow::Value& v) {
return !v.isImpossible() && v.isIntValue() && v.intvalue >= val;
});
CheckFunction: In the <valid> check handle possible/conditional/inconclusive values better
2017-04-20 22:14:54 +02:00
}
Mark members and arguments with 'nonneg'
2019-07-14 12:22:33 +02:00
const ValueFlow::Value * Token::getInvalidValue(const Token *ftok, nonneg int argnr, const Settings *settings) const
CheckFunction: In the <valid> check handle possible/conditional/inconclusive values better
2017-04-20 22:14:54 +02:00
{
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
if (!mImpl->mValues || !settings)
CheckFunction: In the <valid> check handle possible/conditional/inconclusive values better
2017-04-20 22:14:54 +02:00
return nullptr;
const ValueFlow::Value *ret = nullptr;
std::list<ValueFlow::Value>::const_iterator it;
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
for (it = mImpl->mValues->begin(); it != mImpl->mValues->end(); ++it) {
Add impossible values to ValueFlow (#2186) * Add impossible category * Replace values * Try to adjust known values * Add ! for impossible values * Add impossible with possible values * Remove contradictions * Add values when the branch is not dead * Only copy possible values * Dont bail on while loops * Load std lib in valueflow * Check for function calls * Fix stl errors * Fix incorrect impossible check * Fix heap-after-use error * Remove impossible values when they are lowered * Show the bound and remove overlaps * Infer conditions * Dont push pointer values through dynamic_cast * Add test for dynamic_cast issue * Add shifttoomanybits test * Add test for div by zero * Add a test for issue 9315 * Dont make impossible value inconclusive * Fix FP with shift operator * Improve handleKnownValuesInLoop for impossible values * Fix cppcheck warning * Fix impossible values for ctu * Bailout for streams * Check equality conditions * Fix overflows * Add regression test for 9332 * Remove duplicate conditions * Skip impossible values for invalid value * Check for null * Rename bound to range * Formatting
2019-09-20 15:06:37 +02:00
if (it->isImpossible())
continue;
Rename Library::isargvalid()
2018-07-15 23:05:48 +02:00
if ((it->isIntValue() && !settings->library.isIntArgValid(ftok, argnr, it->intvalue)) ||
(it->isFloatValue() && !settings->library.isFloatArgValid(ftok, argnr, it->floatValue))) {
ValueFlow: Put 'inconclusive' state in the ValueKind. A value can't be both known and inconclusive.
2017-09-20 22:41:36 +02:00
if (!ret || ret->isInconclusive() || (ret->condition && !it->isInconclusive()))
CheckFunction: In the <valid> check handle possible/conditional/inconclusive values better
2017-04-20 22:14:54 +02:00
ret = &(*it);
ValueFlow: Put 'inconclusive' state in the ValueKind. A value can't be both known and inconclusive.
2017-09-20 22:41:36 +02:00
if (!ret->isInconclusive() && !ret->condition)
CheckFunction: In the <valid> check handle possible/conditional/inconclusive values better
2017-04-20 22:14:54 +02:00
break;
}
}
token: Fixed potential null pointer dereference. (#1062)
2018-01-28 14:27:01 +01:00
if (ret) {
Refactorization: Improved internal implementation of severity and certainty levels Backported from LCppC.
2021-02-24 22:00:06 +01:00
if (ret->isInconclusive() && !settings->certainty.isEnabled(Certainty::inconclusive))
ValueFlow: Added utility functions getValueLE and getValueGE to simplify usage
2014-04-02 06:49:28 +02:00
return nullptr;
Refactorization: Improved internal implementation of severity and certainty levels Backported from LCppC.
2021-02-24 22:00:06 +01:00
if (ret->condition && !settings->severity.isEnabled(Severity::warning))
ValueFlow: Added utility functions getValueLE and getValueGE to simplify usage
2014-04-02 06:49:28 +02:00
return nullptr;
}
return ret;
}
Set correct type and size of string and char literals (#2275) * Set correct type and size of string and char literals Use that string and char literal tokens store the prefix. This makes it possible to distinghuish between different type of string literals (i.e., utf8 encoded strings, utf16, wide strings, etc) which have different type. When the tokens holding the string and character values have the correct type, it is possible to improve Token::getStrSize() to give the correct result for all string types. Previously, it would return the number of characters in the string, i.e., it would give the wrong size unless the type of the string was char*. Since strings now can have different size (in number of bytes) and length (in number of elements), add a new helper function that returns the number of characters. Checkers have been updated to use the correct functions. Having the size makes it possible to find more problems with prefixed strings, and to reduce false positives, for example in the buffer overflow checker. Also, improve the stringLiteralWrite error message to also print the prefix of the string (if there is one). * Add comment and update string length
2019-10-20 07:11:57 +02:00
const Token *Token::getValueTokenMinStrSize(const Settings *settings) const
BufferOverrun: Use ValueFlow string values more
2014-08-04 08:25:10 +02:00
{
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
if (!mImpl->mValues)
Refactorization: Allocate Token::_values (ValueFlow information) dynamically, reducing size of each token by around 10%
2017-03-27 18:48:34 +02:00
return nullptr;
BufferOverrun: Use ValueFlow string values more
2014-08-04 08:25:10 +02:00
const Token *ret = nullptr;
Avoid 'unsigned' for int variables
2019-07-13 16:06:24 +02:00
int minsize = INT_MAX;
BufferOverrun: Use ValueFlow string values more
2014-08-04 08:25:10 +02:00
std::list<ValueFlow::Value>::const_iterator it;
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
for (it = mImpl->mValues->begin(); it != mImpl->mValues->end(); ++it) {
ValueFlow: allow more value types
2016-11-13 22:33:39 +01:00
if (it->isTokValue() && it->tokvalue && it->tokvalue->tokType() == Token::eString) {
Set correct type and size of string and char literals (#2275) * Set correct type and size of string and char literals Use that string and char literal tokens store the prefix. This makes it possible to distinghuish between different type of string literals (i.e., utf8 encoded strings, utf16, wide strings, etc) which have different type. When the tokens holding the string and character values have the correct type, it is possible to improve Token::getStrSize() to give the correct result for all string types. Previously, it would return the number of characters in the string, i.e., it would give the wrong size unless the type of the string was char*. Since strings now can have different size (in number of bytes) and length (in number of elements), add a new helper function that returns the number of characters. Checkers have been updated to use the correct functions. Having the size makes it possible to find more problems with prefixed strings, and to reduce false positives, for example in the buffer overflow checker. Also, improve the stringLiteralWrite error message to also print the prefix of the string (if there is one). * Add comment and update string length
2019-10-20 07:11:57 +02:00
const int size = getStrSize(it->tokvalue, settings);
BufferOverrun: Use ValueFlow string values more
2014-08-04 08:25:10 +02:00
if (!ret || size < minsize) {
minsize = size;
ret = it->tokvalue;
}
}
}
return ret;
}
const Token *Token::getValueTokenMaxStrLength() const
{
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
if (!mImpl->mValues)
Refactorization: Allocate Token::_values (ValueFlow information) dynamically, reducing size of each token by around 10%
2017-03-27 18:48:34 +02:00
return nullptr;
BufferOverrun: Use ValueFlow string values more
2014-08-04 08:25:10 +02:00
const Token *ret = nullptr;
Avoid 'unsigned' for int variables
2019-07-13 16:06:24 +02:00
int maxlength = 0;
BufferOverrun: Use ValueFlow string values more
2014-08-04 08:25:10 +02:00
std::list<ValueFlow::Value>::const_iterator it;
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
for (it = mImpl->mValues->begin(); it != mImpl->mValues->end(); ++it) {
ValueFlow: allow more value types
2016-11-13 22:33:39 +01:00
if (it->isTokValue() && it->tokvalue && it->tokvalue->tokType() == Token::eString) {
Avoid 'unsigned' for int variables
2019-07-13 16:06:24 +02:00
const int length = getStrLength(it->tokvalue);
BufferOverrun: Use ValueFlow string values more
2014-08-04 08:25:10 +02:00
if (!ret || length > maxlength) {
maxlength = length;
ret = it->tokvalue;
}
}
}
return ret;
}
Dead pointer: Fixed FP for subfunction pointer argument
2014-08-14 06:47:19 +02:00
static const Scope *getfunctionscope(const Scope *s)
{
while (s && s->type != Scope::eFunction)
s = s->nestedIn;
return s;
}
Dead pointer: Added checking for dead pointer usage when pointer alias local variable that has gone out of scope.
2014-08-05 06:24:23 +02:00
const Token *Token::getValueTokenDeadPointer() const
{
Dead pointer: Fixed FP for subfunction pointer argument
2014-08-14 06:47:19 +02:00
const Scope * const functionscope = getfunctionscope(this->scope());
Dead pointer: Added checking for dead pointer usage when pointer alias local variable that has gone out of scope.
2014-08-05 06:24:23 +02:00
std::list<ValueFlow::Value>::const_iterator it;
Refactorization: Allocate Token::_values (ValueFlow information) dynamically, reducing size of each token by around 10%
2017-03-27 18:48:34 +02:00
for (it = values().begin(); it != values().end(); ++it) {
Dead pointer: Added checking for dead pointer usage when pointer alias local variable that has gone out of scope.
2014-08-05 06:24:23 +02:00
// Is this a pointer alias?
ValueFlow: allow more value types
2016-11-13 22:33:39 +01:00
if (!it->isTokValue() || (it->tokvalue && it->tokvalue->str() != "&"))
Dead pointer: Added checking for dead pointer usage when pointer alias local variable that has gone out of scope.
2014-08-05 06:24:23 +02:00
continue;
// Get variable
const Token *vartok = it->tokvalue->astOperand1();
if (!vartok || !vartok->isName() || !vartok->variable())
continue;
const Variable * const var = vartok->variable();
Bailout in Token::getValueTokenDeadPointer() if reference is used (#6399)
2015-01-30 19:29:37 +01:00
if (var->isStatic() || var->isReference())
Dead pointer: Added checking for dead pointer usage when pointer alias local variable that has gone out of scope.
2014-08-05 06:24:23 +02:00
continue;
Remove some code checking for invalid class hierarchy which got obsolete since 480a5672b046ef0017af8bf59b6284895c22123d. Run astyle
2015-07-01 07:50:13 +02:00
if (!var->scope())
return nullptr; // #6804
Fixed false positive #6575
2015-03-11 20:25:27 +01:00
if (var->scope()->type == Scope::eUnion && var->scope()->nestedIn == this->scope())
continue;
Dead pointer: Fixed FP for subfunction pointer argument
2014-08-14 06:47:19 +02:00
// variable must be in same function (not in subfunction)
if (functionscope != getfunctionscope(var->scope()))
continue;
// Is variable defined in this scope or upper scope?
Dead pointer: Added checking for dead pointer usage when pointer alias local variable that has gone out of scope.
2014-08-05 06:24:23 +02:00
const Scope *s = this->scope();
while ((s != nullptr) && (s != var->scope()))
s = s->nestedIn;
if (!s)
return it->tokvalue;
}
return nullptr;
}
Fix issue 9979: false positive: containerOutOfBounds with conditional resize (#3136)
2021-03-30 14:02:28 +02:00
static bool isAdjacent(const ValueFlow::Value& x, const ValueFlow::Value& y)
{
if (x.bound != ValueFlow::Value::Bound::Point && x.bound == y.bound)
return true;
if (x.valueType == ValueFlow::Value::ValueType::FLOAT)
return false;
return std::abs(x.intvalue - y.intvalue) == 1;
}
static bool removePointValue(std::list<ValueFlow::Value>& values, ValueFlow::Value& x)
{
const bool isPoint = x.bound == ValueFlow::Value::Bound::Point;
if (!isPoint)
x.decreaseRange();
else
values.remove(x);
return isPoint;
}
Add impossible values to ValueFlow (#2186) * Add impossible category * Replace values * Try to adjust known values * Add ! for impossible values * Add impossible with possible values * Remove contradictions * Add values when the branch is not dead * Only copy possible values * Dont bail on while loops * Load std lib in valueflow * Check for function calls * Fix stl errors * Fix incorrect impossible check * Fix heap-after-use error * Remove impossible values when they are lowered * Show the bound and remove overlaps * Infer conditions * Dont push pointer values through dynamic_cast * Add test for dynamic_cast issue * Add shifttoomanybits test * Add test for div by zero * Add a test for issue 9315 * Dont make impossible value inconclusive * Fix FP with shift operator * Improve handleKnownValuesInLoop for impossible values * Fix cppcheck warning * Fix impossible values for ctu * Bailout for streams * Check equality conditions * Fix overflows * Add regression test for 9332 * Remove duplicate conditions * Skip impossible values for invalid value * Check for null * Rename bound to range * Formatting
2019-09-20 15:06:37 +02:00
static bool removeContradiction(std::list<ValueFlow::Value>& values)
{
bool result = false;
for (ValueFlow::Value& x : values) {
if (x.isNonValue())
continue;
for (ValueFlow::Value& y : values) {
if (y.isNonValue())
continue;
if (x == y)
continue;
if (x.valueType != y.valueType)
continue;
if (x.isImpossible() == y.isImpossible())
continue;
Deduce symbolic values from conditions (#3406)
2021-08-19 22:01:55 +02:00
if (x.isSymbolicValue() && !ValueFlow::Value::sameToken(x.tokvalue, y.tokvalue))
continue;
Fix issue 9979: false positive: containerOutOfBounds with conditional resize (#3136)
2021-03-30 14:02:28 +02:00
if (!x.equalValue(y)) {
auto compare = [](const ValueFlow::Value& x, const ValueFlow::Value& y) {
return x.compareValue(y, ValueFlow::less{});
};
const ValueFlow::Value& maxValue = std::max(x, y, compare);
const ValueFlow::Value& minValue = std::min(x, y, compare);
// TODO: Adjust non-points instead of removing them
if (maxValue.isImpossible() && maxValue.bound == ValueFlow::Value::Bound::Upper) {
values.remove(minValue);
return true;
}
if (minValue.isImpossible() && minValue.bound == ValueFlow::Value::Bound::Lower) {
values.remove(maxValue);
return true;
}
Add impossible values to ValueFlow (#2186) * Add impossible category * Replace values * Try to adjust known values * Add ! for impossible values * Add impossible with possible values * Remove contradictions * Add values when the branch is not dead * Only copy possible values * Dont bail on while loops * Load std lib in valueflow * Check for function calls * Fix stl errors * Fix incorrect impossible check * Fix heap-after-use error * Remove impossible values when they are lowered * Show the bound and remove overlaps * Infer conditions * Dont push pointer values through dynamic_cast * Add test for dynamic_cast issue * Add shifttoomanybits test * Add test for div by zero * Add a test for issue 9315 * Dont make impossible value inconclusive * Fix FP with shift operator * Improve handleKnownValuesInLoop for impossible values * Fix cppcheck warning * Fix impossible values for ctu * Bailout for streams * Check equality conditions * Fix overflows * Add regression test for 9332 * Remove duplicate conditions * Skip impossible values for invalid value * Check for null * Rename bound to range * Formatting
2019-09-20 15:06:37 +02:00
continue;
Fix issue 9979: false positive: containerOutOfBounds with conditional resize (#3136)
2021-03-30 14:02:28 +02:00
}
const bool removex = !x.isImpossible() || y.isKnown();
const bool removey = !y.isImpossible() || x.isKnown();
if (x.bound == y.bound) {
Add impossible values to ValueFlow (#2186) * Add impossible category * Replace values * Try to adjust known values * Add ! for impossible values * Add impossible with possible values * Remove contradictions * Add values when the branch is not dead * Only copy possible values * Dont bail on while loops * Load std lib in valueflow * Check for function calls * Fix stl errors * Fix incorrect impossible check * Fix heap-after-use error * Remove impossible values when they are lowered * Show the bound and remove overlaps * Infer conditions * Dont push pointer values through dynamic_cast * Add test for dynamic_cast issue * Add shifttoomanybits test * Add test for div by zero * Add a test for issue 9315 * Dont make impossible value inconclusive * Fix FP with shift operator * Improve handleKnownValuesInLoop for impossible values * Fix cppcheck warning * Fix impossible values for ctu * Bailout for streams * Check equality conditions * Fix overflows * Add regression test for 9332 * Remove duplicate conditions * Skip impossible values for invalid value * Check for null * Rename bound to range * Formatting
2019-09-20 15:06:37 +02:00
if (removex)
values.remove(x);
if (removey)
values.remove(y);
return true;
Fix issue 9979: false positive: containerOutOfBounds with conditional resize (#3136)
2021-03-30 14:02:28 +02:00
} else {
result = removex || removey;
bool bail = false;
if (removex && removePointValue(values, x))
bail = true;
if (removey && removePointValue(values, y))
bail = true;
if (bail)
return true;
Add impossible values to ValueFlow (#2186) * Add impossible category * Replace values * Try to adjust known values * Add ! for impossible values * Add impossible with possible values * Remove contradictions * Add values when the branch is not dead * Only copy possible values * Dont bail on while loops * Load std lib in valueflow * Check for function calls * Fix stl errors * Fix incorrect impossible check * Fix heap-after-use error * Remove impossible values when they are lowered * Show the bound and remove overlaps * Infer conditions * Dont push pointer values through dynamic_cast * Add test for dynamic_cast issue * Add shifttoomanybits test * Add test for div by zero * Add a test for issue 9315 * Dont make impossible value inconclusive * Fix FP with shift operator * Improve handleKnownValuesInLoop for impossible values * Fix cppcheck warning * Fix impossible values for ctu * Bailout for streams * Check equality conditions * Fix overflows * Add regression test for 9332 * Remove duplicate conditions * Skip impossible values for invalid value * Check for null * Rename bound to range * Formatting
2019-09-20 15:06:37 +02:00
}
}
}
return result;
}
Fix issue 9979: false positive: containerOutOfBounds with conditional resize (#3136)
2021-03-30 14:02:28 +02:00
using ValueIterator = std::list<ValueFlow::Value>::iterator;
Format with uncrustify (#3388)
2021-08-07 20:51:18 +02:00
template<class Iterator>
Fix issue 9979: false positive: containerOutOfBounds with conditional resize (#3136)
2021-03-30 14:02:28 +02:00
static ValueIterator removeAdjacentValues(std::list<ValueFlow::Value>& values, ValueIterator x, Iterator start, Iterator last)
{
if (!isAdjacent(*x, **start))
return std::next(x);
astyle formatting ci skip
2021-04-04 18:20:32 +02:00
auto it = std::adjacent_find(start, last, [](ValueIterator x, ValueIterator y) {
return !isAdjacent(*x, *y);
});
Fix issue 9979: false positive: containerOutOfBounds with conditional resize (#3136)
2021-03-30 14:02:28 +02:00
if (it == last)
it--;
(*it)->bound = x->bound;
astyle formatting ci skip
2021-04-04 18:20:32 +02:00
std::for_each(start, it, [&](ValueIterator y) {
values.erase(y);
});
Fix issue 9979: false positive: containerOutOfBounds with conditional resize (#3136)
2021-03-30 14:02:28 +02:00
return values.erase(x);
}
static void mergeAdjacent(std::list<ValueFlow::Value>& values)
{
for (auto x = values.begin(); x != values.end();) {
if (x->isNonValue()) {
x++;
continue;
}
if (x->bound == ValueFlow::Value::Bound::Point) {
x++;
continue;
}
std::vector<ValueIterator> adjValues;
for (auto y = values.begin(); y != values.end(); y++) {
if (x == y)
continue;
if (y->isNonValue())
continue;
if (x->valueType != y->valueType)
continue;
if (x->valueKind != y->valueKind)
continue;
Deduce symbolic values from conditions (#3406)
2021-08-19 22:01:55 +02:00
if (x->isSymbolicValue() && !ValueFlow::Value::sameToken(x->tokvalue, y->tokvalue))
continue;
Fix issue 9979: false positive: containerOutOfBounds with conditional resize (#3136)
2021-03-30 14:02:28 +02:00
if (x->bound != y->bound) {
Fixed #10275 (ValueFlow: condition 'x>=0 && x < 10')
2021-05-16 10:34:22 +02:00
if (y->bound != ValueFlow::Value::Bound::Point && isAdjacent(*x, *y)) {
adjValues.clear();
break;
}
Fix issue 9979: false positive: containerOutOfBounds with conditional resize (#3136)
2021-03-30 14:02:28 +02:00
// No adjacent points for floating points
if (x->valueType == ValueFlow::Value::ValueType::FLOAT)
continue;
if (y->bound != ValueFlow::Value::Bound::Point)
continue;
}
if (x->bound == ValueFlow::Value::Bound::Lower && !y->compareValue(*x, ValueFlow::less{}))
continue;
if (x->bound == ValueFlow::Value::Bound::Upper && !x->compareValue(*y, ValueFlow::less{}))
continue;
adjValues.push_back(y);
}
if (adjValues.empty()) {
x++;
continue;
}
std::sort(adjValues.begin(), adjValues.end(), [&values](ValueIterator xx, ValueIterator yy) {
assert(xx != values.end() && yy != values.end());
return xx->compareValue(*yy, ValueFlow::less{});
});
if (x->bound == ValueFlow::Value::Bound::Lower)
x = removeAdjacentValues(values, x, adjValues.rbegin(), adjValues.rend());
else if (x->bound == ValueFlow::Value::Bound::Upper)
x = removeAdjacentValues(values, x, adjValues.begin(), adjValues.end());
}
}
Add impossible values to ValueFlow (#2186) * Add impossible category * Replace values * Try to adjust known values * Add ! for impossible values * Add impossible with possible values * Remove contradictions * Add values when the branch is not dead * Only copy possible values * Dont bail on while loops * Load std lib in valueflow * Check for function calls * Fix stl errors * Fix incorrect impossible check * Fix heap-after-use error * Remove impossible values when they are lowered * Show the bound and remove overlaps * Infer conditions * Dont push pointer values through dynamic_cast * Add test for dynamic_cast issue * Add shifttoomanybits test * Add test for div by zero * Add a test for issue 9315 * Dont make impossible value inconclusive * Fix FP with shift operator * Improve handleKnownValuesInLoop for impossible values * Fix cppcheck warning * Fix impossible values for ctu * Bailout for streams * Check equality conditions * Fix overflows * Add regression test for 9332 * Remove duplicate conditions * Skip impossible values for invalid value * Check for null * Rename bound to range * Formatting
2019-09-20 15:06:37 +02:00
static void removeOverlaps(std::list<ValueFlow::Value>& values)
{
for (ValueFlow::Value& x : values) {
if (x.isNonValue())
continue;
values.remove_if([&](ValueFlow::Value& y) {
if (y.isNonValue())
return false;
if (&x == &y)
return false;
if (x.valueType != y.valueType)
return false;
if (x.valueKind != y.valueKind)
return false;
// TODO: Remove points coverd in a lower or upper bound
// TODO: Remove lower or upper bound already covered by a lower and upper bound
if (!x.equalValue(y))
return false;
if (x.bound != y.bound)
return false;
return true;
});
}
Fix issue 9979: false positive: containerOutOfBounds with conditional resize (#3136)
2021-03-30 14:02:28 +02:00
mergeAdjacent(values);
Add impossible values to ValueFlow (#2186) * Add impossible category * Replace values * Try to adjust known values * Add ! for impossible values * Add impossible with possible values * Remove contradictions * Add values when the branch is not dead * Only copy possible values * Dont bail on while loops * Load std lib in valueflow * Check for function calls * Fix stl errors * Fix incorrect impossible check * Fix heap-after-use error * Remove impossible values when they are lowered * Show the bound and remove overlaps * Infer conditions * Dont push pointer values through dynamic_cast * Add test for dynamic_cast issue * Add shifttoomanybits test * Add test for div by zero * Add a test for issue 9315 * Dont make impossible value inconclusive * Fix FP with shift operator * Improve handleKnownValuesInLoop for impossible values * Fix cppcheck warning * Fix impossible values for ctu * Bailout for streams * Check equality conditions * Fix overflows * Add regression test for 9332 * Remove duplicate conditions * Skip impossible values for invalid value * Check for null * Rename bound to range * Formatting
2019-09-20 15:06:37 +02:00
}
// Removing contradictions is an NP-hard problem. Instead we run multiple
// passes to try to catch most contradictions
static void removeContradictions(std::list<ValueFlow::Value>& values)
{
Fix issue 9979: false positive: containerOutOfBounds with conditional resize (#3136)
2021-03-30 14:02:28 +02:00
removeOverlaps(values);
Add impossible values to ValueFlow (#2186) * Add impossible category * Replace values * Try to adjust known values * Add ! for impossible values * Add impossible with possible values * Remove contradictions * Add values when the branch is not dead * Only copy possible values * Dont bail on while loops * Load std lib in valueflow * Check for function calls * Fix stl errors * Fix incorrect impossible check * Fix heap-after-use error * Remove impossible values when they are lowered * Show the bound and remove overlaps * Infer conditions * Dont push pointer values through dynamic_cast * Add test for dynamic_cast issue * Add shifttoomanybits test * Add test for div by zero * Add a test for issue 9315 * Dont make impossible value inconclusive * Fix FP with shift operator * Improve handleKnownValuesInLoop for impossible values * Fix cppcheck warning * Fix impossible values for ctu * Bailout for streams * Check equality conditions * Fix overflows * Add regression test for 9332 * Remove duplicate conditions * Skip impossible values for invalid value * Check for null * Rename bound to range * Formatting
2019-09-20 15:06:37 +02:00
for (int i = 0; i < 4; i++) {
if (!removeContradiction(values))
return;
removeOverlaps(values);
}
}
fix gcc compiler warnings
2021-08-22 18:06:54 +02:00
static bool sameValueType(const ValueFlow::Value& x, const ValueFlow::Value& y)
Deduce symbolic values from conditions (#3406)
2021-08-19 22:01:55 +02:00
{
if (x.valueType != y.valueType)
return false;
// Symbolic are the same type if they share the same tokvalue
if (x.isSymbolicValue())
return x.tokvalue->exprId() == 0 || x.tokvalue->exprId() == y.tokvalue->exprId();
return true;
}
Refactorization: Allocate Token::_values (ValueFlow information) dynamically, reducing size of each token by around 10%
2017-03-27 18:48:34 +02:00
bool Token::addValue(const ValueFlow::Value &value)
{
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
if (value.isKnown() && mImpl->mValues) {
Fix FP when copying pointer to string (#1479)
2018-11-14 06:59:25 +01:00
// Clear all other values of the same type since value is known
Add symbolic values to ValueFlow (#3367)
2021-07-30 21:29:35 +02:00
mImpl->mValues->remove_if([&](const ValueFlow::Value& x) {
Deduce symbolic values from conditions (#3406)
2021-08-19 22:01:55 +02:00
return sameValueType(x, value);
Fix FP when copying pointer to string (#1479)
2018-11-14 06:59:25 +01:00
});
Refactorization: Allocate Token::_values (ValueFlow information) dynamically, reducing size of each token by around 10%
2017-03-27 18:48:34 +02:00
}
Deduce symbolic values from conditions (#3406)
2021-08-19 22:01:55 +02:00
// assert(!value.isPossible() || !mImpl->mValues || std::none_of(mImpl->mValues->begin(), mImpl->mValues->end(),
// [&](const ValueFlow::Value& x) {
// return x.isKnown() && sameValueType(x, value);
// }));
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
if (mImpl->mValues) {
Refactorization: Allocate Token::_values (ValueFlow information) dynamically, reducing size of each token by around 10%
2017-03-27 18:48:34 +02:00
// Don't handle more than 10 values for performance reasons
// TODO: add setting?
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
if (mImpl->mValues->size() >= 10U)
Refactorization: Allocate Token::_values (ValueFlow information) dynamically, reducing size of each token by around 10%
2017-03-27 18:48:34 +02:00
return false;
// if value already exists, don't add it again
std::list<ValueFlow::Value>::iterator it;
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
for (it = mImpl->mValues->begin(); it != mImpl->mValues->end(); ++it) {
Refactorization: Allocate Token::_values (ValueFlow information) dynamically, reducing size of each token by around 10%
2017-03-27 18:48:34 +02:00
// different types => continue
if (it->valueType != value.valueType)
continue;
ValueFlow: fix false detection of duplicate values
2019-07-24 12:30:33 +02:00
Add impossible values to ValueFlow (#2186) * Add impossible category * Replace values * Try to adjust known values * Add ! for impossible values * Add impossible with possible values * Remove contradictions * Add values when the branch is not dead * Only copy possible values * Dont bail on while loops * Load std lib in valueflow * Check for function calls * Fix stl errors * Fix incorrect impossible check * Fix heap-after-use error * Remove impossible values when they are lowered * Show the bound and remove overlaps * Infer conditions * Dont push pointer values through dynamic_cast * Add test for dynamic_cast issue * Add shifttoomanybits test * Add test for div by zero * Add a test for issue 9315 * Dont make impossible value inconclusive * Fix FP with shift operator * Improve handleKnownValuesInLoop for impossible values * Fix cppcheck warning * Fix impossible values for ctu * Bailout for streams * Check equality conditions * Fix overflows * Add regression test for 9332 * Remove duplicate conditions * Skip impossible values for invalid value * Check for null * Rename bound to range * Formatting
2019-09-20 15:06:37 +02:00
if (it->isImpossible() != value.isImpossible())
continue;
ValueFlow: fix false detection of duplicate values
2019-07-24 12:30:33 +02:00
// different value => continue
Add symbolic values to ValueFlow (#3367)
2021-07-30 21:29:35 +02:00
if (!it->equalValue(value))
ValueFlow: fix false detection of duplicate values
2019-07-24 12:30:33 +02:00
continue;
Fix FP when copying pointer to string (#1479)
2018-11-14 06:59:25 +01:00
if ((value.isTokValue() || value.isLifetimeValue()) && (it->tokvalue != value.tokvalue) && (it->tokvalue->str() != value.tokvalue->str()))
Refactorization: Allocate Token::_values (ValueFlow information) dynamically, reducing size of each token by around 10%
2017-03-27 18:48:34 +02:00
continue;
// same value, but old value is inconclusive so replace it
Add impossible values to ValueFlow (#2186) * Add impossible category * Replace values * Try to adjust known values * Add ! for impossible values * Add impossible with possible values * Remove contradictions * Add values when the branch is not dead * Only copy possible values * Dont bail on while loops * Load std lib in valueflow * Check for function calls * Fix stl errors * Fix incorrect impossible check * Fix heap-after-use error * Remove impossible values when they are lowered * Show the bound and remove overlaps * Infer conditions * Dont push pointer values through dynamic_cast * Add test for dynamic_cast issue * Add shifttoomanybits test * Add test for div by zero * Add a test for issue 9315 * Dont make impossible value inconclusive * Fix FP with shift operator * Improve handleKnownValuesInLoop for impossible values * Fix cppcheck warning * Fix impossible values for ctu * Bailout for streams * Check equality conditions * Fix overflows * Add regression test for 9332 * Remove duplicate conditions * Skip impossible values for invalid value * Check for null * Rename bound to range * Formatting
2019-09-20 15:06:37 +02:00
if (it->isInconclusive() && !value.isInconclusive() && !value.isImpossible()) {
Refactorization: Allocate Token::_values (ValueFlow information) dynamically, reducing size of each token by around 10%
2017-03-27 18:48:34 +02:00
*it = value;
if (it->varId == 0)
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
it->varId = mImpl->mVarId;
Refactorization: Allocate Token::_values (ValueFlow information) dynamically, reducing size of each token by around 10%
2017-03-27 18:48:34 +02:00
break;
}
// Same value already exists, don't add new value
return false;
}
// Add value
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
if (it == mImpl->mValues->end()) {
Refactorization: Allocate Token::_values (ValueFlow information) dynamically, reducing size of each token by around 10%
2017-03-27 18:48:34 +02:00
ValueFlow::Value v(value);
if (v.varId == 0)
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
v.varId = mImpl->mVarId;
astyle formatting [ci skip]
2018-11-14 19:10:52 +01:00
if (v.isKnown() && v.isIntValue())
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
mImpl->mValues->push_front(v);
Fix FP when copying pointer to string (#1479)
2018-11-14 06:59:25 +01:00
else
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
mImpl->mValues->push_back(v);
Refactorization: Allocate Token::_values (ValueFlow information) dynamically, reducing size of each token by around 10%
2017-03-27 18:48:34 +02:00
}
} else {
ValueFlow::Value v(value);
if (v.varId == 0)
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
v.varId = mImpl->mVarId;
mImpl->mValues = new std::list<ValueFlow::Value>(1, v);
Refactorization: Allocate Token::_values (ValueFlow information) dynamically, reducing size of each token by around 10%
2017-03-27 18:48:34 +02:00
}
Add impossible values to ValueFlow (#2186) * Add impossible category * Replace values * Try to adjust known values * Add ! for impossible values * Add impossible with possible values * Remove contradictions * Add values when the branch is not dead * Only copy possible values * Dont bail on while loops * Load std lib in valueflow * Check for function calls * Fix stl errors * Fix incorrect impossible check * Fix heap-after-use error * Remove impossible values when they are lowered * Show the bound and remove overlaps * Infer conditions * Dont push pointer values through dynamic_cast * Add test for dynamic_cast issue * Add shifttoomanybits test * Add test for div by zero * Add a test for issue 9315 * Dont make impossible value inconclusive * Fix FP with shift operator * Improve handleKnownValuesInLoop for impossible values * Fix cppcheck warning * Fix impossible values for ctu * Bailout for streams * Check equality conditions * Fix overflows * Add regression test for 9332 * Remove duplicate conditions * Skip impossible values for invalid value * Check for null * Rename bound to range * Formatting
2019-09-20 15:06:37 +02:00
removeContradictions(*mImpl->mValues);
Refactorization: Allocate Token::_values (ValueFlow information) dynamically, reducing size of each token by around 10%
2017-03-27 18:48:34 +02:00
return true;
}
Refactoring/small corrections to fix warnings from clang -fsanitize=undefined 'member call on null pointer'
2014-05-20 21:55:08 +02:00
void Token::assignProgressValues(Token *tok)
{
Avoid 'unsigned' for int variables
2019-07-13 16:06:24 +02:00
int total_count = 0;
Refactoring/small corrections to fix warnings from clang -fsanitize=undefined 'member call on null pointer'
2014-05-20 21:55:08 +02:00
for (Token *tok2 = tok; tok2; tok2 = tok2->next())
++total_count;
Avoid 'unsigned' for int variables
2019-07-13 16:06:24 +02:00
int count = 0;
Refactoring/small corrections to fix warnings from clang -fsanitize=undefined 'member call on null pointer'
2014-05-20 21:55:08 +02:00
for (Token *tok2 = tok; tok2; tok2 = tok2->next())
Format with uncrustify (#3388)
2021-08-07 20:51:18 +02:00
tok2->mImpl->mProgressValue = count++ *100 / total_count;
Refactoring/small corrections to fix warnings from clang -fsanitize=undefined 'member call on null pointer'
2014-05-20 21:55:08 +02:00
}
SymbolDatabase: Add type information in AST
2015-10-04 19:42:58 +02:00
Token::index(): Created Token member that indicates the Token position in the token list. It can be used to quickly check if tok1 precedes tok2.
2019-06-21 22:16:23 +02:00
void Token::assignIndexes()
{
Avoid 'unsigned' for int variables
2019-07-13 16:06:24 +02:00
int index = (mPrevious ? mPrevious->mImpl->mIndex : 0) + 1;
Token::index(): Created Token member that indicates the Token position in the token list. It can be used to quickly check if tok1 precedes tok2.
2019-06-21 22:16:23 +02:00
for (Token *tok = this; tok; tok = tok->next())
tok->mImpl->mIndex = index++;
}
SymbolDatabase: Add type information in AST
2015-10-04 19:42:58 +02:00
void Token::setValueType(ValueType *vt)
{
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
if (vt != mImpl->mValueType) {
delete mImpl->mValueType;
mImpl->mValueType = vt;
SymbolDatabase: Add type information in AST
2015-10-04 19:42:58 +02:00
}
}
Fixed #7426 (RFC: time to replace simplifyEnum?)
2016-04-22 06:02:54 +02:00
void Token::type(const ::Type *t)
{
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
mImpl->mType = t;
Fixed #7426 (RFC: time to replace simplifyEnum?)
2016-04-22 06:02:54 +02:00
if (t) {
Memoize token nature computation
2017-10-15 01:27:47 +02:00
tokType(eType);
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
isEnumType(mImpl->mType->isEnumType());
Renamed _tokType to mTokType
2018-06-16 16:40:02 +02:00
} else if (mTokType == eType)
Memoize token nature computation
2017-10-15 01:27:47 +02:00
tokType(eName);
Fixed #7426 (RFC: time to replace simplifyEnum?)
2016-04-22 06:02:54 +02:00
}
Fix issue 10076: ValueFlow: False positive after address of var is taken 'T t = {{{&var}}};' (#3283)
2021-06-03 07:31:46 +02:00
const ::Type* Token::typeOf(const Token* tok, const Token** typeTok)
Add check for unnecessary search before insertion This will warn for cases where searching in an associative container happens before insertion, like this: ```cpp void f1(std::set<unsigned>& s, unsigned x) { if (s.find(x) == s.end()) { s.insert(x); } } void f2(std::map<unsigned, unsigned>& m, unsigned x) { if (m.find(x) == m.end()) { m.emplace(x, 1); } else { m[x] = 1; } } ``` In the case of the map it could be written as `m[x] = 1` as it will create the key if it doesnt exist, so the extra search is not necessary. I have this marked as `performance` as it is mostly concerning performance, but there could be a copy-paste error possibly, although I dont think thats common.
2019-05-02 11:04:23 +02:00
{
Fix issue 9710 and 9767: Use AST to get function from argument
2020-06-25 19:54:46 +02:00
if (!tok)
return nullptr;
Fix issue 10076: ValueFlow: False positive after address of var is taken 'T t = {{{&var}}};' (#3283)
2021-06-03 07:31:46 +02:00
if (typeTok != nullptr)
*typeTok = tok;
Add check for unnecessary search before insertion This will warn for cases where searching in an associative container happens before insertion, like this: ```cpp void f1(std::set<unsigned>& s, unsigned x) { if (s.find(x) == s.end()) { s.insert(x); } } void f2(std::map<unsigned, unsigned>& m, unsigned x) { if (m.find(x) == m.end()) { m.emplace(x, 1); } else { m[x] = 1; } } ``` In the case of the map it could be written as `m[x] = 1` as it will create the key if it doesnt exist, so the extra search is not necessary. I have this marked as `performance` as it is mostly concerning performance, but there could be a copy-paste error possibly, although I dont think thats common.
2019-05-02 11:04:23 +02:00
if (Token::simpleMatch(tok, "return")) {
const Scope *scope = tok->scope();
if (!scope)
return nullptr;
const Function *function = scope->function;
if (!function)
return nullptr;
return function->retType;
} else if (Token::Match(tok, "%type%")) {
return tok->type();
} else if (Token::Match(tok, "%var%")) {
const Variable *var = tok->variable();
if (!var)
return nullptr;
return var->type();
} else if (Token::Match(tok, "%name%")) {
const Function *function = tok->function();
if (!function)
return nullptr;
return function->retType;
Fix issue 10076: ValueFlow: False positive after address of var is taken 'T t = {{&var}};' (#3027)
2021-01-08 10:29:32 +01:00
} else if (Token::Match(tok->previous(), "%type%|= (|{")) {
Fix issue 10076: ValueFlow: False positive after address of var is taken 'T t = {{{&var}}};' (#3283)
2021-06-03 07:31:46 +02:00
return typeOf(tok->previous(), typeTok);
Add check for unnecessary search before insertion This will warn for cases where searching in an associative container happens before insertion, like this: ```cpp void f1(std::set<unsigned>& s, unsigned x) { if (s.find(x) == s.end()) { s.insert(x); } } void f2(std::map<unsigned, unsigned>& m, unsigned x) { if (m.find(x) == m.end()) { m.emplace(x, 1); } else { m[x] = 1; } } ``` In the case of the map it could be written as `m[x] = 1` as it will create the key if it doesnt exist, so the extra search is not necessary. I have this marked as `performance` as it is mostly concerning performance, but there could be a copy-paste error possibly, although I dont think thats common.
2019-05-02 11:04:23 +02:00
} else if (Token::simpleMatch(tok, "=")) {
Fix issue 10076: ValueFlow: False positive after address of var is taken 'T t = {{{&var}}};' (#3283)
2021-06-03 07:31:46 +02:00
return Token::typeOf(getLHSVariableToken(tok), typeTok);
Add check for unnecessary search before insertion This will warn for cases where searching in an associative container happens before insertion, like this: ```cpp void f1(std::set<unsigned>& s, unsigned x) { if (s.find(x) == s.end()) { s.insert(x); } } void f2(std::map<unsigned, unsigned>& m, unsigned x) { if (m.find(x) == m.end()) { m.emplace(x, 1); } else { m[x] = 1; } } ``` In the case of the map it could be written as `m[x] = 1` as it will create the key if it doesnt exist, so the extra search is not necessary. I have this marked as `performance` as it is mostly concerning performance, but there could be a copy-paste error possibly, although I dont think thats common.
2019-05-02 11:04:23 +02:00
} else if (Token::simpleMatch(tok, ".")) {
Fix issue 10076: ValueFlow: False positive after address of var is taken 'T t = {{{&var}}};' (#3283)
2021-06-03 07:31:46 +02:00
return Token::typeOf(tok->astOperand2(), typeTok);
Fix issue 9311: False positive duplicateCondition "same if condition" with pointer inside array of struct (#2166) * Check for typeOf through an array * Handle array constructors * Format * Fix compile error on gcc 4.8
2019-09-10 19:41:35 +02:00
} else if (Token::simpleMatch(tok, "[")) {
Fix issue 10076: ValueFlow: False positive after address of var is taken 'T t = {{{&var}}};' (#3283)
2021-06-03 07:31:46 +02:00
return Token::typeOf(tok->astOperand1(), typeTok);
} else if (Token::simpleMatch(tok, "{")) {
int argnr;
const Token* ftok = getTokenArgumentFunction(tok, argnr);
if (argnr < 0)
return nullptr;
if (!ftok)
return nullptr;
if (ftok == tok)
return nullptr;
std::vector<const Variable*> vars = getArgumentVars(ftok, argnr);
if (vars.empty())
return nullptr;
if (std::all_of(
Format with uncrustify (#3388)
2021-08-07 20:51:18 +02:00
vars.begin(), vars.end(), [&](const Variable* var) {
return var->type() == vars.front()->type();
Running astyle [ci skip]
2021-06-03 07:35:50 +02:00
}))
Format with uncrustify (#3388)
2021-08-07 20:51:18 +02:00
return vars.front()->type();
Add check for unnecessary search before insertion This will warn for cases where searching in an associative container happens before insertion, like this: ```cpp void f1(std::set<unsigned>& s, unsigned x) { if (s.find(x) == s.end()) { s.insert(x); } } void f2(std::map<unsigned, unsigned>& m, unsigned x) { if (m.find(x) == m.end()) { m.emplace(x, 1); } else { m[x] = 1; } } ``` In the case of the map it could be written as `m[x] = 1` as it will create the key if it doesnt exist, so the extra search is not necessary. I have this marked as `performance` as it is mostly concerning performance, but there could be a copy-paste error possibly, although I dont think thats common.
2019-05-02 11:04:23 +02:00
}
Fix issue 10076: ValueFlow: False positive after address of var is taken 'T t = {{{&var}}};' (#3283)
2021-06-03 07:31:46 +02:00
Add check for unnecessary search before insertion This will warn for cases where searching in an associative container happens before insertion, like this: ```cpp void f1(std::set<unsigned>& s, unsigned x) { if (s.find(x) == s.end()) { s.insert(x); } } void f2(std::map<unsigned, unsigned>& m, unsigned x) { if (m.find(x) == m.end()) { m.emplace(x, 1); } else { m[x] = 1; } } ``` In the case of the map it could be written as `m[x] = 1` as it will create the key if it doesnt exist, so the extra search is not necessary. I have this marked as `performance` as it is mostly concerning performance, but there could be a copy-paste error possibly, although I dont think thats common.
2019-05-02 11:04:23 +02:00
return nullptr;
}
std::pair<const Token*, const Token*> Token::typeDecl(const Token * tok)
{
Check for mor FPs
2020-06-18 00:06:06 +02:00
if (!tok)
return {};
Add check for unnecessary search before insertion This will warn for cases where searching in an associative container happens before insertion, like this: ```cpp void f1(std::set<unsigned>& s, unsigned x) { if (s.find(x) == s.end()) { s.insert(x); } } void f2(std::map<unsigned, unsigned>& m, unsigned x) { if (m.find(x) == m.end()) { m.emplace(x, 1); } else { m[x] = 1; } } ``` In the case of the map it could be written as `m[x] = 1` as it will create the key if it doesnt exist, so the extra search is not necessary. I have this marked as `performance` as it is mostly concerning performance, but there could be a copy-paste error possibly, although I dont think thats common.
2019-05-02 11:04:23 +02:00
if (Token::simpleMatch(tok, "return")) {
const Scope *scope = tok->scope();
if (!scope)
return {};
const Function *function = scope->function;
if (!function)
return {};
return {function->retDef, function->returnDefEnd()};
} else if (Token::Match(tok, "%type%")) {
return {tok, tok->next()};
} else if (Token::Match(tok, "%var%")) {
const Variable *var = tok->variable();
if (!var)
return {};
if (!var->typeStartToken() || !var->typeEndToken())
return {};
Check for mor FPs
2020-06-18 00:06:06 +02:00
if (Token::simpleMatch(var->typeStartToken(), "auto")) {
const Token * tok2 = var->declEndToken();
if (Token::Match(tok2, "; %varid% =", var->declarationId()))
tok2 = tok2->tokAt(2);
Token::typeDecl: Fix endless recursion for code: void f() { auto val = val; } I did not manage to pinpoint a good location to put a unit test. However this will be tested when daca@home runs the package gcc-avr.
2021-08-26 19:03:35 +02:00
if (Token::simpleMatch(tok2, "=") && Token::Match(tok2->astOperand2(), "!!=") && tok != tok2->astOperand2()) {
Check for mor FPs
2020-06-18 00:06:06 +02:00
std::pair<const Token*, const Token*> r = typeDecl(tok2->astOperand2());
if (r.first)
return r;
}
}
Add check for unnecessary search before insertion This will warn for cases where searching in an associative container happens before insertion, like this: ```cpp void f1(std::set<unsigned>& s, unsigned x) { if (s.find(x) == s.end()) { s.insert(x); } } void f2(std::map<unsigned, unsigned>& m, unsigned x) { if (m.find(x) == m.end()) { m.emplace(x, 1); } else { m[x] = 1; } } ``` In the case of the map it could be written as `m[x] = 1` as it will create the key if it doesnt exist, so the extra search is not necessary. I have this marked as `performance` as it is mostly concerning performance, but there could be a copy-paste error possibly, although I dont think thats common.
2019-05-02 11:04:23 +02:00
return {var->typeStartToken(), var->typeEndToken()->next()};
Check for mor FPs
2020-06-18 00:06:06 +02:00
} else if (Token::Match(tok->previous(), "%name% (")) {
const Function *function = tok->previous()->function();
Add check for unnecessary search before insertion This will warn for cases where searching in an associative container happens before insertion, like this: ```cpp void f1(std::set<unsigned>& s, unsigned x) { if (s.find(x) == s.end()) { s.insert(x); } } void f2(std::map<unsigned, unsigned>& m, unsigned x) { if (m.find(x) == m.end()) { m.emplace(x, 1); } else { m[x] = 1; } } ``` In the case of the map it could be written as `m[x] = 1` as it will create the key if it doesnt exist, so the extra search is not necessary. I have this marked as `performance` as it is mostly concerning performance, but there could be a copy-paste error possibly, although I dont think thats common.
2019-05-02 11:04:23 +02:00
if (!function)
return {};
return {function->retDef, function->returnDefEnd()};
} else if (Token::simpleMatch(tok, "=")) {
return Token::typeDecl(tok->astOperand1());
} else if (Token::simpleMatch(tok, ".")) {
return Token::typeDecl(tok->astOperand2());
} else {
const ::Type * t = typeOf(tok);
if (!t || !t->classDef)
return {};
return {t->classDef->next(), t->classDef->tokAt(2)};
}
}
std::string Token::typeStr(const Token* tok)
{
Fix FP with lifetime containers (#1865)
2019-06-02 10:14:48 +02:00
if (tok->valueType()) {
const ValueType * vt = tok->valueType();
std::string ret = vt->str();
if (!ret.empty())
Running astyle [ci skip]
2019-06-02 13:29:20 +02:00
return ret;
Fix FP with lifetime containers (#1865)
2019-06-02 10:14:48 +02:00
}
Add check for unnecessary search before insertion This will warn for cases where searching in an associative container happens before insertion, like this: ```cpp void f1(std::set<unsigned>& s, unsigned x) { if (s.find(x) == s.end()) { s.insert(x); } } void f2(std::map<unsigned, unsigned>& m, unsigned x) { if (m.find(x) == m.end()) { m.emplace(x, 1); } else { m[x] = 1; } } ``` In the case of the map it could be written as `m[x] = 1` as it will create the key if it doesnt exist, so the extra search is not necessary. I have this marked as `performance` as it is mostly concerning performance, but there could be a copy-paste error possibly, although I dont think thats common.
2019-05-02 11:04:23 +02:00
std::pair<const Token*, const Token*> r = Token::typeDecl(tok);
if (!r.first || !r.second)
return "";
return r.first->stringifyList(r.second, false);
}
std.cfg: Added *experimental* support for math constants liken M_PI. These are *NOT* standard, but they are supported by GCC/Clang and VS.
2019-07-31 11:11:01 +02:00
void Token::scopeInfo(std::shared_ptr<ScopeInfo2> newScopeInfo)
{
Calculate token scopes in advance rather than as the tokenlist is iterated (#2038) * Removed redundant scope calculation * Add scope propagation code to insertToken * Add relevant scope code to Token class * Add code to calculate the scope of Tokens * Add calculateScopes method to class * Add missing include for shared_ptr
2019-07-31 09:19:27 +02:00
mImpl->mScopeInfo = newScopeInfo;
}
std.cfg: Added *experimental* support for math constants liken M_PI. These are *NOT* standard, but they are supported by GCC/Clang and VS.
2019-07-31 11:11:01 +02:00
std::shared_ptr<ScopeInfo2> Token::scopeInfo() const
{
Calculate token scopes in advance rather than as the tokenlist is iterated (#2038) * Removed redundant scope calculation * Add scope propagation code to insertToken * Add relevant scope code to Token class * Add code to calculate the scope of Tokens * Add calculateScopes method to class * Add missing include for shared_ptr
2019-07-31 09:19:27 +02:00
return mImpl->mScopeInfo;
}
Refactorization: Moved code from header to source - from utils.h to new utils.cpp - from token.h to token.cpp - from valueflow.h to valueflow.cpp - from errorlogger.h to errorlogger.cpp
2020-05-10 20:32:59 +02:00
bool Token::hasKnownIntValue() const
{
if (!mImpl->mValues)
return false;
return std::any_of(mImpl->mValues->begin(), mImpl->mValues->end(), [](const ValueFlow::Value& value) {
return value.isKnown() && value.isIntValue();
});
}
bool Token::hasKnownValue() const
{
return mImpl->mValues && std::any_of(mImpl->mValues->begin(), mImpl->mValues->end(), std::mem_fn(&ValueFlow::Value::isKnown));
}
Fix issue 10378: FP derefInvalidIteratorRedundantCheck (#3353)
2021-07-24 22:44:18 +02:00
bool Token::hasKnownValue(ValueFlow::Value::ValueType t) const
{
return mImpl->mValues &&
Format with uncrustify (#3388)
2021-08-07 20:51:18 +02:00
std::any_of(mImpl->mValues->begin(), mImpl->mValues->end(), [&](const ValueFlow::Value& value) {
Running astyle [ci skip]
2021-07-26 16:32:00 +02:00
return value.isKnown() && value.valueType == t;
});
Fix issue 10378: FP derefInvalidIteratorRedundantCheck (#3353)
2021-07-24 22:44:18 +02:00
}
Deduce symbolic values from conditions (#3406)
2021-08-19 22:01:55 +02:00
bool Token::hasKnownSymbolicValue(const Token* tok) const
{
if (tok->exprId() == 0)
return false;
return mImpl->mValues &&
std::any_of(mImpl->mValues->begin(), mImpl->mValues->end(), [&](const ValueFlow::Value& value) {
Fix 10433: assertion failure in ValueFlow (Interval::fromValues) (#3419)
2021-08-26 07:46:40 +02:00
return value.isKnown() && value.isSymbolicValue() && value.tokvalue &&
value.tokvalue->exprId() == tok->exprId();
Deduce symbolic values from conditions (#3406)
2021-08-19 22:01:55 +02:00
});
}
Fix issue 10378: FP derefInvalidIteratorRedundantCheck (#3353)
2021-07-24 22:44:18 +02:00
const ValueFlow::Value* Token::getKnownValue(ValueFlow::Value::ValueType t) const
{
if (!mImpl->mValues)
return nullptr;
auto it = std::find_if(mImpl->mValues->begin(), mImpl->mValues->end(), [&](const ValueFlow::Value& value) {
return value.isKnown() && value.valueType == t;
});
return it == mImpl->mValues->end() ? nullptr : &*it;
}
Refactorization: Moved code from header to source - from utils.h to new utils.cpp - from token.h to token.cpp - from valueflow.h to valueflow.cpp - from errorlogger.h to errorlogger.cpp
2020-05-10 20:32:59 +02:00
bool Token::isImpossibleIntValue(const MathLib::bigint val) const
{
if (!mImpl->mValues)
return false;
for (const auto& v : *mImpl->mValues) {
if (v.isIntValue() && v.isImpossible() && v.intvalue == val)
return true;
if (v.isIntValue() && v.bound == ValueFlow::Value::Bound::Lower && val > v.intvalue)
return true;
if (v.isIntValue() && v.bound == ValueFlow::Value::Bound::Upper && val < v.intvalue)
return true;
}
return false;
}
const ValueFlow::Value* Token::getValue(const MathLib::bigint val) const
{
if (!mImpl->mValues)
return nullptr;
const auto it = std::find_if(mImpl->mValues->begin(), mImpl->mValues->end(), [=](const ValueFlow::Value& value) {
return value.isIntValue() && !value.isImpossible() && value.intvalue == val;
});
return it == mImpl->mValues->end() ? nullptr : &*it;
}
Fix 7524: ValueFlow: false path for 'x<3' (#3393)
2021-08-16 09:19:07 +02:00
const ValueFlow::Value* Token::getMaxValue(bool condition, MathLib::bigint path) const
Refactorization: Moved code from header to source - from utils.h to new utils.cpp - from token.h to token.cpp - from valueflow.h to valueflow.cpp - from errorlogger.h to errorlogger.cpp
2020-05-10 20:32:59 +02:00
{
if (!mImpl->mValues)
return nullptr;
const ValueFlow::Value* ret = nullptr;
for (const ValueFlow::Value& value : *mImpl->mValues) {
if (!value.isIntValue())
continue;
if (value.isImpossible())
continue;
Fix 7524: ValueFlow: false path for 'x<3' (#3393)
2021-08-16 09:19:07 +02:00
if (value.path != 0 && value.path != path)
continue;
Refactorization: Moved code from header to source - from utils.h to new utils.cpp - from token.h to token.cpp - from valueflow.h to valueflow.cpp - from errorlogger.h to errorlogger.cpp
2020-05-10 20:32:59 +02:00
if ((!ret || value.intvalue > ret->intvalue) &&
((value.condition != nullptr) == condition))
ret = &value;
}
return ret;
}
const ValueFlow::Value* Token::getMovedValue() const
{
if (!mImpl->mValues)
return nullptr;
const auto it = std::find_if(mImpl->mValues->begin(), mImpl->mValues->end(), [](const ValueFlow::Value& value) {
return value.isMovedValue() && !value.isImpossible() &&
Format with uncrustify (#3388)
2021-08-07 20:51:18 +02:00
value.moveKind != ValueFlow::Value::MoveKind::NonMovedVariable;
Refactorization: Moved code from header to source - from utils.h to new utils.cpp - from token.h to token.cpp - from valueflow.h to valueflow.cpp - from errorlogger.h to errorlogger.cpp
2020-05-10 20:32:59 +02:00
});
return it == mImpl->mValues->end() ? nullptr : &*it;
}
const ValueFlow::Value* Token::getContainerSizeValue(const MathLib::bigint val) const
{
if (!mImpl->mValues)
return nullptr;
const auto it = std::find_if(mImpl->mValues->begin(), mImpl->mValues->end(), [=](const ValueFlow::Value& value) {
return value.isContainerSizeValue() && !value.isImpossible() && value.intvalue == val;
});
return it == mImpl->mValues->end() ? nullptr : &*it;
}
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
TokenImpl::~TokenImpl()
{
delete mOriginalName;
delete mValueType;
delete mValues;
Optimization: Reduced memory consumption of token list - allocate memory for mTemplateSimplifierPointers only if necessary - reordered member variables in TokenImpl
2020-05-15 09:02:59 +02:00
if (mTemplateSimplifierPointers)
for (auto *templateSimplifierPointer : *mTemplateSimplifierPointers) {
templateSimplifierPointer->token(nullptr);
}
delete mTemplateSimplifierPointers;
Annotations: Add annotation __cppcheck_in_range__(low,high)
2019-07-12 11:09:24 +02:00
while (mCppcheckAttributes) {
struct CppcheckAttributes *c = mCppcheckAttributes;
mCppcheckAttributes = mCppcheckAttributes->next;
delete c;
}
}
void TokenImpl::setCppcheckAttribute(TokenImpl::CppcheckAttributes::Type type, MathLib::bigint value)
{
struct CppcheckAttributes *attr = mCppcheckAttributes;
while (attr && attr->type != type)
attr = attr->next;
if (attr)
attr->value = value;
else {
attr = new CppcheckAttributes;
attr->type = type;
attr->value = value;
attr->next = mCppcheckAttributes;
mCppcheckAttributes = attr;
}
}
bool TokenImpl::getCppcheckAttribute(TokenImpl::CppcheckAttributes::Type type, MathLib::bigint *value) const
{
struct CppcheckAttributes *attr = mCppcheckAttributes;
while (attr && attr->type != type)
attr = attr->next;
if (attr)
*value = attr->value;
return attr != nullptr;
Several fairly significant optimisations (#1518) * Code changes for Token::mImpl optimisation * Added new TokenImpl optimisation Moving members to the TokenImpl struct reduces the size of the Token class, which is a fairly significant optimisation. In my testing on Windows with 32-bit Release-PCRE, this change reduced the size of the Token class from 108 bits to 52 bits and reduced run-time of my test case by around 20%. * Several optimisations Deleted some code that ran very slowly and did nothing, as there is no need to change a Token's string to null if you are about to delete it. Added a frontToken to simplifyCalculations to reduce the amount of work it has to do on already-simplified calculations. Moved template removal to the end of the list as this reduces redundant iteration and saves time. * Added tok argument to simplifyCalculations This means callers can avoid unnecessary work if they know which tokens have already been simplified. Passing nullptr indicates the original behaviour (starting from the front of the list). * Removed mention of member from another change * Re-added and optimised some code deleted in error Changing mTemplateInstantiations to a vector avoids the high cost of doing repeated linear searches. Changing how the code iterates through the array was necessary because the vector can be resized at several points during the loop, which breaks existing references and iterators. * Changed mTemplateInstantiations to a vector This is an optimisation that makes repeated linear searches of this collection significantly faster. Also added a copy constructor to TokenAndName so code can make copies of these objects to keep a reference if a vector gets resized. * A cleaner optimisation to removing template tokens This reverts the previous change to made mInstantiatedTemplates a vector and the iterator changes to support this, and makes mTypesUsedInTemplateInstantiation so the eraseTokens logic can be unified. * Reverted vector to list Also made mTypesUsedInTemplateInstantiation a vector of TokenAndName objects so it can share the same logic as the other members. * Added member for template simplifier pointer This can be used more efficiently than marking Tokens with a flag and then searching through all templates to find the one that matches. * Turned loop inside out This means we only have to iterate through the std::list once. std::list is very expensive to iterate through. * Latest code from danmar and fixed optimisations In particular I have optimised simplifying template instantiation names as this was incredibly slow because of the number of times it had to iterate through the template instantiation list. Previous optimisations to this weren't very effective and broke some edge cases. * Added changes from danmar Made mExplicitInstantiationsToDelete a vector of TokenAndName to be consistent with the rest of the members, which are cleaned up very efficiently. * Tokens can have many templateSimplifierPointers * templateSimplifierPointers must be kept in sync
2018-12-21 13:51:45 +01:00
}