Improve handling of adjacent string literals of different types.
Example of adjacent string literals: "ab" L"cd".
In C89, C++98 and C++03, this is undefined. As of C99 and C++11, this is
well defined and the two string literals are concatenated to L"abcd".
C11 and C++11 introduces the utf16, utf32 and (C++ only) utf8 string
types. Concatenating any of these with a regular c-string works exactely
as the wide string example above. The result of having two adjacent
string literals with different prefix is implementation defined, unless
one is an UTF-8 string literal and the other is a wide string literal.
In this case the behaviour is undefined.
Ignore the undefined and ill-formed programs (this behaviour is unchanged)
and make sure that concatenating a plain c string literal with a prefixed
one works correct (in C99 and C++11 and later versions). It also makes the
behaviour consistent since previously, "ab" L"cd" would result in "abcd"
while L"ab" "cd" would result in L"abcd".
It also means the somewhat awkward updatePropertiesConcatStr() test can
be removed since the added tests would not work if update_properties()
was not called in concatStr().
Since the prefix is stored in the token, testing the type of the string
is not relevant in TestSimplifyTokens. It is tested extensively in
TestToken::stringTypes().
Keeping the prefix in the token allows cppcheck to print the correct
string and char literals in debug and error messages.
To achieve this, move some of the helper functions from token.cpp to
utils.h so that checks that look at string and char literals can reuse
them. This is a large part of this commit.
Note that the only user visible change is that when string and char
literals are printed in error messages, the prefix is now included.
For example:
int f() {
return test.substr( 0 , 4 ) == U"Hello" ? 0 : 1 ;
};
now prints U"Hello" instead of "Hello" in the error message.
Building with enhanced clang warnings indicated a large number of
instances with the warning:
`warning: zero as null pointer constant`
Recommended practice in C++11 is to use `nullptr` as value for
a NULL or empty pointer value. All instances where this warning
was encountered were corrected in this commit.
Where warning was encountered in dependency code (i.e. external library)
no chnages were made. Patching will be offered upstream.
This handles concatenated strings and characters from simplecpp.
Previously, L'c' would be preprocessed to the tokens "L" and "'c'".
cppcheck would then remove the "L" token and set "'c'" to be a wide
character literal. Now, it needs to remove the prefix instead.
When doing this, add handling of utf32 encoded literals (U) and UTF-8
encoded literals (u8).
Paul Fultz II
Robert Reif
Daniel Marjamäki
Oliver Stöneberg
Lars Even Almaas
PKEuS
PKEuS
Rikard Falkeborn
Dmitry-Me
Scott Furry