The number of results from a pattern with groups must never be greater
than the number of token types for that pattern.
Also if a token type was undefined, it's now pushed as a `normal` one.
Before, this was only supported by Lua patterns.
This expects the regex to use the same syntax used for patterns. That
is, the token should be split by empty groups.
Before, syntax patterns/regexes that started with `^` didn't have the
desired effect of matching with the start of the line.
Now those patterns are used only when matching the whole line.
Use regular expressions instead of Lua patterns for find and replace editor commands.
Syntax files can now use regex or Lua patterns as before keeping backward compatibility for plugins.
* Cleaned up tokenizer to make subsyntax operations more clear.
* Explanatory comments.
* Made it so push_subsyntax could be safely called elsewhere.
* Unified terminology.
* Minor bug fix.
* State is an incredibly vaguely named variable. Changed convention to represent what it actually is.
* Also changed function name.
* Fixed bug.
* Only one highlighter state is kept per-document as opposed
to one per-docview
* Fixes a bug with retaining older highlighter state as a
DocView wasn't able to detect lines changing above it's viewport
* Renames `highlighter` module to more descriptive `tokenizer`
Jefferson González
Guldoman
Jan200101
Adam Harrison
Francesco Abbate
takase1121
liquidev
rxi