* Revert "core syntax: strip the path from filename on syntax.get (#1168)"
This reverts commit af6c4bc152.
The previous behavior was correct and allowed access to the full path for path-dependant syntaxes.
* Use `Doc.abs_filename` to obtain syntax when possible
This allows matching full paths in language syntaxes, but we lose the
possibility of matching the project root.
* Introduces a flag that syntax writers can turn off named
space_handling, turning it off means that your syntax will take care
of handling the excessive amount of spaces that can slow down the
tokenizer.
* Adds another pattern at the end of every single table that also
improves tokenizer performance by matching words that weren't match by
any of the synxtax patterns.
* Modifies language_md to turn off the provided space_handling and do its
own since it has rules that require a space at the beginning, also
handles long consecutives amount of dashes used in tables that degrade
performance.
* This changes where discussed in collaboration with @Guldoman and
@takase1121 thanks to all!
* mainly the language_md got affected which has some exotic rules
* some other languages are also using spaces at start of pattern
and even if not affected this change tackles that
George Sokianos
Guldoman
Jefferson González
rxi