* Introduces a flag that syntax writers can turn off named
space_handling, turning it off means that your syntax will take care
of handling the excessive amount of spaces that can slow down the
tokenizer.
* Adds another pattern at the end of every single table that also
improves tokenizer performance by matching words that weren't match by
any of the synxtax patterns.
* Modifies language_md to turn off the provided space_handling and do its
own since it has rules that require a space at the beginning, also
handles long consecutives amount of dashes used in tables that degrade
performance.
* This changes where discussed in collaboration with @Guldoman and
@takase1121 thanks to all!
Guldoman
jgmdev
Adam Harrison
Nightwing
Francesco Abbate
rxi