mirror of
https://github.com/kovidgoyal/kitty.git
synced 2026-11-16 13:08:25 +00:00
Decode 64 byte blocks of valid UTF-8 containing no ESC using the approach of simdutf: blocks are validated with the lookup tables of Keiser and Lemire and converted twelve bytes at a time using shuffles looked up from a mask of the bytes that end code points, with fast paths for runs of ASCII, two byte and three byte sequences. Anything else, ESC, invalid UTF-8 and the end of the input, is decoded as before. The shuffle tables are generated by gen/utf8_decode_tables.py. This decodes the text of the unicode benchmark 2.6 times faster, faster than simdutf's own conversion with error checking, and speeds up the unicode benchmark by ~8%. The new AVX2 decoder is faster than the AVX-512 one, so remove the AVX-512 decoder and use the AVX2 one on CPUs with AVX-512 as well. Also add a test comparing the SIMD decoders with the scalar one on long inputs, which exercise the decoding of blocks, as the existing tests use short inputs. |
||
|---|---|---|
| .. | ||
| __init__.py | ||
| __main__.py | ||
| apc_parsers.py | ||
| bitfields.py | ||
| color_names.py | ||
| config.py | ||
| cursors.py | ||
| go_code.py | ||
| key_constants.py | ||
| README.rst | ||
| rowcolumn-diacritics.txt | ||
| srgb_lut.py | ||
| utf8_decode_tables.py | ||
| wcwidth.py | ||
Scripts to generate code for various things like keys, mouse cursors, unicode data etc. Some of these generate code that is checked into version control. Some generate ephemeral code used during builds. Ephemeral code is in files with a _generated.[h|go|c] extension.