Show HN: Parmar – BPE tokenization as a pre-filter for LZMA (+9.6%, and faster)

Subword-tokenization pre-filter for byte-level compressors, and the harness that tested it. +7-9.6% over raw bytes on LZMA2; the gap widens with corpus size then plateaus at the dictionary window. ...