Liam Chalcroft

Tokenizer-Generator Coupling in Medical Image Generation

L. Chalcroft

Advances in Neural Information Processing Systems (NeurIPS) 2026, poster.

arXivcodemedtokenizersmedlatents

Reading the results

The tokenizer, generator and sampler jointly determine the ranking. The three-seed vocabulary-1024 comparison tests whether the interaction persists across training seeds.

The study concerns unconditional 64 × 64 medical-style images and non-clinical metrics. Continuous references were not given an equivalent sampler sweep.

Original two-panel tokenizer-generator comparison: seed variation and FID measurements across generator families.
Original study figure. The vocabulary-1024 block compares three training seeds; the rest of the grid has one. Source and study scope · Full-size figure

Abstract. Latent medical image generators usually treat the tokenizer as fixed preprocessing. We test whether this separation holds in a controlled ChestMNIST study at 64 × 64 that crosses discrete tokenizers, generator families, and sampler settings under a shared latent grid, with continuous-latent reference cells. The rankings depend jointly on the tokenizer, generator, and sampler. The best quantizer changes with the generator, and validation-based sampler selection changes the apparent generator ranking. We interpret this through a rate-distortion-modelability framing in which modelability is conditional on the generator, sampler, and inference budget. The interaction persists when the vocabulary-1024 block is retrained at three seeds, with 4 of 9 pairwise quantizer comparisons exceeding three seed standard deviations, including a reversal between LFQ and FSQ from MaskGIT to D3PM; the rest of the grid was trained at a single seed and is correspondingly less certain. Reconstruction PSNR alone is not a reliable selection criterion. On LFQ-1024, retuning the D3PM and SEDD samplers on a held-out validation split reduces FID-192 from 0.44/0.41 at the default budget to 0.09/0.10 at lower NFE, replicated across seeds, although the continuous references were not given an equivalent sampler sweep. FID-192, our internal ranking metric, ranks consistently with standard FID-2048 (Spearman 0.89) and with a label-free classifier two-sample test (0.86). All experiments are unconditional, use low-resolution 64 × 64 medical-style images and are evaluated with non-clinical FID-based metrics, and our claims are limited to that setting. Implementations are released at https://github.com/liamchalcroft/medtokenizers and https://github.com/liamchalcroft/medlatents.

arXiv abstract

Year 2026Kind conference paperVenue Advances in Neural Information Processing Systems (NeurIPS 2026)

Materials

  • Experiments, results and figure scripts

    The controlled tokenizer-generator study and its stored result data. Most cells have one training seed; the vocabulary-1024 block includes the multi-seed comparison.

  • Tokenizer library

    Discrete and continuous medical-image tokenizer implementations used by the study.

  • Generator library

    Generative model families over tokenised images, including the samplers whose settings affect the reported rankings.

  • Study scope and comparisons

    The current author manuscript. Results concern unconditional 64 × 64 images and non-clinical generation metrics.

Cite

@inproceedings{chalcroft2026tokenizer,
  author    = {Chalcroft, Liam},
  title     = {{Tokenizer-Generator Coupling in Medical Image Generation}},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS 2026)},
  year      = {2026},
  url       = {https://arxiv.org/abs/2608.07713}
}