Scaling Laws for Sparse Mixture-of-Experts Retrieval Models

We study how retrieval-augmented mixture-of-experts models scale with expert count and index size, and report a regime where adding experts stops improving recall. Results are from a single lab's reproduction of three public benchmarks; the work has been posted as a preprint and has not been through peer review.