Training got GPUs. Decode got fast memory. Sparsity gets Sparxity.
Frontier AI models are moving away from treating every token equally. They increasingly activate only the parameters, memory and context a token needs, and leave the rest untouched. Each new form of sparsity is finer than the last, and further from what today’s hardware was built for.
- 2024Expert routingGoogle DeepMindPEER
Neuron-sized experts - 2024MemoryMetaMemory Layers
Select memory entries - 2025AttentionMoonshotMoBA
Select context blocks - 2026MemoryDeepSeekEngram
N-gram lookup - 2026AttentionMiniMaxMSA
Select key-value blocks
Models get smarter for the same compute as they get sparser. But incumbent hardware was built for dense computation: the sparser the model, the more routing and data-movement overhead it adds, so model builders keep sparsity coarse. Here’s what Sparxity turns it into.
On incumbent hardware, a smarter model costs more to run. On Sparxity, the sparser the model, the smarter it gets and the less each token costs.
Model builders go finer-grained across experts, memory and attention, and build smarter models.
Hyperscalers, neoclouds and cloud providers serve them faster, at a fraction of the cost per token.
- Faster per user
- Long agentic sessions stay fast
- More per megawatt
- Skipped work burns no power
- More per dollar
- And more still as models get sparser
From silicon to multi-rack systems, already running for customers around the world.
- 3Tape-outs
- 70k+Chips produced
- 12+Racks deployed
- 30+Customers