Mixture-of-Experts models make local inference more practical by activating only a small portion of their parameters for each token. A model may have tens of billions of parameters while using a fraction of them for any one computation.
Memory remains the constraint: all experts normally need to be available even when most are idle. Selective parameter loading changes that tradeoff. If a system can identify the experts a prompt needs and move the rest from storage only when required, larger models can run on devices with limited active memory.
Frontier models attract most of the attention, but local models are improving through a different pressure. Limited memory, compute, power, and cost force teams to make inference more efficient. That work can bring capable models onto consumer hardware while keeping data on the device and avoiding a per-token service dependency.