The iPhone 17 Pro has managed to run a 400 billion-parameter large language model, even though such models usually need at least 200GB of memory, which makes this demonstration surprising, given the phone only has 12GB of RAM.
This result was possible because developers avoided loading the full model into memory and instead used a different method that streams data directly from storage, allowing the device to handle workloads that would otherwise exceed its hardware limits.
According to an open-source demo shared by Flash-MoE and developer @anemll, the model runs on the iPhone 17 Pro with heavy limitations but still completes tasks.
The setup uses a Mixture of Experts approach, which means the model activates only a small portion of its parameters for each request, instead of using all 400 billion at once, and this reduces the memory requirement significantly.
At the same time, the system streams model data from the iPhone’s SSD to the GPU, which avoids filling up RAM and allows the device to process queries in parts.
Performance remains slow but functional
The demonstration shows the iPhone 17 Pro generating text at around 0.6 tokens per second, which translates to roughly one word every one to two seconds, so while the model works, it does not deliver practical speeds for most users.
Even with this limitation, the result shows that smartphones can handle much larger AI models than expected when developers use efficient techniques like quantization and selective parameter activation.
Running a model locally gives users full privacy and removes the need for an internet connection, but it also places heavy strain on the battery and hardware during long sessions.
This demonstration shows clear progress in mobile AI, as future improvements in chip design and software optimization will likely push these models closer to usable speeds on consumer devices.
