Apple’s development of a new AI model, FastVLM, points to a technology that could drive its anticipated smart glasses. Reports suggest Apple plans to release AI-enabled wearables, with smart glasses potentially launching around 2027. This new model offers a glimpse into how Apple’s on-device AI might function.
FastVLM, a Visual Language Model, processes high-resolution images with remarkable speed and efficiency. Apple’s Machine Learning Research team designed it using MLX, their open machine learning framework tailored for Apple Silicon. This allows models like FastVLM to train and run locally on Apple devices.
According to Apple, FastVLM demands significantly less computing power than comparable models. The model demonstrates practical capabilities, correctly identifying held-up fingers, on-screen emojis, and handwritten text.
Efficiency and Speed by Design
At the heart of FastVLM is an encoder named FastViTHD, specifically engineered for efficient VLM performance with high-resolution images. Apple states this encoder is up to 3.2 times faster and its model is 3.6 times smaller than similar vision models. Such efficiency is critical if a device needs to process visual information locally, without cloud dependency, for instant responses.
FastVLM also outputs fewer tokens, a key factor for fast inference – the stage where the model interprets data and generates a response. Apple claims its model achieves an 85 times faster time-to-first-token compared to similar models. This metric measures the delay between your initial prompt and the model’s first piece of output. Fewer tokens on a faster, lighter model means quicker processing for you.
The GitHub repository for FastVLM notes its smallest variant outperforms LLaVA-OneVision-0.5B with this 85x faster time-to-first-token and a 3.4x smaller vision encoder. Larger variants using the Qwen2-7B Large Language Model reportedly outperform recent models like Cambrian-1-8B with a 7.9x faster time-to-first-token, using a single image encoder.
Implications for Future Wearables
The combination of speed, low computational demand, and local processing makes FastVLM a strong candidate for devices like smart glasses. These wearables require immediate understanding and reaction to the user’s environment. The ability to perform complex visual tasks without relying on a constant cloud connection is a significant advantage.
Apple has made FastVLM available on GitHub, and a technical report can be found on arXiv. The project also includes a demo iOS app to showcase the model’s performance on a mobile device. This signals Apple’s continued investment in AI capabilities that could define its next generation of personal technology.