DeepGrove, an independent artificial intelligence research lab based in the United States, has introduced Maple-Preview-20B-A1B, a new lightweight large language model (LLM) designed for efficient on-device AI inference. Announced on August 5, the preview model showcases impressive performance on Apple hardware, reportedly achieving inference speeds of up to 127 tokens per second on an iPhone and 281.5 tokens per second on a MacBook Pro powered by Apple’s M5 Pro chip.
The release highlights DeepGrove’s alternative approach to model optimization, replacing conventional quantization techniques with a novel ternary-weight architecture to improve both efficiency and performance.
High-Speed AI Processing on Apple Devices
According to DeepGrove, Maple-Preview can generate 127 tokens per second when running locally on an iPhone, although the company has not disclosed the specific iPhone model used for benchmarking.
DeepGrove also compares the model’s performance with Bonsai, another lightweight on-device AI model, claiming Maple-Preview delivers approximately 13 times higher throughput, compared with Bonsai’s reported 9.6 tokens per second.
If independently verified, this would place Maple-Preview among the fastest publicly demonstrated local AI models available for Apple’s mobile devices.
20.2 Billion Parameters with Efficient MoE Architecture
Despite its focus on mobile deployment, Maple-Preview remains a substantial language model.
Key architectural specifications include:
- 20.2 billion total parameters
- 1.49 billion active parameters during inference
- Mixture of Experts (MoE) architecture
- 24 transformer layers
- 256 experts, with 8 experts activated per token
- 3:1 SWA-512 and Global Attention hybrid attention mechanism
By activating only a subset of experts for each inference step, the model significantly reduces computational requirements while maintaining strong reasoning capabilities.
Moving Beyond Traditional Quantization
DeepGrove argues that conventional model quantization—widely used to compress AI models for smartphones and edge devices—introduces inherent trade-offs that can negatively impact both efficiency and model quality.
Instead, Maple-Preview adopts what the company calls a ternary-weight approach.
Under this method, neural network weights are constrained to only three possible values:
- -1
- 0
- 1
According to DeepGrove, this dramatically reduces memory consumption and memory bandwidth requirements while simplifying computation, making it particularly suitable for resource-constrained devices such as smartphones and laptops.
Strong Mathematical Reasoning
DeepGrove also showcased Maple-Preview’s reasoning capabilities using mathematical benchmarks.
Running locally on a MacBook Pro equipped with Apple’s M5 Pro processor, the model successfully solved Problem 1 of the 2024 International Mathematical Olympiad (IMO), receiving a perfect score of 7 out of 7.
During the demonstration, inference reportedly reached 281.5 tokens per second, highlighting both the model’s computational efficiency and its ability to tackle advanced reasoning tasks.
Efficient Long-Context Memory Management
Another major focus of Maple-Preview is long-context processing.
DeepGrove presented benchmark data showing that the model maintains relatively low memory usage even when handling extremely large context windows.
According to the company:
- Context length: 131,000 tokens
- Additional memory required: 7.69 GB
Efficient memory management is increasingly important for local AI deployment, particularly as users expect models to retain longer conversations, analyze lengthy documents, and process large codebases without relying on cloud computing.
Preview Release with Future Improvements Planned
Maple-Preview is currently available as a preview release, and DeepGrove says development is ongoing.
Looking ahead, the company plans to enhance the model with adaptive capabilities that allow it to automatically adjust its behavior based on conversational context and individual user preferences. Such personalization could improve responsiveness while preserving the privacy advantages of running AI models directly on local devices.
Outlook
The launch of Maple-Preview reflects the growing momentum behind on-device artificial intelligence, where inference is performed locally rather than in the cloud. This approach offers several advantages, including lower latency, improved privacy, offline functionality, and reduced dependence on remote servers.
While DeepGrove’s performance claims will require independent verification, Maple-Preview demonstrates the potential of alternative model architectures such as ternary-weight networks and sparse Mixture-of-Experts systems. As Apple continues to expand the AI capabilities of its silicon platforms, innovations like Maple-Preview could play an important role in enabling faster, more efficient, and privacy-focused AI experiences across future iPhones, iPads, and Macs.