In a nutshell: Apple uses Vision-LLMs for Siri integration without requiring changes to existing apps and provides Core AI PyTorch Extensions to enable developers to run custom models on Apple hardware.
Apple announced new Siri capabilities at WWDC 2026 based on licensed Gemini models that run via Private Cloud Compute. The key innovation is the deployment of Vision-LLMs for screen analysis, along with a new Core AI library for developers.
The new Siri implementation is based on a licensed model derived from Google’s Gemini, which Apple operates through its Private Cloud Compute infrastructure. The approach appears feasible with current technology, but differs significantly from announcements made in 2024, some of which were not implemented.
A central technical innovation is the use of Vision-LLMs: these models extract information directly from the user’s screen content without requiring existing applications to ship specialized integration code. As of June 2024, this model category barely existed in market-ready form. This allows Apple to circumvent the classic integration problem where each app would need to implement independent adapters for new platform features.
New for developers is the Core AI library, specifically tailored to Apple’s hardware. It is based on Meta’s open-source PyTorch project and is provided via Core AI PyTorch Extensions: these Python packages enable existing PyTorch models to be incorporated as torch.export.ExportedProgram and converted into a Core AI AIProgram. The conversion process traverses the FX graph node-by-node and maps ATen operators to Core AI operations.
The new features are available immediately in iOS 27 Developer Beta, though with access restrictions: developers must sign up on a waitlist to test Siri AI features. First practical reports should become available soon.
Source: simonwillison.net · Published June 9, 2026
Lumi AI News — AI-assisted curation in accordance with Art. 50 EU AI Act. Paraphrase and classification by Lumi News Pipeline v1.6.5.