Apple has made several major AI-related announcements and research breakthroughs in recent weeks that could have significant implications for bringing advanced AI capabilities to iPhones and other Apple devices while preserving user privacy.
Researchers Unveil Method for Running LLM AI Models Directly on iPhones
In a research paper, Apple scientists detailed a new technique that would allow large language models (LLMs) like ChatGPT to run directly on iPhones without needing to connect to the internet. This has been a major challenge until now due to the limited memory and computing power of smartphones compared to server farms.
By running AI models directly on devices rather than in the cloud, Apple can enable features like natural conversation while keeping data fully encrypted and not storing personal data on external servers. The researchers used various optimization methods to shrink LLMs to run efficiently on flash storage chips used in iPhones.
While not yet ready for primetime, this research represents a huge step toward bringing more advanced AI capabilities to Apple’s devices while staying true to their privacy commitment. Integrating conversational features by running models like “AppleGPT” natively could be a major selling point of future products.
Ferret: New Open-Source Multimodal LLM Released
In another key AI advancement, Apple quietly released an open-source multimodal LLM system called Ferret in October 2022, representing one of Apple’s only open-source AI projects.
Multimodal LLMs like Ferret can process and generate content across multiple modes like text, images, audio, video, and code. Ferret demonstrates Apple’s continued innovation and investment into on-device AI and shows they are taking steps to allow external researchers to build on internal work.
By releasing Ferret openly and for free for the research community rather than keeping it fully proprietary, Apple facilitates more rapid advancement that ultimately benefits their products and services down the line. The Ferret paper itself even suggests integrating the model into smartphones, showing alignment with the iPhone LLM research.
Apple Execs Tout Custom Silicon and AI Ambitions
In an interview last week, senior VPs Johny Srouji and John Ternus discussed Apple’s ongoing efforts to design more custom silicon and hardware tailored for AI and ML tasks, including the A16 Bionic’s machine learning and neural engine cores.
Ternus highlighted how integrating AI processing and ML accelerators directly into their SoC chipsets rather than offloading to a secondary chip provides higher performance and lower power consumption critical for mobile devices. These chips help enable advanced capabilities like live text translations in the Camera app on recent models.
Srouji also mentioned that further specializing their chips and devices for efficiently running ML models and apps plays a key role in differentiating Apple’s products and user experiences going forward as AI becomes more prevalent across their lineup. Their continued silicon advancements lay the foundation to keep pace with rapidly evolving algorithmic breakthroughs.
HUGS Avatar System Showcases Cutting-Edge AI Research
Apple revealed HUGS, a new AI method for procedurally generating 3D human avatar models from video references. It demonstrates remarkable ability to accurately animate new models performing novel motions and styles based on only small example pose clips.
The technology could significantly advance the state of digital human rendering and animation for films, games, VR, and metaverse applications by greatly reducing the manual effort needed. HUGS and the previously discussed LLM advancements showcase the cutting-edge foundational AI research underway at Apple beyond just product applications.
Continuing to push boundaries in areas like generative video, multimodal understanding, and avatar/scene reconstruction further cements their leadership in delivering the most sophisticated consumer AI experiences powered by custom silicon. Integrating these emerging techniques over time could introduce incredibly immersive and intuitive AR, camera, and conversational interfaces tailored around personal user data while protecting privacy.
What’s Next for Apple’s AI Roadmap?
While Apple tends to keep plans secret, the string of recent announcements and research papers provide valuable clues into their AI strategy and areas of investment.
Everything points to greatly expanding on-device intelligence and ML acceleration to introduce more ambient, assistive and conversational features while eliminating reliance on external servers. Apple’s leading chip design team will continue optimize their silicon to efficiently run advanced neural networks and processes directly on consumer devices.
We will likely see more multimodal capabilities like Ferret integrated into apps like Photos, Camera, and Look Around in Maps to connect images, text, audio, and video. Areas like personalized recommendations, fine-grained location context, augmented reality experiences, and natural language interfaces seem primed for enhancement using Apple’s privacy-centric AI approach.
If these innovations pay off as intended, Apple could cement dominance in consumer AI much like they have in smartphone technologies and chip performance, providing users an unparalleled intelligent experience packaged within their elegantly designed hardware and software ecosystem.
To err is human, but AI does it too. Whilst factual data is used in the production of these articles, the content is written entirely by AI. Double check any facts you intend to rely on with another source.
