Apple isn't backing down from the AI race, and it's got a powerful new tool to prove it. In a bold move, Apple has unveiled its answer to Nvidia's DGX boxes, and it's all about harnessing the potential of its existing Macs.
The Power of Collaboration: Apple and Exo Labs Team Up
Apple has collaborated with Exo Labs to create a groundbreaking AI processing capability, known as EXO 1.0. This innovative tool allows up to four Thunderbolt 5 Mac Studio desktops or two MacBook Pro laptops to work together seamlessly as an "AI cluster." The result? A powerful tandem processing system that can handle AI models with up to a trillion parameters, a feat that would be challenging for a single machine.
The secret lies in the Thunderbolt 5 connections, which enable these systems to unite their resources, creating a unified memory pool for the AI models to access. In a recent web demo, Apple's product team showcased four M3 Ultra-equipped Mac Studio desktops running a 1-trillion parameter model called Kimi-K2-Thinking, all while consuming less than 500 watts of power collectively. This is a significant achievement, as it's far more efficient than traditional GPU-based AI clusters, which can draw up to 700 watts.
But here's where it gets controversial: Nvidia's DGX Spark boxes, while rated for up to 240 watts under maximum load, have faced criticism from developers like John Carmack, who suspect reduced performance. Connecting multiple DGX Spark systems may theoretically draw more power, but Apple's solution seems to have an edge, especially for developers aiming to run multiple clusters.
Unleashing the Power of the M5 Chip
Apple isn't stopping there. In a recent blog post, the company proudly showcased the AI capabilities of its M5 chip. With macOS 26.2, developers can now access the M5's Neural Accelerators through MLX, enhancing memory efficiency for AI workloads. This development is significant because it improves the time-to-first-token (TTFT) metric, which measures how quickly a model generates its first piece of information after a prompt.
The M5 generation of Apple's processors boasts a neural accelerator in each GPU core, significantly boosting AI performance. In our review of the M5 MacBook Pro 14, we found this upgrade to be a game-changer.
Apple's blog post highlights the M5 chip's ability to provide dedicated matrix-multiplication operations, which are crucial for many machine learning tasks. MLX leverages the Tensor Operations (TensorOps) and Metal Performance Primitives framework introduced with Metal 4 to support the Neural Accelerators' features.
The result? A drastic reduction in TTFT for most large language models (LLMs). When evaluating its M5 hardware on the Qwen model developed by Alibaba Cloud, Apple found that M5's TTFT rating was up to four times faster than its M4 counterparts.
This performance improvement extends to both image generation and text. When generating a 1,024-by-1,024 image with FLUX-dev-4bit (12B parameters) using MLX, the M5 hardware performed up to 3.8 times faster than M4 alternatives.
This is a game-changer for anyone utilizing Apple Intelligence features on macOS and for developers aiming to create AI on a Mac. However, if you're looking to accelerate these workloads on a MacBook, using an external RTX graphics card may be the way to go, at least until we see the performance of two M5 MacBook Pro laptops working in tandem.
And this is the part most people miss: Apple's commitment to innovation and collaboration is what sets it apart. By working with developers like Exo Labs and continuously improving its hardware and software, Apple is staying at the forefront of the AI race. So, what do you think? Is Apple's approach to AI development a step in the right direction? We'd love to hear your thoughts in the comments below!
