×
Photo via The IndependentNext

Apple’s New Research Shows AI Might Be Dumber Than We Thought

100% reliable
12 mins
16.55K views
Apple
  • AI’s “intelligence” might just be smoke and mirrors, Apple researchers reveal.
  • Apple’s new research just exposed AI’s Achilles’ heel, and it’s a big one.
  • Even simple math problems are tripping up the smartest AI models—here’s why.
  • Think AI can think? Apple’s study says: Not so fast.

Apple, a global leader in tech innovation, has recently highlighted the limitations of AI in a comprehensive study, shining a spotlight on a critical flaw in AI-driven reasoning capabilities. This revelation comes at a time when AI is being marketed as a transformative force in consumer tech, raising questions about the reliability of these advanced models.

The fact that Apple did this has gotten a lot of attention, but nobody should be surprised at the results.— Gary Marcus, AI critic

Craig Federighi announces Apple Intelligence from Apple Park, California in June 2024.Photo via Apple // Craig Federighi announces Apple Intelligence from Apple Park, California in June 2024.

The Problem: A Simple Math Test

To illustrate, consider this elementary arithmetic challenge:

Oliver picks 44 kiwis on Friday. Then he picks 58 on Saturday. On Sunday, he collects double the amount he did on Friday, but five of those are slightly smaller than average. How many kiwis does Oliver have?

If you arrived at 190 (44 + 58 + 88), you matched the performance of most schoolchildren. However, when posed to over 20 state-of-the-art AI models, the results were startlingly different. Many bots failed to identify that the detail about smaller kiwis was irrelevant, leading to erroneous answers such as 185.

This outcome supports a broader pattern observed in the research: AI systems can be tripped up by seemingly inconsequential details.

Deep Dive into Apple’s Study

Released in October, Apple’s technical paper details how these AI systems struggled with arithmetic questions embedded in complex word problems. The inclusion of distracting yet irrelevant information caused “catastrophic performance drops,” a term the researchers used to describe how the models floundered when challenged by nuanced prompts.

This study not only drew widespread interest due to its rigorous documentation but also because it came from Apple, which is actively integrating AI features into its iPhones and other products. The paper's lead author, Mehrdad Farajtabar, emphasized their core inquiry: “Do these models truly understand mathematical concepts?”

The conclusion? A resounding no.Apple Intelligence is compatible with all iPhone 16 models, as well as Pro model iPhone 15's. Apple Intelligence introduces a suite of new AI-powered tools, mostly launching in the next few months and into 2025.Photo via Apple // Apple Intelligence is compatible with all iPhone 16 models, as well as Pro model iPhone 15's. Apple Intelligence introduces a suite of new AI-powered tools, mostly launching in the next few months and into 2025.

A Familiar Pattern

AI experts are not new to these findings. Similar conclusions have emerged from other research efforts, pointing out that large language models (LLMs), like OpenAI’s GPT, are adept at pattern recognition but lack genuine reasoning abilities. As Melanie Mitchell from the Santa Fe Institute noted, “A large gap in basic abstract reasoning still remains between humans and state-of-the-art AI systems.”

Apple’s research further underscores that these models, while impressive in mimicking human language, are fundamentally constrained. They function by matching learned patterns from extensive training datasets, rather than reasoning through problems. Farajtabar elaborated on this flaw, stating, “They memorized what is out there on the web and do pattern matching... [it’s] not real reasoning.”Photo via Ars Technica // "Simply changing specific names and numbers found in GSM8K tests led to significant decreases in performance in many models." Quote from Ars Technica.

Red Herrings and AI Hallucinations

The study’s results are a stark reminder of AI’s limitations, especially when models produce plausible yet incorrect responses with unwarranted confidence—a phenomenon known as “hallucination.” These issues are more than just curiosities; they have real-world implications. In fields like healthcare or legal documentation, even a minuscule error rate can result in significant consequences.

For instance, a recent analysis of Whisper, an AI-powered transcription tool developed by OpenAI, found that about 1.4% of its transcriptions included fabricated content. These hallucinations could misrepresent statements in sensitive environments, such as court proceedings or monitored phone calls, potentially influencing outcomes based on errors.

Implications for AI’s Future

The broader takeaway is clear: while AI excels in pattern recognition, its failure to engage in true abstract reasoning means it will remain reliant on human oversight, particularly in mission-critical applications. Gary Marcus, an outspoken AI critic, highlighted this limitation: “The ways in which they approach reasoning are an approximation and not the real thing... until we have some new technology.”

Apple’s research, though shedding light on the limits of current AI, suggests Apple is aware of these challenges as it navigates the evolving landscape of AI integration with iPhones, iPads, Macs and more. This transparency could signal a more measured approach in marketing their AI products, contrasting with the often unqualified assurances offered by competitors.Apple Intelligence was marketed as the standout feature of Apple's iPhone 16 line, but many customers are still waiting to try Apple's most advanced AI features.Photo via LifeHacker // Apple Intelligence was marketed as the standout feature of Apple's iPhone 16 line, but many customers are still waiting to try Apple's most advanced AI features.

A Call for Balanced Expectations

The findings should temper the hype surrounding AI capabilities. While AI holds immense potential as a tool to augment human tasks—whether in software development, automated processes, or content generation—it remains essential to apply skepticism when AI steps into roles demanding rigorous logic or high-stakes decisions. Mitchell’s perspective sums it up: even “very young children” can often outperform state-of-the-art AI when it comes to abstract reasoning.

In conclusion, the allure of AI as an all-powerful entity capable of replicating human thought is far from reality. For those invested in the future of AI, Apple’s study is a critical reminder of the technology's current boundaries and a call for innovation that goes beyond mere pattern recognition.

Recommended by the editors:

Thank you for visiting Apple Scoop! As a dedicated independent news organization, we strive to deliver the latest updates and in-depth journalism on everything Apple. Have insights or thoughts to share? Drop a comment below—our team actively engages with and responds to our community. Return to the home page.
Stories related to Apple

iPhone 17 Pro: Rumors, Pricing, Release Date, Design and More

61% reliable12 mins

Why Did Apple Discontinue the iPhone Mini?

100% reliable10 mins

Affordable Apple Vision Pro 'Delayed Until After 2027'

68% reliable10 mins

Apple Exploring iPhone-Connected Smart Glasses, Rumor Says

59% reliable9 mins

Apple Vision Pro: M5 Chip Expected by 2025, Affordable Model Delayed

65% reliable7 mins

What Is Steve Jobs' NeXT? Inside the 'Failure' That Reinvented Apple

100% reliable12 mins

The Original Macintosh: How Apple's Vision Changed Technology Forever

100% reliable14 mins

11 New Apple CarPlay Features You Need to Know from iOS 18 and iOS 18.1

100% reliable9 mins

From the Brink of Bankruptcy to Almost $4 Trillion: How Apple Did It

100% reliable15 mins

Apple's New Smart Home Display Rumored to Launch in 2025

59% reliable9 mins

Microsoft Windows 12: Release Date, Rumors, Leaks and More

79% reliable10 mins

Samsung Confirms Its Apple Vision Pro Rival Headset is Coming 2025

83% reliable10 mins

Apple iPhone 17: Rumors Hint at 120Hz Displays for All Models

64% reliable9 mins

Apple Announces $46.2 Billion in Global iPhone Sales

100% reliable12 mins

Shocking: Here's How Much Money Apple Makes Every Minute

100% reliable9 mins

Apple M4 Pro and M4 Max Chips: Early Benchmark Scores Leak Online

77% reliable8 mins

Apple's M5 Chipset: Release Date, Rumors, Features, and More

62% reliable9 mins

Apple’s M4 MacBook Pro, iMac, and Mac Mini: Price, Release Date, and Key Features

100% reliable12 mins

Apple Ditches Broadcom: iPhone 17 Set to Feature In-House Wi-Fi Chip

67% reliable9 mins

25+ Fascinating Facts About ChatGPT

100% reliable18 mins
More stories

iPhone 17 Pro: Rumors, Pricing, Release Date, Design and More

61% reliable12 mins

Why Did Apple Discontinue the iPhone Mini?

100% reliable10 mins

Affordable Apple Vision Pro 'Delayed Until After 2027'

68% reliable10 mins

Apple Exploring iPhone-Connected Smart Glasses, Rumor Says

59% reliable9 mins

Apple Vision Pro: M5 Chip Expected by 2025, Affordable Model Delayed

65% reliable7 mins

What Is Steve Jobs' NeXT? Inside the 'Failure' That Reinvented Apple

100% reliable12 mins

The Original Macintosh: How Apple's Vision Changed Technology Forever

100% reliable14 mins

11 New Apple CarPlay Features You Need to Know from iOS 18 and iOS 18.1

100% reliable9 mins

100+ Best Free Wallpapers for iPhone 14, 14 Plus, 14 Pro, and 14 Pro Max

100% reliable77 mins

From the Brink of Bankruptcy to Almost $4 Trillion: How Apple Did It

100% reliable15 mins

Apple's New Smart Home Display Rumored to Launch in 2025

59% reliable9 mins

Microsoft Windows 12: Release Date, Rumors, Leaks and More

79% reliable10 mins


More stories
Looking for the perfect wallpaper?
Explore thousands of free, high-quality wallpapers from Apple Scoop, specially crafted for your Apple devices.

Gradient

Apple

4K

HD

Landscape

Beach

Marble

Space

City

Pattern

Sunset

Ocean

Moon

Architecture

Quote




More wallpapers