×
Photo via The IndependentNext

Apple’s New Research Shows AI Might Be Dumber Than We Thought

100% reliable
12 mins
12.38K views
Apple
  • AI’s “intelligence” might just be smoke and mirrors, Apple researchers reveal.
  • Apple’s new research just exposed AI’s Achilles’ heel, and it’s a big one.
  • Even simple math problems are tripping up the smartest AI models—here’s why.
  • Think AI can think? Apple’s study says: Not so fast.

Apple, a global leader in tech innovation, has recently highlighted the limitations of AI in a comprehensive study, shining a spotlight on a critical flaw in AI-driven reasoning capabilities. This revelation comes at a time when AI is being marketed as a transformative force in consumer tech, raising questions about the reliability of these advanced models.

The fact that Apple did this has gotten a lot of attention, but nobody should be surprised at the results.— Gary Marcus, AI critic

Craig Federighi announces Apple Intelligence from Apple Park, California in June 2024.Photo via Apple // Craig Federighi announces Apple Intelligence from Apple Park, California in June 2024.

The Problem: A Simple Math Test

To illustrate, consider this elementary arithmetic challenge:

Oliver picks 44 kiwis on Friday. Then he picks 58 on Saturday. On Sunday, he collects double the amount he did on Friday, but five of those are slightly smaller than average. How many kiwis does Oliver have?

If you arrived at 190 (44 + 58 + 88), you matched the performance of most schoolchildren. However, when posed to over 20 state-of-the-art AI models, the results were startlingly different. Many bots failed to identify that the detail about smaller kiwis was irrelevant, leading to erroneous answers such as 185.

This outcome supports a broader pattern observed in the research: AI systems can be tripped up by seemingly inconsequential details.

Deep Dive into Apple’s Study

Released in October, Apple’s technical paper details how these AI systems struggled with arithmetic questions embedded in complex word problems. The inclusion of distracting yet irrelevant information caused “catastrophic performance drops,” a term the researchers used to describe how the models floundered when challenged by nuanced prompts.

This study not only drew widespread interest due to its rigorous documentation but also because it came from Apple, which is actively integrating AI features into its iPhones and other products. The paper's lead author, Mehrdad Farajtabar, emphasized their core inquiry: “Do these models truly understand mathematical concepts?”

The conclusion? A resounding no.Apple Intelligence is compatible with all iPhone 16 models, as well as Pro model iPhone 15's. Apple Intelligence introduces a suite of new AI-powered tools, mostly launching in the next few months and into 2025.Photo via Apple // Apple Intelligence is compatible with all iPhone 16 models, as well as Pro model iPhone 15's. Apple Intelligence introduces a suite of new AI-powered tools, mostly launching in the next few months and into 2025.

A Familiar Pattern

AI experts are not new to these findings. Similar conclusions have emerged from other research efforts, pointing out that large language models (LLMs), like OpenAI’s GPT, are adept at pattern recognition but lack genuine reasoning abilities. As Melanie Mitchell from the Santa Fe Institute noted, “A large gap in basic abstract reasoning still remains between humans and state-of-the-art AI systems.”

Apple’s research further underscores that these models, while impressive in mimicking human language, are fundamentally constrained. They function by matching learned patterns from extensive training datasets, rather than reasoning through problems. Farajtabar elaborated on this flaw, stating, “They memorized what is out there on the web and do pattern matching... [it’s] not real reasoning.”Photo via Ars Technica // "Simply changing specific names and numbers found in GSM8K tests led to significant decreases in performance in many models." Quote from Ars Technica.

Red Herrings and AI Hallucinations

The study’s results are a stark reminder of AI’s limitations, especially when models produce plausible yet incorrect responses with unwarranted confidence—a phenomenon known as “hallucination.” These issues are more than just curiosities; they have real-world implications. In fields like healthcare or legal documentation, even a minuscule error rate can result in significant consequences.

For instance, a recent analysis of Whisper, an AI-powered transcription tool developed by OpenAI, found that about 1.4% of its transcriptions included fabricated content. These hallucinations could misrepresent statements in sensitive environments, such as court proceedings or monitored phone calls, potentially influencing outcomes based on errors.

Implications for AI’s Future

The broader takeaway is clear: while AI excels in pattern recognition, its failure to engage in true abstract reasoning means it will remain reliant on human oversight, particularly in mission-critical applications. Gary Marcus, an outspoken AI critic, highlighted this limitation: “The ways in which they approach reasoning are an approximation and not the real thing... until we have some new technology.”

Apple’s research, though shedding light on the limits of current AI, suggests Apple is aware of these challenges as it navigates the evolving landscape of AI integration with iPhones, iPads, Macs and more. This transparency could signal a more measured approach in marketing their AI products, contrasting with the often unqualified assurances offered by competitors.Apple Intelligence was marketed as the standout feature of Apple's iPhone 16 line, but many customers are still waiting to try Apple's most advanced AI features.Photo via LifeHacker // Apple Intelligence was marketed as the standout feature of Apple's iPhone 16 line, but many customers are still waiting to try Apple's most advanced AI features.

A Call for Balanced Expectations

The findings should temper the hype surrounding AI capabilities. While AI holds immense potential as a tool to augment human tasks—whether in software development, automated processes, or content generation—it remains essential to apply skepticism when AI steps into roles demanding rigorous logic or high-stakes decisions. Mitchell’s perspective sums it up: even “very young children” can often outperform state-of-the-art AI when it comes to abstract reasoning.

In conclusion, the allure of AI as an all-powerful entity capable of replicating human thought is far from reality. For those invested in the future of AI, Apple’s study is a critical reminder of the technology's current boundaries and a call for innovation that goes beyond mere pattern recognition.

Recommended by the editors:

Thank you for visiting Apple Scoop! As a dedicated independent news organization, we strive to deliver the latest updates and in-depth journalism on everything Apple. Have insights or thoughts to share? Drop a comment below—our team actively engages with and responds to our community. Return to the home page.
Stories related to Apple

Google May Have to Say Goodbye to Chrome

98% reliable10 mins

The Thinnest iPhone Ever: What We Know About the Rumored “iPhone 17 Air”

63% reliable9 mins

Apple News Ads Go In-House: A $10 Billion Gamble

96% reliable11 mins

AirPods Pro 3 Rumors: Release Date, Pricing, Features, and More

66% reliable11 mins

AirTag 2: Release Date, Rumors, Features, and More

67% reliable10 mins

Thinnest iPhone Ever? Latest iPhone 17 Rumors: Features, Leaks, and More

65% reliable15 mins

Apple TV Set Rumors Return: What We Know About the Rumored Smart Display

58% reliable9 mins

Has the Apple Vision Pro Been Discontinued? What We Know So Far

68% reliable12 mins

Apple M4 MacBook Air Rumors: Release Date, Specs, Battery Life, and More

62% reliable9 mins

Upcoming Apple Product Releases: Here’s What’s Coming in 2025

59% reliable17 mins

Apple Vision Pro 2: Expected Launch Date, Features, and Rumored Delays

60% reliable8 mins

100+ Best HD Nature & Landscape Wallpapers for iPhone, iPad, and MacBook

100% reliable6 mins

Apple iPhone SE 4 Rumors: Release Date, Specs, Features, Pricing, and More

67% reliable9 mins

Apple iOS 18.2: Release Date, Features, and More

88% reliable11 mins

Apple’s First Smart Home Display Rumored for 2025 Release

59% reliable8 mins

Apple Reportedly Developing Smart Home Camera for 2026 Launch

58% reliable8 mins

iPhone 16 Only Costs Apple $416 To Manufacture

84% reliable7 mins

iPhone 17 Latest Rumors: Release Date, Camera Upgrades, Slim Model, and More

65% reliable13 mins

Android 16 Rumors: Release Date, New Features, Leaks, and More

71% reliable8 mins

15 Fascinating Facts About NVIDIA

100% reliable13 mins
Apple

Apple

Microsoft

Microsoft

Google

Google

Amazon

Amazon

Meta

Meta

More stories

100+ Free 4K Wallpapers for MacBook Pro

100% reliable89 mins

Apple’s Foldable iPhone: Sizing, Rumors, and What We Know So Far

57% reliable11 mins

Google May Have to Say Goodbye to Chrome

98% reliable10 mins

100+ Best Free Dark Wallpapers for iPhone

100% reliable70 mins

Apple iPhone SE 4 'Confirmed' for March 2025 Launch

68% reliable11 mins

100+ Best HD Space Wallpapers for iPhone

100% reliable76 mins

100+ Best Free Abstract Wallpapers for iPhone

100% reliable85 mins

The Thinnest iPhone Ever: What We Know About the Rumored “iPhone 17 Air”

63% reliable9 mins

Apple News Ads Go In-House: A $10 Billion Gamble

96% reliable11 mins

AirPods Pro 3 Rumors: Release Date, Pricing, Features, and More

66% reliable11 mins

AirTag 2: Release Date, Rumors, Features, and More

67% reliable10 mins

Thinnest iPhone Ever? Latest iPhone 17 Rumors: Features, Leaks, and More

65% reliable15 mins


More stories
Looking for the perfect wallpaper?
Explore thousands of free, high-quality wallpapers from Apple Scoop, specially crafted for your Apple devices.

Gradient

Apple

4K

HD

Landscape

Beach

Marble

Space

City

Pattern

Sunset

Ocean

Moon

Architecture

Quote




More wallpapers