Home Artificial intelligence AI’s Real Bottleneck Isn’t Algorithms, It’s The Rare Earths Of Data
Artificial intelligence

AI’s Real Bottleneck Isn’t Algorithms, It’s The Rare Earths Of Data

Share


The loudest conversations in artificial intelligence are still about models and computers, bigger architectures, faster chips, more parameters. But after years of watching real AI projects succeed or fail, I’ve come to a different conclusion:

AI’s biggest constraint isn’t algorithms anymore. It’s data…specifically, high-quality, forward-looking data.

A useful way to think about this is through an analogy most people already understand: rare earth elements. Rare earths are a group of elements with outsized importance in modern technology despite being difficult to source and refine. You can have brilliant engineering, but without the right input, performance collapses.

AI is entering the same phase.

The Rare Earth Moment for AI

In the early days of AI, the assumption was simple: better models would solve everything. If something didn’t work, you just needed more data, more computers, or a more advanced algorithm.

That assumption is breaking down.

Today, we see massive models trained on oceans of data that still hallucinate, drift, or fail under real-world conditions. We see AI systems that look impressive in demos but struggle when markets shift, consumers change behavior, or the economy turns.

Why?

Because most AI systems are trained on backward-looking exhaust; clicks, transactions, logs, scraped text. Those data sources explain what already happened. They are noisy, biased, and often disconnected from why people actually make decisions.

In other words, AI has plenty of algorithms. What it lacks is signal.

And this is not just my observation; it’s increasingly the consensus among people who live in the ‘last mile’ of AI reliability. Barr Moses (CEO, Monte Carlo) puts the issue plainly: While organizations have invested in resources, technologies, and even teams to drive forward their AI initiatives, all too often they overlook one of AI’s most fundamental building blocks: high quality, reliable data,”

Why Signal Matters More Than Scale

Here’s a truth that doesn’t get enough attention: small improvements in signal quality often outperform large improvements in model complexity.

If your inputs are biased or incomplete, no amount of machine-learning magic will fix that. The system may still run, but it won’t perform reliably. Just like advanced hardware fails without rare earth inputs. The system is technically intact but functionally compromised.

The most valuable data today has a few defining characteristics:

  • It captures intent, confidence, and plans, not just past actions.
  • It is representative, not limited to one platform or behavior.
  • It is cleanly sourced and auditable, not scraped or inferred.
  • It is longitudinal, meaning it persists through economic cycles.

This kind of data is hard to find because it’s expensive to build, difficult to maintain, and requires discipline over long periods of time. But when it exists, it changes what AI can do.

The Data Shortage Is Already Here

Even in the broader AI industry, the “input” constraint is becoming explicit. In October 2025, Goldman Sachs’ chief data officer, Neema Raphael said, “We’ve already run out of data,” arguing that future progress depends less on the open web and more on proprietary, high-context datasets that can be normalized and trusted.

That’s exactly the rare-earth dynamic: value concentrates in scarce, hard-to-replicate inputs.

What Happens When You Get the Inputs Right

Over the past few years, I’ve seen how high-quality consumer signals can be used to generate macro-level insights that arrive weeks or even months ahead of traditional indicators.

My October 2025 article, With Consumer Data Models, Who Needs Government Data, reviewed how models built on forward-looking consumer data have been able to anticipate changes in housing activity, employment trends, consumer spending, and durable goods demand; often before those shifts show up in government reports or consensus forecasts.

These aren’t academic exercises. When macro signals move, they influence trillions of dollars in capital decisions, from equity markets to corporate planning. Getting those signals earlier — and understanding why they’re moving — is far more valuable than shaving milliseconds off the delivery of old data.

Three years ago I interviewed Accenture’s Chief Strategist, Muqsit Ashraf about How AI will Change the C-Suite. In his recent blog about AI discussions at the World Economic Forum in Davos, one statement jumped out: “Use data as a growth engine, not just an operational input: proprietary, high-quality data is the foundation for differentiation and new value pools.”

Why This Changes the AI Conversation

The AI market is slowly realizing that outcomes don’t come from models alone. They come from the quality of the inputs those models depend on.

This has important implications:

  • Bigger models don’t guarantee better decisions.
  • Synthetic data can amplify errors instead of fixing them.
  • Speed doesn’t matter if you’re racing toward the wrong answer.
  • Explainability matters as much as accuracy.

As regulation increases and enterprise buyers demand more accountability, AI systems trained on opaque or fragile data sources face growing scrutiny. In contrast, systems built on trustworthy, well-governed data gain credibility and longevity.

This is why the analogy to rare earth elements resonates so strongly. Rare earths didn’t matter until the world realized technology couldn’t scale without them. AI is having that realization now.

The Next Phase of AI Value Creation

The next wave of AI winners won’t be defined by who builds the largest model. They’ll be defined by who controls the most reliable signal.

In practical terms, that means data that reflects real people, real intentions, and real decision drivers that is collected consistently, ethically, and over time. It means shifting the conversation from “What can this model do?” to “What inputs does it depend on?”

AI doesn’t fail because it’s too advanced. It fails because it’s trained on the wrong ingredients.

Just as rare earth elements quietly underpin the modern economy, high-quality data will quietly determine which AI systems deliver durable value and which ones never move beyond impressive demos.

The future of AI won’t belong to those who shout the loudest about algorithms. It will belong to those who invested early in the rare earths of data.

As Barr Moses said in a recent interview, “no data is better than bad data.”

Disclosure: The consumer sentiment study referenced above was conducted by my company, Prosper Insights & Analytics. This is the same dataset used by the National Retail Federation, and available from Amazon Web Services, Bloomberg, and the London Stock Exchange Group for economic benchmarking.



Source link

Leave a comment

Leave a Reply

Your email address will not be published. Required fields are marked *