But they are not doing it to get a particular project over the line. Or to improve their products. It’s not even about boosting revenue or cutting costs.

In fact, in a lot of ways, it’s the total opposite.

Because the companies in question are doing everything they can to get their staff to use AI as much as possible, and it has led to a phenomenon known as ‘token-maxxing’.

Everybody’s Token

A token is the basic unit of measurement representing the data that’s processed by an AI system – like OpenAI’s ChatGPT or Anthropic’s Claude.

These AI platforms have hoovered up huge swathes of data from countless sources, and all of that information is used to train or teach the platform. Not just the raw information itself, but how different words and ideas interact with each other, how sentences and conversations are structured, and how that information is applied.

And with all of that it’s then supposed to be able to understand what’s been asked of it and “think” of, or create, an appropriate and accurate answer.

That means that every time you ask an AI platform something, it’s essentially doing thousands of searches through all of that data, and through all of the conversations and interactions it’s had with other users, to try to figure out the most helpful answer.

Generally speaking, the more complicated your query the more searches and analysing it will take for the platform to give you the result you want. That means more data is used, which needs to be in some way measurable.

And that’s how we get tokens.

Roughly speaking, a short sentence would represent something in the region of ten to 20 tokens.

But of course all of that data processing comes at a cost, it’s why the power demands from data centres are growing rapidly, it’s the reason why tech firms are racing to build more and more data centres, and it’s the reason why chip-maker Nvidia is one of the few companies that’s so far actually made money from AI – because it’s selling the processors that do the best job at handling all of these queries.

So, more than being a simple measurement of the data that is flowing through an AI platform, a token has also become the basis for how companies are being charged for their use of AI.

Anthropic, for example, charges $10 per million tokens for the latest version of its Claude platform. That’s for the input – or the data you feed into the system – so what you ask it, or what you give it to analyse.

It charges $50 per million tokens of output, so the data it gives back to you based on your query.

Token it to the Maxx

In recent weeks, though, major tech companies have become locked in a race to use as many tokens as possible.

This is being led by workers, but has essentially been encouraged by the companies themselves. Because a number of firms, including some of the biggest firms in the world, have created policies that reward using more tokens.

In some cases these policies could punish those who are not using “enough”.

Some have issued decrees to staff telling them they had to hit certain minimum weekly usage targets around AI, for example.

Others like Amazon and Meta as well as OpenAI itself, set up leaderboards to measure which developers are using the most AI tokens at any given time, essentially creating a competition within teams.

And for those who are topping those leaderboards, the reward can go beyond bragging rights.

Because in some firms token usage is reportedly being taken into consideration when discussing bonuses. So those that are using the most tokens, are essentially getting rewarded financially.

This has also created a sense of fear that those who are deemed to be using too few tokens could ultimately end up losing their jobs altogether.

Token Gesture

Exactly why companies are working so hard to encourage staff to use AI is not straight-forward.

If their AI-enthusiasm is taken at face value, then this is a revolutionary technology that is going to make their workers more productive and more cost-effective, while it will also lead to lots of new applications and systems that will benefit the company.

In that case, it would make sense to try to ensure your company is at the forefront of utilising this tech – which would require staff to familiarise themselves with it, and embedding it into their working routines as quickly and as comprehensively as possible.

To encourage that, and to nudge along the people who might be stuck in their old habits or a bit sceptical about trying something new, creating some kind of incentive system may make sense.

But, of course, there is more to it than that.

In some cases companies are trying to build up more data around how their people work, so they can get a better picture of what can be replaced by the AI itself. Others are, to a certain extent, just keeping up with the Joneses, other tech companies are going big on AI, and don’t want to be left behind.

But perhaps these token-drives should be seen as sales tactics as much as anything else.

Because a lot of the tech companies that have introduced minimum targets and leaderboards are the same companies that are trying to develop AI platforms of their own.

Hundreds of billions of dollars is being invested into AI this year alone, in the hope that it will cut costs and boost revenue, and one way firms hope it’ll boost revenue is because they’ll be able to sell their AI systems on to others who want to use it.

But it’s going to make it harder to convince investors that all of that AI spend is justified, and more importantly it’s going to be harder to convince customers that they should buy your AI platform, if your own staff aren’t using it.

And so pressure has come from the top down, with managers urged to get their teams to make better, or at least more, use of AI.

And that has led to the creation of these leaderboards and rewards systems.

Token Too Far

In the 1970s British economist Charles Goodhart coined ‘Goodhart’s Law’, which said that “when a measure becomes a target, it ceases to be a good measure.”

In other words, once you create a target, people will manipulate the system to hit it. And that’s exactly what’s happened here.

The blind pressure on people to use AI by any means necessary, and the fact that people are being rewarded based on what is a very vague metric, has been taken advantage of very quickly.

Many developers have realised that their managers didn’t actually care how they were using AI, or what results they were getting from it. They were just satisfied that they were using it, full stop.

And that created a perverse incentive for workers.

While the ideal would be to ask an AI platform a precise question in order to get the right result, suddenly it’s become more sensible to give a far more convoluted, complicated prompt – one that would require lots of back and forth in order to get the same result. Because that way, workers are using more tokens.

But even beyond that, it emerged that staff in these companies started to set up AI agents, in many cases, multiple agents, to run automated tasks for them through the day and night. Some created coding assistants that just churned out reams and reams of code. Others set up entire side-projects that had no real purpose other than putting as much pressure as possible on the AI platform so that it burned through more and more tokens.

And none of this was necessarily useful or productive to the person’s work or the company as a whole, all it was designed to do was use lots and lots of tokens.

But that didn’t matter, because if the only thing their boss was measuring them on was how many tokens they used, then they would look like they were doing a good job.

Maxxed Out

Of course this comes at a significant cost, not least from an infrastructural and environmental point of view.

How energy-hungry data centres are, and how much that demand is set to grow in the coming years due to AI, is likely to put upward pressure on fossil fuel emissions.

The pressure data centres are putting on energy systems, in Ireland, as well as in other countries, is also an ever-growing issue.

But, more importantly to the tech firms, token-maxxing has also quickly become a financial burden.

In recent weeks Amazon scrapped its internal token leaderboards, with a senior manager telling staff “don’t use AI for the sake of using AI.”

That is quite an about-face from the ‘let ‘er rip’ approach that had been taken in the weeks and months before.

And while that free-wheeling approach to token usage might have been based on the optimism that it would eventually lead to some kind of financial return, even that hope seems to be fading fast.

Ride sharing app Uber recent said it had burned through its annual token budget for the full year in just four months, and its chief operating officer has said it was hard to draw a line between its spend on AI and the benefits it was getting from it.

Part of the reason for that is because using these AI platforms is gradually getting more expensive.

Initially it was relatively cheap to use AI platforms, they often offered flat rate, per user fees, and had low costs to boost engagement. But now we’re seeing free and unlimited plans being scaled back in favour of ones built around usage.

That’s of course because energy costs are rising, at a time when the platforms are getting more sophisticated and, as a result, more energy intensive. Meanwhile the cost of hardware, like all of those Nvidia chips, is ticking up too.

Meanwhile major AI companies like OpenAI and Anthropic, which until now have been losing billions of dollars, are increasingly coming under pressure from investors to show that there is light at the end of the tunnel. That pressure is only going to grow as they open their books to would-be investors ahead of their stock market flotation.

As a result they’re charging users more.

That means that finance bosses in the companies using these platforms, rather than simply sit back and let people vibe code to their hearts content, are starting to ask questions about the spending, and what they’re getting in return.

The problem is that’s not an easy thing to figure out.

When Elon Musk took over Twitter – which became X – he quickly sought to cut costs, in part by sacking staff that he felt were underperforming. As part of that he briefly became obsessed with reviewing individual developer’s code to see what they were bringing to the table.

But, as experts pointed out, when you have a hundreds or thousands of developers contributing to a piece of software – software that has been built upon for years and years – it’s very hard to pick out what one person has added to the pot.

These things tend to be highly collaborative, and identifying what any one person has done is far from easy. Just because one person has pumped out lots of code, for example, doesn’t mean it’s good, or useful.

In fact having someone who can create lots of code with something shorter and cleaner could be far more attractive. But that’s not always easy to measure.

That’s part of the reason why tech bosses are often so keen to fall back on crude metrics to measure performance.

Float on

So token-maxxing has created a headache for some tech firms, but it could end up a problem for the major AI players too.

The likes of OpenAI and Anthropic have been working hard to drum up business, and they’ll brag about how much compute they have, and how much demand there is on their platform.

A lot of that demand is coming from big tech companies that have been, in turn, encouraging their staff to go all in on AI. The race between firms to embed AI into their business as quickly and as comprehensively as possibly has no doubt helped the actual AI companies’ business metrics.

But if those same users are now suddenly being told to be much more cautious and considered in how they are using AI, and being much more critical in what kind of return they’re getting on the cost they’re putting into all these tokens, that could at the very least slow the growth of these companies.

That’s not to suggest that these firms might be abandoning AI, far from it. But now the focus of some is shifting to what people are calling ‘efficiency-maxxing’- where users seek out ways to get as much out of a platform for as little token expenditure as possible.

And all of this is coming at a really important time for the big AI platforms.

Both Anthrophic and OpenAI recently filed paperwork for what will probably be trillion-dollar-plus stock market flotations in the coming months.

As part of that process, they’ll be opening their books to investors and trying to convince them that their revenue is going to continue to grow rapidly, and that profitability is just around the corner.

Which wouldn’t be a good time to be recording a slowdown, or even dip, in user activity.

Source link