Before asking generative AI ‘Is it a buy?’: How to distinguish between machine learning traders and ChatGPT traders

Recently, more people are using generative AI for investing.
These are commonly known as “ChatGPT traders.”

Even on social media, I often see posts like
“I tried having ChatGPT predict stock prices” or
“I bought a stock that the AI said to buy.”

I think using generative AI for investing is an interesting experiment in itself.

However, from my perspective as someone who has worked in market operations at banks and is currently developing automated trading using machine learning,
there is something that concerns me a little.

That is the fact that asking generative AI for market outlooks and trading with machine learning are being treated as the same thing
.

Both are certainly “investing using AI.”

But what they are actually doing is quite different.

In this article, I will introduce those differences,
while also incorporating research examples
from hedge funds and the financial sector.

Asking ChatGPT, “Will the Nikkei 225 go up tomorrow?”

For example, suppose you ask ChatGPT this question.

“Do you think the Nikkei 225 will go up tomorrow?”

Then, based on recent news, financial policies, exchange rates,
and the movements of U.S. stocks,
it might return a plausible explanation.

I think that explanation can sometimes be helpful.

However, what I want to know is
not just whether that prediction is correct.

For example, if you repeated the same question over the past 1,000 business days,
how often would it be correct?

If the win rate was 55%,
would there be any profit left after deducting transaction costs?

If large losses continued, how much would the capital decrease?

And, will the prediction method
that was effective until yesterday continue to work in the future?

Unless you know these things,
it is insufficient as a basis for making decisions to trade with actual money.

Providing a plausible explanation and
having a trading rule that leads to profit are two different things
.

Machine learning creates predictive models using past results.

So, what is different about trading using machine learning?

In machine learning, you first decide what you want to predict.

For example, ‘whether tomorrow’s closing price will be higher than today’s’.

Then, you prepare data such as past prices, trading volume, and
movements in other markets, and
have it learn how the market actually moved afterward.

Once the model is built, you verify the prediction accuracy with
data that was not used for training.

This is the important part.

It is not very difficult to find plausible-looking patterns
from past data.

What is truly difficult is verifying whether those patterns hold true
even in markets you have never seen before.

This is also the reason why I use machine learning.

I want to verify with data rather than assuming my own market intuition is correct.

In fact, global hedge funds also use machine learning.

Such quantitative investment methods are not just for individual investors.

The U.S. investment firm Two Sigma is
known for utilizing data science and machine learning
in its investment process.

Their official website introduces the concept of creating predictions from various data
and building portfolios while also considering transaction costs and risks.

The UK’s Man AHL is also a company that incorporates machine learning
into its investment research.

At the Oxford-Man Institute, which conducts
joint research with the University of Oxford,
research on machine learning and data analysis is carried out,
and the results are applied to investment strategies.

What is interesting is that they are not simply
‘having AI predict the market’.

They collect data, form hypotheses, verify them,
and incorporate them into operations while considering risks and costs.

The effectiveness of machine learning has also been verified in academic research

Using machine learning for investment is also a subject of academic research.

For example, the 2020 paper
by Gu, Kelly, and Xiu
‘Empirical Asset Pricing via Machine Learning’.

This study used data on approximately 30,000 US stocks over
about 60 years, from 1957 to 2016,
to compare return predictions using various machine learning methods.

The results showed that machine learning can potentially improve
prediction performance by utilizing non-linear relationships
that are difficult to capture with conventional statistical methods.

In particular, methods using neural networks and
regression trees have produced promising results.

However, what this research proved is not
‘if you use machine learning, you will definitely make money’.

Machine learning also has its strengths and weaknesses,
and it is pointed out that financial markets have little information
that can be used for prediction and are extremely noisy.

In other words, machine learning is not a panacea.

Even so, at least I think it can be said that
utilizing machine learning for market prediction
has a rationality backed by actual research.

Reference: Empirical Asset Pricing via Machine Learning (2020)

I wanted to predict the probability of take-profit and stop-loss rather than ‘will it go up or down’

From here on, I will talk about my own automated trading.

What I am building is a machine learning EA
targeting gold (XAUUSD).

However, I am not asking the AI
‘will gold go up or down from here?’

‘If I enter at this price now,
which is more likely to be reached first,
take-profit (TP) or stop-loss (SL)?’

For example, suppose you set the price range to take-profit at 100
and the price range to stop-loss at 50.

In similar situations in the past,
which was reached first?

I analyze those results using machine learning
and apply them to the current market to estimate the probability.

In addition to gold price data,
I also use price data for silver, crude oil, etc., as explanatory variables.

Gold does not move independently of other markets.

I thought that the movements of silver and crude oil might also contain information useful for making gold trading decisions.

Of course, increasing the number of explanatory variables does not necessarily improve accuracy.

It is necessary to verify whether they are actually useful.

However, as a result of incorporating those explanatory variables, the expected return improved dramatically.

By the way, the idea of creating training data from conditions such as profit-taking, stop-loss, and time-out is also being researched in the field of financial machine learning.

A representative example is the ‘Triple Barrier Method’ introduced by Marcos López de Prado.

This is a method of setting three boundaries—profit-taking, stop-loss, and time limit—and classifying trading results based on which one is reached first.

The question I became interested in, ‘which one is reached first, TP or SL,’ is also related to these financial machine learning concepts.

Reference: Research on the Triple Barrier Method (Financial Innovation, 2025)

And calculating the ‘expected value’ from the predicted probability

This is the part I think is particularly important.

In trading, profit is not determined solely by a high win rate.

For example, let’s consider the following trade.

If you can take profit, you gain twice the loss amount set as risk.

If you hit a stop-loss, you lose that risk amount.

Under these conditions, there is a possibility that you will remain profitable even if your win rate does not reach 50%.

Suppose a machine learning model estimates a 45% probability of reaching TP and a 55% probability of reaching SL.

If we set the profit margin to ‘+2R’, the loss margin to ‘-1R’, and the trading cost to ‘0.1R’,

This is how it works.

Under these conditions, the theoretical expected value is positive even with a 45% win rate.

What I want to emphasize here is not simply
whether you think you can win.

Is that trade worth the risk?

The goal is to judge this using numbers.

However, just because the model predicts 45% does not mean
it will actually be correct 45% of the time.

You need to verify whether the predicted probability itself is reliable
using data that was not used for training.

Also, you must consider cases where neither TP nor SL is reached during trading hours,
as well as actual spreads and slippage.

That is why verification after building a machine learning model is so important.

Simply ‘winning in backtesting’ is not enough.

In fact, the biggest thing to be careful about in trading using machine learning is
overfitting to past market conditions.

Changing conditions repeatedly using past data
and selecting the model with the best performance.

If you repeat this, you might end up with
a very impressive backtest.

However, there is no guarantee that those results will be replicated in the future.

In the financial sector, this problem is called ‘backtest overfitting’.

In Bailey et al.’s study, ‘The Probability of Backtest Overfitting,’
the overfitting that occurs when selecting excellent strategies from past performance
is treated as a problem.

I also believe that more than increasing model accuracy,
we need to consider how reliable it will be
in actual operation.

Will the performance be maintained even during periods not used for training?

Has future information been accidentally learned?

Will there be any profit left after including trading costs?

And can you stop it when losses exceed expectations?

Only after considering these aspects can you determine whether it is
usable as an automated trading system.

Reference: The Probability of Backtest Overfitting

So, is market prediction using generative AI meaningless?

That is not necessarily the case.

In fact, there is research showing that generative AI
has the potential to predict future stock price reactions
from financial news.

A study published in the ‘Journal of Financial Economics’ in 2026 reports that
GPT-4 can analyze the meaning of news headlines and has predictive power
regarding stock price reactions and subsequent price movements.

However, correctly predicting price movements immediately after news
is not the same as actually trading and making a profit.

That study also cites transaction costs as a significant factor
hindering actual monetization.

I personally think it is interesting to use generative AI
and machine learning in combination.

Generative AI is for understanding news and text.
What I find particularly interesting is the synergy between the quantification of non-quantitative data by generative AI and
machine learning.
For example, if you survey 10,000 people,
the survey administrator will not aggregate all the free-text responses.
It takes an enormous amount of effort.
Generative AI can summarize, categorize, and
quantify this in one go.

Using that data to make estimates with machine learning might be a very
interesting endeavor.

Reference: Can ChatGPT Forecast Stock Price Movements? (Journal of Financial Economics, 2026)

Individuals are now able to build such systems

In the past, acquiring multiple market data,
building machine learning models, conducting backtests,
and implementing them as automated trading programs
required a fair amount of programming knowledge.

I have also been involved in market operations at a bank
and have been exposed to data analysis and machine learning,
but I am not a programming expert.

That is where I used OpenAI’s Codex.

With Codex helping me with development,
I proceeded with gold market data acquisition, machine learning models,
expected value calculations, and implementation into automated trading.

What is important here is that
I did not leave the trading decisions entirely to generative AI.

What should it be made to predict?

Which data should be used?

How should the expected value be calculated?

Under what conditions should a trade be executed?

I leveraged the power of generative AI to design and implement these aspects.

Even if you cannot write code from scratch yourself, you can turn the mechanisms you conceive into actual working programs.

I believe this is a significant change for individual traders.

The reason I arrived at ‘Machine Learning x Automated Trading x Prop Firms’

Personally, I was unable to generate the profits I wanted through discretionary trading for a long time.

Even when I thought I was analyzing the market, emotions would creep into my actual trading.

I would sometimes make unnecessary trades in an attempt to recover losses.

Therefore, I decided to create a system that converts trading decisions into numbers as much as possible and executes them via a program.

Another issue was my own capital.

Even if I could create an EA with a positive expected value, the profit amount would be limited if the operating capital is small.

That is why I combined it with the prop firm Fintokei.

As a result, I received a total of 2,465,946 yen in payouts between November 2025 and February 2026.

Of course, it does not mean that anyone will get the same results just by creating a similar EA.

Even so, I feel there is potential in the method of using generative AI to build your own machine learning automated trading system and utilizing it at a prop firm.

Finally: Rather than asking AI for predictions, I want to build a system that can verify predictions

I do not mean to deny the act of asking generative AI, ‘Will the market go up tomorrow?’

It is an extremely convenient tool for organizing market news and finding perspectives that I had not noticed myself.

However, if you want to use that for actual trading,
I would like to take it one step further.

How accurate are the predictions?

How much do you make when you are right, and
how much do you lose when you are wrong?

Even after deducting transaction costs, will there be any profit left?

And, is there a possibility that the rules discovered in past market data
will also hold true in future markets?

I want to build a system that can answer these questions
with as much data as possible.

That is the reason why I use machine learning
for my trading.

Regarding the machine learning EA introduced this time,
I have also published the steps to actually build it using Codex in another note.

If you are interested, please take a look at that as well.