Have Chinese AI Models Caught Up to the US Frontier?

On July 16th, 2026, the Chinese start-up MoonshotAI announced Kimi-K3, their new 2.8T parameter open-weight flagship.

Kimi-K3 is by far the largest open-weight model, at almost 2x the size of DeepSeek-V4 and the first open-weight model that is larger than the original 1.8T parameter GPT-4 that started the AI frenzy.

The size and performance of this new model have once again sparked a debate about how far Chinese models are actually behind American frontier models.

---

Before diving into the US - China gap, I first want to paint you a picture of where Kimi-K3 stands after 1 day of benchmarking.

On benchmarks MoonshotAI itself published, Kimi-K3 outperforms Opus 4.8 on 30 of 35, GPT-5.6-Sol on 19 of 35 and Claude Fable 5 in 12 of 35.

In third-party evaluations like Artificial Analysis’ Intelligence Index it is the third highest rated model after Fable 5 and GPT-5.6-Sol and is a noticeable jump over previous Chinese flagship models such as GLM-5.2, Kimi-K2.6, Qwen3.7-Max and DeepSeek-V4-Pro.

Other benchmarks such as the WebDev Arena, where models have to create appealing web pages, show Kimi-K3 dominating all other models including Fable.

There is one last domain I want to highlight where Kimi-K3 is at the frontier.

One example is GPU kernel optimization, as measured by KernelBench. To run models fast and efficiently you need to utilize the hardware as best as you can, as every clock cycle you waste means fewer tokens per second and higher latency, which overall translates to lower GPU throughput and therefore lower revenue.

This is a high impact area that is not only relevant in production environments, but also in the earlier research phase. Each new architectural idea has to be validated by running a series of experiments. Running them in plain PyTorch does not utilize the hardware effectively, which is where optimized kernels come in to speed things up.
A model that can help researchers write kernels for these experiments is valuable, as it saves time and money and Kimi-K3 is together with Fable 5 at the frontier.

For reference:

• GPT-5.6-Sol was launched 7 days before Kimi-K3

• Fable 5 was launched 37 days before Kimi-K3

• Opus 4.8 was launched 49 days before Kimi-K3

Taken at face value, these benchmark results seem to suggest that Kimi-K3 has caught up to the American frontier.

So, the case is closed, right?

Not quite.

For that we have to take a deeper look at the benchmarks, and define what we actually want to measure and why we even care.

---

Why the US-China AI Gap Matters

A lot of people have already written many great articles and scenarios about this.

But the short version is that AI is exponentially getting better at software engineering, where we have measured the effect most extensively, but this is very likely also true for other domains like for example mathematics.

Specifically, for software engineering AI models are doubling their time-horizons approximately every 4 months. Time-horizons measure the amount of time a human needs to complete a task that this model can solve with a 50% probability. Think of it as a proxy for task difficulty. Tasks that take humans more time are more difficult.

Besides the obvious reason that ranking programming tasks by difficulty is itself very difficult, as lines of code or other surface level metrics are not good proxies, using time-horizons also has the benefit of being economically interpretable.

If we project AI staying on this trend for just 2 or 3 more years, we end up with AI that can do work that would take humans months to years.

AI Futures Model

Even if this trend was unique to coding, having superhuman coders is obviously a gigantic strategic advantage for whoever controls them.

So we care about this gap, the time it takes for one country to develop the same capabilities as another, because it changes who controls the future.

We focus on China and the US, because empirically this is where the best models come from, where most AI research happens and where most AI compute is located. Europe could theoretically be in the debate, as they do have the money and talent, but have so far not shown any meaningful interest in competing seriously.

Until now it has been very clear that the US has been ahead of China.
But as we have seen with Kimi-K3 benchmarks, this is no longer as clear as it used to be.

---

How Do We Measure the Gap?

The gap describes the amount of time it takes one lab to reach the same capabilities as another lab.

This definition of the gap is treacherous as it hides several things that are important to distinguish:

• Capabilities are domain dependent. A model can catch up to another model’s coding capabilities, while still being worse in specific other domains or in aggregate.
  

• There is a forward and a backward-looking gap.
  The forward-looking gap asks: “How long will it take the laggard to reach the current SOTA?”
  The backward-looking gap asks: “How long ago did the frontier reach the same performance that the laggard currently has?”
  Typically the gap refers to the backward-looking one, as the forward-looking one is a forecasting exercise and has higher uncertainty, as it depends on assumptions you make about compute, talent, the speedup new AI gives you, and many other things and the growth rates of these things.

  

One problem that occurs when trying to estimate these gaps is that observable progress is discrete as models are only launched every few weeks to several months, depending on the lab. Progress does not happen along smooth curves as shown in the example image above, and models are unlikely to have the exact same scores such that you could simply measure the time between them. You have 3 ways of addressing that via the definition of the gap. You either over- or underestimate the time, by taking the next closest model on the frontier above or below the laggard’s capabilities or you fit smooth regression lines to your data and compare them instead.

Personally, I think the last option is best, as it is symmetric, better reflects the larger underlying trend and also allows you to measure how fast the gap is shrinking or growing. Downsides of this approach are that it depends on the choice of your regression model and it masks breaks in trend in either direction. So this could disadvantage Kimi-K3. But so far we have arguably only seen one, maybe two, changes in trend over the past ~5 years. The first one happened with reasoning models in Q3 2024 and was unrelated to model size. The second more arguable one happened with the release of Opus 4.5 in November 2025, when agentic models started to take off. This was also unrelated to model size, so I would find it surprising if Kimi-K3, being a bit less than 2x larger than DeepSeek-V4-Pro, would suddenly break any existing trends due to its size.

So to be fair I will report two figures, an interval that measures the current, instantaneous gap, which consists of the underestimate and overestimate, and a more general trend estimate based on the fitted regression lines.

Now we will shortly dive into benchmarks and then look at some trends and the gap.

---

Why Benchmark Choice Changes the Answer

MoonshotAI reported 35 benchmarks, amongst them 8 coding benchmarks, 12 agentic benchmarks, 3 “reasoning” and knowledge benchmarks and 12 Vision benchmarks.

What I notice when looking at these benchmarks and their composition is that:

• there are no long-context benchmarks

• there are no cyber, bio, chemistry or any safety-relevant benchmarks

• there are no serious mathematics benchmarks, only MathVision, GPQA-D and HLE

• there are no pure reasoning benchmarks. They only put GPQA-D and HLE under the “reasoning” and knowledge category, which is really just a knowledge category with some mathematics

• vision and agentic benchmarks dominate and make up 24 of the 35 benchmarks

• the coding benchmarks do include some benchmarks, which I consider high signal because of their scope and longer time-horizons, like ProgramBench and SWE-Marathon

• there are no benchmarks quantifying reasoning/token-efficiency

These results therefore cannot support the conclusion that “Kimi-K3 is overall better than Opus 4.8 and GPT-5.6-Sol”. In no case can you make the argument that Kimi-K3 is better than Fable 5, as it loses to Fable in the majority of them.

However, we can say that Kimi-K3 coding capabilities are comparable to current US frontier models.

---

Now let’s look at the other results I shared, like the Artificial Analysis Index (from now on short AAI).
Today it is one of the most popular indexes, but it has similar problems and I don’t think it should be used for assessing general model strength. This has several reasons:

• First, the benchmarks’ composition is heavily skewed towards agents and short time-horizon coding problems, which make up 58% of the index’s weighting.

  (we shouldn’t manually select weights or benchmarks)

Intelligence Benchmarking | Artificial Analysis

• Second, the difficulty and information gain of each benchmark is completely disregarded.
  

That doesn’t make the AAI useless. It still produces good rankings that correlate well with user experience, as it focuses on economically useful tasks.

So, I would not describe it as a benchmark for general capability or intelligence, but everyday usefulness.

This is also why I call it the “Artificial Analysis Index” and not by its official name of “Artificial Analysis Intelligence Index”.

---

This is where EpochAI’s Capability Index, short ECI, comes in.
It addresses these issues and tries to measure the overall latent general capability and unifies it on one scale using Item Response Theory.

In Item Response Theory the probability of answering a question correctly depends on the test-taker’s capability and the difficulty of the question.

ECI applies this to benchmarking. It assumes that we only need to know the latent capability of the model and the benchmark’s difficulty and slope to predict what the model would score.

For each benchmark two parameters are estimated:

• a difficulty parameter, that tells you the capability needed to achieve a 50% score

• and a slope parameter, that tells you how sensitive the benchmark scores are to changes in capability or how discriminative this benchmark is for models of similar capability

At the end a single statistical model is fitted to all of these benchmarks and optimizes the latent capability of the model, such that the latent capability most accurately predicts all of the scores observed.

Unlike the AAI, ECI reduces reliance on manually selected benchmark weights, does not require all benchmarks to be present for all models and most importantly, it accounts for the difficulty and discriminative power of each benchmark.

Anthropic recently also adopted this way of measuring model strength.

Their plot also nicely shows how difficult different benchmarks are such as GPQA Diamond, SWE-Bench-Verified and so on.
For example, the bar for GPQA-Diamond on the far left tells us that a model with an Anthropic ECI of slightly below 130 would score 50% on GPQA-Diamond.
So given that scale and this single latent capability (the Anthropic ECI score) of Claude 3 Opus we can predict that it should score slightly below 50% on GPQA-D.
This is also roughly what we observe. Opus 3 scores slightly above expectation (50.4%).

AECI from the Mythos Preview System Card

That is the beauty of ECI. It lets us infer latent capability on a single shared scale, while being less biased than manually selected and weighted indexes.

---

The Gap According to Artificial Analysis

Let’s get into the fun part of measuring the gap, after clarifying common traps when talking about these things.

Our starting point is the Artificial Analysis Index, as it does provide us with a score for Kimi-K3 and a detailed history of other scores.

I use a sigmoid, because the index is bounded between 0 and 100.

Our adjacent-model lag bracket for the backward-looking gap is 1.22 - 1.61 months, with an interpolated middle estimate of 1.48 months.

The larger trend indicates that the gap is around 2.89 months. Future Chinese model releases will tell us whether Kimi-K3 is a genuine break in trend or an outlier.

Also based on these projections Chinese models should surpass American ones on this Index in late 2027 if dynamics stay the same.

But just looking at the current frontier of both countries hides the trends of individual labs.

So here are the trends of the individual labs for the US (Anthropic, Google, OpenAI) and China (Alibaba, DeepSeek, MoonshotAI, Z AI)

• OpenAI and Anthropic basically the same trajectory

• Google is already behind Chinese models, and their trend doesn’t look good either

• out of the Chinese frontier labs, Moonshot has the steepest trajectory and should overtake Anthropic in February of 2027, given that the trends hold

I already made the case for why we shouldn’t look at AAI, and why the ECI is better.

However, we don’t have an ECI score for Kimi-K3.

---

Estimating Kimi-K3’s ECI

There are several estimates for Kimi-K3’s ECI:

• my community thinks it will fall between 158-159

Community Survey

• Personally, I would have guessed 156-157.

• Teortaxes has guessed 157-158

But we don’t have to rely on intuition and I don’t want to rely on my estimate to pre-empt people calling me biased.

Luckily, Artificial Analysis Index and ECI are highly correlated, which lets us make a more informed guess.

But there’s a problem. As you can see the linear model first overestimates very weak Claude-2 /3 models, then underestimates from around x=[10-30] and then overestimates stronger models again.

The linear model doesn’t quite make sense.
This is because Artificial Analysis Index scores are bounded between 0 and 100 and are better modeled by a sigmoid. ECI, however, is linear and unbounded.

To match these two models, we apply the logit-transform (the inverse of the sigmoid operation) to decompress the AAI scores.

The newly fitted model has the following form, with which we can now transform AAI scores into ECI scores.

This is what it looks like:

It looks much cleaner, and our errors confirm this. The logit-linear model reduces average absolute errors by 37.2%.

This model gives us an estimated ECI score for Kimi-K3 of 158.33 with bootstrapped 80% CIs at [154.49, 162.02].

---

While writing this we actually got some preliminary results for Kimi-K3 by EpochAI.

Kimi-K3 is not competitive at FrontierMath, and is still behind GPT-5.2-Pro, a 7 month old model.

FrontierMath T4 (V2) by EpochAI

With these new results from Epoch, our whole estimate was a bit pointless, as we can just use the benchmarks provided by EpochAI and Moonshot, namely:
AIME 2024-2025, GPQA-D, FrontierMath 1-3 / 4, SimpleQA Verified, Chess Puzzles, PostTrainBench, APEX-Agents and HLE to compute the ECI directly.

Still, this derived formula for converting between AAI and ECI will come in handy in the future.

Computing the preliminary ECI gives us a score of 155.53 with 90% CIs at [153.87, 158.21]. For further analysis we will use these scores.

This is within our correlation-based confidence interval, though a bit lower than expected. Notice that these scores can still change as more benchmarks come in, but this estimate should be more accurate than the correlation-based one.

---

Estimating Mythos Preview’s ECI

First, we need to address the elephant in the room.
We don’t have an ECI value for Mythos Preview, which was launched on April 7, 2026 as part of the Glasswing Project.

The exact date of Mythos Preview’s completion is unclear, but we know from the system card that it was available to Anthropic employees since February 24, 2026.
However, it is not certain whether this was the same Mythos Preview as in the system card and we don’t know when all the other models were available internally, so we will go with the official April 7th date for the release date of this model.

Anthropic also only provided us with their own version of the ECI, the AECI.
But you can convert it, for example:

• How good is Mythos? - Point Estimate: 161.5 using ECI methodology

• Ramez Naam on X: 161 using OLS

Our derived ECI for Mythos Preview is 161, with 90% CIs from 158 to 166.
This is very close to the actual Fable 5 ECI of 160 (158-165).
I think this is fair, considering that Fable 5 is likely a smaller distilled model from Mythos Preview with additional safety training. Both would lower its scores slightly.

---

The US-China Gap According to ECI

First, let us look at the overall frontier trend between Chinese and American AI models.

Based on our “gap methodology” Mythos Preview does not affect the empirical backward-looking gap estimate, because Kimi-K3 is simply nowhere close to it.

Kimi-K3 is currently between GPT-5.3-Codex and GPT-5.4-Pro, which produces an adjacent-model lag bracket of 4.37–5.29 months, with a linearly-interpolated central estimate of 5.27 months, since Kimi-K3 is much closer to GPT-5.3-Codex than it is to GPT-5.4.

Based on our fitted trendlines for frontier models, the backward-looking gap is currently at 6.08 months and there is no crossover or catch-up scenario for Chinese models.

Furthermore, the trendlines would predict that Chinese models will catch up to Mythos Preview’s score of 161 by March 7, 2027, which implies a forward-looking gap of 11 months. Holding the assumed Mythos score and release date fixed, the pointwise 90% confidence band around the Chinese trend reaches ECI 161 between December 28, 2026 and June 18, 2027, or 8.72 to 14.38 months when expressed as lags.

---

Looking only at the frontier trendlines by country ignores potentially steeper trends by individual labs, so we also look at those.

Anthropic is currently on the steepest trendline, increasing its score by 25 ECI points per year, followed by Z-AI at 16.2, OpenAI at 14.3 and Moonshot at 13.8.

Even without Mythos Preview, Anthropic’s rate of improvement is still fastest at 18.1 ECI/year.

Notably, in this scenario Chinese models would also not catch up to American ones.

These are the forward-looking gaps to reach Mythos Preview’s ECI score for Moonshot and Z-AI:

• Moonshot: December 23, 2026; central gap 8.57 months.
  90% crossing interval: November 7, 2026 to February 22, 2027

  or 7.05–10.57 months after Mythos Preview.
  

• Z AI: December 31, 2026; central gap 8.83 months.
  90% crossing interval: October 24, 2026 to April 21, 2027

  or 6.60–12.48 months after Mythos Preview.

---

Conclusion

To summarize, based on the ECI, the backward-looking gap between Kimi-K3 (the current Chinese frontier) and the US frontier appears to be 4.37–5.29 months.

The forward-looking gap, which estimates when Chinese models will first reach a Mythos-Preview-equivalent model, is estimated to be between 6.6 months and 12.48 months if the steep but uncertain Z-AI trend holds. However, the central estimates for the forward-looking gaps are 8.57 and 8.83 months respectively.

In short, Chinese models have not caught up to the American frontier models nor are they projected to catch up to it.

---

I hope you enjoyed the article.
If you did, please like, share and follow me on Substack and Twitter for more :)

---

Some more thoughts on distillation, and on reasons for the gap and apparent catch-up:

• Kimi-K3 is a 2.8T parameter model and does beat Opus 4.8 and GPT-5.6-Sol in some highly specific benchmarks. However, these benchmarks do not consider model size and reasoning/token-efficiency.
  
  From the Cursor CEO Michael Truell we know that Opus and GPT models are similar in size to Cursor’s newly announced 1.5T model. That statement likely refers to Opus 4.8 and GPT-5.4. GPT-5.5 and GPT-5.6 seem to be larger models that are estimated closer to 3T parameters.
  
  That means Kimi-K3 beating Opus 4.8 in some benchmarks would not be impressive from an architectural standpoint. Generally, it also still has worse reasoning efficiency than the US models.
  
  This is one of many reasons why I think both the forward- and backward-looking ECI gaps are underestimated, as model size, reasoning-efficiency, true model serving cost and overall compute are not factored in (because most of these numbers are not actually known publicly).
  
  Imagine the US and China were actually racing or in an all-out-war. Then it does make a difference whether you have a million of the latest Blackwell GPUs or only older Hopper GPUs with lower throughput, because memory/compute matters for the number of instances you can deploy and the speed at which you can serve them.
  
  Another reason is much more thorough safety testing by US labs compared to Chinese labs, which could add another month on top.
  
  The last reason I want to mention for why the gap is likely understated is that most benchmarks are not representative of real-world tasks and do not try to elicit the maximum possible performance, due to cost- and time-constraints.
  

• Mythos is rumored to be a 10T parameter model. We also know from multiple American labs that they are working on their own giant 10T models, like SpaceX AI, OpenAI and Meta. It is entirely plausible that Chinese models have now caught up to the previous generation of models, however US labs have already moved on to the next generation of models.
  
  I think what we are most likely seeing with the apparent catch-up of Kimi-K3 is that frontier labs like OpenAI and Anthropic are holding back their most capable models, because of the uncertain legal situation for frontier model releases, due to their cyber and CBRN capabilities.
  
  Furthermore, in the counterfactual world where Anthropic and OpenAI released their new frontier models Fable 5.1 and GPT-6, which presumably finished training, we wouldn’t be talking about any type of catch-up, because it would be apparent that these new 10T parameter models are entirely different beasts.

• We didn’t look at reasons for why Chinese models have made such rapid progress in the recent months especially on the Artificial Analysis Index, where Chinese models supposedly overtake American frontier models by early 2027.
  
  One possible reason for the rapid catch-up is distillation. Technically, you need access to the models to do real distillation, but using a stronger model as a judge for a weaker model’s output can already count as such.
  
  Anthropic has reported multiple times that they were able to trace distillation attacks back to Chinese labs. So distillation is something real, but we don’t know the exact extent of it and how much it helps Chinese labs improve their models.
  
  At least some non-zero portion of the capabilities gained by Chinese models is attributable to distillation.
  
  I tried to come up with an approximation for when these distillation attacks began on a larger scale. I wasn’t sure how we could even measure this. But one indirect approach could look at the derivative with respect to time of certain indexes like the Artificial Analysis, to see when progress accelerated unnaturally.
  
  Since August to October 2025 the measured progress for Chinese labs has been faster than for American ones, despite American labs having more compute, data, talent, and better models that should speed up the development of new models.

  

  I don’t think this is a particularly good estimate, but it’s still an interesting observation. So why does it happen? Shouldn’t the curves for Chinese labs simply be shifted in time? Why do they reach higher peak velocities?
  
  One explanation could simply be the benchmark being bounded and US labs starting with higher scores where progress is harder.
  So I tried applying the logit-transformation to remove some of the compression that makes progress appear slower. But the logit transformation doesn’t change the story. Moonshot and Z-AI still have higher peak velocities.
  
  So other possible explanations for any kind of catch up are:

  • catching up is simply easier than developing new frontier capabilities, as the general direction for improvement is provided by the frontier

  • the capabilities that AAI tests are publicly very visible and amenable to RL hillclimbing, therefore making them targets for hillclimbing

  • distillation from frontier models

  • some selection bias, as we are only looking at the two fastest growing Chinese labs, whose risky bets have worked out

  • maybe just a lucky window of observations, as we don’t include a whole lot of models

  
  The catch-up story also seems to apply only to AAI, but not the ECI.
  Personally, it’s a combination of all of them.
  
  I think it’s just specific domains and tasks that show any kind of catch-up.
  The reason why the catch-up happens is still interesting as it would tell us more about the future trajectory of capabilities.

• While not proving that distillation was the reason for this, recent UK AISI results and ECI results at least point in the direction of distillation and narrow task hillclimbing.
  
  Specifically, when looking at UK-AISI’s cyber ranges GLM-5.2 is only on par with Opus 4.5, despite being close to Opus 4.8 on many of the self-reported coding benchmarks.
  
  UK AISI’s cyber ranges are also probably the best benchmarks we have, as these are very hard tasks, where models use 100M tokens and are pushed to their limits.
  MirrorCode does this to an even more extreme extent using up to 1 billion tokens per model.

  

• One more odd thing is, that we haven’t seen any scientific breakthroughs by Chinese models. If they really were as good as frontier US models, wouldn’t they also solve previously unsolved mathematics, physics, and other problems?

• We should evaluate Chinese models on cyber and CBRN tasks, because these are exactly the domains where frontier labs sandbag and employ safety filters, meaning that you couldn’t distill these capabilities without having access to universal jailbreaks.
  
  Safety testing and training aren’t really conducted for Chinese models, at least not to the extent they are for US models.

  

  Risk Index from the CAIS Dashboard

  If Chinese models were distilling on a large scale, then we should observe that these models are also weaker in cyber and CBRN tasks, given that they don’t do as much safety testing and training.
  
  At least the most recent GLM-5.2 results on the UK AISI cyber range seem to suggest exactly this, with some weaker evidence from ExploitBench and ExploitGym.