Performance Metrics vs. Practical Application
Zhang Chi, a research scientist and assistant professor at Peking University who previously spent approximately one year developing large language models at ByteDance, argues that China’s artificial intelligence sector is significantly lagging behind its American counterpart. Speaking on the “Into Asia” podcast, he emphasized that the technological divide is not closing but expanding.
While domestic technology giants like ByteDance and Alibaba continue to release models that achieve high scores on standardized evaluations, Zhang contends these metrics do not translate to functional utility. He noted that many development teams prioritize “benchmaxxing,” a practice focused on maximizing test scores rather than enhancing real-world performance. As a result, despite impressive academic results, the systems fall short during actual deployment.
Development Cycles and Infrastructure Hurdles
A primary factor driving this disparity is the pace of innovation. Zhang highlighted that American firms like Google can complete both pre-training and post-training phases for large language models within a three-month timeframe. In contrast, his experience at ByteDance showed that a single iteration typically requires around six months. This slower development cycle is compounded by structural disadvantages, including restricted access to cutting-edge computing hardware, less robust infrastructure, and lower-quality datasets.
Additionally, Zhang pointed out that several domestic developers rely on distilling outputs from leading American models instead of cultivating independent data pipelines. He warned that this shortcut could hinder long-term innovation. He highlighted significant infrastructure disparities, noting that domestic teams struggle to access high-quality datasets.
The Feedback Loop Divide
Zhang explained that American AI products benefit from continuous improvement through direct user interaction. Systems like ChatGPT, Claude, and Gemini are constantly refined based on real-world usage patterns. Chinese models, however, face a detrimental cycle. Because they initially underperform, users avoid deploying them for critical tasks, which limits the feedback necessary for improvement.
He explained that early performance shortcomings discourage users from deploying them for critical tasks, creating a cycle that prevents improvement.
Diverging Industry Perspectives
Zhang’s assessment stands in stark contrast to other prominent industry figures who argue that China is rapidly closing the technological gap. Nvidia CEO Jensen Huang has cautioned that the United States could lose its competitive edge, while Elon Musk has suggested that China’s superior energy resources and computational capacity might enable it to surpass American rivals. AI pioneer Geoffrey Hinton has similarly noted that the American advantage may be smaller than commonly believed and could diminish over time. Meanwhile, Alibaba chairman Joe Tsai argues that the competition will ultimately be determined by the speed of AI implementation rather than raw model capabilities.
Despite these optimistic outlooks from global tech leaders, Zhang remains skeptical about domestic progress. He concluded that the technological divide remains substantial and doubted that domestic firms would close it in the near future.
