Last week, a report from Reuters revealed that China’s People's Liberation Army has used Meta's large language model, Llama, to develop AI tools with potential military applications:
In a June paper reviewed by Reuters, six Chinese researchers from three institutions, including two under the People's Liberation Army's (PLA) leading research body, the Academy of Military Science (AMS), detailed how they had used an early version of Meta's Llama as a base for what it calls "ChatBIT".
The researchers used an earlier Llama 13B large language model (LLM) from Meta, incorporating their own parameters to construct a military-focused AI tool to gather and process intelligence, and offer accurate and reliable information for operational decision-making.
The revelation comes amid a growing U.S. debate over how or whether to regulate powerful AI systems in the context of technological competition with China. As I and many others have argued, poorly constructed regulation runs the risk of stifling innovation and ceding U.S. leadership in AI to our adversaries. On the other hand, America has no hope of maintaining its lead if the Chinese military can simply steal our best AI models with impunity.
Meta’s license prohibits the use of their Llama models for military applications, but to their credit, they recently made an exception for the U.S. military and defense contractors. The U.S. military is thus gaining permission to use Meta’s models at least five months after the People’s Liberation Army already had a Llama-derived model deployed. Better late than never.
The good news is that the model in question, Llama 13B, is not particularly powerful. Nor is Meta’s latest model generation, Llama3.1 405B, that far beyond what can likely be produced by China’s domestic AI ecosystem. The Chinese-made DeepSeek-V2.5, for example, already rivals Llama3 70B on most available benchmarks.
This suggests that China has already largely caught up to the GPT-4 generation of large language models, making their use of Meta’s Llama model not particularly concerning. What is worrying, however, is that Meta has committed to openly releasing the weights of bigger and better AI models indefinitely, up to and including AGI-level systems with powerful dual-use capabilities. This would be a grave mistake. Making genuinely dual-use models available for anyone to download and modify would jeopardize U.S. national security and let China leap-frog to the state of the art overnight.
GPT-4 is two-year-old technology. Building the next generation of models requires scaling-up data and computing resources substantially, and thus access to thousands of the latest AI datacenter chips. The time it takes to design, fabricate and network those chips together creates a roughly two-year lag between each significant increment in model scale. This is why frontier AI companies like OpenAI are only now planning to announce their next model generation, while much of the progress over the last year has focused on making existing models smaller and more efficient.
So while China’s best AI models may be close to parity with the best U.S. models today, the gap is set to widen overtime. Given the 2023 semiconductor export controls, China has been forced to make do with older chips and repurposed technology, along with any chips they obtain illegally through smuggling. The controls are designed to increasingly bind in the years ahead as leading chip designers like Nvidia continue to improve the performance of their chips at an exponential rate. Provided existing gaps in export control enforcement can be closed, the U.S.’s lead in AI hardware will ultimately translate into a formidable lead in the capabilities of our best AI models.
The next major scale-up in AI is expected to enable powerful dual-use cyber and bio capabilities in particular. OpenAI’s new o1 reasoning model was the first to reach a “medium” CBRN risk level in their internal testing. Frontier AI labs like OpenAI and Anthropic are thus investing heavily in AI safety and cybersecurity to prevent the misuse or theft of their future models.
Complementing these efforts, Congress is also considering passing the ENFORCE Act to authorize export controls on AI models that substantially lower the barrier to entry for developing weapons of mass destruction. Meta is actively lobbying against the ENFORCE Act on the basis that it may be used to prevent their future model releases. In so doing, Meta is tacitly acknowledging that they intend to openly release models with dual-use capabilities in the years ahead, knowing full well that they will be immediately harnessed by the Chinese military.
Mark Zuckerberg has defended Meta’s position by appealing to the safety of “open source” software like Unix and Linux, but this is misleading. Open source software is often considered safer because anyone who finds a vulnerability in the code can propose a fix, leading to airtight code after many years of stress testing. In contrast, large language models are giant arrays of inscrutable numbers that rapidly depreciate as new and better models are trained overtime.
Nor are Meta’s models technically open source to begin with. According to the Open Source Initiative’s official definition of “open” artificial intelligence, an AI model is only truly open source if its publisher includes:
- Sufficient information about the data used to train the model for a skilled person to build a substantially equivalent system. This means “the complete description of all data used for training,” including “the provenance of the data, its scope and characteristics, how the data was obtained and selected, the labeling procedures, and data processing and filtering methodologies.”
- The complete source code used to train and run the system.
- The model parameters, including both the weights and other configuration settings.
Meta has openly released Llama’s model weights, but the datasets and protocols used to train the model have not been disclosed. This is a problem, as open source AI models can only be said to be safer to the extent that their code and training data can be audited for backdoors, whether publicly or by an independent third party.
To date, open source AI has been a boon for research and commercialization. The Llama series of models have allowed independent researchers’ to contribute to open problems in AI alignment and interpretability in particular. But as models become much more capable, at some point the costs associated with open model weights will begin to exceed the benefits. In lieu of either regulation or progress in making open models impossible to jailbreak, the world is counting on the leading AI companies to exercise reasonable care and discretion.
That is all the more reason to avoid conflating the open source movement with the motives of a single Big Tech company. Indeed, as the owner of one of the largest AI training clusters in the world, Meta has the unique capacity to train and release genuinely dual-use systems – a capacity they must commit to use responsibly, if not patriotically, going forward.
