The Code of Practice: A Success for Safe AI?

What is the Code of Practice?

The General-Purpose AI Code of Practice of the EU AI Regulation (Code of Practice for short) has been in force since August 2, 2025. It defines guidelines for providers of general purpose AI (GPAI) models. The guidelines give providers a simplified, standardized way to prove that they are compliant with the legislation of the EU AI Act. If providers do not sign this Code of Practice, they are obliged to prove this themselves, which is significantly more complicated and costly. If providers cannot provide proof, they are not allowed to offer their AI system within the EU.

Relevance for Catastrophic Risks

We at Effektiv Spenden are particularly concerned about potential catastrophic risks from AI. In this regard, the sections of the Code of Practice concerning AI models with systemic risks are especially relevant. Models are defined as such above a certain size. They include OpenAI’s GPT models, which are behind ChatGPT, as well as the Grok models from Elon Musk’s xAI. 

According to the Code of Practices, the providers of such AI models have a number of obligations, which are regulated in the “Safety and Security” section of the Code of Practices. Firstly, they must identify potential risks posed by their model. These include:

• Chemical, biological, radiological, and nuclear risks. This is particularly aimed at preventing AI models from being used by terrorists to build dangerous weapons.
• Loss of control: The possibility that an AI system no longer follows user instructions and acts autonomously.
• Cyberattacks: The use of AI systems to carry out cyberattacks, especially on critical infrastructure.
• Manipulation: The risk that AI systems could be used for the widespread influencing of people.

In the next step, providers must analyze the extent to which their models enable these risks and, if in doubt, implement protective measures. One example of such analyses are model evaluations (evals for short), which test the capabilities of AI models. One of the first developers of evals was METR, which we have promoted in the past. On the basis of such evaluations, probabilities should then be derived that these abilities will lead to harm in the wild.

Why This Is a Success for Safe AI

A broad group of stakeholders from the AI industry, research, and civil society was involved in developing the Code of Practice. Among them were organizations that we have supported through our “Safeguarding the Future” and “Ensuring Safe AI” Giving Funds, such as SaferAI. Henry Papadatos, Managing Director of SaferAI, calls the Code of Practice a success. He argues this is the case simply because it regulates GPAI at all, something that was long controversial. Once AI models are already in circulation and in the hands of users, it can be too late to prevent risks. It is therefore important to control them at the beginning of the value chain.

Fortunately, most of the major AI companies have also adopted the Code of Practice signed, including OpenAI, Anthropic, Microsoft, Google, the French company Mistral and the German company Aleph Alpha. (As expected but not Meta, which generally stands for a lax approach to AI security - Mark Zuckerberg seems to have remained true to his motto “move fast and break things” here too).

The fact that companies are cooperating is also an important signal that the regulation of AI can work. In this sense, the AI regulation could be a blueprint for other legislators - the so-called “Brussels effect”. There are initial signs of this: For example, there are significant overlaps between AI regulation in the US state of Colorado and the EU AI Act.

One might fear that the companies are only signing the Code of Practice because it only contains things they would have done anyway. However, Henry Papadatos points out that by no means all companies currently comply with the guidelines of the Code of Practice. For example, Mistral does not yet have a framework for risk analysis, and most companies do not estimate the probabilities of risks at all. If they want to follow the Code of Practice, the affected companies must therefore improve their safety practices.

Ultimately, it is to be welcomed that the AI companies must confirm the risk analyses by external evaluators. This means that providers cannot simply give themselves a “pass” grade.

There Is Still More to Do

The Future Society (TFS) is another organization we have supported. They also call the Code of Practice a “significant achievement of the EU.” However, they point out that there is still room for improvement.

For example, AI companies are only required to share the results of their risk analyses after the deployment of a system in the EU with the authorities there. TFS is pushing for the Code of Practice to be adapted so that such a report should instead be sent to the authorities eight weeks in advance. In addition, no further provisions have been made for the protection of whistleblowers - despite the fact that there are already of known cases, where AI companies have put pressure on employees who wanted to raise concerns about unsafe practices. In particular, TFS would like the EU authorities themselves to act as a clear point of contact for whistleblowers and not leave their protection to the individual 27 EU member states. The authors of the Code of Practice themselves point out this gap and call on the EU authorities to close it.

We Need Independent Research and Civil Society

Even if the Code of Practice could have demanded more from the AI companies, it is, on the whole, a milestone success. It should be noted that stricter rules could have led to the companies not signing the Code of Practice at all. This would have further increased the pressure on Brussels to relax the EU AI Act. In recent weeks, there had been a series of lobbying efforts by the AI industry in this direction. For instance, a number of CEOs as well as the Trump administration had called on the EU Commission to suspend the entire regulation. This would have sent a signal to other legislators: Don’t even try to regulate AI, you will fail anyway.

A detail from the development process of the Code of Practice also shows the unequal relationship between the AI lobby on one side and independent research and civil society on the other. Under pressure from AI companies, the Commission postponed the publication of the Code of Practice originally planned for May, because they were not satisfied with the first version. This then led to some of the independent experts involved in the process having to drop out towards the end because they already had other commitments. This is something that would not happen to a company like OpenAI, which had an estimated budget of USD 8 billion in 2024. If an employee is needed elsewhere, they just send another one to Brussels. Independent research institutions and civil society organizations do not have such deep resources.

If you want to support the work of organizations like The Future Society and SaferAI, you can donate to our Giving Fund Ensuring Safe AI.

About the author

Anne Schulze

Founder & Managing Director

Book a meeting

I am one of the founders and managing directors of Effektiv Spenden. If you have any questions or suggestions, please feel free to send me an email. I'll get right back to you.