(More) philanthropy is needed for safe AI

The release of ChatGPT in November 2022 demonstrated the capabilities of modern artificial intelligence to a broad public. Its fields of application are diverse. They range from programming and the creation of advertising copy to the development of new medicines and military use. In short, AI promises the machine automation of human intelligence. If we succeed, the implications are immense. A world in which machines perform (almost) all cognitive tasks faster and better than humans, and at a fraction of the cost of human labor, represents a paradigm shift. In historical terms, this could only be compared to the industrial revolution. This is how some researchers that AI could lead to annual economic growth of 20 to 30 percent - i.e. the global economy would grow every two and a half years twofold or within 35 years thousandfold.

Such a radically different future promises more than just advantages. A few months after the publication of ChatGPT, the media became aware of a concern that AI researchers, a few philanthropic organizations and think tanks have been worrying about for almost a decade: existential risks posed by AI. An AI that has overtaken humans intellectually could be able to exert considerable influence on its environment and slip out of human control. One reason to assume that this could happen is the so-called instrumental convergence thesis: Intelligent systems will try to attract resources that are helpful for the fulfillment of their goals. This could lead to a competition for these resources with humanity, in which the latter loses out. Another risk is that a race between the superpowers USA and China for the most powerful AI could mutate into a new arms race that shakes up the global world order.

Society has sprung into action

These considerations have Yoshua Bengio and Geoffrey Hintontwo “Godfathers of AI”, whose research laid the foundations for this technology, Public to warn of the danger of humanity being wiped out by AI. Together with other researchers, entrepreneurs, politicians and philanthropists, such as Bill Gates, Audrey Tang, until recently Taiwan’s Minister for Digital Affairs, and OpenAI CEO Sam Altman, stated in May 2023 In May 2023, she stated that preventing an existential catastrophe caused by AI is a monumental societal challenge, comparable to preventing pandemics and nuclear war.

Other players have also responded to these new challenges. While an estimated 9 million US dollars from philanthropic funds went to research into safe AI in 2017, by 2023 this figure had risen to around 100 million US dollars. Governments have also recently taken an interest in the topic of AI security. The British government is leading the way here; last year, it established a “Frontier AI Task Force” to investigate the risks of AI, which was allocated 100 million pounds (and whose advisors include Yoshua Bengio, among others). In May 2023, the first “AI Safety Summit” took place near London, attended by Rishi Sunak and Kamala Harris, among others. Last but not least, the major AI companies also have their own research departments dedicated to AI safety.

So everything under control?

So can we sit back and relax? Unfortunately, it’s not quite that simple.

First of all, it is important to visualize the relationship between these sums and what is flowing into the further development of AI. The USD 100 million that was used from philanthropic funds to mitigate AI risks in 2023 is offset by an estimated 24 billion in contrast, which were invested solely in generative AI—approximately 240 times more.

If we look at AI companies in particular, it should be borne in mind that they have strong economic incentives to put safety concerns to one side in case of doubt. One sign of this was the Exodus of several researchers and developers from OpenAI, who had previously worked on the controllability of AI systems in its “Superalignment Team”. Here we encounter what political economists describe as “externalities”: OpenAI and other companies bear only a tiny fraction of the costs of AI risks but reap significant profits, similar to big oil companies that do not adequately price in the costs of climate change. Furthermore, AI companies focus solely on technical solutions, i.e. trying to build AI systems that act in the inventor’s interest. However, complete risk management also includes preparing for the eventuality that these efforts fail, e.g. by government regulation prohibiting the market entry of AI systems that are not sufficiently safe.

But nation states also have conflicting incentives here. AI could become a decisive advantage in economic and geopolitical competition. In 2023, the US government provided 140 million US dollars for the development of secure AI. At the same time, the Pentagon budgeted 1.8 billion US dollars for AI in the 2024 budget. Here, too, there are valid reasons to believe that, when it comes to safeguarding the national interest, the relevant decision-makers will prioritize maintaining a technological edge. It may be just an anecdote, but the fact that the third edition of the “AI Safety Summit,” which will take place in Paris in February 2025, has been renamed the “AI Action Summit” renamed to “AI Action Summit” and AI safety will play a lesser role there is hardly reassuring. At the same time, democratic states may lack incentives to take sufficient account of the risks posed by AI. Their greatest risks will presumably only come to light decades from now, far beyond the next elections. Finally, government bodies may simply lack the technical expertise to effectively regulate such a complex technology.

Philanthropic engagement needs more diversity

The majority (an estimated 80%) The funding for AI safety from philanthropic sources comes from the American foundation Open Philanthropy (which also supports Effective Giving) and the Survival and Flourishing Fund (SFF), which is supported by Skype founder Jaan Tallinn. This commitment is very important and encouraging. To name a few examples:

• Open Philanthropy has supported OpenMinded, a non-profit that builds software that facilitates the testing of AI systems for risks and is now being used by the UK’s AI Frontier Taskforce mentioned above.
• SFF and Open Philanthropy both have funded the Center for AI Safety funded the group responsible for the above-mentioned open letter on AI.
• Both have co-funded AI Impacts, an organization that conducts surveys among AI experts to find out how this community assesses risks posed by AI (with some worrying results).

However, as our research partner Founders Pledge explains in a new report, such a concentration on just a few foundations can also entail risks and drawbacks.

• Even if it is not to be expected that the organizations mentioned will withdraw from the field of AI, the topic is too important not to be more broadly positioned here.
• The foundations mentioned above have a strong focus on the U.S., partly for historical reasons and partly because the leading AI companies are based there. However, that does not mean there are no levers for action outside the U.S. For example, experts had strongly advocated for the regulation of so-called “foundational models”—such as GPT-4, on which ChatGPT is based—through the EU AI Regulation, which has just gone into effect, and ultimately succeeded in getting their way.
• Competition also stimulates business in philanthropy. If a few foundations set the tone, this can lead to possible solutions being overlooked or assumptions no longer being questioned. In the past, Open Philanthropy and SFF mainly funded technical solutions and neglected regulatory solutions for a long time (although both have now started to invest more in the latter).

Founders Pledge offers orientation

For these reasons, the new report from Founders Pledge is very welcome. In it, they outline the potential risks posed by AI in great detail and develop guiding principles that philanthropists can use as a reference when searching for promising organizations. One of the guiding principles mentioned, for example, is to support interventions that simultaneously address multiple threat scenarios when in doubt. Interventions that seek to foster cooperation among key decision-makers can achieve this, since in many of the scenarios where AI has catastrophic consequences, key actors have failed to coordinate—an arms race for the best AI, as mentioned above, would be one such scenario. One organization that the Founders Pledge itself recommends, based on its new report, is FAR AI. It recently organized a symposium at a Beijing university to bring together renowned Chinese and Western AI researchers. One result of this was a open letter, in which they call for red lines in AI development.

If you’d like to learn more, you can do so here here do. Founders Pledge has summarized its new report for us in this guest post.

What can I do?

Since November 2022, anyone who wants to help shape the future of AI safely can donate to our Giving Fund: Safeguarding the Future , the first donation fund in Germany dedicated to the topic of AI and other existential risks. For donors who would like to delve deeper into the topic and benefit from our network of experts, we are also available for a personal discussion.

About the author

Anne Schulze

Founder & Managing Director

Book a meeting

I am one of the founders and managing directors of Effektiv Spenden. If you have any questions or suggestions, please feel free to send me an email. I'll get right back to you.