Over the past several weeks, a federal jury in Oakland witnessed a spectacular battle between ethics and a trillion-dollar industry. Yet it was a fight without a decisive winner: The jurors found, in less than two hours, that Elon Musk’s claims against OpenAI were barred on the grounds that the statute of limitations had passed. And the judge agreed and dismissed the case. Musk had alleged that the business he helped fund had abandoned its original mission of improving the world through AI by becoming a for-profit company. The decision’s anticlimactic outcome effectively rejected Musk’s extravagant claim that he was out to save humanity.
Lost in the procedural drama was the role of effective altruism, an intellectual movement that once shaped OpenAI’s culture. The movement began decades ago, when the Princeton philosopher Peter Singer posed a question that still unsettles people: If you could save a drowning child at trivial cost to yourself, you would be morally obligated to do so, so why should geographical distance change the equation? Why should a child dying far away matter less than one in front of you?
In practice, effective altruism’s line of questioning brought rigor to philanthropy by asking not whether a charitable act felt virtuous but whether it actually improved human welfare. To answer that question, effective altruists claimed that they could measure the value of human happiness and human life—present and future—and direct giving according to the results. Lives saved were reduced to costs per life or quality-adjusted life years, so that distant strangers and identifiable neighbors could be weighed on the same ledger and a bed-net program to prevent malaria in Malawi could be ranked against cancer research or pandemic preparedness. The strangeness isn’t that the math sometimes points to unexpected outcomes; it’s the underlying assumption that such measurements are possible and that these things can be measured against each other at all, as if human lives, animal welfare, and distant futures shared a common currency.
This is a core error of the basic tenet of effective altruism. Some interventions clearly do more good than others: Feeding the hungry is better than burning money, and to that extent, comparisons are useful. What is not possible is the further claim effective altruism makes: Serious altruistic options can be ranked with the precision the movement urges. Suppose the choice is between saving one hundred lives through one program and one life through another. The hundred is obviously the larger number. But what becomes of those hundred lives is not knowable. One of them may be a tyrant and the other a researcher who cures Alzheimer’s. The variables to make the calculation are not merely unknown but are unknowable. The answer to the impossibility of the calculation is to not claim moral superiority by choosing either over the other based on the calculation.
FUTURES IMAGINED
One branch of the effective altruism movement took this logic further. Devotees of longtermism—a moniker associated with the former Oxford philosophers Nick Bostrom and William MacAskill—argue that moral reasoning should account not only for people alive today but for everyone who might ever live. If humanity survives for millions of years, the number of conscious beings in the future becomes almost unimaginably large, and even a tiny reduction in existential risk could outweigh enormous present-day harms.
The math does not actually get you to “save the trillion-person future.” The variables required to weigh future against present, survival probabilities, welfare of hypothetical future minds, discount rates across millennia, are, again, not merely unknown but unknowable. Longtermism supplies alleged confidence in its outcomes that its calculations do not support. The failure is the same failure that imbues effective altruism more generally, scaled up by enough zeroes to dominate any spreadsheet that admits the variables in the first place.
The problem is not that these concerns are irrational. Existential risks are real. Advanced AI may eventually produce profound societal disruptions involving labor, war, surveillance, misinformation, biosecurity, or institutional stability.
The problem is epistemic. Nobody can confidently quantify the likelihood of AI-driven human extinction, the moral status of hypothetical digital minds, or the trajectory of civilization across millennia.
Nonetheless, this style of reasoning proves especially attractive in Silicon Valley, where many people work in domains where quantitative models genuinely do produce remarkable predictive power. A worldview has emerged there in which probabilistic calculations about civilization-scale outcomes can justify extraordinary intervention in a present-day institution, such as firing a CEO or designing corporate governance structures around speculative estimates of catastrophic risks.
Effective altruism’s drowning child becomes humanity’s future. The longtermists’ spreadsheet becomes civilization itself. These philosophical insights are important because they transform speculative future outcomes into overwhelming present-day moral imperatives, and the institutional consequences are substantial.
OpenAI is an organization whose formation was influenced by effective altruism. Former board member Helen Toner worked at Georgetown’s Center for Security and Emerging Technology, which received support from Coefficient Giving, an organization devoted to “cost-effective, high-impact” contributions. Natasha McCauley served as a trustee of Effective Ventures, a major organizational hub within the effective altruism ecosystem. OpenAI’s former chief scientist Ilya Sutskever had been described as deeply philosophically and emotionally invested in aligning AI and artificial general intelligence for the benefit of humanity. Even Musk had publicly professed admiration for certain aspects of effective altruism and repeatedly framed AI development in existential terms.
RESISTANCE IS FUTILE
When the OpenAI board abruptly removed Sam Altman as CEO in November 2023, the public explanation was opaque. Subsequent reporting and testimony, however, suggested a combination of concerns about governance, honesty, commercialization, and, importantly, safety culture. These were serious allegations and deserved to have been taken seriously, but the board clothed its criticism of Altman in the garb of saving humanity.
The directors apparently believed that OpenAI occupied a uniquely consequential position in the development of artificial general intelligence and that changing leadership could meaningfully alter the pace or direction of civilization-scale technological change. But there was no evidence that that notion had any meaningful connection with reality. Altman returned four days later.
His reinstatement demonstrated how difficult it had become for any single nonprofit board to exercise lasting control over AI development. AI had already become too distributed, too capitalized, and too globally competitive. Employees revolted. Investors intervened. Microsoft, a key investor and customer, applied pressure. The board itself fractured and almost completely dissolved. The episode revealed a mismatch between institutional theory and technological reality.
THE GOD COMPLEX
An examination of Musk’s positions on the risks of AI becomes striking. Musk and the OpenAI board’s safety-focused faction appeared to occupy opposite sides during the recent trial, but they share the same underlying structure of reasoning. Both treat advanced AI as a civilizational inflection point. Both elevate OpenAI to the position of a uniquely consequential institution. Both justify extraordinary interventions on that basis. Both assume that the governance of one organization can meaningfully influence humanity’s future.
But the broader reality is that the development of advanced AI is now distributed across far more actors than any single lawsuit or nonprofit structure can plausibly govern. Google, Meta, Anthropic, xAI, Chinese research labs, European consortia, open-source communities, and university researchers all contribute to the accelerating diffusion of so-called frontier AI capabilities. No single nonprofit board can function as a meaningful civilizational lever. The same was true of the lawsuit. Even a Musk victory would not have halted frontier AI development globally; it would, at most, have rearranged the corporate structure of one important player among many. The risks, whatever they are, are going to arise with or without OpenAI.
That does not mean institutional governance is irrelevant. Frontier labs concentrate unusual amounts of talent, computing resources, capital, and deployment power. OpenAI’s decisions, like those of any major player, unquestionably influence the industry.
The larger question that almost nobody in the courtroom seemed willing to ask is, Isn’t there something absurd about expecting a nonprofit with effective altruism principles to regulate a trillion-dollar, for-profit industry? Can any organization simultaneously pursue commercial dominance and safety, especially when its competition may act otherwise?
The deeper irony of the trial was that both Musk and portions of the effective altruism and AI-safety worlds appear to derive authority from the same underlying claim: that they occupy a uniquely privileged position relative to civilization-scale stakes. Once actors begin reasoning at that scale, ordinary institutional constraints can start to look provincial or insufficient. They often follow a style of reasoning in which speculative future benefits could appear large enough to justify unusual concentrations of wealth, secrecy, or institutional power in the present.
This mentality—moral exceptionalism justified by civilization-level stakes—now permeates much of the AI world, including some of the people who testified against one another in Oakland. Musk and the effective altruists want similar outcomes based on similar moral compulsions.
And that may ultimately be the trial’s most revealing lesson.
The problem with effective altruism is not that it asks people to think hard about doing good. It is that the framework, by claiming to measure what cannot be measured, converts private moral conviction into a warrant for moral approbation of others. Do good. Calculate if it helps you sleep at night. The framework becomes pathological at the point where the calculation tells you that you may override or judge others. That is the move the 2023 board made when it acted on civilizational stakes it could not articulate to Microsoft. It is the move Musk’s lawsuit made by dressing a control fight in a charitable-trust theory of humanity’s interest. Both attempts were dispatched.
The children drowning in front of us are real. The quadrillions of hypothetical digital minds are not. The trial in Oakland was ultimately less about saving humanity than about who gets to steward its future.•
David Mills is a professor of practice and senior lecturer (continuing appointment) at Stanford Law School. He serves on numerous boards and was co-chair of the board of the NAACP Legal Defense Fund for over 10 years.
