Timeline of AI safety

The timeline currently offers focused coverage of the period until February 2023. It is likely to miss important developments outside this period (particularly after this period) though it may have a few events from after this period.

This is a timeline of AI safety. AI safety is the field focused on reducing risks from artificial intelligence (AI).

Big picture

Overall summary

Time period	Development summary	More details
Until 1950	Fictional portrayals only	Most discussion of AI safety is in the form of fictional portrayals. It warns of the risks of robots who, through either stupidity or lack of goal alignment, no longer remain under the control of humans.
1950 to 2000	Scientific speculation + fictional portrayals	During this period, discussion of AI safety moves from merely being a topic of fiction to one that scientists who study technological trends start talking about. The era sees commentary by I. J. Good, Vernor Vinge, and Bill Joy.
2000 to 2012	Birth of AI safety organizations, close connections with transhumanism	This period sees the creation of the Singularity Institute for Artificial Intelligence (SIAI) (which would later become the Machine Intelligence Research Institute (MIRI)) and the evolution of its mission from creating friendly AI to reducing the risk of unfriendly AI. The Future of Humanity Institute (FHI) and Global Catastrophic Risk Institute (GCRI) are also founded. AI safety work during this time is closely tied to transhumanism and has close connections with techno-utopianism. Peter Thiel and Jaan Tallinn are key funders of the early ecosystem.
2013 to 2022	Mainstreaming of AI safety, separation from transhumanism	SIAI changes name to MIRI, sells off the "Singularity" brand to Singularity University, grows considerably in size, and gets a lot of funding. Superintelligence, the book by Nick Bostrom, is released. The Future of Life Institute (FLI) and OpenAI are started, and the latter grows considerably. New organizations founded include the Center for the Study of Existential Risk (CSER), Leverhulme Centre for the Future of Intelligence (CFI), Future of Life Institute (FLI), OpenAI, Center for Human-Compatible AI (CHAI), Berkeley Existential Risk Initiative (BERI), Ought, and the Center for Security and Emerging Technology (CSET). OpenAI in particular becomes quite famous and influential. Prominent individuals such as Elon Musk, Sam Altman, and Bill Gates talk about the importance of AI safety and the risks of unfriendly AI. Key funders of this ecosystem include Open Philanthropy and Elon Musk.
Late 2022 onward	Explosion of AI in the public consciousness	OpenAI releases ChatGPT in late 2022. This triggers an AI "arms race" with players including OpenAI (ChatGPT), Anthropic (Claude), Microsoft (Bing Chat, Copilot), and Google (Google Bard, Gemini). People's estimates of the probability of short AI timelines increase.

Highlights by year (2013 onward)

Year	Highlights
2013	Research and outreach focused on forecasting and timelines continue. Connections with the nascent effective altruism movement strengthen. The Center for the Study of Existential Risk and the Foundational Research Institute launch.
2014	Superintelligence: Paths, Dangers, Strategies by Nick Bostrom is published. The Future of Life Institute is founded and AI Impacts launches. AI safety gets more mainstream attention, including from Elon Musk, Stephen Hawking, and the fictional portrayal Ex Machina. While forecasting and timelines remain a focus of AI safety efforts, the effort shifts toward the technical AI safety agenda, with the launch of the Intelligent Agent Foundations Forum.
2015	AI safety continues to get more mainstream, with the founding of OpenAI (supported by Elon Musk and Sam Altman) and the Leverhulme Centre for the Future of Intelligence, the Open Letter on Artificial Intelligence, the Puerto Rico conference, and coverage on Wait But Why. This also appears to be the last year that Peter Thiel donates in the area.
2016	Open Philanthropy makes AI safety a focus area; it would ramp up giving in the area considerably starting around this time. The landmark paper "Concrete Problems in AI Safety" is published, and OpenAI's safety work picks up pace. The Center for Human-Compatible AI launches. The annual tradition of LessWrong posts providing an AI alignment literature review and charity comparison for the year begins. AI safety continues to get more mainstream, with the Partnership on AI and the Obama administration's efforts to understand the subject.
2017	This is a great year for cryptocurrency prices, causing a number of donations to MIRI from people who got rich through cryptocurrency. The AI safety funding and support landscape changes somewhat with the launch of the Berkeley Existential Risk Initiative (BERI) (and funding of its grants program by Jaan Tallinn) and the Effective Altruism Funds, specifically the Long-Term Future Fund. Open Philanthropy makes several grants in AI safety, including a $30 million grant to OpenAI and a $3.75 million grant to MIRI. AI safety attracts dismissive commentary from Mark Zuckerberg, while Elon Musk continues to highlight its importance. The year begins with the Asilomar Conference and the Asilomar AI Principles, and initiatives such as AI Watch and the AI Alignment Prize begin toward the end of the year.
2018	Activity in the field of AI safety becomes more steady, in terms of both ongoing discussion (with the launch of the AI Alignment Newsletter, AI Alignment Podcast, and Alignment Forum) and funding (with structural changes to the Long-Term Future Fund to make it grant more regularly, the introduction of the annual Open Philanthropy AI Fellowship grants, and more grantmaking by BERI). Near the end of the year, MIRI announces its nondisclosure-by-default policy. Ought, Median Group, and the Stanford Center for AI Safety launch during the year.
2019	The Center for Security and Emerging Technology (CSET), that is focused on AI safety and other security risks, launches with a 5-year $55 million grant from Open Philanthropy. The Stanford Institute for Human-Centered Artificial Intelligence (HAI) launches. Grantmaking from the Long-Term Future Fund picks up pace; BERI hands off its grantmaking of Jaan Tallinn's money to the Survival and Flourishing Fund (SFF). Open Philanthropy begins using the Committee for Effective Altruism Support to decide grant amounts for some of its AI safety grants, including grants to MIRI. OpenAI unveils its GPT-2 model but does not release the full model initially; this sparks discussion on disclosure norms.
2020	Andrew Critch and David Krueger release their ARCHES paper. OpenAI unveils GPT-3, leading to further discussion of AI safety implications. AI Safety Support launches. The funding ecosystem continues to mature: Open Philanthropy and the Survival and Flourishing Fund continue to make large grants to established organizations, while the Long-Term Future Fund increasingly shifts focus to donating to individuals.
2021	Several new AI safety organizations start, including: Anthropic (by Dario and Daniela Amodei, along with several others who left OpenAI in late 2020 or early 2021), Alignment Research Center (by Paul Christiano, who also left OpenAI around that time), and Redwood Research (which appears to have started in 2019, but taken off and scaled up in 2021). A bunch of conversations related to AI safety, many of them including Eliezer Yudkowsky and other researchers (MIRI and non-MIRI), are published in late 2021. Jaan Tallinn's grantmaking, mostly via the Survival and Flourishing Fund, and a lot of it for AI safety, increases a lot in 2021, to $15-20 million, mostly related to the significant increase in Ethereum prices combined with his 2020 pledge.
2022	This year sees the entry of the FTX Future Fund into the AI safety funding space. Anthropic continues to grow by raising a massive Series B, led by the team from FTX and Alameda Research. In November, the collapse of FTX results in the collapse of the FTX Future Fund and uncertainty about the fate of various projects started by the FTX Future Fund. The release by OpenAI of ChatGPT (based on GPT-3.5) at the end of the year raises the profile of AI (and also of AI safety), with the implications only beginning to unfold.
2023	The year sees an "arms race" of AI tools from leading companies such as Microsoft and Google, as well as AI-focused organizations such as OpenAI and Anthropic, with the successful release of ChatGPT in late 2022 a likely trigger. As a result, the profile of AI as well as its critics -- coming from the AI safety perspective as well as other angles -- rises quite a bit. People's estimates of the probability of shorter AI timelines increase. In late 2023, an attempt to remove Sam Altman from OpenAI fails, and the failure is considered by some to be a failure for AI safety, given Altman's desire to move fast with AI.

Full timeline

Inclusion criteria

Here is a partial list of thoughts on what rows we've included:

• For AI safety organizations, we have tried to include a row for their launch where available. When an exact launch date is not available, we have tried to include an approximate launch date based on the history of the website or other public announcements.
• Most grants from Open Philanthropy in AI safety are included. Some grants may be omitted because of small size. For recent grants, we may not have added them yet.
• We have tried to include important fictional portrayals and scientific speculation related to AI before 2000, as comprehensively as possible. For the period from 2000 onward, we have been more selective, and focused more on relatively notable commentary or commentary that played a role in changing the AI safety conversation in specific communities.

Timeline

Year	Month and date	Event type	Details
1630–1650		Fictional portrayal	The publication of the story of the Golem of Chełm dates to around this period. Wikipedia: "Golems are not intelligent, and if commanded to perform a task, they will perform the instructions literally. In many depictions Golems are inherently perfectly obedient. In its earliest known modern form, the Golem of Chełm became enormous and uncooperative. In one version of this story, the rabbi had to resort to trickery to deactivate it, whereupon it crumbled upon its creator and crushed him."
1818		Fictional portrayal	The novel Frankenstein is published. Frankenstein pioneers the archetype of the artificial intelligence that turns against its creator, and is sometimes discussed in the context of an AI takeoff.
1863	June	Publication	In Darwin among the Machines, Samuel Butler raises the possibility that intelligent machines will eventually supplant humans as the dominant form of life.
1920		Fictional portrayal	The science fiction play R.U.R. is published. The play introduces the word "robot" to the English language and the plot contains a robot rebellion that leads to human extinction.
1942	000000002024-03-01-0000March	Fictional portrayal	The Three Laws of Robotics are introduced by Isaac Asimov in his short story "Runaround".
1947	000000002024-07-01-0000July	Fictional portrayal	With Folded Hands, a novelette by Jack Williamson, is published. The novelette describes how advanced robots (humanoids) take over large parts of the world to fulfill their Prime Directive, which is to make humans happy.
1948		Publication	In The general and logical theory of automata, John von Neumann articulates the idea of self-improving AI. Notable quote: "There is, however, a certain minimum level where this degenerative characteristic ceases to be universal. At this point automata which can reproduce themselves, or even construct higher entities, become possible." He would expand on this idea further in 1949, coming close to articulating what is now called an "intelligence explosion."
1950		Publication	Alan Turing publishes Computing Machinery and Intelligence in the philosophy journal Mind. The paper introduces the concept of the Turing test, the simple idea of which is that a machine would be said to have achieved human-level intelligence if it can convince a human that it is human. The Turing test would become a key part of discussions around benchmarking artificial intelligence, and also enter popular culture over the next few decades.
1960	000000002024-05-06-0000May 6	Publication	Norbert Wiener's article Some Moral and Technical Consequences of Automation is published. In 2013, Jonah Sinick would note the similarities between the points raised in this article and the thinking of AI safety leader Eliezer Yudkowsky. Wiener's work would also be cited by Allan Dafoe and Stuart Russell in 2016: "Rather, the risk arises from the unpredictability and potential irreversibility of deploying an optimization process more intelligent than the humans who specified its objectives. This problem was stated clearly by Norbert Wiener in 1960, and we still have not solved it."
1965, 1966		Publication	I. J. Good originates the concept of intelligence explosion in "Speculations Concerning the First Ultraintelligent Machine" where he says that an ultraintelligent machine would be "the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control." This is written in 1965 though formally published in 1966.
1966		Fictional portrayal	The science fiction novel Colossus by British author Dennis Feltham Jones is published. In the novel, both the United States and the USSR develop supercomputers, called Colossus and Guardian respectively, that they connect with each other to share knowledge. The supercomputers coordinate with each other to become increasingly powerful, and humans belatedly try to regain control.
1966		Publication	In Other Worlds than Ours, Cecil Maxwell argues that nations would invest in building powerful intelligent machines and surrender decisionmaking to them, and any nation that succeeded would grow into a major world power. He writes: It seems that, in the foreseeable future, the major nations of the world will have to face the alternative of surrendering national control to mechanical ministers, or being dominated by other nations which have already done this. Such a process will eventually lead to the domination of the whole Earth by a dictatorship of an unparalleled type — a single supreme central authority.
1974		Publication	In The Ignorance Explosion, Julius Lukasiewicz argues that it is very hard to predict the future after we have machine superintelligence.
1977		Fictional portrayal	The science fiction novel The Adolescence of P-1 by Thomas Joseph Ryan is published. It "tells the story of an intelligent worm that at first is merely able to learn to hack novel computer systems and use them to propagate itself, but later (1) has novel insights on how to improve its own intelligence, (2) develops convergent instrumental subgoals for self-preservation and resource acquisition, and (3) learns the ability to fake its own death so that it can grow its powers in secret and later engage in a "treacherous turn"against humans."
1979		Publication	In the book Machines Who Think by Pamela McCorduck, Edward Fredkin descibes the challenge of controlling smarter-than-human artificial general intelligences. He describes the high communication bandwidth between superintelligent machines, the goal-directed nature of intelligent systems, and the challenge of aligning the goals of the machines with those of humans.
1984	000000002024-10-26-0000October 26	Fictional portrayal	The American science fiction film The Terminator is released. The film contains the first appearance of Skynet, a "neural net-based conscious group mind and artificial general intelligence" that "seeks to exterminate the human race in order to fulfill the mandates of its original coding".
1985		Fictional portrayal	In Robots and Empire, Isaac Asimov introduces the Zeroth Law of Robotics, which states: "A robot may not injure humanity, or through inaction, allow humanity to come to harm." This is analogous to, and must take precedence over, the Fist Law "A robot may not injure a human being, or through inaction, allow a human being to come to harm." The law was self-programmed by the robot R. Daneel Olivaw based on the ideas of R. Giskard Reventlov. In particular, the zeroth law now allows a robot to harm or allow harm to individual humans for the greater good of humanity.
1987		Publication	In an article A Question of Responsibility for AI Magazine, Mitchell Waldrop introduces the term machine ethics. He writes: However, one thing that is apparent from the above discussion is that intelligent machines will embody values, assumptions, and purposes, whether their programmers consciously intend them to or not. Thus, as computers and robots become more and more intelligent, it becomes imperative that we think carefully and explicitly about what those built-in values are. Perhaps what we need is, in fact, a theory and practice of machine ethics, in the spirit of Asimov’s three laws of robotics.
1988	000000002024-10-01-0000October 1	Publication	The book Mind Children: The Future of Robot and Human Intelligence by Hans Moravec is published. The book says that machines with human-level intelligence are feasible within fifty years, and should be thought of as the "mind children" of humans. The book would be listed in a 2013 MIRI blog post in a list of past work on AI safety and the AI takeoff.
1993		Publication	Vernor Vinge's article "The Coming Technological Singularity: How to Survive in the Post-Human Era" is published. The article popularizes the idea of an intelligence explosion.
1996	000000002024-05-01-0000May 1	Publication	The book Reflections on Artificial Intelligence: The Legal, Moral and Ethical Dimensions by Blay Whitby is published. A 2013 blog post by MIRI would list this as part of "earlier work on the topic" of AI safety.
1998	000000002024-12-03-0000December 3	Publication	The book Robot: Mere Machine to Transcendent Mind by Hans Moravec is published. The book argues that human levels of intelligence are achievable by 2040, and has an optimistic take on this future: "Intelligent machines, which will grow from us, learn our skills, and share our goals and values, can be viewed as children of our minds." It continues the themes of Moravec's previous book Mind Children, and would be cited in a 2013 MIRI blog post in a list of past work related to AI safety and the AI takeoff.
1999	000000002024-01-01-0000January 1	Publication	The book The Age of Spiritual Machines by Ray Kurzweil is published. The book describes Kurzweil's utopian vision of technological progress and the path to computers achieving superhuman intelligence.
2000	000000002024-04-01-0000April	Publication	Bill Joy's article "Why The Future Doesn't Need Us" is published in Wired.
2000	000000002024-07-27-0000July 27	Organization	Machine Intelligence Research Institute (MIRI) is founded as the Singularity Institute for Artificial Intelligence (SIAI) by Brian Atkins, Sabine Atkins (then Sabine Stoeckel) and Eliezer Yudkowsky. The organization's mission ("organization's primary exempt purpose" on Form 990) at the time is "Create a Friendly, self-improving Artificial Intelligence"; this mission would be in use during 2000–2006 and would change in 2007.
2001	000000002024-06-15-0000June 15	Publication	Version 1.0 of "Creating Friendly AI" by Eliezer Yudkowsky is published.
2002	000000002024-03-08-0000March 8	AI box	The first AI box experiment by Eliezer Yudkowsky, against Nathan Russell as gatekeeper, takes place. The AI is released.
2002	000000002024-07-04-0000July 4–5	AI box	The second AI box experiment by Eliezer Yudkowsky, against David McFadzean as gatekeeper, takes place. The AI is released.
2002	000000002024-10-31-0000October 31	Publication	Bill Hibbard's Super-Intelligent Machines is published.
2003		Publication	Nick Bostrom's paper "Ethical Issues in Advanced Artificial Intelligence" is published. The paper introduces the paperclip maximizer thought experiment.
2004	000000002024-11-11-0000November 11	Publication	In his book Catastrophe: Risk and Response, legal scholar Richard Posner discusses risks from artificial general intelligence at some length. According to a 2013 blog post from the Machine Intelligence Research Institute: "His analysis is interesting in part because it appears to be intellectually independent from the Bostrom-Yudkowsky tradition that dominates the topic today. [...] Still, much of Posner’s analysis is consistent with the basic points of the Bostrom-Yudkowsky tradition [...] One major point of divergence seems to be that Posner worries about a scenario in which AGIs become self-aware, re-evaluate their goals, and decide not to be “bossed around by a dumber species” anymore."
2005		Organization	The Future of Humanity Institute (FHI) is founded.
2005		Conference	The field of machine ethics is delineated at the Fall 2005 Symposium on Machine Ethics held by the Association for Advancement of Artificial Intelligence.
2005	000000002024-08-21-0000August 21	AI box	The third AI box experiment by Eliezer Yudkowsky, against Carl Shulman as gatekeeper, takes place. The AI is released.
2005		Publication	The Singularity is Near by inventor and futurist Ray Kurzweil is published. The book builds upon Kurzweil's previous books The Age of Intelligent Machines (1990) and The Age of Spiritual Machines (1999), but unlike its predecessors, uses the term technological singularity introduced by Vinge in 1993. Unlike Bill Joy, Kurzweil takes a very positive view of the impact of smarter-than-human AI and the upcoming (in his view) technological singularity.
2006	000000002024-11-01-0000November		Robin Hanson starts Overcoming Bias. Eliezer Yudkowsky's posts on Overcoming Bias would form seed material for LessWrong, which would grow to be an important community for discussion related to AI safety.
2008		Publication	Steve Omohundro's paper "The Basic AI Drives" is published. The paper argues that certain drives, such as self-preservation and resource acquisition, will emerge in any sufficiently advanced AI. The idea would subsequently be defended by Nick Bostrom as part of his instrumental convergence thesis.
2008		Publication	Global Catastrophic Risks is published. The book includes Eliezer Yudkowsky's chapter "Artificial Intelligence as a Positive and Negative Factor in Global Risk".
2008	000000002024-10-13-0000October 13	Publication	Moral Machines: Teaching Robots Right from Wrong by Wendell Wallach and Colin Allen is published by Oxford University Press. The book advertises itself as "the first book to examine the challenge of building artificial moral agents, probing deeply into the nature of human decision making and ethics."
2008	000000002024-11-01-0000November–December	Outside review	The AI-Foom debate between Robin Hanson and Eliezer Yudkowsky takes place. The blog posts from the debate would later be turned into an ebook by MIRI.
2009	000000002024-02-01-0000February	Project	Eliezer Yudkowsky starts LessWrong using as seed material his posts on Overcoming Bias. On the 2009 accomplishments page, MIRI describes LessWrong as being "important to the Singularity Institute's work towards a beneficial Singularity in providing an introduction to issues of cognitive biases and rationality relevant for careful thinking about optimal philanthropy and many of the problems that must be solved in advance of the creation of provably human-friendly powerful artificial intelligence". And: "Besides providing a home for an intellectual community dialoguing on rationality and decision theory, Less Wrong is also a key venue for SIAI recruitment. Many of the participants in SIAI's Visiting Fellows Program first discovered the organization through Less Wrong."
2009	000000002024-12-11-0000December 11	Publication	The third edition of Artificial Intelligence: A Modern Approach by Stuart J. Russell and Peter Norvig is published. In this edition, for the first time, Friendly AI is mentioned and Eliezer Yudkowsky is cited.
2010		Organization	DeepMind is founded by Demis Hassabis, Shane Legg, and Mustafa Suleyman. Legg had previously received the $10,000 Canadian Singularity Institute for Artificial Intelligence Prize.
2010		Organization	Vicarious is founded by Scott Phoenix and Dileep George. The company "has publicly expressed some concern about potential risks from future AI development" and the founders are signatories on the FLI open letter.
2011		Publication	Baum, Goertzel, and Goertzel's "How Long Until Human-Level AI? Results from an Expert Assessment" is published.
2011		Organization	The Global Catastrophic Risk Institute (GCRI) is founded by Seth Baum and Tony Barrett.
2011		Organization	Google Brain is started by Jeff Dean, Greg Corrado, and Andrew Ng.
2011–2013		Project	Sometime during this period, the Back of the Envelope Guide to Philanthropy, a website created by Gordon Irlam, includes prevention of "hostile artificial intelligence" as a top 10 philanthropic opportunity by impact.
2011	000000002024-09-01-0000September	Organization	The Oxford Martin Programme on the Impacts of Future Technology (FutureTech) launches.
2013		Publication	Luke Muehlhauser's book Facing the Intelligence Explosion is published.
2013	000000002024-04-13-0000April 13		MIRI publishes an update on its strategy on its blog. In the blog post, MIRI executive director Luke Muehlhauser states that MIRI plans to put less effort into public outreach and shift its research to Friendly AI math research.
2013	000000002024-07-01-0000July	Organization	The Center for the Study of Existential Risk (CSER) launches.
2013	000000002024-07-01-0000July	Organization	The Foundational Research Institute (FRI) is founded. Some of FRI's work discusses risks from artificial intelligence.
2013	000000002024-07-08-0000July 8	Publication	Luke Muehlhauser's blog post Four Focus Areas of Effective Altruism is published. The four focus areas listed are poverty reduction, meta effective altruism, the long term future, and animal suffering. AI safety concerns fall under the "long term future" focus area. This identification of focus areas would persist for several years, and would also be incorporated into the design of the Effective Altruism Funds in 2017. In particular, the blog post encapsulates the central position of AI safety in the then-nascent effective altruist movement.
2013	000000002024-10-01-0000October 1	Publication	Our Final Invention: Artificial Intelligence and the End of the Human Era by James Barrat is published. The book discusses risks from human-level of superhuman artificial intelligence.
2014		Publication	Müller and Bostrom's "Future Progress in Artificial Intelligence: A Survey of Expert Opinion" is published.
2014	000000002024-01-26-0000January 26		Google announces that it has acquired DeepMind. At the same time, it sets up an AI ethics board. DeepMind co-founders Shane Legg and Demis Hassabis, as well as AI safety funders Peter Thiel and Jaan Tallinn, are believed to have been influential in the process.
2014	000000002024-03-01-0000March–May	Organization	Future of Life Institute (FLI) is founded.
2014	000000002024-07-01-0000July–September	Publication	Nick Bostrom's book Superintelligence: Paths, Dangers, Strategies is published. The book would have tremendous impact on defining and raising the status of AI safety concerns, due to "its clear explanation of why superintelligent AI may have arbitrarily negative consequences and why it’s important to begin addressing the issue well in advance."
2014	000000002024-08-01-0000August	Project	The AI Impacts website is launched by Paul Christiano and Katja Grace, with funding channeled through MIRI. AI Impacts would grow into a leading source for understanding AI trends and scenarios, and would later receive funding from th Future of Life Institute and Open Philanthropy.
2014	Fall	Project	The One Hundred Year Study on Artificial Intelligence (AI100) launches.
2014	000000002024-10-22-0000October 22–24	Opinion	During an interview at the AeroAstro Centennial Symposium, Elon Musk calls artificial intelligence humanity's "biggest existential threat".
2014	000000002024-11-04-0000November 4	Project	The Intelligent Agent Foundations Forum, run by MIRI, is launched.
2014	000000002024-11-05-0000November 5	Publication	The book Ethical Artificial Intelligence by Bill Hibbard is released on ArXiV. The book brings together ideas from Hibbard's past publications related to technical AI risk.
2014	000000002024-11-01-0000November	Publication	Edge hosts a conversation on "The Myth of AI" with an introduction by John Brockman and a lead essay by Jaron Lanier, followed by responses from "Reality Club" participants George Church, Peter Diamandis, Lee Smolin, Rodney Brooks, Nathan Myhrvold, George Dyson, Pamela McCorduck, Sendhil Mullainathan, Steven Pinker, Neal Gershenfeld, D.A. Wallach, Michael Shermer, Stuart Kauffman, Kevin Kelly, Lawrence Krauss, Robert Provine, Stuart Russell, Kai Krause. Russell's response is quoted and discussed by Rob Bensinger on LessWrong.
2014	000000002024-12-02-0000December 2	Opinion	In an interview with BBC, Stephen Hawking states that advanced artificial intelligence could end the human race.
2014	000000002024-12-16-0000December 16	Fictional portrayal	The movie Ex Machina is released. The movie highlights the paperclip maximizer idea: it shows how a robot programmed to optimize for being able to make sure it escapes can callously damage human lives in the process. It also covers the ideas of the Turing test (a robot convincing a human that it has human-like qualities, even though the human knows that the robot is not human) and the AI box experiment (a robot convincing a human to release it from its "box" similar to the experiment proposed and carried out by Eliezer Yudkowsky). It leads to more public discussion of AI safety.
2015			Daniel Dewey joins Open Philanthropy. He begins as or would become Open Phil's program officer for potential risks from advanced artificial intelligence.
2015		Publication	The 2015 question on Edge (edge.org) is "What do you think about machines that think?" The question draws responses from a number of intellectuals, including AI researchers, ethicists, AI historians, and other social commentators. Responses from Nick Bostrom, Steve Omohundro, Stuart J. Russell, and Eliezer Yudkowsky discuss AI safety challenges.
2015		Organization	The Strategic Artificial Intelligence Research Centre launches around this time.
2015	000000002024-01-01-0000January	Publication	The Open Letter on Artificial Intelligence, titled "Research Priorities for Robust and Beneficial Artificial Intelligence: an Open Letter", is published. Signatories include Stephen Hawking, Elon Musk, Peter Norvig, and Stuart J. Russell.
2015	000000002024-01-28-0000January 28	Opinion	During an "ask me anything" (AMA) session on reddit, Bill Gates states his concern about artificial superintelligence.
2015	000000002024-01-02-0000January 2–5	Conference	The Future of AI: Opportunities and Challenges, an AI safety conference, takes place in Puerto Rico. The conference is organized by the Future of Life Institute. Nate Soares of the Machine Intelligence Research Institute would later call this the "turning point" of when top academics begin to focus on AI risk.
2015	000000002024-01-22-0000January 22–27	Publication	Tim Urban publishes on Wait But Why a two-part series of blog posts about superhuman AI.
2015	000000002024-02-25-0000February 25	Opinion	Sam Altman, president of Y Combinator, publishes a blog post in which he writes that the development of superhuman AI is "probably the greatest threat to the continued existence of humanity".
2015	000000002024-05-01-0000May 1	Publication	The Wikipedia article on existential risk from artificial general intelligence is published.
2015	000000002024-05-22-0000May 22	Publication	The blog post AI Researchers On AI Risk by Scott Alexander is published. The post lists prominent AI researchers who have expressed concern about AI safety, countering claims by others that serious AI safety concerns lack buy-in from real AI researchers.
2015	000000002024-06-04-0000June 4	Opinion	At Airbnb's Open Air 2015 conference, Sam Altman, president of Y Combinator, states his concern for advanced artificial intelligence and shares that he recently invested in a company doing AI safety research.
2015	000000002024-06-17-0000June 17,21	Publication	The Kindle edition of Artificial Superintelligence: A Futuristic Approach by Roman Yampolskiy is published. The paperback would be published on June 21. Yampolskiy takes an AI safety engineering perspective, rather than a machine ethics perspective, to the problem of AI safety.
2015	000000002024-07-01-0000July 1	Funding	The Future of Life Institute's Grant Recommendations for its first round of AI safety grants are publicly announced. The grants would be disbursed on September 1. The funding for the grants comes from a $10 million grant from Elon Musk.
2015	000000002024-08-01-0000August	Conference	AI safety is a key theme of EA Global 2015, with the panel on AI risk including Elon Musk, Daniel Dewey, Nick Bostrom, Nate Soares and Stuart Russell.
2015	000000002024-08-01-0000August	Funding	Open Philanthropy awards a grant of $1.2 million to the Future of Life Institute, saying that they are impressed with the selections from the first round of grants funded by Elon Musk. This adds to the pool of money from Elon Musk.
2015	000000002024-08-01-0000August	Publication	Open Philanthropy publishes its cause report on potential risks from advanced artificial intelligence.
2015	000000002024-08-29-0000August 29		The ControlProblem subreddit (full title: "The Artificial General Intelligence Control Problem") is created.
2015	000000002024-10-01-0000October	Publication	Open Philanthropy first publishes its page on AI timelines.
2015	000000002024-12-01-0000December	Organization	The Leverhulme Centre for the Future of Intelligence launches around this time.
2015	000000002024-12-11-0000December 11	Organization	OpenAI is announced to the public. (The news articles from this period make it sound like OpenAI launched sometime after this date.)
2016	000000002024-04-28-0000April 28	Publication	The Global Catastrophic Risks 2016 report is published. The report is a collaboration between the Global Priorities Project and the Global Challenges Foundation. The report includes discussion of risks from artificial general intelligence under "emerging risks".
2016	000000002024-04-07-0000April 7	Publication	80,000 Hours releases a new "problem profile" for risks from artificial intelligence, titled "Risks posed by artificial intelligence".
2016	000000002024-05-01-0000May	Publication	Luke Muehlhauser of Open Philanthropy publishes "What should we learn from past AI forecasts?".
2016	000000002024-05-06-0000May 6	Publication	Holden Karnofsky, Executive Director of Open Philanthropy, publishes a blog post on why Open Philanthropy is making potential risks from artificial intelligence a major priority for the year.
2016	000000002024-05-06-0000May 6	Publication	Holden Karnofsky, Executive Director of Open Philanthropy, publishes "Some Background on Our Views Regarding Advanced Artificial Intelligence" on the Open Philanthropy blog.
2016	000000002024-06-01-0000June	Funding	Open Philanthropy awards a grant of $264,525 to George Mason University for work by Robin Hanson.
2016	000000002024-06-21-0000June 21	Publication	"Concrete Problems in AI Safety" by Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané is submitted to the arXiv. The paper would receive a shoutout from Open Philanthropy. It would become a landmark in AI safety literature, and many of its authors would continue to do AI safety work at OpenAI in the years to come.
2016	000000002024-06-27-0000June 27 and 28	Conference	Carnegie Mellon University (CMU) hosts a two-day Workshop on Safety and Control for AI, the first day being an "Exploratory Technical Workshop" (co-sponsored by the Carnegie Mellon Science of Security Lablet) and the seconf day being a "Public Workshop" (co-sponsored by the Office of Science and Technology Policy (OSTP) and CMU). Participants include Edward Felten of OSTP, Bill Scherlis of CMU, Jason Matheny of IARPA, Eric Horvitz of Microsoft, and many others.
2016	000000002024-08-01-0000August	Organization	The UC Berkeley Center for Human-Compatible Artificial Intelligence launches under the leadership of AI expert Stuart J. Russell (co-author with Peter Norvig of Artificial Intelligence: A Modern Approach). The focus of the center is "to ensure that AI systems are beneficial to humans".
2016	000000002024-08-01-0000August	Funding	Open Philanthropy awards a grant of $5.6 million over two years to the newly formed Center for Human-Compatible AI at the University of California, Berkeley.
2016	000000002024-08-01-0000August	Funding	Open Philanthropy awards a grant of $500,000 to the Machine Intelligence Research Institute.
2016	000000002024-08-24-0000August 24		US president Barack Obama speaks to entrepreneur and MIT Media Lab director Joi Ito about AI risk.
2016	000000002024-09-20-0000September 20, November 2	Publication	On September 20, Oren Etzioni, in a piece titled No, the Experts Don’t Think Superintelligent AI is a Threat to Humanity for MIT Technology Review, critiques Nick Bostrom's Superintelligence and includes Etzioni's own survey of AI researchers to refute Bostrom's survey data. In a reply titled Yes, We Are Worried About the Existential Risk of Artificial Intelligence on November 2, Stuart J. Russell and Allan Dafoe argue that the focus of Bostrom's Superintelligence was not on the predictions of imminent AI, but "its clear explanation of why superintelligent AI may have arbitrarily negative consequences and why it’s important to begin addressing the issue well in advance."
2016	000000002024-09-28-0000September 28	Organization	The Partnership on AI is publicly announced.
2016	000000002024-10-12-0000October 12	Publication	Under the Obama Administration, the United States White House releases two reports, Preparing for the Future of Artificial Intelligence and National Artificial Intelligence Research and Development Strategic Plan. The former "surveys the current state of AI, its existing and potential applications, and the questions that progress in AI raise for society and public policy".
2016	000000002024-11-01-0000November	Funding	Open Philanthropy awards a grant of $199,000 to the Electronic Frontier Foundation for work by Peter Eckersley.
2016	000000002024-12-01-0000December	Funding	Open Philanthropy awards a grant of $32,000 to AI Impacts for work on strategic questions related to potential risks from advanced artificial intelligence.
2016	000000002024-12-03-0000December 3, 12	Publication	A couple of posts are published on LessWrong by Center for Applied Rationality (CFAR) president Anna Salamon. The posts discuss CFAR's new focus on AI safety.
2016	000000002024-12-13-0000December 13	Publication	The "2016 AI Risk Literature Review and Charity Comparison" is published on the Effective Altruism Forum. The lengthy blog post covers all the published work of prominent organizations focused on AI safety.
2017		Publication	The Global Catastrophic Risks 2017 report is published. The report discusses risks from artificial intelligence in a dedicated chapter.
2017		Publication	The Global Risks Report 2017 is published by the World Economic Forum. The report contains a section titled "Assessing the Risk of Artificial Intelligence" under "Emerging Technologies".
2017	000000002024-01-05-0000January 5 – 8	Conference	The Asilomar Conference on Beneficial AI is organized by the Future of Life Institute at the Asilomar Conference Grounds. It is the successor to the 2015 Puerto Rico conference. It would result in the creation of the 23 Asilomar AI Principles, a set of guidelines for AI research. This is not to be confused with the Asilomar Conference on Recombinant DNA, that has been examined by the AI safety community as a case study in risk mitigation.
2017	000000002024-02-09-0000February 9	Project	The Effective Altruism Funds (EA Funds) is announced on the Effective Altruism Forum. EA Funds includes a Long-Term Future Fund that is partly intended to support "priorities for robust and beneficial artificial intelligence".
2017	000000002024-03-01-0000March	Funding	Open Philanthropy awards a grant of $2.0 million to the Future of Humanity Institute for general support.
2017	000000002024-03-01-0000March	Funding	Open Philanthropy awards a grant of $30 million to OpenAI for general support.
2017	000000002024-03-05-0000March 5		DeepMind releases AI Safety Gridworlds, which evaluate AI algorithms on nine safety features, such as whether the algorithm wants to turn off its own kill switch. DeepMind confirms that existing algorithms perform poorly, which is "unsurprising" because the algorithms "are not designed to solve these problems"; solving such problems might require "potentially building a new generation of algorithms with safety considerations at their core".
2017	000000002024-04-01-0000April	Organization	The Berkeley Existential Risk Initiative (BERI) launches around this time (under the leadership of Andrew Critch, who previously helped found the Center for Applied Rationality) to assist researchers working at institutions working to mitigate existential risk, including AI risk.
2017	000000002024-04-06-0000April 6	Publication	80,000 Hours publishes an article about the pros and cons of working on AI safety, titled "Positively shaping the development of artificial intelligence".
2017	000000002024-05-01-0000May	Funding	Open Philanthropy awards a grant of $1.5 million to the UCLA School of Law for work on governance related to AI risk; this would lead to the formation of AI Pulse.
2017	000000002024-05-24-0000May 24	Publication	"When Will AI Exceed Human Performance? Evidence from AI Experts" is published on the arXiv. Two researchers from AI Impacts are authors on the paper.
2017	000000002024-06-14-0000June 14	Publication	80,000 Hours publishes a guide to working in AI policy and strategy, written by Miles Brundage.
2017	000000002024-07-01-0000July	Funding	Open Philanthropy awards a grant of $2.4 million to the Montreal Institute for Learning Algorithms.
2017	000000002024-07-01-0000July	Funding	Open Philanthropy awards a grant of about $300,000 to Yale University to support research into the global politics of artificial intelligence led by Allan Dafoe.
2017	000000002024-07-01-0000July	Funding	Open Philanthropy awards a grant of about $400,000 to the Berkeley Existential Risk Initiative to support core functions of grantee, and to help them provide contract workers for the Center for Human-Compatible AI (CHAI) housed at the University of California, Berkeley.
2017	000000002024-07-15-0000July 15–16	Opinion	At the National Governors Association in Rhode Island, Elon Musk tells US governors that artificial intelligence is an "existential threat" to humanity.
2017	000000002024-07-23-0000July 23	Opinion	During a Facebook Live broadcast from his backyard, Mark Zuckerberg reveals that he is "optimistic" about advanced artificial intelligence and that spreading concern about "doomsday scenarios" is "really negative and in some ways […] pretty irresponsible".
2017	000000002024-10-01-0000October	Funding	Open Philanthropy awards MIRI a grant of $3.75 million over three years ($1.25 million per year). The cited reasons for the grant are a "very positive review" of MIRI's "Logical Induction" paper by an "outstanding" machine learning researcher, as well as Open Philanthropy having made more grants in the area so that a grant to MIRI is less likely to appear as an "outsized endorsement of MIRI's approach".
2017	000000002024-10-01-0000October	Project	The first commit for AI Watch, a repository of organizations, people, and products in AI safety, is made on October 23. Work on the web portal at aiwatch.issarice.com would begin the next day.
2017	000000002024-10-13-0000October 13	Publication	Eliezer Yudkowsky's blog post There's No Fire Alarm for Artificial General Intelligence is published on the MIRI blog and on the new LessWrong (this is shortly after the launch of the new version of LessWrong).
2017	000000002024-10-01-0000October  – December	Project	FHI launches its Governance of AI Program (later to be called the Centre for the Governance of AI, and shortened as GovAI), co-directed by Nick Bostrom and Allan Dafoe.
2017	000000002024-11-03-0000November 3	Project	Zvi Mowshowitz and Vladimir Slepnev announce the AI Alignment Prize, a $5,000 prize funded by Paul Christiano for publicly posted work advancing AI alignment. The prize would be discontinued after the fourth round (ending December 31, 2018) due to reduced participation.
2017	000000002024-12-01-0000December	Funding	Jaan Tallinn makes a donation of about $5 million to the Berkeley Existential Risk Initiative (BERI) Grants Program.
2017	000000002024-12-20-0000December 20	Publication	The "2017 AI Safety Literature Review and Charity Comparison" is published. The lengthy blog post covers all the published work of prominent organizations focused on AI safety, and is a refresh of a similar post published a year ago.
2017	Year-round	Funding	The huge increase in cryptocurrency prices in this year would drive a lot of donations to AI safety organizations from people who had held cryptocurrency and got rich from it. MIRI would not be able to match the performance of its 2017 fundraiser in later years, and would cite the unusually high cryptocurrency prices in 2017 as one possible reason.
2018	Early year	Organization	The Median Group, a "non-profit dedicated to research on global catastrophic risks", is formed at around this time. The group would include two former MIRI researchers. As of 2020, the AI safety part of its research would include an interactive model of AI progress as well as a rough estimate of the feasibility of training an AGI using reinforcement learning.
2018	000000002024-02-24-0000February 24	Publication	Paul Christiano publishes a blog post titled "Takeoff speeds" that discusses slow versus fast takeoff. Christiano points out that "slow takeoff" translates to faster initial AI progress (prior to takeoff) and discusses further the implications of the slow and fast takeoff scenarios; overall, Christiano considers slow takeoff a more likely scenario. He also posts a linkpost to his post on LessWrong and the Alignment Forum, that gets several comments. AI Impacts also publishes its reference page "Likelihood of discontinuous progress around the development of AGI" around this time. Over the next few years, there would be a lot of discussion referencing these posts.
2018	000000002024-02-28-0000February 28	Publication	80,000 Hours publishes a blog post A new recommended career path for effective altruists: China specialist suggesting specialization in China as a career path for people in the effective altruist movement. China's likely leading role in the development of artificial intelligence is highlighted as particularly relevant to AI safety efforts.
2018	000000002024-03-07-0000March 7	Conference	During Nextgov and Defense One’s Genius Machines 2018 event, Jason Matheny of IARPA argues that there should be more focus on avoiding bad, hackable design and less concern about superintelligent AI running amok. He says "We’re much less worried about ‘Terminator’ and Skynet scenarios than we are about digital ‘Flubber’ scenarios." Also: "Really badly engineered systems that are vulnerable to either error or malicious attack from outside." Matheny would later go on to lead the Center for Security and Emerging Technology (CSET), and his thinking would influence the direction of the organization.
2018	000000002024-04-04-0000April 4	Podcast	The first AI Alignment Podcast from the Future of Life Institute is on this day. The AI Alignment Podcast would happen at an approximate frequency of one per month from this point onward, but with no set cadence.
2018	000000002024-04-05-0000April 5	Documentary	The documentary Do You Trust This Computer?, directed by Chris Paine, is released. It covers issues related to AI safety and includes interviews with prominent individuals relevant to AI, such as Ray Kurzweil, Elon Musk and Jonathan Nolan.
2018	000000002024-04-09-0000April 9	Newsletter	The first issue of the weekly AI Alignment Newsletter, managed by Rohin Shah of the Center for Human-Compatible AI, is sent. Over the next two years, the team responsible for producing the newsletter would grow to four people, and the newsletter would also be produced in Chinese and in podcast form.
2018	000000002024-04-09-0000April 9	Disclosure norms, competition norms	OpenAI releases a charter stating that the organization commits to stop competing with a value-aligned and safety-conscious project that comes close to building artificial general intelligence, and also that OpenAI expects to reduce its traditional publishing in the future due to safety concerns.
2018	000000002024-04-12-0000April 12 to 22	Conference	The first AI Safety Camp is held in Gran Canaria. The AI Safety Camp team runs about two camps a year.
2018	000000002024-05-01-0000May	Funding	Open Philanthropy announces the first set of grants for the Open Philanthropy AI Fellowship, to 7 AI Fellows pursuing research relevant to AI risk. It also makes a grant of $525,000 to Ought and $100,000 to AI Impacts.
2018	000000002024-05-01-0000May	Publication	The paper "AGI Safety Literature Review" is published in the International Journal Conference on Artificial Intelligence and uploaded to the ArXiV. This would be referenced as an article to read for people interested in contributing technically to AI safety.
2018	000000002024-07-01-0000July	Funding	Open Philanthropy grants $429,770 to the University of Oxford to support research on the global politics of advanced artificial intelligence. The work will be led by Professor Allan Dafoe at the Future of Humanity Institute in Oxford, United Kingdom.
2018	000000002024-07-10-0000July 10 (beta), 000000002024-10-29-0000October 29 (out of beta)	Project	The team behind LessWrong 2.0 launches a beta for the AI Alignment Forum at AlignmentForum.org on July 10, as a successor to the Intelligent Agent Foundations Forum (IAFF) at agentfoundations.org. On October 29, the Alignment Forum exits beta and becomes generally available.
2018	000000002024-08-14-0000August 14	Funding	Nick Beckstead grants the Machine Intelligence Research Institute (MIRI) $488,994 from the Long-Term Future Fund. This is part of his last set of grants as fund manager; he would subsequently step down and the fund management would move to a different team.
2018	000000002024-08-23-0000August 23 onward (fall semester)	Course	The course CS 294-149: Safety and Control for Artificial General Intelligence is taught at UC Berkeley, with instructors Andrew Critch and Stuart Russell, both affiliated with the Center for Human-Compatible AI. The lecture schedule, available online, shows coverage of background concepts (like the VNM utility theorem), AI forecasts, and technical AI safety challenges.
2018	000000002024-09-01-0000September to October	Funding	During this period, the Berkeley Existential Risk Initiative (BERI) makes a number of grants to individuals working on projects related to AI safety.
2018	000000002024-10-25-0000October 25	Organization	The oldest Wayback Machine snapshot of the webpage aisafety.stanford.edu for the Stanford Center for AI Safety is from this date. A more precise launch date or launch announcement could not be found.
2018	000000002024-11-22-0000November 22	Disclosure norms	Nate Soares, executive director of MIRI, publishes MIRI's 2018 update post that announces MIRI's "nondisclosed-by-default" policy for most of its research. The 2018 AI alignment literature review and charity comparison post would discuss the complications created by this policy for evaluating MIRI's research, and so would the 2019 post. In its 2019 fundraiser review, MIRI would mention the nondisclosure-by-default policy as one possible reason for it raising less money in its 2019 fundraiser.
2018	000000002024-11-29-0000November 29	Funding	The Long-Term Future Fund, one of the Effective Altruism Funds, announces a set of grants: $40,000 to Machine Intelligence Research Institute, $10,000 to Ought, $21,000 to AI Summer School, and $4,500 to the AI Safety Unconference.
2018	000000002024-12-17-0000December 17	Publication	The "2018 AI Alignment Literature Review and Charity Comparison" is published on the Effective Altruism Forum. It surveys AI safety work in 2018. It continues an annual tradition of similar blog posts in 2016 and 2017.
2018	000000002024-12-21-0000December 21	Publication	Kelsey Piper writes a lengthy article for the Future Perfect section of Vox titled "The case for taking AI seriously as a threat to humanity" that gives an overview of AI and AI safety. In an interview with 80,000 Hours in February 2019, Piper would say of the article: "I heard from some people that that gave them a clear explanation of what was going on with AI to point to on. That’s something that seems potentially pretty valuable, just to make sure people are on the same page about that."
2019	000000002024-01-01-0000January	Funding	Open Philanthropy grants $250,000 to the Berkeley Existential Risk Initiative (BERI) to temporarily or permanently hire machine learning research engineers dedicated to BERI’s collaboration with the Center for Human-Compatible Artificial Intelligence (CHAI).
2019	000000002024-01-01-0000January	Funding	Open Philanthropy provides a founding grant for the Center for Security and Emerging Technology (CSET) at Georgetown University of $55 million over 5 years. CSET is led by Jason Matheny, formerly of IARPA, dedicated to policy analysis at the intersection of national and international security and emerging technologies. Other founding members include Dewey Murdick from the Chan Zuckerberg Initiative, William Hannas from the CIA, and Helen Toner from Open Philanthropy.
2019	000000002024-01-01-0000January	Conference	The Beneficial AGI 2019 Conference is organized in Puerto Rico by the Future of Life Institute. It is a successor to the 2015 Puerto Rico Conference and the 2017 Asilomar Conference on Beneficial AI, both of which were important milestone conferences.
2019	000000002024-01-01-0000January	Publication	The Future of Humanity Institute publishes a technical report Reframing Superintelligence: Comprehensive AI Services as General Intelligence by K. Eric Drexler (an engineer best known for seminal studies of the potential of molecular nanotechnology). The report introduces to the AI safety community the idea of comprehensive AI services (CAIS) as a possible and likely path to superintelligence. Rohin Shah summarizes the "gargantuan 210 page document" in a post cross-posted to the Alignment Forum and LessWrong.
2019	000000002024-02-01-0000February	Funding	Open Philanthropy grants $2,112,500 to the Machine Intelligence Research Institute (MIRI) over two years. This is part of the first batch of grants decided by the Committee for Effective Altruism Support, which will set "grant sizes for a number of our largest grantees in the effective altruism community, including those who work on long-termist causes." Around the same time, BERI grants $600,000 to MIRI.
2019	000000002024-02-14-0000February 14	Disclosure norms	OpenAI unveils its language-generating system called GPT-2, a system able to write the news, answer reading comprehension problems, and is beginning to show promise at tasks like translation. However, the data or the parameters of the model are not released, under expressed concerns about potential abuse. This would lead to a lot of discussion and controversy.
2019	000000002024-03-11-0000March 11	Organization	OpenAI announces the creation of OpenAI LP, a new “capped-profit” company owned and controlled by the OpenAI nonprofit organization’s board of directors. The new company is purposed to allow OpenAI to rapidly increase their investments in compute and talent while including checks and balances to actualize their mission.
2019	000000002024-03-17-0000March 17	Publication	Paul Christiano's blog post "What failure looks like" is published to LessWrong and the Alignment Forum. The post describes how Christiano thinks AI alignment might fail in the real world: ML training will hyper-optimize for measurable behaviors, giving rise to "greedy" patterns that exploit their own influence and ultimately dominate the behavior of systems. The post would be mentioned by Larks in his 2019 and 2020 alignment literature reviews, nominated for the 2019 review, and generate a lot of discussion in the LessWrong post comments as well as in later posts by others. A post by Ben Pace over a year later would distill the discussion.
2019	000000002024-03-18-0000March 18	Organization	The Stanford Institute for Human-Centered Artificial Intelligence (HAI) launches at Stanford University under the leadership of philosopher John Etchemendy and AI scientist Fei-Fei Li. Partner organizations at launch include AI4All, AI100, AI Index, Center for AI Safety and the Center for the Study of Language and Information. Despite the similarity in name with UC Berkeley's Center for Human-Compatible AI (CHAI), HAI is not focused primarily on AI safety, but on the more general project of making AI beneficial to and supportive of a wider range of human interests; co-founder Fei-Fei Li is involved with partner AI4ALL, that aims to improve diversity and inclusion in AI. The center at Stanford most analogous to CHAI is the Center for AI Safety.
2019	000000002024-04-07-0000April 7	Funding	The Long-Term Future Fund, one of the Effective Altruism Funds, announces a set of 23 grants totaling $923,150. About half the grant money is to organizations or projects directly working in AI safety. Recipients include the Machine Intelligence Research Institute (MIRI), AI Safety Camp, Ought, and a number of individuals working on AI safety projects, including three in deconfusion research.
2019	000000002024-04-11-0000April 11, April 25	Publication	In a two-part podcast with Lucas Perry for the Future of Life Institute, Rohin Shah of the Center for Human-Compatible AI provides an overview of the field of technical AI safety, including different AI safety agendas.
2019	000000002024-05-01-0000May	Funding	Open Philanthropy announces the second class of the Open Philanthropy AI Fellowship, with 8 machine learning researchers in the class, receiving a total of $2,325,000 in grants.
2019	000000002024-06-07-0000June 7	Fictional portrayal	The movie I Am Mother is released on Netflix. According to a comment on Slate Star Codex: "you can use it to illustrate everything from paperclip maximization to deontological kill switches".
2019	000000002024-06-12-0000June 12	Disclosure norms	In a blog post titled "The Hacker Learns to Trust" Connor Leahy explains why he decided against releasing his own replication of OpenAI's GPT-2 model. Leahy says that helpful discussions with OpenAI team members (Jack Clark, Alec Radford and Jeff Wu), and a convincing argument from Buck Shlegeris of MIRI, led him to change his mind. Highlighted text, based on the conversation with Shlegeris: "Because this isn’t just about GPT2. What matters is that at some point in the future, someone will create something truly dangerous and there need to be commonly accepted safety norms before that happens." The post is linkposted to LessWrong, where it attracts more comments.
2019	000000002024-06-13-0000June 13	Publication	The book The AI Does Not Hate You: Superintelligence, Rationality and the Race to Save the World by Tom Chivers is released in the United Kingdom. The book includes a discussion of AI safety (including the AI box experiments), though it is more focused on the rationality community that built around LessWrong and Slate Star Codex.
2019	000000002024-06-20-0000June 20	Publication, organization	The blog post A case for strategy research: what it is and why we need more of it is published by Siebe Rozendal, Justin Shovelain, and David Kristofferson of Convergence Analysis (an organization focused on existential risk strategy, including AI safety strategy); it is published to both LessWrong and the Effective Altruism Forum. This appeaers to be the organization's first publication in its listed publications, although the organization appears to have had a website as far back as December 2015.
2019	000000002024-08-25-0000August 25	Funding	Grantmaking done by Berkeley Existential Risk Initiative (BERI) funded by Jaan Tallinn moves to the newly created Survival and Flourishing Fund (SFF). BERI's grantmaking in this space had previously included AI safety organizations.
2019	000000002024-08-30-0000August 30	Funding	The Long-Term Future Fund, one of the Effective Altruism Funds, announces a set 13 grants totaling $415,697 USD to organizations and individuals. About half the grant money is to organizations or projects working in AI safety and related AI strategy, governance, and policy issues. With the exception of a grant to AI Safety Camp, all the other grants related to AI safety are to individuals.
2019	000000002024-10-08-0000October 8	Publication	The book Human Compatible by Stuart J. Russell (co-author with Peter Norvig of Artificial Intelligence: A Modern Approach and head of the Center for Human-Compatible AI at UC Berkeley) is published by Viking Press. The book is reviewed by The Guardian and interviews with the author are published by Vox and TechCrunch.
2019	000000002024-11-01-0000November	Funding	Open Philanthropy makes a $1 million grant to Ought, double the previous grant of $525,000.
2019	000000002024-11-01-0000November	Funding	Open Philanthropy makes a $705,000 grant to Berkeley Existential Risk Initiative (BERI) to support continued work with the Center for Human-Compatible AI (CHAI) at UC Berkeley. This includes one year of support for machine learning researchers hired by BERI, and two years of support for CHAI.
2019	000000002024-11-06-0000November 6	Publication	An "AI Alignment Research Overview" by Jacob Steinhardt (one of the co-authors of Concrete Problems in AI Safety) is published to LessWrong and the AI Alignment Forum.
2019	000000002024-11-21-0000November 21	Funding	The Long-Term Future Fund, one of the Effective Altruism Funds, announces a set 13 grants totaling $466,000 USD to organizations and individuals. About a quarter of the grant money is to organizations and individuals working on AI safety. With the exception of a grant to AI Safety Camp, all the other grants related to AI safety are to individuals.
2019	000000002024-11-24-0000November 24		Toon Alfrink publishes on LessWrong a postmortem for RAISE, an attempt to build an online course for AI safety. The blog post explains the challenges with running RAISE and the reasons for eventually shutting it down.
2019	000000002024-12-18-0000December 18	Publication	The "2019 AI Alignment Literature Review and Charity Comparison" is published on the Effective Altruism Forum. It surveys AI safety work in 2019. It continues an annual tradition of similar blog posts in 2016, 2017, and 2018. One feature new to the document this year is the author's effort to make it easier for readers to jump to and focus on parts of the document most relevant to them, rather than read it beginning to end. To make this easier, the author ends each paragraph with a hashtag, and lists the hashtags at the beginning of the document.
2020	000000002024-01-01-0000January	Publication	Rohin Shah (the person who started the weekly AI Alignment Newsletter) publishes a blog post on LessWrong (cross-posted to the Alignment Forum) titled "AI Alignment 2018-19 Review" that he describes as "a review post of public work in AI alignment over 2019, with some inclusions from 2018."
2020	000000002024-02-01-0000February	Funding	Open Philanthropy makes grants to AI safety organizations Ought ($1.59 million) and Machine Intelligence Research Institute ($7.7 million) with the money amount determined by the Committee for Effective Altruism Support (CEAS). Other organizations receiving money based on CEAS recommendations at around the same time are the Centre for Effective Altruism and 80,000 Hours, neither of which is primarily focused on AI safety. In April 2020, MIRI would blog about receiving the Open Philanthropy grant (MIRI's "largest grant to date") and reveal that the grant is for two years and that $1.46 million of the grant money comes from Ben Delo, co-founder of the cryptocurrency trading platform BitMEX.
2020	000000002024-02-22-0000February 22	Funding	Jaan Tallinn, a philanthropist who has funded AI safety efforts, and has recently started using the Survival and Flourishing Fund's S-process to allocate money, announces that he has prepared a page explaining his philanthropic policies and making a concrete commitment for 2020-24. Tallinn's commitment is in terms of the larger of $2 million USD and 20,000 times the year-round minimum price of ETH per year in endpoint grants from 2020 to 2024. Based on ETH prices, the ETH-based minimum is the binding factor for 2020 and likely even more so for future years; and is expected to set a pretty high floor for funding in the AI safety field.
2020	000000002024-03-01-0000March	Publication	The Precipice: Existential Risk and the Future of Humanity by Toby Ord (affiliated with the Future of Humanity Institute and with Oxford University) is published by Hachette Books. The book covers risks including artificial intelligence, biological risks, and climate change. The author appears on podcasts to talk about the book, for Future of Life Institute and 80,000 Hours.
2020	000000002024-03-02-0000March 2	Funding	Berkeley Existential Risk Initiative (BERI) grants $300,000 to MIRI. In a blog post in April, MIRI would write: "I’ll note that at the time of our 2019 fundraiser, we expected to receive a grant from BERI in early 2020, and incorporated this into our reserves estimates. However, we predicted the grant size would be $600k; now that we know the final grant amount, that estimate should be $300k lower."
2020	000000002024-04-02-0000April 2 – 3	Publication	Asya Bergal of AI Impacts publishes a series of interviews (done along with Robert Long) of people optimistic about AI safety, along with a summary post to LessWrong. Their one-sentence summary: "Relative optimism in AI often comes from the belief that AGI will be developed gradually, and problems will be fixed as they are found rather than neglected."
2020	000000002024-04-14-0000April 14	Funding	The Long-Term Future Fund, one of the Effective Altruism Funds, announces a set of 10 grants totaling $488,350. Detailed grant writeups are not available at the time of announcement due to "factors arising from the ongoing coronavirus pandemic." The grants include a $100,000 grant to MIRI, a grant for an AI safety unconference, and some other AI safety grants as well as grants not directly related to AI safety.
2020	000000002024-04-28-0000April 28	Publication	The fourth edition of Artificial Intelligence: A Modern Approach by Stuart Russell and Peter Norvig is released. The edition has considerably more coverage of the alignment problem than previous editions.
2020	000000002024-05-01-0000May (announcement: May 12)	Funding	Open Philanthropy announces the third class of the Open Philanthropy AI Fellowship, funding 10 researchers for 5 years for a total of $2.3 million; the fellows were selected from a pool of more than 380 applicants. Catherine Olsson, one of the two people at Open Philanthropy responsible for selecting fellows, responds to a question about the selection of fellows as follows: "But the short answer is I think the key pieces to keep in mind are to view the fellowship as 1) a community, not just individual scholarships handed out, and as such also 2) a multi-year project, built slowly."
2020	May	Funding	The Survival and Flourishing Fund publishes the outcome of its recommendation S-process for the first half of 2020. This includes grant recommendations to Machine Intelligence Research Institute (MIRI) and AI Impacts in the area of AI safety, and other grant recommendations to organizations such as Future of Life Institute and Convergence Analysis that work partly in the area.
2020	May (website launch), November 19 (announcement on EA Forum)	Organization	AI Safety Support is launched by Linda Linsefors and JJ Hepburn. The website launches in May 2020 and it is announced on the EA Forum on November 19. Initiatives include online AI safety discussion days, a list of AI safety resources, a mentorship program, and the AI Alignment Slack.
2020	000000002024-05-28-0000May 28 (release), June and July (discussion and exploration)		OpenAI releases the natural language model GPT-3 on GitHub and uploads to the ArXiV the paper Language Models are Few-Shot Learners explaining how GPT-3 was trained and how it performs. Games, websites, and chatbots based on GPT-3 are created for exploratory purposes in the next two months (mostly by people unaffiliated with OpenAI), with a general takeaway that GPT-3 performs significantly better than GPT-2 and past natural language models. Commentators also note many weaknesses such as: trouble with arithmetic because of incorrect pattern matching, trouble with multi-step logical reasoning even though it could do the individual steps separately, inability to identify that a question is nonsense, inability to identify that it does not know the answer to a question, and picking up of racist and sexist content when trained on corpuses that contain some such content. Writing from an AI safety perspective, Abram Demski asks: How "honest" is GPT-3? Rohin Shah discusses GPT-3 in Alignment Newsletter #102. A post by Andy Jones wonders whether we are in an "AI overhang" with GPT-3 being the trigger for "100x larger projects at Google, Facebook and the like, with timelines measured in months."
2020	000000002024-05-30-0000May 30 (submission), June 11 (date in paper)	Publication	The paper "AI Research Considerations for Human Existential Safety (ARCHES)" by Andrew Critch of the Center for Human-Compatible AI (CHAI) and David Krueger of the Montreal Institute for Learning Algorithms (MILA) is uploaded to the ArXiV. MIRI's July 2020 newsletter calls it "a review of 29 AI (existential) safety research directions, each with an illustrative analogy, examples of current work and potential synergies between research directions, and discussion of ways the research approach might lower (or raise) existential risk." Critch is interviewed about the paper on the AI Alignment Podcast released September 15.
2020	000000002024-05-31-0000May 31		DeepMind's Victoria Krakovna writes a blog post (cross-posted to LessWrong and the Alignment Forum) about possible takeaways from the COVID-19 pandemic for slow AI takeoff.
2020	000000002024-07-27-0000July 27		In a blog post comment (cross-posted between LessWrong and the Alignment Forum), Gwern Branwen talks about the scaling hypothesis underlying OpenAI's work, and the fact that Google Brain and DeepMind refuse to accept or invest in the hypothesis despite increasing amounts of evidence favoring it. The comment is promoted by Raymond Arnold (LessWrong staff member) and gets heavily upvoted.
2020	000000002024-09-03-0000September 3	Funding	The Long-Term Future Fund, one of the Effective Altruism Funds, announces a set of 12 grants totaling $394,200. Two of the grants are to AI safety organizations: the Center for Human-Compatible AI and AI Impacts. 7 of the 8 grants made to indiividuals are in the domain of AI safety.
2020	000000002024-09-18-0000September 18	Publication	Ajeya Cotra of Open Philanthropy makes public a draft report prepared by her on AI timelines for Open Philanthropy, announcing it on LessWrong and the Effective Altruism Forum. The report would be referenced in future discussions of AI timelines. Holden Karnofsky would write a "layperson-compatible summary" in 2021. Matthew Barnett would write in 2022 that it "is the most useful, comprehensive report about AI timelines I've seen so far."
2020	000000002024-09-28-0000September 28	Publication	Richard Ngo publishes "AGI safety from first principles" as a collection of Alignmment Forum blog posts, also available as a PDF, that makes the case for AGI safety. Rohin Shah approvingly discusses it in the Alignment Newsletter, summarizing its argument as the "second species argument", i.e., that sufficiently advanced AGI would make humanity the second species on earth.
2020	000000002024-10-06-0000October 6	Publication	The book The Alignment Problem: Machine Learning and Human Values by Brian Christian is published. Rohin Shah reviews the book for LessWrong (cross-posted to the Alignment Forum).
2020	000000002024-10-22-0000October 22	Publication	Scott Garrabrant publishes (cross-posted to LessWrong and the Effective Altruism Forum) a blog post titled "Introduction to Cartesian Frames" that is is first post in a sequence about Cartesian frames, a new conceptual framework for thinking about agency.
2020	000000002024-11-01-0000November	Funding	The Survival and Flourishing Fund publishes the outcome of its recommendation S-process for the second half of 2020. This includes grant recommendations to several organizations working mostly or partly in AI safety such as Machine Intelligence Research Institute, Center for Human-Compatible AI, Berkeley Existential Risk Initiative, Stanford Existential Risk Initiative, Median Group, Future of Humanity Institute, and Future of Life Institute.
2020	000000002024-11-28-0000November 28	Funding	The Long-Term Future Fund, one of the Effective Altruism Funds, announces a set of 11 grants totaling $505,000. About half of the grant money is for work related to AI safety, and many of the remaining grants are also loosely related to AI, machine learning, and the long-term trajectory of humanity.
2020	000000002024-12-21-0000December 21	Publication	The "2020 AI Alignment Literature Review and Charity Comparison" by Larks is published to the Effective Altruism Forum and cross-posted to LessWrong. It surveys AI safety work in 2020. It continues a tradition of similar posts starting 2016, and follows the same format as 2019.
2020 – 2021	December 2020  – January 2021		Several safety-focused people working at OpenAI announce their departure, including Dario Amodei (who, with many of the others leaving, would co-found Anthropic) and Paul Christiano (who would go on to found the Alignment Research Center). Mira Murati of OpenAI becomes "senior vice president of Research, Product, and Partnerships" as part of "greater focus on the integration of research, product, and safety." Jan Leike joins to lead the alignment effort. The departures are discussed on Reddit and LessWrong. Almost three years later, an open letter allegedly from former OpenAI employees that is critical of Sam Altman's leadership would emerge, with it being likely that this includes employees who left during this tumultous time.
2021	000000002024-01-01-0000January	Funding	Open Philanthropy grants $11,355,246 over five years to the Center for Human-Compatible AI (CHAI) at UC Berkeley. This renews the original founding support in August 2016.
2021	000000002024-01-01-0000January	Funding	Open Philanthropy grants $8 million to the Center for Security and Emerging Technology (CSET) to augment its 5-year, $55 million funding provided in January 2019. Though CSET's activities go beyond AI safety, this grant is specifically for work at the intersection of security and artificial intelligence.
2021	000000002024-02-01-0000February	Funding	Open Philanthropy makes grants to three researchers in the field of adversarial robustness research: Aleksander Madry at MIT and Dawn Song and David Wagner at UC Berkeley.
2021	000000002024-03-18-0000March 18	Organization	The Nonlinear Fund by Kat Woods (co-founder of Charity Entrepreneurship, Charity Science Health, and Charity Science Outreach) and Emerson Spartz (founder of MuggleNet and Dose Media) is announced in a blog post on the Effective Altruism Forum. The Fund plans to generate, identify, and evaluate high-leverage AI safety opportunities, then make them happen using a variety of tools including grantmaking, advocacy, RFPs, and incubation.
2021	000000002024-04-01-0000April (original publication), December (review), June 2022 (ArXiV upload)	Publication	Joseph Carlsmith of Open Philanthropy publishes a draft post "Is power-seeking AI an existential risk?" In December, a compilation of reviews of the post is published by Carlsmith. In June 2022, the post is uploaded to the ArXiV. In a September 2022 blog post, the FTX Future Fund, when announcing its Worldview Prize, would describe this publication as one of the past publications of the standard that would have qualified for the prize.
2021	000000002024-04-01-0000April	Funding	For the fourth class of the Open Philanthropy AI Fellowship, Open Philanthropy provides a total of approximately $1 million over five years to four fellows.
2021	000000002024-04-26-0000April 26	Organization	Paul Christiano (who had left OpenAI at the end of January) announces that he is now working at the Alignment Research Center (ARC), a nonprofit he started that currently has only him. ARC focuses on theoretical research with the hope that it will "yield big alignment improvements within the next few years" that "will be integrated into practice at leading ML labs."
2021	000000002024-04-30-0000April 30  – 000000002024-05-01-0000May 1	Opinion	Paul Christiano conducts an Ask Me Anything (AMA) cross-posted between LessWrong and the Alignment Forum.
2021	000000002024-05-13-0000May 13	Funding	MIRI announces two major donations to it: $15,592,829 in MakerDAO (MKR) from an anonymous donor with a restriction to spend a maximum of $2.5 million per year till 2024, and the remaining funds available in 2025, and 1050 ETH from Vitalik Buterin, worth $4,378,159.
2021	000000002024-05-26-0000May 26	Funding	The Long-Term Future Fund announces its May 2021 funding recommendations, allocating $1,650,795. The majority of grant money is to individuals, and over half the grant money is specific to AI safety (other grants include grants related to biosecurity and pandemic preparedness, and general longtermism).
2021	000000002024-05-28-0000May 28	Organization	The launch of a new AI safety organization, Anthropic, is announced in the Vox Future Perfect newsletter. The organization's founding team include siblings Dario and Daniela Amodei and other people who left OpenAI in December 2020 and January 2021. It starts off with $124 million in Series A funding led by Jaan Tallinn and including Dustin Moskovitz, Eric Schmidt, and James McClave. Anthropic's focus is on building tools that help AI researchers understand AI programs.
2021	May/June	Funding	The Survival and Flourishing Fund publishes the outcome of its recommendation S-process for the first half of 2021 for funders Jaan Tallinn and Jed McCaleb (by this point, the SFF no longer has any of its own funds so it is acting purely as a "virtual fund"). This includes grant recommendations to several organizations working mostly or partly in AI safety such as AI Impacts, AI Safety Support, Centre for the Study of Existential Risk, Convergence Analysis, and the BERI-FHI collaboration. A total of about $9.8 million is granted in this round.
2021	June 15 (announcement), June 25 (workshop)		AI Impacts runs an AI Vignettes Workshop, organized by Katja Grace and Daniel Kokotajlo. The output of this and other similarly themed workshops is available as the AI Vignettes Project, that describes itself as "an ongoing effort to write concrete plausible future histories of AI development and its social impacts."
2021	000000002024-07-01-0000July	Funding	The Long-Term Future Fund makes grant recommendations; these would get announced in January 2022. Several of the grants, particularly to individuals, are for AI safety work.
2021	000000002024-07-01-0000July onward	Publication	Holden Karnofsky, the long-term-focused co-CEO of Open Philanthropy, publishes a series of blog posts called "The Most Important Century" on his personal website, Cold Takes. The series gives "transformative AI" a central role in the long-term trajectory of humanity, breaking down potential outcomes from transformative AI as a world of digital people, a world of misaligned AI, and something else.
2021	000000002024-10-05-0000October 5	Organization	Redwood Research, that appears to have scaled up its activities relatively recently (the oldest Wayback Machine snapshot is from September 22, though Nate Thomas and Bill Zito appear to have started working there in 2019), announces itself in a post cross-posted across LessWrong, the Alignment Forum, and the Effective Altruism Forum. They focus on prosaic alignment approaches and focus on applied alignment research.
2021	000000002024-10-17-0000October 17	Newsletter	The first issue of Dan Hendrycks' ML safety newsletter is published. New issues of the newsletter are published every 1-3 months. The newsletter would be mentioned in a list of existing newsletters in an announcement for another newsletter.
2021	November	Funding	The Survival and Flourishing Fund publishes the outcome of its recommendation S-process for the second half of 2021 for funders Jaan Tallinn, Jed McCaleb, and The Casey and Family Foundation, represented by David Marble (by this point, the SFF no longer has any of its own funds so it is acting purely as a "virtual fund"). This includes grant recommendations to several organizations working mostly or partly in AI safety such as Ought and AI Safety Camp, as well as to the Long-Term Future Fund that would further reallocate the funds in its upcoming grant rounds. A total of $9.6 million is distributed in this round.
2021	November	Funding	Open Philanthropy makes a grant of $9.42 million to Redwood Research.
2021	December	Funding	Open Philanthropy makes a grant of $2,537,600 to the Centre for the Governance of AI.
2021	000000002024-12-23-0000December 23	Publication	The "2021 AI Alignment Literature Review and Charity Comparison" by Larks is published on the Effective Altruism Forum and cross-posted to LessWrong. It surveys AI safety work in 2021. It continues a tradition of similar posts starting 2016, and follows the same format as 2019 and 2020, with one new addition being a "Organisation second preference" section.
2021	December (blog posts announcing the org in August 2022)	Organization	Encultured AI, a for-profit video game company with a public benefit mission related to long-term survival and flourishing, is started by Andrew Critch and Nick Hay. The founding team claims that the video game environment would be a good sandbox to experiment with and learn more about AI safety and alignment.
2022	000000002024-02-28-0000February 28	Funding	The FTX Future Fund, bankrolled by the wealth of Sam Bankman-Fried and his FTX co-founders, announces its launch and the plan to distribute at least $100 million in 2022. At the time of the launch announcement, the areas of interest page lists artificial intelligence as one of the areas of interest,. Also, the project ideas pages lists AI alignment prizes as one of the project ideas it is excited to fund. Two other AI-related topics are listed: AI-based cognitive aids and AI ethics; both are somewhat connected to AI safety, the latter more so.
2022	March	Funding	Open Philanthropy makes two grants related to AI safety: to MIT for Neil Thompson's work on AI trends and impacts research, and to Hofvarpnir Studios to maintain a compute cluster for Jacob Steinhardt's lab and CHAI for their AI safety work.
2022	000000002024-03-26-0000March 26	Project	The AI Safety Arguments Competition is announced on LessWrong and the AI Alignment Forum. The competition "addresses shorter arguments (paragraphs and one-liners) with a total prize pool of $20K. The prizes will be split among, roughly, 20-40 winning submissions."
2022	000000002024-03-26-0000March 26 (first Wayback Machine snapshot), September 12 (first LessWrong post)	Organization	AI safety organization Apart Research launches. The oldest Wayback Machine snapshot of this site is from March 26, 2022. The first LessWrong post for the organization is published on September 12.
2022	April 8}}	Organization	Conjecture, a new AI alignment research startup, announces its founding on the Alignment Forum and LessWrong. Conjecture is a "for-profit company with products on the market".
2022	April, May		Multiple companies release AI tools: OpenAI's DALL-E-2, Google's PaLM, SalesForce's CodeGen, and DeepMind's Gato. Around this time, and possibly partly influenced by these releases, the distribution of estimates of the arrival of artificial general intelligence moves to earlier in time.
2022	000000002024-04-29-0000April 29	Funding	Anthropic raises $580 million in Series B funding; the round is led by Sam Bankman-Fried, CEO of FTX, and also includes participation from Caroline Ellison, Jim McClave, Nishad Singh, Jaan Tallinn, and the Center for Emerging Risk Research (CERR). Anthropic (co-founded by people from OpenAI with AI safety backgrounds) is a private corporation working on making AI more reliable and explainable, and had previously raised $124 million in Series A funding led by Jaan Tallinn and including Dustin Moskovitz.
2022	June	Funding	The Survival and Flourishing Fund publishes the outcome of its recommendation S-process for the first half of 2022, with Jaan Tallinn being the sole funder in this round. This includes grant recommendations to several organizations working mostly or partly in AI safety such as Redwood Research, Median Group, and Nonlinear.
2022	000000002024-06-30-0000June 30	Funding	The FTX Future Fund publishes an update on it grantmaking so far, with AI safety as one of the cause areas it has funded. It estimates a total of 76 grants for "Artificial Intelligence" and a total spend of $20 million (much of which is related to AI safety). Several example grants are listed; some grantees working in AI safety are: Ought, ML Safety Scholars Program, and AI Impacts.
2022	July, August	Organization	The Center for AI Safety starts being active around this time (its website had a "Coming Soon" in December 2021). This is distinct from the Stanford Center for AI Safety, despite the similar name.
2022	000000002024-08-01-0000August	Publication	80,000 Hours publishes a problem profile for AI safety / AI alignment work titled "Preventing an AI-related catastrophe: AI might bring huge benefits — if we avoid the risks" and written by Benjamin Hilton. A summary by an unaffiliated individual is published to LessWrong a few months later.
2022	000000002024-08-24-0000August 24	Publication	OpenAI publishes a blog post titled "Our approach to alignment research" where it describes its alignment plans. The subtitle says: "Our goal is to build a sufficiently aligned AI system that can help us solve all other alignment problems." There would be several followups to this, including a personal blog post by co-author Jan Leike, a post by Rob Bensinger (on behalf of Eliezer Yudkowsky and Nate Soares) asking Anthropic and Deepmind to follow suit and asking readers to critique OpenAI's plans, and posts by Akash Wasil and Søren Elverlin with their thoughts on OpenAI's plan.
2022	000000002024-08-28-0000August 28	Publication	A lengthy blog post by Thomas Larsen (with some help from Eli) goes over what different AI safety groups are doing and why.
2022	000000002024-09-23-0000September 23	Project	The FTX Future Fund announces its Worldview Prize for submissions that challenge the Future Fund's thinking on the trajectory of AI and the importance of AI safety.
2022	000000002024-10-19-0000October 19	Newsletter	The first episode of the ML safety updates newsletter/podcast is published.
2022	000000002024-11-10-0000November 10	Funding	In response to the collapse of FTX, the source of funds for the FTX Future Fund, amidst allegations of fraud on the part of FTX leadership, the entire FTX Future Fund team resigns. The post says: "We are devastated to say that it looks likely that there are many committed grants that the Future Fund will be unable to honor. We are so sorry that it has come to this. We are no longer employed by the Future Fund, but, in our personal capacities, we are exploring ways to help with this awful situation. We joined the Future Fund to support incredible people and projects, and this outcome is heartbreaking to us." The FTX Future Fund had funded several projects and organizations related to AI safety.
2022	000000002024-11-10-0000November 10	Funding	Holden Karnofsky publishes the first public announcement by Open Philanthropy (the other major founder of the EA ecosystem) regarding the FTX collapse. Karnofsky clarifies that funds directed by Open Philanthropy are not invested in or otherwise exposed to FTX or related entities. He also states that if the FTX Foundation stops funding longtermist and effective altruist projects (including projects in AI safety), then Open Philanthropy would have to consider a substantially larger set of funding opportunities, raising their bar for longtermist grantmaking. The post also announces a temporary pause on longtermist grantmaking while awaiting more clarity on the situation.
2022	000000002024-11-13-0000November 13	Funding	Members of non-profit Nonlinear announce funding for Future Fund grantees affected by the FTX collapse, offering a small budget to help grantees cope with financial crisis. This includes grantees working in AI safety, a cause area that the FTX Future Fund had prioritized.
2022	000000002024-11-15-0000November 15	Funding	In response to the collapse of the FTX Future Fund, Open Philanthropy announces that it is seeking applications from FTX Future Fund grantees affected by the collapse. This includes grantees working in AI safety, a cause area that the FTX Future Fund had prioritized.
2022	000000002024-11-21-0000November 21	Project	Open Philanthropy pre-announces that it plans to conduct an AI Worldviews Contest in 2023. This is in lieu of the (now de facto cancelled due to the FTX collapse) Worldview Prize organized by the FTX Future Fund, that had already received several submissions. The post clarifies that this is not a continuation of the Future Fund contest, and various details including the prize pool, judges, and evaluation criteria will be different, but notes: "We expect it will be easy to adapt Future Fund submissions for the Open Phil contest." The post further says: "We are releasing this post now to try to alleviate some of the fear, uncertainty, and doubt surrounding the old Future Fund competition and also to capture some of the value that has already been generated by the Future Fund competition before it dissipates."
2022	000000002024-11-23-0000November 23	Project	The AI safety Mentors and Mentees program is announced to the EA Forum.
2022	000000002024-11-30-0000November 30	AI advance	OpenAI releases GPT-3.5 as well as a conversational chatbot called ChatGPT based on GPT-3.5. This conversational chatbot would grow to being used by millions of users in a few days, and unleash a flurry of activity by competitors to speed up thieir own AI work and invest more in chatbots.
2022	000000002024-12-01-0000December 1	Publication	On behalf of his MIRI colleagues Eliezer Yudkowsky and Nate Soares, Rob Bensinger, who handles MIRI's research communications, publishes a blog post challenging organizations such as Anthropic and Deepmind to follow in OpenAI's footsteps by publicly writing up their alignment plans. The post also challenges readers to write their own critiques of OpenAI's publicly documented alignment plan without looking at the critique that Eliezer and Nate are currently working on writing.
2022	000000002024-12-02-0000December 2	Publication	A blog post by Zvi Mowshowitz, cross-posted to LessWrong, documents the several successes with jailbreaking ChatGPT already (on the day of its release) to get it to reveal information that it tries not to reveal as it could be harmful. This includes directions for AI takeover, bullying, inflicting pain, and making dangerous chemicals. Eliezer Yudkowsky is quoted in the post, and both he and Paul Christiano participate in the comments.
2022	000000002024-12-01-0000December	Project, Publication	Conjecture hosts conversations to help nail down disagreements around AGI and alignment, with the goal of publishing the conversations publicly to advance the conversation. They invite people from OpenAI, DeepMind, Anthropic, Open Philanthropy, FTX Future Fund, ARC, and MIRI, as well as some independent researchers to participate in the discussions. While most people are unable to participate as their organizations prevent them from sharing information publicly, people from the Alignment Research Center, Deepmind, and OpenAI agree to participate. A summary and the conversations are published in February 2023 to LessWrong and the Alignment Forum.
2023	000000002024-01-18-0000January 18, January 20	Publication	On January 18, Sam Harris is interviewed by Connie Loizos about OpenAI, with the recent release of ChatGPT being a key focus of the interview. On January 20, an excerpt of the transcript focused on the parts relevant to AI safety is published to LessWrong, attracting several comments including some from Paul Christiano.
2023	000000002024-01-20-0000January 20	Project	The AI Safety Training website (aisafety.training) is announced to LessWrong and the Effective Altruism Forum. The website describes it as follows: "A database of training programs, courses, conferences, and other events for AI existential safety." The database is managed by Airtable and accepts suggestions for things that it has missed.
2023	000000002024-02-01-0000February 1	Publication	Maheen Shermohammed and Vael Gates publish a report based on interviews with 97 AI researchers and share a summary on LessWrong and the Effective Altruism Forum. The questions include questions about the timeline of AGI and the validity of the alignment problem.
2023	000000002024-02-20-0000February 20	Publication	Eliezer Yudkowsky appears on the Bankless podcast for an interview lasting a little under two hours, where he shares his relatively pessimistic views about the likelihood of catastrophic AGI with his hosts, neither of whom is deep into AI safety. A full transcript is published to LessWrong and the Alignment Forum a few days later. The podcast gets a lot of traction, eliciting several reactions, and leads to a followup Q&A on Twitter Spaces.
2023	000000002024-02-21-0000February 21	Publication	Zvi Mowshowitz publishes (cross-posted to LessWrong and his personal blog) the first of what will become weekly posts about AI; as somebody concerned about AI safety, his posts look at AI developments from a safety-focused lens. The idea behind this series is to copy over to the AI safety area his earlier success with weekly posts about COVID-19 during the COVID-19 pandemic. The first post is about Sydney (the original name of Bing Chat) and Bing.

Visual and numerical data

Mentions on Google Scholar

The following table summarizes per-year mentions on Google Scholar as of May 13, 2021.

Year	artificial intelligence safety	ethical artificial intelligence	friendly artificial intelligence	global risk artificial intelligence
1980	683	478	463	252
1985	1,340	713	1,110	537
1990	3,060	1,410	2,210	1,490
1995	4,160	2,110	2,930	2,510
2000	7,600	3,560	4,860	4,900
2002	8,080	4,510	5,950	6,380
2004	11,400	4,940	6,760	8,440
2006	13,300	6,240	7,820	10,700
2008	16,300	7,770	10,200	12,100
2010	20,500	9,170	12,500	16,400
2012	26,000	11,300	14,800	20,900
2014	29,800	13,300	16,400	24,600
2016	34,600	16,500	18,400	30,500
2017	39,300	20,200	21,000	36,200
2018	47,300	25,300	25,300	43,300
2019	52,700	27,600	28,500	48,300
2020	43,700	31,700	29,200	48,800

Google trends

The chart below shows Google trends data for AI safety (search term) from January 2004 to January 2021, when the screenshot was taken.

Google Ngram Viewer

The chart below shows Google Ngram Viewer data for AI safety from 1900 to 2020.

Meta information on the timeline

How the timeline was built

The initial version of the timeline was written by Issa Rice. The timeline was later expanded considerably by Vipul Naik. Sebastian Sanchez also added graphs to the timeline.

Issa likes to work locally and track changes with Git, so the revision history on this wiki only shows changes in bulk. To see more incremental changes, refer to the commit histories at the old location and the new location.

Funding information for this timeline is available.

Feedback and comments

Feedback for the timeline can be provided at the following places:

• LessWrong link post

What the timeline is still missing

• I think some of Abram Demski's work (radical probabilism?) should be included, but I don't know enough to decide which things are most important. I think similar things can be said for other technical researchers too.
• The Matrix
• maybe more at [1]
• maybe more from the "earlier work on the topic" at Richard Posner on AI Dangers
• maybe more from AI Researchers On AI Risk but no dates; use as starting point to know what people to look for
• more AI box results at [2] but unfortunately no dates
• stuff in [3] and [4]
• siren/marketing worlds
• TDT/UDT
• Paul Christiano AI alignment blog. Also, more on Christiano's trajectory in AI safety
• The launch of Ought
• Translations of Superintelligence?
• universal prior/distant superintelligences stuff
• Steven Pinker?
• AI summer school
• when did the different approaches to alignment come along?
• the name change from "friendly AI" to "AI safety" and "AI alignment" is probably worth adding, though this was gradual so kind of hard to pin down as an event. See also this comment.
• https://youtu.be/pClSjljMKeA?t=3605 -- Robin Hanson talks about "booms" in concerns about robots.

Pingbacks

• The Era of Global Risk: An Introduction to Existential Risk Studies, by SJ Beard, Martin Rees, Catherine Richards, and Clarissa Rios Rojas Reference N°5

• Evan Sandhoefner tweet (December 26, 2017) emphasizes the accelerating nature of progress on AI safety by screenshotting the part till 2002 that makes up 1/5 of the timeline at the time.

See also

Timelines of organizations working in AI safety

• Timeline of Machine Intelligence Research Institute
• Timeline of Future of Humanity Institute
• Timeline of OpenAI
• Timeline of Berkeley Existential Risk Initiative

Other timelines related to AI

• Timeline of Google Brain
• Timeline of DeepMind
• Timeline of machine learning
• Timeline of artificial intelligence
• Timeline of robotics

Other timelines related to topics with potential applications to AI safety

• Timeline of decision theory

Other timelines about cause areas prioritized in effective altruism

• Timeline of global health
• Timeline of animal welfare and rights
• Timeline of wild animal suffering
• Timeline of effective altruism

Other related timelines

• Timeline of the rationality community

External links

• Funding information for AI safety on Vipul Naik's donations portal