Difference between revisions of "Timeline of AI safety"

From Timelines
Jump to: navigation, search
(→‎Visual data)
(→‎Visual data)
Line 43: Line 43:
 
=== Google trends ===
 
=== Google trends ===
  
−
[[ |thumb|center|700px]]
+
[[File:AI safety.jpeg|thumb|center|700px]]
  
 
=== Google Ngram Viewer ===
 
=== Google Ngram Viewer ===

Revision as of 19:05, 15 January 2021

This is a timeline of AI safety. AI safety is the field focused on reducing risks from artificial intelligence (AI).[1][2]

Big picture

Overall summary

Time period Development summary More details
Until 1950 Fictional portrayals only Most discussion of AI safety is in the form of fictional portrayals. It warns of the risks of robots who, through either stupidity or lack of goal alignment, no longer remain under the control of humans.
1950 to 2000 Scientific speculation + fictional portrayals During this period, discussion of AI safety moves from merely being a topic of fiction to one that scientists who study technological trends start talking about. The era sees commentary by I. J. Good, Vernor Vinge, and Bill Joy.
2000 to 2012 Birth of AI safety organizations, close connections with transhumanism This period sees the creation of the Singularity Institute for Artificial Intelligence (SIAI) (which would later become the Machine Intelligence Research Institute (MIRI)) and the evolution of its mission from creating friendly AI to reducing the risk of unfriendly AI. The Future of Humanity Institute (FHI) and Global Catastrophic Risk Institute (GCRI) are also founded. AI safety work during this time is closely tied to transhumanism and has close connections with techno-utopianism. Peter Thiel and Jaan Tallinn are key funders of the early ecosystem.
2013 to present Mainstreaming of AI safety, separation from transhumanism SIAI changes name to MIRI, sells off the "Singularity" brand to Singularity University, grows considerably in size, and gets a lot of funding. Superintelligence, the book by Nick Bostrom, is released. The Future of Life Institute (FLI) and OpenAI are started, and the latter grows considerably. New organizations founded include the Center for the Study of Existential Risk (CSER), Leverhulme Centre for the Future of Intelligence (CFI), Future of Life Institute (FLI), OpenAI, Center for Human-Compatible AI (CHAI), Berkeley Existential Risk Initiative (BERI), Ought, and the Center for Security and Emerging Technology (CSET). OpenAI in particular becomes quite famous and influential. Prominent individuals such as Elon Musk, Sam Altman, and Bill Gates talk about the importance of AI safety and the risks of unfriendly AI. Key funders of this ecosystem include Open Philanthropy and Elon Musk.

Highlights by year (2013 onward)

Year Highlights
2013 Research and outreach focused on forecasting and timelines continue. Connections with the nascent effective altruism movement strengthen. The Center for the Study of Existential Risk and the Foundational Research Institute launch.
2014 Superintelligence: Paths, Dangers, Strategies by Nick Bostrom is published. The Future of Life Institute is founded and AI Impacts launches. AI safety gets more mainstream attention, including from Elon Musk, Stephen Hawking, and the fictional portrayal Ex Machina. While forecasting and timelines remain a focus of AI safety efforts, the effort shifts toward the technical AI safety agenda, with the launch of the Intelligent Agent Foundations Forum.
2015 AI safety continues to get more mainstream, with the founding of OpenAI (supported by Elon Musk and Sam Altman) and the Leverhulme Centre for the Future of Intelligence, the Open Letter on Artificial Intelligence, the Puerto Rico conference, and coverage on Wait But Why. This also appears to be the last year that Peter Thiel donates in the area.
2016 Open Philanthropy makes AI safety a focus area; it would ramp up giving in the area considerably starting around this time. The landmark paper "Concrete Problems in AI Safety" is published, and OpenAI's safety work picks up pace. The Center for Human-Compatible AI launches. The annual tradition of LessWrong posts providing an AI alignment literature review and charity comparison for the year begins. AI safety continues to get more mainstream, with the Partnership on AI and the Obama administration's efforts to understand the subject.
2017 This is a great year for cryptocurrency prices, causing a number of donations to MIRI from people who got rich through cryptocurrency. The AI safety funding and support landscape changes somewhat with the launch of the Berkeley Existential Risk Initiative (BERI) (and funding of its grants program by Jaan Tallinn) and the Effective Altruism Funds, specifically the Long-Term Future Fund. Open Philanthropy makes several grants in AI safety, including a $30 million grant to OpenAI and a $3.75 million grant to MIRI. AI safety attracts dismissive commentary from Mark Zuckerberg, while Elon Musk continues to highlight its importance. The year begins with the Asilomar Conference and the Asilomar AI Principles, and initiatives such as AI Watch and the AI Alignment Prize begin toward the end of the year.
2018 Activity in the field of AI safety becomes more steady, in terms of both ongoing discussion (with the launch of the AI Alignment Newsletter, AI Alignment Podcast, and Alignment Forum) and funding (with structural changes to the Long-Term Future Fund to make it grant more regularly, the introduction of the annual Open Philanthropy AI Fellowship grants, and more grantmaking by BERI). Near the end of the year, MIRI announces its nondisclosure-by-default policy. Ought, Median Group, and the Stanford Center for AI Safety launch during the year.
2019 The Center for Security and Emerging Technology (CSET), that is focused on AI safety and other security risks, launches with a 5-year $55 million grant from Open Philanthropy. The Stanford Institute for Human-Centered Artificial Intelligence (HAI) launches. Grantmaking from the Long-Term Future Fund picks up pace; BERI hands off its grantmaking of Jaan Tallinn's money to the Survival and Flourishing Fund (SFF). Open Philanthropy begins using the Committee for Effective Altruism Support to decide grant amounts for some of its AI safety grants, including grants to MIRI. OpenAI unveils its GPT-2 model but does not release the full model initially; this sparks discussion on disclosure norms.
2020 Andrew Critch and David Krueger release their ARCHES paper. OpenAI unveils GPT-3, leading to further discussion of AI safety implications. AI Safety Support launches. The funding ecosystem continues to mature: Open Philanthropy and the Survival and Flourishing Fund continue to make large grants to established organizations, while the Long-Term Future Fund increasingly shifts focus to donating to individuals.

Visual data

Google trends

AI safety.jpeg

Google Ngram Viewer

AI safety ngram.jpeg

Full timeline

Inclusion criteria

Here is a partial list of thoughts on what rows we've included:

  • For AI safety organizations, we have tried to include a row for their launch where available. When an exact launch date is not available, we have tried to include an approximate launch date based on the history of the website or other public announcements.
  • Most grants from Open Philanthropy in AI safety are included. Some grants may be omitted because of small size. For recent grants, we may not have added them yet.
  • We have tried to include important fictional portrayals and scientific speculation related to AI before 2000, as comprehensively as possible. For the period from 2000 onward, we have been more selective, and focused more on relatively notable commentary or commentary that played a role in changing the AI safety conversation in specific communities.

Timeline

Year Month and date Event type Details
1630–1650 Fictional portrayal The publication of the story of the Golem of Chełm dates to around this period. Wikipedia: "Golems are not intelligent, and if commanded to perform a task, they will perform the instructions literally. In many depictions Golems are inherently perfectly obedient. In its earliest known modern form, the Golem of Chełm became enormous and uncooperative. In one version of this story, the rabbi had to resort to trickery to deactivate it, whereupon it crumbled upon its creator and crushed him."
1818 Fictional portrayal The novel Frankenstein is published. Frankenstein pioneers the archetype of the artificial intelligence that turns against its creator, and is sometimes discussed in the context of an AI takeoff.[3][4][5]
1863 June Publication In Darwin among the Machines, Samuel Butler raises the possibility that intelligent machines will eventually supplant humans as the dominant form of life.[6]
1920 Fictional portrayal The science fiction play R.U.R. is published. The play introduces the word "robot" to the English language and the plot contains a robot rebellion that leads to human extinction.
1942 March Fictional portrayal The Three Laws of Robotics are introduced by Isaac Asimov in his short story "Runaround".
1947 July Fictional portrayal With Folded Hands, a novelette by Jack Williamson, is published. The novelette describes how advanced robots (humanoids) take over large parts of the world to fulfil their Prime Directive, which is to make humans happy.
1948 Publication In The general and logical theory of automata, John von Neumann articulates the idea of self-improving AI. Notable quote: "There is, however, a certain minimum level where this degenerative characteristic ceases to be universal. At this point automata which can reproduce themselves, or even construct higher entities, become possible."[6] He would expand on this idea further in 1949, coming close to articulating what is now called an "intelligence explosion."[6]
1950 Publication Alan Turing publishes Computing Machinery and Intelligence in the philosophy journal Mind. The paper introduces the concept of the Turing test, the simple idea of which is that a machine would be said to have achieved human-level intelligence if it can convince a human that it is human. The Turing test would become a key part of discussions around benchmarking artificial intelligence, and also enter popular culture over the next few decades.[7][8]
1960 May 6 Publication Norbert Wiener's article Some Moral and Technical Consequences of Automation is published.[9] In 2013, Jonah Sinick would note the similarities between the points raised in this article and the thinking of AI safety leader Eliezer Yudkowsky.[10] Wiener's work would also be cited by Allan Dafoe and Stuart Russell in 2016: "Rather, the risk arises from the unpredictability and potential irreversibility of deploying an optimization process more intelligent than the humans who specified its objectives. This problem was stated clearly by Norbert Wiener in 1960, and we still have not solved it."[11]
1965 Publication I. J. Good originates the concept of intelligence explosion in "Speculations Concerning the First Ultraintelligent Machine".
1966 Fictional portrayal The science fiction novel Colossus by British author Dennis Feltham Jones is published. In the novel, both the United States and the USSR develop supercomputers, called Colossus and Guardian respectively, that they connect with each other to share knowledge. The supercomputers coordinate with each other to become increasingly powerful, and humans belatedly try to regain control.[6]
1966 Publication In Other Worlds than Ours, Cecil Maxwell argues that nations would invest in building powerful intelligent machines and surrender decisionmaking to them, and any nation that succeeded would grow into a major world power. He writes: It seems that, in the forseeable future, the major nations of the world will have to face the alternative of surrendering national control to mechanical ministers, or being dominated by other nations which have already done this. Such a process will eventually lead to the domination of the whole Earth by a dictatorship of an unparalleled type — a single supreme central authority.[6]
1974 Publication In The Ignorance Explosion, Julius Lukasiewicz argues that it is very hard to predict the future after we have machine superintelligence.[6]
1977 Fictional portrayal The science fiction novel The Adolescence of P-1 by Thomas Joseph Ryan is published. It "tells the story of an intelligent worm that at first is merely able to learn to hack novel computer systems and use them to propagate itself, but later (1) has novel insights on how to improve its own intelligence, (2) develops convergent instrumental subgoals for self-preservation and resource acquisition, and (3) learns the ability to fake its own death so that it can grow its powers in secret and later engage in a "treacherous turn"against humans."[6]
1979 Publication In the book Machines Who Think by Pamela McCorduck, Edward Fredkin descibes the challenge of controlling smarter-than-human artificial general intelligences. He describes the high communication bandwidth betweeen superintelligent machines, the goal-directed nature of intelligent systems, and the challenge of aligning the goals of the machines with those of humans.[12]
1984 October 26 Fictional portrayal The American science fiction film The Terminator is released. The film contains the first appearance of Skynet, a "neural net-based conscious group mind and artificial general intelligence" that "seeks to exterminate the human race in order to fulfill the mandates of its original coding".
1985 Fictional portrayal In Robots and Empire, Isaac Asimov introduces the Zeroth Law of Robotics, which states: "A robot may not injure humanity, or through inaction, allow humanity to come to harm." This is analogous to, and must take precedence over, the Fist Law "A robot may not injure a human being, or through inaction, allow a human being to come to harm." The law was self-programmed by the robot R. Daneel Olivaw based on the ideas of R. Giskard Reventlov. In particular, the zeroth law now allows a robot to harm or allow harm to individual humans for the greater good of humanity.
1987 Publication In an article A Question of Responsibility for AI Magazine, Mitchell Waldrop introduces the term machine ethics. He writes: However, one thing that is apparent from the above discussion is that intelligent machines will embody values, assumptions, and purposes, whether their programmers consciously intend them to or not. Thus, as computers and robots become more and more intelligent, it becomes imperative that we think carefully and explicitly about what those built-in values are. Perhaps what we need is, in fact, a theory and practice of machine ethics, in the spirit of Asimov’s three laws of robotics.[13]
1988 October 1 Publication The book Mind Children: The Future of Robot and Human Intelligence by Hans Moravec is published. The book says that machines with human-level intelligenece are feasible within fifty years, and should be thought of as the "mind children" of humans. The book would be listed in a 2013 MIRI blog post in a list of past work on AI safety and the AI takeoff.[14]
1993 Publication Vernor Vinge's article "The Coming Technological Singularity: How to Survive in the Post-Human Era" is published. The article popularizes the idea of an intelligence explosion.[15]
1996 May 1 Publication The book Reflections on Artificial Intelligence: The Legal, Moral and Ethical Dimensions by Blay Whitby is published. A 2013 blog post by MIRI would list this as part of "earlier work on the topic" of AI safety.[14]
1998 December 3 Publication The book Robot: Mere Machine to Transcendent Mind by Hans Moravec is published. The book argues that human levels of intelligence are achievable by 2040, and has an optimistic take on this future: "Intelligent machines, which will grow from us, learn our skills, and share our goals and values, can be viewed as children of our minds." It continues the themes of Moravec's previous book Mind Children, and would be cited in a 2013 MIRI blog post in a list of past work related to AI safety and the AI takeoff.[14]
1999 January 1 Publication The book The Age of Spiritual Machines by Ray Kurzweil is published. The book describes Kurzweil's utopian vision of technological progress and the path to computers achieving superhuman intelligence.
2000 April Publication Bill Joy's article "Why The Future Doesn't Need Us" is published in Wired.
2000 July 27 Organization Machine Intelligence Research Institute (MIRI) is founded as the Singularity Institute for Artificial Intelligence (SIAI) by Brian Atkins, Sabine Atkins (then Sabine Stoeckel) and Eliezer Yudkowsky. The organization's mission ("organization's primary exempt purpose" on Form 990) at the time is "Create a Friendly, self-improving Artificial Intelligence"; this mission would be in use during 2000–2006 and would change in 2007.[16]:3[17]
2001 June 15 Publication Version 1.0 of "Creating Friendly AI" by Eliezer Yudkowsky is published.[18]
2002 March 8 AI box The first AI box experiment by Eliezer Yudkowsky, against Nathan Russell as gatekeeper, takes place. The AI is released.[19]
2002 July 4–5 AI box The second AI box experiment by Eliezer Yudkowsky, against David McFadzean as gatekeeper, takes place. The AI is released.[20]
2002 October 31 Publication Bill Hibbard's Super-Intelligent Machines is published.[21]
2003 Publication Nick Bostrom's paper "Ethical Issues in Advanced Artificial Intelligence" is published. The paper introduces the paperclip maximizer thought experiment.[22]
2004 November 11 Publication In his book Catastrophe: Risk and Response, legal scholar Richard Posner discussses risks from artificial general intelligence at some length. According to a 2013 blog post from the Machine Intelligence Resarch Institute: "His analysis is interesting in part because it appears to be intellectually independent from the Bostrom-Yudkowsky tradition that dominates the topic today. [...] Still, much of Posner’s analysis is consistent with the basic points of the Bostrom-Yudkowsky tradition [...] One major point of divergence seems to be that Posner worries about a scenario in which AGIs become self-aware, re-evaluate their goals, and decide not to be “bossed around by a dumber species” anymore."[14]
2005 Organization The Future of Humanity Institute (FHI) is founded.[23]
2005 Conference The field of machine ethics is delineated at the Fall 2005 Symposium on Machine Ethics held by the Association for Advancement of Artificial Intelligence.[24]
2005 August 21 AI box The third AI box experiment by Eliezer Yudkowsky, against Carl Shulman as gatekeeper, takes place. The AI is released.[25]
2005 Publication The Singularity is Near by inventor and futurist Ray Kurzweil is published. The book builds upon Kurzweil's previous books The Age of Intelligent Machines (1990) and The Age of Spiritual Machines (1999), but unlike its predecessors, uses the term technological singularity introduced by Vinge in 1993. Unlike Bill Joy, Kurzweil takes a very positive view of the impact of smarter-than-human AI and the upcoming (in his view) technological singularity.
2006 November Robin Hanson starts Overcoming Bias.[26] Eliezer Yudkowsky's posts on Overcoming Bias would form seed material for LessWrong, which would grow to be an important community for discussion related to AI safety.
2008 Publication Steve Omohundro's paper "The Basic AI Drives" is published. The paper argues that certain drives, such as self-preservation and resource acquisition, will emerge in any sufficiently advanced AI. The idea would subsequently be defended by Nick Bostrom as part of his instrumental convergence thesis.[27]
2008 Publication Global Catastrophic Risks is published. The book includes Eliezer Yudkowsky's chapter "Artificial Intelligence as a Positive and Negative Factor in Global Risk".[28]
2008 October 13 Publication Moral Machines: Teaching Robots Right from Wrong by Wendell Wallach and Colin Allen is published by Oxford University Press. The book advertises itself as "the first book to examine the challenge of building artificial moral agents, probing deeply into the nature of human decision making and ethics."
2008 November–December Outside review The AI-Foom debate between Robin Hanson and Eliezer Yudkowsky takes place. The blog posts from the debate would later be turned into an ebook by MIRI.[29][30]
2009 February Project Eliezer Yudkowsky starts LessWrong using as seed material his posts on Overcoming Bias.[31] On the 2009 accomplishments page, MIRI describes LessWrong as being "important to the Singularity Institute's work towards a beneficial Singularity in providing an introduction to issues of cognitive biases and rationality relevant for careful thinking about optimal philanthropy and many of the problems that must be solved in advance of the creation of provably human-friendly powerful artificial intelligence". And: "Besides providing a home for an intellectual community dialoguing on rationality and decision theory, Less Wrong is also a key venue for SIAI recruitment. Many of the participants in SIAI's Visiting Fellows Program first discovered the organization through Less Wrong."[32]
2009 December 11 Publication The third edition of Artificial Intelligence: A Modern Approach by Stuart J. Russell and Peter Norvig is published. In this edition, for the first time, Friendly AI is mentioned and Eliezer Yudkowsky is cited.[33][34]
2010 Organization DeepMind is founded by Demis Hassabis, Shane Legg, and Mustafa Suleyman. Legg had previously received the $10,000 Canadian Singularity Institute for Artificial Intelligence Prize.[35]
2010 Organization Vicarious is founded by Scott Phoenix and Dileep George. The company "has publicly expressed some concern about potential risks from future AI development" and the founders are signatories on the FLI open letter.[36]
2011 Publication Baum, Goertzel, and Goertzel's "How Long Until Human-Level AI? Results from an Expert Assessment" is published.[37]
2011 Organization The Global Catastrophic Risk Institute (GCRI) is founded by Seth Baum and Tony Barrett.[38]
2011 Organization Google Brain is started by Jeff Dean, Greg Corrado, and Andrew Ng.
2011–2013 Project Sometime during this period, the Back of the Envelope Guide to Philanthropy, a website created by Gordon Irlam, includes prevention of "hostile artificial intelligence" as a top 10 philanthropic opportunity by impact.[39][40]
2011 September Organization The Oxford Martin Programme on the Impacts of Future Technology (FutureTech) launches.[41]
2013 Publication Luke Muehlhauser's book Facing the Intelligence Explosion is published.[42]
2013 April 13 MIRI publishes an update on its strategy on its blog. In the blog post, MIRI executive director Luke Muehlhauser states that MIRI plans to put less effort into public outreach and shift its research to Friendly AI math research.[43]
2013 July Organization The Center for the Study of Existential Risk (CSER) launches.[44][45]
2013 July Organization The Foundational Research Institute (FRI) is founded. Some of FRI's work discusses risks from artificial intelligence.[46]
2013 July 8 Publication Luke Muehlhauser's blog post Four Focus Areas of Effective Altruism is published. The four focus areas listed are poverty reduction, meta effective altruism, the long term future, and animal suffering. AI safety concerns fall under the "long term future" focus area.[47] This identification of focus areas would persist for several years, and would also be incorporated into the design of the Effective Altruism Funds in 2017. In particular, the blog post encapsulates the central position of AI safety in the then-nascent effective altruist movement.
2013 October 1 Publication Our Final Invention: Artificial Intelligence and the End of the Human Era by James Barrat is published. The book discusses risks from human-level of superhuman artificial intelligence.
2014 Publication Müller and Bostrom's "Future Progress in Artificial Intelligence: A Survey of Expert Opinion" is published.[48]
2014 January 26 Google announces that it has acquired DeepMind. At the same time, it sets up an AI ethics board. DeepMind co-founders Shane Legg and Demis Hassabis, as well as AI safety funders Peter Thiel and Jaan Tallinn, are believed to have been influential in the process.[49][50]
2014 March–May Organization Future of Life Institute (FLI) is founded.[51]
2014 July–September Publication Nick Bostrom's book Superintelligence: Paths, Dangers, Strategies is published. The book would have tremendous impact on defining and raising the status of AI safety concerns, due to "its clear explanation of why superintelligent AI may have arbitrarily negative consequences and why it’s important to begin addressing the issue well in advance."[52]
2014 August Project The AI Impacts website is launched by Paul Christiano and Katja Grace, with funding channeled through MIRI.[53] AI Impacts would grow into a leading source for understanding AI trends and scenarios, and would later receive funding from th Future of Life Institute and Open Philanthropy.
2014 Fall Project The One Hundred Year Study on Artificial Intelligence (AI100) launches.[54]
2014 October 22–24 Opinion During an interview at the AeroAstro Centennial Symposium, Elon Musk calls artificial intelligence humanity's "biggest existential threat".[55][56]
2014 November 4 Project The Intelligent Agent Foundations Forum, run by MIRI, is launched.[57]
2014 November 5 Publication The book Ethical Artificial Intelligence by Bill Hibbard is released on ArXiV. The book brings together ideas from Hibbard's past publications related to technical AI risk.[58]
2014 November Publication Edge hosts a conversation on "The Myth of AI" with an introduction by John Brockman and a lead essay by Jaron Lanier, followed by responses from "Reality Club" participants George Church, Peter Diamandis, Lee Smolin, Rodney Brooks, Nathan Myhrvold, George Dyson, Pamela McCorduck, Sendhil Mullainathan, Steven Pinker, Neal Gershenfeld, D.A. Wallach, Michael Shermer, Stuart Kauffman, Kevin Kelly, Lawrence Krauss, Robert Provine, Stuart Russell, Kai Krause.[59] Russell's response is quoted and discussed by Rob Bensinger on LessWrong.[60]
2014 December 2 Opinion In an interview with BBC, Stephen Hawking states that advanced artificial intelligence could end the human race.[61]
2014 December 16 Fictional portrayal The movie Ex Machina is released. The movie highlights the paperclip maximizer idea: it shows how a robot programmed to optimize for being able to make sure it escapes can callously damage human lives in the process. It also covers the ideas of the Turing test (a robot convincing a human that it has human-like qualities, even though the human knows that the robot is not human) and the AI box experiment (a robot convincing a human to release it from its "box" similar to the experiment proposed and carried out by Eliezer Yudkowsky). It leads to more public discussion of AI safety.[62][63][64][65]
2015 Daniel Dewey joins Open Philanthropy.[66] He begins as or would become Open Phil's program officer for potential risks from advanced artificial intelligence.
2015 Publication The 2015 question on Edge (edge.org) is "What do you think about machines that think?" The question draws responses from a number of intellectuals, including AI researchers, ethicists, AI historians, and other social commentators.[67] Responses from Nick Bostrom, Steve Omohundro, Stuart J. Russell, and Eliezer Yudkowsky discuss AI safety challenges.
2015 Organization The Strategic Artificial Intelligence Research Centre launches around this time.[68][36]
2015 January Publication The Open Letter on Artificial Intelligence, titled "Research Priorities for Robust and Beneficial Artificial Intelligence: an Open Letter", is published. Signatories include Stephen Hawking, Elon Musk, Peter Norvig, and Stuart J. Russell.
2015 January 28 Opinion During an "ask me anything" (AMA) session on reddit, Bill Gates states his concern about artificial superintelligence.[69][70]
2015 January 2–5 Conference The Future of AI: Opportunities and Challenges, an AI safety conference, takes place in Puerto Rico. The conference is organized by the Future of Life Institute.[71] Nate Soares of the Machine Intelligence Research Institute would later call this the "turning point" of when top academics begin to focus on AI risk.[72]
2015 January 22–27 Publication Tim Urban publishes on Wait But Why a two-part series of blog posts about superhuman AI.[73][74]
2015 February 25 Opinion Sam Altman, president of Y Combinator, publishes a blog post in which he writes that the development of superhuman AI is "probably the greatest threat to the continued existence of humanity".[75]
2015 May 1 Publication The Wikipedia article on existential risk from artificial general intelligence is published.[76]
2015 May 22 Publication The blog post AI Researchers On AI Risk by Scott Alexander is published. The post lists prominent AI researchers who have expressed concern about AI safety, countering claims by others that serious AI safety concerns lack buy-in from real AI researchers.[77]
2015 June 4 Opinion At Airbnb's Open Air 2015 conference, Sam Altman, president of Y Combinator, states his concern for advanced artificial intelligence and shares that he recently invested in a company doing AI safety research.[78]
2015 June 17,21 Publication The Kindle edition of Artificial Superintelligence: A Futuristic Approach by Roman Yampolskiy is published. The paperback would be published on June 21.[79] Yampolskiy takes an AI safety engineering perspective, rather than a machine ethics perspective, to the problem of AI safety.[80]
2015 July 1 Funding The Future of Life Institute's Grant Recommendations for its first round of AI safety grants are publicly announced. The grants would be disbursed on September 1. The funding for the grants comes from a $10 million grant from Elon Musk.[81][82][83]
2015 August Conference AI safety is a key theme of EA Global 2015, with the panel on AI risk including Elon Musk, Daniel Dewey, Nick Bostrom, Nate Soares and Stuart Russell.[84][85]
2015 August Funding Open Philanthropy awards a grant of $1.2 million to the Future of Life Institute, saying that they are impressed with the selections from the first round of grants funded by Elon Musk. This adds to the pool of money from Elon Musk.[86][87]
2015 August Publication Open Philanthropy publishes its cause report on potential risks from advanced artificial intelligence.[88]
2015 August 29 The ControlProblem subreddit (full title: "The Artificial General Intelligence Control Problem") is created.[89]
2015 October Publication Open Philanthropy first publishes its page on AI timelines.[90]
2015 December Organization The Leverhulme Centre for the Future of Intelligence launches around this time.[91]
2015 December 11 Organization OpenAI is announced to the public. (The news articles from this period make it sound like OpenAI launched sometime after this date.)[92][93]
2016 April 28 Publication The Global Catastrophic Risks 2016 report is published. The report is a collaboration between the Global Priorities Project and the Global Challenges Foundation.[94] The report includes discussion of risks from artificial general intelligence under "emerging risks".[95][96]
2016 April 7 Publication 80,000 Hours releases a new "problem profile" for risks from artificial intelligence, titled "Risks posed by artificial intelligence".[97][98]
2016 May Publication Luke Muehlhauser of Open Philanthropy publishes "What should we learn from past AI forecasts?".[99]
2016 May 6 Publication Holden Karnofsky of Open Philanthropy publishes a blog post on why Open Philanthropy is making potential risks from artificial intelligence a major priority for the year.[100]
2016 May 6 Publication Holden Karnofsky of Open Philanthropy publishes "Some Background on Our Views Regarding Advanced Artificial Intelligence" on the Open Philanthropy blog.[101]
2016 June Funding Open Philanthropy awards a grant of $264,525 to George Mason University for work by Robin Hanson.[87]
2016 June 21 Publication "Concrete Problems in AI Safety" by Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané is submitted to the arXiv.[102] The paper would receive a shoutout from Open Philanthropy.[103] It would become a landmark in AI safety literature,[104] and many of its authors would continue to do AI safety work at OpenAI in the years to come.
2016 June 27 and 28 Conference Carnegie Mellon University (CMU) hosts a two-day Workshop on Safety and Control for AI, thee first day being an "Exploratory Technical Workshop" (co-sponsored by the Carnegie Mellon Science of Security Lablet) and the seconf day being a "Public Workshop" (co-sponsoreed by the Office of Science and Technology Policy (OSTP) and CMU). Participants include Edward Felten of OSTP, Bill Scherlis of CMU, Jason Matheny of IARPA, Eric Horvitz of Microsoft, and many others.[105]
2016 August Organization The UC Berkeley Center for Human-Compatible Artificial Intelligence launches under the leadership of AI expert Stuart J. Russell (co-author with Peter Norvig of Artificial Intelligence: A Modern Approach). The focus of the center is "to ensure that AI systems are beneficial to humans".[106]
2016 August Funding Open Philanthropy awards a grant of $5.6 million over two years to the newly formed Center for Human-Compatible AI at the University of California, Berkeley.[87]
2016 August Funding Open Philanthropy awards a grant of $500,000 to the Machine Intelligence Research Institute.[87]
2016 August 24 US president Barack Obama speaks to entrepreneur and MIT Media Lab director Joi Ito about AI risk.[107]
2016 September 20, November 2 Publication On September 20, Oren Etzioni, in a piece titled No, the Experts Don’t Think Superintelligent AI is a Threat to Humanity for MIT Technology Review, critiques Nick Bostrom's Superintelligence and includes Etzioni's own survey of AI researchers to refute Bostrom's survey data.[108] In a reply titled Yes, We Are Worried About the Existential Risk of Artificial Intelligence on November 2, Stuart J. Russell and Allan Dafoe argue that the focus of Bostrom's Superintelligence was not on the predictions of imminent AI, but "its clear explanation of why superintelligent AI may have arbitrarily negative consequences and why it’s important to begin addressing the issue well in advance."[52]
2016 September 28 Organization The Partnership on AI is publicly announced.
2016 October 12 Publication Under the Obama Administration, the United States White House releases two reports, Preparing for the Future of Artificial Intelligence and National Artificial Intelligence Research and Development Strategic Plan. The former "surveys the current state of AI, its existing and potential applications, and the questions that progress in AI raise for society and public policy".[109][110]
2016 November Funding Open Philanthropy awards a grant of $199,000 to the Electronic Frontier Foundation for work by Peter Eckersley.[87]
2016 December Funding Open Philanthropy awards a grant of $32,000 to AI Impacts for work on strategic questions related to potential risks from advanced artificial intelligence.[87]
2016 December 3, 12 Publication A couple of posts are published on LessWrong by Center for Applied Rationality (CFAR) president Anna Salamon. The posts discuss CFAR's new focus on AI safety.[111][112]
2016 December 13 Publication The "2016 AI Risk Literature Review and Charity Comparison" is published on the Effective Altruism Forum. The lengthy blog post covers all the published work of prominent organizations focused on AI safety.[113]
2017 Publication The Global Catastrophic Risks 2017 report is published.[114] The report discusses risks from artificial intelligence in a dedicated chapter.[115]
2017 Publication The Global Risks Report 2017 is published by the World Economic Forum. The report contains a section titled "Assessing the Risk of Artificial Intelligence" under "Emerging Technologies".[116]
2017 January 5 – 8 Conference The Asilomar Conference on Beneficial AI is organized by the {{w}Future of Life Institute}} at the Asilomar Conference Grounds. It is the successor to the 2015 Puerto Rico conference. It would result in the creation of the 23 Asilomar AI Principles, a set of guidelines for AI research.[117][118][119] This is not to be confused with the Asilomar Conference on Recombinant DNA, that has been examined by the AI safety community as a case study in risk mitigation.[120]
2017 February 9 Project The Effective Altruism Funds (EA Funds) is announced on the Effective Altruism Forum. EA Funds includes a Long-Term Future Fund that is partly intended to support "priorities for robust and beneficial artificial intelligence".[121][122]
2017 March Funding Open Philanthropy awards a grant of $2.0 million to the Future of Humanity Institute for general support.[87]
2017 March Funding Open Philanthropy awards a grant of $30 million to OpenAI for general support.[87]
2017 April Organization The Berkeley Existential Risk Initiative (BERI) launches around this time (under the leadership of Andrew Critch, who previously helped found the Center for Applied Rationality) to assist researchers working at institutions working to mitigate existential risk, including AI risk.[123][124]
2017 April 6 Publication 80,000 Hours publishes an article about the pros and cons of working on AI safety, titled "Positively shaping the development of artificial intelligence".[125][126]
2017 May Funding Open Philanthropy awards a grant of $1.5 million to the UCLA School of Law for work on governance related to AI risk; this would lead to the formation of AI Pulse.[87][127]
2017 May 24 Publication "When Will AI Exceed Human Performance? Evidence from AI Experts" is published on the arXiv.[128] Two researchers from AI Impacts are authors on the paper.[129]
2017 June 14 Publication 80,000 Hours publishes a guide to working in AI policy and strategy, written by Miles Brundage.[130]
2017 July Funding Open Philanthropy awards a grant of $2.4 million to the Montreal Institute for Learning Algorithms.[87]
2017 July Funding Open Philanthropy awards a grant of about $300,000 to Yale University to support research into the global politics of artificial intelligence led by Allan Dafoe.[87]
2017 July Funding Open Philanthropy awards a grant of about $400,000 to the Berkeley Existential Risk Initiative to support core functions of grantee, and to help them provide contract workers for the Center for Human-Compatible AI (CHAI) housed at the University of California, Berkeley.[87]
2017 July 15–16 Opinion At the National Governors Association in Rhode Island, Elon Musk tells US governors that artificial intelligence is an "existential threat" to humanity.[131]
2017 July 23 Opinion During a Facebook Live broadcast from his backyard, Mark Zuckerberg reveals that he is "optimistic" about advanced artificial intelligence and that spreading concern about "doomsday scenarios" is "really negative and in some ways […] pretty irresponsible".[132]
2017 October Funding Open Philanthropy awards MIRI a grant of $3.75 million over three years ($1.25 million per year). The cited reasons for the grant are a "very positive review" of MIRI's "Logical Induction" paper by an "outstanding" machine learning researcher, as well as Open Philanthropy having made more grants in the area so that a grant to MIRI is less likely to appear as an "outsized endorsement of MIRI's approach".[133][134]
2017 October Project The first commit for AI Watch, a repository of organizations, people, and products in AI safety, is made on October 23.[135] Work on the web portal at aiwatch.issarice.com would begin the next day.[136]
2017 October 13 Publication Eliezer Yudkowsky's blog post There's No Fire Alarm for Artificial General Intelligence is published on the MIRI blog and on the new LessWrong (this is shortly after the launch of the new version of LessWrong).[137][138]
2017 October  – December Project FHI launches its Governance of AI Program (later to be called the Centre for the Governance of AI, and shortened as GovAI), co-directed by Nick Bostrom and Allan Dafoe.[139]
2017 November 3 Project Zvi Mowshowitz and Vladimir Slepnev announce the AI Alignment Prize, a $5,000 prize funded by Paul Christiano for publicly posted work advancing AI alignment.[140] The prize would be discontinued after the fourth round (ending December 31, 2018) due to reduced participation.[141]
2017 December Funding Jaan Tallinn makes a donation of about $5 million to the Berkeley Existential Risk Initiative (BERI) Grants Program.[142]
2017 December 20 Publication The "2017 AI Safety Literature Review and Charity Comparison" is published. The lengthy blog post covers all the published work of prominent organizations focused on AI safety, and is a refresh of a similar post published a year ago.[143]
2017 Year-round Funding The huge increase in cryptocurrency prices in this year would drive a lot of donations to AI safety organizations from people who had held cryptocurrency and got rich from it. MIRI would not be able to match the performance of its 2017 fundraiser in later years, and would cite the unusually high cryptocurrency prices in 2017 as one possible reason.[144]
2018 Early year Organization The Median Group, a "non-profit dedicated to research on global catastrophic risks", is formed at around this time. The group would include two former MIRI researchers.[145] As of 2020, the AI safety part of its research would include an interactive model of AI progress as well as a rough estimate of the feasibility of training an AGI using reinforcement learning.[146]
2018 February 28 Publication 80,000 Hours publishes a blog post A new recommended career path for effective altruists: China specialist suggesting specialization in China as a career path for people in the effective altruist movement. China's likely leading role in the development of artificial intelligence is highlighted as particularly relevant to AI safety efforts.[147]
2018 March 7 Conference During Nextgov and Defense One’s Genius Machines 2018 event, Jason Matheny of IARPA argues that there should be more focus on avoiding bad, hackable design and less concern about superintelligent AI running amok. He says "We’re much less worried about ‘Terminator’ and Skynet scenarios than we are about digital ‘Flubber’ scenarios." Also: "Really badly engineered systems that are vulnerable to either error or malicious attack from outside."[148] Matheny would later go on to lead the Center for Security and Emerging Technology (CSET), and his thinking would influence the direction of the organization.
2018 April 4 Podcast The first AI Alignment Podcast from the Future of Life Institute is on this day. The AI Alignment Podcast would happen at an approximate frequency of one per month from this point onward, but with no set cadence.[149]
2018 April 5 Documentary The documentary Do You Trust This Computer?, directed by Chris Paine, is released. It covers issues related to AI safety and includes interviews with prominent individuals relevant to AI, such as Ray Kurzweil, Elon Musk and Jonathan Nolan.
2018 April 9 Newsletter The first issue of the weekly AI Alignment Newsletter, managed by Rohin Shah of the Center for Human-Compatible AI, is sent. Over the next two years, the team responsible for producing the newsletter would grow to four people, and the newsletter would also be produced in Chinese and in podcast form.[150][151][152]
2018 April 9 Disclosure norms, competition norms OpenAI releases a charter stating that the organization commits to stop competing with a value-aligned and safety-conscious project that comes close to building artificial general intelligence, and also that OpenAI expects to reduce its traditional publishing in the future due to safety concerns.[153][154][155][156][157]
2018 April 12 to 22 Conference The first AI Safety Camp is held in Gran Canaria.[158] The AI Safety Camp team runs about two camps a year.
2018 May Funding Open Philanthropy announces the first set of grants for the Open Philanthropy AI Fellowship, to 7 AI Fellows pursuing research relevant to AI risk. It also makes a grant of $525,000 to Ought and $100,000 to AI Impacts.[87]
2018 May Publication The paper "AGI Safety Literature Review" is published in the International Journal Conference on Artificial Intelligence and uploaded to the ArXiV.[159] This would be referenced as an article to read for people interested in contributing technically to AI safety.[127]
2018 July Funding Open Philanthropy grants $429,770 to the University of Oxford to support research on the global politics of advanced artificial intelligence. The work will be led by Professor Allan Dafoe at the Future of Humanity Institute in Oxford, United Kingdom.[87]
2018 July 10 (beta), October 29 (out of beta) Project The team behind LessWrong 2.0 launches a beta for the AI Alignment Forum at AlignmentForum.org on July 10, as a successor to the Intelligent Agent Foundations Forum (IAFF) at agentfoundations.org.[160] On October 29, the Alignment Forum exits beta and becomes generally available.[161][162]
2018 August 14 Funding Nick Beckstead grants the Machine Intelligence Research Institute (MIRI) $488,994 from the Long-Term Future Fund. This is part of his last set of grants as fund manager; he would subsequently step down and the fund management would move to a different team.[163][164]
2018 August 23 onward (fall semester) Course The course CS 294-149: Safety and Control for Artificial General Intelligence is taught at UC Berkeley, with instructors Andrew Critch and Stuart Russell, both affiliated with the Center for Human-Compatible AI. The lecture schedule, available online, shows coverage of background concepts (like the VNM utility theorem), AI forecasts, and technical AI safety challenges.[165]
2018 September to October Funding During this period, the Berkeley Existential Risk Initiative (BERI) makes a number of grants to individuals working on projects related to AI safety.[166]
2018 October 25 Organization The oldest Wayback Machine snapshot of the webpage aisafety.stanford.edu for the Stanford Center for AI Safety is from this date.[167] A more precise launch date or launch announcement could not be found.
2018 November 22 Disclosure norms Nate Soares, executive director of MIRI, publishes MIRI's 2018 update post that announces MIRI's "nondisclosed-by-default" policy for most of its research.[168] The 2018 AI alignment literature review and charity comparison post would discuss the complications created by this policy for evaluating MIRI's research,[169] and so would the 2019 post.[170] In its 2019 fundraiser review, MIRI would mention the nondisclosure-by-default policy as one possible reason for it raising less money in its 2019 fundraiser.[144]
2018 November 29 Funding The Long-Term Future Fund, one of the Effective Altruism Funds, announces a set of grants: $40,000 to Machine Intelligence Research Institute, $10,000 to Ought, $21,000 to AI Summer School, and $4,500 to the AI Safety Unconference.[164]
2018 December 17 Publication The "2018 AI Alignment Literature Review and Charity Comparison" is published on the Effective Altruism Forum. It surveys AI safety work in 2018. It continues an annual tradition of similar blog posts in 2016 and 2017.[169]
2018 December 21 Publication Kelsey Piper writes a lengthy article for the Future Perfect section of Vox titled "The case for taking AI seriously as a threat to humanity" that gives an overview of AI and AI safety.[171] In an interview with 80,000 Hours in February 2019, Piper would say of the article: "I heard from some people that that gave them a clear explanation of what was going on with AI to point to on. That’s something that seems potentially pretty valuable, just to make sure people are on the same page about that."[172]
2019 January Funding Open Philanthropy grants $250,000 to the Berkeley Existential Risk Initiative (BERI) to temporarily or permanently hire machine learning research engineers dedicated to BERI’s collaboration with the Center for Human-compatible Artificial Intelligence (CHAI).[87]
2019 January Funding Open Philanthropy provides a founding grant for the Center for Security and Emerging Technology (CSET) at Georgetown University of $55 million over 5 years.[87][173] CSET is led by Jason Matheny, formerly of IARPA, dedicated to policy analysis at the intersection of national and international security and emerging technologies. Other founding members include Dewey Murdick from the Chan Zuckerberg Initiative, William Hannas from the CIA, and Helen Toner from Open Philanthropy.
2019 January Conference The Beneficial AGI 2019 Conference is organized in Puerto Rico by the Future of Life Institute. It is a successor to the 2015 Puerto Rico Conference and the 2017 Asilomiar Conference on Beneficial AI, both of which were important milestone conferences.[174][175]
2019 January Publication The Future of Humanity Institute publishes a technical report Reframing Superintelligence: Comprehensive AI Services as General Intelligence by K. Eric Drexler (an engineer best known for seminal studies of the potential of molecular nanotechnology). The report introduces to the AI safety community the ideae of comprehensive AI services (CAIS) as a possible and likely path to superintelligence.[176] Rohin Shah summarizes the "gargantuan 210 page document" in a post cross-posted to the Alignment Forum and LessWrong.[177][178]
2019 February Funding Open Philanthropy grants $2,112,500 to the Machine Intelligence Research Institute (MIRI) over two years. This is part of the first batch of grants decided by the Committee for Effective Altruism Support, which will set "grant sizes for a number of our largest grantees in the effective altruism community, including those who work on long-termist causes."[87] Around the same time, BERI grants $600,000 to MIRI.[166]
2019 February 14 Disclosure norms OpenAI unveils its language-generating system called GPT-2, a system able to write the news, answer reading comprehension problems, and is beginning to show promise at tasks like translation.[179] However, the data or the parameters of the model are not released, under expressed concerns about potential abuse. This would lead to a lot of discussion and controversy.[180]
2019 March 11 Organization OpenAI announces the creation of OpenAI LP, a new “capped-profit” company owned and controlled by the OpenAI nonprofit organization’s board of directors. The new company is purposed to allow OpenAI to rapidly increase their investments in compute and talent while including checks and balances to actualize their mission.[181][182]
2019 March 18 Organization The Stanford Institute for Human-Centered Artificial Intelligence (HAI) launches at Stanford University under the leadership of philosopher John Etchemendy and AI scientist Fei-Fei Li. Partner organizations at launch include AI4All, AI100, AI Index, Center for AI Safety and the Center for the Study of Language and Information.[183] Despite the similarity in name with UC Berkeley's Center for Human-Compatible AI (CHAI), HAI is not focused primarily on AI safety, but on the more general project of making AI beneficial to and supportive of a wider range of human interests; co-founder Fei-Fei Li is involved with partner AI4ALL, that aims to improve diversity and inclusion in AI. The center at Stanford most analogous to CHAI is the Center for AI Safety.
2019 April 7 Funding The Long-Term Future Fund, one of the Effective Altruism Funds, announces a set of 23 grants totaling $923,150. About half the grant money is to organizations or projects directly working in AI safety. Recipients include the Machine Intelligence Research Institute (MIRI), AI Safety Camp, Ought, and a number of individuals working on AI safety projects, including three in deconfusion research.[164]
2019 April 11, April 25 Publication In a two-part podcast with Lucas Perry for the Future of Life Institute, Rohin Shah of the Center for Human-Compatible AI provides an overview of the field of technical AI safety, including different AI safety agendas.[184][185]
2019 May Funding Open Philanthropy announces the second class of the Open Philanthropy AI Fellowship, with 8 machine learning researchers in the class, receiving a total of $2,325,000 in grants.[87]
2019 June 7 Fictional portrayal The movie I Am Mother is released on Netflix. According to a comment on Slate Star Codex: "you can use it to illustrate everything from paperclip maximization to deontological kill switches".[186]
2019 June 12 Disclosure norms In a blog post titled "The Hacker Learns to Trust" Connor Leahy explains why he decided against releasing his own replication of OpenAI's GPT-2 model. Leahy says that helpful discussions with OpenAI team members (Jack Clark, Alec Radford and Jeff Wu), and a convincing argument from Buck Shlegeris of MIRI, led him to change his mind. Highlighted text, based on the conversation with Shlegeris: "Because this isn’t just about GPT2. What matters is that at some point in the future, someone will create something truly dangerous and there need to be commonly accepted safety norms before that happens."[187] The post is linkposted to LessWrong, where it attracts more comments.[188]
2019 June 13 Publication The book The AI Does Not Hate You: Superintelligence, Rationality and the Race to Save the World by Tom Chivers is released in the United Kingdom. The book includes a discussion of AI safety (including the AI box experiments), though it is more focused on the rationality community that built around LessWrong and Slate Star Codex.[189][190]
2019 June 20 Publication, organization The blog post A case for strategy research: what it is and why we need more of it is published by Siebe Rozendal, Justin Shovelain, and David Kristofferson of Convergence Analysis (an organization focused on existential risk strategy, including AI safety strategy); it is published to both LessWrong[191] and the Effective Altruism Forum.[192] This appeaers to be the organization's first publication in its listed publications,[193] although the organization appears to have had a website as far back as Decembr 2015.[194]
2019 August 25 Funding Grantmaking done by Berkeley Existential Risk Initiative (BERI) funded by Jaan Tallinn moves to the newly created Survival and Flourishing Fund (SFF).[195] BERI's grantmaking in this space had previously included AI safety organizations. However, as of April 2020, SFF's grants have not included any grants to organizations exclusively focused on AI safety, but rather been to organizations working on broader global catastrophic risks and other adjacent topics.
2019 August 30 Funding The Long-Term Future Fund, one of the Effective Altruism Funds, announces a set 13 grants totaling $415,697 USD to organizations and individuals. About half the grant money is to organizations or projects working in AI safety and related AI strategy, governance, and policy issues. With the exception of a grant to AI Safety Camp, all the other grants related to AI safety are to individuals.[196]
2019 October 8 Publication The book Human Compatible by Stuart J. Russell (co-author with Peter Norvig of Artificial Intelligence: A Modern Approach and head of the Center for Human-Compatible AI at UC Berkeley) is published by Viking Press. The book is reviewed by The Guardian[197] and interviews with the author is published by Vox[198] and TechCrunch.[199]
2019 November Funding Open Philanthropy makes a $1 million grant to Ought, double the previous grant of $525,000.[87]
2019 November Funding Open Philanthropy makes a $705,000 grant to Berkeley Existential Risk Initiative (BERI) to support continued work with the Center for Human-Compatible AI (CHAI) at UC Berkeley. This includes one year of support for machine learning researchers hired by BERI, and two years of support for CHAI.[87]
2019 November 6 Publication An "AI Alignment Research Overview" by Jacob Steinhardt (one of the co-authors of Concrete Problems in AI Safety) is published to LessWrong and the AI Alignment Forum.[200]
2019 November 21 Funding The Long-Term Future Fund, one of the Effective Altruism Funds, announces a set 13 grants totaling $466,000 USD to organizations and individuals. About a quarter of the grant money is to organizations and individuals working on AI safety. With the exception of a grant to AI Safety Camp, all the other grants related to AI safety are to individuals.[201]
2019 November 24 Toon Alfrink publishes on LessWrong a postmortem for RAISE, an attempt to build an online course for AI safety. The blog post explains the challenges with running RAISE and the reasons for eventually shutting it down.[202]
2019 December 18 Publication The "2019 AI Alignment Literature Review and Charity Comparison" is published on the Effective Altruism Forum. It surveys AI safety work in 2019. It continues an annual tradition of similar blog posts in 2016, 2017, and 2018. One feature new to the document this year is the author's effort to make it easier for readers to jump to and focus on parts of the document most relevant to them, rather than read it beginning to end. To make this easier, the author ends each paragraph with a hashtag, and lists the hashtags at the beginning of the document.[170]
2020 January Publication Rohin Shah (the person who started the weekly AI Alignment Newsletter) publishes a blog post on LessWrong (cross-posted to the Alignment Forum) titled "AI Alignment 2018-19 Review" that he describes as "a review post of public work in AI alignment over 2019, with some inclusions from 2018."[203]
2020 February Funding Open Philanthropy makes grants to AI safety organizations Ought ($1.59 million) and Machine Intelligence Research Institute ($7.7 million) with the money amount determined by the Committee for Effective Altruism Support (CEAS). Other organizations receiving money based on CEAS recommendations at around the same time are the Centre for Effective Altruism and 80,000 Hours, neither of which is primarily focused on AI safety.[87] In April 2020, MIRI would blog about receiving the Open Philanthropy grant (MIRI's "largest grant to date") and reveal that the grant is for two years and that $1.46 million of the grant money comes from Ben Delo, co-founder of the cryptocurrency trading platform BitMEX.[204]
2020 March Publication The Precipice: Existential Risk and the Future of Humanity by Toby Ord (affiliated with the Future of Humanity Institute and with Oxford University) is published by Hachette Books. The book covers risks including artificial intelligence, biological risks, and climate change. The author appears on podcasts to talk about the book, for Future of Life Institute[205] and 80,000 Hours.[206]
2020 March 2 Funding Berkeley Existential Risk Initiative (BERI) grants $300,000 to MIRI.[207] In a blog post in April, MIRI would write: "I’ll note that at the time of our 2019 fundraiser, we expected to receive a grant from BERI in early 2020, and incorporated this into our reserves estimates. However, we predicted the grant size would be $600k; now that we know the final grant amount, that estimate should be $300k lower."[204]
2020 April 2 – 3 Publication Asya Bergal of AI Impacts publishes a series of interviews (done along with Robert Long) of people optimistic about AI safety, along with a summary post to LessWrong. Their one-sentence summary: "Relative optimism in AI often comes from the belief that AGI will be developed gradually, and problems will be fixed as they are found rather than neglected."[208][209]
2020 April 14 Funding The Long-Term Future Fund, one of the Effective Altruism Funds, announces a set of 10 grants totaling $488,350. Detailed grant writeups are not available at the time of announcement due to "factors arising from the ongoing coronavirus pandemic." The grants include a $100,000 grant to MIRI, a grant for an AI safety unconference, and some other AI safety grants as well as grants not directly related to AI safety.[210][204]
2020 April 28 Publication The fourth edition of Artificial Intelligence: A Modern Approach by Stuart Russell and Peter Norvig is released. The edition has considerably more coverage of the alignment problem than previous editions.[211]
2020 May (announcement: May 12) Funding Open Philanthropy announces the third class of the Open Philanthropy AI Fellowship, funding 10 researchers for 5 years for a total of $2.3 million; the fellows were selected from a pool of more than 380 applicants.[87] Catherine Olsson, one of the two people at Open Philanthropy responsible for selecting fellows, responds to a question about the selection of fellows as follows: "But the short answer is I think the key pieces to keep in mind are to view the fellowship as 1) a community, not just individual scholarships handed out, and as such also 2) a multi-year project, built slowly."[212]
2020 May Funding The Survival and Flourishing Fund publishes the outcome of its recommendation S-process for the first half of 2020. This includes grant recommendations to Machine Intelligence Research Institute (MIRI) and AI Impacts in the area of AI safety, and other grant recommendations to organizations such as Future of Life Institute and Convergence Analysis that work partly in the area.[213]
2020 May (website launch), November 19 (announcement on EA Forum) Organization AI Safety Support is launched by Linda Linsefors and JJ Hepburn. The website launches in May 2020 and it is announced on the EA Forum on November 19. Initiatives include online AI safety discussion days, a list of AI safety resources, a mentorship program, and the AI Alignment Slack.[214]
2020 May 28 (release), June and July (discussion and exploration) OpenAI releases the natural language model GPT-3 on GitHub[215] and uploads to the ArXiV the paper Language Models are Few-Shot Learners explaining how GPT-3 was trained and how it performs.[216] Games, websites, and chatbots based on GPT-3 are created for exploratory purposes in the next two months (mostly by people unaffiliated with OpenAI), with a general takeaway that GPT-3 performs significantly better than GPT-2 and past natural language models.[217][218][219][220] Commentators also note many weaknesses such as: trouble with arithmetic because of incorrect pattern matching, trouble with multi-step logical reasoning even though it could do the individual steps separately, inability to identify that a question is nonsense, inability to identify that it does not know the answer to a question, and picking up of racist and sexist content when trained on corpuses that contain some such content.[221][222][223] Writing from an AI safety perspective, Abram Demski asks: How "honest" is GPT-3?[224] Rohin Shah discusses GPT-3 in Alignment Newsletter #102.[225] A post by Andy Jones wonders whether we are in an "AI overhang" with GPT-3 being the trigger for "100x larger projects at Google, Facebook and the like, with timelines measured in months."[226]
2020 May 30 (submission), June 11 (date in paper) Publication The paper "AI Research Considerations for Human Existential Safety (ARCHES)" by Andrew Critch of the Center for Human-Compatible AI (CHAI) and David Krueger of the Montreal Institute for Learning Algorithms (MILA) is uploaded to the ArXiV.[227] MIRI's July 2020 newsletter calls it "a review of 29 AI (existential) safety research directions, each with an illustrative analogy, examples of current work and potential synergies between research directions, and discussion of ways the research approach might lower (or raise) existential risk."[228] Critch is interviewed about the paper on the AI Alignment Podcast released September 15.[229]
2020 May 31 DeepMind's Victoria Krakovna writes a blog post (cross-posted to LessWrong and the Alignment Forum) about possible takeaways from the COVID-19 pandemic for slow AI takeoff.[230]
2020 July 27 In a blog post comment (cross-posted between LessWrong and the Alignment Forum), Gwern Branwen talks about the scaling hypothesis underlying OpenAI's work, and the fact that Google Brain and DeepMind refuse to accept or invest in the hypothesis despite increasing amounts of evidence favoring it. The comment is promoted by Raymond Arnold (LessWrong staff member) and gets heavily upvoted.[231]
2020 September 3 Funding The Long-Term Future Fund, one of the Effective Altruism Funds, announces a set of 12 grants totaling $394,200. Two of the grants are to AI safety organizations: the Center for Human-Compatible AI and AI Impacts. 7 of the 8 grants made to indiividuals are in the domain of AI safety.[232]
2020 September 28 Publication Richard Ngo publishes "AGI safety from first principles" as a collection of Alignmment Forum blog posts, also available as a PDF, that makes the case for AGI safety.[233] Rohin Shah approvingly discusses it in the Alignment Newsletter, summarizing its argument as the "second species argument", i.e., that sufficiently advanced AGI would make humanity the second species on earth.[234]
2020 October 6 Publication The book The Alignment Problem: Machine Learning and Human Values by Brian Christian is published. Rohin Shah reviews the book for LessWrong (cross-posted to the Alignment Forum).[235]
2020 October 22 Publication Scott Garrabrant publishes (cross-posted to LessWrong and the Effective Altruism Forum) a blog post titled "Introduction to Cartesian Frames" that is is first post in a sequence about Cartesian frames, a new conceptual framework for thinking about agency.[236][237]
2020 November Funding The Survival and Flourishing Fund publishes the outcome of its recommendation S-process for the second half of 2020. This includes grant recommendations to several organizations working mostly or partly in AI safety such as Machine Intelligence Research Institute, Center for Human-Compatible AI, Berkeley Existential Risk Initiative, Stanford Existential Risk Initiative, Median Group, Future of Humanity Institute, and Future of Life Institute.[238]
2020 November 28 Funding The Long-Term Future Fund, one of the Effective Altruism Funds, announces a set of 11 grants totaling $505,000. About half of the grant money is for work related to AI safety, and many of the remaining grants are also loosely related to AI, machine learning, and the long-term trajectory of humanity.[239]
2020 December 21 Publication The "2020 AI Alignment Literature Review and Charity Comparison" by Larks is published to the Effective Altruism Forum and cross-posted to LessWrong. It surveys AI safety work in 2020. It continues a tradition of similar posts starting 2016, and follows the same format as 2019.[127]

Meta information on the timeline

How the timeline was built

The initial version of the timeline was written by Issa Rice. The timeline was later expanded considerably by Vipul Naik.

Issa likes to work locally and track changes with Git, so the revision history on this wiki only shows changes in bulk. To see more incremental changes, refer to the commit histories at the old location and the new location.

Funding information for this timeline is available.

What the timeline is still missing

  • The Matrix
  • maybe more at [1]
  • maybe more from the "earlier work on the topic" at Richard Posner on AI Dangers
  • maybe more from AI Researchers On AI Risk but no dates; use as starting point to know what people to look for
  • more AI box results at [2] but unfortunately no dates
  • stuff in [3] and [4]
  • siren/marketing worlds
  • TDT/UDT
  • Paul Christiano AI alignment blog. Also, more on Christiano's trajectory in AI safety
  • The launch of Ought
  • Translations of Superintelligence?
  • universal prior/distant superintelligences stuff
  • Steven Pinker?
  • AI summer school
  • when did the different approaches to alignment come along?
  • the name change from "friendly AI" to "AI safety" and "AI alignment" is probably worth adding, though this was gradual so kind of hard to pin down as an event. See also this comment.
  • https://youtu.be/pClSjljMKeA?t=3605 -- Robin Hanson talks about "booms" in concerns about robots.

Timeline update strategy

Pingbacks

  • Evan Sandhoefner tweet (December 26, 2017) emphasizes the accelerating nature of progress on AI safety by screenshotting the part till 2002 that makes up 1/5 of the timeline at the time.

See also

Timelines of organizations working in AI safety

Other timelines related to AI

Other timelines related to topics with potential applications to AI safety

Other timelines about cause areas prioritized in effective altruism

Other related timelines

External links

References

  1. ↑ Paul Christiano (November 19, 2016). "AI "safety" vs "control" vs "alignment"". AI Alignment. AI Alignment. Retrieved November 18, 2017. 
  2. ↑ Eliezer Yudkowsky (November 16, 2017). "Hero Licensing". LessWrong. Retrieved November 18, 2017. I'll mention as an aside that talk of "Friendly" AI has been going out of style where I'm from. We've started talking instead in terms of "aligning smarter-than-human AI with operators' goals," mostly because "AI alignment" smacks less of anthropomorphism than "friendliness." 
  3. ↑ Michael Nuschke (October 10, 2011). "Seven Ways Frankenstein Relates to Singularity". RetirementSingularity.com. Retrieved July 27, 2017. 
  4. ↑ Mitchell Howe (2002). "What is the intellectual history of the Singularity concept?". Retrieved July 27, 2017. Bearing little resemblance to the campy motion pictures he would inspire, Dr. Frankenstein's monster was a highly intelligent being of great emotional depth, but who could not be loved because of his hideous appearance; for this, he vowed to take revenge on his creator. The monster actually comes across as the most intelligent character in the novel, making Frankenstein perhaps the first work to touch on the core idea of the Singularity. 
  5. ↑ Alan Winfield (August 9, 2014). "Artificial Intelligence will not turn into a Frankenstein monster". The Guardian. Retrieved July 27, 2017. From the Golem to Frankenstein's monster, Skynet and the Matrix, we are fascinated by the old story: man plays god and then things go horribly wrong. 
  6. ↑ 6.0 6.1 6.2 6.3 6.4 6.5 6.6 Muehlhauser, Luke (March 31, 2012). "AI Risk & Opportunity: A Timeline of Early Ideas and Arguments". LessWrong. Retrieved September 14, 2019. 
  7. ↑ Bradley, Peter. "Turing Test and Machine Intelligence". Consortium on Computer Science Instruction. 
  8. ↑ Muehlhauser, Luke (August 11, 2013). "What is AGI?". Machine Intelligence Research Institute. Retrieved September 8, 2019. 
  9. ↑ Wiener, Norbert (May 6, 1960). "Some Moral and Technical Consequences of Automation". Retrieved August 18, 2019. 
  10. ↑ Sinick, Jonah (July 20, 2013). "Norbert Wiener's paper "Some Moral and Technical Consequences of Automation"". LessWrong. Retrieved August 18, 2019. 
  11. ↑ Dafoe, Allan; Russell, Stuart. "Yes, We Are Worried About the Existential Risk of Artificial Intelligence. A defense of the warnings about AI in philosopher Nick Bostrom's book Superintelligence.". Technology Review. 
  12. ↑ Muehlhauser, Luke (July 12, 2013). "Miles Brundage recently pointed me to these quotes from Ed Fredkin, recorded in McCorduck (1979).". LessWrong. Retrieved May 7, 2020. 
  13. ↑ Waldrop, Mitchell (Spring 1987). "A Question of Responsibility". AI Magazine. 8 (1): 28–39. doi:10.1609/aimag.v8i1.572. 
  14. ↑ 14.0 14.1 14.2 14.3 Muehlhauser, Luke (October 18, 2013). "Richard Posner on AI Dangers". Machine Intelligence Resarch. Retrieved May 7, 2020. 
  15. ↑ "History of AI risk thought". Lesswrongwiki. LessWrong. Retrieved July 28, 2017. 
  16. ↑ "Form 990-EZ 2000" (PDF). Retrieved June 1, 2017. Organization was incorporated in July 2000 and does not have a financial history for years 1996-1999. 
  17. ↑ "About the Singularity Institute for Artificial Intelligence". Retrieved July 1, 2017. The Singularity Institute for Artificial Intelligence, Inc. (SIAI) was incorporated on July 27th, 2000 by Brian Atkins, Sabine Atkins (then Sabine Stoeckel) and Eliezer Yudkowsky. The Singularity Institute is a nonprofit corporation governed by the Georgia Nonprofit Corporation Code, and is federally tax-exempt as a 501(c)(3) public charity. At this time, the Singularity Institute is funded solely by individual donors. 
  18. ↑ Eliezer Yudkowsky (2001). "Creating Friendly AI 1.0: The Analysis and Design of Benevolent Goal Architectures" (PDF). The Singularity Institute. Retrieved July 5, 2017. 
  19. ↑ "SL4: By Thread". Retrieved July 1, 2017. 
  20. ↑ "SL4: By Thread". Retrieved July 1, 2017. 
  21. ↑ "Amazon.com: Super-Intelligent Machines (Ifsr International Series on Systems Science and Engineering) (9780306473883): Bill Hibbard: Books". Retrieved July 26, 2017. Publisher: Springer; 2002 edition (October 31, 2002) 
  22. ↑ "Ethical Issues In Advanced Artificial Intelligence". Retrieved July 25, 2017. 
  23. ↑ "About". Oxford Martin School. Retrieved July 25, 2017. The Future of Humanity Institute was established in 2005 with funding from the Oxford Martin School (then known as the James Martin 21st Century School). 
  24. ↑ "Papers from the 2005 AAAI Fall Symposium". Archived from the original on 2014-11-29. 
  25. ↑ "SL4: By Thread". Retrieved July 1, 2017. 
  26. ↑ "Overcoming Bias : Bio". Retrieved June 1, 2017. 
  27. ↑ "Basic AI drives". Lesswrongwiki. LessWrong. Retrieved July 26, 2017. 
  28. ↑ "AIPosNegFactor.pdf" (PDF). Retrieved July 27, 2017. 
  29. ↑ "The Hanson-Yudkowsky AI-Foom Debate". Lesswrongwiki. LessWrong. Retrieved July 1, 2017. 
  30. ↑ "Eliezer_Yudkowsky comments on Thoughts on the Singularity Institute (SI) - Less Wrong". LessWrong. Retrieved July 15, 2017. Nonetheless, it already has a warm place in my heart next to the debate with Robin Hanson as the second attempt to mount informed criticism of SIAI. 
  31. ↑ "FAQ - Lesswrongwiki". LessWrong. Retrieved June 1, 2017. 
  32. ↑ "Recent Singularity Institute Accomplishments". Singularity Institute for Artificial Intelligence. Retrieved July 6, 2017. 
  33. ↑ Muehlhauser, Luke (March 10, 2011). "AGI and Friendly AI in the dominant AI textbook". LessWrong. Retrieved December 20, 2020. 
  34. ↑ Muehlhauser, Luke (October 19, 2013). "Russell and Norvig on Friendly AI". Machine Intelligence Research Institute. Retrieved December 20, 2020. 
  35. ↑ Legg, Shane. "About". Retrieved September 15, 2019. 
  36. ↑ 36.0 36.1 "Landscape of current work on potential risks from advanced AI". Google Docs. Retrieved July 27, 2017. 
  37. ↑ "How Long Untill Human-Level AI - 2011_AI-Experts.pdf" (PDF). Retrieved July 28, 2017. 
  38. ↑ "About". Global Catastrophic Risk Institute. Retrieved July 26, 2017. The Global Catastrophic Risk Institute (GCRI) is a nonprofit, nonpartisan think tank. GCRI was founded in 2011 by Seth Baum and Tony Barrett. 
  39. ↑ "Back of the Envelope Guide to Philanthropy". Retrieved July 28, 2017. 
  40. ↑ "Gordon Irlam on the BEGuide". Meteuphoric. WordPress.com. October 16, 2014. Retrieved July 28, 2017. 
  41. ↑ "Welcome". Oxford Martin Programme on the Impacts of Future Technology. Retrieved July 26, 2017. The Oxford Martin Programme on the Impacts of Future Technology, launched in September 2011, is an interdisciplinary horizontal Programme within the Oxford Martin School in collaboration with the Faculty of Philosophy at Oxford University. 
  42. ↑ "About". Facing the Intelligence Explosion. Retrieved July 27, 2017. 
  43. ↑ Luke Muehlhauser (December 11, 2013). "MIRI's Strategy for 2013". Machine Intelligence Research Institute. Retrieved July 6, 2017. 
  44. ↑ Sylvia Hui (November 25, 2012). "Cambridge to study technology's risk to humans". Retrieved July 26, 2017. The university said Sunday the center's launch is planned next year. 
  45. ↑ "Centre for the Study of Existential Risk". 
  46. ↑ "Transparency". Foundational Research Institute. Retrieved July 27, 2017. 
  47. ↑ Muehlhauser, Luke (July 8, 2013). "Four Focus Areas of Effective Altruism". LessWrong. Retrieved September 8, 2019. 
  48. ↑ "Future Progress in Artificial Intelligence: A Survey of Expert Opinion - survey.pdf" (PDF). Retrieved July 28, 2017. 
  49. ↑ Efrati, Amir (January 26, 2014). "Google Beat Facebook for DeepMind, Creates Ethics Board". Huffington Post. Retrieved September 15, 2019. 
  50. ↑ Bosker, Bianca (January 29, 2014). "Google's New A.I. Ethics Board Might Save Humanity From Extinction". Retrieved September 15, 2019. 
  51. ↑ Victoria Krakovna. "New organization - Future of Life Institute (FLI)". LessWrong. Retrieved July 6, 2017. As of May 2014, there is an existential risk research and outreach organization based in the Boston area. The Future of Life Institute (FLI), spearheaded by Max Tegmark, was co-founded by Jaan Tallinn, Meia Chita-Tegmark, Anthony Aguirre and myself. 
  52. ↑ 52.0 52.1 Dafoe, Allan; Russell, Stuart J. (November 2, 2016). "Yes, We Are Worried About the Existential Risk of Artificial Intelligence. A defense of the warnings about AI in philosopher Nick Bostrom's book Superintelligence.". MIT Technology Review. Retrieved May 7, 2020. 
  53. ↑ "MIRI's September Newsletter". Machine Intelligence Research Institute. September 1, 2014. Retrieved July 15, 2017. Paul Christiano and Katja Grace have launched a new website containing many analyses related to the long-term future of AI: AI Impacts. 
  54. ↑ Peter Stone; et al. (AI100 Standing Committee and Study Panel) (September 2016). "One Hundred Year Study on Artificial Intelligence: Report of the 2015-2016 Study Panel" (PDF). Retrieved July 27, 2017. The One Hundred Year Study on Artificial Intelligence, launched in the fall of 2014, is a longterm investigation of the field of Artificial Intelligence (AI) and its influences on people, their communities, and society. 
  55. ↑ Samuel Gibbs (October 27, 2014). "Elon Musk: artificial intelligence is our biggest existential threat". The Guardian. Retrieved July 25, 2017. 
  56. ↑ "AeroAstro Centennial Webcast". Retrieved July 25, 2017. The high point of the MIT Aeronautics and Astronautics Department's 2014 Centennial celebration is the October 22-24 Centennial Symposium 
  57. ↑ Benja Fallenstein. "Welcome!". Intelligent Agent Foundations Forum. Retrieved June 30, 2017. post by Benja Fallenstein 969 days ago 
  58. ↑ Hibbard, Bill (2014): Ethical Artificial Intelligence. https://arxiv.org/abs/1411.1373
  59. ↑ "The Myth Of AI". Edge. November 14, 2014. Retrieved December 20, 2020. 
  60. ↑ "Stuart Russell: AI value alignment problem must be an "intrinsic part" of the field's mainstream agenda". LessWrong. November 26, 2014. Retrieved December 20, 2020. 
  61. ↑ "Stephen Hawking warns artificial intelligence could end mankind". BBC News. December 2, 2014. Retrieved July 25, 2017. 
  62. ↑ "Ex Machina's Scientific Advisor – Murray Shanahan". Y Combinator. June 28, 2017. Retrieved August 18, 2019. 
  63. ↑ "Ex Machina movie asks: is AI research in safe hands?". January 21, 2015. Retrieved August 18, 2019. 
  64. ↑ "Go see Ex Machina". LessWrong. February 26, 2016. Retrieved August 18, 2019. 
  65. ↑ Hardawar, Devindra (April 1, 2015). "'Ex Machina' director embraces the rise of superintelligent AI". Engadget. Retrieved August 18, 2019. 
  66. ↑ "Daniel Dewey". Open Philanthropy. Retrieved July 25, 2017. 
  67. ↑ "2015: What do you think about machines that think?". Edge. Retrieved May 7, 2020. 
  68. ↑ Future of Humanity Institute - FHI. "Strategic Artificial Intelligence Research Centre - Future of Humanity Institute". Future of Humanity Institute. Retrieved July 27, 2017. 
  69. ↑ "Hi Reddit, I'm Bill Gates and I'm back for my third AMA. Ask me anything. • r/IAmA". reddit. Retrieved July 25, 2017. 
  70. ↑ Stuart Dredge (January 29, 2015). "Artificial intelligence will become strong enough to be a concern, says Bill Gates". The Guardian. Retrieved July 25, 2017. 
  71. ↑ "AI safety conference in Puerto Rico". Future of Life Institute. October 12, 2015. Retrieved July 13, 2017. 
  72. ↑ Nate Soares (July 16, 2015). "An Astounding Year". Machine Intelligence Research Institute. Retrieved July 13, 2017. 
  73. ↑ "The Artificial Intelligence Revolution: Part 1". Wait But Why. January 22, 2017. Retrieved July 25, 2017. 
  74. ↑ "The Artificial Intelligence Revolution: Part 2". Wait But Why. January 27, 2015. Retrieved July 25, 2017. 
  75. ↑ "Machine intelligence, part 1". Sam Altman. Retrieved July 27, 2017. 
  76. ↑ "Existential risk from artificial general intelligence". May 1, 2015. Retrieved August 18, 2019. 
  77. ↑ Alexander, Scott (May 22, 2015). "AI Researchers On AI Risk". Retrieved May 7, 2020. 
  78. ↑ Matt Weinberger (June 4, 2015). "Head of Silicon Valley's most important startup farm says we're in a 'mega bubble' that won't last". Business Insider. Retrieved July 27, 2017. 
  79. ↑ Yampolskiy, Roman (June 17, 2015). "Artificial Superintelligence: A Futuristic Approach". Retrieved August 20, 2017. 
  80. ↑ Muehlhauser, Luke (July 15, 2013). "Roman Yampolskiy on AI Safety Engineering". Machine Intelligence Research Institute. Retrieved August 20, 2017. 
  81. ↑ "Grants Timeline - Future of Life Institute". Future of Life Institute. Retrieved July 13, 2017. 
  82. ↑ "New International Grants Program Jump-Starts Research to Ensure AI Remains Beneficial: Press release for FLI grant awardees. - Future of Life Institute". Future of Life Institute. Retrieved July 13, 2017. 
  83. ↑ "AI Safety Research - Future of Life Institute". Future of Life Institute. Retrieved July 13, 2017. 
  84. ↑ Matthews, Dylan (August 10, 2015). "I spent a weekend at Google talking with nerds about charity. I came away … worried.". Vox. Retrieved April 19, 2020. 
  85. ↑ Bensinger, Rob (August 28, 2015). "AI and Effective Altruism". Machine Intelligence Research Institute. Retrieved April 19, 2020. 
  86. ↑ "Future of Life Institute — Artificial Intelligence Risk Reduction". Open Philanthropy. Retrieved May 7, 2020. 
  87. ↑ 87.00 87.01 87.02 87.03 87.04 87.05 87.06 87.07 87.08 87.09 87.10 87.11 87.12 87.13 87.14 87.15 87.16 87.17 87.18 87.19 87.20 87.21 "Open Philanthropy donations made (filtered to cause areas matching safety)". Retrieved May 16, 2020. 
  88. ↑ "Potential Risks from Advanced Artificial Intelligence". Open Philanthropy. Retrieved July 27, 2017. 
  89. ↑ "The Artificial General Intelligence Control Problem". Retrieved April 18, 2020. 
  90. ↑ "What Do We Know about AI Timelines?". Open Philanthropy. Retrieved July 25, 2017. 
  91. ↑ "The future of intelligence: Cambridge University launches new centre to study AI and the future of humanity". University of Cambridge. December 3, 2015. Retrieved July 26, 2017. 
  92. ↑ John Markoff (December 11, 2015). "Artificial-Intelligence Research Center Is Founded by Silicon Valley Investors". The New York Times. Retrieved July 26, 2017. The organization, to be named OpenAI, will be established as a nonprofit, and will be based in San Francisco. 
  93. ↑ "Introducing OpenAI". OpenAI Blog. December 11, 2015. Retrieved July 26, 2017. 
  94. ↑ "Global Catastrophic Risks 2016". The Global Priorities Project. April 28, 2016. Retrieved July 28, 2017. 
  95. ↑ "Global-Catastrophic-Risk-Annual-Report-2016-FINAL.pdf" (PDF). Retrieved July 28, 2017. 
  96. ↑ George Dvorsky. "These Are the Most Serious Catastrophic Threats Faced by Humanity". Gizmodo. Retrieved July 28, 2017. 
  97. ↑ "How and why to use your career to make artificial intelligence safer". 80,000 Hours. April 7, 2016. Retrieved July 25, 2017. 
  98. ↑ "Risks posed by artificial intelligence". 80,000 Hours. 
  99. ↑ "What should we learn from past AI forecasts?". Open Philanthropy. Retrieved July 27, 2017. 
  100. ↑ "Potential Risks from Advanced Artificial Intelligence: The Philanthropic Opportunity". Open Philanthropy. Retrieved July 27, 2017. 
  101. ↑ "Some Background on Our Views Regarding Advanced Artificial Intelligence". Open Philanthropy. Retrieved July 27, 2017. 
  102. ↑ "[1606.06565] Concrete Problems in AI Safety". June 21, 2016. Retrieved July 25, 2017. 
  103. ↑ Karnofsky, Holden (June 23, 2016). "Concrete Problems in AI Safety". Retrieved April 18, 2020. 
  104. ↑ For instance, the 2019 AI Alignment Literature Review has this line, talking about AI safety publications: "While low-quality contributions might help improve Concrete Problems’ citation count, they may use up scarce funding."
  105. ↑ "Workshop on Safety and Control for Artificial Intelligence". June 28, 2016. Retrieved May 16, 2020. 
  106. ↑ "UC Berkeley launches Center for Human-Compatible Artificial Intelligence". Berkeley News. August 29, 2016. Retrieved July 26, 2017. 
  107. ↑ Scott Dadich (October 12, 2016). "Barack Obama Talks AI, Robo Cars, and the Future of the World". WIRED. Retrieved July 28, 2017. 
  108. ↑ "No, the Experts Don't Think Superintelligent AI is a Threat to Humanity. Ask the people who should really know.". MIT Technology Review. September 20, 2016. Retrieved May 7, 2020. 
  109. ↑ "The Administration's Report on the Future of Artificial Intelligence". whitehouse.gov. October 12, 2016. Retrieved July 28, 2017. 
  110. ↑ "The Obama Administration's Roadmap for AI Policy". Harvard Business Review. December 21, 2016. Retrieved July 28, 2017. 
  111. ↑ "CFAR's new focus, and AI Safety - Less Wrong". LessWrong. Retrieved July 13, 2017. 
  112. ↑ "Further discussion of CFAR's focus on AI safety, and the good things folks wanted from "cause neutrality" - Less Wrong". LessWrong. Retrieved July 13, 2017. 
  113. ↑ Larks (December 13, 2016). "2016 AI Risk Literature Review and Charity Comparison". Effective Altruism Forum. Retrieved August 18, 2019. 
  114. ↑ "Annual Report on Global Risks". Global Challenges Foundation. Retrieved July 28, 2017. 
  115. ↑ "Global Catastrophic Risks 2017.pdf" (PDF). Retrieved July 28, 2017. 
  116. ↑ "Acknowledgements". Global Risks Report 2017. Retrieved July 28, 2017. 
  117. ↑ "Asilomar AI Principles". Future of Life Institute. Retrieved April 19, 2020. 
  118. ↑ Alexander, Scott (February 6, 2017). "Notes from the Asilomar Conference on Beneficial AI". Retrieved April 19, 2020. 
  119. ↑ "Beneficial AI conference develops 'Asilomar AI principles' to guide future AI research". kurzweilai.net. February 3, 2017. Retrieved April 19, 2020. 
  120. ↑ Bensinger, Rob (June 30, 2015). "New report: "The Asilomar Conference: A Case Study in Risk Mitigation"". Machine Intelligence Research Institute. Retrieved April 19, 2020. 
  121. ↑ "EA Funds". Retrieved July 27, 2017. In the biography on the right you can see a list of organizations the Fund Manager has previously supported, including a wide variety of organizations such as the Centre for the Study of Existential Risk, Future of Life Institute and the Center for Applied Rationality. These organizations vary in their strategies for improving the long-term future but are likely to include activities such as research into possible existential risks and their mitigation, and priorities for robust and beneficial artificial intelligence. 
  122. ↑ William MacAskill (February 9, 2017). "Introducing the EA Funds". Effective Altruism Forum. Retrieved July 27, 2017. 
  123. ↑ "May 2017 Newsletter". Machine Intelligence Research Institute. May 10, 2017. Retrieved July 25, 2017. Interested parties may also wish to apply for the event coordinator position at the new Berkeley Existential Risk Initiative, which will help support work at CHAI and elsewhere. 
  124. ↑ "Update on Effective Altruism Funds". Effective Altruism Forum. April 20, 2017. Retrieved July 25, 2017. 
  125. ↑ "Positively shaping the development of artificial intelligence". 80,000 Hours. Retrieved July 25, 2017. 
  126. ↑ "Completely new article on the pros/cons of working on AI safety, and how to actually go about it". April 6, 2017. 
  127. ↑ 127.0 127.1 127.2 Larks (December 21, 2020). "2020 AI Alignment Literature Review and Charity Comparison". Effective Altruism Forum. Retrieved December 21, 2020. 
  128. ↑ "[1705.08807] When Will AI Exceed Human Performance? Evidence from AI Experts". Retrieved July 13, 2017. 
  129. ↑ "Media discussion of 2016 ESPAI". AI Impacts. June 14, 2017. Retrieved July 13, 2017. 
  130. ↑ "New in-depth guide to AI policy and strategy careers, written with Miles Brundage, a researcher at the University of Oxford's Future of Humanity Institute". 80,000 Hours. June 14, 2017. 
  131. ↑ "Elon Musk Warns Governors: Artificial Intelligence Poses 'Existential Risk'". NPR.org. July 17, 2017. Retrieved July 28, 2017. 
  132. ↑ Catherine Clifford (July 24, 2017). "Facebook CEO Mark Zuckerberg: Elon Musk's doomsday AI predictions are 'pretty irresponsible'". CNBC. Retrieved July 25, 2017. 
  133. ↑ Malo Bourgon (November 8, 2017). "A major grant from Open Philanthropy". Machine Intelligence Research Institute. Retrieved November 11, 2017. 
  134. ↑ "Machine Intelligence Research Institute — General Support (2017)". Open Philanthropy. November 8, 2017. Retrieved November 11, 2017. 
  135. ↑ Rice, Issa (October 23, 2017). "Initial commit: AI Watch". Retrieved April 19, 2020. 
  136. ↑ Rice, Issa (October 24, 2017). "start on portal: AI Watch". Retrieved April 19, 2020. 
  137. ↑ "There's No Fire Alarm for Artificial General Intelligence". Machine Intelligence Research Institute. October 13, 2017. Retrieved April 19, 2020. 
  138. ↑ Yudkowsky, Eliezer (October 13, 2017). "There's No Fire Alarm for Artificial General Intelligence". LessWrong. Retrieved April 19, 2020. 
  139. ↑ Future of Humanity Institute - FHI (January 19, 2018). "Quarterly Update Winter 2017 - Future of Humanity Institute". Future of Humanity Institute. Retrieved March 14, 2018. 
  140. ↑ Slepnev, Vladimir (November 3, 2017). "Announcing the AI Alignment Prize". LessWrong. Retrieved April 19, 2020. 
  141. ↑ Slepnev, Vladimir (January 20, 2019). "Announcement: AI alignment prize round 4 winners". Alignment Forum. Retrieved April 19, 2020. 
  142. ↑ "Berkeley Existential Risk Initiative | Activity Update - December 2017". Retrieved February 8, 2018. 
  143. ↑ Larks (December 20, 2017). "2017 AI Safety Literature Review and Charity Comparison". Effective Altruism Forum. Retrieved August 18, 2019. 
  144. ↑ 144.0 144.1 "Our 2019 Fundraiser Review". Machine Intelligence Research Institute. February 13, 2020. Retrieved April 19, 2020. 
  145. ↑ "Team". Median Group. Retrieved May 16, 2020. 
  146. ↑ "Research". Median Group. Retrieved May 16, 2020. 
  147. ↑ Todd, Benjamin; Tse, Brian (February 28, 2018). "'A new recommended career path for effective altruists: China specialist". 80,000 Hours. Retrieved September 8, 2019. 
  148. ↑ "This is Why the IARPA Director Doesn't Worry About Self-Aware Artificial Intelligence". NextGov. March 8, 2018. Retrieved May 16, 2020. 
  149. ↑ "AI Alignment Podcast". Future of Life Institute. Retrieved April 4, 2020. 
  150. ↑ "Alignment Newsletter One Year Retrospective". Effective Altruism Forum. April 10, 2019. Retrieved September 8, 2019. 
  151. ↑ "Alignment Newsletter 1". April 9, 2018. Retrieved September 8, 2019. 
  152. ↑ "Alignment Newsletter". Retrieved September 8, 2019. 
  153. ↑ "OpenAI Charter". OpenAI Blog. April 9, 2018. Retrieved May 5, 2018. 
  154. ↑ wunan (April 9, 2018). "OpenAI charter". LessWrong. Retrieved May 5, 2018. 
  155. ↑ "[D] OpenAI Charter • r/MachineLearning". reddit. Retrieved May 5, 2018. 
  156. ↑ "OpenAI Charter". Hacker News. Retrieved May 5, 2018. 
  157. ↑ Tristan Greene (April 10, 2018). "The AI company Elon Musk co-founded intends to create machines with real intelligence". The Next Web. Retrieved May 5, 2018. 
  158. ↑ "Previous Camps". Retrieved September 7, 2019. 
  159. ↑ Everitt, Tom; Lea, Gary; Hutter, Marcus (May 3, 2018). "AGI Safety Literature Review". Retrieved December 21, 2020. 
  160. ↑ Arnold, Raymond (July 10, 2018). "Announcing AlignmentForum.org Beta". LessWroong. Retrieved April 18, 2020. 
  161. ↑ Habryka, Oliver; Pace, Ben; Arnold, Raymond; Babcock, Jim (October 29, 2018). "Introducing the AI Alignment Forum (FAQ)". LessWrong. Retrieved April 18, 2020. 
  162. ↑ "Announcing the new AI Alignment Forum". Machine Intelligence Research Institute. October 29, 2018. Retrieved April 18, 2020. 
  163. ↑ "July 2018 - Long-Term Future Fund Grants". Effective Altruism Funds. August 14, 2018. Retrieved August 18, 2019. 
  164. ↑ 164.0 164.1 164.2 "Effective Altruism Funds donations made (filtered to cause areas matching AI safety)". Retrieved August 18, 2019. 
  165. ↑ "CS 294-149: Safety and Control for Artificial General Intelligence (Fall 2018)". EECS (UC Berkely). Retrieved May 16, 2020. 
  166. ↑ 166.0 166.1 "Berkeley Existential Risk Initiative donations made (filtered to cause areas matching AI safety)". Retrieved August 18, 2019. 
  167. ↑ "Stanford Center for AI Safety (Wayback Machine snapshot)". October 25, 2018. Retrieved May 9, 2020. 
  168. ↑ "2018 Update: Our New Research Directions - Machine Intelligence Research Institute". Machine Intelligence Research Institute. November 22, 2018. Retrieved February 14, 2019. 
  169. ↑ 169.0 169.1 Larks (December 17, 2018). "2018 AI Alignment Literature Review and Charity Comparison". Effective Altruism Forum. Retrieved August 18, 2019. 
  170. ↑ 170.0 170.1 Larks (December 18, 2019). "2019 AI Alignment Literature Review and Charity Comparison". Effective Altruism Forum. Retrieved April 18, 2020. 
  171. ↑ Piper, Kelsey (December 21, 2018). "The case for taking AI seriously as a threat to humanity. Why some people fear AI, explained.". Vox. Retrieved April 19, 2020. 
  172. ↑ Wiblin, Robert; Harris, Kieran (February 27, 2019). "Can journalists still write about important things?". 80,000 Hours. Retrieved April 19, 2020. 
  173. ↑ Williams, Tate (March 22, 2019). "Important But Neglected: Why an Effective Altruist Funder Is Giving Millions to AI Security". Inside Philanthropy. Retrieved May 16, 2020. 
  174. ↑ "Beneficial AGI 2019". Future of Life Institute. Retrieved April 19, 2020. 
  175. ↑ "CSER at the Beneficial AGI 2019 Conference". Centre for the Study of Existential Risk. January 31, 2019. Retrieved April 19, 2020. 
  176. ↑ Drexler, K. Eric. "Reframing Superintelligence: Comprehensive AI Services as General Intelligence". Future of Humanity Institute. 
  177. ↑ Shah, Rohin (January 7, 2019). "Reframing Superintelligence: Comprehensive AI Services as General Intelligence". Alignment Forum. Retrieved May 2, 2020. 
  178. ↑ Shah, Rohin (January 7, 2019). "Reframing Superintelligence: Comprehensive AI Services as General Intelligence". LessWrong. Retrieved May 2, 2020. 
  179. ↑ "An AI helped us write this article". vox.com. Retrieved 28 June 2019. 
  180. ↑ Lowe, Ryan. "OpenAI's GPT-2: the model, the hype, and the controversy". towardsdatascience.com. Retrieved 10 July 2019. 
  181. ↑ Johnson, Khari. "OpenAI launches new company for funding safe artificial general intelligence". venturebeat.com. Retrieved 15 June 2019. 
  182. ↑ Trazzi, Michaël. "Considerateness in OpenAI LP Debate". medium.com. Retrieved 15 June 2019. 
  183. ↑ Adams, Amy (March 18, 2019). "Stanford University launches the Institute for Human-Centered Artificial Intelligence. The new institute will focus on guiding artificial intelligence to benefit humanity.". Retrieved May 9, 2020. 
  184. ↑ Perry, Lucas (April 11, 2019). "AI Alignment Podcast: An Overview of Technical AI Alignment with Rohin Shah (Part 1)". Future of Life Institute. Retrieved May 2, 2020. 
  185. ↑ Perry, Lucas (April 25, 2019). "AI Alignment Podcast: An Overview of Technical AI Alignment with Rohin Shah (Part 2)". Retrieved May 2, 2020. 
  186. ↑ "OPEN THREAD 129.25". June 8, 2019. Retrieved August 18, 2019. 
  187. ↑ Leahy, Connor (June 12, 2019). "The Hacker Learns to Trust". Retrieved April 19, 2020. 
  188. ↑ Pace, Ben (June 21, 2019). "The Hacker Learns to Trust". LessWrong. Retrieved April 19, 2020. 
  189. ↑ "Some Notes on The AI Does Not Hate You: Superintelligence, Rationality and the Race to Save the World". Reddit (Slate Star Codex subreddit). June 24, 2019. Retrieved May 2, 2020. 
  190. ↑ McCluskey, Peter (Octobeer 24, 2019). "The AI Does Not Hate You". Retrieved May 2, 2020.  Check date values in: |date= (help)
  191. ↑ Rozendal, Siebe; Shovelain, Justin; Kristofferson, David (June 20, 2019). "A case for strategy research: what it is and why we need more of it". LessWrong. Retrieved May 2, 2020. 
  192. ↑ Rozendal, Siebe; Shovelain, Justin; Kristofferson, David (June 20, 2019). "A case for strategy research: what it is and why we need more of it". Effective Altruism Forum. Retrieved May 2, 2020. 
  193. ↑ "Convergence publications". Convergence Analysis. Retrieved May 2, 2020. 
  194. ↑ "Convergence analysis". Retrieved December 8, 2015. 
  195. ↑ "The Future of Grant-making Funded by Jaan Tallinn at BERI". Berkeley Existential Risk Initiative. August 25, 2019. Retrieved April 18, 2020. 
  196. ↑ "August 2019: Long-Term Future Fund Grants and Recommendations". Effective Altruism Funds. August 30, 2019. Retrieved April 18, 2020. 
  197. ↑ Sample, Ian (October 24, 2019). "Human Compatible by Stuart Russell review -- AI and our future. Creating machines smarter than us could be the biggest event in human history -- and the last". The Guardian. Retrieved April 18, 2020. 
  198. ↑ Piper, Kelsey (October 26, 2019). "AI could be a disaster for humanity. A top computer scientist thinks he has the solution. Stuart Russell wrote the book on AI and is leading the fight to change how we build it.". Vox. Retrieved April 18, 2020. 
  199. ↑ Coldewey, Devin (March 20, 2020). "Stuart Russell on how to make AI 'human-compatible': 'We've actually thought about AI the wrong way from the beginning'". TechCrunch. Retrieved April 18, 2020. 
  200. ↑ Pace, Ben (November 6, 2019). "AI Alignment Research Overview (by Jacob Steinhardt)". LessWrong. Retrieved April 18, 2020. 
  201. ↑ "November 2019: Long-Term Future Fund Grants". Effective Altruism Funds. November 21, 2019. Retrieved April 18, 2020. 
  202. ↑ Alfrink, Toon (November 24, 2019). "RAISE post-mortem". LessWrong. Retrieved April 18, 2020. 
  203. ↑ Shah, Rohin (January 27, 2020). "AI Alignment 2018-19 Review". LessWrong. Retrieved April 18, 2020. 
  204. ↑ 204.0 204.1 204.2 Bensinger, Rob (April 27, 2020). "MIRI's largest grant to date!". Machine Intelligence Research Institute. Retrieved May 2, 2020. 
  205. ↑ Perry, Lucas (March 31, 2020). "FLI Podcast: The Precipice: Existential Risk and the Future of Humanity with Toby Ord". Future of Life Institute. Retrieved April 18, 2020. 
  206. ↑ Wiblin, Robert; Koehler, Arden; Harris, Kieran (March 7, 2020). "Toby Ord on the precipice and humanity's potential futures". 80,000 Hours. Retrieved April 18, 2020. 
  207. ↑ "Grants". Berkeley Existential Risk Initiative. Retrieved May 2, 2020. 
  208. ↑ Bergal, Asya (April 3, 2020). "Takeaways from safety by default interviews". LessWrong. Retrieved April 18, 2020. 
  209. ↑ "Interviews on plausibility of AI safety by default". AI Impacts. April 2, 2020. Retrieved April 18, 2020. 
  210. ↑ "April 2020 – Long-Term Future Fund Grant Recommendations". Effective Altruism Funds. April 14, 2020. Retrieved May 2, 2020. 
  211. ↑ "Artificial Intelligence: A Modern Approach (4th edition) on the Alignment Problem". LessWrong. September 16, 2020. Retrieved December 20, 2020. 
  212. ↑ Olsson, Catherine (May 14, 2020). "Hi Ryan - in terms of the Fellowship, I have a lot of thoughts about what we're trying to do". Effective Altruism Forum. Retrieved May 16, 2020. 
  213. ↑ "SFF-2020-H1 S-process Recommendations Announcement". Survival and Flourishing Fund. May 29, 2020. Retrieved October 10, 2020. 
  214. ↑ Linsefors, Linda (November 19, 2020). "Announcing AI Safety Support". Effective Altruism Forum. Retrieved December 20, 2020. 
  215. ↑ "GPT-3 on GitHub". OpenAI. Retrieved July 19, 2020. 
  216. ↑ "Language Models are Few-Shot Learners". May 28, 2020. Retrieved July 19, 2020. 
  217. ↑ "Nick Cammarata on Twitter: GPT-3 as therapist". July 14, 2020. Retrieved July 19, 2020. 
  218. ↑ Gwern (June 19, 2020). "GPT-3 Creative Fiction". Retrieved July 19, 2020. 
  219. ↑ Walton, Nick (July 14, 2020). "AI Dungeon: Dragon Model Upgrade. You can now play AI Dungeon with one of the most powerful AI models in the world.". Retrieved July 19, 2020. 
  220. ↑ Shameem, Sharif (July 13, 2020). "Sharif Shameem on Twitter: With GPT-3, I built a layout generator where you just describe any layout you want, and it generates the JSX code for you.". Twitter. Retrieved July 19, 2020. 
  221. ↑ Lacker, Kevin (July 6, 2020). "Giving GPT-3 a Turing Test". Retrieved July 19, 2020. 
  222. ↑ Woolf, Max (July 18, 2020). "Tempering Expectations for GPT-3 and OpenAI's API". Retrieved July 19, 2020. 
  223. ↑ Asparouhov, Delian (July 17, 2020). "Quick thoughts on GPT3". Retrieved July 19, 2020. 
  224. ↑ "How "honest" is GPT-3?". LessWrong. July 8, 2020. Retrieved December 20, 2020. 
  225. ↑ Shah, Rohin (June 3, 2020). "Alignment Newsletter #102: Meta learning by GPT-3, and a list of full proposals for AI alignment". Retrieved December 20, 2020. 
  226. ↑ Jones, Andy (July 27, 2020). "Are we in an AI overhang?". LessWrong. Retrieved December 20, 2020. 
  227. ↑ "AI Research Considerations for Human Existential Safety (ARCHES)". May 30, 2020. Retrieved June 11, 2020. 
  228. ↑ Bensinger, Rob (July 8, 2020). "July 2020 Newsletter". Machine Intelligence Research Institute. 
  229. ↑ "Andrew Critch on AI Research Considerations for Human Existential Safety". Future of Life Institute. September 15, 2020. Retrieved December 20, 2020. 
  230. ↑ Krakovna, Victoria (May 31, 2020). "Possible takeaways from the coronavirus pandemic for slow AI takeoff". LessWrong. Retrieved December 20, 2020. 
  231. ↑ Branwen, Gwern (July 27, 2020). "As far as I can tell, this is what is going on: they do not have any such thing, because GB and DM do not believe in the scaling hypothesis the way that Sutskever, Amodei and others at OA do.". LessWrong. Retrieved December 20, 2020. 
  232. ↑ "September 2020: Long-Term Future Fund Grants". Effective Altruism Funds. September 3, 2020. Retrieved December 20, 2020. 
  233. ↑ Lua error in package.lua at line 80: module 'Module:Citation/CS1/Suggestions' not found.
  234. ↑ Shah, Rohin. "Alignment Newsletter #122". 
  235. ↑ Shah, Rohin (October 6, 2020). "The Alignment Problem: Machine Learning and Human Values". LessWrong. Retrieved December 20, 2020. 
  236. ↑ Garrabrant, Scott (October 22, 2020). "Introduction to Cartesian Frames". LessWrong. Retrieved December 20, 2020. 
  237. ↑ Bensinger, Rob (October 23, 2020). "October 2020 Newsletter". Machine Intelligence Research Institute. Retrieved December 20, 2020. 
  238. ↑ "SFF-2020-H2 S-process Recommendations Announcement". Survival and Flourishing Fund. Retrieved December 10, 2020. 
  239. ↑ "November 2020: Long-Term Future Fund Grants". November 28, 2020. Retrieved December 20, 2020. 
Retrieved from "http://timelines.issarice.com/index.php?title=Timeline_of_AI_safety&oldid=47915"