Edge Rewrite
everything Source code View original
// HTMLRewriter · presentation

This page was redesigned at the edge.

Cloudflare fetched the original article and streamed it through HTMLRewriter to apply an entirely new visual system without rebuilding the source page.

// request.cf · coarse context

A page that knows where it met you.

Only coarse request metadata is shown. This demo does not display or persist visitor IP addresses.

Country
US
Cloudflare location
CMH
Connection
HTTP/2
Language
Not provided

Ray ID: a402daccef3789da

Jump to content

Machine Intelligence Research Institute

From Wikipedia, the free encyclopedia
(Redirected from Singularity Institute)
Machine Intelligence Research Institute
Formation2000; 26 years ago (2000)
TypeNonprofit research institute
Tax ID no.58-2565917
Location
Key people
Eliezer Yudkowsky
Websiteintelligence.org Edit this at Wikidata

The Machine Intelligence Research Institute (MIRI), formerly the Singularity Institute for Artificial Intelligence (SIAI), is a non-profit research institute focused on identifying and managing potential existential risks from artificial intelligence. MIRI's work has focused on a friendly AI approach to system design and on predicting the rate of technology development. Since 2023, it has prioritized policy and public communication while continuing research.[1]

History

[edit]
Yudkowsky at Stanford University in 2006

In 2000, Eliezer Yudkowsky founded the Singularity Institute for Artificial Intelligence with funding from Brian and Sabine Atkins, with the purpose of accelerating the development of artificial intelligence (AI).[2][3][4] However, Yudkowsky began to be concerned that AI systems developed in the future could become superintelligent and pose risks to humanity.[2] In 2005 the institute began operating in Silicon Valley and focused on ways to identify and manage those risks, which were at the time largely ignored by scientists in the field.[3][1]

Starting in 2006, the Institute organized the Singularity Summit to discuss the future of AI including its risks, initially in cooperation with Stanford University and with funding from Peter Thiel. The San Francisco Chronicle described the first conference as a "Bay Area coming-out party for the tech-inspired philosophy called transhumanism".[5][6] In 2011, its offices were four apartments in downtown Berkeley.[7] In December 2012, the institute sold its name, web domain, and the Singularity Summit to Singularity University,[8] and in the following month took the name "Machine Intelligence Research Institute".[9]

In 2014 and 2015, public and scientific interest in the risks of AI grew, increasing donations to fund research at MIRI and similar organizations.[4][10]: 327 

In 2019, Open Philanthropy recommended a general-support grant of approximately $2.1 million over two years to MIRI.[11] In April 2020, MIRI announced an additional $7.7 million grant from Open Philanthropy over two years.[12][13]

In 2021, Vitalik Buterin donated several million dollars' worth of Ethereum to MIRI.[14]

Research and approach

[edit]
Nate Soares presenting an overview of the AI alignment problem at Google in 2016

MIRI's earlier research on identifying and managing the risks of AI, led by Yudkowsky, primarily addressed the AI control problem and how to design friendly AI, covering both the initial design of AI systems and the creation of mechanisms to ensure that evolving AI systems remain friendly.[4][15][16]

MIRI researchers advocate early safety work as a precautionary measure.[17] In 2015, MIRI researchers expressed skepticism about the reliability of forecasts of when advanced AI would emerge.[15] MIRI has funded forecasting work through an initiative called AI Impacts, which studies historical instances of discontinuous technological change, and has developed new measures of the relative computational power of humans and computer hardware.[18]

MIRI aligns itself with the principles and objectives of the effective altruism movement.[19]

In January 2024, MIRI announced that it had shifted its priorities in 2023 toward policy and public communication. It advocated an international agreement to halt progress toward smarter-than-human AI until understanding of such systems substantially improved and they could be secured against misuse, while continuing technical research.[1]

Works by MIRI staff

[edit]
  • Graves, Matthew (8 November 2017). "Why We Should Be Concerned About Artificial Superintelligence". Skeptic. The Skeptics Society. Retrieved 28 July 2018.
  • LaVictoire, Patrick; Fallenstein, Benja; Yudkowsky, Eliezer; Bárász, Mihály; Christiano, Paul; Herreshoff, Marcello (2014). "Program Equilibrium in the Prisoner's Dilemma via Löb's Theorem". Multiagent Interaction without Prior Coordination: Papers from the AAAI-14 Workshop. AAAI Publications.
  • Soares, Nate; Levinstein, Benjamin A. (2017). "Cheating Death in Damascus" (PDF). Formal Epistemology Workshop. Retrieved 28 July 2018.
  • Soares, Nate; Fallenstein, Benja; Yudkowsky, Eliezer; Armstrong, Stuart (2015). "Corrigibility". AAAI Workshops: Workshops at the Twenty-Ninth AAAI Conference on Artificial Intelligence, Austin, TX, January 25–26, 2015. AAAI Publications. Archived from the original on 2016-01-15. Retrieved 2015-10-16.
  • Soares, Nate; Fallenstein, Benya (2017). "Agent Foundations for Aligning Machine Intelligence with Human Interests: A Technical Research Agenda" (PDF). In Callaghan, Victor; Miller, James; Yampolskiy, Roman; Armstrong, Stuart (eds.). The Technological Singularity: Managing the Journey. Springer. pp. 103–125. doi:10.1007/978-3-662-54033-6_5. {{cite book}}: Invalid |display-editors=4 (help)
  • Yudkowsky, Eliezer (2008). "Artificial Intelligence as a Positive and Negative Factor in Global Risk" (PDF). In Bostrom, Nick; Ćirković, Milan (eds.). Global Catastrophic Risks. Oxford University Press. ISBN 978-0199606504.
  • Taylor, Jessica (2016). "Quantilizers: A Safer Alternative to Maximizers for Limited Optimization". Workshops at the Thirtieth AAAI Conference on Artificial Intelligence.
  • Yudkowsky, Eliezer (2011). "Complex Value Systems in Friendly AI" (PDF). Artificial General Intelligence: 4th International Conference, AGI 2011, Mountain View, CA, USA, August 3–6, 2011. Berlin: Springer.
  • Yudkowsky, Eliezer; Soares, Nate (2025). If Anyone Builds It, Everyone Dies: Why Superhuman AI Would Kill Us All. Little, Brown and Company. ISBN 9780316595643.

See also

[edit]

References

[edit]
  1. 1 2 3 Bourgon, Malo (January 4, 2024). "MIRI 2024 Mission and Strategy Update". Machine Intelligence Research Institute.
  2. 1 2 "MIRI: Artificial Intelligence: The Danger of Good Intentions - Future of Life Institute". Future of Life Institute. 11 October 2015. Archived from the original on 28 August 2018. Retrieved 28 August 2018.
  3. 1 2 Khatchadourian, Raffi. "The Doomsday Invention". The New Yorker. Archived from the original on 2019-04-29. Retrieved 2018-08-28.
  4. 1 2 3 Waters, Richard (31 October 2014). "Artificial intelligence: machine v man". Financial Times. Archived from the original on 27 August 2018. Retrieved 27 August 2018.
  5. ↑ Abate, Tom (2006). "Smarter than thou?". San Francisco Chronicle. Archived from the original on 11 February 2011. Retrieved 12 October 2015.
  6. ↑ Abate, Tom (2007). "Public meeting will re-examine future of artificial intelligence". San Francisco Chronicle. Archived from the original on 14 January 2016. Retrieved 12 October 2015.
  7. ↑ Kaste, Martin (January 11, 2011). "The Singularity: Humanity's Last Invention?". All Things Considered, NPR. Archived from the original on August 28, 2018. Retrieved August 28, 2018.
  8. ↑ "Press release: Singularity University Acquires the Singularity Summit". Singularity University. 9 December 2012. Archived from the original on 27 April 2019. Retrieved 28 August 2018.
  9. ↑ "Press release: We are now the "Machine Intelligence Research Institute" (MIRI) - Machine Intelligence Research Institute". Machine Intelligence Research Institute. 30 January 2013. Archived from the original on 23 September 2018. Retrieved 28 August 2018.
  10. ↑ Tegmark, Max (2017). Life 3.0: Being Human in the Age of Artificial Intelligence. United States: Knopf. ISBN 978-1-101-94659-6.
  11. ↑ "Machine Intelligence Research Institute — General Support (2019)". Open Philanthropy Project. 2019-03-29. Archived from the original on 2019-10-08. Retrieved 2019-10-08.
  12. ↑ "Machine Intelligence Research Institute — General Support (2020)". Open Philanthropy Project. 10 March 2020. Archived from the original on April 13, 2020.
  13. ↑ Bensinger, Rob (April 27, 2020). "MIRI's largest grant to date!". MIRI. Archived from the original on April 27, 2020. Retrieved April 27, 2020.
  14. ↑ Maheshwari, Suyash (2021-05-13). "Ethereum creator Vitalik Buterin donates $1.5 billion in cryptocurrency to India COVID Relief Fund & other charities". MSN. Archived from the original on 2021-08-24. Retrieved 2023-01-23.
  15. 1 2 LaFrance, Adrienne (2015). "Building Robots With Better Morals Than Humans". The Atlantic. Archived from the original on 19 August 2015. Retrieved 12 October 2015.
  16. ↑ Russell, Stuart; Norvig, Peter (2009). Artificial Intelligence: A Modern Approach. Prentice Hall. ISBN 978-0-13-604259-4.
  17. ↑ Sathian, Sanjena (4 January 2016). "The Most Important Philosophers of Our Time Reside in Silicon Valley". OZY. Archived from the original on 29 July 2018. Retrieved 28 July 2018.
  18. ↑ Hsu, Jeremy (2015). "Making Sure AI's Rapid Rise Is No Surprise". Discover. Archived from the original on 12 October 2015. Retrieved 12 October 2015.
  19. ↑ "AI and Effective Altruism". Machine Intelligence Research Institute. 2015-08-28. Archived from the original on 2019-10-08. Retrieved 2019-10-08.

Further reading

[edit]
[edit]
Retrieved from "https://en.wikipedia.org/w/index.php?title=Machine_Intelligence_Research_Institute&oldid=1376467869"