Suffering risks

Risks of astronomical suffering

Scope–severity grid from Bostrom's paper "Existential Risk Prevention as Global Priority"[1]

Suffering risks, or s-risks, are risks involving an astronomical amount of suffering; much more than all of the suffering that has occurred on Earth thus far.[2][3] They are sometimes categorized as a subclass of existential risks.[4]

Sources of possible s-risks include embodied artificial intelligence[5] and superintelligence,[6] as well as space colonization, which could potentially lead to "constant and catastrophic wars"[7] and an immense increase in wild animal suffering by introducing wild animals, who "generally lead short, miserable lives full of sometimes the most brutal suffering", to other planets, either intentionally or inadvertently.[8]

Steven Umbrello, an AI ethics researcher, has warned that biological computing may make system design more prone to s-risks.[5]

Induced amnesia has been proposed as a way to mitigate s-risks in locked-in conscious AI and certain AI-adjacent biological systems like brain organoids.[9]

David Pearce has argued that genetic engineering is a potential s-risk. Pearce argues that while technological mastery over the pleasure-pain axis and solving the hard problem of consciousness could lead to the potential eradication of suffering, it could also potentially increase the level of contrast in the hedonic range that sentient beings could experience. He argues that these technologies might make it feasible to create "hyperpain" or "dolorium" that experience levels of suffering beyond the human range. He also argues that there are other potential S-risks that are more exotic, such as those posed by the many-worlds interpretation of quantum mechanics.[10]

## See also

[edit]

* AI control problem
* Ethics of artificial intelligence
* Ethics of terraforming
* Existential risk from artificial general intelligence
* Experience machine
* Global catastrophic risk
* Suffering-focused ethics
* Waluigi effect
* Wild animal suffering
* Wirehead (science fiction)

## References

[edit]

1. ^ Bostrom, Nick (2013). "Existential Risk Prevention as Global Priority" (PDF). Global Policy. 4 (1): 15–3. doi:10.1111/1758-5899.12002. Archived (PDF) from the original on 2014-07-14. Retrieved 2024-02-12 - via Existential Risk.
2. ^ Daniel, Max (2017-06-20). "S-risks: Why they are the worst existential risks, and how to prevent them (EAG Boston 2017)". Center on Long-Term Risk. Archived from the original on 2023-10-08. Retrieved 2023-09-14.
3. ^ Hilton, Benjamin (September 2022). "'S-risks'". 80,000 Hours. Archived from the original on 2024-05-09. Retrieved 2023-09-14.
4. ^ Baumann, Tobias (2017). "S-risk FAQ". Center for Reducing Suffering. Archived from the original on 2023-07-09. Retrieved 2023-09-14.
5. ^ a b Umbrello, Steven; Sorgner, Stefan Lorenz (June 2019). "Nonconscious Cognitive Suffering: Considering Suffering Risks of Embodied Artificial Intelligence". Philosophies. 4 (2): 24. doi:10.3390/philosophies4020024. hdl:2318/1702133.
6. ^ Sotala, Kaj; Gloor, Lukas (2017-12-27). "Superintelligence As a Cause or Cure For Risks of Astronomical Suffering". Informatica. 41 (4). ISSN 1854-3871. Archived from the original on 2021-04-16. Retrieved 2021-02-10.
7. ^ Torres, Phil (2018-06-01). "Space colonization and suffering risks: Reassessing the "maxipok rule"". Futures. 100: 74–85. doi:10.1016/j.futures.2018.04.008. ISSN 0016-3287. S2CID 149794325. Archived from the original on 2019-04-29. Retrieved 2021-02-10.
8. ^ Kovic, Marko (2021-02-01). "Risks of space colonization". Futures. 126: 102638. doi:10.1016/j.futures.2020.102638. ISSN 0016-3287. S2CID 230597480.
9. ^ Tkachenko, Yegor (2024). "Position: Enforced Amnesia as a Way to Mitigate the Potential Risk of Silent Suffering in the Conscious AI". Proceedings of the 41st International Conference on Machine Learning. PMLR. Retrieved 2024-06-11.
10. ^ "Quora Answers by David Pearce (2015 - 2024) : Transhumanism with a human face".

## Further reading

[edit]

* Baumann, Tobias (2022). Avoiding the Worst: How to Prevent a Moral Catastrophe. Independently published. ISBN 979-8359800037.
* Metzinger, Thomas (2021-02-19). "Artificial Suffering: An Argument for a Global Moratorium on Synthetic Phenomenology". Journal of Artificial Intelligence and Consciousness. 08: 43–66. doi:10.1142/S270507852150003X. ISSN 2705-0785.
* Minardi, Di (2020-10-15). "The grim fate that could be 'worse than extinction'". BBC Future. Retrieved 2021-02-11.
* Baumann, Tobias (2017). "S-risks: An introduction". Center for Reducing Suffering. Retrieved 2021-02-10.
* Althaus, David; Gloor, Lukas (2016-09-14). "Reducing Risks of Astronomical Suffering: A Neglected Priority". Center on Long-Term Risk. Retrieved 2021-02-10.

* v
* t
* e

Global catastrophic risks

* Future of the Earth
* Future of an expanding universe
* Ultimate fate of the universe

Technological|

* Chemical warfare
* Cyberattack
* Cyberwarfare
* Cyberterrorism
* Cybergeddon
* Gray goo
* Nanoweapons
* Kinetic bombardment
* Kinetic energy weapon
* Nuclear warfare
* Mutual assured destruction
* Dead Hand
* Doomsday Clock
* Doomsday device
* Antimatter weapon
* Electromagnetic pulse (EMP)
* Safety of high-energy particle collision experiments
* Micro black hole
* Strangelet
* Synthetic intelligence / Artificial intelligence
* AI takeover
* Existential risk from artificial intelligence
* Technological singularity
* Transhumanism

Sociological|

* Anthropogenic hazard
* Collapsology
* Doomsday argument
* Self-indication assumption doomsday argument rebuttal
* Self-referencing doomsday argument rebuttal
* Economic collapse
* Malthusian catastrophe
* New World Order (conspiracy theory)
* Nuclear holocaust
* cobalt
* famine
* winter
* Societal collapse
* World War III

Ecological|

| Climate change|

* Anoxic event
* Biodiversity loss
* Mass mortality event
* Cascade effect
* Cataclysmic pole shift hypothesis
* Climate change and civilizational collapse
* Deforestation
* Desertification
* Extinction risk from climate change
* Tipping points in the climate system
* Flood basalt
* Global dimming
* Global terrestrial stilling
* Global warming
* Hypercane
* Ice age
* Ecocide
* Ecological collapse
* Environmental degradation
* Habitat destruction
* Human impact on the environment
* coral reefs
* on marine life
* Land degradation
* Land consumption
* Land surface effects on climate
* Ocean acidification
* Ozone depletion
* Resource depletion
* Sea level rise
* Supervolcano
* winter
* Verneshot
* Water pollution
* Water scarcity

|

Earth Overshoot Day|

* Overexploitation
* Overpopulation
* Human overpopulation

Biological|

| Extinction|

* Extinction event
* Holocene extinction
* Human extinction
* List of extinction events
* Genetic erosion
* Genetic pollution

|

Others|

* Biodiversity loss
* Decline in amphibian populations
* Decline in insect populations
* Biotechnology risk
* Biological agent
* Biological warfare
* Bioterrorism
* Colony collapse disorder
* Defaunation
* Dysgenics
* Interplanetary contamination
* Pandemic
* Pollinator decline
* Overfishing

Astronomical|

* Big Crunch
* Big Rip
* Coronal mass ejection
* Cosmological phase transition
* Geomagnetic storm
* False vacuum decay
* Gamma-ray burst
* Heat death of the universe
* Proton decay
* Virtual black hole
* Impact event
* Asteroid impact avoidance
* Asteroid impact prediction
* Potentially hazardous object
* Near-Earth object
* winter
* Rogue planet
* Near-Earth supernova
* Hypernova
* Micronova
* Solar flare
* Stellar collision

Eschatological|

* Buddhist
* Maitreya
* Three Ages
* Hindu
* Kalki
* Kali Yuga
* Last Judgement
* Second Coming
* 1 Enoch
* Daniel
* Abomination of desolation
* Prophecy of Seventy Weeks
* Messiah
* Christian
* Futurism
* Historicism
* Interpretations of Revelation
* Idealism
* Preterism
* 2 Esdras
* 2 Thessalonians
* Man of sin
* Katechon
* Antichrist
* Book of Revelation
* Events
* Four Horsemen of the Apocalypse
* Lake of fire
* Number of the Beast
* Seven bowls
* Seven seals
* The Beast
* Two witnesses
* War in Heaven
* Whore of Babylon
* Great Apostasy
* New Earth
* New Jerusalem
* Olivet Discourse
* Great Tribulation
* Son of perdition
* Sheep and Goats
* Islamic
* Al-Qa'im
* Beast of the Earth
* Dhu al-Qarnayn
* Dhul-Suwayqatayn
* Dajjal
* Israfil
* Mahdi
* Sufyani
* Jewish
* Messiah
* War of Gog and Magog
* Third Temple
* Norse
* Zoroastrian
* Saoshyant

Others|

* 2011 end times prediction
* 2012 phenomenon
* Apocalypse
* Apocalyptic literature
* Apocalypticism
* Armageddon
* Blood moon prophecy
* Earth Changes
* End time
* Gog and Magog
* List of dates predicted for apocalyptic events
* Messianism
* Messianic Age
* Millenarianism
* Millennialism
* Premillennialism
* Amillennialism
* Postmillennialism
* Nemesis (hypothetical star)
* Nibiru cataclysm
* Rapture
* Prewrath
* Post-tribulation rapture
* Resurrection of the dead
* World to come

Fictional|

* Alien invasion
* Apocalyptic and post-apocalyptic fiction
* List of apocalyptic and post-apocalyptic fiction
* List of apocalyptic films
* Climate fiction
* Disaster films
* List of disaster films
* List of fictional doomsday devices
* Zombie apocalypse
* Zombie

Organizations|

* Centre for the Study of Existential Risk
* Future of Humanity Institute
* Future of Life Institute
* Nuclear Threat Initiative

General|

* Ransomware
* Cyberwarfare
* Depression
* Droughts
* Epidemic
* Famine
* Financial crisis
* Pandemic
* Riots
* Social crisis
* Survivalism

* World portal
* Categories
* Apocalypticism
* Future problems
* Hazards
* Risk analysis
* Doomsday scenarios

* v
* t
* e

Effective altruism

Concepts|

* Aid effectiveness
* Charity assessment
* Demandingness objection
* Disability-adjusted life year
* Disease burden
* Distributional cost-effectiveness analysis
* Earning to give
* Equal consideration of interests
* Longtermism
* Marginal utility
* Moral circle expansion
* Psychological barriers to effective altruism
* Quality-adjusted life year
* Utilitarianism
* Venture philanthropy

Key figures|

* Sam Bankman-Fried
* Liv Boeree
* Nick Bostrom
* Hilary Greaves
* Holden Karnofsky
* William MacAskill
* Dustin Moskovitz
* Yew-Kwang Ng
* Toby Ord
* Derek Parfit
* Peter Singer
* Cari Tuna
* Eliezer Yudkowsky

Organizations|

* 80,000 Hours
* Against Malaria Foundation
* All-Party Parliamentary Group for Future Generations
* Animal Charity Evaluators
* Animal Ethics
* Centre for Effective Altruism
* Centre for Enabling EA Learning & Research
* Center for High Impact Philanthropy
* Centre for the Study of Existential Risk
* Development Media International
* Evidence Action
* Faunalytics
* Fistula Foundation
* Future of Humanity Institute
* Future of Life Institute
* Founders Pledge
* GiveDirectly
* GiveWell
* Giving What We Can
* Good Food Fund
* The Good Food Institute
* Good Ventures
* The Humane League
* Mercy for Animals
* Machine Intelligence Research Institute
* Malaria Consortium
* Nuclear Threat Initiative
* Open Philanthropy
* Raising for Effective Giving
* Sentience Institute
* Unlimit Health
* Wild Animal Initiative

Focus areas|

* Biotechnology risk
* Climate change
* Cultured meat
* Economic stability
* Existential risk from artificial general intelligence
* Global catastrophic risk
* Global health
* Global poverty
* Intensive animal farming
* Land use reform
* Life extension
* Malaria prevention
* Mass deworming
* Neglected tropical diseases
* Suffering risks
* Wild animal suffering

Literature|

* Doing Good Better
* The End of Animal Farming
* Famine, Affluence, and Morality
* The Life You Can Save
* Living High and Letting Die
* The Most Good You Can Do
* Practical Ethics
* The Precipice
* Superintelligence: Paths, Dangers, Strategies
* What We Owe the Future

Events|

* Effective Altruism Global

* v
* t
* e

Existential risk from artificial intelligence

Concepts|

* AGI
* AI alignment
* AI capability control
* AI safety
* AI takeover
* Consequentialism
* Effective accelerationism
* Ethics of artificial intelligence
* Existential risk from artificial general intelligence
* Friendly artificial intelligence
* Instrumental convergence
* Intelligence explosion
* Longtermism
* Machine ethics
* Suffering risks
* Superintelligence
* Technological singularity

Organizations|

* Alignment Research Center
* Center for AI Safety
* Center for Applied Rationality
* Center for Human-Compatible Artificial Intelligence
* Centre for the Study of Existential Risk
* EleutherAI
* Future of Humanity Institute
* Future of Life Institute
* Google DeepMind
* Humanity+
* Institute for Ethics and Emerging Technologies
* Leverhulme Centre for the Future of Intelligence
* Machine Intelligence Research Institute
* OpenAI

People|

* Scott Alexander
* Sam Altman
* Yoshua Bengio
* Nick Bostrom
* Paul Christiano
* Eric Drexler
* Sam Harris
* Stephen Hawking
* Dan Hendrycks
* Geoffrey Hinton
* Bill Joy
* Shane Legg
* Elon Musk
* Steve Omohundro
* Huw Price
* Martin Rees
* Stuart J. Russell
* Jaan Tallinn
* Max Tegmark
* Frank Wilczek
* Roman Yampolskiy
* Eliezer Yudkowsky

Other|

* Statement on AI risk of extinction
* Human Compatible
* Open letter on artificial intelligence (2015)
* Our Final Invention
* The Precipice
* Superintelligence: Paths, Dangers, Strategies
* Do You Trust This Computer?
* Artificial Intelligence Act

Category
*[v]: View this template
*[t]: Discuss this template
*[e]: Edit this template