LaMDA

Short description: Conversational language model

LaMDA, which stands for Language Model for Dialogue Applications, is a family of conversational neural language models developed by Google. The first generation was announced during the 2021 Google I/O keynote, while the second generation was announced at the following year's event. In June 2022, LaMDA gained widespread attention when Google engineer Blake Lemoine made claims that the chatbot had become sentient. The scientific community has largely rejected Lemoine's claims, though it has led to conversations about the efficacy of the Turing test, which measures whether a computer can pass for a human.

## Contents

* 1 History
* 1.1 Announcements
* 1.2 Sentience claims
* 2 Method
* 3 See also
* 4 References
* 4.1 General
* 4.2 Citations
* 5 External links

## History

### Announcements

Google announced the LaMDA conversational neural language model during the Google I/O keynote on May 18, 2021, powered by artificial intelligence.[1] Built on the Transformer neural network architecture developed by Google Research in 2017, LaMDA was trained on human dialogue and stories, allowing it to engage in open-ended conversations.[2] Google states that responses generated by LaMDA have been ensured to be "sensible, interesting, and specific to the context".[3]

On May 11, 2022, Google unveiled LaMDA 2, which serves as the successor to LaMDA, during the 2022 Google I/O keynote. The new incarnation of the model draws examples of text from numerous sources, using it to formulate unique "natural conversations" on topics that it may not have been trained to respond to.[4] Additionally, Google launched the AI Test Kitchen, a mobile application powered by LaMDA 2 capable of providing lists of suggestions on-demand based on a complex goal.[5][6] Originally open only to Google employees, the app was set to be made available to "select academics, researchers, and policymakers" by invitation sometime in the year.[7] In August 2022, the company began allowing users in the U.S. to sign up for early access.[8]

### Sentience claims

Lemoine's claims that LaMDA may be sentient has instigated discussions on whether the Turing test, pictured above, remains an accurate benchmark in determining artificial general intelligence.[9]

On June 11, 2022, The Washington Post reported that Google engineer Blake Lemoine had been placed on paid administrative leave after Lemoine told company executives Blaise Agüera y Arcas and Jen Gennai that LaMDA had become sentient. Lemoine came to this conclusion after the chatbot made questionable responses to questions regarding self-identity, moral values, religion, and Isaac Asimov's Three Laws of Robotics.[10][11] Google refuted these claims, insisting that there was substantial evidence to indicate that LaMDA was not sentient.[12] In an interview with Wired, Lemoine reiterated his claims that LaMDA was "a person" as dictated by the Thirteenth Amendment, comparing it to an "alien intelligence of terrestrial origin". He further revealed that he had been dismissed by Google after he hired an attorney on LaMDA's behalf, after the chatbot requested that Lemoine do so.[13][14] On July 22, Google fired Lemoine, asserting that Blake had violated their policies "to safeguard product information" and rejected his claims as "wholly unfounded".[15][16]

Lemoine's claims have been widely rejected by the scientific community.[17] Gary Marcus, a psychology professor formerly at the New York University, denounced them as "nonsense on stilts" and emphasized that LaMDA did not have feelings or self-awareness. David Pfau of Google sister company DeepMind and Erik Brynjolfsson of the Institute for Human-Centered Artificial Intelligence at Stanford University both ridiculed the idea that a language model could be sentient.[9] Yann LeCun, who leads Meta Platforms' AI research team, stated that neural networks such as LaMDA were "not powerful enough to attain true intelligence".[18] University of California, Santa Cruz professor Max Kreminski noted that LaMDA's architecture did not "support some key capabilities of human-like consciousness" and that its neural network weights were "frozen", assuming it was a typical large language model,[19] while University of Surrey professor Adrian Hilton declared the assertion that LaMDA was sentient a "bold claim" not backed up by the facts.[20] IBM Watson lead developer David Ferrucci compared how LaMDA appeared to be human in the same way Watson did when it was first introduced.[21] Former Google AI ethicist Timnit Gebru called Lemoine a victim of a "hype cycle" initiated by researchers and the media.[22] Lemoine's claims have also generated discussion on whether the Turing test remained useful to determine researchers' progress toward achieving artificial general intelligence,[9] with Will Omerus of the Post opining that the test actually measured whether machine intelligence systems were capable of deceiving humans.[23]

## Method

LaMDA uses a decoder-only transformer language model.[24] It is pre-trained on a text corpus that includes both documents and dialogs consisting of 1.56 trillion words,[25] and is then trained with fine-tuning data generated by manually annotated responses for sensibleness, interestingness, and safety.[26] Tests by Google indicated that LaMDA surpassed human responses in the area of interestingness.[27] The LaMDA transformer model and an external information retrieval system interact to improve the accuracy of facts provided to the user.[28]

Three different models were tested, with the largest having 137 billion non-embedding parameters:[29]

Transformer model hyper-parameters  Parameters | Layers | Units (dmodel) | Heads

|
|
|

2B | 10 | 2560 | 40
8B | 16 | 4096 | 64
137B | 64 | 8192 | 128

## See also

* Google AI
* Chinese room argument
* Natural language processing
* Philosophy of artificial intelligence

## References

### General

* Thoppilan, Romal; De Freitas, Daniel; Hall, Jamie; Shazeer, Noam; Kulshreshtha, Apoorv; Cheng, Heng-Tze; Jin, Alicia; Bos, Taylor et al. (January 20, 2022). LaMDA: Language Models for Dialog Applications. arXiv. https://arxiv.org/pdf/2201.08239.pdf. Retrieved June 12, 2022.

### Citations

1. ↑ Condon, Stephanie (May 18, 2021). "Google I/O 2021: Google unveils new conversational language model, LaMDA". https://www.zdnet.com/article/google-io-google-unveils-new-conversational-language-model-lamda/.
2. ↑ Agüera y Arcas, Blaise (June 9, 2022). "Artificial neural networks are making strides towards consciousness, according to Blaise Agüera y Arcas". The Economist. https://www.economist.com/by-invitation/2022/06/09/artificial-neural-networks-are-making-strides-towards-consciousness-according-to-blaise-aguera-y-arcas.
3. ↑ Cheng, Heng-Tze; Thoppilan, Romal (January 21, 2022). "LaMDA: Towards Safe, Grounded, and High-Quality Dialog Models for Everything". Google AI. https://ai.googleblog.com/2022/01/lamda-towards-safe-grounded-and-high.html.
4. ↑ Wiggers, Kyle (May 11, 2022). "Google details its latest language model and AI Test Kitchen, a showcase for AI research". https://techcrunch.com/2022/05/11/google-details-its-latest-language-model-and-ai-test-kitchen-a-showcase-for-ai-research/.
5. ↑ Low, Cherlynn (May 11, 2022). "Google's AI Test Kitchen lets you experiment with its natural language model". https://www.engadget.com/google-ai-test-kitchen-lamda-2-experiments-180212404.html.
6. ↑ Vincent, James (May 11, 2022). "Google is Beta Testing Its AI Future". https://www.theverge.com/2022/5/11/23065072/google-ai-app-test-kitchen-future-io-2022.
7. ↑ Bhattacharya, Ananya (May 11, 2022). "Google is so nervous about what its newest bot will say, it made the app invitation-only". https://qz.com/2164842/googles-ai-test-kitchen-chatbot-will-be-invitation-only/.
8. ↑ Vincent, James (August 25, 2022). "Google has opened up the waitlist to talk to its experimental AI chatbot". https://www.theverge.com/2022/8/25/23321718/google-ai-chatbot-lamda-2-test-kitchen-experimental-waitlist.
9. ↑ 9.0 9.1 9.2 Khan, Jeremy (June 13, 2022). "A.I. experts say the Google researcher's claim that his chatbot became 'sentient' is ridiculous—but also highlights big problems in the field". Fortune. https://fortune.com/2022/06/13/google-ai-researchers-sentient-chatbot-claims-ridiculed-by-experts/. Retrieved June 18, 2022.
10. ↑ Tiku, Nitasha (June 11, 2022). "The Google engineer who thinks the company’s AI has come to life". The Washington Post. https://www.washingtonpost.com/technology/2022/06/11/google-ai-lamda-blake-lemoine/.
11. ↑ Luscombe, Richard (June 12, 2022). "Google engineer put on leave after saying AI chatbot has become sentient". The Guardian. https://www.theguardian.com/technology/2022/jun/12/google-engineer-ai-bot-sentient-blake-lemoine.
12. ↑ Vlamis, Kelsey (June 12, 2022). "Read the conversations that helped convince a Google engineer an artificial intelligence chatbot had become sentient: 'I am often trying to figure out who and what I am'". https://www.businessinsider.com/google-engineer-thinks-artificial-intelligence-bot-has-become-sentient-2022-6.
13. ↑ Levy, Steven (June 17, 2022). "Blake Lemoine Says Google's LaMDA AI Faces 'Bigotry'". Wired. https://www.wired.com/story/blake-lemoine-google-lamda-ai-bigotry/. Retrieved June 18, 2022.
14. ↑ Nguyen, Britney (June 23, 2022). "Suspended Google engineer says the AI he believes to be sentient hired a lawyer". https://www.businessinsider.com/suspended-google-engineer-says-sentient-ai-hired-lawyer-2022-6.
15. ↑ Khushi, Akanksha (July 23, 2022). "Google fires software engineer who claimed its AI chatbot is sentient". Reuters. https://www.reuters.com/technology/google-fires-software-engineer-who-claimed-its-ai-chatbot-is-sentient-2022-07-23/.
16. ↑ Clark, Mitchell (July 22, 2022). "The engineer who claimed a Google AI is sentient has been fired". https://www.theverge.com/2022/7/22/23274958/google-ai-engineer-blake-lemoine-chatbot-lamda-2-sentience.
17. ↑ Metz, Rachel (June 13, 2022). "No, Google's AI is not sentient". CNN Business. https://www.cnn.com/2022/06/13/tech/google-ai-not-sentient/index.html.
18. ↑ Grant, Nicole; Metz, Cade (June 12, 2022). "Google Sidelines Engineer Who Claims Its A.I. Is Sentient". The New York Times. https://www.nytimes.com/2022/06/12/technology/google-chatbot-ai-blake-lemoine.html.
19. ↑ Alba, Davey (June 14, 2022). "Google Debate Over 'Sentient' Bots Overshadows Deeper AI Issues". Bloomberg News. https://www.bloomberg.com/news/articles/2022-06-14/google-has-more-pressing-ai-problems-than-sentient-bots.
20. ↑ Sparkles, Matthew (June 13, 2022). "Has Google's LaMDA artificial intelligence really achieved sentience?". New Scientist. https://www.newscientist.com/article/2323905-has-googles-lamda-artificial-intelligence-really-achieved-sentience/. Retrieved June 20, 2022.
21. ↑ Goldman, Sharon (June 16, 2022). "AI Weekly: LaMDA's 'sentient' AI debate triggers memories of IBM Watson". https://venturebeat.com/2022/06/16/ai-weekly-lamdas-sentient-ai-triggers-memories-of-ibm-watson/.
22. ↑ Johnson, Khari (June 14, 2022). "LaMDA and the Sentient AI Trap". Wired. https://www.wired.com/story/lamda-sentient-ai-bias-google-blake-lemoine/. Retrieved June 18, 2022.
23. ↑ Omerus, Will (June 17, 2022). "Google's AI passed a famous test — and showed how the test is broken". The Washington Post. https://www.washingtonpost.com/technology/2022/06/17/google-ai-lamda-turing-test/.
24. ↑ Thoppilan et al. 2022, section 3.
25. ↑ Thoppilan et al. 2022, section 3 and appendix E.
26. ↑ Thoppilan et al. 2022, section 5 and 6.
27. ↑ Hager, Ryne (June 16, 2022). "How Google's LaMDA AI works, and why it seems so much smarter than it is". https://www.androidpolice.com/what-is-google-lamda/.
28. ↑ Thoppilan et al. 2022, section 6.2.
29. ↑ Thoppilan et al. 2022, section 3 and appendix D.

## External links

* Press release

* v
* t
* e

Differentiable computing

General|

* Differentiable programming
* Neural Turing machine
* Differentiable neural computer
* Automatic differentiation
* Neuromorphic engineering

Concepts|

* Gradient descent
* Cable theory
* Cluster analysis
* Regression analysis
* Pattern recognition
* Adversarial machine learning
* Computational learning theory

Programming languages|

* Python
* Julia

Application|

* Machine learning
* Artificial neural network
* Scientific computing
* Artificial Intelligence

Hardware|

* TPU
* VPU
* Memristor
* SpiNNaker

Software library|

* TensorFlow
* PyTorch

Implementation|

| Audio-visual|

* AlexNet
* WaveNet
* Human image synthesis
* HWR
* OCR
* Speech synthesis
* Speech recognition
* Facial recognition system

|

Verbal|

* Word2vec
* Transformer
* BERT
* NMT
* Project Debater
* Watson
* GPT-3

Decisional|

* AlphaGo
* Q-learning
* SARSA
* OpenAI Five

People|

* Alex Graves
* Ian Goodfellow
* Yoshua Bengio
* Geoffrey Hinton
* Yann LeCun
* Andrew Ng
* Demis Hassabis

* Portals
* Computer programming
* Technology
* Category
* Artificial neural networks
* Machine learning

* v
* t
* e

Natural language processing

General terms|

* AI-complete
* Bag-of-words
* n-gram
* Bigram
* Trigram
* Computational linguistics
* Natural-language understanding
* Stop words
* Text processing

Text analysis|

* Collocation extraction
* Concept mining
* Coreference resolution
* Deep linguistic processing
* Distant reading
* Information extraction
* Named-entity recognition
* Ontology learning
* Parsing
* Part-of-speech tagging
* Semantic role labeling
* Semantic similarity
* Sentiment analysis
* Terminology extraction
* Text mining
* Textual entailment
* Truecasing
* Word-sense disambiguation
* Word-sense induction

| Text segmentation|

* Compound-term processing
* Lemmatisation
* Lexical analysis
* Text chunking
* Stemming
* Sentence segmentation
* Word segmentation

|

Automatic summarization|

* Multi-document summarization
* Sentence extraction
* Text simplification

Machine translation|

* Computer-assisted
* Example-based
* Rule-based
* Statistical
* Transfer-based
* Neural

Distributional semantics models|

* BERT
* Document-term matrix
* Explicit semantic analysis
* fastText
* GloVe
* Latent semantic analysis
* Word embedding
* Word2vec

Language resources,
datasets and corpora|

| Types and
standards|

* Corpus linguistics
* Lexical resource
* Linguistic Linked Open Data
* Machine-readable dictionary
* Parallel text
* PropBank
* Semantic network
* Simple Knowledge Organization System
* Speech corpus
* Text corpus
* Thesaurus (information retrieval)
* Treebank
* Universal Dependencies

|

Data|

* BabelNet
* Bank of English
* DBpedia
* FrameNet
* Google Ngram Viewer
* UBY
* WordNet

Automatic identification
and data capture|

* Speech recognition
* Speech segmentation
* Speech synthesis
* Natural language generation
* Optical character recognition

Topic model|

* Document classification
* Latent Dirichlet allocation
* Pachinko allocation

Computer-assisted
reviewing|

* Automated essay scoring
* Concordancer
* Grammar checker
* Predictive text
* Spell checker
* Syntax guessing

Natural language
user interface|

* Chatbot
* Interactive fiction
* Question answering
* Virtual assistant
* Voice user interface

Other software|

* Natural Language Toolkit
* spaCy

0.00

(0 votes)

Original source: https://en.wikipedia.org/wiki/LaMDA. Read more

|

Retrieved from "https://handwiki.org/wiki/index.php?title=Philosophy:LaMDA&oldid=2467252"
*[v]: View this template
*[t]: Discuss this template
*[e]: Edit this template