And just for a moment he forgot, or didn’t want to remember, that other robots might be more ignorant than human beings. His very superiority caught him.
Dr. Susan Calvin in “Little Lost Robot” by Isaac Asimov, first published in Astounding Science Fiction, 1947 and anthologized by Isaac Asimov in I, Robot, Gnome Press, 1950.
Version 0.1 (Revision history at the bottom) December 28th, 2023
When I was a doctoral student working on my thesis in computer science in an earlier heyday of artificial intelligence, if you’d ask me how I how I’d find out why a program did not perform as expected, I would come up with a half dozen heuristics, most of them near cousins of standard computer programming debugging techniques.1 Even though I was a diehard science fiction reader, I gave short shrift to the techniques illustrated by the expert robopsychologist—Dr. Susan Calvin—introduced into his robot short stories in the 1940’s by Isaac Asimov. These seemed more akin the the logical dissections performed by Conan Doyle’s Sherlock Holmes than anything I could recognize as computer science.
Yet over the last five years, particularly since 2020, English (and other language) prompts—human-like statements or questions, often called “hard prompts” to distinguish them from “soft prompts”2 —have come into wide use. Interest in hard prompts grew rapidly after the release of ChatGPT and was driven by creative individuals who figured out, through experimentation, which prompts worked particularly well for specific tasks. This was jarring to many computer scientists such as Andrej Karpathy who declared “The hottest new programming language is English.” Ethan and Lilach Mollick are exemplars of non-computer scientist creatives that have pushed the envelope in their own domain using mastery of hard prompts. They have been inspired leaders in developing sets of prompts for many common educational tasks that resulted in functionality that has surpassed and replaced whole suites of commercial educational software.
After the initial culture shock, many researchers have started working on ways to automate optimization of hard prompts (e.g. Wen et al., Sordoni et al.) How well this works for all applications of generative AI (now less frequently referred to as large language models, and foundation models, even though technically they do not denote the same thing) in medicine in particular remains to be determined. I’ll try to write a post about optimizing prompts for medicine soon, but right now, I cannot help but notice that in my interactions with GPT-4 or Bard, when I do not get the answer I expect, my interactions resemble a conversation with a sometimes reluctant, sometimes confused, sometimes ignorant assistant who has frequent flashes of brilliance.
Early on, some of the skepticism about the performance of large language models centered on the capacity of these models for “theory of mind” reasoning. Understanding the possible state of mind of a human was seen as an important measure of artificial general intelligence. I’ll step away from the debate of whether or not GPT-4, Bard et al, show evidence of theory of mind but instead posit that having of theory of the “mind3” of the generative AI program gives humans better results when using such a program.
What does it mean to have a theory of the mind of the generative AI? I am most effective in using a generative AI program when I have a set of expectations of how it will respond to a prompt based on both my experience with that program over many sessions and its responses so far in this specific session. That is, what did they “understand” from my last prompt and what might that understanding be as informed by my experience with that program? Sometimes, I check on the validity of my theory of their mind by asking an open ended question. This leads to a conversation which is much closer to the work of Dr. Susan Calvin than to that of a programmer. Although the robots had complex positronic brains, Dr. Calvin did not debug the robots by examining their nanoscale circuitry. Instead she conducted logical and very rarely emotional conversations in English with the robots. The low level implementation layer of robot intelligence were NOT where her interventions were targeted. That is why her job title was robopsychologist and not computer scientist. A great science fiction story does not serve as technical evidence or a scientific proof but thus far it has served as a useful collection of metaphors for our collective experience working with generative AI using these Theory of AI-Mind (?TAIM) approaches.
In future versions of this post, I’ll touch on the pragmatics of Theory of AI-Mind for effective use of these programs but also on the implications for “alignment” procedures.
| Version | |
| 0.1 | Initial presentation of theory mind of humans vs programming generative AI with a theory of mind of the AI. |
- Some techniques were more inspired by the 1980’s AI community’s toolit including dependency directed backtracking and Truth Maintenance Systems. ↩︎
- Soft prompts are frequently implemented as embeddings, vectors representing the relationship between tokens/words/entities across a training corpus. ↩︎
- I’ll defer the fun but tangential discussion of what mind means in this cybernetic version of the mind-body problem. Go read I Am A Strange Loop if you dare, if you want to get ahead of the conversation. ↩︎
