On September 1, 2020, Mislav Juric interviewed Steve Omohundro for his series “Interesting Conversations with Mislav Juric”:
In this podcast episode, I have a conversation with Steve Omohundro. Steve is one of the first people to point out the potential dangers of advanced AI systems and in this podcast we discuss topics related to AI, mainly personal AI and AGI (Artificial General Intelligence). Hope you enjoy!
Here is the video version:
and the audio version:
Things mentioned in this podcast episode:
- TikTok
- Snapchat
- Sapiens: A Brief History of Humankind
- Possibility Research
- The Bitter Lesson
- GPT-3
- Safe-AI Scaffolding
- Software as a service (SaaS)
- The Origin of Species
- The Moral Animal: Why We Are, the Way We Are: The New Science of Evolutionary Psychology
- Antifragile: Things That Gain from Disorder
- Nonzero: The Logic of Human Destiny
- The Basic AI Drives
- Future of Life Institute
- Future of Humanity Institute
- Superintelligence
- Human Compatible
- Trolley problem
- Slaughterbots
- Pyro (programming language)
- PyTorch
Timestamps:
- 00:00:00 – 00:01:40 Introduction
- 00:01:40 – 00:06:26 Steve’s experience with startups
- 00:06:26 – 00:10:49 Personal AI
- 00:10:49 – 00:12:28 Steve’s research company
- 00:12:28 – 00:20:37 Combining symbolism and connectionism in AI
- 00:20:37 – 00:25:22 Can GPT-3’s successors eventually build an accurate world model?
- 00:25:22 – 00:30:27 Contributing to AI or AI safety research as an individual?
- 00:30:27 – 00:34:28 Entrepreneurship opportunities for individuals in AI
- 00:34:28 – 00:45:28 Personal AI capabilities
- 00:45:28 – 00:49:14 The outcome of AGI
- 00:49:14 – 00:56:26 The reasoning behind The Basic AI Drives
- 00:56:26 – 01:00:01 Can we mathematically formalize emotions?
- 01:00:01 – 01:03:42 Can we slow down AI progress?
- 01:03:42 – 01:06:35 Next steps for AGI and personal AI
- 01:06:35 – 01:10:39 Ideal educational background for AI researchers?
- 01:10:39 – 01:13:05 How to approach learning math?
- 01:13:05 – 01:14:21 Parting thoughts
Steve Omohundro 
Fascinating conversation! An astronaut once said, “When you’re up here you don’t see borders, you see one beautiful planet with humanity living on it.” When immature narcissistic mindsets take over leadership positions in cult-ures mature philosophies that resonate with our egalitarian social nature can be lost. You said the goal of A.I. “is really to support humanity.” Isn’t this shared goal just a natural result of paying attention to everyone’s experience, of choosing to use your compassionate motivations in your learning brain? You alluded to the problem of economic growth and how it’s not necessarily interested in all humans “cruising along really well” (as you said and as I would love right now). But when individuals focus on their own wealth accumulation and privelaged access to resources that money allows, how COULD their be room for a moral philosophy that wasn’t too full of worry and fear? Along the lines of your thoughts on the Slaughterbots video, I wondered if your concern there is one you are overlooking with the moral dilemmas fantasized in the Trolley problem- “too much worry and fear is not a good thing.” Protecting ourselves and preventing moral dilemmas would seem like a more careful and thoughtful way to move forward in moral philosophy. Zero sum incentives seem to be holding back our best human nature and our best technology. This monopoly game we play isn’t logical. Instead of working to create value for owners, why not do things to honor our value, the value of all humans and even all living beings? Why not have a human standard instead of a gold standard? Would it be that hard to design A.I. to distribute the resources to help every human being live well, to recognize each person’s value, an equal dignity that can never be improved or diminished but honored and optimize the sustainable ways for doing this? Cesar Hidalgo had some interesting ideas for improving democracy in a recent Ted Talk. Isn’t it time to use A.I. for humanity and a new more mature economy as well?
Thanks for the comment Rich! I think AI will enable a much richer palette of possibilities for both governance and for economics. If we can clearly identify the outcomes we truly desire, I think we will be able to soon do a much better job at making out systems deliver those outcome!