Interesting Conversations with Mislav Juric Interview with Steve Omohundro

On September 1, 2020, Mislav Juric interviewed Steve Omohundro for his series “Interesting Conversations with Mislav Juric”:

In this podcast episode, I have a conversation with Steve Omohundro. Steve is one of the first people to point out the potential dangers of advanced AI systems and in this podcast we discuss topics related to AI, mainly personal AI and AGI (Artificial General Intelligence). Hope you enjoy!

Here is the video version:

and the audio version:

Things mentioned in this podcast episode:


  • 00:00:00 – 00:01:40 Introduction
  • 00:01:40 – 00:06:26 Steve’s experience with startups
  • 00:06:26 – 00:10:49 Personal AI
  • 00:10:49 – 00:12:28 Steve’s research company
  • 00:12:28 – 00:20:37 Combining symbolism and connectionism in AI
  • 00:20:37 – 00:25:22 Can GPT-3’s successors eventually build an accurate world model?
  • 00:25:22 – 00:30:27 Contributing to AI or AI safety research as an individual?
  • 00:30:27 – 00:34:28 Entrepreneurship opportunities for individuals in AI
  • 00:34:28 – 00:45:28 Personal AI capabilities
  • 00:45:28 – 00:49:14 The outcome of AGI
  • 00:49:14 – 00:56:26 The reasoning behind The Basic AI Drives
  • 00:56:26 – 01:00:01 Can we mathematically formalize emotions?
  • 01:00:01 – 01:03:42 Can we slow down AI progress?
  • 01:03:42 – 01:06:35 Next steps for AGI and personal AI
  • 01:06:35 – 01:10:39 Ideal educational background for AI researchers?
  • 01:10:39 – 01:13:05 How to approach learning math?
  • 01:13:05 – 01:14:21 Parting thoughts


  1. Fascinating conversation! An astronaut once said, “When you’re up here you don’t see borders, you see one beautiful planet with humanity living on it.” When immature narcissistic mindsets take over leadership positions in cult-ures mature philosophies that resonate with our egalitarian social nature can be lost. You said the goal of A.I. “is really to support humanity.” Isn’t this shared goal just a natural result of paying attention to everyone’s experience, of choosing to use your compassionate motivations in your learning brain? You alluded to the problem of economic growth and how it’s not necessarily interested in all humans “cruising along really well” (as you said and as I would love right now). But when individuals focus on their own wealth accumulation and privelaged access to resources that money allows, how COULD their be room for a moral philosophy that wasn’t too full of worry and fear? Along the lines of your thoughts on the Slaughterbots video, I wondered if your concern there is one you are overlooking with the moral dilemmas fantasized in the Trolley problem- “too much worry and fear is not a good thing.” Protecting ourselves and preventing moral dilemmas would seem like a more careful and thoughtful way to move forward in moral philosophy. Zero sum incentives seem to be holding back our best human nature and our best technology. This monopoly game we play isn’t logical. Instead of working to create value for owners, why not do things to honor our value, the value of all humans and even all living beings? Why not have a human standard instead of a gold standard? Would it be that hard to design A.I. to distribute the resources to help every human being live well, to recognize each person’s value, an equal dignity that can never be improved or diminished but honored and optimize the sustainable ways for doing this? Cesar Hidalgo had some interesting ideas for improving democracy in a recent Ted Talk. Isn’t it time to use A.I. for humanity and a new more mature economy as well?

  2. Thanks for the comment Rich! I think AI will enable a much richer palette of possibilities for both governance and for economics. If we can clearly identify the outcomes we truly desire, I think we will be able to soon do a much better job at making out systems deliver those outcome!

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s