In this podcast episode, I have a conversation with Steve Omohundro. Steve is one of the first people to point out the potential dangers of advanced AI systems and in this podcast we discuss topics related to AI, mainly personal AI and AGI (Artificial General Intelligence). Hope you enjoy!
PwC predicts that Artificial Intelligence (AI) will create $90 trillion of value between now and 2030.[1] But this huge economic value only hints at AI’s profound impact on information networks, commerce, and governance. Many are worried that powerful AI will disempower individuals. The Wall Street Journal recently published best selling author Yuval Harari’s commencement speech to the class of 2020 entitled “Rebellion of the Hackable Animals.”[2] He argued that AI will allow corporations and governments to manipulate individuals and challenged the students to find ways to counteract this manipulation.
This article describes “Personal AI” and argues that it will be the antidote to AI-powered manipulation. It will, instead, dramatically empower individuals to reshape their social and economic networks. We define “Personal AIs” as artificial intelligences trusted by individual “owners” to represent them in interactions with other individuals, organizations, and networks. There are great challenges in building personal AIs, but their impact will be profoundly positive for humanity. To understand why, we must first understand the current role of AI in society.
The Rise of Platform AI
Flashy AI applications like self-driving cars, deepfake videos, and the Sophia robot have dominated news headlines. But the AI technology with the greatest economic impact has actually been “recommender systems.”[3] These simple AI systems model users to make recommendations such as movies on Netflix, products on Amazon, and friends on Twitter.
Recommender systems were only invented in the 1990’s but have had an enormous impact. Netflix reports that their movie recommender has been responsible for creating more than $1 billion of business value. Amazon’s recommenders generate 35% of the purchases on their site. ByteDance, the parent company of TikTok, was recently privately valued at $140 billion primarily due to their innovative recommender AI.
One reason that recommender systems have had such a big impact is that they enable the “Platform Business Model.” Platform companies match up producers and consumers and take a cut from each transaction. For example, Uber’s AI connects nearby drivers with people who need rides.
The platform business model creates sustainable outsized profits and is responsible for the rise of the most valuable companies over the past 15 years. In 2004, the top ten companies were General Electric, Exxon, Microsoft, Pfizer, Citigroup, Walmart, BP, AIG, Intel, and Bank of America. By 2019, they were Microsoft, Amazon, Apple, Alphabet, Facebook, Berkshire Hathaway, Alibaba, Tencent, Visa, and Johnson and Johnson.[4] Seven of these are based on an AI-driven platform business model.
According to Applico, 60% of the billion-dollar “unicorn” startups are platform companies and most IPOs and acquisitions also make use of this model. It is estimated to have created over $3 trillion in market capitalization.
Many aspects of platform companies are counter-intuitive from a traditional business perspective. A popular meme states that:
● Uber, the world’s largest taxi company, owns no vehicles.
● Airbnb, the largest accommodation provider, owns no real estate.
● Facebook, the most popular media provider, creates no content.
● Instagram, the most valuable photo company, sells no cameras.
● Netflix, the fastest-growing television network, lays no cables.
● Alibaba, the most valuable retailer, has no inventory.
While recommender systems are critical to platforms, several other forms of AI are also important. On the producer side, platform companies provide: AI-driven content creation tools, AI-driven auctions for placement, AI-driven A/B testing for optimization, AI analytics to track performance, AI-based producer reputations and AI-driven malicious content blocking. On the consumer side, platform companies use: AI-based gamification for engagement, AI-personalized marketing, AI-driven pricing, AI-based consumer reputation and AI-driven malicious consumer blocking. Each of these functions will improve as AI technologies improve.
The remarkable rise of platform companies can be understood through “Coase’s Theorem.” Ronald Coase was an economist in the 1930s who studied the nature of the firm. Economists understood that market mechanisms produced efficient results and Coase wondered why firms weren’t organized as markets internally. He showed that if information and contracting were inexpensive enough, then market mechanisms produce the most efficient outcomes. He concluded that traditional firms are organized hierarchically because business information was not freely available and contracting was too expensive.
AI dramatically lowers the costs of both information gathering and contracting. Traditional taxi companies owned their own cars, hired drivers as employees, and had managers who determined which car would transport which customer. Uber’s AI systems enable their cellphone app to turn the traditional taxi company “inside out” and to profit by intermediating between external drivers and riders.
This “inversion of the firm” is also happening in HR, marketing, innovation, finance, logistics, etc. An extreme example was Instagram which had only 13 employees when it was bought by Facebook for $1 billion. This remarkable purchase has been called the “most brilliant tech acquisition ever made.”
Many of the consequences of the platform revolution are quite positive for society. Airbnb unlocked resources (people’s spare bedrooms) which would otherwise have gone unused. Individual consumer needs can be better met by platforms (eg. the long tail of demand met by Amazon’s many sellers). Platforms enable more producers (eg. Uber’s many part-time drivers). We can understand Platform AI as creating both business value and social value.
Platform companies gain value through network effects on both the producer and the consumer side. These networks create strong “moats” around their businesses and allow them to sustain outsized profits. In typical platform niches, one company is dominant (eg. Uber) with a much smaller company in second-place (eg. Lyft) and third-place being insignificant. The strong position of the dominant company gives them great power in interactions with both producers and consumers. As AI improves, you might think that this platform power will only increase and that Harari’s fears are justified.
Platforms use their power over producers to gain the advantage. Uber has been criticized for squeezing drivers and taking a bigger share of profits. Amazon has repeatedly created their own branded versions of products which they observe are profitable for third-party sellers. Netflix notices what elements of movies and TV shows are most liked by customers and creates their own shows using that knowledge.
Platforms also use their power over consumers. Platform advertising has been criticized for being manipulative and for rewarding click-bait headlines. YouTube has been blamed for “radicalizing” viewers who watch a video out of curiosity and then receive recommendations for increasingly extreme related videos. Deceptive news stories generate outrage which causes clicks and recommender systems incentivize their creation in a vicious loop. There is increasing concern about privacy and the use of personal information by platform companies.
The Rise of “Personal AI”
If the simple AI underlying platform companies have had such a transformative societal effect, what will be the impact of more powerful AI? All indications are that AI is improving at a rapid pace and is likely to power another phase of Coase’s theorem. This will create more market-like structures and will spread power throughout networks. While AI is the enabler, the underlying forces are economic.
Two technological trends, “Moore’s Law” and “Nielsen’s Law,” are driving the improvement in AI. Moore’s Law says that the number of transistors in a CPU grows by 60% per year and has held since 1970. Nielsen’s Law says that internet bandwidth grows by 50% per year and has held since 1984. Together, they give AI learning systems increasing amounts of computation and data to improve independently from algorithmic innovation.
But learning and reasoning algorithms are also rapidly improving. The last decade has seen dramatic improvements in machine vision, natural language processing, and game playing. As advanced AI becomes more commercially viable, it attracts more investment, students, researchers, and practitioners.
Rich Sutton’s influential essay “The Bitter Lesson”[5] argued that simple algorithmic techniques like search and statistical learning have always overcome clever human-designed algorithms as computation and data increase. OpenAI’s GPT-3 “transformer” language model is essentially a scaled-up version of their GPT-2 model, but exhibits a wide range of new behaviors. Many are speculating that scaling up this class of models by another factor of 10 or 100 may lead to dramatically improved AI systems.
What will these more powerful AI’s be used for? “Digital Twins” are an AI application that has seen increasing interest over the past decade. These are digital AI replicas of living or non-living physical systems. The physical systems are continuously monitored by sensors which are used to update the corresponding AI twin models. The digital twin models are then used for estimation, diagnosis, policy design, control, and governance. Each of these is first tested on the twin and then deployed on the real system. Monte Carlo simulations estimate interactions between multiple twins for game-theoretic analysis, contract design, and analysis of larger system dynamics.
“Personal AIs” are related to digital twins but model a human “owner” and act for that owner’s benefit. They are trusted AI agents which model their owners’ values, beliefs, and goals, are continually updated based on their owner’s actions, and act as the owner’s proxy in interacting with other agents. They filter ads, news, and other content according to their owners’ preferences. They control the dissemination of the owner’s personal information according to the owner’s preferences. They continually search for new business and purchase opportunities for their owners. They communicate their owners’ preferences to governmental and other organizations. When personal AIs become widespread, they will have a profound impact on the nature of human society.
What AI advances are needed to create personal AIs? Simple versions could be built today but powerful versions will require advances in natural language processing, modeling of human psychology, and smart contract design. Each of these areas is undergoing active research and powerful personal AIs should be possible within a few years.
The simplest personal AI contract is making a purchase. If an owner trusts their personal AI, they will allow it to search Amazon and other sellers for the best product at the best price for their needs. More complex contracts will allow an owner to contract to watch a video in return for watching ads that meet their value criteria. More complex purchase contracts could include terms for insurance, shipping, return policies, and put constraints on the sourcing of components and labor. As personal AIs become more powerful, contracts can become arbitrarily complex. A new era of highly personalized purchases and interactions will follow that better meets each person’s needs and desires.
Personal AI will dramatically change the nature of marketing. If an owner knows they are emotionally vulnerable to depictions of alcohol, fast cars, or chocolate cake, they can instruct their personal AI to refuse advertising with that content. In today’s internet, recommender systems might discover an owner’s vulnerability and start specifically showing them the manipulative content they are sensitive to because it generates a stronger response. This is disempowering for the viewer and harmful for society.
With personal AI negotiation owners can block manipulative advertisements and only enable calm, informative ads about products they are interested in. If enough individuals use personal AIs, advertisers will no longer have an incentive to create manipulative ads. Cigarette advertising was only banned after governmental intervention, but personal AIs provide a more effective direct mechanism to move advertising in a positive direction.
Personal AI will also dramatically change the nature of social media. Today’s popular social media sites have power because no one wants to spend time on sites that their friends aren’t on. Lock-in is maintained by the annoyance of maintaining accounts on multiple sites. Each site has its own user interface, profiles, password, and identity system. Tracking content on multiple sites is time-consuming and confusing for users. But powerful personal AIs will easily be able to interface with multiple social media sites. They will present their owners with unified interfaces for information from a wide variety of sites personalized to their owner’s tastes. The owner need not even be aware of which site particular messages or interactions are from. This new flexibility will put additional pressure on social media sites to truly meet their user’s needs rather than relying on the power of network effects for lock-in.
Personal AI will also dramatically change the nature of governance. Today, voting gives citizens a small bit of influence over governmental decisions. But the expense and complexity of voting mechanisms means that elections happen rarely and only support a limited expression of preferences. New voting procedures like “range voting”, “quadratic voting”, and “liquid democracy” would improve the current system. But personal AIs will allow detailed “semantic voting” in which citizens can express their ideas and preferences in real-time. Governments will be able to create detailed models of their citizen’s actual needs moment by moment.
Personal AI will also dramatically change the nature of commerce. Instead of being locked into a few online marketplaces, personal AIs can explore the entirety of the web for products and deals. Complex negotiations with a wide variety of sellers will allow personalized contracts that better meet the owner’s true needs. As increasing numbers of people shop using personal AIs, this will change the nature of commerce. Buyers will be able to demand greater transparency about supply chains, counterfeiting, and forced labor. They will be able to know the exact history of a product and the exact ingredients in food and supplements.
Perhaps the largest impact of personal AI will be in the transformation of information gathering. The internet shifted news from a few powerful channels to a wide variety of sources and networks. Unfortunately, this has also enabled the spread of disinformation and misinformation. Recent AI technologies can create fake text, audio, images, and video which is indistinguishable from real content. Various groups are developing AI to detect fake content but it appears that the fakers will ultimately win the arms race. That means that careful tracking of the source and “provenance” of content will be fundamental to future information networks. Today, various gatekeepers are attempting to take control of “fact-checking” and information tracking but many are themselves being questioned.
Personal AI enables individuals to choose their own sources of validation. New sources of validation, reputation, and information tracking will arise and personal AIs will be able to choose among these according to their owner’s preferences. “Liquid Democracy” allows voters to delegate their votes to trusted knowledgeable third parties (eg. the Sierra club) who may in turn delegate their votes to even more informed groups. A similar mechanism can be used to create networks of information validated by an owner’s trusted groups. The societal effect of these kinds of information networks will be to democratize knowledge and to weaken the power of centralized information sources.
Our Empowered AI Future
W. Edwards Deming helped create the Japanese “post-war economic miracle” from 1950–1960. He proposed management and manufacturing processes that dramatically improved Japanese productivity and the quality of their goods. The Japanese word “Kaizen” means “change for the better” and has come to represent continuous improvement of all functions and full engagement of all stakeholders. Personal AI will enable a kind of “Deming 2.0” for the whole of society.
Interactions between an owner and their personal AI continuously improves the AI’s model of its owner’s ideas, values, and beliefs. Interactions between personal AIs and AIs associated with larger groups will enable those groups to integrate the detailed knowledge and needs of all stakeholders in a kind of societal “Kaizen”. This responsive interaction will happen from the local level up to the global level improving effectiveness at all scales.
The impact on the global level is especially interesting given the huge number of global crises we are currently struggling with: climate change, pandemic, economic crises, poverty, pollution, and transformative technological change. The United Nations maintains a list of the 17 most important “Sustainable Development Goals.”[6] Every one of these goals can be addressed with advanced artificial intelligence and extensive networks of personal AIs will enable every human to contribute their perspective.
The picture of our future that emerges when we include the personal AI revolution is a far cry from the “Hackable Animals” dystopia that Harari worries about. It is a future of extensive inclusiveness and individual empowerment. It is a future in which global problems are solved through careful consideration of every human’s needs and ideas. It is a future in which empowered networks enable each person to contribute and connect to the whole of humanity through their unique individual gifts.
On August 27, 2020, Dan Fagggella’s Emerj AI Futures published the podcast “The Transition to AGI Governance – with Dr. Steve Omohundro (S1E10)”:
Today’s guest is the great and brilliant Dr. Steve Omohundro, Chief Scientist at AIBrain. AIBrain is creating Turingworld, a powerful AI learning social media platform based on AI-optimized learning, AI-powered gamification, and AI-enhanced social interaction. Dr. Steve Omohundro received his Ph.D. in Physics from U.C. Berkeley. He also founded an organization to support AI safety and another organization to advance new intelligence architectures based on the programming language Omda, the specification language Omex, and the semantics language Omai. Episode topics include: how humans can build safe AI, what facets of AI development might/might-not require global governance, how the international community might best collaborate to prioritize AGI development efforts, and how AI may influence our lives as consumers.
On July 3, 2020, Steve Omohundro and Puja Ohlhaver discussed “Pluralism Through Personal AIs” at the 2020 RadicalxChange Conference:
Artificial Intelligence is transforming every aspect of business and society. The usual narrative focuses on monolithic AIs owned by large corporations and governments that promote the interests of the powerful. But imagine a world in which each person has their own “personal AI” which deeply models their beliefs, desires, and values and which promotes those interests. Such agents enable much richer and more frequent “semantic voting” improving feedback for governance. They dramatically change the incentives for advertisers and news sources. When personal agents filter manipulative and malicious content, it incentivizes the creation of content that is aligned with a person’s values. Economic transactions, social interactions, personal transformation, and ability to contribute to the greater good will all be dramatically transformed by personal AI agents. But there are also many challenges and new ideas are needed. Come join this fireside chat to discuss the possibilities and perils of personal AIs and how they relate to the RadicalXChange movement.
SPEAKERS
Puja Ohlhaver is a technologist and lawyer who explores the intersection of technology, democracy, and markets. She is an advocate of digital social innovation, as a path to rebooting democracy and testing regulatory innovations. She is an inventor and founder of ClearPath Surgical, a company that seeks to improve health outcomes in minimally invasive surgery. She holds a law degree from Stanford Law School and was previously an investment management attorney.
Steve Omohundro has been a scientist, professor, author, software architect, and entrepreneur and is developing the next generation of artificial intelligence. He has degrees in Physics and Mathematics from Stanford and a Ph.D. in Physics from U.C. Berkeley. He was an award-winning computer science professor at the University of Illinois at Champaign-Urbana and cofounded the Center for Complex Systems Research. He is the Chief Scientist of AIBrain and serves on its Board of Directors. AIBrain is creating new AI technologies for learning, conversation, robotics, simulation, and music and has offices in Menlo Park, Seoul, Berlin, and Shenzhen. It is creating Turingworld, a powerful AI learning social media platform based on AI-optimized learning, AI-powered gamification, and AI-enhanced social interaction. He is also Founder and CEO of Possibility Research which is working to develop new foundations for Artificial Intelligence based on precise mathematical semantics and Self-Aware Systems which is working to ensure that intelligent technologies have a positive impact. Steve published the book “Geometric Perturbation Theory in Physics”, designed the first data parallel language StarLisp, wrote the 3D graphics for Mathematica, developed fast neural data structures like balltrees, designed the fastest and safest object-oriented language Sather, invented manifold learning, co-created the first neural focus of attention systems, co-designed the best lip reading system, invented model merging for fast one-shot learning, co-designed the best stochastic grammar learning system, co-created the first Bayesian image search engine PicHunter, invented self-improving AI, discovered the Basic AI Drives, and proposed many of the basic AI safety mechanisms including AI smart contracts. Steve is an award-winning teacher and has given hundreds of talks around the world. Some of his talks and scientific papers are available here. He holds the vision that new technologies can help humanity create a more compassionate, peaceful, and life-serving world.
Here is the conference website with the other presentations:
On July 2, 2020, Steve Omohundro gave a keynote talk on “From Scientist to Entrepreneur” to the “Deep Dive into the Silicon Valley” Schoolab course by BPIFrance.
On July 1, 2020, Steve Omohundro gave a talk on GPT-3 and it’s implications for artificial intelligence to Numenta’s Research Meeting:
In this research meeting, guest Stephen Omohundro gave a fascinating talk on GPT-3, the new massive OpenAI Natural Language Processing model. He reviewed the network architecture, training process, and results in the context of past work. There was extensive discussion on the implications for NLP and for Machine Intelligence / AGI.
On June 17, 2020, Steve Omohundro spoke about “Platform AI, Personal AI, and Global AI” to The Hive’s excellent “Think Tank” group:
Platform AI, Personal AI, and Global AI
Wednesday, Jun 17, 2020, 11:00 AM
Online event ,
70 Data Bees Went
WEBINAR URL: https://us02web.zoom.us/webinar/register/WN_Ti0L6e1AQReihaTNVgINNg ABSTRACT: Simple artificial intelligence has transformed the world economy over the past 15 years. In 2004, the 5 largest companies were GE, Exxon, Microsoft, Pfizer, and Citigroup. By 2019, they were Microsoft, Amazon, Apple, Alphabet, and Facebook, all based on the po…
Simple artificial intelligence has transformed the world economy over the past 15 years. In 2004, the 5 largest companies were GE, Exxon, Microsoft, Pfizer, and Citigroup. By 2019, they were Microsoft, Amazon, Apple, Alphabet, and Facebook, all based on the powerful “AI Platform Model”. These companies use simple AI-based search, recommendation, matchmaking, ad serving, and malicious content filtering to create new channels between producers and consumers. Today’s most valuable startup is ByteDance (recently valued at $140 billion) whose TikTok platform is driven by 3 simple AI technologies.
As AI becomes more powerful, these basic channels will expand into a wide array of new forms of business and social interaction. We argue that every person will have a trusted “Personal AI” that promotes their interests and filters content and interactions not aligned with their values. At a large scale, “Global AI” will improve governance through increasingly detailed world simulations to manage global challenges like pandemics, global warming, financial crises, etc. New social mechanisms like “quadratic voting” and “semantic voting” will enable society to better meet citizen’s needs. AI will help people filter false and manipulative content which will shift the incentives for advertisers and news sources. The impact of this “Multi-Scale AI” is likely to be immense. We describe recent ideas from the science of complex systems that help us to analyze and manage it.
SPEAKER:
Steve Omohundro has done fundamental research in AI for the past 35 years. He has a PhD in physics, was an AI professor at the University of Illinois, was a scientist at several research labs and worked with many startups. He is the Chief Scientist at AIBrain, works with Facebook on bringing advanced technologies to ad serving, and founded Possibility Research to help ensure that AI will be beneficial for humanity. He co-founded one of the first complex systems institutes, designed the first data-parallel programming language, invented manifold learning, co-developed the first image recommender system, co-developed the first attention-driven neural nets, co-built the first lip reading system, and developed many other learning algorithms. His work on AI’s social impact was featured in the book “Our Final Invention” and he appears in the recent Universal Pictures documentary “We Need to Talk About AI”.
Steve Omohundro was interviewed for the Universal Pictures documentary film “We Need to Talk About AI” which was released in the United States on May 18, 2020. It explores the impact of AI in an even-handed way and features James Cameron and a number of AI scientists.
Steve Omohundro 