Soul Machines are the world leader in humanizing AI to create astonishing Digital People. Driven by deep research into neuroscience, psychology and cognitive science- they create autonomous, hyper realistic, digital characters with whom we can interact like any other human being. Built around their Human OS platform, and with a patented Digital Brain, Soul Machines are at the forefront of some of the most exciting advances in computing.
Double Academy Award winner Dr. Mark Sagar is the CEO and co-founder of Soul Machines and Director of the Laboratory for Animate Technologies at the Auckland Bioengineering Institute. He has a Ph.D. in bioengineering and previously worked as the Special Projects Supervisor at Weta Digital and Sony Pictures Imageworks and developed technology for the digital characters in blockbusters such as Avatar, King Kong, and Spiderman 2. His pioneering work in computer-generated faces was recognised with two consecutive Scientific and Engineering Oscars in 2010 and 2011, and Mark was elected as a Fellow of the Royal Society of New Zealand in 2019 in recognition of his world-leading research.
In this exclusive interview, I speak to Mark Sagar about creating digital people, how technology is helping us to understand humanity, and how interaction with intelligent machines will open up the next era of our civilisation.
Q: What did you learn from the world of film about creating realistic digital people?
[Mark Sagar]: Creating digital people is really hard. The quality really matters, especially when it comes to the face. We’re so attuned to seeing faces that we pick up anything that doesn’t look right. Faces communicate so much- when we see someone else, our imaginations go into overdrive.
When we create digital faces, they have to look and behave organically- they have to trigger that part of your brain that starts to think about what that ‘person’ would be like. Some time ago, we did a short film with a digital actress. The point of the film was that it tried to make you think about the introspection of the character, even though she was artificial. When I saw her face on screen, it made me think about her life rather than having an analytical view of her face. That’s when I knew we’d crossed a threshold.
Q: Can we form emotional bonds with digital people?
[Mark Sagar]: We form emotional bonds in interesting and different ways. Some people form emotional bonds with their car- for example. They might get upset if their car gets scratched! It is a real emotional bond with an inanimate object. Then of course you have emotional bonds with pets, and ultimately with people. We are incredibly emotional beings and much of our cognition is tied-up with emotion. Emotions are how we communicate and understand many essential things that are going on – and we need to communicate emotion to cooperate with each other. That’s why we’ve evolved to have so many incredible facial expressions, and the ability to read them. At a glance we can interpret another being – whether they want to do something, whether they need help, whether they’re paying attention, how they feel.
If a machine is able to express like that- it’s a very efficient way of getting time-critical information across or communicating an additional layer. When we’re speaking, we’re saying messages through words but emoting through our faces. Our expressions give cues to what is and is not important in what we’re saying. Adding emotion to the written word allows you to communicate incredibly powerfully.
Q: How are digital people being deployed?
[Mark Sagar]: We have a great project right now with the World Health Organisation where Florence, one of our digital people, is helping to dispel Covid myths and encourage people to stop smoking. Both of these are behavioural changes and that’s where the presence of a digital person amplifies what would normally be just an interaction with a web page into more of a social contract. As humans we will always be more compelled to do something if the interaction is with another person – we have to actively participate in communication.
The ultimate goal is to be in a state of flow with machines. Think about people working with horses, or herding cattle with a dog, they are examples of interactions with other intelligent creatures in a way which is fluid and allows us to achieve something we couldn’t do ourselves. Dogs and horses have superhuman abilities. Dogs are faster and can sense more than we can, and horses are much stronger than us. Think about intelligent machines in the same way- they have superhuman abilities to compute and getting to a similar flow state with our interactions with them would be powerful.
Q: What are the ethics of creating digital people?
[Mark Sagar]: We take ethics very seriously – and it’s important to never fool the user. It must be clear that you are dealing with a digital person, not a real person, and at the same time you cannot create any dependency. It’s also important that we use our technology for social good and so we choose to not work with politics, sex, weapons and things like that.
Through our work, we’re trying to draw attention to humanity in many ways. I personally have a love-hate relationship with technology. Technology can do wonderful things, but we need to spend lots of time with real people, with nature. Technological interactions can- however- be hugely beneficial. For example, if someone has dementia, they need to keep their brain engaged. Ideally this would be done by a human carer, but in the absence of a human, a digital person can provide that service. Our economy has forced us into a state where we cannot afford to have one-to-one care, and so digital people are a form of solution to that.
We’ve also been working with the psychologist, Elizabeth Broadbent, on using digital people as part of a loneliness study. In this case, our digital person is advising people- we have introduced in modules based on kindness, on coaching, and we’re looking into areas such as cognitive behavioural therapy. A real therapist is not available all the time, but a digital therapist can see you whenever you need, and take you through practices such as mindfulness, breathing and so on. There are all kinds of things where we can use technology to reflect our own humanity to better us.
Q: How do you create a technology stack to build digital people?
[Mark Sagar]: The experience of interacting with a digital person is anchored around coherence. Essentially, we’ve been building an artificial nervous system which has a brain that coordinates behaviours, and virtual muscles that create movements. The system has eyes and ears that take information in, and our ‘digital brain’ allows for perception, cognition and simulated emotions. We’ve taken a biological approach because we’re trying to make something that humans can relate to.
Technology is improving all the time. AR and VR aren’t quite there yet, but are getting better and better. That’s where we will start to feel totally immersed. Imagine if your encounter with a digital person has the fidelity of a real experience.
Advances in the speed and capability of computing is critical too. Typically, machines will run all the time and generate lots of heat- they’re super-inefficient. Currently we use CPUs and GPUs but want to explore upcoming technologies such as graph based processors and neuromorphic computing. Neuromorphic computing simulates more how our brains work, where neurons only fire when they get a signal.
[Vikas: Do you have a testbed for this technology?]
[Mark Sagar]: We created a project called BabyX. Essentially, it’s part science project, part philosophy project, and part art project. The aim is to understand the building blocks of human interaction and behaviour.
Before we even touch on neuromorphic computing or anything like that- we have to look at how a system is constructed. We have a spinal cord that directs our limbs. We have a brain-stem which controls our vital functions, and we have a limbic system which controls our emotions and behaviours. On top of this sits a cortex which allows us to plan and imagine and inhibit low level behaviours. Together they act to reduce dimensionality to allow us to take decisions in a complex world. Our environment is unpredictable – not only do we have to deal with the weather, finding food and shelter, we have to deal with other brains! That latter area introduces us to the idea of abstract concepts, of shared myths and shared understandings. This is the world where we manage and interleave the many nuanced relationships we have between us and our children, colleagues and friends.
BabyX is a project that tries to build these layers of cognition. It starts with reflexes (if something comes at you, you’ll startle) but goes on to have emotional and cognitive reactions (which are often entwined). Emotion helps coordinate body-brain responses. If you have a goal which is cognitive, you may feel sad emotionally if you fail to achieve it. That’s your emotional system conditioning you, and pumping chemicals around to modulate your behaviour. Our aim is to create a general model of processing that allows BabyX to respond to situations in the human world. This effectively becomes a platform that allows us to introduce increasing autonomy into our commercial digital people – ultimately allowing them to cooperate with humans.
Q: What do you hope your legacy will be?
[Mark Sagar]: At a scientific level, I hope the work we’re doing with projects like BabyX will help illuminate a better understanding of the working of the mind and how we interact with each other. Digital technologies can actually draw attention to our humanity as we take so much of this for granted. How we behave really affects others, and has repercussions, and by understanding behaviour as deeply as we need to, it can draw attention to ourselves and make us more mindful as a result.
At a philosophical level, I would like our technology to push the discussion on understanding consciousness and sentience – two of the greatest mysteries of our universe. Understanding consciousness and sentience would be incredibly important in our understanding of ourselves, treatment of animals and society.
At a practical level, if this technology could help empower people to do amazing things that would make me very happy. Human cooperation is the greatest force in history- it enabled us to put people on the moon, build microprocessors and advance medicine. Human cooperation with intelligent machines will define the next era of history.