Joanna J Bryson

Interview with Dr Joanna J Bryson

Lecturer, Department of Computer Science
University of Bath, Bath, BA2 7AY, United Kingdom
Personal website

What do you think cognitive systems are?

Let me start with cognition. I think cognition is choosing action through a process of real-time search. This is in contrast to completely reflexive action selection, e.g. by look-up based on the current agent and environmental context. However, since we know full search is intractable, in fact all intelligence is at most partially cognitive, the rest provided by evolution, design or another form of prior learning.

Cognitive systems are just systems that perform cognition. This implies that they are real-time agents situated in complex dynamic environments expected to perform intelligent behavior that cannot be entirely prespecified. The most typical examples are domestic robots (which need to interact with humans and other domestic pets and robots), but this definition applies equally to other sorts of artifacts such as intelligent environments (e.g. houses that assist the disabled), "believable" computer game characters and interactive tutors. It also applies to animals.

What is your area of research within Cognitive Systems?

There are two aspects of my research. From an engineering perspective, I work on making it easier to create agents capable of human-like behaviour. This includes figuring out how much of the problem of a cognitive system we should leave the system itself to solve (e.g. by planning and learning), and how we can leverage the knowledge of developers and users to make the agents smarter faster.

From a scientific perspective, I try to understand intelligent behavior in real animals. I used to focus on understanding how modular interacting learning systems gave rise to animal behavior, but then I got very interested in the difference between modular individual agents and multiple agents in a society, and how intelligence evolves and develop in both models. 

Currently I am researching the origins of primate social structure and social intelligence. I want to understand why only one species currently on Earth seems to be increasingly leveraging cultural evolution rather than biological evolution to accumulate intelligence.  Given how quickly the system works, and that we now know many species transmit behavior socially, it is interesting that we seem unique in this.

Why did you become a researcher?

I always wanted to be a researcher. When I was five after a unit on dinosaurs in 1st grade, I wanted to be a paleontologist so I could study them all day long. When I was in middle school, I wanted to be one of those people on Nova (a US science show) that the narrator comes and asks interesting questions, and they sit in these offices full of books and answer them, looking smart & enthusiastic. It's just what I always wanted.

How did you get into Cognitive System research?

I think you have to balance what you love with what you are good at. By the time I got to middle school I knew I was really interested in animal behavior. By university I was interested in human intelligence too, particularly what we shared with animals. But the best jobs on campus were for people who could teach programming, so I took a computer science course and got an A so I'd be invited to help tutor it.

From there I eventually did a conversion MSc in AI at the University of Edinburgh, and that's where I got involved in robotics. John Hallam taught this great course I only applied for because it involved LEGO, but it included control theory and behaviour-based AI, and I thought it was the most interesting thing going on in AI.

Where did you study and what subjects did you study?

I studied Behavioural Science at  University of Chicago, then AI at Edinburgh, then Computer Science at MIT (with a little more Psychology at Edinburgh thrown in during the middle.) Also, I worked in the financial industry between my first and second degree -- it was during the PC revolution. We were just starting to try to network PCs, so I learned a lot about distributing control and data which has helped me throughout my research. If you don't have experience building real-time systems, I think it's hard to really understand cognition, though clearly some people manage to imagine it themselves.

Can you describe briefly how you are doing what you do?

Currently I am building artificial-life models of theories of non-human primate cognition. This lets me both do science (by testing and extending established theories) and engineering (by testing, breaking and extending my own AI software.) My students also work on other systems, such as AI game characters and virtual reality avatars for assisting the elderly.

What are the techniques used in your research?

Agent-based modelling, also real-time action selection, and relatively standard machine learning techniques.

Can you tell me why they are important?

People are not good about reasoning about time or concurrency. I think this is because our consciousness has evolved to handle one thing that otherwise we are not naturally good at -- sequencing actions. So this is what we have language about and theories of. But most of intelligence is a massively concurrent process, both within and between individuals. And time matters.

AI right now is where mechanics was in the time of Decartes --- it's as if events have to slide against each other to exist.  We need a Newtonian revolution where we come to understand how actions are suspended across time.

What are the major implications of your work?

I suppose the major implication of my work is helping people understand themselves better, for example understanding that cognition and consciousness are only a small part of our intelligence. We can't fully understand or control ourselves just by wanting to. For engineering, I think the main implication of my work is getting people to realize intelligence is not just one homegenous system, rather it is a system composed of many parts, some of which operate in very different ways.

I hope to help people understand there are computational similarities between all evolved systems, whether they are single animals or communities, and between the systems we build. But also to understand the differences between the different structures for intelligence we see --- what these differences mean and what they can be used for. Finally, I think my work has important implications for what we can really expect from AI in terms of cognition, and about what roles artificial cognitive systems will have in our lives and in our societies.

Who will benefit from your research / techniques?

This is always difficult to predict. I am publishing in proper science journals and presenting in scientific meetings, so in some ways from that alone humanity benefits. I know that people who download my action selection system include educators, games programmers, public policy generators and military contractors. I've worked directly with people looking to create intelligent environments to assist people with dementia.

What skills do you think are most important to a Cognitive Systems researcher?

Discipline and organisation --- Cognitive systems are extremely complex and to keep track of them takes structure. Scientific / experimental skills --- cognitive systems' behavior is dependent on complex dynamic environments and therefore will always be stochastic. So they can only really be assessed in rigerous experimental frameworks. And of course real-time programming skills. Many cognitive systems also require knowledge of either graphics and/or mechanical and electrical engineering. Real contributions can be made to the field with any of these skill sets though.

What do you think is most satisfying about Cognitive Systems research?

For me, it's that as I make progress, I suddenly get insights into myself and the things, animals and people around me. I've been working in this area for 15 years and I keep learning more and more.

It is also nice to build something that really moves around & acts intelligent --- this is what AI is supposed to be about!

What do you consider is the most challenging about being a Cognitive Systems researcher?

It can be a very, very long time between having a good idea and then testing it. This isn't like theoretical computer science or chemistry. There are so many interacting components in cognitive systems that maintaining them all and understanding their interactions is very difficult. 

When I went to MIT in 1993 I had the good fortune to work with the leader of my field, Rod Brooks, on a humanoid robot project. He told us we would spend 6 months building the prototype of a baby robot (he was originally thinking of just one camera and one arm/hand per robot, much like what Deb Roy eventually built.) Then we would send the prototype to a shop and by the Fall of 1994 every PhD student would have one on their desk and be running experiments with it. I am still waiting for a robot like this I could have on my desk!

What do you think are the main challenges for the future?

The future is now! The challenges are complexity, concurrency and mechanical reliability.

There are several discussions or debates associated with Cognitive Systems research.

Could you mention issues relating to your work?

One issue is whether a system needs robotics to be cognitive. Personally I think being situated in time is the only requirement. The more the agent has human-like constraints, the more human-like an intelligent solution will look. But robot control can be even more different from animal control than say VR control is.

Another debate is about the importance of language and communication. Language clearly affects human cognition, but I believe most animals with brains can be said to be cognitive, and that human intelligence is not entirely non-linear from these other systems. For example, the basics of perception, action & motivation we share with other primates, and the basic mathematical problems of computing we share with all animals, and to a large degree with computers.

Can you outline the arguments of the opposing sides of the debate?

I did above implicitly. If you think you need robots to be cognitive, then you think other artifacts are not human-like enough to need cognition. If you think you need language for cognition, then you think that language enabled a fundamental change in intelligent behavior and we have no commonality with other animals, at least behaviourally. I think anyone who has worked with primates finds that laughable, but I have known people to argue it.

Thank you!

Text only
An euCognition Network Action
Department of Computer Science
University of Bath
Send us an email:
euCognition -
HomeFor PressFor KidsFor AcademicsAbout Us
Meet the Scientists
Explore the Debates
Study at University
Robot Gallery
Home > Meet the Scientists > Dr Joanna J Bryson