265 – Connectionist Versus Symbolist AI

[00:00:07] David: Hi, and welcome to the IDEMS Podcast. I’m David Stern, founding director of IDEMS, and it’s my pleasure to be here again with Kate Fleming, as we’re continuing our discussions about AI and the AI empires.

[00:00:19] Kate: Hey, David. Yeah, this has all come out of your Earthkeepers versus AI Empires event in Zambia. And then the fact that that led me to read Karen Hao’s book, Empire of AI, which, I think for both of us, gave us information, different pieces of the puzzle where we had had gaps. Certainly more for you. For me, it was more just educational in some ways that were broadly educational. I think for you, there were gaps in your understanding of why things are the way they are. 

[00:00:52] David: It was more the language to communicate it. I had such gaps in that, and Karen was just brilliant at that.

[00:00:58] Kate: I think we are going to be in conversation about some of those topics and some of that history that has shaped the narrative, that has shaped the direction of travel, all of those different things. So the biggest thing here, just from our perspective, is that you had felt quite frustrated with the fact that a model of AI that you think about seemed to be very underrepresented in the culture. And you felt like it had had a heyday, but it had kind of just disappeared from view.

And so a lot of what we’re going to talk through is a history, and we are getting into what are fundamentally two camps of AI, where one right now is in ascendants and kind of has a tyranny, but there is this other model of AI that is really interesting.

So, just to take a step back – this might be interesting for you too – I learned from Karen’s book that a lot of these ideas of artificial intelligence emerged, took shape, in 1956, when there was this workshop at Dartmouth. Maybe you knew about this, I did not. But it’s 20 scientists representing mathematicians, cryptographers, cognitive scientists who came together, and at the time they were seeing themselves as part of this “automata studies” field, which I think is not very romantic, it doesn’t sound very, you know, highfalutin and kind of having this grand vision. So, out of that workshop – and I think this is a key thing – it gets repackaged by the director of the workshop as “artificial intelligence”.

And so, I would argue this is a huge deal because it sets up a kind of marketing play that is maybe conceptually creating the wrong idea of what automata studies is actually doing, or creating a popular conception of what it is that might be obscuring what the actual science is and what is actually happening.

So that foundational conceptualization creates this logic where “computer intelligence” – even just using “intelligence” is the wrong word – this sort of “computer automation” work is pitted against human intelligence that is set up as the kind of standard. So, just to start there, is that something you were very aware of? 

[00:03:18] David: Absolutely. I mean, I often said, in previous episodes even, this goes back 80 years, and it comes back again to the Turing test, which is all related to these things as this way of perceiving artificial intelligence as intelligence, as opposed to what it is, which is a computational tool, a tool in automation.

And that sparks the imagination. It has led to these boom and bust cycles within artificial intelligence as progress is made, and genuine progress is made. The way you’ve just framed it, which I quite like, is that it makes it highly marketable in certain contexts. And that has been a cycle, which has happened time and time again around artificial intelligence.

[00:04:09] Kate: And I guess this gets back to the last episode when I referenced that quote, that technology emerges from this rallying grand vision that might not really be accurately representative of the truth, but it’s almost like the charismatic direction setting that people can get behind. Because, in some sense, this is like, I don’t wanna say a “cultish religion” or anything, but it has this framing that gets outside of the what’s under the hood, how this works, and it really gets into these much more human ideals, ambitions, all of these things that we would like to see for our society, for our world, for civilization.

And I think that if we take that artificial intelligence foundation, then it doesn’t become difficult to see – and Karen makes this connection – why artificial general intelligence then becomes the rallying ambition. And this is going to lead into the topics I wanna talk about, which is: is that the right ambition to have, what are the problems with that?

And I just wanna read this quote because I think this is very relevant to our thinking. Again, this is from Karen’s book, “To justify the elongating timeline and the ever-expanding costs of pursuing the ambition of AI, the promises we’re told about it have grown more grandiose than ever before […]. AGI (Artificial General Intelligence), if ever reached, will solve climate change, enable affordable healthcare, provide equitable education. Current companies…” – actually her book is very much about OpenAI, but I think by extension this becomes about all mainstream AI companies right now – “cannot say how the technology will deliver on these promises – only that the staggering price society needs to pay for what they are developing will someday be worth it.”

I think that is where we are right now. We are bearing the staggering costs on this future promise. So yeah, I think just to set that up as a framework.

[00:06:13] David: I know, I shouldn’t get sidetracked by this, but it’s so mind blowing that so much of society gets caught up in this and has bought into this because the idea that we would want artificial general intelligence to solve those problems is beyond me. These are very human problems, you know, and we would want to use technological tools to help us solve these problems, yes, but not to be the solution. This is so counterintuitive. 

[00:06:48] Kate: Well, and I think this gets back to the idea that, once you said that it was intelligence and not automata studies, then for the average person you are conceiving – and I think this is part of the conversation right now – people are conceiving technology as like, it’s just going to replace us, it’s just going to do these things.

And of course that’s not how it works. It can’t work that way, it needs humans, it is a tool that humans direct. And so I think I found that really what is a communication sin or play, just a very helpful thing for everyone to have in mind. Because I think there is this sense right now, if you’re the average person you are being a luddite if you’re not on board, you must be against discovering the cure for cancer if you don’t wanna support data centres and AI.

It’s these false narratives that have been constructed out of this view of AI. So that is kind of the foundation on which I wanted to then go into these two camps. So if we look at this Dartmouth workshop.. 

[00:07:59] David: Maybe just before we go into that, the idea that it is artificial general intelligence which is needed for this is the particularly perplexing piece for me, because actually a lot of the things that would be really useful, we already have the answers with the tools that we have. We’re just not using them in that way because we are being sidetracked by this magical, mystical thinking.

Now, it’s not to say that if we did reach what people would classify as artificial general intelligence that there aren’t useful things that would emerge from that. But there are plenty of useful things which already could be solved. And this is the thing where it’s a distraction, we are being distracted from actually making use of what we have.

[00:08:45] Kate: Yeah, in a later episode I’d like to talk about why, what are the incentives, what are the reasons that that happens? But yeah, I think that for now it is a really helpful point to make.

And then one more quote from Karen’s book, and I will then just leave it to us to talk. But out of this workshop at Dartmouth, there emerged these two camps of AI. And I’m sure one could argue there could be other camps, but for now, let’s just say these are the only two models you could have. So there is “Symbolist AI” and there is “Connectionist AI”. And I will read you just the overview description, and there’s a lot that we are going to unpack in it.

“The first camp, known as the “symbolists”, believed that intelligence comes from knowing. Humans know more than animals and can use that knowledge to understand and act on the world. Achieving AI must then involve encoding symbolic representations of the world’s knowledge into machines, creating so-called expert systems. The second camp, called the “connectionists”, believed that intelligence comes from learning. Humans have a greater capacity to learn than animals and can use that ability to acquire and advance different skills. Developing AI should focus instead on creating so-called machine learning systems, such as by mimicking the way our brains process signals and information. This hypothesis would eventually lead to the popularity of neural networks, data processing software loosely designed to mirror the brain’s interlocking connections, now the basis of modern AI, including all generative AI systems.”

[00:10:24] David: Yeah.

[00:10:25] Kate: So those two elements are what I wanna talk about really in this episode, Symbolist versus Connectionist. Also, just to put something else in the mix, the variables of AI do not change. They are data, computing power, and the models that are making sense of the data that need the computing power to run all of this together. So I think with those foundations, I’d like to just sort of talk through what these two models are, and the fact that everything we, the people, the consumers of technology, experience as AI right now is Connectionist AI.

[00:11:05] David: Almost everything. 

[00:11:07] Kate: Almost everything. So, well, maybe that’s a good starting point for diving into this. What if we look at the almost, what are examples of Symbolist AI right now?

[00:11:20] David: So, interestingly, I would argue the weather models that people use, that lead to the predictions every day. Traditionally, these weather models have been physics based models, which are essentially process based, built on the knowledge of how physical systems work.

Arguably, people don’t call that AI, they call it models, but these models are still at the heart of most of the predictions that happen and the accuracy of those predictions. They sometimes have a neural network, a learning module on top, but they still form the core. So that would be an example of where you do still have that knowledge base system at the heart of it.

And, in the scientific communities, there’s good recognition of the value that brings in a context such as weather prediction, where actually there are physical processes which are happening, which are understood, which we can be knowledgeable about, which aren’t just about amassing more data, they are about understanding the nature of how clouds form and how climate happens, if you want.

[00:12:33] Kate: So that would be an example of an expert system where the data that you want is actually very constrained. The models you might want to use to make sense of that data can vary, but the actual data you’re working with, the constraints even for learning within that would be sort of a sidebar to the need for the real expertise.

[00:12:58] David: Well, the constraints, the system itself in that context isn’t learning, it’s building on the knowledge of how physics works, what the physical processes are, and what the current situation is, and therefore predicting how the situation changes. So this is very much constructed from expert knowledge.

[00:13:18] Kate: Is the assumption of an expert system that there is always a right answer?

[00:13:23] David: Arguably yes, in the sense that a system works and there is a question of having the right knowledge to understand how the system works, what are the rules of engagements. The physical systems of the world, you can model them, standard models work quite well on them.

And what’s so interesting, of course, is that we work with these biologists who have for 40, 50 years been building similar process systems related to biology. And so, actually for natural systems, a lot of this can be identified by expert knowledge. And that expert knowledge can be incredibly complex in these biological systems. This can involve complex interactions between different species, and that interplay is really delicate and important. But you can understand that by observing it over time.

And so this knowledge of biological processes, physical processes, chemical processes, these base sciences, these are sort of quite well studied and there is a lot of expert knowledge.

[00:14:30] Kate: So I think part of what I’m hearing is that a symbolist system also requires a lot less data. That the data is more knowledgeable in some sense as there’s already a tonne of expertise. It might be complex, there might be a lot of layers to that, which is where a computer system would be very helpful and models are helpful, that for a human to make sense of that is too much, but actually that knowledge is held.

It’s not like you need to pull the scope of human history to try to learn what the meaning is. No, humans have already learned what the meaning is, they’re still learning, but there are these foundational elements that make building AI in this context actually, you know, it’s bounded, it’s constrained, you know in many ways what the variables are. I can see that there is a place for layering on learning.

[00:15:22] David: Yeah, so maybe let me just get down to the fact that many people aren’t calling this AI at the moment, they’re calling this modelling. And one of the things which is interesting is that AI has taken a lot of the emphasis in terms of needing more compute, needing more data. But these models, actually having more compute, having more data, allows you to do much more complex models that you would never have been able to do before.

Models that might even require different layers, different levels, having things that are a microscopic layer and a sort of macroscopic layer would’ve been really impossibly computationally demanding in the past. Whereas now these things are actually imaginable. We could do this, different time steps where different things work on different levels, but they interact with one another. Putting together these sorts of models is something which is feasible now in a way that a while back it would have been unimaginable.

And I want to come back to the fact that if you think about this in terms of the actual knowledge, the expert knowledge which is being used, these are putting together different experts. Potentially, this is what you can do with that additional compute, with that additional data.

What’s interesting is very little work has gone into this. Most of these models actually were last seriously worked on 40 years ago.

[00:16:54] Kate: I would argue that, and this is only from my reading and from Karen’s book, and also because I knew this from elsewhere, IBM was working on this. Watson, I think, worked on an expert cancer diagnosis technology. And what I heard in what you just said is data sharing, cooperation, interoperability. Those are hard problems to solve that involve humans, they involve trust, they involve sharing value, reputation, wealth, for people to contribute meaningfully to those systems.

So what I hear partly in what you’re saying is for AI to be valuable in a symbolist context, it needs to think very differently, it needs to be designed, and these are social problems as much as they are technical problems, which is not the domain of computer scientists. And so I can easily imagine why that’s not very appealing to a tech company.

[00:17:56] David: Absolutely. I mean, if we think about actually accumulating the world’s knowledge, this can only be done if this knowledge is able to be put together in interoperable and compatible ways where it isn’t owned or held by an individual or an individual group,but it is something which is built collaboratively.

This is something where, inherently, the amount of work that needs to go into the details in a given context is so great. This is an expensive venture, so to speak. This is knowledge. Knowledge is expensive in this process.

[00:18:32] Kate: Yeah, I think that distinction of you need real experts to be encoding meaning in data as opposed to, “well, we can just outsource this to a contractor who will identify that this is a bicycle and this is not a bicycle, or whatever the sort of low scale variable of data training or data cleaning or whatever you wanna call that is”.

So I think that’s a really helpful distinction and it makes sense of why Symbolist AI would be harder to pursue, harder to have value. I guess the other piece of it that I think is really important and that’s implicit in this is that then the underpinnings of whatever the AI spits out is actually understandable. You can go back and you can trace the history of which the decisions went into that.

And if you see something like, “ooh, that was bad AI”, then you can say, well, here’s where a bad human decision happened, where a human encoded something or put information in that was flawed or didn’t take into consideration this other variable. Not even a bad human, just flawed human, or holding a narrow view human, where we didn’t get this other bit of data, this other bit of insight, this other piece of knowledge encoded into the system that would have trained the AI to understand that there’s something different happening here.

[00:19:50] David: Well, possibly except that that view of training the AI is not a Symbolist view.

[00:20:00] Kate: Ah, okay, good, talk about that. 

[00:20:02] David: The whole point is that the simplest variant of this in some sense is not trained, it is expert based, it is traceable. Whereas the idea of training the AI, of the AI learning itself, is inherently a view which has been put on it by the connectionist view.

[00:20:22] Kate: Right, right. Oh, that’s so interesting. So language that has become default about how AI works is not default at all. It’s all part of this prioritisation of this connectionist model of AI.

[00:20:35] David: Exactly. And the point is that it is not necessarily bad to distinguish these two. As we say, AI is not necessarily the best way to call this anyway. So the fact that your Symbolist approach is no longer really considered AI wouldn’t be a bad thing, except that it means it’s been totally unfunded, it has been devalued and really pushed to the sides because of the mystique and what has emerged through the Connectionist AI. 

[00:21:13] Kate: It’s interesting, so it’s actually that the black box of Connectionist AI, which is we put in all this data, we trained it, it’s learning, it’s making decisions, we don’t actually know what connections it has made to come to these outputs, we certainly couldn’t audit them, correct anything. I think you hear this a lot of times from AI researchers.

This is a marketing issue, I think, if you can’t even create a pathway for an alternative because funders, people who could actually make stuff happen, they have so been sold on this idea that it is not AI unless it is a black box, unless it is magic, unless it’s these neural networks just doing their science, which we can’t even understand to that proximates intelligence, it’s not AI and so you shouldn’t invest in it, and this gets back to that artificial general intelligence view.

[00:22:05] David: Well, almost, but let’s just draw it back into ourselves to finish this episode, because I think when we talk about what we believe is needed for responsible AI, we talk about having a deterministic core. And so, really, what this comes down to is it was never a choice. It was never a choice between connectionist and symbolist. Both are needed, both add value in different ways.

Really what we need is to recognise that both of these approaches, there is value at their core, and so both are needed, both add value. What I believe is clear from any perspective is that we don’t need another big revolution in AI to add value to our current society based on the tools we have. We need to just get much better at using the tools that exist.

Now, if there is another breakthrough that happens, great. That might be useful as well. But we don’t need it. Right now, we can use the tools that exist much better than they’re currently being used, if only we actually get our act together. This is the thing which I think is at the heart of what we’re saying. Real progress has been made on both of the sides, both the symbolist and the constructivist, and they should be working together not in opposition with one another.

[00:23:32] Kate: I want us to do a whole episode about that, but I think just one follow up question on the impact side, why is the symbolist model of AI so important for impact? I think I could answer this question, but I think you will answer it better. 

[00:23:46] David: You mentioned traceability, and of course traceability is really important. In places where traceability matters, where it really matters why you are making certain decisions and what decisions could be made, that’s really important. But I would argue it’s more than that.

So traceability alone is only part of the problem. It is the fact that there are, in the context of impact, there are rules and things which we want to bake into our AI systems. We don’t want the AI systems to go rogue and do things that we never imagined before. If we’re wanting to be giving parenting advice, then we don’t want parenting advice to emerge which might be popular, like violence against children, but has been demonstrated to actually have negative impacts, not just on the children but also on society as a whole, to lead to a whole set of issues in the future.

So this area of being able to say, yes, it might exist in the world, but just because it exists doesn’t mean it’s what we want the AI systems that we are building to actually promote it. Where you’ve got those limits, that’s the sort of thing where actually we want to be able to have certain rules baked into a system, certain things which have evidence behind them.

And the same holds for education, the same holds for many areas of life. Where we’re wanting to have a positive social impact, we want that impact to be based on how the world works. Preserving biodiversity, this is something which, if we affect something and it has knock on effects, we want to know those knock on effects to be able to make sure that the decisions that we make are safe and sound.

[00:25:37] Kate: In the context of impact, and I would argue in a lot of societies, there are right answers, there are known better approaches. Those might evolve over time with better research, but there is a state of the field where we know, where there’s enough evidence.But right now there’s this fictional idea of like a marketplace of ideas, where often by algorithms, by publicity, whatever the forces are, you can have a lot of data that exists that might suggest that something is really high quality or really good just by the preponderance of information out there about it. But that doesn’t actually reflect reality. So it is this idea of very deliberately building systems around the known quality.

[00:26:29] David: Seat belts. You know, before seat belts were required by law, lots of people didn’t bother wearing seat belts. And the impact that this has had on reducing death and serious injury on the roads is huge. It’s a very simple thing that sometimes rules can have a really positive effect on society and on individuals.

And it does sometimes come at the expense of the fact that, well, I quite like driving in a car without a seatbelt, it’s a personal choice where actually that choice led to societal and personal issues which were avoidable. Now, I’m not saying that it’s about having seat belts, but I am saying this is a simple example of a rule where most societies now accept seat belts as a sensible societal rule.

[00:27:21] Kate: And you design for what that best standard is. And if people wanna opt out, they can, they can have other experiences, but at least you know that when you do get into a car, you have the system that is designed to keep you as safe as possible.

Okay, so, David, in the next episode let’s talk about . and this has come out of this – that relationship between LLM Connectionist AI that’s learning and generalist versus expert Small Language Models knowing AI.

[00:27:59] David: Okay. And a small language model of course is still learning. So anyway, yeah, let’s dig into that in the next episode.

[00:28:04] Kate: Okay.

[00:28:05] David: Great.