09. Why AI Matters Now

[00:00:07] David: Hi, and welcome to the IDEMS podcast. I’m David Stern, a founding director of IDEMS, and it’s my pleasure to be here again with Kate Fleming, another director.

Kate, we’ve had this whole series now, which emerged from the meeting I went to in Zambia, Earthkeepers versus AI Empires, which got you stuck into the “Empire of AI” book, and which we’ve then been in discussion on, as to the AI empires and how this relates to our work in different ways, but particularly just how we are thinking about AI in relation to these ideas.

And I feel we need to dig into a little bit more. I’ve been having a whole series with Lily on AI, but that hasn’t been central to our discussions before this. This is something which was something we do, but it was on the periphery. Why is it that it’s so relevant to what we’re doing?

This is what I’d like to come out, how does this really relate to the bigger picture of our work? It’s always been there, it’s always been something which I’ve been keeping a finger on the pulse of, but it’s just now that we seem to be emerging into this space. The two of us discussing this more, this coming out, why is that happening now?

And I guess that’s what I want us to discuss today. Of course, I have thoughts on this, but do you wanna start? When did AI start being important for you? How did you get onto this?

[00:01:33] Kate: Hi, David. This is what I’ve done to you on other episodes, I just introduce a lot and then I say, talk to me David. It’s funny because I don’t think I would have thought of AI as being a focus, because I understand, and we will get into this, I understand that there are all these other foundational problems that need to be solved first. But I’ve always been aware. Well, if you can start to solve these issues that are mostly related to data, that are related to community ownership, how you create systems of trust that enable a system to access data, information, practises, all of those different things. So I was interested in that piece.

And then you can imagine quite easily, well, when you start to have those systems, they are going to be very complex and unmanageable really by humans. And then you get into kind of the logic of a massive bureaucratic system with a very different logic. But the problems of it, where you just have so much information and it’s so chaotic and it’s so distributed, and your ability to make sense of that is nearly impossible, or to make sense of it in a way that really moves things forward and just doesn’t feel like you’ve opened up a whole other set of problems, which was the conversation about big data for a long time, it was just like, we’ve got all this data, but what do we do with it? And so I think it is this progression where you can see, okay, if we start to solve these data problems, then we’re going to have these other problems that are more easily solved by AI. 

And then there is one other piece that I will throw in – I’ve done a bit of what you did in your setup, where I’m throwing a lot out there. But I can also see that what starts to happen is that non technologists can be the keepers of technology.

Yes, things require maintenance, they require keeping by people who really understand how the systems work or can manage them, and have the expertise to manage them. But mostly in the day-to-day, non technologists can get about their business and do things. And so that’s where I see AI has the potential to be something really interesting. But it’s also where you see, okay, AI needs to look different from whatever this current, the Karen Hao Empire of AI, this lane of AI needs to look like something quite different.

[00:03:55] David: And let’s be clear when you’re saying it needs to look different, that is what Karen is actually saying. She’s saying in her book that the AI of the AI empires is not taking us down the route which is gonna serve society in the way that, as society, we should be driving towards.

[00:04:14] Kate: Correct. I will jump in and say I don’t think that anyone has a problem with the current direction of AI as one path. And that has come up again and again in our conversation. It is a totally valid direction of travel, it is just one of many potential pathways. And when you’re looking, depending on the problem, you need very different AI solutions.

So I think that was what we wanted to pull back into: what are the problems we see that need to be solved and the way they need to be solved, and what does that mean for how we are conceiving AI?

[00:04:52] David: Yes. I think the point and the naming of this, the Empires of AI, is that this being the empire we’re aiming for is what we are pushing back against. Not that this is not technological and societal advances, which could happen based on this, but it is leading to a capture, which is not what we believe is in the interest of border society, and, to be honest, even in the interest of the companies themselves in the long term.

If you take a long term interest, if you forget the short term where they can’t do better than they’re currently doing in the short term, but if you think of them actually, what might they look like in a hundred years time, well, I think they’d look much better in a hundred years time if we navigate these few years in a way which actually has them in balance with society rather than just extracting from society.

[00:05:49] Kate: Also, what I was thinking about as you were talking is that there is a kind of innovation that is very centralised. I feel like there’s this narrative right now, it’s almost an authoritarian view of what technology should look like. You know, everything’s going the same direction, we can’t compete with these places where it’s just like there’s one model and they’re putting all their resources into it. But what makes our society interesting is pluralism, is the idea that you have all these different forces and it’s that compliment of forces that actually drives real innovation over time.

In the short term, that concerted, concentrated effort, yeah, you can see centralising that, putting that towards something. But then you need to spread back out and let a lot of different things be happening to get to the next level and kind of have things be in competition, and really good ideas incubate and bubble up.

[00:06:43] David: Yeah. Let’s come back, so you are interested in this really from a human perspective. You see that the systems that you are wanting to work in are complex. They’re deeply complex, and that complexity has in the past been held back by the bureaucracy that has to get created to manage it.

And that in some sense, AI has the potential, and maybe not the AI that we are seeing being pushed at the moment, but elements of the methods, the computation or the mathematics behind AI, these could actually build things which really enable that bureaucracy to be managed and allow non-experts to be able to interact with artefacts and actually take the important decisions on behalf of community, society.

That’s where you are seeing that there is a role that we can play. That the technology is potentially enabling us to make better systems work where the bureaucracy is not holding back the interests of society or community.

[00:07:54] Kate: Yeah, I wouldn’t even frame it as bureaucracy. I would say the better framing, when I think about it at the problem level, is just dependencies that often keep the people who know the most and are best suited to tackling a problem from ever getting close to even contributing to what a solution might look like.

You can think of all kinds of communities where there’s somebody who, often an expert, often at the systemic level, where they have an idea of the solution. But then when you get on the ground, well, what a community actually needs, the knowledge they hold of how that really should work, and it’s not that they don’t need that outside information and research because it helps them to frame their own practises and thinking and various things, but to just replace whatever that local knowledge or expertise is with this outside thing is not helpful and it’s missing where there’s a lot of value that could actually have real impact.

I think about, well, how are you starting to remove those dependencies, collapse the current power dynamics that create those dependencies? Again, it’s not going to be that it’s all or nothing, there are going to be times when someone who currently has agency should continue to have agency. They know things, they have information, they have skills, whatever it is, it’s just they also need to get out of the way.

And if technology is a barrier, which is how it is right now, you can’t ever have that happen. I mean, right now we are living through the age when a very small pool of people have designed the systems, the digital systems that shape all of our lives. And they have made decisions that I think are incredibly consequential for society in good ways, but also in really problematic ways.

[00:09:45] David: What I want to just come back to is the bureaucracy piece on this. If you think about in absence of it being technology, often this is a debate between having good rules which are set to support people, having social safety nets and these sorts of things that come with bureaucratic burden, or, at the other hand, very light intervention where you don’t have the bureaucracy but then you don’t have the social services, the actual support services.

And part of what I hear in some sense is that the technologies are presenting themselves as the latter, whereas actually they are partly the former, the bureaucratic processes. They’re actually gate holding a lot of the different experiences, what you can do. Although they’re presented as being the latter, they are actually holding quite a lot of the structure, the decision making power, in a way in which, if we think about how this interacts with society, actually the tools that they’re building can help us to navigate that space between the bureaucratic and the individual freedoms much better.

But it requires a different sort of mindset to be able to bring these together and to actually cut through the bureaucracy while actually offering the services and recognising the services for what they are.

[00:11:11] Kate: That’s such a good point. It’s that we’re still being sold this idea that a lot of what’s happening is these resourceful startups, these are just lean, these are the creators, the ideas, this is where stuff is coming from, they need to be unleashed. But actually a lot of it is bureaucracy.

[00:11:30] David: It’s taxation as well. They’re taxing every interaction we have but they’re taxing it at the digital level, they’re the tax collectors.

[00:11:38] Kate: Yeah. We need our Boston Tea Party. It’s such an interesting framing. Again, I just come back to the fact that this has been captured as if like this is the free, open-minded exchange of ideas, all of that, but it’s actually deeply bureaucratic, it is like the entrenched apparatus of the technology state that has basically set up, exactly it’s a bureaucracy. That’s such an interesting framing, I really hadn’t thought of it that way before, but yeah, that makes sense.

[00:12:08] David: I think the taxation to me is the key one. They are in the middle of every transaction and they tax every transaction. They’re just not doing it at a government level, they’re doing it at a global level. So your global tax is what you are paying to your AI empire that they are just taking their tax on everything.

How people who want freedom from tax and don’t want government taxation support this is something mind blowing to me.

[00:12:34] Kate: Because they haven’t thought about it this way. And it’s funny because when people say they want a revolution, I always think, ugh, revolution, be careful what you wish for, because obviously revolution is hugely destabilising. Often the people who come out on top are sort of the same people just wearing different clothes and the same people who are marginalised end up at the bottom in whatever the new system is. Or maybe it’s different people, but somehow they’re always the same power dynamics.

Whereas the soft revolution, and I guess in some ways, even though obviously the American Revolution was – sorry, I’m bringing my American lens – but there are different kinds of revolutions, and early conceptualizations of independence were just about breaking free of those systems that were imposed that had kind of imposed this tyranny.

But to bring it back, the problem that we’re really solving here, or we’re thinking about here, is how do we give people access to the things they need, the tools they need, the resources they need without this machine that is taxing them, that is controlling them, where they don’t really have a say.

And this will come back to, again, sorry, politics, but there’s been a lot that’s been written about the authoritarian nature of big tech where it’s like an authoritarian bureaucracy that’s been created where they create the illusion of free will and choice, but in fact you have very little free will and choice, it’s deeply constrained. 

[00:14:04] David: Let’s just take this analogy further ’cause we’ll get to why I’m discussing AI. This has got in a direction which I quite enjoy. The point is, if we actually take the governance analogy and we recognise that the AI empires are currently empires, well, America led the way in terms of deciding, you didn’t want empires, you didn’t want to be governed by an empire, and so the American Revolution was all about moving to a republic and to have democratic systems behind the republic. Okay, so what happens? What does a democratic governance of an AI system look like?

It would probably still have taxation, just like the Republic did not do away with taxation. It probably still needs that to be able to function, but it could be governed differently. And there are other people thinking about this and thinking about governance in these ways. And so there are ways in which that parallel is actually really useful. AI is a technology, it is not a governance mechanism. And the fact that we are confusing these and conflating them is actually quite interesting. 

[00:15:20] Kate: I was going to say, actually, you remind me of something. I was listening to some conversation between historians talking about ancient Greece or Rome or something. It was basically one of the things that came out with somebody who said, well, maybe democracy isn’t the right model anymore.

And part of it was like, what, how could you possibly think that way? This person is against democracy. But actually the point he was making is these are outdated systems and we could imagine new systems and maybe there are better ways, with technology, to represent different groups, different categories, to give people ownership where they have expertise.

I don’t know what it would look like. I don’t even begin to know what it would look like. I guess I just, you set me down the path of thinking of two divergent things. One is how AI is governed and the other is how AI can help with governance. So the idea is, if you have a well-governed system, you could actually begin to do things that are much more modern than going to the ballot box, which is thousands of years old, I guess. That’s like a very old way of representing a viewpoint. It means that sometimes I’m voting about something I don’t really know anything about and I’m pretty informed.

Whereas what if we elected people.. well this, okay, sorry, now I’m getting into the way the system should work because democracy is always representative. But yeah, it’s just how might technology help with that? So yeah, you’re right, that is a problem I find very interesting.

[00:16:49] David: Yes, I’m aware that this is something which you have dragged me into and I am now interested in. It is something I thought beyond my expertise, but you are exposing me to actually think deeply about this. And the thing I do want to say is that it isn’t that democracy as an idea is outdated, it is, possibly, that our implementations of that idea are outdated. The idea itself is about representation, it’s about all sorts of other things. And what we’re saying is that the type of democracy could change.

There is also another point, and it’s a really critical point. You know, authoritarianism isn’t always bad. It is really good at certain phases. It’s very good in a young phase for very strategic directed growth, that’s what authoritarianism does really well. So we shouldn’t be surprised that while AI was young, it was the authoritarians who actually developed it faster and better because that’s what authoritarianism as a governance mechanism does really well.

The question is, is the technology now mature enough that it is going to evolve or shift towards what are sometimes more stable governance mechanisms? Now, the stability or not of democracy is something we could debate on another court – I don’t wanna dig into that – but I think there is that element of: where are you in a growth cycle?

[00:18:17] Kate: And those are hard problems because obviously in politics and business, what happens is the bigger you get, the less you wanna cede the control you have and you want to keep out innovation, you wanna acquire it and squash it. If you’re a company, you might blend it in, but I think what we see so often in tech is, I mean, we have been living through the internet long enough, there are so many products that have been relegated to the dust bin of history that I think kind of nostalgically about. I’m like, that was a great product. What happened to it?

Well, it got acquired, usually it got acquired, sometimes somebody took it and did something with it, but often it just got folded in, they couldn’t quite do anything with it, and then it just got shut down.

[00:18:59] David: I can give you a really concrete example of that because of course it’s a domain which I know really well, but which is not of interest to many people, statistics education or data education. The best toolbar known on this, a statistics software which has inspired me since, which had a group of developers, it’s called TinkerPlots, and it was a fantastic tool, beautifully conceived, really at the forefront and really innovative.

It then got acquired, but it didn’t actually get acquired. It was part of a company which had multiple tools, and what got acquired was the interest in the other tool. The people who acquired it didn’t understand that, to the community, to the statistics or data education community, TinkerPlots was gold.

And it just got relegated to the sideline. The developer team got totally disbanded and went off in different directions. And so all the knowledge of how to develop it was lost. Now the technologies are out of date, it was never opened up and there’s still a decade later nothing which is even close.

Now, this is exactly the phenomena you are describing for a very specific use case. But the thing that I want to draw out on that, is that, actually, maybe this is part of what’s motivated me and has got me really interested in AI. I’ve always had that in the back of my mind. This happened before we started, before IDEMS existed, and it was this realisation that actually for us to build software which is really going to be impactful, it isn’t enough to just build the structures which build good software.

Building good software is actually secondary to figuring out what are the structures that will enable good software to sustain, to grow, to actually evolve over time, and emerge even if it’s for a niche community.

And this is exactly what motivated IDEMS to be set up in a different way. At times, as you know, people have come to us and they’ve offered us mentorship and advice and told us you are idiots for being a community interest company limited by guarantee. Why are you making life hard for yourself? But it is exactly that vision to recognise that actually I don’t want to build things which then get relegated to history, I’d rather not get them finished than get them built and then see them acquired and torn apart.

So I want to solve that broader problem. And that’s exactly where I was aware that AI is part of what can change and be the heart of changing that dynamic of what gets built, what gets evidenced and all of this and how these sorts would get built out. And so a lot of what we’ve been doing is building the underlying infrastructure that would allow for this different model.

And so we’ve been waiting for this model where the AI agents are good enough to be able to actually make a different system work. This is the heart.

[00:22:22] Kate: Yeah, and what I hear in what you’re saying is, right now it’s very, either you’re deeply centralised and you’re ambitious of your scale, or you’re deeply decentralised, and then it’s a lot of kind of scattershot, a lot of creativity, but nothing can ever get anywhere. I mean, we know so many people like this in the open source community who are working on really interesting things, but they’re just this one lane, it’s often held by one person who’s really passionate about it, and you recognise when that one person goes away, it kind of languishes and dies.

And so what we see and are working toward, and we can save more of this for the next episode, but is how do you build and how much do you build what is the underlying wiring to enable all this local creativity, local flourishing, but to do it within a networked, connected, interoperable and not sort of loosely interoperable, really deeply interoperable, which is how it would need to be for it to actually work, in a way that becomes competitive at scale with the big players.

It’s not that we’re saying they can’t exist, and there’s a place for them, great. But what does it look like to build scale alternatives?

[00:23:37] David: And yeah, as you say, this should be another episode. I won’t dig into that, but what I will do is just finish with the fact that, really, AI was always something that we knew was needed, but it wasn’t ready until recently. And what I believe in some sense is that the structures we were building, we got criticised, these are too complex, they’re too hard. Actually, what we’re building is not easy enough to use. And that was valid criticism until the latest breakthroughs in AI.

And this is what’s so exciting, now we have a foundation where all that complexity, everything we were building, which others were saying, can’t you just make this simple? And we’re saying, no, you can’t make it simple ’cause the problems underlying this are complex, we need to expose that complexity. And that’s what we’ve been working on in the background for years now.

And now with AI, there’s no problem with that complexity because that’s exactly what the AI agents can help you with. As long as you have those underlying structures right, we always knew the technology would catch up to enable us to then action that. And this is why it feels like it’s the right moment to try and really move on this.

Because once you have that underlying structure, you don’t need the big models, you can do the same things the big models can do with smaller models. The environmental issues go away, all of this, we’ve been waiting for this moment. We are not there yet, but it is exactly what we’ve been preparing for for a decade or more.

And it comes back to what you articulated as the reason you’ve been thinking about this. That it is not about technology, it’s about the community, it’s about the role technology plays within society. And this is what we could actually reimagine, I believe, if we can get these underlying foundations right and build the systems on that.

Anyway, let’s dig in in another episode to what you were just saying about what we see as being this good AI system?

[00:25:49] Kate: Sounds good. Thanks, David.

[00:25:51] David: Thanks.