272 – The Foundations of Community-Centred AI

[00:00:07] David: Hi, and welcome to the IDEMS podcast. I’m David Stern, a founding director of IDEMS, and it’s my pleasure to be here today with Kate Fleming, another of our directors, and we’re keeping going in this series that you’ve instigated us on related to the Empires of AI.

We hinted at, in the last episode, this thinking process we’ve started to have of, well, what would good AI look like? What is the alternative which is possible? What would the foundations of that be? That’s something which I’d love to now dig into with you.

[00:00:45] Kate: Hey, David. Yeah, and I would say, which is quite clear from everything we’ve said in this, this is our vision. Other people can have their visions too. I just think we have this idea based on the spaces we work in. The problems we see that are systemic and scalable happen at scale but are also quite local.

We see the need for a very particular thing, but I’m happy for other people to have other ideas, I’m not saying this is the right alternative, as we dig into that.

[00:01:12] David: Let’s just be clear on what you are actually saying there. It is that at the heart of what we believe is that there isn’t one right vision, at the heart is that we need this diversity of different visions. We could not make the progress we believe we can now make if the AI empires had not made the progress that they’ve made.

So let’s give credit where it’s due. In Karen Hao’s book, she articulates how it was a push that OpenAI took to just go down a particular lane, and it bore fruit. So, if they hadn’t done that, we’d never have done that. We’d never have wanted to do that. If they hadn’t done that, we’d still be playing the waiting game because we’d be building the foundations, but what we’d be building, it would be too complex to be sensibly managed and all the rest of it.

So we need that environment, which has these diverse actors. Now, I do wish there was a little bit more balance, but that’s a separate discussion.

[00:02:08] Kate: Well, I was also gonna say, I would love for us to take the torch and be a catalyst that then sets in motion, okay, great, we hit some level of sustainability, we’ve achieved something that’s really useful for other people, which is going to be quite difficult and expensive to get there. But then we open up, great, now we hand the baton, the torch, whatever it is, over to the next generation of innovators who are based on our technology, able to do something incredibly interesting and carry us into some new era of innovation.

[00:02:40] David: That is very explicit. It is not somebody taking the torch, we are splitting the torch. This is really at the heart of what we’re wanting to do, we’re wanting to enable that multitude of people to then actually run with the foundations that we built. At the heart of our innovation of what we see is that ability to democratise the innovation that can be coming out in a way that is interestingly contrasting, because this is the narrative that’s already there, anyone can innovate.

But I think there’s good reasons why actually we are seeing that’s not how the systems are set up. If we wanted to set up systems so that anyone could innovate, we’re seeing the current systems are not really supporting that. There are layers, and we talked about this as being bureaucracy before, which have been put into the system, which are unnecessary and they’re actually bogging down the real innovators, or gate keeping the real innovators through a particular lane. 

[00:03:46] Kate: And I think that a lot of the societal rage and discontent right now is that people aren’t feeling excited about the innovation. When the internet emerged, people were excited, there was a flourishing. This isn’t feeling exciting to people, it’s feeling like it’s being imposed, it’s top down, someone in some room somewhere decided that this is what their future’s going to look like and they’re just, you know, “eat your dog food or be left behind”. And it’s very destabilising and you see a lot of discontent fomenting.

But yeah, so that is…

[00:04:27] David: Another story, and we will get to that.. You’ve got at least a couple more episodes after this where you’ll get into this, the future of work and this sort of thing. So we will come back to that.

But let’s try and make sure this episode focuses on, well, what is this alternative, what’s the foundation? And one of the things we’ve got to recognise is that a lot of the language we want to use for this has sort of been captured. That’s part of the challenge, human-centered means a very specific thing, and so human centering is a word that we would like to frame things as, but in some sense that has been captured to mean something different.

[00:05:05] Kate: I think it’s also not actually the ideal framing because, from the beginning, the place where we have had common ground is recognising that the real locus of power isn’t the individual, it’s the community. You need these small collectives of knowledge holders, of skill sets, whatever it is, but at the local level, that is what you should be designing to unlock.

And because communities work because they are systems, they do not work because you have a bunch of empowered individuals who are just working in their own lane, that’s when society really starts to break down in many ways. So human centering is good, but it doesn’t necessarily mean that you’re starting to really solve social impact problems. I would start at that point.

[00:05:46] David: Can I just check? So what you are saying is community centering was always a better language for us.

[00:05:52] Kate: I think so, yes. 

[00:05:54] David: This is really interesting, you have communication skills that I don’t have, which is great, because to me, when I hear human centering, I think of community, I include community within that. But that’s exactly where you are absolutely right, calling out the community aspect of human centering and saying, no, that’s the really important piece.

And then we get to the question, well, what is community? And even just that language is hard, you know, recognising that in some ways every big corporation has created a community of members of that organisation. There’s academic communities who are expert communities that cover the globe, there’s very local communities.

So community centering is really interesting because it’s so diverse.

[00:06:40] Kate: Yes. And I worry that we’re going to spin out into things that are quite abstract. But I think having that framing in mind is useful because it doesn’t mean that a community has a very narrow definition, it means that it’s a group of people who are organised around shared values, a shared sense of purpose, belonging, whatever it is, to create an environment where things get done, where you have a quality of life, whatever those things are that create community.

And so, to come back to the foundations of what we’re working on, these are really data and community problems. So what is the system that needs to be designed to both give communities agency over data in all its forms – that could be software, that could be content, whatever it is, we use data very broadly, it could be analytics – but how do you give communities that agency? How do you put them in conversation – conversation is not the right word – in really deep collaboration with other communities if they want to be? So when you are developing community technology, it is not this isolated, spun off fork, it is something that is part of a networked system that is deeply interoperable, not just our email protocols can exchange, but really every aspect, if you wanted them to work together could. 

[00:08:07] David: Yeah, I think this needs.. you are getting quite deep already. Normally, this is my failing, so it’s really nice to be able to jump in and say, wait a second, let’s make this a bit clearer. Because I think the thing that you are sort of articulating is that if we are centering community, then collaboration is key.

It’s collaboration within community, collaboration across communities. Community is about working together, it’s not about just working on your own. Human centering has the image of you helping the individual, you are just worried about that unit of exchange. Community centering, you have to worry about governance, you have to worry about ownership, you have to worry about all these things. There is an element of compromise about being in community and communities interacting that needs to then be resolved through collaboration.

If we are going to collaborate, then we need to be speaking the same language, whatever that means. And that’s what you are talking about, this deep interoperability. This is this ability to be able to share and to be able to speak the same language in what we share, so we can learn within our community from other people, and yet there is that sort of common language to do so in ways where that sharing and that collaboration can emerge in really positive ways. It’s not something which is siloed.

So this is, I think, the big thing that you are articulating. In some sense is moving to community centering rather than just individual centering, a lot of it is breaking down silos. And to break down those silos, that’s where we need what you were calling “deep interoperability”.

[00:09:50] Kate: That was a very good clarification. And I think the emphasis on these as social problems to be navigated, not technology problems that can be optimised for. There is not a right answer often, there is a negotiation that happens and then you have to decide, well, this is the best solution here for our community. Or maybe we need two versions and those two versions can speak to each other. We need to solve these two different problems in our community to really meaningfully make progress, whatever it is.

I think something I hear even as you’re talking, which someone will infer in this, is the idea of doing this in the entirety of everything that humans have to work through socially. Just in your day, the things that you are having to do that involve social exchanges, which are either you’re avoiding them entirely because tech has just created efficiency and you don’t deal with things and then you move out into the real world and you think, whoa, people are so difficult. It’s like, well, welcome to reality.

But, I guess the point I wanna make is, there’s a reason that we very much focus on this thinking within the lane of a single impact problem. Because when we are working in that very bounded context, it’s not that the variables are finite, in fact they’re very much not finite, but there is a core set of variables that we can at least get our heads around, or the deeply brilliant mathematical minds on our team can begin to kind of model as systems.

And then you can imagine there are other variables that we keep building out and adding on, but they are things that are manageable complexity, even within those narrow lanes, although they’re not that narrow, there is some ability to parameterize them.

[00:11:48] David: I want to draw out again on this point of breaking down barriers as being this deeply interoperable piece, that often collaboration is held back because it’s an all or nothing game. I don’t want to collaborate with you because I’m worried about what you’ll do with this private data. But maybe you don’t need that bit that I’m worried about, but what you do need is some aggregation of this in a way that actually I don’t mind, but that is maybe not possible in this way.

And so, if we have this so that actually the forms of collaboration which are possible can protect the things that people care about and we can inform people on what are the things you want to protect, what are the things that you are happy to share? At the moment it’s very simple. You wanna use the technology, you tick the box and you don’t know what it means, you don’t know what you’ve just given away, and there’s no sense of being able to sort of, no, that’s the technology piece.

If we can do this right, this can actually get down to the stage where we can have, and we have inspiring partners who are doing things which are really informing our thinking, this Farmer Federation in Niger, where they own the data for their farmers and that’s changed power dynamics with researchers, with other collaborators, with the government, because they’re able to provide information. 

[00:13:15] Kate: You got me thinking about a talk I went to that was with a negotiator, someone whose whole job is negotiations. And his point was, every single negotiation basically has a win-win. You just have to figure out what that win-win is. We can have what we want and you can have what you want, and we can have that system coexist.

But often people are negotiating from the point of view of fear or they’re trying to negotiate across everything at the same time. You’re not ever surfacing like, oh, I’m just worried about this thing, you’re just worried about this thing, oh, I don’t care about that, you don’t care about that. All right, this is fine, we just own our data, you just own your data, but we can use it at these points, no problem.

And so this is a bit, it’s designing the systems basically where you are creating technology that is…

[00:14:03] David: Enabling those negotiations, I love that framing. I’ve not heard that before, but I love it because I think it’s exactly right. It doesn’t matter whether it’s a business transaction for two companies to work together. This is not something which is just in the impact space, this is why we recognise the implications of this could be huge to enable company collaboration to happen better.

Now, of course, that’s not gonna be the starting point of this, mainly because it’s so much easier to see this problem in the impact spaces where everyone wants to collaborate, but there’s these constraints and so if we break this down, this would be visible and it’s easy.

But I believe the same elements could apply in the business sector as well. Actually there’s collaboration there in certain ways. Now, of course, who it helps, that’s the whole point. Might it help those at the top? Maybe, maybe not. It might help those who are coming up through, I don’t know. But that’s something which we’d find out if and when we get to this.

But that’s the idea of actually enabling proper collaboration to happen and for us to negotiate the solutions, where we can articulate exactly what it is you need to protect, what it is you are happy it can be shared. We’re coming through this in our own ways. We believe in open software, we believe in open source, and yet at the moment we’re going through the process of actually registering patents.

[00:15:27] Kate: Oh, well you introduced a new topic. Yes. I think we are aware that we need to be competitive, and you know, we need to play the game. I think this is so often what happens when doing things for good. There’s almost this, I don’t wanna say naive, but there’s this belief like, well, just the goodness will speak for itself and people will get behind it.

And, as a social enterprise, we are trying to walk that line between what it looks like to have a sustainable, competitive business, but also be delivering impact. So that’s where I think we think across different lanes or categories of opportunity. But one, we’re really focused on impact because it’s what we care deeply about and see that it’s so deeply underserved. Also, low resource variability means that people are very aware of the need for negotiations and compromise all the time.

I designed this app, but it doesn’t work in your context. So I need to let go of this grand vision I had for this perfect app and realise, uh, this needs to be completely redesigned for this context for it to actually work, to be something people will use. I think in impact people, there’s less ego sometimes, most of the time.

[00:16:43] David: Less ego is, I think, fair. That’s not to say there isn’t ego.

[00:16:46] Kate: No ego, no. But yeah, there is a shared commitment. You are working in the field because you want things to have social impact. So if you recognise the ability to have social impact means giving control to other people, and there’s evidence that that is going to advance impact, that’s kind of hard to argue with, unless you’re really working on things for the wrong reason.

So yeah, I think the opportunity in this space, because negotiations are just part of the way things work and work well, that is where we see this is the space where we can really build these systems that work quite differently.

[00:17:28] David: And let’s just try to recap because I’m conscious we’ve gone all around the houses. So, broadly, what I hear in what you are saying, which is I think what resonates and what I agree with, is we would like AI systems and the foundation of those systems to be community centred. If we want things to be community centred, we need to be able to have this deep interoperability of both data and the actual tools themselves, so that the silos that they’re built in, we can cross those boundaries, there can be sharing, there can be collaboration.

And in fact, one of the ways to talk about that is sort of to talk about distributed ownership. It isn’t owned by someone, there is a distributed ownership with collaboration and sharing of these components. And, in some sense, by working in the context of impact, we are designing for resource scarcity because every community in the impact space that you work with has something which is often not easy for them, there’s an element of scarcity, there’s an element of resource constraint, which is imposed, and that’s different for each community. But when you are working in the social impact space, that’s the nature of how you’re designing.

And this has the advantage that if we are building the foundational systems based on resource scarcity and collaboration, then we actually know what is needed where and why. So it is not that we’re just saying, well, this is the best thing that’s available, let’s use it. We are actually saying this is the right tool for the job here because, and then we can actually tailor for that and we can improve, and that actually leads to this recognition that we need development to be happening in this diverse set of ways based on a diverse set of constraints.

It’s not just building one set of advances in AI, it is actually recognising we need AI models to serve many different purposes and they lead to different constraints under different constraints. And so we need that diversity of advances to be coming out.

The final thing I was just gonna say is that I also heard this need for adaptation. As you go from one community to the other, let’s not get stuck on what we thought we were doing, but let’s adapt it, so it’s not people adopting, it is people adapting. And so we are building these systems to be adaptable to context.

[00:20:02] Kate: Yeah, I think the core of it is that how do communities adapt and own a social impact programme application for their context, which might vary in everything from the foundational software architecture to details of how the programme gets delivered. So how do they have that but it remains connected with a continuously evolving programme where there is a constant exchange between all these different community adaptations, community work, and sort of a global programme that is getting better, getting smarter, that learnings from one community can be easily picked up and adopted by another community so they’re not learning from scratch when they start the programme.

That is how we think about this in the context of impact, because I think, as soon as we get into talking about different pieces, it’s like, well, where are you actually envisioning? And I know that’s still quite abstract, there’s a lot of details in there about data and governance structuring and operationalization.. What’s the word there? Turning into something that’s operational. 

[00:21:09] David: Operationalisation, yes.

[00:21:11] Kate: And then you know how humans and AI can work together to actually use that, because humans would find that all overwhelming.

And beyond that, how does that scale in a connected way? In the same way that if you were scaling a centralised architecture, you would want it to scale and be connected, but how do you do that in a distributed system? So I would say those are the kinds of things that we are broadly addressing.

[00:21:39] David: Let me reign this in again, because what you are explaining, which is great, is the vision of what this AI would actually be serving, where we are wanting to use it. But let’s come back to what is the foundation of good AI for us. It is AI which is, in some sense specialist, it is serving specific pieces within this, it is not replacing the humans in the system, it is enhancing the humans in the system.

And because it’s specialist, therefore we envisage it as AI which can be actually resource efficient, because we are working in contexts of resource scarcity. And so actually recognising that we can build a lot of these AI systems to be resource efficient if we make them specialist, working in specific areas. It is AI which is both enabling this deep interoperability, but also enabled by a sense of deep interoperability, that learning from one context can be shared with another context and so on.

And so that’s something which is also really subtle, but good. The deep interoperability is central to this, the interoperability of data, the interoperability of software, of AI agents, and so on. This is something which is actually the key, removing those barriers so that we can be exchanging across context is really critical to the foundations we see for these new types of AI.

And finally, the thing which I think you started with is sort of correcting me on the “human centred” to say it’s “community centred”. This is a really interesting concept to say, well, what does it mean to have AI which is not coming out of individuals or of organisations, but community owned AI? What does that look like?

And this is, I think, really, putting it in juxtaposition with the ideas of the empires of AI. We are looking at AI, community owned and led AI. That’s a future, this is maybe a democratised AI. I don’t know, it is what we’re aiming for.

[00:23:55] Kate: Just one final thought there. I would also say I don’t think all decentralised or all centralised is good. We always need some centralization, you want organisations that run things, you want companies, you don’t wanna just send stuff all down to communities.

[00:24:09] David: Wait a second. Let’s be clear in our definition of community. Companies, organisations, these are instances of communities at different levels.

[00:24:18] Kate: Sure. I think I was just making the distinction that sometimes community things can sound like they’re just going to live out in these communities, and then people spin off into, oh my gosh, that sounds like a lot of work. And I don’t think that’s what we’re envisioning. The idea is that there are core organisations that get these going and manage them and keep them up, but that fundamentally a community can be the keepers of them, they understand how they work, they can audit them, whatever those things are that they would need to do.

[00:24:47] David: I think this is really important. What we’re saying is that it is not that we don’t want a sense of hierarchy. The things that we build, there’s a need for hierarchy, there’s having experts, and so on. But maybe what we are trying to do is say that hierarchy can be adaptive, it can be fluid, it can be something which actually evolves over time because of the deep interoperability.

And so it’s not that you are saying, okay, right now this company is leading everything, actually maybe in the future we could use someone else, we’re not tied in.

[00:25:16] Kate: Yes. I think that is a really good point, that you could just buy your AI agent from us and then it’s yours. And then if you wanted to hire a service person to come and fix it at times or help you because the person on your team doesn’t quite have the expertise to do something, you could.

What tech has done right now is we are all just locked into these bureaucratic systems where we are just permanent taxed servers on the land. This is like going back to, I bought my washing machine, it’s mine, but sometimes it breaks and I need a bit of help, but it’s mine. It’s not that you have locked me into this is gonna be taxed in perpetuity for me to even be able to use it.

So yeah. Okay, we have run a bit long on this, so I think let’s end there, as always there’s so much more we could talk about here. But this is a really interesting conversation and it’s so exciting, but we do need money to do a lot of these things. But we’re getting there. I think we’re getting better at articulating, even if we didn’t do it very concisely in this episode, but we’re getting better at articulating what we are working on, what we are working to build and in what context and the value that that has. So I will leave it there.

[00:26:30] David: No, this is good, it’s great to have these conversations, and hopefully the series will actually stimulate thinking for others as well. Being in collaboration is something which is central to our DNA, and so actually we are not wanting to do this alone, we want to do this in collaboration. So, if this is sparking an interest to anyone listening out there and you want to get engaged, get in touch, happy to collaborate on this.

[00:26:55] Kate: Thanks, David.