28 – The Developing Research Principle

[00:00:07] Lucie: Hi and welcome to the IDEMS Principle. I’m Lucie Hazelgrove Planel, a Social Impact Scientist, and I’m here with David Stern, one of the founding directors of IDEMS. 

Hi, David.

[00:00:17] David: Hi, Lucie. Which principle are we digging into today? 

[00:00:20] Lucie: One called Developing Research. 

[00:00:24] David: Oh, yes, this one’s one of our newer ones.

[00:00:26] Lucie: To me, the wording trips me up. 

[00:00:29] David: Yes. It is deliberate I’m afraid.

[00:00:33] Lucie: Okay. We’ve got Developing Research as the title of it. The explanation of it is: this principle is born from our experiences being involved in research for development. Now that’s a surprise for me, given the title of it. 

[00:00:44] David: Yes. 

[00:00:45] Lucie: But it sort of explains where the developing comes from. And then the explanation goes on: it recognises the active need to develop new research methodologies that make use of new data opportunities that are increasingly available. 

[00:00:59] David: Yes, so this is the key point. And so it is this sort of play of words. It’s sort of, research for development. This is where it’s come from. But actually what we found is that when you really want to do good research for development, the current research methodologies don’t apply. They don’t serve the right purposes. 

So what we’re finding time and time again, as we enter into different areas of work, is that there is a need to rethink research methodologies and develop new research types and do new types of research to be able to actually do impact oriented research, to really focus on research which is serving a purpose.

Now, there’s all sorts of academic discussions around this. And many academics say, oh, serving a purpose, that’s not deep research in the same way. The deep research is moving our conceptual thinking forward. My claim is that actually, at the moment, right now, our thinking in most research areas is pretty static.

And really, we could change the way we think if we did research differently, much more, and much more substantially. And there are elements which I’ve learned, I’ve totally changed, I was a pure mathematician. You don’t get much more abstract than that. But I’ve learned about how even elements of the pure maths I was doing apply and are useful to me when I’m thinking and when I’m working on things which are really related to impact.

And so this is where the change which has happened is most research methodologies were born in an era of data scarcity. It’s the way I like to think of that. Agricultural research, Fisher, they didn’t have much data and they couldn’t get much data. And so the statistical methods were all developed to maximise the amount of reliable information you could get from a small amount of data.

You would do your sampling calculations to say how small can it be, because I don’t have much money, I can’t do that much, can I get reliable results? And that’s why you do these sort of sample size calculations, it’s extremely valuable. Right now we have data, you know we’re drowning in data almost.

[00:03:25] Lucie: All of the data that has come out to all of the digital, how our world has gone digital. 

[00:03:30] David: Exactly, the fact that there’s probably more data coming out of your fridge than there was data in the world a few hundred or so years ago. That might not be quite true, but it is an interesting analogy that just any of our devices that we’re using, you know, maybe your phone rather than your fridge, your fridge does give quite a lot of data, but not… 

[00:03:53] Lucie: Yeah, exactly, and like the contents, because I don’t have a smart fridge, but something like the contents of the fridge and then how you bought your fridge and all of that sort of information, I guess, too. 

[00:04:00] David: All of this, if you have a smart fridge, which is actually trying to sort of store the data, that fridge, the amount of data it generates, it’s just immense compared to the data that was available a hundred years ago.

And so we’re drowning in this data. We’re not making very good use of it yet. It’s only just in the last 18 months that people have realised that you can take all the data on the internet and turn it into a large language model, into something which can speak and act and behave in very human like ways.

And of course, that was, this was a huge resource of data which had been accumulating over a number of years, which is now available to be reused and repurposed. And that’s only a small part of the data our world is generating every single minute of every single day. 

And so this sort of data abundance, this era of data abundance, we should be thinking about research differently. If we’re not thinking about research differently, I’m afraid you’re stuck in the past. You’re stuck in the era when you were in data scarcity. 

[00:04:59] Lucie: Exactly, and this comes to a word that you use often in terms of being opportunistic. Why do the extra work when you have something right there? When you have all of the data there, why not make use of it?

[00:05:10] David: There are good reasons why certain bits of knowledge, you need to go through formal processes to do experimentation. But even experimentation, if you take it and you do it at scale using digital interventions, it can be transformed. Actually your big tech companies, they’re experimenting all the time.

They’re not doing it in formal research methods approaches, but they could be. Actually, they have the ability to do better formalised research than any research organisation I know. Now, admittedly, what they use it for is to try and see, can they keep your attention on the advert a little bit longer? That’s what their research is used for.

[00:05:49] Lucie: It’s really going to change the world. 

[00:05:52] David: Yes. Now, what if we were using that incredible infrastructure and power to answer other questions. Of course, part of the worry is that people have used that infrastructure to answer questions which people weren’t comfortable with, and that has caused problems. So this is not all good, misinformation is something which has come out of the fact that people learned how to use that data to spread misinformation and now it’s omnipresent in our society. 

The difference between truth and alternative facts, these are things which are sometimes difficult to distinguish now because there’s a real corruption of the mechanisms that people use to disseminate information and distinguish between truth and falsehood. And part of that, of course, is subjective, but it’s a lot of things which are not subjective. 

[00:06:39] Lucie: So interesting, yeah. 

[00:06:41] David: And so, the idea that the research methods of the past are the research methods we need in the future, this just needs to be put into question. It amazes me that our journals, top academic journals, there was a wonderful paper in Nature a few years ago, which basically said that 50 percent of good articles in good journals use statistics incorrectly. 

Now, that’s just using the traditional statistics incorrectly, let alone actually how many are using modern methods which are taking into account sort of different data sources, which there are some there. 

But actually a lot of research journals aren’t looking for that. They’re looking for things which follow traditional research patterns. These are things where thinking about how we change the nature of research across disciplines, that’s really important. It really is. 

I think there’s an opportunity to really rethink how we build our knowledge as a society in ways which will come out from the fact that data is available in ways it wasn’t before. You know, the census, this has been around for an awfully long time. It’s an incredibly powerful tool of the past. 

[00:07:59] Lucie: I think it’s one person generally going around every house and asking a whole list of questions. 

[00:08:05] David: Not one person. 

[00:08:06] Lucie: No, sorry. 

[00:08:08] David: That would take a long time. 

[00:08:11] Lucie: An actual person going to an individual house.

[00:08:16] David: And now in some countries, it’s gone digital. And what does that mean, and how does that change things? But this is just one of many of these sorts of data sources, which actually have been there for a long time. These are things of the past. And yet still, we don’t use them as much as maybe we should.

And now there are other opportunities to rethink this, and think about, well, actually, are there other or better ways to be able to have better data on some of these things? And if you’re thinking of the census, that’s just one. And I’m not, you know, I’m a big fan of the census, it’s incredible data.

There’s a group that actually has been making efforts to make this more widely available and doing some fantastic work. And we should be making more use of that data. It’s incredible, all over the world, we’ve been doing census. 

[00:09:03] Lucie: Exactly, I used census data from Vanuatu in my PhD.

[00:09:06] David: There you go! So when did Vanuatu start its census, do you know? 

[00:09:11] Lucie: No, I don’t know, but the last one, it was fairly old, it was about seven years old or something, which was annoying. But they just redid one after… 

[00:09:18] David: Of course, it’s about a 10 year cycle.

[00:09:20] Lucie: Okay. 

[00:09:20] David: So this is the thing, they’re 10 year cycles. So traditionally a census is done every 10 years. 

[00:09:26] Lucie: Because of all of the human effort it takes to do it though too. 

[00:09:29] David: Very expensive. It’s extremely expensive. 

[00:09:31] Lucie: Yeah, human and financial. 

[00:09:33] David: Well, I mean, the human effort is made possible because of the finances. But this is something which is such an important part of the legacy that we have from the approaches of the past. And we’re not yet making even good use of that. Actually for a long time, these censuses, they were stuck behind closed walls. A researcher could access it here or there, but they weren’t really being used that much. 

More recently, quite a lot of places are making at least part of the census data really openly available at the sort of granular level. And what that means is you can actually answer questions about societies and look across the time, you know, because it makes, it’s such rich information. It doesn’t mean it’s perfect, it has its own problems. But it is extremely rich, extremely valuable. 

Now I don’t know quite why, from developing research we got onto the census, because… 

[00:10:28] Lucie: Well there’s two interesting things that have been popping out of that anyway, which is one in terms of the opportunities, and two just improving how research is done in general, because we started off saying that Developing Research comes from research for development.

[00:10:42] David: Yeah. 

[00:10:42] Lucie: I perhaps have a too narrow idea of what research for development means, but here we’re really talking about, very broadly, research for a purpose, perhaps, research for impact, research to change things. 

[00:10:54] David: Yes, absolutely. The point is, and this is in the UK, the Research Excellence Framework, the REF, which all universities go through this process to sort of be allocated funding in the future and so on, over the recent cycles the importance of what they call impact case studies has just grown and grown each time. And this is where the research that’s actually produced is finding a way out into society to give value back to society. And that’s something which is valued more and more.

But it’s amazing how journals are not responding to that in the way that I think they should. If only journals and researchers valued more impact oriented research differently, then of course this might lead to deeper learnings related to this. Because in some sense, well, let’s put it this way, if you make a really good advance in a theoretical framework and then lots of people use it. 

If you would value the advance, then that’s one paper, and then it’s other people’s job to use that and to actually apply it. That’s not valued as much as the one paper which actually advanced the thinking, which led to everyone else doing this. That’s a very traditional way of thinking about research.

In agricultural research, what this led to was things like new crop varieties and so on, where you would do the research which would develop a new variety, that research then would go out and once it existed it would get registered in the catalogues and then you’d have an extension service that then goes out and gets it to people.

And if you’re working in Africa, what you’d find when you get it to people is that they don’t tend to adopt it because the variety wasn’t adapted to their local environment for whatever reason and it didn’t fit into the things they needed. 

And so, more recently, people have been doing much more participatory breeding, where they’ve actually been doing the breeding on the farms with the farmers to understand what their needs are and why they need certain things, why do they value one thing over another? 

[00:12:55] Lucie: Yeah. 

[00:12:55] David: Now, let’s think about this differently. So now think about that same process, you know, your theoretical advance, your breeding of a new variety going out and not being used. That’s something which can be replicated in most domains. Whereas actually your participatory breeding, this is really rather more interesting, because actually that doesn’t need to be thought of as a single process. This can be a continual loop of learning and of dissemination and so on. 

And that’s only possible because you can get data at scale now in ways that you couldn’t in the past. And now you can actually be having the research happening not just at the level of the breeding of the variety, but also on the implementation. Why are farmers choosing this? 

Now that might not be a breeding research thesis, but it might now be an anthropologist who goes in and understands what’s the social reasons behind the adoption of certain varieties in certain places and not in others, or whatever it might be. But you think about different research angles coming out much more strongly.

Why are people adopting certain things and not others? How are they using it? How is it changing the food systems where they are? This new variety has come in, suddenly, actually that’s really good for making beer, and so suddenly a lot of the land is being used up to grow the sorghum which is used for beer, and now there’s a food insecurity problem, but people are much richer, and so they’re buying less healthy food. I’m making this up, but it’s not so far from what I’ve seen.

[00:14:23] Lucie: We’ve discussed this in other contexts about different ways of having impact, that traditionally research has been about having sort of theoretical impact, whereas actually there’s loads of potential for having impact, I don’t want to say on the ground, but real lived impact, I guess, perhaps by people.

[00:14:40] David: Yes, let me give a very concrete example of that, I’m sure I’ve used this in other cases because I love it. This is a group in Niger, Fuma Gaskya, who have basically created a market in human urine in eastern Niger, or central Niger. And because they did the whole research process, very participatory in different ways, really, I loved it.

They’ve basically shown that actually, by integrating it into the farming system, there’s now a demand for it, people are repurposing it, they’re recycling it in all sorts of interesting ways, and it’s being used really at relatively large scales now, and it was not being used before. And so this has really had an impact in that zone.

Now on the one hand, you can say, well, there’s nothing interesting or new about this because everybody already knew that human urine was a valid, a sensible fertiliser. So they haven’t done anything new. I disagree. The way they took this from a sort of holistic approach of actually making sure they had religious leaders, scientific leaders on board, they had local chiefs, they had everybody going through the process together to be able to get a local innovation.

Yes, of course it was known that human urine could be used as a fertiliser, but it was not known how to integrate it into a religious society, which was sort of, in very harsh conditions, in this sort of way. So all you need to do is frame your research question differently. And that research question is a much more powerful research question than just, is human urine a sensible fertiliser?

Well, you can do a chemical analysis and find, yes, it will probably work. That’s not an interesting question. Most of the sort of bigger questions aren’t actually that interesting. I know this because I’m a mathematician. The questions I love to work on, we’re not sure of man or beast for a hundred years or more. I love tilting t-structures, I strongly recommend it to anyone with a totally abstract mind who doesn’t want to do anything useful but just wants to solve puzzles. 

So if that’s what you want to do, great, but why is that the focus of our research, doing things which aren’t the really useful and insightful knowledge game? The knowledge game of the complexity. And what blows my mind is when you talk to researchers about this, they’re so domain specific that they’re only interested in the advance in their domain. The power of what Fuma did was they considered everything, they considered religious aspects, social aspects, they considered the scientific aspects, you know, they brought everything together in this holistic way. They cut across disciplinary boundaries because that wasn’t what was important to them. 

That’s what we need in terms of research. We need to be able to know what is the limiting factor. Is the problem really a scientific problem? Is it a problem which is cultural? Actually, maybe the reason something’s not getting improved or integrated into society is because of cultural reasons, that’s important. 

[00:17:39] Lucie: And this absolutely links, this explains why Developing Research as a principle fits under the Informed Decision Making principle for exactly what you’ve just said, both in terms of the research questions but the use of the research, who’s going to use it? 

[00:17:53] David: What if we thought, at the moment researchers are put on a pedestal quite often and they’re kept in the ivory tower. What if we thought of researchers as serving others much more? What if they were something where they were actually asked to go outside of their comfort zone, to be transdisciplinary, to really try and sort of build evidence around things, investigate things more deeply?

[00:18:18] Lucie: I’ve been interested to see, I think it’s in Burkina Faso that they have a lot of research institutes which, at least in terms of the way they talk about it, it’s about serving their country’s needs. 

[00:18:30] David: Absolutely. 

[00:18:30] Lucie: I found that really interesting. 

[00:18:32] David: This is common across Africa now. And this is why I’m so excited to work in Africa. I feel very privileged because I do think that there is a recognition that actually, they want to serve others, they want to serve their community, they recognise the value that knowledge can bring. 

[00:18:49] Lucie: Exactly, I think that’s what I’ve witnessed, that the researchers I’ve worked with, they’re really motivated to do that, whereas a lot of European ones aren’t so.

[00:18:57] David: Exactly, but they’re not often empowered to do that, because of course there’s issues about financing, you know, labs, but more than that, more fundamentally. If they do do good research like that, it’s not recognised internationally. People don’t value it as good research. Oh, you’re doing something in your local environment, oh, good for you.

But no! That’s deep learning you can get out. We should be recognising what deep learning actually is. Deep learning is not people like me, because I loved being an abstract researcher, tilting t-structures on my own in my room where nobody had to bother me. It was fantastic, what a privilege! We need to get out of that privilege. We need to actually do research with purpose.

That’s where I think, you know, I believe this is going to come out of Africa. My guess is that the ‘developed countries’ are going to get way behind on this. They’re going to really struggle to do this, even though they’ve got all the resources.

[00:19:55] Lucie: For people who couldn’t see David just then, he put quotes up when he said developed countries. Important to say in a podcast. 

[00:20:03] David: You’re absolutely right. Those quotes are important. 

But I think the point there, which is so important, is that if you want to compete, if you’re in Burkina Faso and you want to compete internationally, the odds are stacked against you in every single way. And it’s just, it’s not right, in many different ways. And yet what I see happening is that the innovations which are happening, the really insightful research, which I’ve experienced, it’s coming out of these environments, it’s coming from Niger, it’s coming from Burkina, it’s coming from Kenya in some cases. 

And other places, you know, Ethiopia, there’s an incredible culture of the sort of universities serving their communities. The outreach programs are really important, valued and powerful. And these environments, actually I have more hope maybe in some ways for Ethiopia, despite the troubles they’re going through at the moment, because they’ve built a local infrastructure where the only problem it really has is it’s totally isolated.

You know, it’s really very isolated from the international world because the outside world is very expensive compared to doing things internally in Ethiopia. And so they tend to just do it internally. 

[00:21:20] Lucie: What sorts of infrastructure do you mean, sorry? 

[00:21:23] David: Even just conferences. I’ve been privileged to go to a number of sort of conferences in Ethiopia, where I’m the only outsider there, and I look around, I think, this is you know, this is an amazing group, they’re big, hundreds of people, coming from all over the country, they’ve got a lot of institutions now doing very interesting pieces of work. 

It’s of varying quality, but the point is a lot of it is trying to be more impact focused. And they’re basically doing it on their own, in a totally isolated way. Now, I go in and I say, this shouldn’t be isolated, you guys should be getting out more. And then I think, but actually, the fact that you value what’s happening internally and you’re able to build things internally, maybe this isn’t such a bad thing. 

I’ve been going to Ethiopia now for, on these sorts of research visits and so on, for 12, 13 years. And I’m so impressed. The evolution that’s happened even in that period of time has been incredible. And the isolation, you know, I tend not to go to Addis, I’ve worked in Addis and done things in Addis before, but Addis is more internationally connected. But Ethiopia is a huge country. It’s the second most populous African country after Nigeria. 

[00:22:34] Lucie: Okay, didn’t realise that. 

[00:22:36] David: And the point is that they’ve now built these amazing sets of institutions, lecturers, I remember discussions over ten years ago with lecturers at Bahir Dar University, where we were discussing, compared to their Kenyan colleagues, how terribly paid they were. And they said, we accept this. And you say, oh, that’s very good of you. Well, why? Well, we recognise that, because we’re so badly paid, look, they’re able to employ so many of us, and we’re able to grow as an institution, and we’re able to offer this service to all these students, and to actually make the systems work.

And those students, that generation, they’re going to be able to be properly paid. They’re going to be able to get proper salaries, and they’re going to be able to then be part of an international world, and so on. 

[00:23:18] Lucie: That’s interesting. 

[00:23:20] David: Maybe they were a bit optimistic, because I think they were thinking 10 years down the line, 10 years ago. But… 

[00:23:27] Lucie: But what you’ve just explained, it ties again with the first podcast, which was titled Mathematical Scientists Thinking Hard for a Collaborative Society, is that right? 

[00:23:35] David: Mathematical Minds Thinking Hard for a Collaborative Society. 

[00:23:39] Lucie: Yeah, that one. But so the Collaborative Society bit then, it seems that in Ethiopia they have, or they are working on an interesting… 

[00:23:48] David: I can’t speak for everyone in Ethiopia and I know there’s been immense issues with Ethiopia over the last few years and the internal struggles are serious and I don’t want to downplay the problems. But I do want to say my personal experience with people across the country that I’ve met and that I’ve worked with, Ethiopia is the only country on the African continent never to be colonised. I think Lesotho might have a similar claim. But Ethiopia was never colonised. 

[00:24:15] Lucie: That’s quite something to say. 

[00:24:17] David: And the point there, they do have Italian influence. 

[00:24:21] Lucie: Yeah, I thought so. 

[00:24:21] David: They were not colonised. That’s a whole other story. But the point there which is so interesting that I find now is that you actually, when you go around, there’s an ancient, culture, this is more like in, in many ways, I find it very similar to going to places like Bangladesh, Pakistan, India, where these ancient cultures are sort of, they have lots of different influences, and they’re very deep and they’re very old. You can’t understand them. 

The one thing I know from spending a little bit of time in India is that if ever I think I know anything about India, I’m wrong. So it’s sort of, it’s just a deep cultural mix which is so diverse and so incredible. And Ethiopia has a bit of this as well, that it really has very old cultures which are very complex, but they’re very proud of them and correctly so. 

And there is this sense of actually working together, or there was a sense of working together, whether that’s been eroded over the last few years because of the situation, and so on, and of course it’s not everyone, but it is generally this sort of sense that actually, as a country, they are building something, or they were building something really substantial. The way the university came up and the way they built their research infrastructure to me highlights that there is a possibility of developing new ways of doing research. 

And maybe these will come out of contexts like that where they’ll value them more, value them differently, maybe, rather than more. And that would enable the innovation to happen, which would develop new methods for research, which I believe will lead to better knowledge, more contextualised knowledge, knowledge which sort of isn’t overly simplistic.

One of my big criticisms of a lot of the international research which I interact with, is it removes complexity, it tries to search for simplicity. Now, as a mathematician… 

[00:26:28] Lucie: It’s easier to handle! 

[00:26:30] David: Well, economic theories, my wife’s in economics, and economic theories are sort of, you simplify down until, actually, I don’t believe it anymore.

[00:26:39] Lucie: Yes! Exactly, it’s easier to understand, but it’s more likely to be wrong. 

[00:26:45] David: And then you accept that, well, everything’s wrong because these are models and so nothing’s perfect. That’s okay. But does it have substance? Is it telling you the right things? Is it putting you in the right direction? Those are whole different matters. And I don’t know that we know that because things have been oversimplified. 

Dealing with complexity requires totally different methods. And we’re not developing that much. There are people who do try to develop methods to deal with complexity, but it’s hard, and so they don’t tend to be as well received.

People like simplicity, and even in research which really worries me, because most of the things which are worth studying are complex, in my opinion. There are some simple questions still out there to study, but most of the simple questions that were worth studying have been studied. A lot of the research that’s left, and there’s so much that needs to be done, deals with complexity and accepts complexity.

To come back to the principle, and this has been a rather rambling podcast. 

[00:27:42] Lucie: No, we’ve gone from like the very sort of narrow of what data exists in research and how that can be better used, how it can lead to better research questions, and then how it can actually have impact more widely. So there has been, well I’ve been following a thread. 

[00:27:55] David: I really appreciate the fact that you’ve managed to put a thread to my rambles. But it’s important, whether there was a thread or not, I feel this particular principle is, the wording is deliberately problematic because we don’t know how to frame it.

But it is this element that current research processes are not serving the purpose that I think they could serve if we had new research processes, which were more tailored to dealing with complexity, to looking at impact, to investigating different types of questions in different contexts. 

[00:28:31] Lucie: Thank you. That’s a great place to end, so thank you for your time, David. 

[00:28:35] David: No, no, thank you for a nice discussion.