040 – The STACK Internships

[00:00:00] Santiago: Hi and welcome to the IDEMS podcast. I am Santiago Borio, an Impact Activation Fellow, and I’m here with David Stern, one of the founding directors of IDEMS.

Hi David.

[00:00:17] David: Hi Santiago, I’m looking forward to hearing about STACK, which is I presume our topic for today.

[00:00:24] Santiago: It is our topic for today, and it’s motivated by quite a few members of our team attending the officially, international meeting of the STACK community, unofficially the Stack Conference 2024. And we should in one minute, no more, explain what STACK is to give a bit of context.

[00:00:47] David: I would put it very simply that STACK is the open source intervention which is transforming maths education in ways which I feel are word leading. It’s very specific what it does but it’s about giving interesting automated feedback derived from a computer algebra system.

[00:01:09] Santiago: To mathematics questions online.

[00:01:12] David: Not just mathematics. I mean, it’s been used in all sorts of areas, physics, engineering. We’ve used it beyond in statistics, data. The key point is because it can do the maths, it can be used anywhere where that is useful.

[00:01:27] Santiago: And we will probably have another episode going into what STACK is and how it’s used and how we’re using it in depth. But that’s not the focus of today.

[00:01:36] David: I guess part of today is the fact that at the STACK event, two of our former team from Kenya, who are now PhD students at Trieste, were present. And their journey along with others’ journey is part of what we might want to be talking about.

[00:01:52] Santiago: So beyond that, not just two of our former team members. We had the main driver of STACK in Africa, Mike Obiero, doing a presentation online, and we had four other people who work with us, they’re now at INNODEMS, two of them gave a joint presentation, one gave a poster presentation, and the fourth gave a lightning talk. And it’s really about them that we want to talk about today.

[00:02:29] David: And I should clarify that this is something where there’s other episodes which talk about internships more broadly, which IDEMS has been involved in across different contexts, particularly in Africa. But the STACK internships, which have emerged, I think they’re pretty special, these recent ones. So yeah, that’s what we’re really discussing.

[00:02:52] Santiago: So, let’s go back a bit. I want to mention Mike in a bit more detail. He went back to Kenya after doing his PhD in the US and wanted to innovate and talked to you and others.

[00:03:04] David: Let me just correct one detail there. He did his Fulbright scholarship PhD at University of Illinois Urbana Champaign, and came back turning down a permanent position at Urbana Champaign because of what he’d been doing in e-learning there to take a position at Maseno University because he wanted to make a difference, he really wanted to change the system.

And after less than six months back, he was almost ready to throw in the towel. He needed to innovate. He didn’t want to innovate. He did want to innovate, but he needed to innovate. Otherwise, he was not going to survive coming back to Kenya to work in the Kenyan system because he just couldn’t see how with the classes of… I think that time he had a relatively small class, it was only four to five hundred but having classes of four to five hundred students, he couldn’t give the type of education, with no tutorial assistant, nobody helping him, he couldn’t give the sort of education and feedback that he thought would lead to learning. And so he was after six months feeling he’d made the wrong decision. I should definitely do an episode with Mike.

[00:04:18] Santiago: You should definitely do an episode with Mike. So let’s not get too much into the detail of what happened. I disagree that he couldn’t… he’s quite a resourceful guy.

[00:04:28] David: He is an extremely resourceful guy. He’s one of the smartest people I’ve ever met. But the point was, this was back in 2018. We were able to get him to come to visit the UK, came to an event in Oxford, and then we got him to go to Edinburgh to meet Chris Sangwin to find out about STACK.

[00:04:44] Santiago: And Chris Sangwin is the main developer of Stack.

[00:04:49] David: Yes. And that then meant that we supported him back in 2019 to try and use that to change his situation there a bit. And that’s actually how you got involved.

[00:05:03] Santiago: I got involved actually creating questions for some of his courses. I was still back in Argentina before I was the first IDEMS employee.

[00:05:12] David: Exactly.

[00:05:13] Santiago: I was working freelance for IDEMS and I got involved because I had experience of STACK. STACK is about 20 years old and I got involved on its very infancy through a summer internship that I got while I was doing my studies to create questions.

[00:05:36] David: I think it’s worth putting the year on that, this is worth stating. You were not only an intern on STACK, but you were one of the very first interns on STACK.

[00:05:44] Santiago: I think I might have been the first. It was 2005, summer of 2005. It was a two month internship transforming a resource that was given to starting students at the University of Birmingham to make sure they are up to scratch with the basics.

[00:06:03] David: Yeah. This was not the same open source product that now exists. This was before 20 years of development had really, really turned it into the powerhouse it now is, but it was already of value then, and it’s done nothing but strengthen since.

[00:06:20] Santiago: Indeed. But going back to Mike, we worked with him in collaboration. He saw the impact and he said, I can’t be the only one doing this in Africa. And he organized a workshop and other universities, local universities got involved. And well, not so local necessarily.

[00:06:41] David: So this is 2019, the first workshop, and we had people from the whole East African region, and so there was real interest, real demand right away. It’s taken time for people to really gather speed on this, but the community that Mike’s built up is incredible. And the first African conference last year was amazing.

[00:07:02] Santiago: Let me mention particularly in that first workshop in 2019 two universities, one in Ethiopia, Bahir Dar University, and one in Tanzania, the University of Dar es Salaam, both took their learnings and tried to implement with different success.

[00:07:22] David: Well, they weren’t the only ones. I mean, others took it back and used it as well. But both of them, not only had different levels of success, but they both persevered in different ways over the years. And so both of them, things have grown, they have evolved, they’ve remained really very much, you know, part of the community in East Africa, pushing this.

[00:07:47] Santiago: I was just trying to correct my local comment by demonstrating the breadth of the impact of that workshop. But then he didn’t stay still. He organized another workshop in 2022 after having to take a break from COVID on in-person workshops. And, it was wonderful, and more people attended.

[00:08:10] David: I didn’t because I had COVID. I was there. I was ready to meet everyone on the first day and then actually was driven away because I had COVID. So yes, it was still disrupted by COVID.

[00:08:25] Santiago: It was a hugely successful event. Higher attendance, much better. Well, better …

[00:08:33] David: It was different. But our story does start here.

[00:08:37] Santiago: The real story that we want to speak about starts there in that workshop because we realised that we needed local talent and local support to create resources.

[00:08:54] David: It’s not just to create the resources. By this point we had an open question bank with sufficient number of courses to be of interest to other institutions. But the need for people to actually adapt those questions, to work on them, and the demands that placed on lecturers was just unreasonable. And so actually being able to bring in people who could specialize and take those skills to be able to help other institutions, you know, adapt them for themselves and integrate them into their courses became really apparent then.

[00:09:29] Santiago: And I must say that I was taking a break from IDEMS when that event happened. So it was only when I joined back a couple of months later that I heard about what had happened. And I was both excited and a bit pessimistic, I must say. But what happened was that, I think Mike had the idea, of hiring recent graduates as interns to learn how to create these materials.

[00:10:05] David: And we did go back a step because Juma, who is now in Trieste doing his PhD, and Wastalas, who is now doing his PhD in Trieste, were already sort of identified and on board at that point. And so these were master’s – PhD level students who had been involved and who were doing bits of work. And Mike said, no, we need to get more people on this. We need more people and we could just bring in recent graduates and enable them to grow with this.

[00:10:36] Santiago: And you did! And it was a quickly advertised position, four positions for an internship.

[00:10:44] David: We didn’t advertise four positions, but it just so happened that four exceptional candidates came forward and we agreed to have those four.

[00:10:52] Santiago: There were about 20 candidates that applied and four were selected. There was an interview process and four were selected.

[00:11:01] David: INNODEMS at this point, who was hosting them, has huge numbers of interns always interested. They always have a backlog of people interested. They always have this sort of process now. That’s been established over the last ten plus years, predating IDEMS and INNODEMS, where this sort of internship process has been evolved. That’s a whole other discussion.

The thing which I think is so interesting with the fact that these four were selected is that for STACK, it was felt that we couldn’t have too many, but we needed enough to make it worthwhile to develop the training. And it was that balance between those two of actually having enough but not too many that led to us getting these four, and they were really exceptional candidates who then have grown into that role through the guidance of others.

[00:11:53] Santiago: Let me perhaps add a bit of clarity. It wasn’t four initially, it was eight.

[00:12:01] David: Well, it was four interns plus four others who had been involved. So we were doing training for eight people because there were four others who were already involved in other work and interested in learning how to work with STACK. But they weren’t full time in that internship in the same way. They had other roles.

[00:12:22] Santiago: And I got back to IDEMS. I was told about this and I got very excited, I finally can have a team of people working with me creating questions rather than doing it myself. Creating a question is very time-consuming. I would consider myself, a high quality author of questions and a quick question takes two hours. A very complex questions that are quite rare can take two weeks or more but between two and five hours, I would say a standard question. So it can be very time-consuming and sometimes a bit tedious.

[00:13:07] David: Well, many of those questions that you’re talking about are actually relatively similar and the questions you can do in two hours, these aren’t groundbreaking new questions, they’re just questions which have a specific pedagogical focus and where you’re trying to achieve something very specific.

The two week questions, those are the interesting ones. Anyway, that’s a whole different story again. So I think the point is that the need, when you have a new lecturer who gets on board who wants to do this, to support them to get high quality questions is really there. Even if you have an open question bank where they can be selecting questions, it’s not the same as being able to have people who know that question bank who can then say, yes, okay, I understand what you’re trying to teach and we can now try and adapt the question for that. Which is what this group of interns now do for partners, particularly in Kenya, but beyond as well.

[00:14:01] Santiago: And I think it’s a good time to mention another powerhouse in Kenya which is Professor George Lawi who, it coincided that he wanted to institutionalize STACK at Masinde Muliro University for Science and Technology, MMUST, and he had huge ambitions. And we really didn’t have the capacity to satisfy those ambitions.

[00:14:33] David: He was looking quite quickly at doing multiple courses at once, most of which based on existing materials, but with lecturers who had never used STACK before, and he was aiming to do it so that it reduced their workload. This is something which I love, this idea that he was going into the lecturers and saying, yeah, just teach as normal, but your continuous assessment, which you normally have to mark hundreds and hundreds of scripts, well, we’ll do that with electronic assessment for you. You don’t need to worry about it. Your workload has just gone down. That was the way he approached it, which is so powerful.

[00:15:06] Santiago: And we’ll get into that in more detail in another episode, explaining the impact that that had as well and give examples, but the reality is, we didn’t have capacity to satisfy the requests for support that we were receiving.

[00:15:20] David: Well, we did because that’s why we took on the interns.

[00:15:24] Santiago: So we took on the interns. I started working with them. I started training them. I must mention Georg…

[00:15:29] David: He’s now a Mathematical Scientist within IDEMS.

[00:15:33] Santiago: He was then, if I’m not mistaken, an Impact Activation Fellow.

[00:15:36] David: But he’s been involved in STACK as well and was deeply engaged in this sort of process of bringing on these interns, working with Juma, working with Wastalas and trying to get a team together to do more.

[00:15:49] Santiago: Yes, and he laid the foundations for this to happen. And we started together training these eight people with the support of Juma and Wastalas in country, locally, supporting them, helping them. We had two meetings every week. We set tasks for them. Georg’s tasks were more standard in the sense of this will help you develop a particular skill. My tasks were more just author, get it wrong, we’ll analyse where it’s gone wrong and we’ll improve. It’s a different style and both combined work very well.

[00:16:31] David: And the point which I think is critical here is that there was, over an extended period of time, a real effort put in. And let’s be honest, at the beginning, you didn’t think it was working. It took a long time.

[00:16:44] Santiago: For the first two months, at least, I was arguing to you, quite strongly, that the work that was being produced, Juma, myself, Wastalas and Georg could do it in a better quality, more efficiently. And we could deliver the same materials that were being delivered through the internships with higher quality.

[00:17:15] David: Absolutely. I remember that. And I was sort of saying, you know, I understand, and yes, this might not work. We may need to sort of pull back on this, but let’s give it a bit more time.

[00:17:27] Santiago: And I think the pivotal point was when we realized that the additional four, they were also engaged in other projects. They were also doing other education work because of course they weren’t STACK interns. They were working with INNODEMS, they were doing all sorts of other interesting stuff.

[00:17:49] David: Maths camp, maths clubs, they were running all sorts of things. They were doing these wonderful elements, painting primary school classrooms, the Happy Classrooms project. They had all sorts of things going on.

[00:18:02] Santiago: And they were very valuable things. But really, if you want to learn STACK you need to throw yourself into it. And especially, and I don’t want to be negative about the education systems in Kenya or anything of the sort, but especially with the skill sets that recent graduates have.

[00:18:26] David: They haven’t had necessarily this exposure that you might get in other contexts that would enable you to support this sort of broader skill set which you’d need to go into this project.

[00:18:38] Santiago: Yeah, and that was a pivotal change. We had a discussion and we said, let’s let the four prior INNODEMS people focus on the rest of the education work that INNODEMS is doing and let’s focus deeply with a more personalized program with the four interns that we got specifically for STACK.

[00:19:09] David: Yep.

[00:19:09] Santiago: And that changed everything. Not just that. There’s one more person I want to mention, who is Dan Kelly, a current undergraduate student at Oxford University who is passionate about our work, particularly our education work in Africa.

[00:19:32] David: Well, the STACK work. I mean, very explicitly, STACK, he experienced this during the pandemic when he was at school, and he grew from there. But he was being really passionate about STACK. And this was his first chance, I believe, to go to Kenya. And he visited them and worked with them intensely for a number of weeks, which I think was an inspiration for them, and really very exciting.

[00:19:59] Santiago: Him and Georg.

[00:20:01] David: Yes.

[00:20:02] Santiago: They both travelled to Kenya. I think this was December 2022.

[00:20:07] David: That’s right.

[00:20:08] Santiago: The progress they made… There is a lot of progress that can be made with virtual training, but having that sprint face to face, it just transformed the quality of questions, the quality of resources that the interns were creating. And it just started growing and growing and growing in quality from there. And I suddenly said to myself, actually, this could work.

[00:20:38] David: And let’s be clear, it wasn’t until about February, March 2023 that you really started saying that. And that was exciting because then, from then, we were in the run-up to the next face to face event.

[00:20:54] Santiago: Which was the Ethiopian workshop.

[00:20:57] David: Which was the Ethiopian workshop. But I was actually thinking of the first African STACK conference.

[00:21:03] Santiago: Which is where they actually contributed.

[00:21:06] David: Yeah.

[00:21:06] Santiago: So the first African STACK conference, again, Mike, the powerhouse, Lawi, the powerhouse really drove that…

[00:21:16] David: It’s them driving it with our support. And we weren’t the only supporters. Danilo, of course, who’s now recruited Juma and Wastalas as PhD students in Trieste, it was central in that as well. Chris Sangwin himself was there, which without him, it wouldn’t have had the same stature, and he was exposed to what’s happening there and was really impressed, just as everybody there was motivated by the presence of Chris. And so, these things came together in very interesting ways but let’s come back to the interns themselves. You focused on the four interns but I want to come back to Juma and Wastalas.

[00:21:56] Santiago: Okay, go on.

[00:21:58] David: Well, because they came in in different ways. They predated this. Juma was doing his Masters in Educational Technology and Mike was one of his supervisors, who got him on to stack through that as part of this. And then he took on an internship role within IDEMS.

So Juma was volunteering, as you say, in the e-campus. And he was involved with Mike there, but Mike was the person who had really identified him already, which is why you were introduced.

And not just Mike, I mean, this is where we should also mention Mildred, who’s the head of the e-campus in Masseno, and is just fantastic. I’ve worked with her for many, many years. And she actually identified him even before that. There was history which sort of led to your meeting and your actually getting him involved in STACK.

But I think the key point here is that that journey that he went on, where he then took this on as part of his Master’s, he actually ended up getting an internship with us where part of his role was to support this broader team and bring them up. He originally was there to support Mike and to sort of get involved, but it was clear that there was just so much need, and he’d recognised that, and he was trying to do his own studies, and he got involved and therefore got sucked in to actually doing bits of research related to STACK.

But it’s something where his journey there was central to enabling there to be some face to face contact for the other internships. And so this layering of internships was important. And alongside him, Wastalas, who was a PhD student at Masseno, originally from Benin, got involved as well. And so we have these two more senior members who had more experience outside of STACK, who were then playing a critical role in supporting the team on the ground.

[00:23:47] Santiago: And by that point, if I’m not mistaken, they were subcontracted by IDEMS, they were beyond interns.

[00:23:55] David: Yes, they weren’t interns, that’s right, they were being subcontracted by IDEMS, and it’s an interesting one, that it was something we’ve done in other ways. They were still studying, both of them, and it was a way of supporting their studies by giving them bits of work. This was sort of, if you want, more like a study scholarship. And Juma finished his studies and actually transitioned into a slightly different role. But it was really a way to support them through their degrees while they were, actually doing interesting bits of work, particularly supporting the local team.

And I should be clear that that need to work while you study is very common in Kenya and elsewhere. That you don’t get scholarships to study generally, and so finding ways that the work you’re doing actually supports your studies is really very challenging, and that’s part of what we try to do.

[00:24:52] Santiago: And it’s very rare, not just challenging.

[00:24:55] David: Yes, it’s a really tough scenario and tough situation.

[00:25:01] Santiago: So, we were in our story about the conference, you added a bit of context about Juma and Wastalas because they deserve that.

[00:25:09] David: It’s not just that they deserve that, it’s their role in this was central, you mentioned the importance of face to face. If we want to make systems like this work, you have to have this layering. The amount of effort which is needed is substantial, and this was something where everybody along those layers gains.

[00:25:32] Santiago: You say substantial, let me clarify, in terms of person hours, it was about two days a week for almost eight months.

[00:25:44] David: Yeah.

[00:25:44] Santiago: And only then it started to go down.

[00:25:46] David: Yeah, absolutely. And that was remote. You know, this is where there’s a question about whether this would have been more effective or more efficient if this was done actually face to face, exactly as you said. And so there are really interesting issues. Of course, it would have been more effective face to face. But that would have been unaffordable, this would never have happened under those contexts and in that situation. And so, the amount of effort which is needed to make this work is so substantial. But at the same time, the rewards and what’s come out, and I think getting to the conference, the role that that team played, and that goes from Juma and Wastalas down to the interns and the opportunities that came out. And the reason I keep coming back to Juma and Wastalas is their evolution, then out of that to get these PhD positions is really also central.

[00:26:40] Santiago: Let me add a tiny bit of context. The conference that we helped organize was hosted at Masinde Muliro, which we mentioned earlier in the episode, run by Lawi, and it included two days of STACK workshop to explore the question banks and to get lecturers to be able to grab the materials and use them. And I think that’s where a lot of the contribution from the interns came.

[00:27:13] David: Rather than saying two days of STACK workshop, because that’s not how it happened, the afternoons every day, 40 percent of it was STACK workshop, is the key point. That it was done throughout, that we had elements of that throughout the week.

[00:27:29] Santiago: Of course, Juma, Wastalas, the interns, they gave presentations, they interacted incredibly well with the audience and so on, but that is where they really shone, they really demonstrated that they had achieved a huge skill set.

[00:27:53] David: Absolutely. This is where they were now interacting with lecturers in their environment, being recognised as experts. One year out of graduation, they had expertise in something specific, but they had something which was of value to the lecturers and the lecturers couldn’t have done this without them.

And that recognition was huge: huge for their confidence, huge for the lecturers. And that’s part of where, since then, they’ve now been growing further in terms of actually that support offered.

And I do want to just come back to this sort of Juma and Wastalas on that. We’re now sort of outside of that. They were already a bit more senior, they were not only recognized locally, they were sort of beyond that. It was about the fact they were recognized by the international partners as people who could now help move forward, be at the cutting edge of what’s happening in STACK, not just in Africa, but internationally. And this is where their PhD positions have sort of come. They’re now at Trieste studying essentially STACK and how this can be used to transform education, not just for Africa, but more generally.

[00:29:02] Santiago: And you mentioned where Juma and Wastalas are now. I’d like to mention where the former interns are now. They are now employed full time by INNODEMS to continue this work. And they just started training the next person, the next generation. It’s one individual because we have to be opportunistic, and we have to think about sustainability and scalability carefully.

[00:29:31] David: And there simply isn’t the financing at the moment to support a bigger group.

[00:29:35] Santiago: And we have this one individual that they are now starting to take Juma and Wastalas’ role, and train this person. And as well, thanks to SAMI, the charity that we work very closely with, we have a small budget for them to go and visit universities on their own to train lecturers. And this, it just brings me happiness to see their growth and their development. And I’m so looking forward to what’s coming next for them because they may move on to do postgraduate studies, some related to STACK, some not, but they have such a bright future ahead of them.

[00:30:22] David: It’s not just that they have such a bright future ahead of them, it’s the impact that they are achieving within this role, so recently out of university. We’ve, in the Kenyan context, been working for, as I say, myself personally, over 10 years, getting on for 15 years, getting graduates, trying to give them opportunities of internship.

I’ve never had a group grow so quickly, really in stature and in presence. We’ve had very good groups before. We’ve had people come through and go on to do amazing things. But them as a group of four, all of them growing the way they are, we’ve never really achieved this in the same way.

And that’s despite the virtual nature of the support. We already said how difficult that was at the start, but I feel that in some sense those difficulties have paid off in the long run because now the virtual nature is, I think, to their advantage.

[00:31:17] Santiago: Yes. And the quality of their contribution to the International Stack Conference, not the African one, the international one that is recognized as the main international conference was fantastic. I had so much positive feedback, because unfortunately they couldn’t be there in person, they attended remotely. I was there in person and people coming to me to say, wow, these guys are great. And Juma was there in person and Wastalas was there in person as well.

[00:31:49] David: And to me, what’s so incredible is, as with many of our things, this is all bootstrapped. We don’t have any specific funding to do this. But this is how we achieve real impact. We can’t achieve real impact in Kenya on the ground unless we have people on the ground in Kenya who can do this, who can understand the context and adapt and do things.

Our role has to be a support role in this. And this is what we’re seeing. We’re seeing that by putting in place the structures to just change the system a bit and have people coming through with real skills and provide that capacity building, we’re able to then get outcomes which just blow me away. This is the real impact that we can have going forward. And it’s what I’m excited about.

[00:32:35] Santiago: And I’m excited about looking at ways of scaling this as well. But that is a different story, which we don’t have time.

[00:32:43] David: Well, that’s a whole different story, and I should be clear that my impact, I wasn’t talking about the impact on the interns. Yes, that’s good, but I want to be clear, this is multipurpose. Yes, it’s good for them, but the real impact for me is what you’re seeing happening in some of the institutions they’re supporting, because that’s then reaching thousands of students. This is four interns, but the impact in terms of numbers, we’re looking at institutions. Even if you just take Masinde Muliro and what they’ve supported there, that’s thousands of students they’ve impacted.

[00:33:18] Santiago: Yep. We might continue discussing this because it’s so exciting and it’s such a wonderful experience. I loved it and I can’t wait to keep working with them and others in similar ways.

[00:33:31] David: Yeah, it is, and there are so many other stories which relate to this. There will be other episodes, I’m sure, which maybe dig into some of these impacts on the institutions themselves. We keep coming back to Masinde Muliro, but the way they’ve institutionalized it, I will just say one thing to finish this.

The thing that blew me away was this lecturer, who didn’t care about STACK, wasn’t interested, was approached and just told: ‘you just don’t have to do the continuous assessment, you just teach as normal’. And she did that for that semester. And the results she got in the exam on a course where normally most of the students failed, meant that she is now the biggest believer in this whatsoever, totally engaged. Because suddenly she said, you know, how could this happen? You know, this is an intervention which is meaning the teaching I’ve been doing without changing for years, suddenly, this makes all the difference. Oh, that, it’s one story, but that’s the one that really caught me.

[00:34:35] Santiago: It’s one of the many, one of the many.

[00:34:38] David: Yes.

[00:34:39] Santiago: More episodes to come. Thank you, David.

[00:34:42] David: Great, thank you.