216 – The Five Quiz Framework for Electronic Assessment

[00:00:07] Santiago: Hi, and welcome to the IDEMS podcast. I am Santiago Borio, an Impact Activation Fellow, and I’m here with David Stern, one of the founding directors of IDEMS. Hi David.

[00:00:17] David: Hi Santiago. We’re discussing the five quiz model today, aren’t we?

[00:00:22] Santiago: That’s the plan. It’s been an idea that, we’ve been discussing for several years now.

[00:00:29] David: Several being at least four to my recollection.

[00:00:33] Santiago: Yeah, shortly after I joined IDEMS, during the pandemic I think it was, we were staying together in your house, I had just moved to the UK, so we had a lot of time to discuss and have interesting conversations. I was, in some ways, considering pursuing a PhD and we were discussing potential areas of research and we came up together with this sort of pedagogy of quizzes that I think it’s quite interesting and could be quite impactful. I’m not sure if it’s a pedagogy, framework?

[00:01:16] David: Yeah, it’s a framework. It’s a really nice framework actually. I found it surprisingly, well, good frameworks are obvious, so I don’t think it’s anything which is revolutionary or new, but that’s actually what makes it a good framework, that it’s something which is just capturing what others already know.

[00:01:37] Santiago: Yes. And part of it came from some of the COVID responses that we did, that we had all sorts of different types of quizzes and materials for different audiences to help them deal with school closures and distance learning and so on. And we were discussing how do we deliver content, and what type of content and what type of feedback is useful at different times?

The conclusion for my potential PhD topic was more than this framework was layered feedback and when do we get feedback, for what purpose, how? But this is very much related to that.

[00:02:23] David: So we’re talking about a five quiz model, a five quiz framework if you want, and as you say, this dates back, so you are right, it is five years, it’s over five years now that we’ve been talking about this. We’ve not yet written it up, we now have plans to do so, but it’s something which we’ve used and tried out in a number of different contexts, and I found it really useful.

[00:02:47] Santiago: Hang on, you say we used it. I don’t think we managed to use the full framework in any particular context.

[00:02:56] David: Wait a second. No, we have never had any context in which the implementation of all five types of quizzes has been something we could achieve or necessary at this point. However, we have used the framework in many different contexts to identify what sort of quizzes are needed, what sort of assessment is needed, what its role is and how it will add value.

And that we have used, and this is why it’s not a mantra, it’s a framework. It’s helped us to understand what is it we are trying to achieve by having resources of a particular type.

[00:03:38] Santiago: Yes, and the framework was initially conceptualized for online courses, and they are STACK quizzes in its conception. But I think that it goes a lot further beyond that and it can be applied in many different areas. It doesn’t need to be an electronic assessment thing, it could be also how teachers work in the classroom.

[00:04:06] David: Well, this is where, as I say, we conceived this during COVID when people were being forced online and there was a recognition that we really needed to have good interactive assessment to be able to get the same sort of deep learning happening in this remote situation. That’s where this was conceived.

Let’s go through the framework quickly, because it’s not that complicated, do you wanna talk it through quickly?

[00:04:36] Santiago: I can do, yes. We have five types of quizzes. The first one is a prerequisites quiz. The name gives it away, is all you need to know in order to be able to tackle a particular topic.

The second one is instructional quiz.

[00:04:52] David: I like calling it content quiz.

[00:04:55] Santiago: You like calling it content quiz, I like calling it instructional, and we’ll talk about where the motivation came from, but essentially what it is, is presentation of content in an interactive way.

[00:05:14] David: It’s an active presentation of content, this is the point, as you articulate, for instruction. The idea is this is instructive, it is your learning resource.

[00:05:26] Santiago: The third quiz is a mastery quiz where you have questions that follow on from the content or instruction, and it’s where you actually practice. This framework is conceptualized mostly for mathematics and STEM subjects where mastery is hugely important, repetition is hugely important. But not just rote repetition, repetition with high quality feedback, and targeted feedback.

[00:06:00] David: The idea here very much, and it applies in a number of areas, particularly STEM, as you say, is that this is formative assessment which enables you to know when you have mastered these concepts, when you are on top of the material.

[00:06:17] Santiago: And again, we’ll go into more detail into it and tell some fantastic stories that we have of using this type of quiz. But once you’ve achieved a certain level or you are confident enough with the mastery, you move on to the fourth type of quiz, which is more of a summative assessment, it’s a testing quiz, it’s just to check formally that you have achieved the level that you need, it’s a bit less formative, a bit more summative assessment.

[00:06:48] David: Absolutely. This is a test, this is testing, where are you at this point in time right now? A higher stakes exam of some form, a test.

[00:07:00] Santiago: And I like to think about it as questions that are similar in style and structure to the mastery quiz, not exactly the same, but following the same sort of patterns.

[00:07:14] David: If you have mastered the content, you should do well on the test is the design that we would envisage.

[00:07:21] Santiago: And, finally, once you’ve completed the test, once you succeeded and got your result and you are happy with it and you sort of passed, there are students or learners that need that extra bit, need to either understand a bit more of the context or understand different applications of the concepts or want a bit more, going to a bit more depth.

And I say want, but there is an idea that it’s also a need, not just a want, for all, but it’s essentially an extension quiz, a context quiz, something that tells you a bit more, or hints as to what’s to come.

[00:08:17] David: It might hint at what’s to come, it might be putting it into context or application, making it more real or realistic in context, as you said, it could serve a whole set of different purposes. I don’t agree that it’s necessarily something to be done last. I think that good extension quizzes might be things that some people actually take on before they finish the mastery because they’re actually wanting the motivation, why do I care about this? What is it which makes this interesting or enjoyable? Some people might need that to be able to motivate themselves to do the mastery.

So the extension can serve different purposes, but what it should be, is it should be optional, and so different people might need different extensions. So you might have a set of different extension quizzes where people choose the things that interest them in a sense, and it could be, some are more challenging, for good students who really want to be intellectually challenged. For others, it is about making it real and making it more realistic, which other people might not want to put the time onto.

So extension quizzes for me, they’re taking the topic and they’re adding something beyond what is needed for everyone, but it’s what is needed for individuals to engage well with the topic.

[00:09:40] Santiago: Well, I always, in my teaching, I always included extension materials, all my worksheets included optional questions and so on. But really the idea of separating this, for me, came from an intervention that we did during COVID, during school closures, for transitioning from GCSE to A Level, where a couple of teachers came to us and said our students who finished GCSE are not doing the exam, the exam got cancelled, we are doing teacher grades, and we really need to make sure that when they start A Level they have a solid understanding of the algebra that they would normally get through the revision process for the exam, which was not gonna happen because the exam was cancelled.

And what we did in collaboration with these teachers, who by the way are members of SAMI, the charity that we work very close to and support. They presented the different topics that they wanted to consolidate the understanding with five, six questions with good feedback, sort of mastery style.

But then they had four different types of activities, one was geometric investigation using the dynamic geometry package called GeoGebra, one was a modelling activity related to the topic, one was purely problem solving, and another one was, I believe, an activity very much inspired by activities from the website NRICH. So four different types of extension work that were optional, but a lot of them really engaged and provided very interesting solutions.

[00:11:53] David: And the point was, within that, with all this really exciting content, actually both teachers and students found that in this remote context, it was the STACK questions that really added value to the students’ learning. Both students and teachers single the STACK work out.

And this was part of what really transformed our thinking into, well, this is really important because all of these other interventions we’ve been putting lots of time and effort into, but, actually the STACK work was enabling student learning in a way that was so appreciated.

[00:12:33] Santiago: Yes. And even though a lot of the tasks were STACKified, the extension tasks were also STACKified, I believe that the STACK questions that you refer to were the more mastery style questions.

[00:12:49] David: Yeah.

But let’s come back now to these sort of five topics. Let’s start with the prerequisites. This is something where a lot of the work I’ve been involved in within African contexts in particular, there is often this issue about students coming from different places, having different background knowledge, this is a common problem everywhere. But this is often debilitating when you don’t have the human resource to support those students who might be struggling.

And the idea that actually this could be supported through electronic assessment is not new. I mean, I know groups who have been doing this in various forms for years, using multiple choice as ways to diagnose and then offering human support when things are identified. So this is not new as a concept, but I think actually thinking about this being systematic for each topic, that isn’t something I’ve seen happen much.

[00:13:55] Santiago: Baseline assessments are essentially prerequisite quizzes in some ways. But what this type of quiz with quality feedback allows you to do, and the ability to do it multiple times and potentially get specific support, get linked through the feedback to different areas or materials, what this does is it can raise the baseline so that we’d have a more level playing field before we start instructing students.

[00:14:31] David: And let’s be clear, we don’t believe that in every context you need to do all the quizzes. In fact, in some of the work that we’ve done recently in Caltech, there was a particular course where the only piece they wanted was this prerequisite to prepare people so that they could come in ready to do that week’s activities.

[00:14:53] Santiago: And we have informal feedback questionnaires, at the end of every quiz we asked how useful do you think the previous quiz was to help you tackle the worksheets that the students were getting, and some students were saying, not at all, it was too easy, and some other students were saying it’s just wonderful, I went to the lecture and I managed to follow everything much better.

Because the way we designed the questions were not just consolidating the algebraic understanding that they needed for the materials that were going to be presented, but also were about following the processes, following the problem solving that they were going to be presented formally in an informal way without really having to know the concepts, so that when they get to the concepts, when they get to the theorem and its application, they say, oh, that’s exactly what we did in the prerequisite, it’s just a bit more formal, a bit more algebraic, a bit more abstract.

[00:16:03] David: And this is the thing, mathematics is often seen as being this linear progression of mathematical ideas where if you’ve missed something you’re then lost forevermore. Whereas, actually, if you can articulate what the prerequisites are, those gaps are often quite easy to fill at that point in time for a particular topic.

And so, the prerequisite quiz may not be something that everyone does. It may be just be something that people who struggle with the content go back and do, and then they say, oh, I was missing this. Or it may be something that people do exactly as you are articulating, and it means that they’re prepared and they’re ready for it.

So I love this idea and I love this as a component, which I feel we don’t often put enough weight on. Because you can’t get this right for everyone if you’re doing this as sort of a lecture or individual teaching, or even in a classroom context because everybody needs something different at this level to bring them to that same point, to be ready for the topic.

[00:17:05] Santiago: Yes. I love to think about it as an equaliser.

[00:17:09] David: Yes.

[00:17:10] Santiago: You know, there’s a lot of talk about inclusive education and what that means, and I think this is a clear example of what inclusive education is, it just brings everyone together to the level that is required in order to be able to tackle the challenges to come.

[00:17:30] David: And if you find it easy, that’s great, that means you are ready to get on with the actual resources with the actual topic.

Let’s get to the topic and the content, because this is the whole point. If you are introducing a new topic, then there’s going to be something which you are expecting the learners not to know at this point, and that you are gonna be introducing something.

[00:17:51] Santiago: And this is perhaps what happens normally in the classroom. In the classroom, I explain a concept and I normally give a few examples on that, and I have discussions about it. Textbooks do the same thing, traditional textbooks tend to have a few paragraphs on a topic and then give you examples and then exercises.

However, there was this paper published by George Kinnear, I think it was, which talked about bringing the textbook into a quiz. And we were, I think, both really inspired by that paper.

[00:18:32] David: Can I just come in on this? Because the key point is whether you are at school and you are in a classroom doing this, or whether you are lecturing and you are actually presenting this, you know, it’s the same basic principle, you are presenting new content in some form. And what the paper articulated was, well, there are ways in which, actually, done as active learning, this can add value. And I was very lucky, this resonated for me because when I was an undergraduate at Warwick many years ago, I was part of a research study which investigated using worksheets and workbooks as opposed to lectures to introduce new topics, which is exactly what this is.

So I’d already experienced that and I’d loved those workbooks. And it’s actually, there’s a whole lot of educational research now on using active learning and group learning to be able to do these things. And this is what it is. It’s about turning that introduction of new material into a process which is active rather than passive consumption.

[00:19:40] Santiago: And what is I think worth pointing out is that we came up, or we started discussing about this framework during the pandemic, and a lot of the solutions that were presented were to record videos for the students to watch. I don’t know about evidence, but my belief is that watching a video in mathematics is completely pointless, you just watch it and forget it.

It’s passive learning, passive learning is short term, it doesn’t help you consolidate any understanding, it’s the active learning that is most impactful.

[00:20:23] David: There is evidence around this and there’s lots of work which has been done on this, but as you say, it’s a very intuitive aspect from personal experience. Watching videos can be nice, it can be fun, but does it lead to deep retention? Very rarely. And I should be clear, I don’t believe that the content quiz or this instructional quiz is where the deep learning happens anyway, but it is a really nice way to introduce new concepts.

I can talk to a very specific case. In my first year undergraduate doing a worksheet where I realised, oh yes, I’m a mathematician, I like this. In the worksheet there was this question about continuity and finding a nowhere continuous function. And suddenly this is where I realised, yes, this relates to infinity, this relates to all sorts of, I understand how these concepts come together, number systems, you gotta bring it all together to be able to answer these questions, and I loved it.

My passion for mathematics really grew out of that active learning. If someone had just told me the answers, it would’ve been nowhere near as much fun as actually just sitting on that problem thinking, well, how could it be nowhere continuous, until it actually dawns on you what it means and what the definitions tell you.

[00:21:51] Santiago: For the non mathmatic audience, a function is not continuous if there is a break, if there is a jump, if the graph has a jump. So how can you have a function that at any point on the function, there will always be a jump? There’s no point for which there isn’t a jump. And the solution is beautifully simple, but it’s one of those that it can take a while to get, and once you get it, it’s so obvious. And I’m not gonna give out the solution.

[00:22:26] David: Not on this episode anyway. But this is an example of where active learning is so much more powerful than just being passively presented with this and with an answer to this.

[00:22:37] Santiago: What might this look like? So I want to give an example, secondary level, there’s a process that you have to learn, it’s important, it has its uses, it’s called completing the square. Some people will be familiar by it, some people will be terrified by it, it might bring nightmares back because of the way it’s normally taught.

It’s taught in a very algorithmic way. Just follow the process, turn the handle, and you get the solution. And the way I implemented this question for the bridging course was a little paragraph explaining what the objective was, an example and a similar situation with fill in the blanks.

And the beautiful thing about that question is that it identified, you know, fill in the blanks, it wasn’t just fill in two boxes, it was fill in the blanks for pretty much the whole exercise, the whole thing that the students had to replicate. It is a bit turning the handle except that at any point, any common misconception, any typical mistake would lead to follow through marks throughout the question. And it would say, okay, your answer here is incorrect, this is your mistake, but everything below this would have been correct if your answer was correct.

[00:24:04] David: And the reason that’s so important with a topic like that is it reinforces success rather than failure. Because, actually, you’ve got this, you can do almost everything, you’ve just gotta remember this bit, you’ve just gotta actually understand this little step. And once you’ve understood that, then everything will follow and it’ll all make sense.

And so it reinforces the fact that you’ve almost got this. These are the things that you need to just pay attention to as opposed to, if you are just getting the answer and you get it wrong and then you think, oh, where did I go wrong? Why? I’m just useless at this, it’s no good. And so that’s an example where it can then, presented in this, you know, what they talked about in the paper that we referred to before is these faded worked examples where you’d start with something really detailed like that, and eventually in something like your mastery or your testing, you just have the, you know, put the answer in.

[00:25:00] Santiago: Yes, exactly. That same question of completing the square in a mastery context, I would put it as complete the square of this quadratic, and it would have the same sort of checks for common misconceptions, and it would provide the feedback, oh, this is probably where you went wrong. But it wouldn’t have all the little step by step fill in the blanks, all the building blocks of the solution.

[00:25:27] David: Yeah, the scaffolding of it, which is so valuable when you are first being introduced to it and so frustrating if you have to do it time and time again. But this is the point, the content quiz, that instructional quiz, we wouldn’t think of these generally as being randomised. You tend to think of these as being you do it once and then you come back and you look at it, it sort of now becomes your textbook, but you’ve written your own solutions to it, and then you can go back and correct them.

[00:25:55] Santiago: And if you got it wrong, you got a common misconception that is written out with why that is wrong, textbooks don’t tend to have that.

[00:26:04] David: Exactly. And that then leads us to the mastery because once you’ve been introduced to the topic, exactly as you’ve articulated, you need to be able to practice. In many different contexts, the practice can take different forms, it can be very similar to what you’ve been taught, or it can be using the ideas you’ve been taught to do something else.

But the point is, mastery is where you want to be able to be given multiple versions, multiple variants, where maybe the first time you do it, you are sort of saying, wait a second, I need to go back to the content and check, I hadn’t understood it as well as I thought, but then, once you know it, you know it.

[00:26:46] Santiago: And the mastery, together with the test quiz, are the type of quizzes that we applied when we first started using STACK in Kenya at Maseno University. There was this idea that students could do the same quiz multiple times, and by the same quiz, what I mean is you get the same questions but with different parameters, randomised variant of the question.

[00:27:17] David: Sorry, let me be clear on this, there are different forms of randomisation, and the same quiz could have multiple different types of randomisation. So when you talk about different parameters, people might just think this is the numbers in an equation have changed, but it could be more interesting things, it could even be your context has changed. The randomisation gives you the same quiz but with different variants.

[00:27:41] Santiago: Okay, thank you for clarifying that, you are perfectly right. And we had it in such a way that students had to score a given percentage depending on the course, one lecturer preferred 70%, another one preferred 80% in order to progress to the test quiz. But what we found was that certain students weren’t satisfied achieving the minimum requirement to move on to the next activity. Certain students wanted to get a hundred percent. Certain students wanted to really master the content, and mastering the content meant achieving a hundred percent in the mastery quiz.

I mean, this is potentially a good thing or a bad thing, but we had one student attempting a quiz 17 times until they got a hundred percent. It shows the dedication, it could show maybe a bit of, I’m not a psychiatrist or a psychologist, but obsessive behavior as well.

[00:28:39] David: I remember looking at that case, this was a student in Kakamega, at Masinde Muliro University. And actually when we looked at it, it wasn’t just that they did it 17 times one week, they’d been doing it eight times or more almost every week. Within that class, they were an outlier just in terms of the fact that they were really engaged and wanted to master the content to its fullest, even when they’ve got some questions right, they wanted to cover the variants to check that they could get it right. It was very interesting looking at that data.

[00:29:14] Santiago: It was indeed. But what I say to my students all the time when I’m teaching is, you know, there’s only one way to learn maths it’s by doing maths, you have to do it, you have to practice. And again, feedback, targeted feedback, high quality feedback, without good feedback, the mastery quizzes, they’re not as useful. If you do the mastery quiz and you don’t get sound advice on how to improve, you are not going to improve.

[00:29:46] David: Yeah, absolutely. And what’s interesting here is when MOOCs first came out, these are Massive Open Online Courses, I guess I’m getting old, it must be almost 20 years ago now, but I still remember when they first came out, they weren’t about the videos, the content, they were about the assessment.

The hypothesis was if you have a hundred thousand students doing the same assessment, then you are able to put the time in to find really rare misconceptions, identify them and give tailored feedback to them better than you could do as a single lecturer even, or as a single tutor. If this is built over time by a team of people who are putting effort in to make sure really good feedback is there. The idea of getting really exceptional feedback you could for mastery outperform a private tutor.

[00:30:42] Santiago: Yep, certainly. And with the numbers that we were working with in Kenya, it doesn’t necessarily get to the order of magnitude of some of the MOOCs that you’re referring to, but it’s a constant cycle of iterations, looking at where students were wrong and improving the questions so that they include better feedback. And it’s almost a virtuous cycle, it’s hugely time consuming.

[00:31:15] David: But this is what, if we can get these open question banks, common resources used across institutions, this is exactly where that then becomes a shared effort. And that’s sort of what we’re working towards at the moment, which is very exciting. So the mastery quizzes, these are really, this is the heart of where we see the learning happening and it’s really when we started using STACK to support learning, it was the mastery quizzes that we cared about, this formative assessment.

[00:31:45] Santiago: And we also cared about the test quizzes. And the reason that we cared about the test quizzes was that, previously there were lecturers in Kenyan universities that we were supporting that had perhaps over a thousand students and no teacher assistants, no support, it was just one lecturer. And the continuous assessment was two class tests, one in the middle of the term, and one shortly before the final exam.

And because of the teaching load and the number of courses and the number of students and all the other responsibilities that these lecturers had, they really couldn’t mark it in time to give something positive, give positive feedback or valuable feedback to the students. And I even had a conversation with one of the lecturers that we supported who told me, I actually don’t have the time to mark all the half term assessment and give them back before the final exam.

And so replacing, and it took quite a lot of work, quite a lot of convincing to certain people to run this experiment, but replacing the continuous assessment in the way it was done with two class tests that really didn’t provide any sort of feedback, it was just summative, completely summative, it didn’t give any indication to the students as to where they were, with weekly assessments on the topics that had just been presented and that were similar in structure and style to the mastery quizzes, that you could only get to if you scored a certain percentage in the mastery quiz, it just changed hugely how continuous assessment worked in those contexts.

[00:33:52] David: And it led to learning outcomes, which we can talk about and we have talked about in previous episodes. But I think the thing which I want to highlight here is that the mastery quizzes are great for the student learning. But it is also important to remember that assessment is partly about differentiation. It is, whether we like it or not, it is about identifying students and differentiating them. That’s part of what summative assessment does.

And so this combination of the mastery and the summative is extremely powerful to be able to do a bit of both and to be able to enable students to actually judge not just where they are in terms of their mastery, but how are they doing at a given point in time under test style, well, in a test.

[00:34:46] Santiago: And there are concerns about how online assessments are used for continuous assessment or formal summative assessment. There’s cheating that was evident. There was this informal, well, semi-formal study that was done.

[00:35:02] David: I think Juma published it.

[00:35:04] Santiago: Oh, it was a formal study then, where, it was amazing, we had some fantastic outliers where the continuous assessment was wonderfully done and the final exam was awfully done, and the questions in the final exam were fairly similar to the continuous assessment. Those outliers clearly indicate someone else did this for you.

[00:35:27] David: And more than that, the qualitative study that then accompanied this indicated that there was a business emerging where people were paying others to do their assessments, their continuous assessments for them. This is something which is happening in all environments in different ways.

What was so interesting is that actually, within these contexts, that was normalised. But it didn’t make any sense anymore, it wasn’t cost effective because these were low stakes exams. You know, when you have 10 continuous assessment, when you have two, then it’s different, but when you have 10 it’s getting expensive. The students were actually joking about this, the fact that those who paid really didn’t get a good deal because not only did they have to pay for it, but then they failed anyway. And the students had a great laugh about that. Of course, it was the ones who didn’t fail, who actually did the work, who were laughing. But this is how it should be.

[00:36:29] Santiago: Yeah, it is.

We talked enough about the fifth quiz, the extension quiz. I just wanted to perhaps wrap up, considering the time, with a summary of this framework. You know, we never managed to study it properly. We discussed it with academics, with teachers, with educators in general, and we only got positive responses.

However, it is quite time consuming to produce resources like this, and it’s not something that happens from one day to the next. And that’s perhaps why five years on, four or five years on, we still haven’t managed to study it. However, I heard recently that one of our academic colleagues may be interested in studying it, and it’s wonderful that it’s perhaps gaining a bit of momentum.

And we are at a point where we created questions that are suitable for each one of the types of quizzes, we’re almost at a position where we could, for certain courses, just create the five quiz framework.

[00:37:44] David: Not yet. I mean, when we take pretty much any implementation, it’s too much work for that particular implementation and it’s not effective because there are components which are already covered by other elements of the system. In the Kenyan context, the lectures cover the content, and so really what you needed it was the mastery and the testing. That’s where there was the gap. Yes, of course, extension and prerequisites would’ve been nice, but it wasn’t necessary. What was needed was the mastery and the testing.

When we go to the high resource environments like Caltech, it was the prerequisites that came as filling the gap. They have all the human resource that you’d need, but the prerequisites were a way of actually making something low stakes, where different students could spend different amount of time as they need on it, and it’s something which is low stakes, which was useful in that context, and didn’t really exist within the resources they already had.

They already had good content. They already had good ways of doing mastery and testing, and they have human feedback for that. What added value for them was this prerequisite component in that particular course. In other courses, it’s other things.

But this is what I think, to me, the power of the framework now is to help have those discussions about in which way do you want to use electronic assessment as part of your course, your teaching.

Yes. My dream is still to have full textbooks or full implementations of five quiz models where all of these are tied together into a learning process where this becomes your textbook and you as an educator now you can choose the pieces you want to use in your course. That would be the dream. But getting to that is a lot more work than supporting people to achieve what they need in their context at the moment, which is where we’re at.

[00:39:43] Santiago: Good, I think that is a lovely place to finish.

[00:39:46] David: Well, thanks for the discussion, I’ve enjoyed this and you know, five years after actually coming up with the ideas, it’s nice to actually have a discussion about it to make it sort of public in some form. So this good.

[00:39:59] Santiago: Thank you very much, David.