Unlock Efficiency with the Blank Bill Book for Research and Development

Blank bill book for Research and Development

good afternoon and thanks for the warm welcome see is this um audible um so as Curtis uh mentioned um it's an exciting place to be I think it's a delight to be here in New Orleans and uh I'm grateful for the invitation to share some of the things that get me all fired up today and also for the wonderful hospitality that we've all been able to enjoy is the sound okay okay so as Curtis mentioned I have several affiliations um but across those different positions I work on the same kinds of things I look at curriculum development teacher professional development and the supportive roles that technology can play in each of these or in putting these things together and um this means that I spend a lot of time designing which um is something that I really enjoy doing and that ranges from designing Cutting Edge uh um Technologies for use in modern classrooms to very simple low to the ground use of basic Technologies like text messaging to support rural villager workers uh or Preschool teachers like the one that you see uh on this picture here um my passion for design has a long history I come from a family of of Educators and and uh I think one of the things that as part of my upbringing and certainly part of my professional career has been a really Keen interest in not just becoming exposed to interesting ideas or cool widgets and tools but to finding ways to really make meaningful change on the ground and it's that kind of work that I would like to share with you today so the uh conference flyer says it we are here to engage in this multidisciplinary forum and to exchange information and during this session the kind of information I would like to exchange with you and I do hope we have time for exchange at the end of this session that we can have some conversation um relates especially to the idea of integrating research and development uh in the applications concerning e e-learning and designing and realizing and implementing e-learning so this means looking at the role of research in design for design and also through design maybe it takes it's worth taking a moment though just just Define what I mean by e-learning um and who are the designers who actually create e-learning there are many people who participate in design even though not everybody who I would view as a designer would necessarily view themselves as a designer because they wear often multiple hats but in my mind this includes software developers or technology developers it includes researchers but it also includes teachers to a a large extent and when I talk about design I'm not just talking about making the stuff of course the tools are interesting and and and important um but but it's more than that um and designing uh for change on the ground means uh not only creating the the tools that we use but the processes and the mechanisms upon which they will be built that are uh where a vision for use is founded on a realistic and well articulated um theory of change the kinds of things that we use might include ebooks or readers they might include software they could include learning environments which are fully or maybe just partially integrated with technology they could also include Pathways for example to teach your professional development here too technology may play a supportive role but it's not the centerpiece always there are many ways um and many models for Designing learning and instruction but um there aren't that many that actually explicitly focus on integration of research but this is important I think and that's the thesis of my message today it's it's important because integration of research into development yields robust and well-informed Designs it's increasingly important given our current stage of accountability um our society now is growing increasingly um and I think justifiably also concerned with evidence informed practices research-based design or at least research in formed design can speak to that so today what I'd like to discuss are two uh some thoughts about two particular approaches to design that really work hard uh to integrate the processes of R&D more so than others one is called research-based design or R RBD and the other one is called design-based research or dbr they both research-based design and design-based research both share the same phases of analysis design and evaluation and they're integrated and and iterative and they there are many commonalities across the two but their outputs are quite different so um this also implies that there are different relationships with research throughout the entire process in research-based design the focus is purely on the iterative development of solutions to problems that can be uh implemented in practice in contrast to that educational design research has twin goals in addition to the pursuit of Sol solutions that can address real problems educational design research is also aiming explicitly at deriving theoretical understanding that can inform the work of the others of other people outside the uh developments or uh context or the target setting where um the design is going to be used so this means that there are some some overlaps but also some differences in what uh the final outputs are of each approach so in research paced design um there are many different kinds of things that can come out educational programs processes products even policies to facilitate the use of programs processes or products in contrast uh or in addition I would actually say design based research aims to do this but at the same time has that additional goal of wanting to develop theoretical understanding that can be informative for the work of others so this is understanding that can be used for different purposes right to to describe explain predict or even manipulate phenomena in the real world so uh that could includes um teaching learning motivation self-regulation all of these um relat to the process or the processes or the products of design um both of these approaches feature research and uh quite prominently and both of these approaches feature integrated research integrated R&D but because they're striving towards different outcomes the processes tend to look a little bit uh not exactly the same the the the relationship with research is is different between uh the one RBD and dbr and I think it's important to untie these to be able to choose what am I working on and uh and to to act ingly um it's also a lot easier to integrate research based uh sorry Research into design or into uh design based research it's easier to say that we're going to do that than it actually is to do that anyone who's worked in a development team knows that the time scales uh of developers and the time scales of researchers are often poorly aligned um and there's also a lack of of alignment between research and development activities due to maybe denial so we get results from an evaluation back and we just don't want to believe them it happens sometimes it's due to Omission maybe there's no time no budget no Personnel no expertise to do the thoughtful considered steps that we would ideally do if we had uh other options available sometimes research and design um is not integrated in ways that that that you see with there are maybe some front-end analysis that has has taken place or if it did though it wasn't uh of a high enough quality to really uh take into consideration during design um or maybe some of the evaluations that are conducted could better be labeled as quasi evaluations because they're really more to justify what was done than they are to actually inform decision-making or ways forward in the remainder of this session what I would like to do is share with you some considerations uh on balancing the relationship which is like I said much easier said than done U between research and development and these two approaches and so I'd like to look at RBD and dbr and think about um which might be the cart and which might be the horse given certain situations um to talk a little bit more about ramifications of the similarities and differences that I already mentioned and then to think about the affordances limitations and synergies of these different processes so which is horse and which is cart let's start with what's what um in my experience it's kind of like Beauty it's in the eyee of the beholder so um researchers tend to think that it's all about uh the research and practitioners and developers tend to think it's all about the tools or the designs and um that's sensible it's understandable it also makes things a little bit complicated when you're working in teams where goals and values and uh um ways to move forward need to be negotiated and um so there are certain consequences of these differ kinds of views in teams and let's take a little look at that some of the consequences can be positive some may be less so um from a strategic standpoint being able to separate out horse and cart can actually work out pretty well I have here some examples on the screen of some projects that I admire very much um where different groups have been able to pull the cart using funding from different um sources so so for example the seeds of science roots of reading uh curriculum which is widely used curriculum in 50 states here in the US for elementary school science and literacy that are integrated those developers decided that they needed uh to do something to um invigorate their their product and their also their design processes and they turned to research not only for funding to get the job done but also to improve the quality of their internal process and so they have NSF grants and grants from uh the Bill and Melinda Gates Foundation to um refine their already outstanding products so the research here is trailing but it's serving a wonderful purpose in in the development work um on the other hand you've got products like a very famous example from some decades ago was the Jasper Woodberry series uh for learning mathematics video based um vignettes that that help students connect to uh mathematics learning in ways that are are very uh realistic and relevant to kids in uh secondary schools in this particular example was more research um researchers and research money driving what was actually done and that also had a lot of implications for the way the the projects uh panned out on the other hand you see people who want to engage in integr research and design looking for uh Roots through practical uh sources of financing and I'm not not saying that financing is the only thing that matters here of course but uh it does have a a big say in what we are actually able to do so um for example uh the the jump in Project which is a project in the Netherlands is a a project where um uh researchers in the fields of of health and education teamed up and they had money from um a local city to uh develop interventions and Implement them uh and it was the the funding for uh Public Funding for um health and fed of uh children that was able to feed their work so there's a couple of examples here that there are also those are some implications and some consequences of looking at strategic alignment or or um between research and development goals money being W of course other resources the human resources the temporal resources the material resources they matter as well um another consequence for thinking about the integration of research and design um is the process I love this model which took me a while to learn to embrace it because it it sort of blew my mind when I first saw it but I I've begun to to get a sense of it um it's by um some Danish colleagues um and uh what you do when you look at the model is look kind of in the Middle where it says problem uh and it shows the cycles of um of research and development starting from the same Point of Departure the problem and then working in parallel at this like this so if you you could imagine the wheels turning would go like this and uh I think they did a brilliant job of conceptualizing how these processes can be parallel and yet separate and on the one hand I like the fact that they make things uh so transparent on the other hand in reading the book chapter in which this model was presented they were very brave I think and said that this is the ideal model uh in real life we've never done a project where it worked like this we've done projects where it went like this and like this and in all different kinds of ways but the sequence is never as pretty as it is when we draw pictures of it we also know that to be true with our research reports and many other things um that is the way of it but I think it's uh important to note that um there are potential ways to align things and if it's not done carefully as these people have indicated it can get out of syn very easily another consequence of considering uh research or development one to be the horse and one to be the cart could be uh related to the outcomes um there might be if there are tensions uh in in what are the main goals and what are the supporting goals then that leads to different ideas about uh you know when pressures are tight when when money is tight or time is tight or a production schedule has to be adhere to and we have to um cut back well maybe cutting back on Research rigor would be the priority of some in a team and maybe cutting back on uh development uh detail would be a priority for some others and so um there there are some very in some ways having to innovate can certainly year to uh very um productive tensions my mother used to call that hybrid Vigor where you have different ideas coming together to really bring forth something new and inspiring and at the same time it could be quite draining and uh painful if uh not resolved within a a reasonable time um another important consideration in terms of what actually influences the outcomes if if research and development are not integrated closely um has to do with the different reward systems of the individuals that could be participating in design teams so in most of the design projects that I've been involved in it's not just uh uh people from One domain from like Academia or industry or school practice but usually there's there's Partners from different organizations and again that has many affordances at the same time different organizations have their own reward system so where I work at my multiple institutions they're pretty big on Journal articles and scientific output and you know as a preschool teacher I'm pretty big on making change on the ground and I'm not saying my University doesn't appreciate that they certainly do but uh it weigh in differently for that organization as opposed to for example the teacher who feels accountable for making sure that her students do well on the national exams or the trainer who feels accountable for making sure that um his his Learners are able to implement new technologies in in ways that make sense for the business and so on so there are varying expectations about outcomes this can lead to varying priorities in pursuing those outcomes and um it works generally speaking I think it works better when uh those have been negotiated early on and explicitly within a team and that can happen when research and development are well integrated so I think that um uh that to conclude my argument about why I think research and development need to be integrated I think that the horse and cart mindset can be very helpful for um uh from a strategic standpoint especially for obtaining funding and uh in the short term that's lovely but in the long term separated processes can be informative but maybe insufficient for building Theory or creating robust design and I think that the focus on different kinds of goals which is in part um uh brought in made clear by the different reward systems of different parties involved um can risk exacerbating the tensions that already exist so I think that integrated research and development can yield greater benefits and I'm hoping that you're here at least in part because you think that's not a completely crazy idea um some of the potential benefits I see to be on the flip side strategically um by integrating R&D uh we demonstrate added value um for researchers and for developers uh and to other practitioners I think that um robust design and research processes um can help sensitize researchers for example to the realities of developers or of practitioners um also I think that when you've got research and development practices that are integrated especially when they involve uh practitioners on design and development teams uh that can help to sensitize practitioners um in a way that inculcates ownership and um that can be uh facilitating for implementation and sustaining Innovation consequences of uh integrated R&D can uh for for the for the final product I think that um if we have integrated processes especially with members of a design team who understand accept and live the day-to-day of the target implementation context we can develop designs uh and um also uh create research findings that are ecologically valid and um I think that this stands to inrease the increase the quality of the designs themselves and also the research for having an uh potential impact on practice so um what might an integrated process look like uh there are many ways to roam of course and uh what I'd like to do now is present a model which uh my good friend and colleague Tom Reeves and I cooked up together um uh as the backbone of a book that we recently published this model is based on scientific insights um from the fields of curriculum and instruction and uh also software design and a couple of others um and it also aims to explicitly blend both analytic and creative approaches to design the starting point for the model is problems in practice like I said earlier both research based design and design-based research are fundamentally concerned with um creating productive change on the ground and um so that's where we start with our model um there are a little bit of differences though in research based design uh the main audience for uh for the work are are um users clients U this could include Educators or Learners um but it's mostly those that are within the Innovation System and in contrast to that design-based research has the additional audience of um practitioners and um researchers that are tend to be further Outside The Innovation System um in both cases um a first step tends to be the identification and Analysis of problems in practice upon which to focus the activity so here an attempt is made to understand the problem its root causes um the concerns of those people who are involved it also means beginning to look at the room for maneuverability what I like to think of as the jurisdiction of change you know what's within the purview of this project and what is just too big for us to tackle and um beginning to look at the affordances and constraints within a situation that might lend themselves well uh to being used or mitigated during design so at the same time there's a kind of an exploration more open-ended that takes place where we go looking for how other people have tackled this kind of thing in other situations and with what kinds of results in research-based design this process is largely informal um and while scientific literature may be used um it's it's it's tends to be more limited than in design-based research but in sound research-based design there will be some evidence informed uh framing of the problem and uh uh that will be used for example to gather data uh to understand the problem better in design based research where you have an explicit goal of also generating relevant usable theore iCal understanding that other people can benefit from the way one looks at the analysis and exploration phase is different um not necessarily fundamentally different but I would say that uh the the rigor and the detail that one um uh seeks in order to really understand the problem is more intense and um this could range um from all the different kinds of of insights one needs to begin to understand a problem and develop Solutions um and this kind of understanding that we seek in this phase can be useful for design but like I said at each stage we're also looking at generating usable knowledge so doing a study to understand kindergarten teachers attitudes towards technology use for early literacy is something that can inform design and at the same time be valuable in its own right if it's conducted well enough so um the Frameworks that are used to to uh study um the problem situation and the context tend to be more robust heavily based on literature um and um that's a little bit different from research-based design a second typically a second any anyway uh phase in the in both processes is uh design and construction and this is where uh we have ideation uh so beginning to think about possibilities generating ideas weighing off Alternatives checking which ones will actually make sense thinking through if they are feasible for the given um problem and context and uh the development of prototypes which tend to mature over time in research based design this process is largely determined by um existing organizational practices what have the people in this organization done before which is a very Fine Place to Start I'm not knocking that um and what about the what expertise is available on a particular design team so what is the repertoire of the designers that are involved those are two things that play a determinant role in uh what the design process actually looks like whereas in design-based research um there's an explicit focus on looking out uh at the big wide world and uh inventory and what's already known and how are we going to make a contribution that um has an added value uh above and beyond what's already known and so this has to do with um looking through literature that can help U frame up a design that sound so if you take for example technology for Teacher professional development there's an awful lot that we already know and there's an awful lot that we can build on there's there's an awful lot in terms of Dos and probably way lots more in terms of don'ts and I'm not saying that research-based design would ignore those but design-based research will take those and uh use them in a much more more systematic and thorough way typically um so um we use design principles for um in research-based design or in design-based research we might be using existing design principles for example um principles on how to shape authentic uh e-learning tasks uh in higher education um this could be the result of a design based research study and it could certainly feed into future efforts on research-based design or design-based research um another step another core process in design-based research and research-based design has to do with when uh the ideas and prototypes of interventions begin to take shape they're tested and we reflect upon them and this process typically feeds back into design and the iterations uh and cycles of of development are common to both processes in research-based design uh the evaluation and reflection phase tends to be shorter and the emphasis is primarily on formative evaluation occasionally also on what I mentioned earlier the Quasi evaluation where we're really just doing the evaluation to justify that the money was spent or the time was spent or so on but um in design-based research the evaluation tends to be quite rigorous and uh a bit more formal um it also is based on some uh very sound theoretical grounding and the finding of the work are not only relevant to design but also useful for other people tackling similar challenges and similar kinds of settings as mentioned previously a key difference between design based research and research-based design um has to do with the outcomes that they aim to produce there's some overlap but also some differences so in research based design uh we're primarily aiming towards an intervention which as you see here um may start off as an idea as a first prototype and gradually through different passes through this uh model and there's many ways to pass through it um it will mature over time and that is also true of design-based research but at the same time this is a process that uh aims explicitly for developing theoretical understanding simultaneously so um what does this look like in the real world there are many examples that I would have loved to share today and I had to pick and so I hope you like the choices and if not well maybe we'll have time to discuss other ones um later on but um um I think it's helpful to take two examples and begin to think about what a lovely model that has nice square boxes actually looks like in the messy World um so one example I'd like to give you uh is a a research-based design example and that has to do with um creating communities uh uh to advance workplace learning and the design based research example I would like to share with you um is uh got to do with um was the development of an electronic performance support system which is actually a rather old study but I mention it today uh because uh the the work is still relevant to the topic but also because I'm excited it's one of those old projects that had its own life we thought it was over and is now getting reincarnated uh in a different context with different users and maybe we'll have a chance to talk about that so for each of these I'd like to just briefly mention the nature of the project what the core processes of analysis design and evaluation actually look like in these projects and what the main outcomes were so um the Technologies for workplace learning project had the main goal of working to improve the alignment of workplace learning um with uh U the school setting in iary tertiary learning we we worked with six teams of um intern and each team had an intern mentors so people from the workplace uh and they're in different sectors so the Ed people from the educational sector that would be teachers from the workplace uh from the e economics sector we had small businesses and bankers and we also had a um a technique I don't know if that's the right translation uh but a technical sector um which had people from from um businesses that do Woodworking and another one that does um Phillips you know Phillips I don't know how to describe Philips a giant company produces all kinds of Technologies um and we had so we had mentors from intern mentors from the workplace and uh also the school-based mentors working together in teams tackling the problem of improving uh the connection or if you want to go half empty uh dealing with the gap between the school-based learning and the workplace learning and what we did with these teams is um uh ask them to understand their own situation uh look at root causes of the misalignment between workplace learning and school-based learning and uh begin to develop solutions to um not the entire problem because that was beyond the scope of our project but to some of the big causes that they thought were responsible for the project problem and uh begin to chip away at those one at a time so basically we had six Innovation teams from the sectors of Education technique and um um business and um what we did was we facilitated each of these teams in the creation of their Innovations through Technology based materials and uh we also gave them resources and and uh workshops and coaching so across the process of developing our research-based Innovation uh and there's two layers to the onion here doing this for our project was a research-based Innovation and at the same time we uh were encouraging the participants to develop their own research-based Innovation um one of the first things that popped up as a big red flag and it was predictable but um the extent uh of it was still uh shocking for a lot of participants was the fact that um there's different agendas different people in the in these te these teams were small there were four to six people in a team so two or three people from the workplace and two or three people from the school you would think that that's fairly Nimble but it wasn't um and uh there were there were differences between you know were you from school or were you from the workplace but the bigger differences were of course um across those two uh contexts and the the values the the the reward systems and the priorities in each group um it was identified early on that they were uh quite difficult to negotiate and one team basically broke up but they got back together and move forward in the end um so I guess what I want to emphasize here is that taking time to um articulate that and and to um accommodate that was was very important um during the design and and and one of the ways that we did that uh was through taking inventory not only with the people that were on the teams but also uh uh sending them back to their own institutions for a reality check with their colleagues and very often we found out that the people who were on these design teams were much more flexible and yielding than their colleagues and so they would come back to the table and say I I know we thought of this and I know we thought of that but it's not going to fly so we have to rethink this so um during design and construction I would say that the design uh uh activities uh ranging from ideation to um initial mapping of what the interventions were going to look like was all done face to face and then the process of developing and constructing the prototypes was a mixture of face-to-face online offline uh work um we facilitated that through um uh apps that we had um downloaded and and populated with specific information for them on iPads that they received as part of this uh project um also through a dedicated website and um some encouragement uh using social media the evaluation and reflection phase of this project um focused a lot on ownership by colleagues as we had discovered earlier on uh the people in the design teams tended to be more motivated and flexible than uh everybody else back at their own institutions so a lot of the evaluation work looked at uh what needed to be done to initiate and maintain Buy in in the in the different institutions um another focus of the evaluation work was uh was on looking at actual results are we meeting our goals are we closing the Gap and ultimately is this um contributing to improved student learning or at least improved student outcomes some groups looked more at uh motivational issues and um interest and enjoyability of learning and others looked more at grades and outcomes uh on exams but um the design teams across the board found the assessment of outcomes extremely challenging and I I don't blame them it's real hard stuff but they did um here they are on the day that they got their iPads they don't look as happy as they were actually but that was that was while they were negotiating some serious tension but in the end we have some happy results to report we have six locally driven research informed interventions um we're still Gathering the data on actual Effectiveness but so far perceived value is quite high and we're optimistic um we also have developed technology res sources and um uh also sort of a repertoire of tools um that we can uh use with other teams so these are definitely reusable okay let's talk a little bit about a design based research example we're going to switch um settings from uh higher education in the Netherlands uh higher education business sector in the Netherlands and now we're going to Southern Africa uh where uh we're talking about the development of a tool um an electronic performance support system to support curriculum developers in creating paper-based lesson materials so essentially teacher guides that accompany a textbook um and this was a project that uh survived on the uh or was fueled by the iter develop iterative development of software it was informed by research and as a design-based research project it also had the explicit goal of producing theoretical understanding for use by others during the analysis and exploration phase of the study um a lot of attention was paid to the state of practice so we were fairly well aware of the state of art of uh and at this particular time uh computer software that was available but in that particular context the state-ofthe-art was not relevant the state of practice was much more important so we looked at the technologies that were available the technologies that we were thought would be likely available uh in the next 5 years in uh the teacher Resource Centers where we were working across uh Namibia batswana swasiland South Africa lutu Tanzania and so on um we also looked at the state of practice not just regard with regard to the technology but also with regard to curriculum developer practices what do they already do and what do they want to maybe do differently what do they like about what they're already doing and with both um parts of state of practice so looking at the technology but also looking at the human-based Technologies and uh we wanted to make sure that the Innovation leap uh no I guess I would say we wanted to make sure that what we took was an innovation jump that everybody could make and not an innovation leap that would leave too many people lagging behind and so we chose our focus and we chose our um our technology uh based heavily on what we thought would be a reasonable jump within what I uh have written about elsewhere as the zone of proximal implementation during um during this process we also paid attention to what we thought would be their most urgent needs and wants not only because we want to make a difference but also because we know that if we don't tap into what people feel is um the most necessary for their daily survival we're not going to have much of a shot at uptake and actual use so we definitely attended to those in our design took those studied them carefully in an analysis phase and then attended to those in our design the design and construction phase was um I would call it user centered uh I had the Good Fortune with a background as a preschool teacher to um get trained as a software developer and learn how to program stuff thank goodness there's nice systems out there called like authorware and other icon based systems that allow um people like me who are who are not uh expertise who don't have that kind of expertise to still jump in and play the game um this enabled some I wouldn't call it rapid but I call it semi-rapid prototyping so throughout the course of several years um different prototypes were then um developed and tested with the target users in the Target setting um using um a combination of mixed methods the uh across the years there were 510 respondents from 15 different countries and the kinds of uh evaluation data that were collected um on each of the different prototypes ranged from sit down oneon-one and walk through the Prototype and give me lots of qualitative feedback to also including more quantitative stuff there was also a little bit of Quasi experimental design in uh the final round looking at uh comparing how users of this system performed without the computer as opposed to with the computer the products uh of this study were exactly what you would expect from a design based research study um you have a system so a tool for use uh in the real world that attends to or is meant to address practical problems and at the same time um theoretical understanding that can inform the work of others so there's some examples here of the various books and book chapters and journal articles um it's not complete but this gives an impression of the kinds of Publications that uh resulted from this work so let's think a little bit about what these approaches uh can mean what what what are the affordances what are the drawbacks and uh where might things um the research-based design and design based research be uh synergistic for one thing I think that uh tightly integrated uh design and research trajectories allow us to meet our goals in sustainable ways um and I certainly um have experienced that often times unforeseen benefits come along the way a big one being for example the teacher professional development that can result from being involved on a design team whether that design is more reactionary or proactive um uh there are many opportunities to make more change than might additionally or initially have been intended um there are opportunities for collaborations between uh education industry and Academia there are very few projects that I've been involved in uh where these processes are integrated tightly that do not have some border crossing happening which can be complicated but usually in the end the net the Net's positive uh makes it worth it um and um doing this can yield uh all kinds of new opportunities one of the ones that I think of uh that jumps to my mind especially is the use of agile approaches um in curriculum design so having you know become familiar with um techniques like scrum for example to to do some some rapid and very user- centered design for software we begin to think about the implications of that for creating um uh other kinds of things which might be Technology based or technology supported but maybe aren't necessarily and um so bringing on uh different kinds of expertise and crashing that together can have some very powerful affordances at the same time we need to be realistic uh about the limitations and um I think accept them because that will enable us to confront them headon and actually work towards resolution as opposed to you know stand in denial and uh freaking out when things don't pan out uh and the big ones uh there have to do with the tensions between the stakeholders like I mentioned earlier especially with regard to different priorities different goals different reward systems basically wanting to get different things out of the Enterprise um another one is um just got to do with the resources it takes awful lot of time it takes money it takes expertise to integrate these processes well it's not for the fainthearted at all uh so um some situations May simply render it unfeasible but I think it's worth trying for in most cases I think there might also be some synergies if we think about the different orientations and mindsets behind uh design-based research and research-based design I like to imagine that um some of the development expertise coming from research-based design might begin to inform design-based research in ways that that we haven't uh thought about yet um and I I think I mentioned earlier the idea of principles that come out of design-based research so design principles that can uh guide the process of design or inform what the products of design ought to look like could be usable certainly in research-based design so how do we take these ideas forward I think um a key thing is to accept that integration is difficult and uh to commit to doing it anyway and to face those challenges headon um not not uh doing doing one's best um to use uh rigorous research methods even in the face of other tensions and other constraints and um uh striving to derive products and uh processes that accommodate the real world not just the ideal world uh in certain situations uh research-based designers and designers in general or Developers who work or at least have one foot planted real firmly in practice have very little difficulty with that researchers tend to um idealize things more than uh we'd like to admit um this is something we can work on um I also uh have been through a major reorganization at the open University and and I love the acronym of the group uh that I've I've landed in I think that this is something we we could all enjoy working towards uh uh developing integ ated trajectories that feel good so looking for um in this case e-learning designs that can foster effective enjoyable and efficient learning and looking at how research-based design and design-based research can actually contribute to that is important but then what we need to do is share those stories within the community so within the e-learning community it's important to first of all uh voice our concerns our struggles um possibly even our VI iies and uh begin to collaborate maybe on uh tackling different challenges and uh we talked about this yesterday maybe this is something that we could be doing within sigs here at e-learn maybe it's even something that could happen across uh different special interest groups so uh to wrap up for today I think that um separate research and development processes are a reality but they bring with them certain risks and maybe they can't be avoided at all times but where possible I would uh advocate for um robust and integrated approaches where research and development aren't just parallel but are really tightly integrated to the benefit of both and I think that um this brings along with it some challenges that we should tackle together um and uh begin to look at how we can and share our stories about how we can integrate research and development processes in ways that don't dilute the quality of either in case you're interested in learning more about this kind of stuff um well I uh there's some resources available uh in print and I'll I'll post this online um there's also plenty of web-based resources uh that teach more especially about design-based research and of course I'm always open to talk with people about design so if you're interested I uh would be delighted to have a conversation with you and I believe that we actually even have time to do that right now so perhaps we can hello Susan thank you so much it it was really good thank you um the there's as you know I do quite a bit of design-based research myself and supervise quite a bit of it and then by necessity I also tend to have to examine quite a bit of it and one of the questions that I keep on having to ask people who do that is when you're designing something and then testing what you are designing how do you protect yourself against finding what you want to find um I I'm I'm finding it very hard with and and I'd like you to comment a little finding it very hard when students come to me and they say I want to do this research because I want to prove to my boss that my job is necessary and I say you can't do that because you're not protecting yourself against finding what you want to find right so that's a great question and I don't have I don't have the answers but I can share with you some of the things that we have done because I I I uh I get that and we've we've all lived with those challenges so um like so many things I think we get by with a Little Help from our friends and and um I think that one thing that can help a lot is to make sure that in a design-based research project there's enough uh people who are involved um uh and preferably that would include the design researcher but it isn't always feasible who don't have a stake in the outcome uh so of course people you're if you're you know looking to make a change on the ground everybody involved wants to see that change no question but um if I think about some of the technology projects especially that I've done where we go into classrooms then and we see what people actually do you know whether our designs degrade gracefully or become lethal mutations um they're never used the way we expect and uh that kind of humbling experience can be an amazing tool for redesign if we embrace it as such but if we're really concerned with seeing it do the thing that we really want it to do then we don't have an open mind to begin with and we're not going to be able to make the use of that the evaluation finding so I like and for Theory building I would emphasize also especially to students for whom the whole the whole word theory just makes them get all nervous I would emphasize that finding surprising results is really interesting and when things don't do what you expect um well you know for some cases there's there's because what you expected was really naive and that needs to be addressed but maybe it's because the theoretical understanding that you have so far needs tweaking or maybe it's because it's the theoretical understanding that we have so far isn't as robust or reliable as we thought it was so um I think that the that that helpful things have to do with having different um advisors in this case I think it would be advisers on a project and even even a small do doctoral student project can have a a a group of critical friends that you go to once every so often in Belgium I'm on a couple of committees where they have an annual review like that and uh and we just go in and toss our two cents once a year um another one like I said not having a stake in the outcomes being able to feel like I did usable valuable research even if the findings aren't what I expected I can still and you can do that if you are able to put them to productive use to still make a good change on the ground so I think those would be at at the top of my list of of conversations uh that I would have with with students on that topic thank you okay Susan thank you very much for your presentation thank you very much