MeMyself&I

Posts Tagged ‘design-based research

I am reflecting on another recent success of my authentic learning research that has become a part of my being. I submitted a paper to HERDSA for the first time this year. My intention when I submitted was to get feedback on my work, I heard that HERDSA has a very good reputation for its rigorous review process.

And rigorous it was, the most informative feedback I’ve received by far. The only ever time I was asked to respond to every reviewer’s feedback and suggestions. I was required to document my reaction/response to each and every recommendation made by the reviewers. The learning for a relatively new researcher like me had been tremendous. There were many praises and encouragement as well included in the reviewers’ feedback, which was motivating for me and my co-researcher.

At the HERDSA Conference in Darwin, I was informed by folks from Sydney University that my research had influenced their presentation on role play, which was surprising to know. However, the biggest surprise came during the closing ceremony when my name was called to receive the best paper award for authentic learning. I couldn’t help but reflect on what the conclusion of my PhD thesis holds. I dream of contributing to knowledge to improve educational practice. With the two recent successes at conferences, I wonder if I am on track to make a difference.

Well, the PhD journey has just began, it is a while to go yet. It is good to know for now that interests in my research project continue to come my way. I have been invited again to present at another university on the same topic below. 

Advertisements

As I engage in my PhD work and get feedback from ‘critical friends’ I am seeing a slightly different path to how I’ve originally conceived how my thesis will unfold. The new title summarises or captures a more definitive focus for the thesis. This reframing was also heavily influenced by recent readings and updates to my literature review. The literature review is still in progress and it could change again, but I thought I should record my current thinking and see the complete evolution of my PhD journey 🙂

A special thank you to Tim Lever for capturing the essence of my original proposal better than I could, Tim suggested a more focused topic for my work. 

Research topic/title

Enculturation into discipline specific ways of thinking and acting whilst at university: Comparative evaluation of approaches in higher education to preparation for professional practice

Background
The educational issue that my research proposes to address is that traditional learning environments do not represent the values or practices of the profession for which educators intend to prepare students. In these types of environments students acquire abstract and decontextualised knowledge coming from teaching approaches that tend to separate “knowing” from “doing” (Brown, Collins & Duguid, 1989). In approaches where the interdependence of situation and cognition are ignored, the students do not perceive the knowledge acquired as being useful in solving real problems outside university, so what they develop is only “inert knowledge” (Collins, Brown & Newman, 1989). This is a type of knowledge that people can recall when prompted but cannot recall in problem solving situations (Bransford, et al, 1990; Cognition & Technology Group of Vanderbilt, 1990; Herrington & Oliver, 2000). Within the accounting domain, for example, researchers and practitioners both identify inert knowledge in graduate accountants, claiming that graduate accountants accumulate a storehouse of technical knowledge they cannot apply to solve real world problems (Catanach, et al., 2000, Freeman, 2008; Sundem, 1994).

Aims
In response to this educational dilemma, the central aim of my research is to propose, exemplify and test alternative approaches to learning and teaching in higher education that provide students bridges rather than gaps between learning at university and professional practice. Re-engineering the traditional learning environment, and developing and testing prototypes form part of this aim, in which the design of curricula integrates opportunities for students to perform authentic practices and activities that practitioners and experts engage in during real problem solving situations. 

Methodology
I envisage adopting an iterative empirical approach, which will be guided by an experimental framework known as ‘design-based research’ (Brown, 1992; Collins, 1992). Design-based research ‘blends empirical educational research with the theory-driven design of learning environments [thereby shaping] an important methodology for understanding how, when and why educational innovations work in practice’ (The Design-Based Research Collective, 2003, p.5). Consistent with the principles of design-based research, both quantitative and qualitative methods will be employed in the iterative cycle of design, implementation, analysis and modification. Ethnographic approaches will be used as it provides qualitative methods for looking carefully at how a design plays out in practice and how social and contextual variables interact with cognitive variables (Collins, Joseph & Bielaczyc, 2004). Additionally, the quantitative method of administering survey questionnaires provides a means for evaluating the effects of independent variables on the dependent variables (Collins, Joseph & Bielaczyc, 2004). The adoption of mixed methods is useful for the description of phenomena reflecting the complex issues that should be addressed (Brown 1992).

Expected outcomes
The conclusion of my PhD will deliver a greater understanding of how students build their knowledge in learning environments where preparation for professional practice is the explicit aim. It is also anticipated that the outcomes of my research will include practical guidelines for designing learning environments that facilitate better prepared students for their chosen professional careers.

Timetable
As this research will involve human participants approval for Ethics in Human Research will be sought by the end of Year 1. The study will proceed in three phases, as follows:

Phase 1: Development of framework – Year 1
This phase will involve a review of literature on educational innovations facilitated by previous research initiatives. The different educational approaches that have evolved will be contrasted, and I will identify the theoretical frameworks that align with the conventions of selected disciplines as case studies. The goal is to ‘specify the significant disciplinary ideas and forms of reasoning that constitute the prospective goals or endpoints for student learning’ (Cobb et al, 2003, p. 11).

Phase 2: Design and development of prototypes – First half of Year 2
Evaluation of the existing learning designs in use in selected discipline areas will be documented using both archived and classroom observation data. Evaluative comparisons will be drawn about features, strengths and weaknesses between the existing design in use and the intended design of prototypes. The curriculum design principles formulated in the previous phase will underpin the development of prototypes, within which intended learning outcomes, instructional materials, activity structures and assessment are central to the design. Because the focus of this study is on the design of a model of learning, the new learning design is reified in the learning environments. As Kelly (2004) observes, the exploration of research questions about learning are reified, explored, and tested by the design and use of the learning environment.

Phase 3: Implementation, data collection, analysis, and modification – Second half of Year 2
Multiple cycles of implementations and evaluations will be carried out in partnership with academic teachers at different campuses of a regional university. Both students’ and teachers’ perceptions of their physical and online “classroom” experiences will be investigated and, following the main tenets of design-based research, evaluation will be carried out using multiple strategies, e.g. online observations, recording of classroom episodes, survey questionnaire, individual interviews and focus group interviews.  

Within the different cycles of the implementation, the iteration process for analysis – prototypes modifications – evaluations will be documented in my thesis, and will focus on validating assumptions embodied in the prototypes, developing new ones, and finally refining and enhancing the different prototypes.

By Year 3 if not sooner, I will take leave of absence from work to take up full-time engagement in my PhD and complete the study. By the second half of Year 3 I envisage the writing and compilation of the thesis chapters will commence, which will include revisiting the literature review again along with methodological assumptions, following the lived experience of this research.

References 

Bransford, J.D., Sherwood, R.D., Hasselbring, T.S., Kinzer, C.K. & Williams, S.M. (1990). Anchored Instruction: Why we need it and how technology can help. In D. Nix & R. Spiro (Eds.), Cognition, education and multimedia: Exploring ideas in high technology, pp.115-141, Hillsdale: Lawrence Erlbaum.

Brown, A.L. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. The Journal of the Instructional Sciences, 2(2), pp. 141-178.

Brown, J.S., Collins, A. & Duguid, P. (1989). Situated Cognition and the Culture of Learning, Educational Researcher 18(1), pp.32-42.

Catanach, A.H., Croll, D.B. & Grinaker, R.L. (2000). Teaching Intermediate Financial Accounting Using a Business Activity Model, Issues in Accounting Education, 15(4), pp.583-603.

Cobb, P., diSessa, A., Lehrer, R., Schauble, L. (2003) Design Experiments in Educational Research, Educational Researcher, 32(1), pp. 9-13.

Cognition and Technology Group at Vanderbilt (1990). Anchored instruction and its relationship to situated cognition, Educational Researcher, 19 (6), pp.2-10.

Collins, A. (1992). Towards a design science of education, In E Scanlon & T O’Shea (Eds), New directions in educational technology, Springer, Berlin.

Collins, A., Brown, J.S., & Newman, S.E. (1989). Cognitive apprenticeship: teaching the crafts of reading, writing and mathematics. In L.B. Resnick (Ed.), Knowing, learning, and instruction: essays in honor of Robert Glaser. Hillsdale, NJ: Erlbaum.

Collins, A., Joseph, D. & Bielaczyc, K. (2004). Design research: Theoretical and methodological issues. The Journal of the Instructional Sciences, 13(1), pp. 15-42.

Design-Based Research Collective (2003). Design-based research: An emerging paradigm for educational enquiry, Educational Researcher, (32)1, pp. 5–8.

Freeman, M., Hancock, P., Simpson, L., & Sykes, C. (2008). Business as usual: a collaborative and inclusive investigation of existing resources, strengths, gaps and challenges to be addressed for sustainability in teaching and learning in Australian university business faculties. ABDC Scoping Report, March, 1-54. 

Herrington, J., & Oliver, R. (2000). An instructional design framework for authentic learning environments. Educational Technology Research and Development, 48(3), 23-48.

Sundem, G.L. (1994) Scholarship in four dimensions, CA Magazine, 127(3), pp.39-44.

After long and laborious work, thinking and writing about a myriad of ideas relating to the educational issue I wanted to pursue in my PhD study, I think I finally got it.

My working title is something like this:

Enculturation into discipline specific ways of thinking and acting whilst at university: Implications for professional practice

 I have synthesized  the problematic for the research, as follows:

The educational issue that my research proposes to address is that traditional learning environments do not represent the values or practices of the profession for which educators intend to prepare students. In these types of environments students acquire abstract and decontextualised knowledge coming from teaching approaches that tend to separate “knowing” from “doing” (Brown, Collins & Duguid, 1989). In approaches where the interdependence of situation and cognition are ignored, the students do not perceive the knowledge acquired as being useful in solving real problems outside university, so what they develop is only “inert knowledge” (Collins, Brown & Newman, 1989). This is a type of knowledge that people can recall when prompted but cannot recall in problem solving situations (Bransford, et al, 1990; Cognition & Technology Group of Vanderbilt, 1990; Herrington & Oliver, 2000). Within the accounting domain, for example, researchers and practitioners both identify inert knowledge in graduate accountants, claiming that graduate accountants accumulate a storehouse of technical knowledge they cannot apply to solve real world problems (Freeman, 2008; Sundem, 1994; Catanach, et al., 2000).

The central aim for my research that I have so far identified is:

To propose, exemplify and test alternative approaches to learning and teaching in higher education that provide students bridges rather than gaps between learning at university and professional practice.

The preliminary research plan goes something along these lines:

I envisage adopting an iterative empirical approach, which will be guided by an experimental framework known as ‘design-based research’ (Brown, 1992; Collins, 1992). Design-based research ‘blends empirical educational research with the theory-driven design of learning environments [thereby shaping] an important methodology for understanding how, when and why educational innovations work in practice’ (The Design-Based Research Collective, 2003, p.5). Consistent with the principles of design-based research, both quantitative and qualitative methods will be used in the iterative cycle of design, implementation, analysis and modification. Ethnographic approaches will be used as it provides qualitative methods for looking carefully at how a design plays out in practice and how social and contextual variables interact with cognitive variables (Collins, Joseph & Bielaczyc, 2004). Additionally, the quantitative method of administering survey questionnaires provides a means for evaluating the effects of independent variables on the dependent variables (Collins, Joseph & Bielaczyc, 2004). The adoption of mixed methods is useful for the description of phenomena reflecting the complex issues that should be addressed (Brown 1992). As this research involves human participants, approval for Ethics in Human Research will be sought. The study will proceed in three phases, as follows:

Phase 1: Development of framework

This phase will involve a review of literature on educational innovations facilitated by previous research initiatives. The different educational approaches that have evolved will be contrasted, and I will identify the theoretical frameworks that chime with the conventions of selected disciplines as case studies. The goal is to ‘specify the significant disciplinary ideas and forms of reasoning that constitute the prospective goals or endpoints for student learning’ (Cobb et al, 2003, p. 11).

Phase 2: Design and development of prototypes

The curriculum design principles formulated in the previous phase will underpin the development of prototypes, within which intended learning outcomes, instructional materials, activity structures and assessment are central to the design. Because the focus of this study is on the design of a model of learning, this is reified in the technology-enhanced learning environments. As Kelly (2004) observes, the exploration of research questions about learning are reified, explored, and tested by the design and use of the learning environment.

Phase 3: Implementation, data collection, analysis, and modification

Multiple cycles of implementations and evaluations will be carried out in partnership with academic teachers at different campuses of a regional university. Both students’ and teachers’ perceptions of their “classroom” experiences will be investigated and, following the main tenets of design-based research, evaluation will be carried out using multiple strategies, e.g. observations and recording of classroom episodes, survey questionnaire, individual interviews and focus group interviews.  Within the different cycles of the implementation, the iteration process for analysis – prototypes modifications – evaluations will be documented in my thesis, and will focus on validating assumptions embodied in the prototypes, developing new ones, and finally refining and enhancing the different prototypes.

And finally the intended outcomes of my research:

The conclusion of my PhD will deliver a greater understanding of how students build their knowledge in learning environments where preparation for professional practice is the explicit aim. It is also anticipated that the outcomes of my research will include practical guidelines for designing learning environments that facilitate better prepared students for their chosen professional careers.

References 

Bransford, J.D., Sherwood, R.D., Hasselbring, T.S., Kinzer, C.K. & Williams, S.M. (1990). Anchored Instruction: Why we need it and how technology can help. In D. Nix & R. Spiro (Eds.), Cognition, education and multimedia: Exploring ideas in high technology, pp.115-141, Hillsdale: Lawrence Erlbaum.

Brown, A.L. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. The Journal of the Instructional Sciences, 2(2), pp. 141-178.

Brown, J.S., Collins, A. & Duguid, P. (1989). Situated Cognition and the Culture of Learning, Educational Researcher 18(1), pp.32-42.

Catanach, A.H., Croll, D.B. & Grinaker, R.L. (2000). Teaching Intermediate Financial Accounting Using a Business Activity Model, Issues in Accounting Education, 15(4), pp.583-603.

Cobb, P., diSessa, A., Lehrer, R., Schauble, L. (2003) Design Experiments in Educational Research, Educational Researcher, 32(1), pp. 9-13.

Cognition and Technology Group at Vanderbilt (1990). Anchored instruction and its relationship to situated cognition, Educational Researcher, 19 (6), pp.2-10.

Collins, A. (1992). Towards a design science of education, In E Scanlon & T O’Shea (Eds), New directions in educational technology, Springer, Berlin.

Collins, A., Brown, J.S., & Newman, S.E. (1989). Cognitive apprenticeship: teaching the crafts of reading, writing and mathematics. In L.B. Resnick (Ed.), Knowing, learning, and instruction: essays in honor of Robert Glaser. Hillsdale, NJ: Erlbaum.

Collins, A., Joseph, D. & Bielaczyc, K. (2004). Design research: Theoretical and methodological issues. The Journal of the Instructional Sciences, 13(1), pp. 15-42.

Design-Based Research Collective (2003). Design-based research: An emerging paradigm for educational enquiry, Educational Researcher, (32)1, pp. 5–8.

Freeman, M., Hancock, P., Simpson, L., & Sykes, C. (2008). Business as usual: a collaborative and inclusive investigation of existing resources, strengths, gaps and challenges to be addressed for sustainability in teaching and learning in Australian university business faculties. ABDC Scoping Report, March, 1-54. 

Herrington, J., & Oliver, R. (2000). An instructional design framework for authentic learning environments. Educational Technology Research and Development, 48(3), 23-48.

Sundem, G.L. (1994) Scholarship in four dimensions, CA Magazine, 127(3), pp.39-44.

 

Grades have been finalised and are about to be released for T2 2008, with pleasing results and important points for reflection that are critical to future implementation of the auditing course. One important point in particular is the frequency of course offering. Attrition rates and performance-related issues are significantly higher in T2 for off-campus students historically. One of the possible links might be high demands in the workplace with end of financial year reporting and planning activities for many working students.

In the second cycle of implementation of our design-based research in the auditing course, preliminary data suggests the need to revisit the timing of course offering for off-campus students. Currently, the course is offered every term, i.e. Term 1 – AICs, Regional, Flex; Term 2 – AICs, Flex; Term 3 – AICs. 

The design of the auditing course is such that it requires a minimum of 12 hours in-depth participation with the course material, with teaching staff and with other students to complete course work. One of the inherent weaknesses in the design is the prescriptive-nature of the activities which suits one cohort of students but problematic for another particularly those who are working full-time.On the one hand, some teaching staff at AICs report that the design is effectively helping students’ active engagement in the course. Some off-campus students also report that the design of the course gives them focus and is helpful to their time management for work, home and study. However, on the other hand, some off-campus students report that they just want to complete assignments on their own, prepare for the exam without the need to interact with others, and that they can manage their time better this way.  The latter was the approach used in previous offerings of the auditing course, where the failure rate was significantly higher and a great number of students were failing the course multiple times.

Should we consider embedding choices for students in the design of learning and assessment? What might be the implications for practice, and the quality of student learning experience? One of the tacit design aims for the auditing course is to develop capacity for students to become better learners, many approaches for which require time investment on the part of the students. But many in the off-campus cohort are time-poor students whose motivation for enrolling in a degree is not always intrinsic.

It is perhaps worthwhile to survey both 2008 T1 and T2 off-campus students to determine future improvements to the auditing course that meet the needs of different cohorts of students.

Well I have procrastinated long enough, and my boss David Jones knows this trait of mine only too well. David happens to be one of the most prolific bloggers I know… check him out, and gave me an ultimatum to blog my thinking about my PhD enrolment. I will start by reflecting on the following questions/ideas in separate posts:

What is the aim of your thesis? What’s your question or problem? I think this is something to do with your perception that what you are currently doing is not scalable.

– Why isn’t scalable? What is the source of the problems you are facing?

– What’s your definition of capacity building?

– Why is capacity building important to learning design/universities?

– What have people already written about this?

– What are the problems with what they’ve written?

One of my problems is narrowing down the problem/question, so hopefully this blogging exercise will finally get me focused.

It has been a while since I posted, which can only suggest that I’ve been busy elsewhere. Indeed I have been pre-occupied, implementing and evaluating the first cycle of design-based research involving the machinima initiative. But I should have been formatively sharing the research journey here… no excuses in the second cycle of implementation 🙂

This project is funded by CQUni Learning and Teaching Grants in which the aim is to:

investigate ways in which a particular model of learning, namely cognitive apprenticeship may be embedded in traditional pedagogical approaches such as lectures and tutorials. The key objectives of the study are threefold: identify learning and teaching strategies based on the principles of cognitive apprenticeship; develop teaching and learning resources that support this model of learning within the accounting education context; and evaluate the impact of this pedagogical approach with a group of students in an auditing course. An increased value placed on teaching and quality of student learning outcomes is the core aim of the research.

We have disseminated preliminary findings at a couple of internal forums at CQUni available here: Presentation 1 – Cognitive Apprenticeship in Accounting Education, Presentation 2 – Second Life Machinimas.

A refereed paper will also be published at the proceedings of ascilite Melbourne 2008 Conference, titled Bringing ‘second life’ to a tough undergraduate course: Cognitive apprenticeship through machinimas.


Blog Stats

  • 10,459 hits

My Tweets

RSS My Links

  • An error has occurred; the feed is probably down. Try again later.

Flickr Photos