Jean Macnish - Academia.edu (original) (raw)
Papers by Jean Macnish
Background: Standard setting is integral in clinical skill assessments in outcome and competency ... more Background: Standard setting is integral in clinical skill assessments in outcome and competency based MBBS curriculum. To determine the most credible and feasible standard setting method for clinical examinations (OSCEs), the outcomes from three different methods were compared. Summary of work: Modified Angoff (MAM), Borderline Group (BGM), and Borderline Regression (BRM) standard setting methods were applied to nine OSCE stations for 103 first year and 106 second year student. The same set of five examiners standard set each OSCE station using each of the three methods. Summary of results: Pass marks and standard errors for MAM were higher than the other two methods in most stations. Pass / fail decisions agreement between MAM and BGM/BRM was statistically significant, but marginally convincing (Kappa = 0.488, p Conclusions: Empirical evidence seems to indicate that BGM/BRM is more credible and defensible than MAM for standard setting pass mark in OSCE. Take-home messages: Evidenc...
Early Child Development and Care
Http Dx Doi Org 10 1080 17439884 2013 872655, May 20, 2014
ABSTRACT This paper discusses gaming in a Western Australian school for boys. The overriding etho... more ABSTRACT This paper discusses gaming in a Western Australian school for boys. The overriding ethos of the school is supportive of the potential of ICT to better engage students and deliver enhanced educational outcomes. The school sees game-based design as at the vanguard of innovation, but also accepts its important duty of care responsibilities. Tensions were revealed between the opportunities presented by educational gaming and the perceived problem of managing student distraction, particularly the tendency for students to spend large amounts of time playing games that have little or no educational value. The paper describes the forms of gaming that emerged at the school, considering both their educational impact and propensity to detract from students' opportunities to learn. It is argued that the perceived benefits and risks of gaming are not well understood, and that powerful political forces are at play which shape school policy, teachers' pedagogy, parent perceptions and student actions.
Providing accessible Web pages is becoming a key concern for many providers of electronic informa... more Providing accessible Web pages is becoming a key concern for many providers of electronic information. There are many people who find accessing Web pages difficult and among these, vision impaired users are perhaps the group with the greatest needs. The Web is a strong visual environment and most designers use this aspect of the environment as a critical element in their interface and information design. Such strategies, while providing many opportunities for mainstream Web users, provide limiting and impeding outcomes for visually impaired Web users. There are a number of accessibility standards that now exist to inform and guide the designers of Web pages but little is known about precisely how best these standards can be applied and achieved. This paper will describe a study undertaken in the Australian context that sought to explore how the goals of accessibility influenced the design process and the design outcomes of an online learning environment designed to cater for visually impaired users. It is a study of the TruVision Project, a Web-based learning setting, designed to aid visually impaired users to gain an elementary qualification in Information Technology.
INTED2016 Proceedings, 2016
This paper provides some initial findings from a current longitudinal study that examines the imp... more This paper provides some initial findings from a current longitudinal study that examines the implementation of a student-owned 1:1 laptop program in a school for boys in Perth, Western Australia. This research tracks 196 students, their families and associated teachers for a 3-year period (2010-2012). Underpinning this research is a mixed methods approach investigating how boys use their laptops for learning, teachers' pedagogy and use of ICT, implementation differences between a junior (primary) and middle (secondary) school, and possible impact of the laptops on learning. One theme that emerged from the first year of data collection was a decrease in parent satisfaction with the extent to which the educational objectives of the laptop initiative are being met. This paper explores possible reasons for this decline in satisfaction, focusing on parent and student perceptions of (a) the time spent on laptops and (b) the activities that students are seen to be engaging with on their laptops. These perceptions are discussed in the context of parents' own knowledge of, and skills in, information and communications technologies (ICT) and relate to both school and home-based settings.
Edulearn12 Proceedings, 2012
This paper provides some initial findings from a current longitudinal study that examines the imp... more This paper provides some initial findings from a current longitudinal study that examines the implementation of a 1:1 laptop program in a school for boys in Perth, Western Australia. One issue that has emerged from the study is the problem of managing student distraction. The school in this study has taken a proactive approach to managing student conduct on its own network. Two student monitoring initiatives were implemented during the course of the research. The first: parental control software sought to integrate the parental control features of the laptops with the school network. The second initiative: e-safe is a web tracking service that records suspicious searches and URLs that students visit. When used in tandem, these tools were shown to have a marked impact on the conduct of students in using their laptops. This paper describes these initiatives including their effect on the broader school community, and suggests some ways in which student distraction can be best managed in future practice.
... Whilst learners found this a straightforward task, the technical infrastructure was not condu... more ... Whilst learners found this a straightforward task, the technical infrastructure was not conducive to being on ... Selecting learners with an interest in the content, and in learning flexibly is a sensible ... The inclusion of too many learners with low levels of motivation may create a peer ...
This paper provides some initial findings from a current longitudinal study that examines the imp... more This paper provides some initial findings from a current longitudinal study that examines the implementation of a student-owned 1:1 laptop program in a school for boys in Perth, Western Australia. This research tracks 196 students, their families and associated teachers for a 3-year period (2010-2012). Underpinning this research is a mixed methods approach investigating how boys use their laptops for learning, teachers’ pedagogy and use of ICT, implementation differences between a junior (primary) and middle (secondary) school, and possible impact of the laptops on learning. One theme that emerged from the first year of data collection was a decrease in parent satisfaction with the extent to which the educational objectives of the laptop initiative are being met. This paper explores possible reasons for this decline in satisfaction, focusing on parent and student perceptions of (a) the time spent on laptops and (b) the activities that students are seen to be engaging with on their l...
This paper provides some initial findings from a current longitudinal study that examines the imp... more This paper provides some initial findings from a current longitudinal study that examines the implementation of a student-owned 1:1 laptop program in a school for boys in Perth, Western Australia. This research tracks 196 students, their families and associated teachers for a 3-year period (2010-2012). Underpinning this research is a mixed methods approach investigating how boys use their laptops for learning, teachers’ pedagogy and use of ICT, implementation differences between a junior and middle school, and possible impact of the laptops on learning. One theme that emerged from the first two years of data collection was a decrease in parent satisfaction with the extent to which the educational objectives of the laptop initiative were being met. This paper explores possible reasons for this decline in satisfaction, focusing on parent and student perceptions of (a) the time spent on laptops and (b) the activities that students are seen to be engaging with on their laptops. These pe...
This exploratory research characterised the degree of adoption of mobile learning (ML) devices am... more This exploratory research characterised the degree of adoption of mobile learning (ML) devices among academic staff at an Australian university. It also sought to evaluate the impact of academics' perceptions about possibilities and constraints in the adoption of these technologies. A zone of free movement (ZFM) scale was developed and validated to quantify the magnitude and direction of those perceptions. Results showed that academic staff are characteristically at the third of the Russell's (1995) six developmental stages of technological adoption. Lack of time to integrate ML into courses, limited availability of mobile devices, little familiarity with the tools, as well as the perception that students cannot use them as a wordprocessor, act as inhibitors to the adoption of the technology. In turn, the perception that mobile tools enhance student-lecturer communication outside class was found to be a positive predictor of adoption.
Education Sciences, 2013
This qualitative study looked at the instructional, curricular and organizational factors impacti... more This qualitative study looked at the instructional, curricular and organizational factors impacting on the adoption of mobile learning in a higher education institution. Academics expressed their views on a variety of educational issues likely to enhance or hinder the effectiveness of the innovation. Teachers requested more professional development in a number of key areas including the integration of the technology into teaching and learning. Likewise, resolving vital issues such as workload, equity to access and effective policy making were seen as key to successful adoption. Technical issues such as having good wireless connectivity, need for efficient technical support, access to mobile devices and an understanding of their operational limitations with respect to desk/laptops were also highlighted. The study revealed a number of alternative perceptions and misconceptions, about articulating effective mobile learning pedagogies. For instance, staff expressed concerns about the risk of exposing students to superficial learning when mobile learning experiences were not well designed, the prospect that the devices might distract students from learning, as well as a possible deterioration of the quality of interaction between academics and students. Recommendations to reconcile those alternative conceptions with best practice principles are outlined.
The objective and authentic assessment of practical skills in tertiary settings is a challenge. T... more The objective and authentic assessment of practical skills in tertiary settings is a challenge. The purpose of this project was to evaluate the usefulness of a video analysis software program as a teaching and/or assessment tool for practical and professional skills in five different undergraduate programs. Barnett’s (1989) model of ‘reflective thought through action’ highlights the importance of reflective observation
The Topic Sevenoaks Senior College is an innovative new secondary school located in a lower socio... more The Topic Sevenoaks Senior College is an innovative new secondary school located in a lower socioeconomic suburb of Perth, Western Australian. This area has a high rate of unemployment and a low rate of further tertiary study by students leaving those schools (Wood & Trinidad, 2001). Sevenoaks opened its doors in February 2001 with a strong focus on industry and community partnerships. The driving force behind the college is its client perspective including strategies for customised learning so that programs are developed for each student available online, incorporate vocational learning and enterprise outcomes, use information technology and are contextualised to meeting individual learning rates and styles. As Wood and Trinidad (2001) state "Sevenoaks Senior College has been built on a clear vision of an adult and workplace oriented culture. It has a high degree of flexibility (operating from 8am to 5pm) with all students involved in Structured Workplace Learning (SWL), and a culture of enterprise where students are actively engaged in their own learning through the online curriculum and strong connections to the learning community that is crucial in providing young people with an environment where they feel valued and supported" (p.7).
Background: As an integral tool for clinical skill assessment, the quest to enhance the validity ... more Background: As an integral tool for clinical skill assessment, the quest to enhance the validity and reliability of OSCE scores should be ongoing. Summary of work: A new instrument with a wide spectrum of scales that allow objectivity and precision (i.e. checklist, process performance rating, and an overall global rating) was introduced, replacing the use of holistic numerical scoring. The new marking format was piloted and used in nine OSCE stations for first year students and 14 OSCE stations for second year students in 2010. It was also specifically designed for computerized scoring using Optical Mark Recognition software. Summary of results: Findings suggest that the new instrument is well accepted by examiners for its ease of use and enhanced objectivity. Empirical evidence also indicates better discrimination for students’ clinical skills, stronger internal consistency, greater inter- and intra-station reliability, and improved individual examiner marking consistency. Conclusi...
This paper provides some initial findings from a current longitudinal study that examines the imp... more This paper provides some initial findings from a current longitudinal study that examines the implementation of a 1:1 laptop program in a school for boys in Perth, Western Australia. One issue that has emerged from the study is the problem of managing student distraction. The school in this study has taken a proactive approach to managing student conduct on its own network. Two student monitoring initiatives were implemented during the course of the research. The first: parental control software sought to integrate the parental control features of the laptops with the school network. The second initiative: e-safe is a web tracking service that records suspicious searches and URLs that students visit. When used in tandem, these tools were shown to have a marked impact on the conduct of students in using their laptops. This paper describes these initiatives including their effect on the broader school community, and suggests some ways in which student distraction can be best managed i...
Background: Standard setting is integral in clinical skill assessments in outcome and competency ... more Background: Standard setting is integral in clinical skill assessments in outcome and competency based MBBS curriculum. To determine the most credible and feasible standard setting method for clinical examinations (OSCEs), the outcomes from three different methods were compared. Summary of work: Modified Angoff (MAM), Borderline Group (BGM), and Borderline Regression (BRM) standard setting methods were applied to nine OSCE stations for 103 first year and 106 second year student. The same set of five examiners standard set each OSCE station using each of the three methods. Summary of results: Pass marks and standard errors for MAM were higher than the other two methods in most stations. Pass / fail decisions agreement between MAM and BGM/BRM was statistically significant, but marginally convincing (Kappa = 0.488, p Conclusions: Empirical evidence seems to indicate that BGM/BRM is more credible and defensible than MAM for standard setting pass mark in OSCE. Take-home messages: Evidenc...
Early Child Development and Care
Http Dx Doi Org 10 1080 17439884 2013 872655, May 20, 2014
ABSTRACT This paper discusses gaming in a Western Australian school for boys. The overriding etho... more ABSTRACT This paper discusses gaming in a Western Australian school for boys. The overriding ethos of the school is supportive of the potential of ICT to better engage students and deliver enhanced educational outcomes. The school sees game-based design as at the vanguard of innovation, but also accepts its important duty of care responsibilities. Tensions were revealed between the opportunities presented by educational gaming and the perceived problem of managing student distraction, particularly the tendency for students to spend large amounts of time playing games that have little or no educational value. The paper describes the forms of gaming that emerged at the school, considering both their educational impact and propensity to detract from students' opportunities to learn. It is argued that the perceived benefits and risks of gaming are not well understood, and that powerful political forces are at play which shape school policy, teachers' pedagogy, parent perceptions and student actions.
Providing accessible Web pages is becoming a key concern for many providers of electronic informa... more Providing accessible Web pages is becoming a key concern for many providers of electronic information. There are many people who find accessing Web pages difficult and among these, vision impaired users are perhaps the group with the greatest needs. The Web is a strong visual environment and most designers use this aspect of the environment as a critical element in their interface and information design. Such strategies, while providing many opportunities for mainstream Web users, provide limiting and impeding outcomes for visually impaired Web users. There are a number of accessibility standards that now exist to inform and guide the designers of Web pages but little is known about precisely how best these standards can be applied and achieved. This paper will describe a study undertaken in the Australian context that sought to explore how the goals of accessibility influenced the design process and the design outcomes of an online learning environment designed to cater for visually impaired users. It is a study of the TruVision Project, a Web-based learning setting, designed to aid visually impaired users to gain an elementary qualification in Information Technology.
INTED2016 Proceedings, 2016
This paper provides some initial findings from a current longitudinal study that examines the imp... more This paper provides some initial findings from a current longitudinal study that examines the implementation of a student-owned 1:1 laptop program in a school for boys in Perth, Western Australia. This research tracks 196 students, their families and associated teachers for a 3-year period (2010-2012). Underpinning this research is a mixed methods approach investigating how boys use their laptops for learning, teachers' pedagogy and use of ICT, implementation differences between a junior (primary) and middle (secondary) school, and possible impact of the laptops on learning. One theme that emerged from the first year of data collection was a decrease in parent satisfaction with the extent to which the educational objectives of the laptop initiative are being met. This paper explores possible reasons for this decline in satisfaction, focusing on parent and student perceptions of (a) the time spent on laptops and (b) the activities that students are seen to be engaging with on their laptops. These perceptions are discussed in the context of parents' own knowledge of, and skills in, information and communications technologies (ICT) and relate to both school and home-based settings.
Edulearn12 Proceedings, 2012
This paper provides some initial findings from a current longitudinal study that examines the imp... more This paper provides some initial findings from a current longitudinal study that examines the implementation of a 1:1 laptop program in a school for boys in Perth, Western Australia. One issue that has emerged from the study is the problem of managing student distraction. The school in this study has taken a proactive approach to managing student conduct on its own network. Two student monitoring initiatives were implemented during the course of the research. The first: parental control software sought to integrate the parental control features of the laptops with the school network. The second initiative: e-safe is a web tracking service that records suspicious searches and URLs that students visit. When used in tandem, these tools were shown to have a marked impact on the conduct of students in using their laptops. This paper describes these initiatives including their effect on the broader school community, and suggests some ways in which student distraction can be best managed in future practice.
... Whilst learners found this a straightforward task, the technical infrastructure was not condu... more ... Whilst learners found this a straightforward task, the technical infrastructure was not conducive to being on ... Selecting learners with an interest in the content, and in learning flexibly is a sensible ... The inclusion of too many learners with low levels of motivation may create a peer ...
This paper provides some initial findings from a current longitudinal study that examines the imp... more This paper provides some initial findings from a current longitudinal study that examines the implementation of a student-owned 1:1 laptop program in a school for boys in Perth, Western Australia. This research tracks 196 students, their families and associated teachers for a 3-year period (2010-2012). Underpinning this research is a mixed methods approach investigating how boys use their laptops for learning, teachers’ pedagogy and use of ICT, implementation differences between a junior (primary) and middle (secondary) school, and possible impact of the laptops on learning. One theme that emerged from the first year of data collection was a decrease in parent satisfaction with the extent to which the educational objectives of the laptop initiative are being met. This paper explores possible reasons for this decline in satisfaction, focusing on parent and student perceptions of (a) the time spent on laptops and (b) the activities that students are seen to be engaging with on their l...
This paper provides some initial findings from a current longitudinal study that examines the imp... more This paper provides some initial findings from a current longitudinal study that examines the implementation of a student-owned 1:1 laptop program in a school for boys in Perth, Western Australia. This research tracks 196 students, their families and associated teachers for a 3-year period (2010-2012). Underpinning this research is a mixed methods approach investigating how boys use their laptops for learning, teachers’ pedagogy and use of ICT, implementation differences between a junior and middle school, and possible impact of the laptops on learning. One theme that emerged from the first two years of data collection was a decrease in parent satisfaction with the extent to which the educational objectives of the laptop initiative were being met. This paper explores possible reasons for this decline in satisfaction, focusing on parent and student perceptions of (a) the time spent on laptops and (b) the activities that students are seen to be engaging with on their laptops. These pe...
This exploratory research characterised the degree of adoption of mobile learning (ML) devices am... more This exploratory research characterised the degree of adoption of mobile learning (ML) devices among academic staff at an Australian university. It also sought to evaluate the impact of academics' perceptions about possibilities and constraints in the adoption of these technologies. A zone of free movement (ZFM) scale was developed and validated to quantify the magnitude and direction of those perceptions. Results showed that academic staff are characteristically at the third of the Russell's (1995) six developmental stages of technological adoption. Lack of time to integrate ML into courses, limited availability of mobile devices, little familiarity with the tools, as well as the perception that students cannot use them as a wordprocessor, act as inhibitors to the adoption of the technology. In turn, the perception that mobile tools enhance student-lecturer communication outside class was found to be a positive predictor of adoption.
Education Sciences, 2013
This qualitative study looked at the instructional, curricular and organizational factors impacti... more This qualitative study looked at the instructional, curricular and organizational factors impacting on the adoption of mobile learning in a higher education institution. Academics expressed their views on a variety of educational issues likely to enhance or hinder the effectiveness of the innovation. Teachers requested more professional development in a number of key areas including the integration of the technology into teaching and learning. Likewise, resolving vital issues such as workload, equity to access and effective policy making were seen as key to successful adoption. Technical issues such as having good wireless connectivity, need for efficient technical support, access to mobile devices and an understanding of their operational limitations with respect to desk/laptops were also highlighted. The study revealed a number of alternative perceptions and misconceptions, about articulating effective mobile learning pedagogies. For instance, staff expressed concerns about the risk of exposing students to superficial learning when mobile learning experiences were not well designed, the prospect that the devices might distract students from learning, as well as a possible deterioration of the quality of interaction between academics and students. Recommendations to reconcile those alternative conceptions with best practice principles are outlined.
The objective and authentic assessment of practical skills in tertiary settings is a challenge. T... more The objective and authentic assessment of practical skills in tertiary settings is a challenge. The purpose of this project was to evaluate the usefulness of a video analysis software program as a teaching and/or assessment tool for practical and professional skills in five different undergraduate programs. Barnett’s (1989) model of ‘reflective thought through action’ highlights the importance of reflective observation
The Topic Sevenoaks Senior College is an innovative new secondary school located in a lower socio... more The Topic Sevenoaks Senior College is an innovative new secondary school located in a lower socioeconomic suburb of Perth, Western Australian. This area has a high rate of unemployment and a low rate of further tertiary study by students leaving those schools (Wood & Trinidad, 2001). Sevenoaks opened its doors in February 2001 with a strong focus on industry and community partnerships. The driving force behind the college is its client perspective including strategies for customised learning so that programs are developed for each student available online, incorporate vocational learning and enterprise outcomes, use information technology and are contextualised to meeting individual learning rates and styles. As Wood and Trinidad (2001) state "Sevenoaks Senior College has been built on a clear vision of an adult and workplace oriented culture. It has a high degree of flexibility (operating from 8am to 5pm) with all students involved in Structured Workplace Learning (SWL), and a culture of enterprise where students are actively engaged in their own learning through the online curriculum and strong connections to the learning community that is crucial in providing young people with an environment where they feel valued and supported" (p.7).
Background: As an integral tool for clinical skill assessment, the quest to enhance the validity ... more Background: As an integral tool for clinical skill assessment, the quest to enhance the validity and reliability of OSCE scores should be ongoing. Summary of work: A new instrument with a wide spectrum of scales that allow objectivity and precision (i.e. checklist, process performance rating, and an overall global rating) was introduced, replacing the use of holistic numerical scoring. The new marking format was piloted and used in nine OSCE stations for first year students and 14 OSCE stations for second year students in 2010. It was also specifically designed for computerized scoring using Optical Mark Recognition software. Summary of results: Findings suggest that the new instrument is well accepted by examiners for its ease of use and enhanced objectivity. Empirical evidence also indicates better discrimination for students’ clinical skills, stronger internal consistency, greater inter- and intra-station reliability, and improved individual examiner marking consistency. Conclusi...
This paper provides some initial findings from a current longitudinal study that examines the imp... more This paper provides some initial findings from a current longitudinal study that examines the implementation of a 1:1 laptop program in a school for boys in Perth, Western Australia. One issue that has emerged from the study is the problem of managing student distraction. The school in this study has taken a proactive approach to managing student conduct on its own network. Two student monitoring initiatives were implemented during the course of the research. The first: parental control software sought to integrate the parental control features of the laptops with the school network. The second initiative: e-safe is a web tracking service that records suspicious searches and URLs that students visit. When used in tandem, these tools were shown to have a marked impact on the conduct of students in using their laptops. This paper describes these initiatives including their effect on the broader school community, and suggests some ways in which student distraction can be best managed i...