Evaluation of the role of near-peer teaching in critical appraisal skills learning: a randomized crossover trial

Introduction

To process research findings and other information, medical doctors need critical appraisal skills for careful and systematic assessment of the trustworthiness, value, and relevance of the available evidence in particular contexts. Medical students must, therefore, be prepared to learn and master these skills, and various approaches have been implemented to that end.^1-3 Within medical education, peer teaching, peer-assisted learning, peer tutoring, peer assessment, and other methods involving the deployment of students or trainees in a teaching role have attracted increasing interest.^4,5 Peer teaching can be defined as “an educational arrangement in which one student teaches one or more fellow students.” When the teacher is a student who is more advanced (by at least one year) in the same curriculum as the learner, this is known as “near-peer teaching,” which is one of the most common forms of peer teaching.⁴ Some studies have also classified recently graduated medical doctors as near-peer teachers by virtue of their proximity in age and experience.^6,7 Peer teaching and near-peer teaching methods have many advantages, which include alleviating faculty teaching burden, providing role models for junior students, enhancing students’ intrinsic motivation, and preparing physicians for their future role as educators.⁵ Although the effectiveness and efficiency of peer-assisted learning have been demonstrated in previous studies, these related mainly to a facilitation of psychomotor skills. Several of the published randomized controlled trials assessing the effectiveness of peer teaching relate to clinical skills such as intravenous (IV) access and bladder catheterization,⁸ neurological examination and lumbar puncture,⁹ resuscitation skills,¹⁰ interpretation of musculoskeletal ultrasound of the shoulder (as part of a sports medicine module),¹¹ and basic clinical examination skills.¹²

Among the limited number of studies comparing student-led tutorials (SLT) and faculty-led tutorials (FLT), Kassab and colleagues¹³ studied the use of problem-based learning (PBL) to teach haematology. Other non-randomized controlled studies of peer teaching of evidence-based medicine include Josephson et al. in the US¹⁴ and Rees and colleagues in the UK.¹⁵ However, both of those aimed solely to demonstrate the feasibility of such programs rather than to provide evidence of the effectiveness of peer teaching interventions. Sabouni and colleagues ¹⁶ also explored the utilization of peer teaching in an evidence-based medicine course in a resource-poor setting. Although the knowledge and skills of course participants were found to have improved significantly after the course, the study did not report any change in attitude. In addition, the pre-post study design is not the most robust means of determining the effectiveness of a given intervention.

Although the literature notes many benefits of peer-assisted learning, these are largely confined to the acquisition of psychomotor skills. In light of the growing need for innovative ways of incorporating EBM into medical school curricula, especially where human resources are limited, the present study aimed to evaluate the effectiveness of recently graduated medical doctors as near-peer tutors as compared to experienced medical staff. Specifically, we hypothesized that near-peer tutors would be as effective as medical staff in teaching critical appraisal skills to medical students as part of a Clinical Epidemiology and Evidence-based Medicine (CE-EBM) program.

Methods

Study design and participants

The study was designed as a randomized crossover trial. The participants were fourth-year medical students completing the CE-EBM module in the Faculty of Medicine Universitas Indonesia (FMUI) in the academic year 2014–2015. As the CE-EBM module is compulsory for fourth-year students, the entire cohort of 241 students participated. The module was conducted in two rotations: 116 students in rotation 1 (May–June 2015) and 125 students in rotation 2 (June–July 2015).  The crossover design meant that every student had an equal opportunity of being tutored by staff members or recently graduated medical doctors. The Research Ethics Committee of FMUI granted ethical clearance.

CE-EBM module and intervention

The content and structure of the CE-EBM module at FMUI has been described elsewhere.^17,18 Adapted from the University Medical Center Utrecht, the module is completed over four weeks. Informed by the principles of the Sicily statement on EBM teaching,¹⁹ the module includes lectures, tutored group discussions, and moderated plenary presentations on the design and conduct of diagnostic, therapeutic, prognostic, and etiologic studies, as well as computer-based literature search and data analysis.

Skills for critical appraisal of scientific papers were practiced during four 2-hour tutored group discussions. Journal articles on diagnosis, therapy, prognosis, or etiologic studies were provided by the module coordinators to ensure that the studies in question were well designed and could be appraised by students at this level. Led by a tutor, the participating students undertook a critical appraisal of the studies in their respective groups, using the worksheets provided by the Oxford Centre for EBM.²⁰ Prior to these discussions, the students attended corresponding lectures delivered by experts in clinical epidemiology and evidence-based medicine.

The voluntary near-peer tutors were recruited from FMUI’s newly graduated medical doctors, who had passed the CE-EBM module in their fourth year and had been practising EBM during their clerkship. The selected candidates were trained in a three-day training of teachers (TOT) that focused on tutoring skills and group dynamics. These new graduates were selected as near-peer tutors on the basis of their level and proximity in age to the fourth-year medical students. Five recently graduated medical doctors who were awaiting internship placement ultimately participated as near-peer tutors (of whom four participated in both rotations while one participated only in the first rotation).

The control groups were taught by experienced medical staff who had already participated in a two-day EBM course conducted by FMUI or in a three-day EBM course by Oxford CEBM and were practicing EBM in their clinical work. In total, 15 medical staff from various departments participated as tutors (seven in rotation 1 and eight in rotation 2), having attended a similar three-day TOT on EBM.

The students were randomly assigned to groups of 10 or 11. For the first two group discussion sessions, each group was randomly scheduled to be tutored either by medical staff or junior doctors. During the last two group discussion sessions, the groups were crossed over (see Figure 1). Randomization was implemented using computer software. As medical staff tutors and near-peer tutors were unequal in number, five groups did not participate in the crossover and were tutored by medical staff throughout.

Data collection

Evaluation of the teaching strategies was conducted in accordance with the Classification Rubric for Evidence-based Practice Assessment Tools in Education (CREATE) framework²¹ as shown in Figure 2.

Figure 1

The flow-chart of study

Student knowledge and skills were evaluated using a written test comprising 30 multiple choice questions (MCQ) with five options (similar to the Berlin questionnaire²² and Fresno test²³ as validated assessments of EBM skills). The CE-EBM faculty team developed the questions, based on predefined learning outcomes and in line with the national standard for medical education assessment in Indonesia to ensure the content and face validity of the test as an educational measurement tool. The test was conducted twice; the first (mid-test, after the first two discussion sessions) pertained to clinical question formulation, search, diagnosis, and prognosis, and the second (final test, after the last two sessions) related to therapy, etiology, and systematic review.

Evaluation of self-efficacy utilized a modified version of the Evidence-based Practice Confidence (EPIC) scale;²⁴ students rated their level of confidence in their ability to perform each evidence-based practice (EBP) activity on an 11-point scale ranging from 0% (cannot do at all) to 100% (certain can do). Item-level responses were averaged to obtain a summary score, which ranged from 0% to 100%. The evaluation was completed at the beginning and end of the module. Cronbach’s α was computed to assess the internal consistency of the EPIC questionnaire during baseline (pre-module) assessment. The pre-module Cronbach’s α value for the EPIC questionnaire was 0.95.

Attitudes were measured using part of the Knowledge Attitudes and Behaviors questionnaire developed by Johnston and colleagues,²⁵ which had previously been used to assess the first implementation of the CE-EBM module in FMUI and had been validated accordingly, with a Cronbach’s α value of 0.76.¹⁸ Students were asked to score each item on a 6-point Likert scale (from strongly agree to strongly disagree). The attitude questionnaire was also administered at the beginning and end of the module.

Figure 2

The Classification Rubric for EBP Assessment Tools in Education (CREATE) framework21 for the Faculty of Medicine Universitas Indonesia (FMUI) Clinical Epidemiology and Evidence-Based Medicine module evaluation

Finally, student educational experience was also evaluated, although EBP steps were not explicitly measured. This final questionnaire assessed more general aspects of the tutorial process as evaluated by students; it was previously utilized by Knobe and colleagues¹¹ to evaluate the effectiveness of peer-assisted teaching of technical skills for interpreting ultrasound images of the shoulder. Students rated their experience using a 5-point Likert scale, ranging from 1 (strongly disagree) to 5 (strongly agree). All evaluations were conducted after the first two discussion sessions and again after the last two sessions following crossover.

Data analysis

The primary outcome measure was the difference in mean score between intervention and control groups; independent t-testing was used to assess the significance of any discrepancies. For outcomes that differed significantly from baseline scores, analysis of covariance (ANCOVA) was utilized, using the baseline score as a covariate. The analysis was based on the intention-to-treat principle. All analyses were performed using SPSS version 22.0.

Results

The mean + SD age of participating students was 22.1+1.1 years (range 18.9–26.7 years), of whom 89 (36.9%) were male. The median age of near-peer tutors was 24 years (range 23–26 years). The median age of staff tutors was 41.5 years (range 37–55 years).

A mid-evaluation following the first two discussion sessions found no statistically significant difference (t₍₂₄₁₎ = -1.520, p = 0.130) between mean + SD of written test scores for the near-peer tutored group (65.89 + 12.86) and the staff-tutored group (63.18 + 13.69). The groups then exchanged tutors for the next two discussion sessions. At the end of the module, the difference in mean + SD of written test scores for the near-peer tutored group (68.20 + 12.50) and the staff-tutored group (70.61+10.99) was again statistically non-significant (t₍₂₄₁₎ = 1.553, p = 0.122).

At the beginning of the module, student attitude scores were comparable (Table 1). After the module, there were no statistically significant differences in the mean overall attitude score or for the majority of attitude items other than the statement “If evidence-based medicine is valid, then anyone can see patients and do what doctors do.”

The overall score for level of student confidence in Evidence-based Practice (based on the EPIC questionnaire) at the beginning of the module was also comparable for both groups (Table 2). However, a statistically significant lower score was observed for medical staff on two items (4 - Critically appraise the strengths and weaknesses of study methods, and 8 - Determine if evidence from the research literature applies to your patient's or client's situation). No statistically significant difference in mean EPIC scale score was observed either for individual items or overall scores at the end of the module. When the students were asked to evaluate the tutorial process, they assigned more positive scores to the near-peer tutor for the majority of evaluation items (Table 3).

Discussion

In the present study, near-peer tutored students and staff-tutored students achieved comparable scores on knowledge, skills, attitudes, and confidence in relation to critical appraisal skills. Although confirming what has been reported in earlier studies, these findings must nevertheless be interpreted with a degree of caution. For example, the similarity of these results, especially on the written test, may reflect the fact that all students attended lectures delivered by experts in clinical epidemiology. Another possible explanation (as suggested by ten Cate and colleagues)²⁶ is that peer-teaching may also encourage students to invest more effort in the study because of concerns about inadequate teaching.

Additionally, students scored better on the tutorials led by near-peer tutors than on those led by medical staff tutors. Tolsgaard and colleagues reported a similar result in a study employing second-year medical students as tutors to teach IV access and bladder catheterization to first-year medical students at the University of Copenhagen.⁸ As explained by cognitive congruence theory, a teacher whose knowledge base is similar (i.e., congruent) to the learner’s is likely to be more effective than an expert in the field who has a dissimilar knowledge base (i.e., who is cognitively incongruent or is perceived as more “cognitively distant”). Peer-tutors are also believed to be able to provide a more comfortable and safe learning environment for students, as they come from a similar social environment.⁵

A systematic review by Burgess and colleagues²⁷ indicated that practical issues such as limited faculty resources were most frequently reported to account for the implementation of peer or near-peer teaching. Another justification for the use of near-peer tutors is the need to develop the teaching skills of future doctors. While the Indonesian Medical Council does not explicitly list teaching skills among the competencies of medical doctors, Indonesian medical graduates are expected to be able to provide health education to patients, families, and communities.²⁸ In Tomorrow’s Doctors, the UK General Medical Council states that medical graduates must be able to demonstrate appropriate teaching skills,²⁹ and the Royal College of Physicians and Surgeons of Canada states in its CanMEDS Physicians Competency framework that the physician’s role includes being a “scholar” who are competent to “teach students, residents, the public, and other health care professionals”.³⁰

As well as the benefits for the medical students, the peer tutors also gained, as the teaching experience motivated them to learn more and to deepen their understanding of the subject. This is an expected aspect of EBM skills, which are currently seen as one of the core competencies of medical doctors. Experience in teaching and preparation of teaching materials is also known to build leadership capability and confidence.⁵

Although peer teaching has been widely implemented in medical education, we are not aware of any previous studies of this kind from Indonesia, although in our experience the practice is undoubtedly performed informally. We recognize that as educational interventions are generally complex, it is often difficult to distinguish which element accounts for which effect or to what extent it might be adapted to local circumstances while remaining effective.³¹ Moreover, certain particulars may limit the generalization of the present findings, such as student characteristics and EBM as a distinctive subject. As the oldest and most prominent medical school in Indonesia, FMUI students are a select group, and earlier research found that tutor expertise did not affect FMUI students’ performance during problem-based learning.³² Furthermore, the ethics of medical education require the minimization of any potential disadvantage to students assigned to a near-peer tutor – hence the need for a crossover. As a result, none of the students in this study were assigned only to a near-peer tutor, and this may have led to over-estimation of results, especially in the final test. Future studies should, therefore, extend the utilization of near-peer tutors to all tutorial sessions.

Conclusions

The present findings confirm that near-peer tutors can be as effective as and more readily accepted than staff tutors. This is based on the performance of students in critical appraisal skills in EBM, extending the evidence of the benefits of peer-assisted learning beyond the area of psychomotor skills. This study also provides preliminary evidence that medical school educational policy will be accommodated to the assignment of trained near-peer tutors to teach critical appraisal skills to medical students.

Acknowledgements

We would like to express our gratitude to the Dean, the Vice Dean for Education, Research, and Student Affairs and the Manager of Education and Student Affairs at FMUI for lending their full support to this study. The study received full financial support from the Medical Education Unit of FMUI.

Conflict of Interest

The authors declare that they have no conflict of interest.

References

1. Rezaeian M. How to teach medical students to critically appraise a published article in the public health domain. J Educ Health Promot. 2013; 2: 8.
   Full Text PubMed
2. Ilic D and Maloney S. Methods of teaching medical trainees evidence-based medicine: a systematic review. Med Educ. 2014; 48: 124-135.
   Full Text PubMed
3. Maggio LA, Cate OT, Irby DM and O'Brien BC. Designing evidence-based medicine training to optimize the transfer of skills from the classroom to clinical practice: applying the four component instructional design model. Acad Med. 2015; 90: 1457-1461.
   Full Text PubMed
4. Ten Cate O and Durning S. Dimensions and psychology of peer teaching in medical education. Med Teach. 2007; 29: 546-552.
   Full Text
5. Ten Cate O and Durning S. Peer teaching in medical education: twelve reasons to move from theory to practice. Med Teach. 2007; 29: 591-599.
   Full Text
6. Rashid MS, Sobowale O and Gore D. A near-peer teaching program designed, developed and delivered exclusively by recent medical graduates for final year medical students sitting the final objective structured clinical examination (OSCE). BMC Med Educ. 2011; 11: 11.
   Full Text PubMed
7. Mabvuure NT, Rodrigues J, Cumberworth A and Mahmud M. Twelve tips for running successful junior doctor-led teaching programmes for medical students. Med Teach. 2013; 35: 628-632.
   Full Text PubMed
8. Tolsgaard MG, Gustafsson A, Rasmussen MB, Høiby P, Müller CG and Ringsted C. Student teachers can be as good as associate professors in teaching clinical skills. Med Teach. 2007; 29: 553-557.
   Full Text PubMed
9. Heckmann JG, Dütsch M, Rauch C, Lang C, Weih M and Schwab S. Effects of peer-assisted training during the neurology clerkship: a randomized controlled study. Eur J Neurol. 2008; 15: 1365-1370.
   Full Text PubMed
10. Hughes TC, Jiwaji Z, Lally K, Lloyd-Lavery A, Lota A, Dale A, Janas R and Bulstrode CJ. Advanced Cardiac Resuscitation Evaluation (ACRE): a randomised single-blind controlled trial of peer-led vs. expert-led advanced resuscitation training. Scand J Trauma Resusc Emerg Med. 2010; 18: .
    Full Text PubMed
11. Knobe M, Münker R, Sellei RM, Holschen M, Mooij SC, Schmidt-Rohlfing B, Niethard FU and Pape HC. Peer teaching: a randomised controlled trial using student-teachers to teach musculoskeletal ultrasound. Med Educ. 2010; 44: 148-155.
    Full Text PubMed
12. Blank WA, Blankenfeld H, Vogelmann R, Linde K and Schneider A. Can near-peer medical students effectively teach a new curriculum in physical examination? BMC Med Educ. 2013; 13: 165.
    Full Text PubMed
13. Kassab S, Abu-Hijleh MF, Al-Shboul Q and Hamdy H. Student-led tutorials in problem-based learning: educational outcomes and students' perceptions. Med Teach. 2005; 27: 521-526.
    Full Text PubMed
14. Josephson SA and Whelan AJ. A new first-year course designed and taught by a senior medical student. Acad Med. 2002; 77: 1207-1211.
    PubMed
15. Rees E, Sinha Y, Chitnis A, Archer J, Fotheringham V and Renwick S. Peer-teaching of evidence-based medicine. Clin Teach. 2014; 11: 259-263.
    Full Text PubMed
16. Sabouni A, Bdaiwi Y, Janoudi SL, Namous LO, Turk T, Alkhatib M, Abbas F and Yafi RZ. Multiple strategy peer-taught evidence-based medicine course in a poor resource setting. BMC Med Educ. 2017; 17: 82.
    Full Text PubMed
17. Widyahening IS, van der Heijden GJ, Moy FM, van der Graaf Y, Sastroasmoro S and Bulgiba A. From west to east; experience with adapting a curriculum in evidence-based medicine. Perspect Med Educ. 2012; 1: 249-261.
    Full Text PubMed
18. Widyahening IS, van der Heijden GJ, Moy FM, van der Graaf Y, Sastroasmoro S and Bulgiba A. Direct short-term effects of EBP teaching: change in knowledge, not in attitude; a cross-cultural comparison among students from European and Asian medical schools. Med Educ Online. 2012; 17: 19623.
    Full Text PubMed
19. Dawes M, Summerskill W, Glasziou P, Cartabellotta A, Martin J, Hopayian K, Porzsolt F, Burls A and Osborne J. Sicily statement on evidence-based practice. BMC Med Educ. 2005; 5: 1-127.
    Full Text PubMed
20. Center for Evidence Based Medicine. Center for Evidence Based Medicine. Oxford: University of Oxford; 2013 [cited 7 March 2014]; Available from: www.cebm.net.
21. Tilson JK, Kaplan SL, Harris JL, Hutchinson A, Ilic D, Niederman R, Potomkova J and Zwolsman SE. Sicily statement on classification and development of evidence-based practice learning assessment tools. BMC Med Educ. 2011; 11: 78.
    Full Text PubMed
22. Fritsche L, Greenhalgh T, Falck-Ytter Y, Neumayer HH and Kunz R. Do short courses in evidence based medicine improve knowledge and skills? Validation of Berlin questionnaire and before and after study of courses in evidence based medicine. BMJ. 2002; 325: 1338-1341.
    PubMed
23. Ramos KD, Schafer S and Tracz SM. Validation of the Fresno test of competence in evidence based medicine. BMJ. 2003; 326: 319-321.
    PubMed
24. Salbach NM and Jaglal SB. Creation and validation of the evidence-based practice confidence scale for health care professionals. J Eval Clin Pract. 2011; 17: 794-800.
    Full Text PubMed
25. Johnston JM, Leung GM, Fielding R, Tin KY and Ho LM. The development and validation of a knowledge, attitude and behaviour questionnaire to assess undergraduate evidence-based practice teaching and learning. Med Educ. 2003; 37: 992-991000.
    PubMed
26. ten Cate O, van de Vorst I and van den Broek S. Academic achievement of students tutored by near-peers. Int J Med Educ. 2012; 3: 6-13.
    Full Text
27. Burgess A, McGregor D and Mellis C. Medical students as peer tutors: a systematic review. BMC Med Educ. 2014; 14: 115.
    Full Text PubMed
28. Indonesian Medical Council. Standard of medical doctors competencies. Jakarta: Indonesian Medical Council; 2012.
29. General Medical Council. Tomorrow's doctors. London: General Medical Council; 2009.
30. Frank JR, Snell L, Sherbino J. CanMEDS 2015 Physician competency framework. Ottawa: Royal College of Physicians and Surgeons of Canada; 2015.
31. Murray E. Challenges in educational research. Med Educ. 2002; 36: 110-112.
    Full Text
32. Wahid M, Kumara A, Savitri T, Rahayu GR, Werdhani RA. Students' characteristics related to their performances in problem-based learning. South East Asian Journal of Medical Education. 2015;9(2):50-55.