Students’ perception of problem-based learning at a Japanese medical school: an exploratory sequential mixed method

Introduction

After the introduction of problem-based learning (PBL) into medical education at McMaster University in Canada in the 1960s,¹ PBL has been applied to medical education curricula around the world. Previous studies reported that PBL is advantageous for knowledge retention, attaining problem-solving abilities, and integration of basic science and clinical medicine.^2,3 Moreover, PBL is assumed to enhance social learning and promote interpersonal communication and presentation skills.^4-6 In addition, it was reported that PBL encourages self-directed learning and increases student’s enthusiasm and motivation for learning.⁷ On the other hand, factors affecting the effectiveness of PBL programs have been reported.^8,9 These include the quality of case scenarios at tutorials, tutor’s tutoring ability, active participation of student-peers, and the process of self-learning.

Mie University Faculty of Medicine in Japan introduced the concept of PBL into the 6-year medical education curriculum in 1995. The original program was planned as PBL tutorials for the 3rd to 4th year students in pre-clinical education. However, it faced some problems such as student’s motivation for self-learning and a shortage of tutors. The staffing issue became more serious in association with the increasing number of students. Furthermore, some faculty members and students complained about the replacement of conventional lectures with PBL tutorials. To solve these issues, the pre-clinical education program was amended to operate with limited resources. In addition, it aimed to match the preference of the faculties for lectures although it may be criticized for deviating from authentic PBL tutorials. The program was modified to be composed of self-learning, PBL tutorials, team-based learning (TBL), and lectures. The modified program has continued for more than 10 years. However, the program may have weaknesses in facilitating active learning of the students such as tutors feeling that students have a passive attitude at PBL tutorials. A previous study conducted during the introductory period of PBL in Asia reported the incompatibility of Asian cultural attitudes with active learning methods.¹⁰ The study reported characteristics of Asian students, such as disliking open criticism and lacking motivation to ask questions, that made the use of such methods difficult. A recent article from Japan reported that some Japanese medical students may be reluctant to shift towards active learning because they have become accustomed to passive learning since childhood.¹¹

To clarify learning effects and challenges of a PBL program conducted in parallel with lectures for Japanese medical students, we studied students’ perception of the program. Because there are few reports focusing on this issue, we adopted an exploratory sequential mixed method for the analysis. This methodology provides a broader spectrum of ways to understand students’ perception in different contexts than those that could be done through either quantitative or qualitative approaches alone.¹² Our findings may contribute to the improvement of pre-clinical education at medical schools with similar problems in Japan and other Asian countries.

Methods

Medical education curriculum

At Mie university, medical students join liberal arts education and community-based education in their 1st and 2nd years. In these years, they also learn basic medical sciences by attending lectures. In the 2nd and 3rd years, they participate in practical trainings in basic sciences. Afterwards, students move to a preclinical education curriculum composed of PBL tutorials, TBL, lectures, clinical skill training, and laboratory internship in the 3rd and 4th years. Then the students participated in clinical clerkship for 70 weeks.

PBL program

In the PBL program, a total of 125 students were allocated into 16 groups. Each group consisted of seven to eight students who took part in collaborative learning. They were requested to practice self-directed learning before and after group work. The PBL program contains 10 units, which were organized according to organ systems. A single unit was carried out for 4 to 6 weeks. The units were organized into three types of group studies: PBL tutorials (Units 2, 3, 6, and 8), mixture of PBL tutorials and TBL (Units 1, 4, 5, and 7), and exclusively TBL (Units 9 and 10). In addition, lectures related to the topics of the PBL programs were given. A PBL tutorial session was facilitated by a tutor for each group. A TBL session was carried out without the tutor facilitation. Instead, one to two teachers guided a whole-class activity in a single classroom, and students dealt with tasks at a single table with members of their group. The formal PBL tutorial had two sessions per week for analyzing and reporting. Each session time was set to 90 minutes. Students were sequentially provided with a patient scenario containing information on medical interview, physical examination, laboratory tests, and therapeutic intervention. Through discussions at each tutorial, students were expected to improve skills of problem distillation, problem-solving and clinical reasoning. Students were evaluated with a portfolio submitted in every group session, a paper test, student-and-peer assessment, and tutor's assessment at the end of each unit. The peer and tutor assessments were carried out in three domains of learning: self-learning in preparation, active participation in group discussions, and cooperative attitude in group work. A tutor attending the tutorials evaluated all students in the group. The tutors also provided formative feedback to the students. TBL consisted of one session of 120 minutes a week. The supervising teacher gave the students learning topics, and then they had a group discussion on the topic. After the discussion, several groups presented a summary of their discussions at a plenary session, which was followed by a small lecture by the supervising teacher. Students were evaluated for their learning achievements similarly to the PBL tutorials, but a tutor assessment was not included in TBL.

Study design

This study employed an exploratory sequential mixed method to understand students’ perception systematically by integrating qualitative and quantitative findings. In the first phase, qualitative data collection and analysis were performed using focus group discussions (FGDs) and a thematic analysis. Based on the results, we explored students’ perception of PBL and identified specific themes. Then we developed a questionnaire and conducted a questionnaire survey in the second phase to measure the students’ perception quantitatively. This process corresponded to “the building approach” of the exploratory sequential mixed method. Furthermore, “the merging approach” was applied through integration and visualization of the findings at each phase.^13,14

FGDs were performed in 2019. FGDs were held in four groups comprising of students or residents who joined the PBL program in the same academic year. All participants were Japanese. The participants were requested to discuss their experiences of the PBL program at Mie University Faculty of Medicine. The research questions at the first phase were: 1) outcome of PBL, 2) application of PBL outcome, 3) challenges of PBL, and 4) suggestions for better PBL. The actual questions to the participants were as follows: 1) What ability and knowledge have you obtained from the PBL program? 2) How would you apply the ability and knowledge that you obtained through the PBL program to your later academic life? 3) What challenges and difficulties of the PBL program did you face? 4) What advice would you give to make the PBL programs better? The group discussion was facilitated by an investigator who was not involved in the management of the PBL program. The discussion lasted about 90 minutes for each group. The discussion was recorded using a voice recorder and transcribed verbatim into written documents. The data were anonymized throughout the process. The discussions were carried out in Japanese. The textual documents were translated from Japanese to English, reviewed by three investigators and then analyzed. Thematic analysis was conducted based on the six-phase framework by Baum and Clarke.^15,16 The words and phrases used in the discussion were coded with Atlas.ti MacOS Software Version 8.3. Two investigators reviewed the result and defined the themes.

To quantitatively assess the students’ perception of PBL, a cross-sectional questionnaire survey was conducted among 5th and 6th year students in 2019. The research questions at the second phase were developed based on the results of the thematic analysis. A self-administered anonymous questionnaire was distributed by hand and by e-mail if requested. The answered questionnaire was posted in a collection box in the classroom. We analyzed responses posted within two months from distribution. A questionnaire constructed using the results of the qualitative study was used in the survey. The questionnaire was composed of several categories defined according to the research questions at the second phase. The collected data were analyzed for each question and category. The information of student characteristics was also collected with the survey, which included sex, age, school year, and the result of the computer-based test (CBT). CBT was carried out just after the PBL program by the Common Achievement Tests Organization in Japan for the qualification of entering clinical clerkship. Students self-reported their rank of CBT among their classmates. The questionnaire was produced in English at first, translated into Japanese, and validated by counter-translation into English before the survey. Students responded to 5-point Likert scale questions. The definitions of each point in the scale were as follows: 1, strongly disagree; 2, disagree; 3, neither agree nor disagree; 4, agree; and 5, strongly agree. For a question item with negative meaning, reversed scoring was adopted for data analysis. The responses to each question in the questionnaire survey were divided into three classifications: strongly agree/agree, neutral, or strongly disagree/disagree in the analytical process. The result was tested for independence with a contingency table, and then a residual analysis in the contingency table was conducted if any significant difference was noted in the test for independence. The internal reliability of question items was assessed with 20 students attending PBL tutorials before the survey.

Study participants

This study was composed of a qualitative analysis phase with FGDs followed by a quantitative analysis phase with a questionnaire survey. For FGDs, the 4th to 6th year students and junior residents who had completed the PBL program at Mie University Faculty of Medicine were recruited to form a group with four to eight persons. The participants joined the study voluntarily.

A questionnaire survey was conducted to 258 students, 126 5th year and 132 6th year students. A total of 119 out of 258 students responded to the survey (Table 1). The response rate was 46.1%. The female to male ratio was 1:2.5, corresponding with the sex ratio in the whole student body. As for the age composition of respondents, 79.8% were aged 22-25 years, and 20.2% were aged 26 years and above. We further divided the subjects by CBT rank in class. The number of students was similarly distributed as in the whole student body when divided into five groups by every 25 ranks.

This study was approved by the Institutional Review Board at Mie University Faculty of Medicine. The study plan was open to the public on the homepage of Mie University. In the qualitative study with FGDs, informed consent was obtained from the participants before group discussions. The documents applied to the qualitative study were completely anonymized. In the questionnaire survey, an informed consent form was provided to students together with a questionnaire. The answered questionnaire was returned to the investigator if they agreed to participate. The questionnaire survey was anonymous.

Statistics

The questionnaire constructed in this study was validated for internal reliability with Cronbach’s alpha. An item-total correlation was analyzed with Spearman’s correlation coefficient between the total scores of responses to all questions in each category and all categories. The data in a contingency table were statistically tested for independence with two-sided chi-square test, and then examined with a residual analysis if any significant difference was noted in the test for independence. Significance in the residual analysis was indicated with a p-value for the adjusted standardized residual (ASR).¹⁷ A p-value less than .05 was regarded as significant where a p-value was estimated from the standard normal distribution. A total score of responses to all questions in each category was calculated and indicated as a percentage to the maximum score (the number of questions x5). Statistical analyses were carried out using IBM-SPSS Statistics 24.0 (IBM Corp., Armark, NY, USA) and BellCurve for Excel (Social Survey Research Information Co. Ltd., Tokyo, Japan).

Results

Qualitative study

Four groups were organized with a total of 27 students and residents. Eight medical students in the 4th year (male/female = 4/4), eight (male/female = 4/4) in the 5th year, seven in the 6th year (male/female = 4/3), and four junior residents (male/female = 3/1) working at Mie University Hospital participated in the FGDs. The four groups in the FGDs were organized according to the academic year when the participants joined the PBL program. The textual documents were analyzed with an inductive thematic analysis for each research question at the first phase. We derived three themes from 17 codes in FGDs for the outcome of PBL: effective learning, cognitive thinking, and social skills (Table 2ََA). Discussions on the application of PBL outcome identified three themes from 12 codes: social interaction, continuing learning, and clinical practice (Table 2B). Discussions on challenges of PBL identified five themes from 28 codes: learning resources, assessment, student’s learning behaviors, limitations of application, and reflection of student perspectives (Table 2C). Moreover, discussions on suggestions for better PBL derived three themes from 17 codes: quality of tutors, quality of case-scenarios, and innovation of learning strategy (Table 2D).

Along with the concept of an exploratory sequential mixed method, the phrases specific to each category in the qualitative analysis and the results of the quantitative analysis were indicated as “a joint display” in Table 3. Students acknowledged that the PBL program was effective for acquiring skills of communication, presentation, collaborative learning, cognitive thinking, and clinical reasoning. They also regarded the PBL program as a better learning method than lectures. The following comment was a good example: “Ability of summarizing my idea and speaking precisely would not be improved by learning at lectures while that was improved at PBL tutorials. I believe that such ability is useful after becoming a doctor.” However, some students had hesitation in stating their own opinion before group members and experienced difficulty getting along with other students at PBL tutorials. The following comments showed one student’s thoughts: “I always worried about saying something wrong about my learning achievements.” “I thought about how to get along well with others every time at tutorials.” Accordingly, some students seemed to attach a certain value to lectures. The following comments were given at FGDs: “At Mie University, the general knowledge is learned in lectures, and the symptomology of individual diseases are learned in PBL tutorials.” “It might be nice to have a preparatory lecture at the beginning of a PBL tutorial. If I received a summary lecture, it could be effective.” Furthermore, one student gave the following comment: “PBL works well in Canada, but we need to modify it for Japanese students”, which suggests that the student felt a barrier to effective PBL among Japanese medical students. In the discussions on challenges of PBL and suggestions for PBL, quality of tutoring, case scenarios, group discussions, and assessments were recognized as issues to be improved. Student’s learning behavior was also discussed. The participants of the FGDs commented that there were problems in active participation in PBL tutorials and proper usage of learning resources.

In general, the residents gave more supportive comments for PBL while the students presented more critical opinions. Especially, the 4th year students joining the PBL program at the time of FGDs tended to prefer a lighter workload than a better learning outcome. In this regard, a student commented that “the purpose of PBL was not clearly indicated to students”. In addition, the learning effects of PBL for clinical practice were likely to be recognized after they had become physicians.

Questionnaire survey

We developed four research questions in the second phase based on the results of the first phase. These questions consisted of 1) learning effects of PBL, 2) social interaction during and after PBL, 3) application of PBL experiences to clinical clerkship, and 4) satisfaction with PBL. Then we generated 24 questions for the questionnaire survey and categorized these questions into four categories according to the research questions in the second phase (Appendix). Before the survey, the internal reliability of 24 questionnaire items was validated among 20 students. The value of Cronbach’s alpha was .875, indicating a good reliability index. The value was also assessed with the 119 students that responded to the questionnaire survey. The value of Cronbach’s alpha was .932. An analysis of the item-total correlation resulted in statistically significant correlations for all of four categories, learning effects of PBL (r = .836, n = 112, p < .001), social interaction during and after PBL (r = .882, n = 112, p < .001), application of PBL experiences to clinical clerkship (r = .868, n = 112, p < .001) and satisfaction of PBL (r = .674, n = 112, p < .001) (Table 3).

The results of the questionnaire survey are summarized in Table 4. The answers to 24 questions were classified to “strongly agree/agree”, “neutral”, or “strongly disagree/disagree”. A chi-square test of independence found the significant difference in the contingency table (χ² (48, 2848) = 398.3, p < .001). The ASR in the residual analysis for each category of the responses to each question was calculated. A significantly higher ASR for the response of “strongly agree/agree” was found in four out of six questions of Category I, learning effects of PBL (Q1, ASR = 5.901, n = 119, p < .001; Q2, ASR = 2.846, n = 118, p = .004; Q4, ASR = 3.303, n = 119, p < .001; Q6, ASR = 3.591, n = 118, p < .001) and three out of seven questions of Category IV, satisfaction with PBL (Q20, ASR = 4.231, n = 119, p < .001; Q21, ASR = 2.190, n = 119, p = .029; Q22, ASR = 4.046, n = 119, p < .001). A significantly lower ASR for the response of “strongly agree/agree” was found in five out of six questions of Category II, social interaction during and after PBL (Q7, ASR = -3.303, n = 118, p < .001; Q8, ASR = -2.078, n = 119, p = .038; Q9, ASR = -3.006, n = 119, p = .003; Q10, ASR = -2.449, n = 119, p = .014; Q11, ASR = -5.790, n = 119, p < .001). Among five questions of Category III, application of PBL experiences to clinical clerkship, two had a significantly higher ASR for the response of “strongly agree/agree” (Q14, ASR = 4.973, n = 119, p < .001; Q17, ASR = 4.417, n = 119, p < .001) and another two had a significantly lower ASR (Q13, ASR = -4.657, n = 116, p < .001; Q16, ASR = -5.975, n = 119, p < .001).

Table 3 shows a joint display of the results from qualitative and quantitative analyses in an exploratory sequential method. The percentage of total scores to the maximum value in each category is indicated. In accordance with the results of the residual analysis for all questions, Categories I (learning effects of PBL) and IV (satisfaction with PBL) showed higher values (68.9 and 67.9%) and Category II (social interaction during and after PBL) had the lowest value (57.7%). These findings mostly reflected the results of the qualitative study; however, opposite views were observed in the qualitative study.

Discussion

In our curriculum, students in the preclinical years attend the three different types of classes: PBL tutorials, TBL, and large lecture classes. In this study, we aimed to clarify the students’ perception of PBL in this situation using an exploratory sequential mixed method. Furthermore, we aimed to elucidate the challenges of PBL for Japanese medical students.

Students appreciated that PBL was an effective learning strategy. A positive response was significantly higher for the question on self-learning skills and clinical reasoning skills. These findings are consistent with previous studies.^18-22

However, the positive response was not significantly higher for the question on critical thinking ability. This may be related to the learning attitude of the students as discussed later. Pre-clinical education is expected to foster the abilities needed for clinical clerkship. But this study showed that there were arguments for and against on the application of PBL experiences to clinical clerkship among students. Significantly more students disagreed with the question “PBL tutorials were instructive for me to improve my study-style”. Most interestingly, social interaction in PBL was not perceived affirmatively as there were significantly fewer students who agreed that PBL improved communication skills, responsibility and accountability at group discussion, and social networking. Moreover, the qualitative study indicated the passive attitude to group work and preference for the traditional style of learning in some students. A previous study from a Japanese medical school reported that the majority of students were satisfied with PBL; however, 5% preferred traditional lecture-style learning.²³ In Japan, students may not have had exposure to learning through open discussion, debating, or group work since childhood. Moreover, they are also not familiar with constructive criticism. Such a learning culture may influence Japanese students’ attitude for learning. To improve this situation, we should emphasize the importance of active learning in medical education and provide some introductory sessions before the PBL program.

Previous studies reported that PBL achieved better learning outcomes in various aspects.^4-6 However, the findings in the present study were not completely consistent with these previous studies. The passive attitude of students may be associated with these negative findings. The qualitative study extracted other problems in the quality of tutoring, case scenarios, and assessments. Davis et al. reported that in small-group teaching sessions, student groups facilitated by content-experts had higher levels of satisfaction and higher examination scores than those led by non-experts.²⁴ The selection of tutors is one of the critical components of PBL; however, content-experts are not always available due to the limited resources of teaching staff. We should enhance the tutoring ability of teachers effectively through faculty development. For better case scenarios, the participants of FGDs suggested the use of visual materials and simulated patients. The production of case scenarios that are attractive to students is critical for effective PBL. An effective assessment is also important to facilitate student’s learning. The results of FGDs suggested that students looked for a fair assessment and timely feedback by tutors. The assessment system should be continuously improved with feedback from students; however, previous studies reported that the majority of students were not satisfied with the evaluation system in medical education curriculum.^25,26

The findings in the present study need to be considered in the context of its limitations. FGDs were conducted by students and residents participating voluntarily but not randomly assigned. Thus, this is likely to have resulted in bias towards choosing more active students.

The respondents in the questionnaire survey may also have such characteristics. The collection rate at the survey was relatively low; however, the ratio of respondents by school year, sex, or age composition reflected the ratios of the whole student body. In addition, the CBT rank divided by every 20% matched the actual distribution. Thus, we assumed that the participants represented the student cohort planned in the study design. The present study was a cross-sectional study; therefore, we could not assess the change of students’ perception longitudinally with the progress of their experiences in clinical practice.

For the qualitative study, the results could be influenced by the investigators’ values and beliefs although we tried to avoid subjective decisions. Furthermore, as the medical education curriculum varies from school to school, the findings in this study may not be applicable to other schools.

Conclusion

PBL was perceived as an effective learning strategy in the pre-clinical curriculum of a Japanese medical school. However, some medical students had problems with social interaction, which made it difficult for them to actively participate in group work. Moreover, some students preferred lectures even if the passive learning approach produced less learning effects. Although students were generally satisfied with PBL, the qualitative study found dissatisfaction with the quality of tutoring, case scenarios, and assessments.

Acknowledgements

The study was supported by an official budget from Mie University Graduate School of Medicine. The fund covered all costs related to the study.

Conflict of Interest

The authors declare that they have no conflict of interest.

Supplementary materials

References

1. Barrows HS, Tamblyn RM. Problem-based learning: an approach to medical education. New York: Springer; 1980.
2. Hmelo-Silver CE. Problem-based learning: what and how do students learn? Educational Psychology Review. 2004;16:235-66.
3. Yaqinuddin A. Problem-based learning as an instructional method. J Coll Physicians Surg Pak. 2013; 23: 83-85.
   PubMed
4. Yew EH and Schmidt HG. Evidence for constructive, self-regulatory, and collaborative processes in problem-based learning. Adv Health Sci Educ Theory Pract. 2009; 14: 251-273.
   Full Text PubMed
5. Wun YT, Tse EY, Lam TP and Lam CL. PBL curriculum improves medical students' participation in small-group tutorials. Med Teach. 2007; 29: 198-203.
   Full Text PubMed
6. Khalid BA. The current status of medical education in the Gulf Cooperation Council countries. Ann Saudi Med. 2008; 28: 83-88.
   Full Text PubMed
7. Kaufman DM and Mann KV. Comparing students' attitudes in problem-based and conventional curricula. Acad Med. 1996; 71: 1096-1099.
   Full Text PubMed
8. Van Berkel HJ and Dolmans DH. The influence of tutoring competencies on problems, group functioning and student achievement in problem-based learning. Med Educ. 2006; 40: 730-736.
   Full Text PubMed
9. Dolmans DH and Schmidt HG. What drives the student in problem-based learning? Med Educ. 1994; 28: 372-380.
   Full Text PubMed
10. Khoo HE. Implementation of problem-based learning in Asian medical schools and students' perceptions of their experience. Med Educ. 2003; 37: 401-409.
    Full Text PubMed
11. Iwata K and Doi A. Can hybrid educational activities of team and problem based learning program be effective for Japanese medical students? Kobe J Med Sci. 2017; 63: 51-57.
    PubMed
12. Creswell JW, Plano Clark VL. Designing and conducting mixed methods research. Los Angeles, CA: SAGE publications; 2018.
13. Fetters MD, Curry LA and Creswell JW. Achieving integration in mixed methods designs-principles and practices. Health Serv Res. 2013; 48: 2134-2156.
    Full Text PubMed
14. Guetterman TC, Fetters MD and Creswell JW. Integrating quantitative and qualitative results in health science mixed methods research through joint displays. Ann Fam Med. 2015; 13: 554-561.
    Full Text PubMed
15. Braun V, Clarke V. Using thematic analysis in psychology. Qualitative Research in Psychology. 2006;3 (2):77-101.
16. Maguire M, Delahunt B. Doing a thematic analysis: a practical, step-by-step guide for learning and teaching scholars. All Ireland Journal of Higher Education. 2017;9:3351-14.
17. Haberman SJ. The analysis of residuals in cross-classified tables. Biometrics. 1973;29:205-20.
18. Emerald NM, Aung PP, Han TZ, Yee KT, Myint MH and Soe TT, et al. Students' perception of problem-based learning conducted in phase 1 medical program, UCSI University, Malaysia. South-East Asian Journal of Medical Education.. 2013; 7: 45-48.
    Full Text
19. Yan Q, Ma L, Zhu L and Zhang W. Learning effectiveness and satisfaction of international medical students: Introducing a Hybrid-PBL curriculum in biochemistry. Biochem Mol Biol Educ. 2017; 45: 336-342.
    Full Text PubMed
20. Musal B, Taskiran C and Kelson A. Opinions of tutors and students about effectiveness of PBL in Dokuz Eylul University School of Medicine. Med Educ Online. 2003; 8: 4345.
    Full Text PubMed
21. Alkhuwaiter SS, Aljuailan RI and Banabilh SM. Problem-based learning: Dental student's perception of their education environments at Qassim University. J Int Soc Prev Community Dent. 2016; 6: 575-583.
    Full Text PubMed
22. Callis AN, McCann AL, Schneiderman ED, Babler WJ, Lacy ES and Hale DS. Application of basic science to clinical problems: traditional vs. hybrid problem-based learning. J Dent Educ. 2010; 74: 1113-1124.
    PubMed
23. Iwatsuki J, Kondo T, Takahashi N, Takami H, Nishigori H, Bustos-Villalobos I, Aleksic B, Kasuya H, Ban N, Yagi T and Skokauskas N. Problem-based learning in child and adolescent psychiatry: a perspective from Japan. Adv Med Educ Pract. 2021; 12: 1329-1335.
    Full Text PubMed
24. Davis WK, Nairn R, Paine ME, Anderson RM and Oh MS. Effects of expert and non-expert facilitators on the small-group process and on student performance. Acad Med. 1992; 67: 470-474.
    Full Text PubMed
25. Al-Shaikh G, Al Mussaed EM, Altamimi TN, Elmorshedy H, Syed S, Habib F. Perception of medical students regarding problem based learning. Kuwait Medical Journal. 2015;47:133-8.
26. Shamsan B and Syed AT. Evaluation of problem based learning course at college of medicine, qassim university, saudi arabia. Int J Health Sci (Qassim). 2009; 3: 249-258.
    PubMed