Assessing the effectiveness of a digital, case-based learning platform for cancer pain management in residency training

Introduction

During the COVID-19 pandemic, many medical school classes were changed to case-based online learning because a limited number of patients were available and because of concerns regarding the students' safety.¹ Additionally, many providers who would normally teach trainees were diverted to the provision of health care during the outbreak. A self-directed e-learning (SDL) platform allows students to learn at their own pace and convenience. Case-based learning (CBL) is a problem-based method that can help students connect knowledge and medical practice, develop professional skills, and enhance higher-order thinking.²  Compared to traditional teaching methods, CBL improves knowledge acquisition, problem-solving skills, and learner satisfaction.³ Moreover, SDL using CBL in graduate medical education improves knowledge, clinical skills and critical thinking skills to a similar degree as didactic teaching.^4-8 A systematic review that included 51 studies of 6,750 students in medicine, nursing, dentistry, physical therapy and pharmacy showed that e-learning provides similar learning outcomes and satisfaction compared to traditional methods but is more flexible and cost-effective.⁹

Inadequate cancer pain management is a global problem despite increased awareness and initiatives, updated guidelines, and an increasing number of studies investigating its barriers. The rate of undertreatment of pain in cancer patients has not decreased substantially from 43.4% in 2016 to 40.0% in 2022.¹⁰ Moreover, a systematic review in 2016 that included 117 studies of 65,533 cancer patients demonstrated no significant change in the prevalence of cancer pain compared in the previous decade, with up to 38% of cancer patients reporting moderate-severe pain.¹¹

In Asia, cancer pain is generally undermanaged. Factors that might explain this phenomenon are shortage of trained healthcare workers, and inadequate knowledge regarding assessment and management of cancer pain.¹² A study of pain management in 10 countries in Asia demonstrated that more than half of physicians did not know how to prescribe pain medications properly, and they wanted more training to improve their knowledge and skills.¹³ Undertreatment of cancer pain in Thailand is well documented; 40% of hospitalized cancer patients experienced pain, and half of those who received pain medication had to request it.¹⁴ Chinda M and colleagues showed that a lack of knowledge of pain medications and inappropriate attitudes about pain were associated with poor pain management skills in general practitioners and residents in Thailand.¹⁵

Pain education for health professionals is repeatedly identified as a key to improving pain management. A comprehensive review of twenty-five years of research on pain education suggested moving forward to "advanced pain education", which must shift from a theoretical basis to a clinical environment where contextual decision-making is practised.¹⁶ A study of electronically delivered undergraduate pain education that included expert-facilitated collaboration online or face-to-face significantly improved pain knowledge.¹⁷ However, the effectiveness of pure e-learning, which is a more sustainable system of pain education for health professionals, has not been studied. The goal of this study was to assess the effectiveness of an SDL electronic platform on the management of cancer pain using a case-based approach in improving knowledge and critical thinking skills of first- and second-year trainees in anesthesiology, surgery and family medicine. The secondary outcome was to examine the residents' experiences and satisfaction.

Methods

Study design

The study was approved by the Ethics Committee of the Faculty of Medicine Ramathibodi Hospital, Mahidol University. The prospective observational study was conducted from February 2022 to June 2022 at the Faculty of Medicine Ramathibodi Hospital, Bangkok, Thailand.

Participants

First and second-year residents from anesthesiology, surgery, and family medicine who had not received training in the management of cancer pain at the pain clinic or who had not undergone a pain rotation were eligible to enroll in the study, and they were invited to participate by their Program Director. Participation was voluntary, and all participants provided written informed consent. We aimed to enroll at least forty participants based on the suggestion of an expert in digital education (PW). A total of 51 residents (F: M; 30:21), aged M=28.11, SD=0.6 years, completed four modules of e-CBL and pre and post-test of knowledge including residents from the Department of Anesthesiology 28 (55%), Family Medicine 14 (25%) and Surgery 11 (20%). However, only 32 participants completed the knowledge pre-and post-tests and the Critical thinking tests (32/51; 63%).

E-learning platform design and development

The e-learning platform was designed by three pain specialists (RS, SS, NT) and one specialist in digital education (PW). It contained four learning modules. The contents of each module were assessed and refined by four senior pain specialists who worked at Ramathibodi Hospital. The Modules included: 1) Pain assessment in cancer patients (12.51 minutes); 2) Principles of pain management (1.59 minutes); 3) Pharmacological treatment (22.43 minutes); and 4) Non-pharmacological management (2.26 minutes). Each module had five sequential steps:1:) case presentation, 2) creating a problem list, 3) summary of recent knowledge or guideline or recommendation, 4) making a relevant plan and 5) summary and application.

To enhance attention during each module, participants underwent quizzes that consisted of 8 multiple-choice questions either in or between the learning module. Eight quizzes contained one question about 1) pain assessment, 2) the type of pain, 3) the principle of acute pain management, 4) the principle of chronic pain management, 5) the type of analgesics, 6) the conversion ratio of morphine to fentanyl, 7) opioid-induced constipation, and 8) non-pharmacological pain management technique.

The e-learning process

Each module was posted in the Moodle learning platform and was available for seven days after students started the first module. The residents could complete the learning module at their own pace on any web-compatible device, including a smartphone, tablet, or PC. If the students had a question, they could send a message to the web administrator directly via Line-application.

Learning outcome and user experience - Pain knowledge

Pre and post-tests consisting of 30 MCQs were used to evaluate knowledge. Then, the score was converted to percentage. All MCQs were evaluated by six pain specialists from different University hospitals and each had an index of consistency (IOC) of more than 0.6. The MCQs assessed pain assessment (3 questions), pain classification (2 questions), principles of pain management (2 questions), techniques of pain management (1 question), choices of analgesics (10 questions), opioid conversion (5 questions), opioid side effects (5 questions) and non-pharmacological techniques (2 questions).

Critical thinking skills

The Cornel Critical Thinking Test Level Z measures cognitive abilities in students above grade 11. The Thai version is validated (criterion-related validity - concurrent validity; 0.70 and reliability; 0.75),¹⁸ and is widely used to investigate critical thinking ability.¹⁹ The test consists of 52 MCQs, which assess six domains based on the Theory of Ennis and Millman: 1) Deductive reasoning (10 questions); 2) Semantic (11 questions); 3) Credibility (4 questions); 4) Inductive reasoning (13 questions); 5) Prediction in planning experiment (4 questions); and 6) Definition and identification of assumptions (10 questions). The student is given 50 minutes to complete the test. One point was given for each correct response. The residents took the Critical Thinking Test before beginning the modules, and within 14 days after completing them.

For the pre and post-tests and the critical thinking test. The Moodle program randomly changed the order of choices in MCQs each time a test was given.

Learning experiences

At the end of study participation, participants filled out a satisfaction survey via the Moodle platform (0-10; 0 = unlikely and 10 = extremely likely) including 1) accessibility and convenience, 2) appropriateness and understandability and 3) satisfaction with the digital learning platform. Respondents were also invited to comments or suggestions.

Statistical Analysis 

The SPSS version 18.0 was used for statistical analysis The descriptive data was presented as percentages, and continuous data as mean and standard deviation. Normality of the continuous data was assessed with the Kolmogorov-Smirnov test. A paired t-test was used to compare pre vs. post-test of both knowledge and critical thinking skills if the data had a normal distribution. The effect size was calculated to demonstrate the strength of differences between pair measures and estimated by Cohen's d. A p-value < .05 was considered statistically significant.

Results

There was a normal distribution of scores for both the knowledge and critical thinking tests. The results of the pre and post-tests of knowledge and critical thinking were shown in Table 1. Overall, knowledge and critical thinking significantly improved after completing the electronically delivered instructional modules, with an extremely large effect size for knowledge and a medium effect size for critical thinking level. Table 2 showed changes in performance on the individual subscales of the critical thinking test, with significant improvement in deductive reasoning, semantics, inductive reasoning and definition and identification of assumptions, again with medium effect sizes.

Overall, the learning experience was rated at a high level, including accessibility and convenience (M=9.4, SD=1.0), appropriateness and understandability (M=9.5, SD= 0.8) and satisfaction with the digital learning platform (M=9.4, SD=0.9). There was a wide range of learning time (Mdn =3.9 hours, IQR=0.8, 165 hours). There was no significant correlation between learning time and change in knowledge score, r₍₅₂₎= -.08, p = .56 and Cornell critical thinking score, r₍₅₂₎ = -.04, p = .78.

Discussion

Our study demonstrated that use of the newly developed self-directed case-based e-learning platform in cancer pain management improved both knowledge and critical thinking skill levels in multidisciplinary residents with high satisfaction scores. Our findings are similar to other studies that demonstrate that online courses in medical education can increase knowledge and learning satisfaction and provide a low cost-effectiveness ratio.²⁰ Compared with traditional lectures, Koth and colleagues showed that an e-learning module provided similar knowledge outcomes without compromising critical thinking skills in dental education.²¹ For cancer pain education, Leung YW and colleagues showed that a facilitator-led online educational intervention can significantly enhance not only knowledge and confidence but also clinical skills in nurses.²²

Three components of our e-learning modules could explain our successful outcomes, including 1) an appropriate learning technique (e-CBL) and material that included well-selected cases and teaching materials, 2) appropriate active tools to increase student engagement such as quizzes in each module, and 3) the flexibility of e-learning that allowed students to learn at their own pace and at convenient times.

First, the five steps in each case-based module allowed the students to consider the patient's current and past medical history, generate a problem list, take into account existing knowledge, and form and apply a plan. We carefully developed the material in each module and verified it with four senior pain specialists. The unified theory of acceptance and use of technology (UTAUT) model specifies four aspects that enhance e-learning, including performance expectancy (learning material or education is applicable), effort expectancy (ease of use), social influence (other influence on individuals' engagement) and facilitating condition (availability of assisting resources).²³ Well-prepared material that is relevant to daily practice could improve in performance expectancy.²⁴ Moreover,  students enjoy CBL and think that it enhances their learning.³ Similarly, teachers prefer CBL because it engages and motivates students.³

Second, our learning platform included active learning tools in the form of quizzes in the modules and between the modules to increase students' engagement. Lau and colleagues demonstrated that a dynamic e-learning module (e.g. interactive learner-centric dynamic scenario-based education) was more enjoyable than a static module (e.g. video record of linear static education), which impacts engagement and satisfaction.²⁵ Rossi and colleagues showed that an online course with active learning methods can improve critical thinking skills, motivation and altitudes in sciences, especially the activities that require the interaction of information, prediction and reasoning.²⁶ Additionally, the case that we included who had severe cancer pain and suffering may increase the emotional engagement of learners.

Third, the flexibility of e-learning is suitable for students with variable learning styles. Bahrambeygi and colleagues showed that self-pacing improved nurses' knowledge regarding venous thromboembolism.²⁷ Shikino and colleagues demonstrated that using e-learning videos provided higher diagnostic accuracy scores for funduscopic examinations compared to didactic lectures.²⁸ Moreover, we used the Moodle e-learning platform, which is familiar to many residents, thus enhancing effort expectancy according to the UTAUT model.

Critical thinking is essential for students to evaluate the accuracy of information that they receive and to develop new ideas. Additionally, the ability to think critically is crucial in diagnosis and practice, for example, in the intensive care unit.²⁹ There is a significant relationship between critical thinking and self-directed learning in higher education^30-32 which is an essential prerequisite for lifelong education, continuing professional development or public health leadership.³³ Importantly, our case-based e-learning tools improved the critical thinking skills of learners.

Self-directed case-based e-learning can be an effective tool during challenging times such as pandemics or when there is limited time or access to educational resources, for example, a limited number of patients. Residents can access e-learning easily via their tablet, smartphone, or computer and learn at their own pace and time. Participants in our study rated accessibility and convenience, appropriateness and understandability, and satisfaction with the digital learning platform as highly satisfactory, enhancing the effort and performance expectancy of the UTAUT model.

Several limitations should be acknowledged when considering the results of our study. Our study sample was small. The pre-and post-learning evaluations only measured immediate, short-term effects. Repeat measures may be necessary to identify retention of learning and stabilization of critical thinking skills. The generalizability of our results is limited because most of the residents who participated were from the department of Anesthesiology. Due to the voluntary nature of residents' participation, selection bias cannot be excluded because more motivated residents were presumably more likely to enroll. Additionally, confounding factors might affect the internal validity of the results. For example, pain education before or during the intervention or enrollment of those with very little pain education could impact changes in scores. However, we excluded residents with formal pain education, and we limited the e-learning time to 7 days, which limited the available time for additional pain training. Finally, our study lacked a control group that underwent conventional didactic education on pain, and we cannot exclude an improvement in the pre-test assessment with repeated testing, particularly for the Critical Thinking test.

Conclusions

The self-directed case-based e-learning platform in cancer pain management significantly improved knowledge and critical thinking in multidisciplinary residents. This e-learning platform might play an important role in pain education. Future studies should include larger numbers of participants from additional disciplines and compare the effectiveness of our platform to conventional didactics.

Acknowledgements

The authors gratefully acknowledge the assistance of Ms. Wareeya Vongspanich for statistical analysis. Also, we thank the post-graduate medical department, including Mrs. Rujira Pethrak, for technical support. Lastly, we appreciated Prof. Christina Marra, Department of Neurology, University of Washington, USA, for English proofreading and generosity in manuscript editing.

Conflict of Interest

The authors declare that they have no conflict of interest.

References

1. Seymour-Walsh AE, Bell A, Weber A and Smith T. Adapting to a new reality: COVID-19 coronavirus and online education in the health professions. Rural Remote Health. 2020; 20: 6000.
   Full Text PubMed
2. Harman T, Bertrand B, Greer A, Pettus A, Jennings J, Wall-Bassett E and Babatunde OT. Case-based learning facilitates critical thinking in undergraduate nutrition education: students describe the big picture. J Acad Nutr Diet. 2015; 115: 378-388.
   Full Text PubMed
3. Thistlethwaite JE, Davies D, Ekeocha S, Kidd JM, MacDougall C, Matthews P, Purkis J and Clay D. The effectiveness of case-based learning in health professional education. A BEME systematic review: BEME Guide No. 23. Med Teach. 2012; 34: 421-444.
   Full Text PubMed
4. Fewox CS and Clearfield A. Synthesis and characterization of protonated zirconium trisilicate and its exchange phases with strontium. J Phys Chem A. 2008; 112: 2589-2597.
   Full Text PubMed
5. O'Doherty D, Dromey M, Lougheed J, Hannigan A, Last J and McGrath D. Barriers and solutions to online learning in medical education - an integrative review. BMC Med Educ. 2018; 18: 130.
   Full Text PubMed
6. Vedi N and Dulloo P. Students' perception and learning on case based teaching in anatomy and physiology: An e-learning approach. J Adv Med Educ Prof. 2021; 9: 8-17.
   Full Text PubMed
7. Gaupp R, Fabry G and Körner M. Self-regulated learning and critical reflection in an e-learning on patient safety for third-year medical students. Int J Med Educ. 2018; 9: 189-194.
   Full Text PubMed
8. Zottmann JM, Horrer A, Chouchane A, Huber J, Heuser S, Iwaki L, Kowalski C, Gartmeier M, Berberat PO, Fischer MR and Weidenbusch M. Isn't here just there without a "t" - to what extent can digital Clinical Case Discussions compensate for the absence of face-to-face teaching? GMS J Med Educ. 2020; 37: 99.
   Full Text PubMed
9. George PP, Papachristou N, Belisario JM, Wang W, Wark PA, Cotic Z, Rasmussen K, Sluiter R, Riboli-Sasco E, Tudor Car L, Musulanov EM, Molina JA, Heng BH, Zhang Y, Wheeler EL, Al Shorbaji N, Majeed A and Car J. Online eLearning for undergraduates in health professions: A systematic review of the impact on knowledge, skills, attitudes and satisfaction. J Glob Health. 2014; 4: 010406.
   Full Text PubMed
10. Roberto A, Greco MT, Uggeri S, Cavuto S, Deandrea S, Corli O and Apolone G. Living systematic review to assess the analgesic undertreatment in cancer patients. Pain Pract. 2022; 22: 487-496.
    Full Text PubMed
11. van den Beuken-van Everdingen MH, Hochstenbach LM, Joosten EA, Tjan-Heijnen VC and Janssen DJ. Update on Prevalence of Pain in Patients With Cancer: Systematic Review and Meta-Analysis. J Pain Symptom Manage. 2016; 51: 1070-1090.
    Full Text PubMed
12. Javier FO, Irawan C, Mansor MB, Sriraj W, Tan KH and Thinh DHQ. Cancer Pain Management Insights and Reality in Southeast Asia: Expert Perspectives From Six Countries. J Glob Oncol. 2016; 2: 235-243.
    Full Text PubMed
13. Kim YC, Ahn JS, Calimag MM, Chao TC, Ho KY, Tho LM, Xia ZJ, Ward L, Moon H and Bhagat A. Current practices in cancer pain management in Asia: a survey of patients and physicians across 10 countries. Cancer Med. 2015; 4: 1196-1204.
    Full Text PubMed
14. Vatanasapt P, Lertsinudom S, Sookprasert A, Phunmanee A, Pratheepawanit N, Wattanaudomrot S, et al. Prevalence and management of cancer pain in Srinagarind Hospital, Khon Kaen, Thailand. J Med Assoc Thai. 2008;91(12):1873-7.
15. Chinda M, Hathirat S, Waikakul W, Ittichaikulthol W. Physician's basic knowledge and attitudes toward pain medication in cancer patients. WoncaEurope 2006. Italy, Florence: Florence Publishing Company; 2005.
16. Thompson K, Johnson MI, Milligan J and Briggs M. Twenty-five years of pain education research-what have we learned? Findings from a comprehensive scoping review of research into pre-registration pain education for health professionals. Pain. 2018; 159: 2146-2158.
    Full Text PubMed
17. Watt-Watson J, McGillion M, Lax L, Oskarsson J, Hunter J, MacLennan C, Knickle K and Victor JC. Evaluating an Innovative eLearning Pain Education Interprofessional Resource: A Pre-Post Study. Pain Med. 2019; 20: 37-49.
    Full Text PubMed
18. Sirichai Kanchanawasi. Classical Test Theory. Bangkok: Chulalongkorn University Printing House; 2013.
19. Wannapiroon P. Development of a problem-based blended learning model to develop undergraduate students' critical thinking. [Doctoral Disserta-tion]. Faculty of Education; Bangkok: Chulalongkorn University Printing House; 2008.
20. Yeung AWK, Parvanov ED, Hribersek M, Eibensteiner F, Klager E, Kletecka-Pulker M, Rössler B, Schebesta K, Willschke H, Atanasov AG and Schaden E. Digital Teaching in Medical Education: Scientific Literature Landscape Review. JMIR Med Educ. 2022; 8: 32747.
    Full Text PubMed
21. Koth AJ, Focken AG, Lyden ER and Yoachim SD. Effectiveness of an E-module at teaching novice learners critical thinking skills related to dentistry. J Dent Educ. 2021; 85: 1879-1888.
    Full Text PubMed
22. Leung YW, Wong J, Kiteley C, Ellis J and Esplen MJ. Addressing Educational Needs in Managing Complex Pain in Cancer Populations: Evaluation of APAM: An Online Educational Intervention for Nurses. Am J Hosp Palliat Care. 2019; 36: 587-597.
    Full Text PubMed
23. Venkatesh V, Morris MG, Davis GB and Davis FD. User Acceptance of Information Technology: Toward a Unified View. MIS Quarterly. 2003; 27: 425-475.
    Full Text
24. Dyke EV, Jauncey-Cooke J and Johnston ANB. e-Learning interventions for nurses to prevent venous thromboembolism in patients: A realist review. J Clin Nurs. 2023; 32: 4441-4453.
    Full Text PubMed
25. Lau BD, Shaffer DL, Hobson DB, Yenokyan G, Wang J, Sugar EA, Canner JK, Bongiovanni D, Kraus PS, Popoola VO, Shihab HM, Farrow NE, Aboagye JK, Pronovost PJ, Streiff MB and Haut ER. Effectiveness of two distinct web-based education tools for bedside nurses on medication administration practice for venous thromboembolism prevention: A randomized clinical trial. PLoS One. 2017; 12: 0181664.
    Full Text PubMed
26. Rossi IV, de Lima JD, Sabatke B, Nunes MAF, Ramirez GE and Ramirez MI. Active learning tools improve the learning outcomes, scientific attitude, and critical thinking in higher education: Experiences in an online course during the COVID-19 pandemic. Biochem Mol Biol Educ. 2021; 49: 888-903.
    Full Text PubMed
27. Bahrambeygi F, Roozbahani R, Shojaeizadeh D, Sadeghi R, Nasiri S, Ghazanchaei E, et al. Evaluation of the effects of e-learning on nurses' behavior and knowledge regarding venous thromboembolism. Tanaffos. 2019;18(4):338–345.
28. Shikino K, Rosu CA, Yokokawa D, Suzuki S, Hirota Y, Nishiya K and Ikusaka M. Flexible e-learning video approach to improve fundus examination skills for medical students: a mixed-methods study. BMC Med Educ. 2021; 21: 428.
    Full Text PubMed
29. Hayes MM, Chatterjee S and Schwartzstein RM. Critical Thinking in Critical Care: Five Strategies to Improve Teaching and Learning in the Intensive Care Unit. Ann Am Thorac Soc. 2017; 14: 569-575.
    Full Text PubMed
30. Kreber C. The relationships between self-directed learning, critical thinking, and psychological type, and some implications for teaching in higher education. Studies in Higher Education. 1998; 23: 71-86.
    Full Text
31. Chen J, Zhou J, Wang Y, Qi G, Xia C, Mo G and Zhang Z. Blended learning in basic medical laboratory courses improves medical students' abilities in self-learning, understanding, and problem solving. Adv Physiol Educ. 2020; 44: 9-14.
    Full Text PubMed
32. Sajid MR, Laheji AF, Abothenain F, Salam Y, AlJayar D and Obeidat A. Can blended learning and the flipped classroom improve student learning and satisfaction in Saudi Arabia? Int J Med Educ. 2016; 7: 281-285.
    Full Text PubMed
33. Könings KD, de Jong N, Lohrmann C, Sumskas L, Smith T, O'Connor SJ, Spanjers IAE, Van Merriënboer JJG and Czabanowska K. Is blended learning and problem-based learning course design suited to develop future public health leaders? An explorative European study. Public Health Rev. 2018; 39: 13.
    Full Text PubMed