Approach to learning and educational environment: time to rethink measurement tools in postgraduate medical training?

Introduction

Anesthesiology training in the United States is oriented to the development of core competencies defined by the Accreditation Council for Graduate Medical Education (ACGME).¹ Residency programs must enact their academic curriculum to support learning for their trainees. Although residency programs have adjusted the curriculum to the new paradigm shifts regarding teaching and competence assessment, an individual component of the equation unique to each resident determines the learning outcomes.^2,3 The analysis of individual approach to learning addresses why and how students engage in learning and the factors that influence the selection of specific approaches.⁴

Extensive research into the phenomenon of higher learning has unearthed several theories regarding the approaches to learning based on the characteristics of each approach and the motivation of each student. Learning approaches have been divided into three distinct, but not dissimilar categories: the surface approach, the deep approach, and the strategic approach. The surface approach to learning involves the memorization and the reproduction of the desired information. The deep approach involves the organization and extrapolation of the desired information, and the strategic approach involves the academic success of the student and the highest efficiency possible.⁵

The learning environment is an important factor that influences the approach to learning adopted by a student, in aspects as diverse as content, context, and demands.⁶ Biggs proposed that the approach to learning was determined by factors inherent to each student under the influence of the educational context.⁷ This association has led researchers to postulate that modifications in the learning environment lead to better academic performance via adoption of a deep approach to learning fostered by student-centered methods. ⁸ The effect of the educational environment on learning affects undergraduate students from different disciplines and medical students.^9,10 The approaches to learning are not static; on the contrary, the educational environment determines the transition from one approach to another;¹¹ however, the relationship between learning approaches and the environment is rather complex, and is dependent on multiple factors whose interaction has not been fully elucidated.¹²  In addition, student perception rather than the learning environment itself is the variable that influences the approach to learning.¹³ Several authors have shown the association between approach and environment in relation to learning as well as its impact on academic performance in the context of general education.^14-18 To our knowledge, the relationship between the perception of the learning environment and learning approaches have not been described in postgraduate medical trainees. We conducted this study to evaluate the effect of the perception of the learning environment on the approaches to learning adopted by anesthesiology residents, and whether the approach to learning changes as the resident progresses through training.

This study is aimed to assess the correlation of the perception of the learning environment and the approach to learning adopted by anesthesiology residents throughout training. The secondary aims are to evaluate the effect of years of training on the resident’s learning approach and to assess the association between the domains of the Dundee Ready Environment Measure (DREEM) questionnaire used to evaluate learning environment, and the learning approaches adopted by anesthesiology residents.

Methods

Design and participants

A cross-sectional study was conducted in forty-one residents enrolled in the anesthesiology program at Augusta University. The sample was collected by convenience to include the current anesthesiology residents enrolled in the program. The study was approved by the Institutional Board Review of Augusta University. Written consent was obtained after explaining the participants the ethical aspects of the study, including the quality and integrity of the research, respect for confidentiality and anonymity, and the independent nature of the study.  Participants were asked to complete a demographic survey and the electronic version of two questionnaires:  the Revised Two-Factor Study Process Questionnaire (R-SPQ-2F) to assess learning approaches¹⁹ and the Dundee Ready Educational Environment Measure (DREEM) to assess learning environment.

Data Collection

The R-SPQ-2F instrument consists of twenty items evaluating two scales, each one composed of ten items, assessing either the superficial or the deep approach to learning. Additionally, the scales can be subdivided into two subscales of five items each, which reveal the strategies and motivations underlying the learning approaches. Responses to each item were categorized according to a 5-point Likert scale. This questionnaire evaluates the deep dimension (items 1, 2, 5, 6, 9, 10, 13, 14, 17, 18), and superficial dimension of learning approaches (items 3, 4, 7, 8, 11, 12, 15, 16, 19, 20). Biggs reported both the validity and reliability of the scale to discriminate superficial and deep learning approaches.¹⁹

The DREEM questionnaire consists of fifty items evaluating a range of topics in relation to the learning environment. The participant’s responses fall within five Likert categories from strongly agree to strongly disagree. The instrument evaluates five different dimensions: perceptions of learning (12 questions), the perception of course organizers (11 questions), perceptions of the atmosphere (12 questions), social self-perceptions (7 questions) and academic self-perceptions (8 questions). The total score is interpreted to allocate the participant to the following categories based on the perception of the learning environment: very poor (0-50), plenty of problems (51-100), more positive than negative (101-150), and excellent (151-200).²⁰ McAleer reported on validation and reliability of the DREEM questionnaire.²¹ Demographic variables (age, gender, relationship status) and variables related to learning approach, DREEM score, and the post-graduate year from each resident were recorded.

Procedure

The residents were invited to participate in the study in a departmental meeting. Then, an electronic link to the questionnaires as well as a consent form was sent via electronic mail. The surveys were collected ensuring anonymity. Residents received two electronic mail prompts in July 2018. Data were collected by one investigator, exported to an Excel Microsoft worksheet, and tabulated for analysis.

Statistical analysis

Baseline characteristics including age, relationship status, and category of medical school (i.e., allopathic US medical school, osteopathic medical school or foreign medical school) were compared between junior residents at post-graduate years 1-2 (PGY-1 and PGY-2) and senior residents at PGY-3 and 4 using chi-square test. Cronbach α was used to assess internal consistency of responses within each domain of both questionnaires.²² A value above 0.60 was considered as satisfactory for internal consistency and values above 0.70 as good for internal consistency in a particular domain.²³ In domains where Cronbach  α  was less than 0.60, questions were sequentially excluded beginning with the question with the lowest consistency to obtain a Cronbach α of more than 0.60.

Analysis of variance was performed to elucidate differences in the mean domain and total scores among the two residency categories (junior versus senior residents). We performed multivariate logistic regression model to assess the association between total R-SPQ-2F score, deep motive, surface motive, deep strategy, surface strategy, total surface approach, total deep approach domains, and resident category. We also performed the above analyses using the year of residency as the categorical variable i.e., PGY-1 versus PGY-2, PGY-3, and PGY-4 residents. Fisher’s least significance difference was used for post-hoc analysis. A p-value of 0.05 was considered statistically significant.

The total DREEM score and mean DREEM domain scores were standardized by dividing the total score with the number of questions included in that particular domain. Analysis of variance was performed to assess the difference in mean scores among the residency categories. Interpretation of each DREEM domain was done using modified recommendations of McAleer and colleagues to account for questions excluded to maintain internal consistency.^19,21 Chi-square test was used to assess the association between standardized domain categories among the two residency categories (junior versus senior residents). We also performed the above analyses using the year of residency as a categorical variable, i.e., PGY-1 versus PGY-2, PGY-3, and PGY-4 residents. Fisher’s least significance difference was used for post-hoc analysis. A p-value of 0.05 was considered statistically significant.

Regarding R-SPQ-2F and DREEM correlation analyses, we assessed the correlation between each domain and total R-SPQ-2F scores with each domain and total DREEM scores using the multivariate logistic regression model with the Pearson correlation coefficient (R). An R-value of greater than 0.8 was classified as strong, 0.5-0.79 as moderate, and less than 0.5 as a weak association. A p-value of 0.01 was considered statistically significant. All statistical analyses were performed using SPSS 20 (IBM, Armonk, New York, USA).

Results

Forty-one residents completed the DREEM and R-SPQ-2F questionnaires. The study population included 8 post-graduate year-1 (PGY-1) residents, 12 PGY-2 residents, 9 PGY-3 residents, and 12 PGY-4 residents. Age of residents varied between 27-45 years. There was no statistically significant difference in baseline characteristics except for marital status (Table 1).

R-SPQ-2F Questionnaire

Cronbach α varied between 0.604 and 0.76 among the domains in R-SPQ-2F with question number two excluded in deep strategy domain to achieve Cronbach α of more than 0.60 (Table 2). There were no significant differences in mean scores of individual responses to questions in each domain between junior and senior residents (Table 3). There were no differences in mean domain-specific or total scores among the junior and senior residents. There was no significant association between specific domains and resident category (junior versus senior). Similarly, there was no significant association between specific domains and resident year of training (Table 4).

DREEM Questionnaire

With respect to DREEM questionnaire, questions 1, 2 and 6 were excluded from the subjective perception of teachers domain, and question 3 was excluded from the subjective academic perception domain in order to achieve Cronbach  greater than 0.60. There was no significant difference in mean scores of individual questions between junior and senior residents, except for two questions: “Last year's work has been good preparation for this year's work,” and “I seldom feel lonely.”  There was no significant association between domain category responses and resident category (junior versus senior). Similarly, there was no significant association between specific domains and resident year of training.

R-SPQ-2F and DREEM Correlation

There was a moderate correlation between total deep approach scores of the R-SPQ-2F questionnaire with the total subjective perception of teachers scores of the DREEM questionnaire (R²= - 0.507, p <0.01). There was no significant correlation between domains and total scores of R-SPQ-2F and DREEM questionnaire.

Discussion

Our study shows the results of anesthesiology residents’ learning approaches and perception of the educational environment in our institution. Our results indicate that there is no correlation between the perception of the learning environment and the approach to learning adopted by anesthesiology residents and that these finding does not change with years of training.

The internal consistency for R-SPQ-2F questionnaire was acceptable with a Cronbach alpha ranging between 0.604 and 0.76 between domains, whereas for the DREEM questionnaire, the internal consistency was questionable (Cronbach alpha of 0.6). We considered that adjustment for inconsistent responses by the exclusion of some questions was necessary for better assessment of these multi-item scales.²⁴ The relatively low Cronbach alpha values may be due to the limited number of questions, poor interrelatedness between items or heterogeneous constructs.²⁵ We consider that both learning approaches and educational environment present themselves as a continuous heterogeneous construct in residency training, rather than a dichotomous one. In our opinion, the correlation of learning approaches and environment are complex, requiring equally complex non-homogeneous predictors for objective and subjective criteria of evaluation. In other words, the application of undergraduate instruments to evaluate these two important dimensions of medical education, in the context of residency training is an inadequate practice.

Intuition leads researchers to think that the more advanced a student is in years of education, the greater their tendency to adopt deep approaches to learning. Mirghani et al. showed that medical students preferred deep approaches to learning as opposed to a superficial approach preferred by first and second-year students.²⁶ Delva and colleagues evidenced that the approach to learning was related to workplace environment when medical students and residents were evaluated with the Workplace Learning Questionnaire and that this correlation varied with years of training. However, there is no mention of an approach to learning in relation to years of education.²⁷ Our study did not show a correlation between learning environment perception/approach and years of training for anesthesiology residents. These results may be related to the fact that by the time residents enter their postgraduate education cycle, they have already defined their preferred approach to learning, and that the learning environment does not change significantly during training. On the other hand, we acknowledge that the adoption of a determinate approach to learning by a resident is a dynamic process. Depending on the particular situation, a resident adopts one approach over the other.²⁸

We found a moderate correlation between total deep approach scores of the R-SPQ-2F questionnaire and the total subjective perception of teacher’s scores of the DREEM questionnaire. Campbell et al. found that students with deep approaches to learning demonstrate a more sophisticated understanding of the teaching they receive compared to a student with superficial approaches.²⁹ The teacher as a role model is a significant contributor to the approach to learning adopted by students. Clinical teacher awareness of their role has significant influence on the teaching-learning experience.³⁰ Postgraduate clinical teaching demands for trained trainers.³¹ Some authors advocate for a structured journey that a clinician should follow to become a medical teacher.³² Our findings support the notion that the way teaching is conveyed to anesthesiology residents might be related to their learning approach and potentially to learning outcomes.

The 3P model has outlined the complexity of the learning process, that conceives the learning process as a chronological sequence of consecutive phases.^33,34 These phases are called presage, process, and product. The presage phase details the personality distinctions between all of the people involved in the learning process.³⁵ The process phase describes the specific learning approach or process that both students and teachers utilize in order to either learn or teach information. In the product phase, all of the learning is completed, regardless of the approach, and the evaluation has been graded. A student’s conception of learning, their conception of their teacher, and a student’s individual contextual experience combine to determine their perception of their learning environment. Although the three phases are well defined in undergraduate education, where most of the learning process takes place in a classroom, in residency training the definition of the phases is blurred. In our study, administration of the R-SPQ-2F questionnaire did not discriminate learning approaches per year of training. We might argue that the three phases are experienced in a different way as the resident progresses through training, and an instrument specifically designed to evaluate these dimensions might be necessary to evaluate learning approaches at different levels of training.

Our study has limitations. We evaluated a population of anesthesiology residents of a single program in the United States, limiting the generalizability of our results. Residency programs outside the United States may be structured differently in terms of team-based care. In American residency programs, the post-graduate student receives constant input from diverse sources, including nursing and administrative staff as well as from medical personnel, which has a direct impact on the perception of the educational environment. On the other hand, the differences between general educational systems across countries may account for differences in learning approaches adopted by students. All in all, extrapolation of our results to post-graduate medical programs must be done with caution. In addition, we consider that including residents of different years of training and from different backgrounds provide a rich sample of individuals at different stages of their educational progress. Overall, our results should be taken within the context for a residency program with a size similar to the one at our institution. A Cronbach value of 0.60 was deemed satisfactory. However, values greater 0.70 were considered good, and results should be interpreted with this understanding. Future research is warranted to develop and validate an instrument able to assess learning approaches and perception of the educational environment in postgraduate anesthesiology education.

Conclusions

The learning approaches adopted by anesthesiology residents and the perception of the educational environment are not correlated with years of training. Additionally, the DREEM and R-SPQ-2F questionnaires are not adequate for evaluation of the correlation between learning environment perception and approaches to learning in anesthesiology residents. Our study has implications for post-graduate education in anesthesiology. The adoption of DREEM and R-SPQ-2F questionnaires that have been validated in general educational contexts, should not be recommended in the post-graduate anesthesiology setting. In addition, future research should focus on the design, development, and validation of instruments to assess the relationship between the learning approach and the perception of educational climate in post-graduate medical training, as well as the effect of progress on these dimensions during the years of residency.

Conflict of Interest

The authors declare that they have no conflict of interest.

References

1. Lurie SJ, Mooney CJ and Lyness JM. Measurement of the general competencies of the accreditation council for graduate medical education: a systematic review. Acad Med. 2009; 84: 301-309.
   Full Text PubMed
2. Fraser AB, Stodel EJ and Chaput AJ. Curriculum reform for residency training: competence, change, and opportunities for leadership. Can J Anaesth. 2016; 63: 875-884.
   Full Text PubMed
3. Biggs J. Individual differences in study processes and the quality of learning outcomes. Higher Education. 1979;8(4):381-394.
4. Coertjens L, Vanthournout G, Lindblom-Yianne S, Postareff L. Understanding individual differences in approaches to learning across courses: a mixed method approach. Learning and Individual Differences. 2016;51:69-80. doi: 10.1016/j.lindif.2016.07.003
5. Biggs JB. Student approaches to learning and studying. Research Monograph. Newcastle: ERIC; 1987.
6. Richardson JT. Researching student learning: approaches to studying in campus-based and distance education. Buckingham: Society for Research into Higher Education & Open University Press; 2000.
7. Biggs JB. From theory to practice: a cognitive systems approach. Higher Education Research & Development. 1993;12(1):73-85.
8. Struyven K, Dochy F, Janssens S, Gielen S. On the dynamics of students' approaches to learning: The effects of the teaching/learning environment. Learning and Instruction. 2006;16(4):279-294.
9. Eleazer CD and Scopa Kelso R. Influence of study approaches and course design on academic success in the undergraduate anatomy laboratory. Anat Sci Educ. 2018; 11: 496-509.
   Full Text PubMed
10. Ullah R. Learning environment, approaches to learning and learning preferences: medical students versus general education students. J Pak Med Assoc. 2016; 66: 541-544.
    PubMed
11. Entwistle NJ, Peterson ER. Conceptions of learning and knowledge in higher education: relationships with study behaviour and influences of learning environments. International Journal of Educational Research. 2004;41(6):407-428.
12. Baeten M, Kyndt E, Struyven K, Dochy F. Using student-centred learning environments to stimulate deep approaches to learning: factors encouraging or discouraging their effectiveness. Educational Research Review. 2010;5(3):243-260.
13. Entwistle NJ. Approaches to learning and perceptions of the learning environment. Higher Education. 1991;22(3):201-204.
14. Fraser BJ. International handbook of science education. New York: Springer; 1998.
15. Genn JM. AMEE Medical Education Guide No. 23 (Part 1): Curriculum, environment, climate, quality and change in medical education–a unifying perspective. Medical Teacher. 2009; 23: 337-344.
    Full Text
16. Gustin MP, Abbiati M, Bonvin R, Gerbase MW and Baroffio A. Integrated problem-based learning versus lectures: a path analysis modelling of the relationships between educational context and learning approaches. Med Educ Online. 2018; 23: 1489690.
    Full Text PubMed
17. Müller FH and Louw J. Learning Environment, Motivation and Interest: Perspectives on Self-Determination Theory. South African Journal of Psychology. 2004; 34: 169-190.
    Full Text
18. Wayne SJ, Fortner SA, Kitzes JA, Timm C and Kalishman S. Cause or effect? The relationship between student perception of the medical school learning environment and academic performance on USMLE Step 1. Med Teach. 2013; 35: 376-380.
    Full Text PubMed
19. Biggs J, Kember D, Leung DY. The revised two‐factor study process questionnaire: R‐SPQ‐2F. Br J Educ Psychol. 2001;71(1):133-149. doi: 10.1348/000709901158433
20. Vaughan B, Carter A, Macfarlane C and Morrison T. The DREEM, part 1: measurement of the educational environment in an osteopathy teaching program. BMC Med Educ. 2014; 14: 99.
    Full Text PubMed
21. Genn JM. AMEE Medical Education Guide No. 23 (Part 1): Curriculum, environment, climate, quality and change in medical education-a unifying perspective. Med Teach. 2001; 23: 337-344.
    Full Text PubMed
22. Bland JM, Altman DG. Statistics notes: Cronbach's alpha. BMJ.1997;314(7080):572. doi:10.1136/bmj.314.7080.572
23. George D. SPSS for Windows step by step: a simple guide and reference 11.0 Update. New York: Pearson Education; 2003.
24. Fong DY, Ho SY and Lam TH. Evaluation of internal reliability in the presence of inconsistent responses. Health Qual Life Outcomes. 2010; 8: 27.
    Full Text PubMed
25. Tavakol M and Dennick R. Making sense of Cronbach's alpha. Int J Med Educ. 2011; 2: 53-55.
    Full Text PubMed
26. Mirghani HM, Ezimokhai M, Shaban S and van Berkel HJ. Superficial and deep learning approaches among medical students in an interdisciplinary integrated curriculum. Educ Health (Abingdon). 2014; 27: 10-14.
    Full Text PubMed
27. Delva MD, Kirby J, Schultz K and Godwin M. Assessing the relationship of learning approaches to workplace climate in clerkship and residency. Acad Med. 2004; 79: 1120-1126.
    PubMed
28. Zeegers P. Approaches to learning in science: a longitudinal study. Br J Educ Psychol. 2001; 71: 115-132.
    PubMed
29. Campbell J, Smith D, Boulton-Lewis G, Brownlee J, Burnett PC, Carrington S and Purdie N. Students' Perceptions of Teaching and Learning: The influence of students' approaches to learning and teachers' approaches to teaching. Teachers and Teaching. 2010; 7: 173-187.
    Full Text
30. Jayasuriya-Illesinghe V, Nazeer I, Athauda L and Perera J. Role Models and Teachers: medical students perception of teaching-learning methods in clinical settings, a qualitative study from Sri Lanka. BMC Med Educ. 2016; 16: 52.
    Full Text PubMed
31. Ortwein H, Blaum WE and Spies CD. Anesthesiology residents' perspective about good teaching--a qualitative needs assessment. Ger Med Sci. 2014; 12: 05.
    Full Text PubMed
32. Riveros-Perez E and Rodriguez-Diaz J. The journey from clinician to undergraduate medical educator involves four patterns of transformation. Adv Med Educ Pract. 2018; 9: 7-15.
    Full Text PubMed
33. Biggs JB. The role of metalearning in study processes. Br J Educ Psychol. 1985;55(3):185-212. doi: 10.1111/j.2044-8279.1985.tb02625.x
34. Price L. Learning patterns in higher education. London: Routledge; 2013.
35. Chatti MA, Jarke M, Specht M. The 3P learning model. Journal of Educational Technology & Society. 2010;13(4):74-85.