Using a structured questionnaire improves seizure description by medical students

Introduction

Seizures are common, affecting about 4% of the population by age 80, and reoccurring in about 30-40% of those with a first seizure.¹ Of greater concern, misdiagnosis is common; about 30-40% of patients evaluated at epilepsy centers are not epileptic.² Accurate seizure identification and classification requires an accurate account of the behavioral events occurring during the seizure (seizure semiology). This account of behavioral events contributes significantly to identifying the seizure onset site.³ A detailed account of behavioral events also contributes to improved differential diagnosis of non-epileptic events from epileptic events, selection of patients for clinical trials, and selection of patients for epilepsy surgery.⁴

Accurate seizure identification and classification remains a challenging task because of the complexity of seizure behavior and the limited accuracy of witnessed reports.  The International League Against Epilepsy (ILAE) Glossary exemplifies the complexity of seizure behaviors, listing 91 distinct terms for characterizing seizure semiology.⁵  A twin questionnaire study exemplified the limited accuracy of seizure reports; in one subpopulation, 18.5% of participants with verified epilepsy reported no prior history of seizures.⁶  In a study of eyewitness reports of seizure video recordings, eyewitnesses often overlook or inaccurately recall salient features of seizures.⁷  Even among clinicians, seizure reports obtained from the same patient by different clinicians from may vary.⁸ In a study of children with a first unprovoked seizure, clinicians had only a moderate to high level of agreement for only 68% of the 31 clinical variables studied.⁸  Similarly, medical students often fail to distinguish psychogenic non-epileptic seizures from epileptic seizures.⁹  Varying interpretations of witnessed reports also lead to inconsistent seizure classification. Among trained lay reviewers, agreement of seizure diagnosis varied significantly.¹⁰ In a study comparing initial to final diagnosis, 22% of first seizure cases were initially incorrectly classified as non-epileptic events by accident and emergency department physicians.¹¹

The challenge of accurate seizure diagnosis has led to research in epilepsy education.  Isler et al. developed an effective modular epilepsy educational program for medical residents, nurses and EEG technicians.¹² Using a randomized crossover trial of epilepsy education, Bye et al. found that although interactive lecturing and a computerized tutorial were both effective, students preferred interactive learning¹³.  Bye et al. also evaluated medical students’ interest, clinical confidence and perceived usefulness of their epilepsy educational program.¹³

The focus of this study was to develop an educational resource that would improve a student's ability to interpret a witnessed seizure.  Our hypothesis was that a structured questionnaire would guide a student's recall, improving the student's ability to identify, describe and interpret witnessed seizures.   

Methods

Setting and participants

The Georgia Regents University Institutional Review Board provided ethical approval for this research.  This study recruited 3rd year medical students who were enrolled in the Georgia Regents University Neurology Clerkship over a 1 year interval.  This study was in compliance with ethical regulations at Georgia Regents University.

Seizure observation and questionnaire completion

During the medical student’s second year, all students received a 1 hour epilepsy lecture that included video recordings of complex partial and primary generalized seizures.  During the first day of their 3rd year neurology clerkship, students were asked to view 2 videos.  These students were not told that the videos recorded seizures or that they would receive a questionnaire.  The first video recorded an adult male with a 43 second primary generalized tonic-clonic seizure (juvenile myoclonic epilepsy) with bilateral myoclonus and ictal vocalization.  The second video recorded an adult female with a 63 second complex partial seizure (temporal lobe epilepsy) with pause of activity, oral automatisms and manual automatisms.  The seizures occurred in alert patients, lying in bed with scalp electrodes in an inpatient epilepsy monitoring unit.  The patients provided written consent for viewing of the video recordings. 

After viewing both videos, the students first completed the unstructured questionnaire which contained 2 questions: Please describe what you see in video 1, please describe what you see in video 2. 

After completing the unstructured questionnaire, the students completed the structured questionnaire. The students identified their gender, reported if they had seen a seizure previously, and reported if they knew someone who had a seizure disorder. The structured questionnaire contained 17 questions, evaluating 8 features from the primary generalized seizure and 9 features from the complex partial seizure.  The structured questionnaire asked the student if the event was a seizure, how did the seizure begin, how long did the seizure last, which limb jerked first, did an automatism occur, did any vocal activity occur, did the seizure appear to rise from the right or left side of the brain, and did the patient appear to lose consciousness; for the complex partial seizure, the questionnaire asked if the head turned to the right or left side. Seizure duration was considered correct if the time estimate agreed to within +15 seconds of the actual seizure duration.  We selected complex partial and primary generalized seizures and the specific seizure features because these seizure types and seizure features are commonly encountered in clinical practice.  We evaluated these 17 features because our educational objective was for the students to identify, describe and interpret complex partial and primary generalized witnessed seizures.

Data analysis

The first outcome measurement was the total number of correctly identified seizure features using the unstructured questionnaire and total number of correctly identified features using the structured questionnaire.  We determine this outcome measure for each student. We applied the paired student t test to this data to determine if the structured questionnaire significantly improved the total of number of correctly identified features. Using a linear regression model, we determined if two potentially confounding variables, prior experience of seeing a seizure and knowing someone with a seizure disorder, could predict the change in total number of correctly identified items from using the structured questionnaire.

The second outcome measure was a binary variable representing either improved or worsened ability to identify a specific seizure feature.  Correct identification of the seizure feature on the structured questionnaire and incorrect identification on the unstructured questionnaire represented an improved outcome.  Incorrect identification of a seizure feature on the structured questionnaire and correct identification on the unstructured questionnaire represented worsened outcome. We determined this outcome measure for each student and for each of the 17 seizure features.  We used the sign test to determine if the improved outcomes were more likely than worsened outcomes for each individual seizure feature.  Using a logistic regression model, we determined if two potentially confounding variables, prior experience of seeing a seizure and knowing someone with a seizure disorder could predict, improved or worsened outcome for each seizure feature.  We used a p < 0.0029 for statistical significance which corresponds to familywise p < 0.05 adjusted for 17 repeated measurements.  We performed statistical analysis using R statistical software and the mid-p value method.^14,15

Results

This study enrolled 92 participants, 35 (38%) women and 57 (62%) men.  An additional 2 participants initially participated but were later excluded because they did not complete the questionnaire. 

The overall number of participants who correctly identified each feature varied over a wide range (Table 1). Overall, the structured questionnaire was more effective in eliciting an average of 9.25 correct responses compared to the unstructured questionnaire eliciting an average of 5.30 correct responses (p < 0.001).  The 2 potential confounding factors, prior experience of seeing a seizure and knowing someone with a seizure disorder, were not statistically significant predictors of the total number of correct responses from using a structured questionnaire.

The relative improvement from using a structured questionnaire was significant for 10 of the 17 features using a familywise p-value 0.05 adjusted for 17 repeated measures (Table 2).   For both seizures, the structured questionnaire improved the student's recognition of seizure occurrence, absence or presence of ictal vocalization, and impaired consciousness (Table 2).  The structured questionnaire also improved student recognition for absence of automatisms for the primary generalized seizure and pause of motor activity, duration, and absence of head or eye deviation for the complex partial seizure (Table 2).  The 2 potential confounding factors were not statistically significant for predicting the improved performance from using a structured questionnaire for any of the individual 17 features (Table 3).

Discussion

In comparison to prior research, our study was unique in verifying that a specific intervention, a study questionnaire, more effectively elicited the major features of an observed seizure compared to an unstructured questionnaire.  This benefit occurs for distinct seizure types, complex partial and primary generalized seizures. Also, important is that structured questionnaire improves recognition of some but not all seizure features and prior experience of seeing a seizure or knowing someone with seizures does not improve the student’s ability to recognize seizure features.

There are some limitations of this study. This study’s questionnaire identified the major seizure features in the 2 videos viewed by the students. However, a single standard questionnaire surveying the full range of recognized behavioral features could apply to a broader range of seizure types.⁵ In this study, our selection of a limited number of questionnaire items could cause potential bias.  A comprehensive standard seizure questionnaire could reduce this potential for bias.  The unstructured questionnaire asked the students to describe what they saw but did not specifically request a detailed description; a request for a detailed description may have led to a greater number of correct responses on the unstructured questionnaire. This study relied on 2 seizures; however, variability occurs among seizures and use of a greater number of seizures would generate a more accurate estimate of which seizure features are likely to be recognized by medical students.

In this study, we restricted the study to complex partial and primary generalized seizures, but for medical student instruction, video instruction would need to include a broad range of seizure disorders, syncope and non-epileptic psychogenic seizures. Our data shows that the combined sequence of unstructured and structured questionnaire is more effective than unstructured questionnaire alone.  Due to the study design, we cannot conclude that structured questionnaire alone is better than unstructured questionnaire alone. Although the structured questionnaire improved student performance, the mean score of 9.35 items correct was below the ideal score of all 17 items correctly identified. Another limitation of this study was that although we identified which features were improved with the structured questionnaire, our data did not indicate why performance improved on only 10 of the 17 seizures features.

Two prior studies also investigated diagnostic accuracy of seizure classification using video recorded seizures.  Using a logistic regression model, Azar et al. constructed a model based on a 12 item questionnaire that would distinguish epileptic from non-epileptic seizures with a predicted 84.4% accuracy.¹⁶ Unlike our study, Azar et al. did not determine the relative benefit of their questionnaire in a controlled trial.¹⁶ Hirfanoglu et al. compared seizure classification before and after viewing seizure video recordings.¹⁷  Hirfanoglu et al. found a moderate to high consistency of seizure classification for 6 of the 22 seizure types.¹⁷ Unlike our study, Hirfanoglu et al. investigated a far larger number of seizure types, but did not investigate identification of specific seizure features.¹⁷ Also unlike our study, Hirfanoglu et al. investigated the influence of a different manipulated variable, seizure video data, on seizure classification.¹⁷ In comparison to the prior studies, less experienced medical students in our study observed and interpreted seizure behavior rather than the more experienced physicians in these prior studies.^16,17

The results of this study may lead to additional research and new applications in the future.  Future research could determine if repeated application of the unstructured and structured questionnaire sequence would train students to consistently perform better, ultimately recognizing all features of a video recorded event.  Future research could focus on the effects of structured questionnaires on recognizing key features of other clinical events by participants other than medical students: the lay person, non-physician professionals, primary care physicians and neurologists.  Questionnaires could be developed to evaluate related disorders such as psychogenic nonepileptic seizures, convulsive syncope and other seizure types. The questionnaire could be applied prior to and following epilepsy student lecture to determine if the lecture was effective in promoting student learning. Another approach would be for a neurologist, blinded to seizure diagnosis, to diagnose the seizure type from the unstructured student questionnaire responses; the features from the unstructured questionnaires leading to correct diagnosis would identify the features sufficient for diagnosis.  A seizure questionnaire, modified for use by a layperson, may be useful to evaluate responses to anti-epileptic medications, investigational medications or vagus nerve stimulator therapy in the home environment.

Conclusions

This study shows that a structured questionnaire significantly improves a medical student’s ability to accurately identify, describe and interpret the key features of primary generalized and complex partial witnessed seizures.  This study identified 10 specific features improved by using the structured questionnaire. 

Acknowledgements

We gratefully acknowledge the participation of all the students involved in this study. We also acknowledge the patients who consented to the viewing of video recordings of their seizures.

Conflict of Interest

The authors declare that they have no conflict of interest.

References

1. Herman ST. Single Unprovoked Seizures. Curr Treat Options Neurol. 2004; 6: 243-255.
   Full Text PubMed
2. Benbadis S. The differential diagnosis of epilepsy: a critical review. Epilepsy Behav. 2009; 15: 15-21.
   Full Text PubMed
3. Tufenkjian K and Lüders HO. Seizure semiology: its value and limitations in localizing the epileptogenic zone. J Clin Neurol. 2012; 8: 243-250.
   Full Text PubMed
4. Berg AT, Berkovic SF, Brodie MJ, Buchhalter J, Cross JH, van Emde Boas W, Engel J, French J, Glauser TA, Mathern GW, Moshé SL, Nordli D, Plouin P and Scheffer IE. Revised terminology and concepts for organization of seizures and epilepsies: report of the ILAE Commission on Classification and Terminology, 2005-2009. Epilepsia. 2010; 51: 676-685.
   Full Text PubMed
5. Blume WT, Lüders HO, Mizrahi E, Tassinari C, van Emde Boas W and Engel J. Glossary of descriptive terminology for ictal semiology: report of the ILAE task force on classification and terminology. Epilepsia. 2001; 42: 1212-1218.
   Full Text PubMed
6. Corey LA, Kjeldsen MJ, Solaas MH, Nakken KO, Friis ML and Pellock JM. The accuracy of self-reported history of seizures in Danish, Norwegian and U.S. twins. Epilepsy Res. 2009; 84: 1-5.
   Full Text PubMed
7. Thijs RD, Wagenaar WA, Middelkoop HA, Wieling W and van Dijk JG. Transient loss of consciousness through the eyes of a witness. Neurology. 2008; 71: 1713-1718.
   Full Text PubMed
8. Dayan PS, Lillis K, Bennett J, Conners G, Bailey P, Callahan J, Akman C, Feldstein N, Hauser WA and Kuppermann N. Interobserver agreement in the assessment of clinical findings in children with first unprovoked seizures. Pediatrics. 2011; 127: 1266-1271.
   Full Text PubMed
9. Ristić AJ, Drašković M, Bukumirić Z and Sokić D. Reliability of the witness descriptions of epileptic seizures and psychogenic non-epileptic attacks: a comparative analysis. Neurol Res. 2015; 37: 560-562.
   Full Text PubMed
10. Ottman R, Lee JH, Hauser WA, Hong S, Hesdorffer D, Schupf N, Pedley TA and Scheuer ML. Reliability of seizure classification using a semistructured interview. Neurology. 1993; 43: 2526-2530.
    Full Text PubMed
11. Leung H, Man CY, Hui AC, Wong KS and Kwan P. Agreement between initial and final diagnosis of first seizures, epilepsy and non-epileptic events: a prospective study. J Neurol Neurosurg Psychiatr. 2008; 79: 1144-1147.
    Full Text PubMed
12. Isler A, Basbakkal Z, Serdaroglu G, Tosun A, Polat M, Gokben S and Tekgul H. Semiologic seizure classification: the effectiveness of a modular education program for health professionals in pediatrics. Epilepsy Behav. 2008; 13: 387-390.
    Full Text PubMed
13. Bye AME, Connolly AM, Netherton C, Looker P, Burgess A and Lonergan A. A triangulated approach to the assessment of teaching in childhood epilepsy. Med Teach. 2007; 29: 255-257.
    Full Text PubMed
14. R Development Team. R: a language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing; 2011.
15. Venables W, Ripley B. Modern applied statistics with S. New York: Springer; 2002.
16. Azar NJ, Pitiyanuvath N, Vittal NB, Wang L, Shi Y and Abou-Khalil BW. A structured questionnaire predicts if convulsions are epileptic or nonepileptic. Epilepsy Behav. 2010; 19: 462-466.
    Full Text PubMed
17. Hirfanoglu T, Serdaroglu A, Cansu A, Bilir E and Gucuyener K. Semiological seizure classification: before and after video-EEG monitoring of seizures. Pediatr Neurol. 2007; 36: 231-235.
    Full Text PubMed