A mixed methods study to explore the process of clinical reasoning employed by clinical students during primary clinical assessments

Background and Purpose

Clinical reasoning is required at all points of patient interaction yet is confused due to a lack of coherence in the language used and description within the literature. Clinical educators must develop the skills of clinical reasoning amongst students as a core skill of the professions. Failure to do so risks students having good knowledge but a lack of effective cognitive tools to apply it. Understanding how students undertake the clinical reasoning process as they leave clinical programmes can enable us to ensure the appropriate development of educational interventions to develop their expertise in this skill.
This research project sought to answer the question as to how clinical students undertake the process of clinical reasoning in Single Best Answer (SBA) questions and Objective Structured Clinical Examinations (OSCE).

Methods

A narrative literature review was undertaken to formulate a concept framework for the clinical reasoning process and explore the implications of the literature regarding education and assessment. From this a working definition for clinical reasoning was formulated for use in this study.
The concept framework was used as the basis for a nominal group approach to formulate ten key questions to use in the primary study to explore different aspects of clinical reasoning as set out in the concept framework.
The primary study was a convergent mixed methods study. The questions generated from the nominal group were used in the quantitative arm, as questionnaire items, and the qualitative focus groups as the interview guide. In the quantitative arm students answered 40 SBA questions, or completed an OSCE, and then immediately answered the ten-item questionnaire. In the qualitative arm students were interviewed in focus groups. A pilot of the methodology was conducted to ensure feasibility.

Results

Three universities took part in the study, two with both medical and physician associate students and one with physician associate students.
In the quantitative element a total of 595 data points (an individual participant completing all 10 questions) were collected, 291 for SBA and 304 for OSCE. Statistical analysis was undertaken in Microsoft Excel and SPSS (IBM, 2021; Microsoft, 2023). The quantitative results permitted comparison of groups with different characteristics. These showed differences in reasoning between SBA and OSCE, but these were very small. There were three questions with statistically significant differences, in both SBA and OSCE, between medical and PA students although these were small. There was a slightly higher tendency toward pattern recognition in medical students. There was one question showing a statistically significant difference between genders for both SBA and OSCE. Observed difference was small however females felt being in an assessment affected their reasoning more. There was one question showing a statistically significant difference between ethnicities for SBA and three for OSCE. The observed difference was, again, small.
The qualitative arm included seven focus groups of between 2 and 18 participants and a total of 18 medical students and 28 physician associate students. These were transcribed and analysed using thematic analysis (Clarke and Braun, 2013). Four key themes were identified. The qualitative results and quantitative results were then converged, and conclusions drawn.
• Students felt being in an exam affected their reasoning, but this appeared linked to the difference between eustress and distress.
• SBA is heavily pattern recognition, OSCE less so than the quantitative data alone would suggest. OSCE discrepancy (e.g. patient appearance differing from the instructions) lowered the value of demographic information and clinical setting in the stem.
• Early hypothesis generation was typical in both assessments, but SBA was answer driven, OSCE was plan driven.
• Student thinking process tended toward inductive reasoning, but this was stronger for SBA.
• Evidence for metacognition was seen. The results suggest abductive reasoning is the most appropriate description which is about using the data available to draw a “most likely” conclusion.
• There is some evidence for students getting to answers without really understanding the cognitive approach they have taken however this appears to link to satisficing in a position of lack of knowledge rather than being linked to inductive reasoning.
• Satisficing occurs commonly but is multifactorial in nature, particularly around knowledge and time pressure.
• Students are aware of biases, although there is confusion with heuristics. The evidence that they actively seek to manage them is less clear.

Discussion and Implications

The results facilitated a revision of the concept framework and consideration of number of key areas. Issues for assessment include time pressure, question ordering, assessment schemes and implications for workplace-based assessment. Issues for education include diagnostic stewardship, situated cognition, teaching approach, bias and technological implications.
The convergent results show a number of implications for education, policy and research.
For education improving clinical reasoning will require a shift in the way that curricula are planned and delivered. This will have implications for faculty training, including in the workplace with our non-core staff, particularly more junior clinicians. This work also challenges our assumptions that our assessments are effective in ensuring that students are appropriately prepared for decision making in practice.
For policy, the equal footing of clinical reasoning with respect to clinical knowledge needs to be emphasised by regulators and educational bodies. Evidence demonstrates that clinical reasoning errors directly lead to poor patient outcomes thus instilling a requirement for this in policy makes sense.
Further research on aspects of clinical reasoning within assessments and curricula is crucial including looking at workplace-based assessment and whether it compliments these current methodologies. Further research into the delivery of education in clinical reasoning, both in terms of faculty and student understanding, will further improve the delivery of education in this area.
The design, and subsequent revision, of the concept framework through this piece of work can act as a guide for educators when designing programmes and underpin the education and curricula plans.