0
£0.00
0 items
Over recent years the Society of Radiologists, the Society of Radiographers and the national government have introduced initiatives for non-medical health professionals to extend their roles and develop their skill-mix in order to meet increasing service demands (Royal College of Radiologists, 2001, Department of Health, 2000 & Society of Radiographers, 2003). The number of health professionals performing and independently reporting ultrasound (US) examinations has gradually risen over the past few decades. . More recently this has been the case in musculoskeletal (MSK) US. Despite its undeniable value, concerns about the training competency of practitioners still wait to be addressed (Taggart et al, 2006). To prove and maintain competency in US, the clinician must perform regular audits to ensure the delivery of a quality service
Clinical audit is defined as a dynamic quality-assurance tool that is used to sample the performance in a department for comparison against the desired standard.
Clinical audit is promoted by the National Institute of Clinical Excellence (NICE) and the healthcare commission to improve patient care and clinical outcomes to monitor standards and implement change. It was introduced into the National Health Service (NHS) in 1993, and was traditionally carried out by doctors but now involves all health care professionals including sonographers (Price et al, 2002). The Royal College of Radiologists and the United Kingdom Association of Sonographers (UKAS) deem clinical audit to evaluate clinical practice as a required component of implementing clinical governance (Stoyles & Harrison, 2006).
Clinical audit is thought of as a cycle (Figure 1). The cycle revolves around choosing an explicit benchmark, evaluating local clinical practice against that benchmark and coming up with an implementation strategy to introduce change into local clinical practice. If changes are introduced re-auditing is essential to confirm the implementation strategy was effective and the required standard has been maintained in clinical practice (Price et al, 2002).
The aim of this audit was to assess the diagnostic accuracy of the student sonographer by attempting to quantify inter-operator agreement between the student and an experienced musculoskeletal (MSK) sonographer and comparison with magnetic resonance imaging (MRI).
Figure 1 – The audit cycle (Al-Quasi et al, 2007)
Data from 150 (1 – 150) MSK US examinations were included in the audit. The examinations were performed over a 9 month period (April 2009 – Feb 2010) and took place in the Physiotherapy Department at St.Leonards Hospital, City & Hackney Primary Care Trust (PCT).
Of the 150 scans in the audit, 125 were audited against one of two supervisors, JL and RL. JL and RL are physiotherapists and sonographers, both with over 10 years experience in MSK US. The clinics took place in a busy clinic in a primary care setting and with a student - supervisor ratio of 2:1. Consequently, there had to be flexibility in the assessment process. For each examination the student recorded appropriate images and clips for each scan and wrote a diagnostic report. The supervisor then carried out a scan on the patient immediately after the student or the supervisor reviewed saved images and clips retrospectively at the end of the clinic. As MSK ultrasound image interpretation relies on a dynamic assessment of the patient, a majority of the saved images were in a clip format. For both methods, the supervisors were blinded to the interpretation of the student to limit any potential source of bias. If there was any disagreement, the scan findings were in agreement with the supervisor.
Of the 150 scans 25 of the examinations were reviewed against an MRI scan. In all 25 cases the MRI scan was either requested by the student sonographer following the US scan, or by an orthopaedic consultant following onward referral by the student. Auditing practice with other imaging techniques is essential for assessing the accuracy of diagnostic practice (Allen & Wilson., 1999).
Prior to commencing a scan, the student, in his role as a musculoskeletal practitioner, carried out a clinical examination which involved basic history taking and assessing the appropriate joint(s) or soft tissue(s) for the presenting condition. This represents the clinical context the student uses US in their daily practice. It has been
suggested that US might be at its most valuable when the practitioner interprets the scan findings and compares them immediately to the clinical findings (Awerbuch, 2008). The student carried out the clinical examination prior to the scan as they felt this enabled them to target the scan to a certain area to answer a specific clinical question.
Ultrasound is known to be an operator dependent imaging modality. It has been suggested that standardisation of the examination technique can improve the interobserver reliability and improve diagnostic accuracy (Rutten et al., 2007). The student had received approximately 10 hours of tuition with the supervisors prior to commencing the audit, which helped to standardise the examinations and agree on a glossary of terms for the diagnostic reports. The examinations were performed as standard protocols as recommended by the European Society of Musculoskeletal Radiology, with an additional group classified as muscle injuries.
The same machine was used by both the student and the supervisors on the same patient to reduce interobserver variability. Two machines were used throughout the audit process; a Sonosite Micromaxx and a Siemens Acuson (Figure 2).
The 150 examinations were graded for discrepancy (Table 1). For the 25 scans compared with MRI, the student allocated and recorded a score for each scan on the audit form after discussion with one of the supervisors. If there were any uncertainties regarding the MRI report, the student discussed them with the supervisor or reporting radiologist. For the 125 unsupervised scans, a supervisor recorded the agreement scores. Any discordant cases were reviewed by the student and the supervisor together. The data from this audit was collected concurrently to ensure the student was reflecting prospectively and learning throughout the audit process.
Figure 2 – The machines used a Sonosite Micromaxx (left) and a Siemens Acuson (right)
 |
 |
The agreement scores were then reviewed by the student and the percentages for each grade were calculated. In cases of significant discrepancy, e.g. grades 3 and 4, an amended report was sent to the relevant health professional if e.g. GP, physiotherapist and/or consultant to ensure patient management was not compromised.
When carrying out a clinical audit to evaluate local clinical practice an acceptable benchmark must be defined (Stoyles & Harrison, 2006). It has been suggested in the literature, for a group of rheumatologists, that an 80% agreement with an examiner is an acceptable level (Taggart, et al., 2006). In a recent audit, involving MSK sonographers, they used a 95% level of agreement between the sonographer and a consultant MSK radiographer (Riley, 2010). This study was auditing a newly- appointment sonographer with 8 years MSK experience, and so the agreement level was deemed a little high, as the student in this audit had approximately one year of previous scanning experience.
At present there is very little evidence regarding a standardised agreement score to achieve a recognised competency for non-medical practitioners. The author of this audit suggested an 85% level of agreement was used to benchmark the student’s performance.
The breakdown of the anatomical sites examined and the agreement scores of the students’ performance against the experienced sonographers (JL and RL) and MRI, are presented in Table 2. Shoulder examinations accounted for 38% of all examinations, 23% knee and 20% shin, ankle, and foot.
The overall grading scores and percentage agreement for the 150 cases audited are presented in Table 3. The student was in complete agreement with the experienced sonographers and MRI in 135 of the 150 (90%) cases. The student scored 116 of the 125 (93%) examinations grade 1 (complete agreement) when compared to the supervisors US and when compared to MRI he scored 19 of the 25 (76%) examinations grade 1 (complete agreement). Nine of the 125 (7%) examinations were scored as grade 2 (minor disagreement) when compared to the supervisor and compared to MRI, 3 of the 25 (12%). No examinations when compared to the
supervisors US were scored as grade 3 (potentially significant) or grade 4 (definite, significant discrepancy). Two of the 25 (8%) examinations scored a grade 3 (potentially significant) and 1 of the 25 (4%) examinations a grade 4 (definite, significant discrepancy). Table 4 shows a detailed description of the examinations with discrepancy between the student and the supervisor or MRI findings.
Table 4 – Details of the 15 cases of discrepancy between the student and the supervisor or MRI
A vast majority (125 of the 150) examinations measured inter-observer agreement between the student and experienced sonographers, and did not consider specificity and sensitivity, as this would have required a diagnostic gold standard. Only, 25 scans were compared to MRI, for financial and ethical reasons, and the author did not feel this was enough to calculate a meaningful sensitivity and specificity. Surgical evidence of pathology would be considered the diagnostic gold standard for a majority of the pathology seen in this audit (Fotiadou et al., 2008; Madhusudhan et al, 2008; de Jesus, 2009). A vast majority of the patients were treated conservatively and so this gold standard could not be achieved. The student plans to re-audit in 6- 12 months to ensure he is improving on the standards set in the current audit and further develop his knowledge and skills. Since the audit the student is working very closely with a shoulder and knee surgeon and is likely to be referred a greater percentage of patients with significant structural abnormalities that may require surgery.
Table 2 shows that the students report when compared with the supervisors’ findings, or an MRI scan, were in complete agreement (grade 1) in 90% of the cases. This level of agreement is considered high and exceeded the standards of practice set for this audit (85%). Image 1 shows an example of an examination that was in complete agreement with an MRI scan.
Thirty eight percent of the examinations were carried out on shoulders, 23% knee and 20% shin, ankle, and foot. This is a fair reflection of the student’s caseload and common MSK pathologies seen in a primary care MSK clinic. This non-uniform number of examinations performed for each anatomical area would have affected the percentage agreement.
Image 1a & b – Scan 225 - Rupture of the Achilles tendon on longitudinal section (1a) and transverse section (1b). The blue crosses in image 1a show the two ends of the tendon.
The student achieved a good agreement level with shoulder diagnosis. All 9 full thickness tears of supraspinatus were correctly identified by the student. The current evidence base has demonstrated high sensitivity and specificity for detection of full thickness tears of the rotator cuff (Naqvi et al, 2009; de Jesus et al, 2009). Most of these scans were carried out on the older population, in these cases the detection rates have been shown to be very high and missing even one is considered unacceptable (McNally, 2004, de Jesus et al, 2009).
In 15 (10%) of the 150 cases it was considered there was a discrepancy between the students report and the supervisor or MRI findings. In 12 (8%) of the 150 cases this was graded as a grade 2 i.e. a minor discrepancy unlikely to affect patient management. One example of this was scan 140, where the student reported supraspinatus tendinosus. The examiner felt there was an articular sided PTT in supraspinatus (Image 3). It is acknowledged in the literature that differentiation between tendinosus and partial thickness tears is very difficult on ultrasound (McNally, 2004). In this case the patient was treated conservatively which is the recommended management for partial thickness tears of the rotator cuff. Only if conservative measures fail would surgical management be sought (Matava et al, 2005). Therefore, the discrepancy did not affect patient care. The discrepancy in scan 157 was also related to differentiating a PTT from tendinosus. The student reported tendinosus and failed to identify a PTT tear of supraspinatus. A cause of failure to detect PT RC tears on US can relate to the chronicity of the lesion. A recent tear with some effusion can be easily identified, whereas a chronic lesion, like the tears in this case, without an effusion and heterogeneous appearance may mimic tendinopathy (Fotiadou et al., 2008).
In 2 (1.3%) of the 150 cases the grade of discrepancy was 3 (potentially significant discrepancy). Both cases were in comparison with MRI. In one case the student missed a significant PTT of infraspinatus and the other case a significant PTT of supraspinatus. Although, the patients’ immediate management was not affected by the discrepancy, both patients failed to improve with conservative measures and were referred for an MRI and a surgical review. The supervisor and the student deemed these to big significant PTT that the student should have identified on ultrasound. The supervisor reviewed the patients with the student following the MRI result and the tears were visualised clearly by placing the tendon on a significant stretch and following the individual fibres of the tendon. The amount of stretch used was more than used by the student on the original examination.
In 1 (0.7%) case of the 150 examinations the grade of discrepancy was 4 (definite, significant discrepancy likely to have adverse concequences for the patient). In this case the student reported a FTT of subscapularis, however on MRI, the subscapularis was intact. A FTT of subscapularis on a 46 year old, who is otherwise fit and well, is likely to require surgery and therefore the patients care could have been compromised (McNally, 2004, de Jesus et al, 2009). The student re-scanned the patient on follow up and could clearly visualise the intact subscapularis. Poor scanning technique of subscapularis was responsible for the error. An oblique probe placement in the longitudinal section gave the appearance of discontinuity in the tendon and in the transverse section, anisotropy gave the appearance of a hypoechoic lesion in the tendon.
This was a prospective audit which has been shown to be a more accurate reflection of practice than a retrospective audit (Al-Qaisi et al, 2007). Approximately a third of the scans assessed by the examiners were carried out retrospectively. Retrospective review of images and clips is not a complete reflection of the original examination.
However, reviewing the images and clips is a relatively simple and cost-effective way of carrying out quality assurance audit. Cases with a significant discrepancy were discussed with the two examiners to ensure patient management was not compromised. The agreement between the examinations and reports of the two supervisors (JL and RL) is unknown.
US provides a cost effective, safe, efficient and portable imaging technique. It allows immediate diagnosis and can be used for therapeutic interventions. It forms an essential part of a ‘one stop shop’ for MSK conditions in primary care. The advantages of US in the delivery of MSK care have increased the demand for more MSK sonographers (Department of Health, 2000). Despite its undeniable value, concerns about the training competency of practitioners still wait to be addressed (Taggart et al, 2006). It has been shown that the experience of the sonographer directly relates to diagnostic accuracy (O’Connor et al, 2005). It is essential that individuals and departments carry out regular audits, implement changes and review those changes to ensure best standards are maintained.
Despite achieving a 90% agreement percentage during this audit, it is essential that the student continues to audit his own practice to further maintain and develop his skills, improve quality and clinical outcomes (Al-Qaisi et al, 2007). This is also a
basic requirement of UKAS. The student has identified the following areas to improve practice and develop his skills and knowledge:
The audit shows a high level (90%) agreement between the student and the experienced sonographer and MRI. This audit has been an invaluable learning experience for the student and it has identified key learning areas for future development. It is essential that a standard for future quality assurance audit of non- medical MSK reporting practitioners is set. The standards will help to ensure the delivery of a safe, high quality and reflective MSK ultrasound service.
Allen, G., Wilson, D. 1999. Ultrasound and the diagnosis of orthopaedic disorders. Journal of Bone and Joint Surgery Br. (6): pp. 944 - 951
Al-Qaisi, M., Deane, C., Goss, D., et al. 2007. Clinical Audit – an update for ultrasound practitioners. Ultrasound, 15(4); pp. 236-239
Awerbuch, M., 2008. The clinical utility of ultrasonography for rotator cuff disease, shoulder impingement syndrome and sub-acromial bursitis. Medical Journal of Australia. 188(1), pp. 50-53
de Jesus , J., Parker, L., Frangos, A., Nazarian, L., 2009. Accuracy of MRI, MR Arthrography, and Ultrasound in the Diagnosis of Rotator Cuff Tears: A Meta- Analysis. American Journal of Roentgenology. 2009; 192:1701-1707
Department of Health. Radiology skills mix: A report on the four tier service delivery model. London: Department of Health, 2003
Fotiadou, A., Vlychou, M., Papadopoulos, P., et al. 2007. Ultrasonography of symptomatic rotator cuff tears compared with MR imaging and surgery. Europeon Journal of Radiology. 1-6
Madhusudhan, T., Kumar, T., Bastawrous, S., et al. 2008. Clinical examination, MRI and arthroscopy in meniscal and ligamentous knee Injuries – a prospective study.
Journal of Orthopaedic Surgery. 3: 19
Matava, M., Pursell, D., Rudzki, J., 2005. Partial thickness tear rotator cuff tears.
American Journal of Sports Medicine, 33, pp. 1405 - 1417
McNally, E., 2004. Practical Musculoskeletal Ultrasound. Churchill Livingstone
Naqvi, G., Jadaan, M., Harrington, P., 2009. Accuracy of ultrasonography and magnetic resonance imaging for detection of full thickness rotator cuff tears.
International Journal of Shoulder Surgery. 3(4): pp. 94-97
O’Connor, P., Rankine, J, Gibbon, V. 2005. Interobserver variation in sonography of the painful shoulder. Journal of Clinical Ultrasound, 33: pp.53 - 56
Price, R. Miller, L., Mellor, F., 2002. Longitudinal changes in extended roles in radiography. Radiography. 8: pp. 223-234
Riley, S., Groves, C., Chandramohan. 2010. Musculoskeletal ultrasound: audit of sonographer reporting. Ultrasound. 18: pp. 36-40
Royal College of Radiologists. Clinical Radiology: A workforce in crisis. London. Royal College of Radiologist, 2001
Rutten, M., Maresch, B., Jager, G., et al. 2007. Ultrasound of the rotator cuff with MRI and anatomical correlation. European Journal of Radiology. 62: pp.427-436
Society of Radiographers. Role development revisited The research evidence, 2003. London: Society of radiographers, 2003
Stoyles, T., Harrison, G., 2006. A national survey of sonographers and their reporting practices in the general medical ultrasound field. Ultrasound. 14: pp. 109-114
Taggart, A., Filippucci, E., Wright, G., et al. 2006. Musculoskeletal ultrasound training in Rheumatology: the Belfast experience. Rheumatology. 45: pp. 102-105