The Annual Review of Consultant Outcomes (ARCO) audit is an ongoing project which measures variation in mortality outcomes in Scotland. This is done through support of Public Health Scotland (PHS), which collates routinely collected data, and adopts a case-mix adjustment tool. This is to ensure that factors such as age, comorbidities, and likely prognosis are taken into account when establishing a picture of mortality in Scotland. The audit involves a multi-step verification of data, from case ascertainment, to clinical coding and comorbidities.
About the project
The MSN collaborated with PHS to develop a case mix adjustment model for named consultant outcomes. It is important when reporting on patients who die following neurosurgery that we take into account a number of factors, these include age, gender, deprivation and any other illnesses the patient may have. The Society of British Neurological Surgeons (SBNS) has published outcomes for neurosurgeons in England and Wales. Scotland is uniquely positioned to develop a case mix adjustment model using routinely collected data which can identify and quantify variation in consultant mortality across Scotland. However our wider aim is to align the Scottish data with data collected for consultant outcomes in England and Wales, which will allow the SBNS to complete the UK picture of case-mix adjusted mortality.
Aims
- Ensure validated information that is factually correct is used to determine whether or not an individual surgeon has an unexpectedly high number of deaths
- Ensure factors out with the surgeons control (e.g. long term conditions such as diabetes or heart disease) are taken into account when information is put into the public domain
Data validation
The case listings have been generated from SMR data which is extracted from either the interim discharge letter (IDL) or, ideally, the final discharge letter (FDL) for each patient. Each consultant will have the opportunity to work with their MSN Audit Facilitator to validate their data in advance of publication of the series of performance indicators.
There are four steps involved in validation: case ascertainment, type of admission (elective or non-elective), accuracy of coding (patient’s diagnosis ICD10 and their operation OPCS4), and the patient outcome (alive or dead).
Development of a case mix adjusted model
All neurosurgical cases are extracted from the hospital inpatient and day-case dataset (SMR01), which collects episode level data on all discharges from acute specialities across Scotland. The data is combined into long-term patient histories comprising hospital activity and death registrations supplied by National Records for Scotland (NRS), which can then be used for data analysis in a number of ways.
A series of clinical data items comprising diagnoses and procedures are recorded on all SMR01 records. The GMC number of the clinician undertaking the procedure and date of operation is recorded alongside every procedure code. There are also a series of episode management codes which define the elective or non-elective nature of the admission, and comprehensive demographic data such as age, sex and postcode, which can be used to allocate the patient to the Scottish Index of Multiple Deprivation.
The procedure codes have been used to identify all relevant surgical events that have been undertaken by a neurosurgeon in a Scottish hospital between 2013 and 2015. Then by looking for a death registration and comparing the date of operation with the date of death, we were able to generate an outcome ‘yes/no’ for deaths within 30-days of the procedure. The diagnostic codes comprise a main condition and up to five underlying conditions not necessarily related to their neurosurgery (comorbidities).
The latest Scottish model has adjusted each consultants 30-day mortality for various factors including age-group, sex, type of admission (elective or non-elective), main diagnosis (grouped), comorbidity (Charlson Index) and deprivation (SIMD). We do this by using regression techniques to estimate a likelihood of death for each procedure based on the characteristics recorded on SMR01 (adjustment factors). The number of observed deaths within 30 days for each individual consultant are then compared with the number of deaths that would be expected given the case-mix profile of their individual caseload. The measure is expressed as the Standardised Mortality Ratio. When expressed as a percentage, a value greater than 100% implies more deaths than expected, and a value of less than 100% implies less deaths than expected. The variation in these ratios can then be statistically tested using control limits, commonly in a graphical format known as ‘funnel plots’.
Outcome
The final output is a funnel plot which will show the position of each consultant in relation to the Scottish/UK ‘norm’ and identify if any are outliers.