Healthy Skepticism

Canadian Medicine/National Review of Medicine recently featured an Annals of Internal Medicine paper that reported an attempt to implement Advanced Access in several American primary care practices. The Canadian Medicine post summarizes the study's findings (you can read the abstract here); essentially that it was difficult to maintain improved wait times in the study groups. Also, the study didn't find any improvement in other parameters like no-show rates, and patient and staff satisfaction.

As noted by commentators on both the Canadian Medicine and Annals sites, the lack of change in these measurements is not surprising, as the practices didn't successfully implement Advanced Access, and therefore couldn't be expected to reap its benefits. Advanced Access-expert Mark Murray pointedly diagnoses the problems with this study.

It may be that there was a lack of buy-in among the clinic staff in the practices studied. Even though the investigators who wrote the paper and supported the implementation efforts may have been highly committed, if the "troops on the ground" weren’t engaged, the initiative would fall apart.

This report raises the issue of the tension between evidence-based medicine’s rigid approach to assessment and the Quality Improvement movement’s "just do something" mantra. IHI’s Don Berwick commented on this in a March 2008 JAMA editorial. He advocates embracing methods of statistical proof other than randomized clinical trials (RCT). RCTs are notoriously difficult to conduct, and are resistant to mid-course modification should unexpected findings arise. However, other commentators stand by RCTs’ proven value in eliminating unforeseen biases when new treatments, technologies, and techniques are studied.

A classically-designed RCT puts me in mind of a "square wheel." It takes a huge amount of preparation and energy to achieve one lurching movement of such a wheel. Whereas a Quality Improvement "round wheel" (epitomized by the PDSA cycle), can be kept in motion by many small pushes (tests of change).

In the case of this study of Advanced Access, if you had your shoulder to the square wheel, and the first big push didn’t seem to make progress, you’d be very tempted to quit. However, if you were rolling the round wheel, the momentum built up by successful pushes will keep you going despite the occasional failure.

In an attempt to quantify Advanced Access, I wonder if the study authors ignored the fact that Advanced Access is really part of a larger initiative: Clinical Practice Redesign (CPR). That is, you can’t succeed with AA unless you’re seeing it as part of a broader attempt to eliminate inefficiencies in a practice. And, I think CPR is not going to achieve its optimal effect unless it’s considered as part of the Patient and Family-Centred Care concept: All the changes we make should be focused on improving value for our patients/clients – as determined by our patients/clients!

Even though this study has been touted as a failure of Advanced Access to achieve its goals, I think it’s really a demonstration of the challenges of implementing change. It would be interesting to see a measure of the degree of commitment among the staff and team leaders at the start of the project (we’ve used the DICE tool in our office AA project).