Wednesday, March 23, 2011

97% is not a passing mark! (More to CIHI report on wait times than meets the eye)

Can you score 97% on a test and still fail?

CIHI just released its 2010 results on which provinces are meeting national benchmarks on wait times for specified procedures.   I took vague note of the results for hip and knee replacement, surgery for fractured hips, cataracts and cardiac bypass.  I don't have much to do with those procedures in my urology practice.

But, I was interested in the Star-Phoenix's report that 97% of Saskatchewan patients receive radiotherapy (for cancer) within the benchmark time (4 weeks).   Nice!  But, wait...

I regularly refer patients for radiation treatment, usually for prostate cancer.  My impression is that patients usually wait longer than 4 weeks for their treatment.  I often get phone calls from patients I've referred, asking when their treatment will start.  I usually quote a wait of 8-10 weeks from when I send a referral letter to when they start their treatment.

Perhaps I'm not speaking the same language as CIHI.

The CIHI report states that the 4 week wait is measured from when patients are "ready to receive care".  Interesting.  I would consider most of my patients to be ready to receive care from the moment I refer them.  Some still require xray testing to be completed, but there would be very few men who are medically unfit to receive treatment.  So, why the discrepancy between my perception of patient wait and CIHI's report?

On the Saskatchewan Cancer Agency website, "ready to treat" is explained as "the date that the patient is ready to be treated, taking into account clinical factors and patient preference".  So, "ready to treat" equals "ready to be treated"...

I was no further ahead after reading this, so I asked a senior physician at our Cancer Clinic what "ready to treat" meant.  His answer was more enlightening:

Ready to Treat means the patient has been assessed by a Radiation Oncologist with all necessary work-up completed, treatment options considered and a consent for Radiation therapy signed.  It means that if simulation and planning could be done in minutes the patient is ready to start treatment that day. It means the patient is available and willing to start.
OK, now I get it.  It means we've stacked the measurement deck by ignoring all the heavy lifting necessary to get the patient to the point of "ready to treat".  Here's what goes on before the official clock starts:

Referral letter generated and sent to Cancer Clinic
Letter reviewed by triage clerk
Letter reviewed by Radiation Oncologist
Appointment date assigned
Consultation with Radiation Oncologist
Further testing (possibly)
Patient decision to proceed with treatment
Each of the spaces in the above list equals its own wait time.  Who is measuring those waits?  Our patients sure are, but CIHI isn't.  I have no doubt that CIHI recognizes the importance of each of these wait times.  But, there isn't a system in place to track them.

It's relatively easy to track wait times once a patient is in the Cancer Clinic system.  It's harder to track all the other times.  It's even more difficult if you want to measure the patient's real waiting experience, that is, from the time the patient is referred by their family doctor, or even when they first consult their doctor with symptoms.  Who decided on this benchmark anyway?  Did anyone ask patients whether this was truly reflective of what was important to them?

There's a chance that making these easy measurements could actually hamper efforts at overall system improvement.  What if health administrators and politicians look at the "success story" of radiotherapy across the country and decide that it's "fixed", and that attention and resources can be moved elsewhere?

If my son came home from his basketball game and told me that he had scored 50 points, I'd be curious how that had happened.  I wouldn't be surprised to find out that the baskets had been lowered to 6 feet high.  Easy slam dunk.

In a health care system that has universal struggles with access, we should be suspicious when one area seemingly slam dunks the access problem.  Their basket is too low.

97% = Fail.


  1. Couldn't agree more. There has been talk for some time in Ontario that people are consciously not entering patients on wait lists until they will fall into the new guideline. Guideline met and any financial incentives received.

  2. Great point. I am suspicious of the cheer leading going on when my own experience with real patients indicates otherwise.

    Thanks for your excellent blog.


  3. Thanks for the comments and the encouragement.

    I see the ubiquitous Anonymous is suggesting that people might game the system if financial incentives were at stake. How cynical. Tsk, tsk.

  4. (different Anonymous)
    An excellent blog. All of this points to the continual data problems in health care, and in Canada in general (like getting rid of the 'mandatory' in the long census form *cough*).

    I understand that establishing medical benchmarks is difficult as well. How do you know when exactly a particular patient needs to receive care? How do you determine "medically necessary", especially if someone is in pain but is not at risk for it to develop into other medical problems. Some may be a higher priority than others, but in general I can see the difficulty.

    Perhaps it would be better to blatantly describe the methodology in these results. I think CIHI is doing the right thing by at least starting somewhere, but there is definitely room for improvement.

  5. Thanks, different Anonymous. CIHI is making a start at measurement and transparency - that's great. I am certain that CIHI staff understand that they are not measuring the whole patient experience, especially with their radiation therapy wait times. I worry that the end-users of the CIHI information (policy makers and administrators) don't understand that and might simply slap a "Mission Accomplished" sticker on that procedure.

    As you suggest, it might be better to clearly describe what is being measured and also point out the other components to the wait time that patients actually experience.

  6. (different Anonymous)

    Point taken.

  7. I find this posting interesting and as of yet never been compelled to post a note to your blog. I have to state I know a lot of the great work that the Saskatchewan Cancer Agency is doing and find this posting both interesting and somewhat insulting. Canadian (not only Saskatchewan) health care is working hard on improving. Just as you are working on improving service to your practice so is the SCA working on reducing wait times to their practise. I see no mention of the recent SCA press release of “reducing the time between the patient’s first appointment with an oncologist and their CT planning
    appointment”. The SCA team worked their butts off and improved the patient experience by 92%, actually giving the patient the option for almost same hour appointment. Pretty darn good by any measure and they did it using Lean Six Sigma, team work, and sweat. I know you are also very aware that there are many steps to a patient journey and when someone has prostate cancer there may be extra steps required before they can receive actual treatment e.g. markers placed and the swelling must go down before treatment thus increasing the time for treatment. I also know that you fully understand that if we want to improve the health system we have to work together and not instigate a mindset of us against the man. Should I post on your blog that it took me four months to get in to see my urologist? That serves no purpose. (I am feeling much better now though)
    I am extremely fortunate to be working in health care from BC to PEI and get to meet all the great people that come in each and every day for no accolades and do the work needed. CIHI may set a “standard” of measure; that’s fine, we as a society set a standard of tolerance. If CIHI wants to set the current bench mark as four weeks and their start and end points are established so be it. When all the Saskatchewan data is reviewed it shows that for years 06 and 07 a different reporting method was used, it also shows that under a new reporting method for years 08 and 09 an increase from 78% to 97% was achieved. Regardless of how it’s measured that fact is it got better. Baseline is baseline, CIHI may be off on their baseline start and finish points but we have to live with what we are given sometimes. I believe the saying is “don’t sweat the small stuff”. Beating people up that are trying to do their best in the incongruent system they have been handed won’t help them or us.
    Keep up the good work, my only suggestion would be to swing back to the positives you used to write and help transform our health system as I know you passionately want to do.

    Hope I did not open a hornets’ nest.....

    Dale Schattenkirk

  8. Thanks, Dale, for the comment. Your passion for improvement comes through clearly.

    I may not have been clear with my point. I don't fault the people who are working hard to improve the healthcare system. My concern is with the way results are reported. I think that measurements should be made in ways that are meaningful to patients. The 97% success rate is for a very small part of patient experience. What do patients think when they see a supposed success story that doesn't fit with their own experience? I think they will lose confidence in the reporting.

    I haven't said anything negative about the clinicians and administrators who continue to work to improve patient experience and reduce waste.

    As for your question as to whether you should "post on your blog that it took me four months to get in to see my urologist?" - of course you should! That was your experience, honestly reported. That's the type of information that makes us continue to look for ways to improve how we serve our patients.

    As for your suggestion that I "swing back to the positives" I used to write, hmmm. That got me thinking in this post.

  9. Simple improvements:

    Remove First Referral Letter, which can be sent while waiting for your first appointment (appointment having already been made).

    Remove Referral by Triage

    Remove Letter reviewed by oncologist (he will have the letter by the time your appointment rolls around).

    Your GP gets your results and calls the triage clerk and says, "I have this person with prostate cancer/symptoms of prostate cancer. When can I get him in?

    The reason I believe this would work is because I live in a country (U.S.) where both my in-laws have beaten cancer (one prostate, one colon) and it was fast. FAST.

    Any specialist I see is done just the way I mentioned. I get an apology if I wait longer than a week.

    Rather than ANYONE telling Dr. Visvanathan that he should change ANYTHING about his blog, they should instead go straight to their politicians and say, "I will donate real dollars if you can start legitimately fixing the mess that is Saskatchewan (Canadian) medicine.

    I am so sorry to see the mess that is Saskatchewan medicine. Tommy Douglas must be spinning in his grave.

    Dr. Visvanathan: Thank you for staying in a province that doesn't treat you with the respect you deserve. I love SK and my deepest wish is to come back to retire, but the medical care scares the heck out of me. I'm sure what I suggested sounds hopelessly unrealistic. Maybe it's just something to strive for?

  10. Thanks for the comment, Bohica. Thanks also for sticking up for me, but I don't feel that anyone's comments on this blog have showed a lack of respect. I relish comments that challenge what I've written because that's the way I'll learn and think more deeply about these issues.

    And, finally, thanks for inspiring this post.