5 Stories: Breast Cancer Screening

Late last year the U.S. Preventive Services Task Force released a report that significantly changed the breast cancer screening guidelines for women. They raised the age where mammograms are recommended from 40 to 50. This set off a firestorm in the country and perhaps rightly so as a significant number of cancers are found via mammograms in women between 40 and 50. The report remained in the national spotlight for several weeks and the task force was brought before Congress to testify as to what the report really meant. Furthermore, since this recommendation was published in the middle of the health care debate, many wondered if the motivation was a cost saving move, implying the science was not as conclusive as the financing. Finally amid all this controversy the task force revised its recommendation to more clearly state that women should consult with their doctors and follow their doctor’s guidelines.

This article is one in a series of articles that asks whether a data graphic would have altered the course of a recent news story. Actual data was not collected for this particular article (though a future article may show actual data) in part to allow us to focus on the issue of how the story would have changed with a good data graphic. Indeed without a data graphic many woman and insurance companies may have thought the recommendations meant that mammograms were 0% effective for women under 50 and 100% effective for women over 50. Subsequent interviews with the task force members made it clear that’s not what they meant, but words alone are weak tools at conveying the proper meaning here.

sampleChartHThe graph above charts a hypothetical “effectiveness” factor for mammograms by age. Effectiveness is intentionally vague here, a real chart would undoubtedly blend many complicated factors together to get a single “effectiveness” number. Those factors might include the rate of false positives, risks and inconvenience from unnecessary biopsies, risks from unnecessary x-ray exposure and etc. But let’s assume the task force could have created a graphic like the one above as part of their recommendation. The task force might also have agreed to a “cut-off” value ahead of time, e.g. an effectiveness of greater than 50% would be recommended and one of less than 50% would not be recommended. This isn’t unusual, doctors have similar guidelines for other tests. Amniocentesis testing for example. The risk of miscarriage from an amniocentesis test is between 1 in 200 and 1 in 400, similarly for mothers between the ages of 34 and 36 the risk of having a downs syndrome child is between 1 in 200 and 1 in 400. Consequently the guidelines are for women to be tested when they’re 35, approximately where these 2 risk factors are equal. Similarly drawing a mammography effectiveness line at 50% is entirely plausible.

The hypothetical chart above shows that mammograms are clearly not the optimal screening method for women under 40. The ‘effectiveness’ ranking is below 50. Similarly it clearly shows that for women over 50 mammograms clearly are effective. But this middle ground between 40 and 50 is tricky to describe. Technically it too falls under the threshold, but only just. If you were on the task force and saw this graph in your mind’s eye and were forced to explain your recommendation in words alone, how would you do it?

Once you include a graph the question naturally turns to other graphs. The graph above is for the “average” woman. How would the graph look for women with a family history of breast cancer? How would it look for women with no risk factors? The graph below shows these possibilities. In this case women over 40 with a family history are shown above the 50% line and women over 40 with no risk factors are clearly below the 50% line.
sampleMultiChartHIf the Task Force’s report were issued with (actual versions of) these graphs (and if the news media carried them) the weeks long national firestorm that accompanied the announcement could have been minimized.

4 comments for “5 Stories: Breast Cancer Screening

  1. Kevin Henry
    March 18, 2010 at 3:23 pm

    This is a fine idea, but the y-axis is ill-chosen.

    It implies that for any population group the test is at least somewhat effective – after all, something 10% effective is better than nothing, right? In fact, for pretty much any medical procedure – including mammography – there are measurable risks that make a procedure actually harmful for certain population groups.

    Your discussion of the amniocentesis test makes this clear, but it’s a mistake to refer to the balancing point as being “%50 effective” – that should be the “0%” point. Values above that are helpful, and values below that are harmful. Which is very different from calling something “40% effective”.

  2. numbersguy
    March 19, 2010 at 10:07 am

    Hi Kevin:

    What would you use instead? I chose “effectiveness” here and a 50% cut off because it’s a simple concept that would convey the gist w/o the burden of a more accurate but harder to describe metric. This is a hypothetical graph after all. In reality I expect the task force looked at data from a variety of sources not necessarily directly compatible with each other. Risk of false positives times risks from unnecessary biopsies could be compared to risks from either missing or diagnosing cancer outside of a routine mammogram. Those risks could be portrayed in a compatible way, but I suspect that the actual evaluation methodology might have included other features not so easily comparable. (This might be why the task force didn’t create such a chart, it was hard, but IMHO would have been immensely useful even if approximate.)

    Additionally, graphing results like this might have shed light on how effective the current “gold standard” method is. One thing that struck me in the task force’s recommendation is that mammography is a pretty lousy screening technique for women under 50. (A graph might have made this clearer.) I’m puzzled as to why that aspect wasn’t discussed more heavily and why no one advocated more research dollars into creating a better screening technique for younger women. That might have been a more accurate headline, “Task Force Finds Mammograms Less Effective Than Thought For Women Between 40 And 50”

  3. Kevin Henry
    March 20, 2010 at 2:27 am

    Hello,

    I’m not disagreeing with the usefulness of such a graph, and my comment isn’t related to the complexities and difficulties of creating such a metric.

    I’m saying that whatever the metric is, it is combining costs and benefits (risks and rewards, etc.) into a single number, and that the natural way to express this is with a metric that centers on 0, with values above that being, on balance, helpful, and values below that, harmful. Expressing this as a percentage scale with a 50% cutoff point seems arbitrary and non-standard for a cost / benefit analysis.

    My other point is that the word “effectiveness” is misleading. In the everyday use of this word, something that is a little bit effective is better that nothing at all – but that’s not true here, where values below 50% effectiveness are worse than nothing.

    Again, I agree with your main point, but a confusing or misleading graph can be worse than none at all.

  4. August 3, 2011 at 8:06 am

    Thank you so much for giving everyone remarkably wonderful possiblity to read from this site. It’s usually so superb and also stuffed with a great time for me personally and my office acquaintances to visit your blog a minimum of three times a week to see the new things you have. And lastly, I’m so always fulfilled with your awesome methods you give. Certain two areas in this article are unequivocally the most suitable we’ve ever had.

Leave a Reply

Your email address will not be published. Required fields are marked *