If you’re stuck in the education bubble, the big news for this week is Education Week‘s release of its annual “Quality Counts” report. The national publication uses a wide range of metrics to rate states on their K-12 performance and policies. Since I can guarantee you that some prominent official or media figure in 2011 will turn to “Quality Counts” to make this or that case about Colorado K-12 education, I figure a little up-front clarification is in order.

First of all, let me say that the report produced by Education Week contains a lot of valuable information — and especially in the categories of K-12 Achievement; Standards, Assessment and Accountability; and the Teaching Profession. But I admit to find it confusing that one-third of the overall rankings is built around two factors that rely heavily on inputs, rather than outputs. I’m talking about Chance of Success and School Finance.

Writing at the Flypaper blog, Fordham’s Liam Julian does a good job of explaining how the “Chance of Success” index continues to be misleading, though it has improved somewhat from earlier editions. Why are we combining factors outside schools’ and policymakers’ control — such as family income and parental education — to determine a state’s letter grade? And this concern doesn’t even include the major part of the School Finance category, which downgrades states (as Fordham’s Emmy Partin notes) “for not spending enough on K-12 education without examining whether they actually need to be investing more dollars in their schools.”

The good news, though, is that if you don’t like the factors Education Week uses to rate the states, you can use their handy calculator to assign your own weights to all the different categories and subcategories. You may or may not like what such an adjustment does to your state’s letter grade (making my adjustments dropped Colorado ever so slightly from 73.7 to 73.1, still a C). However, you can advance the discussion about which factors should be given more or less consideration in making these important judgments.

So just remember… the next time someone trots out a letter grade or ranking from “Quality Counts,” ask yourself what it’s really showing. (And ask why the quality and availability of educational choices isn’t included anywhere in the calculation.) Look at the merits of each category and subcategory, and decide for yourself whether and to what extent the factors measured truly tell us what we need to know.