The Danger in State Rankings

Governing Magazine published an article that ranks the Arizona economy 41st worst in the nation. They took the time to examine six data points that were easily collected from federal government websites. This wasn’t a discussion of school funding, or tax cuts, or anything like that. It was just based on broader economic measures like the unemployment rate, state GDP, and personal income. On the surface this sounds okay. Here is why it’s not.

In this case, while they tried to stay out of political issues, they failed to take the time to understand the individual economies. If you don’t understand something, then don’t write about it. I have delivered hundreds of economic presentations and have written dozens upon dozens of economic papers, and I have never reached a conclusion by reviewing such a limited list of statistics. Each state or metro region has an economic base that is influenced by many factors. Other things to review include infrastructure quality; workforce availability, affordability, and quality; select education issues; real estate availability; marketing and economic development programs; tax base; etc. etc. etc. A full list would take several paragraphs to write. 

Furthermore, while sometimes it’s okay to compare points across states, other times we must track how we compare to ourselves over time. Think about it like golf. You might have a buddy that is better than you, but you compare your performance on any given outing to how you have been doing overall. Your friend may indeed be better with his or her drives and putting, but you may be improving more rapidly overall. This is usually the more important statistic. There was also a failure to examine the economic data over a broader period of time. It’s a basic rule in economics to never take a statistic from a very short period of time and form these bold conclusions.


Then there is statistical error. This is less the case with Governing Magazine’s article which is plagued more by laziness than statistical error. But, other professional organizations that provide rankings similarly look at too few data points and form conclusions that may or may not be correct. I once reviewed a document that ranked Arizona as #1 in the nation one year on an economic development issue, then dropped us to twenty-something the following year. After doing some digging, I applied their margin of error to the ranking and found that we could be ranked #1…or #50.

I don’t mind if people or groups provide an opinion and write a blog or professional article to make a point or vent about an issue. This is a good thing; it gets a conversation started. However, I do mind when normally credible sources form widely distributed conclusions based on lazy work, and mask it by showing some numbers in between the text.

Author’s note: If there is a particular economic or policy issue you would like covered in a later publication feel free to send your ideas and/or comments to [email protected]