About that water affordability study
in The Guardian...
Water is a ZILLION PERCENT unaffordable! Also, aliens.
The Guardian recently published a big story on water utility affordability in the United States. The headline was shocking: “Millions of Americans Can't Afford Water, as Bills Rise 80% in a Decade,” and “Analysis of U.S. cities shows emergency on affordability of running water amid COVID-19 pandemic.” The story was based on a study that The Guardian commissioned from Boston-area attorney Roger Colton. Colton’s report—which The Guardian called “the first nationwide research of its kind”* —was the basis for the story’s claims.
The Colton report sets out to do three things:
1) “examine whether the affordability of water is common in the U.S. today”;
2) examine whether “water affordability has changed in recent years;” and
3) “examine the extent to which… reasonable projections of water rate increases will affect water affordability in the near future.”
Put simply, the report aims to measure the level of water affordability in the U.S., past affordability trends, and projected future affordability.
Unfortunately, the Colton report is deeply flawed. Not just flawed in some narrow, technical sense; it’s flawed in ways that grossly misrepresent the state of water affordability in the U.S. and point to the wrong approaches to addressing the issue.
I really didn’t want to write this post. It’s heartening when mainstream journalists pay attention to the oft-neglect water sector, especially on matters of affordability, so I hate to be negative about it. I'm also painfully aware of Brandolini’s Law, and any time spent counteracting bad research is time not spent on my own research.
But over the past month I’ve fielded several inquiries about The Guardian article asking whether the Colton report is valid. The short answer is no.
The long answer requires wading into the tall methodological grass. So grab your weed eater and follow me.
The study
Although The Guardian article’s claims are national in scope, the underlying analysis is limited to twelve cities. The dozen were not selected at random, nor are they the twelve largest. Rather, they were selected by Guardian staff, supposedly “to provide a diversity of geographic regions… population sizes… and poverty.” All inferences from the Colton report come from this hand-picked, non-representative sample of systems that collectively serve a little more than 2% of the U.S. population.
But the Colton study’s deepest problems are about measurement, not sampling. Answering the questions that the study raises requires four things:
a) prices for water service in the United States
b) household resources available to pay for water
c) projected future prices and resources
d) defining affordability
You can think of those as the numerator, the denominator, the forecast, and the judgment. All of them flawed in ways that compound and confound.
The numerator
A study of water and sewer affordability should start with the price of water and sewer service, right? Strangely, the Colton study didn’t collect any rates data or calculate prices. Instead, Colton got price data from Circle of Blue’s annual reports on water prices in 30 U.S. cities. Circle of Blue calculates prices at three benchmark volumes, representing demand for a family of four at 50 gallons per capita per day (gpcd), 100 gpcd, and 150 gpcd. That works out to roughly 6,000, 12,000, and 18,000 gallons per month. Nationally, residential indoor water demand averages around 50-60 gpcd and has been falling steadily for the past twenty years. This indoor demand represents basic needs for drinking, cooking, cleaning, and sanitation. That’s the main public policy concern for affordability—from a public health standpoint we don’t really care about the affordability of lawn watering or car washing.
But the Colton report used Circle of Blue’s 12,000 gallon monthly price as the basis for its numerator. So The Guardian’s startling graphs reflect the prices for for very high volumes of water.
Next Colton used the 12,000 gallon monthly price to “derive estimated bills given average household sizes for each Census Tract,” according to a footnote on page 6. How Colton derived the prices is a complete mystery; he never explains the procedure, and the footnote is the only mention of how he calculated prices.† A simple table of monthly prices is strangely absent from the 88-page report.
The denominator
Claims about the affordability of anything implies some relationship between costs and resources. Colton’s sole measure of resource affordability is the Federal Poverty Level (FPL). The FPL’s quirky origin story is fascinating—it’s based on 1962 food costs—and the problems with FPL are well-documented in voluminous research. Despite its quirks and flaws, FPL remains a touchstone in policy discussions nonetheless, mainly because it’s familiar and easily available from the US Census Bureau.
The main problem with FPL for purposes of assessing water and sewer affordability is that it’s insensitive to local costs of living. FPLS also a one-size-fits-all number national number (hence, “federal”): whether you live in pricey San Francisco or cheap Buffalo, the denominator is the same. Home energy, health care, taxes, and especially housing varies wildly across the country, but FPL is insensitive to those costs. Efforts like United Way’s ALICE are aimed at providing a more realistic assessment of cost of living by accounting for that variation.
The projection
Colton’s study projected affordability through 2030, which involved forecasting both numerators (price) and denominators (FPL). Once again, the report is vague about how it made those projections. A footnote on p.33 indicates that Colton used inflation rates from a 2017 U.S. Department of Energy’s report, which calculated 2008-2016 increases in water/sewer prices for selected U.S. cities based on average volumetric rate. It’s not clear whether Colton used city-specific projections for his twelve cities, or a 4.71% annual inflation rate, which a footnote in a DOE study says was the “national average annual price increase for water and sewerage maintenance” from 1996-2016. It’s impossible to know exactly how Colton projected prices because price projections never appear in the report. To project the denominator, Colton uses 2010-2018 increase in FPL and assumes that the same growth will continue through 2030.
It appears likely that Colton used straightline projection to forecast an already-inflated numerator using with either an eight- or twenty-year retrospective rate of change (it’s not clear which), either nationally or regionally (again not clear). He projected the FPL denominator using a different eight years of national FPL escalation.
The judgment
All of this culminates in a declarations of how many people or what percentage of the population suffers from “unaffordable” water. Those conclusions fuel The Guardian headlines, but require a binary definition of what is affordable. So how did the Colton report define affordability?
“In assessing whether a water bill was ‘affordable,’ the base level of affordability was set at 4% of income,” says the report (p.8). That 4% threshold is the main definition of affordability in both the Colton report and The Guardian stories. The Colton report goes on to set “Affordable Burdens” for various income ranges.
Where did these affordability burden thresholds come from?
Colton made them up.
No poll, no Blue-Ribbon Committee, no philosophical inquiry on the meaning of affordability, no analysis of economic tradeoffs. They’re just arbitrary numbers. Every claim about the nation’s water affordability crisis and most of the scary graphs in The Guardian articles boil down to these thresholds. They’re based on nothing.
The stakes
Does any of this matter? Shouldn’t we just be happy that mainstream publications like The Guardian and Consumer Reports are paying attention to water affordability? Who cares if the methodological details are a bit dodgy, if the overall point—that water prices are rising faster than poor folks’ incomes—is generally fair?
There are at least three big reasons we should care about assessments that are so egregiously inaccurate. First, this study grossly misstates the scope and nature of the problem. It’s good to draw attention, but bad measurement can lead to the wrong inferences about what’s wrong and how to fix it.
The Colton report’s treatment of Austin, Texas is a great example of what’s so pernicious about poor measurement. The Guardian screams that water bills in the Texas capital increased 154% from 2010-2018, and will be unaffordable for 26% of all Austin residents by 2030. The naïve reader could be forgiven for thinking that Austin Water executives are mustache-twirling villains trying to squeeze money out of poor folks. But remember that Colton’s calculations are based 100 gpcd of demand—far higher than basic indoor needs. Austin Water employs a progressive rate structure that’s actually designed to protect affordability for basic needs and curb inefficiently high water use--and the big price increases start at 11,000 gallons monthly. Here’s the Circle of Blue plot of Austin’s water rates from 2010-2018:
At the more reasonable 50 gpcd, Austin’s monthly water prices increased an average of $2.28 annually from 2010-2018—still an increase, to be sure, but not quite the rolling disaster The Guardian describes. Meanwhile, Colton’s 100 gpcd assumption makes Philadelphia’s regressive, declining block rates look relatively affordable, even as the City of Brotherly Love sticks low-volume customers with higher prices than Austin's. Austin’s progressive pricing is exactly the kind of thing that we ought to encourage to help affordability! Bad measurement leads to bad inferences about rate design.
Second, the arbitrary affordability thresholds that create sensational headlines preempt public debate over what affordability really means. Understanding the burdens and economic tradeoffs that low-income households face is critical to tackling the affordability challenge. But what exactly constitutes affordable water ought to be a matter of community values, not an analyst’s arbitrary judgment.
Finally, the (literally) incredible claims in The Guardian undermine legitimate efforts to assess and address the water affordability challenge. Studies that emphasize impact over accuracy risk achieving neither. Over time, slipshod studies can cause officials and the public to become cynical and dismiss an issue as overblown and ideological (see, for example, political discourse on COVID-19 prevention or climate change).
Water affordability is too important to allow to suffer that fate. Important issues demand responsible research.
*Apparently The Guardian missed this study (2015), this study (2017), this study (2018), this study (2019), and this study (2020). Each was peer-reviewed, nationwide in scope, and included far more utilities than the Colton report.
†I’m guessing that Colton did something like multiply 12,000 gallon price to the ratio of average census tract household size and Circle of Blue’s assumed 4-person household. But that’s pure speculation.