From Water

Misoverestimickated

About that water affordability study
in The Guardian...

Water is a ZILLION PERCENT unaffordable! Also, aliens.

The Guardian recently published a big story on water utility affordability in the United States. The headline was shocking: “Millions of Americans Can't Afford Water, as Bills Rise 80% in a Decade,” and “Analysis of U.S. cities shows emergency on affordability of running water amid COVID-19 pandemic.” The story was based on a study that The Guardian commissioned from Boston-area attorney Roger Colton. Colton’s report—which The Guardian called “the first nationwide research of its kind”* —was the basis for the story’s claims.

The Colton report sets out to do three things:

1) “examine whether the affordability of water is common in the U.S. today”;

2) examine whether “water affordability has changed in recent years;” and

3) “examine the extent to which… reasonable projections of water rate increases will affect water affordability in the near future.”

Put simply, the report aims to measure the level of water affordability in the U.S., past affordability trends, and projected future affordability.

Unfortunately, the Colton report is ​deeply flawed. Not just flawed in some narrow, technical sense; it’s flawed in ways that grossly misrepresent the state of water affordability in the U.S. and point to the wrong approaches to addressing the issue.

I really didn’t want to write this post. It’s heartening when mainstream journalists pay attention to the oft-neglect water sector, especially on matters of affordability, so I hate to be negative about it. ​I'm also painfully aware of Brandolini’s Law​, and ​any time spent counteracting bad research is time not spent on my own research.

​But over the past month I’ve fielded several inquiries about The Guardian article asking whether the Colton report is valid. The short answer is no.

​The long answer requires wading into the tall methodological grass. So grab your weed eater and follow me.

The study

Although The Guardian article’s claims are national in scope, the underlying analysis is limited to twelve cities. The dozen were not selected at random, nor ​are they the twelve largest. Rather, they were selected by Guardian staff, supposedly “to provide a diversity of geographic regions… population sizes… and poverty.” All inferences from the Colton report come from this hand-picked, non-representative sample of systems that collectively serve a little more than 2% of the U.S. population.

But the Colton study’s deepest problems are about measurement, not sampling. Answering the questions that ​the study raises requires four things:

a) prices for water service in the United States

b) household resources available to pay for water

c) projected future prices and resources

d) defining affordability

You can think of those as the numerator, the denominator, the forecast, and the judgment. All of them flawed in ways that compound and confound.

The numerator

A study of water and sewer affordability should start with the price of water and sewer service, right? Strangely, the Colton study didn’t collect any rates data or calculate prices. Instead, Colton got price data from Circle of Blue’s annual reports on water prices in 30 U.S. cities. Circle of Blue calculates prices at three benchmark volumes, representing demand for a family of four at 50 gallons per capita per day (gpcd), 100 gpcd, and 150 gpcd. That works out to roughly 6,000, 12,000, and 18,000 gallons per month. Nationally, residential indoor water demand averages around 50-60 gpcd and has been falling steadily for the past twenty years. This indoor demand represents basic needs for drinking, cooking, cleaning, and sanitation. That’s the main public policy concern for affordability—from a public health standpoint we don’t really care about the affordability of lawn watering or car washing.

But the Colton report used Circle of Blue’s 12,000 gallon monthly price as the basis for its numerator. So ​The Guardian’s startling graphs ​reflect the prices for for very high volumes of water.

Next Colton used the 12,000 gallon monthly price to “derive estimated bills given average household sizes for each Census Tract,” according to a footnote on page 6. How Colton derived the prices is a complete mystery; he never explains the procedure, and the footnote is the only mention of how he calculated prices.​† A simple table of monthly prices is ​strangely absent from the 88-page report.

The denominator

Claims about the affordability of anything implies some relationship between costs and resources. Colton’s sole measure of resource affordability is the Federal Poverty Level (FPL). The FPL’s quirky origin story is fascinating—it’s based on 1962 food costs—and the problems with FPL are well-documented in voluminous research. Despite its quirks and flaws, FPL ​remains a touchstone in policy discussions nonetheless, mainly because it’s familiar and easily available from the US Census Bureau.

The main problem with FPL for purposes of assessing water and sewer affordability is that it’s insensitive to local costs of living. FPLS also a one-size-fits-all number national number (hence, “federal”): whether you live in pricey San Francisco or cheap Buffalo, the denominator is the same. Home energy, health care, taxes, and especially housing varies wildly across the country, but FPL is insensitive to those costs. Efforts like United Way’s ALICE are aimed at providing a more realistic assessment of cost of living by accounting for that variation.

The projection

Colton’s study projected affordability through 2030, which involved ​forecasting both numerators (price) and denominators (FPL). Once again, the report is vague about how it made those projections. A footnote on p.33 indicates that Colton used inflation rates from a 2017 U.S. Department of Energy’s report, which calculated 2008-2016 increases in water/sewer prices for selected U.S. cities based on average volumetric rate. It’s not clear whether Colton used city-specific projections for his twelve cities, or a 4.71% annual inflation rate, which ​​a footnote ​in a DOE study ​says was the “national average annual price increase for water and sewerage maintenance” from 1996-2016. It’s impossible to know exactly how Colton projected prices because price projections never appear in the report. To project the denominator, Colton uses 2010-2018 increase in FPL and assumes that the same growth will continue through 2030.

​It appears likely that Colton used straightline projection to forecast an already-inflated numerator using with either an eight- or twenty-year retrospective rate of change (it’s not clear which), either nationally or regionally (again not clear). He projected the FPL denominator using a different eight years of national FPL escalation.

The judgment

All of this culminates in a declarations of how many people or what percentage of the population suffers from “unaffordable” water. Those conclusions fuel The Guardian headlines, but require a binary definition of what is affordable. So how did the Colton report define affordability?

“In assessing whether a water bill was ‘affordable,’ the base level of affordability was set at 4% of income,” says the report (p.8). That 4% threshold is the main definition of affordability in both the Colton report and The Guardian stories. The Colton report goes on to set “Affordable Burdens” for various income ranges.

Where did these affordability burden thresholds come from?

Colton made them up.

No poll, no Blue-Ribbon Committee, no philosophical inquiry on the meaning of affordability, no analysis of economic tradeoffs. They’re just arbitrary numbers. Every claim about the nation’s water affordability crisis and most of the scary graphs in The Guardian articles boil down to these thresholds. They’re based on nothing.

The stakes

Does any of this matter? Shouldn’t we just be happy that mainstream publications like The Guardian and Consumer Reports are paying attention to water affordability? Who cares if the methodological details are a bit dodgy, if the overall point—that water prices are rising faster than poor folks’ incomes—is generally fair?

There are at least three big reasons we should care about assessments that are so egregiously inaccurate. First, this study grossly misstates the scope and nature of the problem. It’s good to draw attention, but bad measurement can lead to the wrong inferences about what’s wrong and how to fix it.

The Colton report’s treatment of Austin, Texas is a great example of what’s so pernicious about poor measurement. The Guardian screams that water bills in the Texas capital increased 154% from 2010-2018, and will be unaffordable for 26% of all Austin residents by 2030. The naïve reader could be forgiven for thinking that Austin Water executives are mustache-twirling villains trying to squeeze money out of poor folks. But remember that Colton’s calculations are based 100 gpcd of demand—far higher than basic indoor needs. Austin Water employs a progressive rate structure that’s actually designed to protect affordability for basic needs and curb inefficiently high water use--and the big price increases start at 11,000 gallons monthly. Here’s the Circle of Blue plot of Austin’s water rates from 2010-2018:

At the more reasonable 50 gpcd, Austin’s monthly water prices increased an average of $2.28 annually from 2010-2018—still an increase, to be sure, but not quite the rolling disaster The Guardian describes. Meanwhile, Colton’s 100 gpcd assumption makes Philadelphia’s regressive, declining block rates look relatively affordable, even as the City of Brotherly Love sticks low-volume customers with higher prices than Austin's. Austin’s  progressive pricing is exactly the kind of thing that we ought to encourage to help affordability! Bad measurement leads to bad inferences about rate design.

Second, the arbitrary affordability thresholds that ​create sensational headlines ​preempt public debate over what affordability really means. Understanding the burdens and economic tradeoffs that low-income households face is critical to tackling the affordability challenge. But what exactly constitutes affordable water ought to be a matter of community values, not an analyst’s arbitrary judgment.

Finally, the ​(literally) incredible claims in The Guardian undermine legitimate efforts to assess and address the water affordability challenge. Studies that emphasize impact over accuracy risk achieving neither. Over time, ​slipshod studies can cause officials and the public to become cynical and dismiss an issue as overblown and ideological (see, for example, political discourse on COVID-19 prevention or climate change).

Water affordability is too important to allow to suffer that fate. Important issues demand responsible research.



*Apparently The Guardian missed this study (2015), this study (2017), this study (2018), this study (2019), and this study (2020). Each was peer-reviewed, nationwide in scope, and included far more utilities than the Colton report.

I’m guessing that Colton did something like multiply 12,000 gallon price to the ratio of average census tract household size and Circle of Blue’s assumed 4-person household. But that’s pure speculation.

Graphic Realism

Organization of U.S. drinking water utilities in a few simple figures

Here are some graphs that convey ​a few key things about the organization of drinking water utilities in the United States​.*

​Source: EPA SDWIS

​There's a lot of important information ​in those graphs, but these are the most important for policymaking purposes:

  1. Fragmentation. There are nearly 50,000 community water systems in the United States, an order of magnitude more than electrical and gas utilities combined.
  2. Ownership & governance. The overwhelming majority of Americans (84%) get their drinking water service from local government utilities, rather than investor-owned utilities. This proportion is opposite from the energy sector, where investor-owned firms ​hold the lion's share of the market.
  3. Size. The distribution of systems is highly skewed in size: over half of American community water systems are very small, serving populations of less than 500​; the largest 434 systems serve nearly half of the U.S. population.

These three realities inform virtually every aspect of water system management, operations, finance, and regulation. Any successful effort to improve or reform American drinking water utilities must account for the political and administrative challenges that these realities present.

Organizations are human creations, so we can change them if we want to. But we can’t ignore them.



*Feel free to copy and use; please link to this ​page. 

Clean water regulation in Indian Country

Sovereignty isn’t what’s on paper, it’s what flows through taps and rivers

Environmental sovereignty (Photo: nativenewsonline.net)

America is slowly awakening to the dire state of tribal water and sewer systems. Access to drinking water and sanitation services are severely limited on many reservations, and where such systems exist, many are in poor shape. A couple years ago ​the first systematic study of Safe Drinking Water Act (SDWA) and Clean Water Act (CWA) ​implementation for tribal facilities yielded alarming results: tribal systems violated the SDWA 57% more and the CWA 23% more than similar non-tribal facilities. The disparities extended to enforcement, too: formal SDWA enforcement was 12% lower and CWA inspections 44% less frequent for tribal facilities. Evidence of systemic environmental injustice is seldom so glaring.

But there is hope. A new study offers promising evidence for a way to tackle the daunting challenge of tribal water systems. This time instead of comparing tribal and non-tribal systems, ​Mellie Haider and I looked at differences across tribal facilities to see whether regulatory institutions might hold the key to better environmental management in Indian Country. To understand why, we have to start with the foundations of federal environmental regulation and the peculiar legal status of Indian nations.

Environmental federalism & tribal governance

The landmark laws of the 1970s that form the core of American environmental protection (e.g., the Clean Air Act, Resource Conservation & Recovery Act, SDWA, CWA) were built with a system of federal-state cooperative implementation. Under these laws, the U.S. Environmental Protection Agency (EPA) sets rules, and states are responsible for implementation and enforcement of those rules for the facilities operating in their jurisdictions.

Thing is, tribes are sovereign nations under the U.S. Constitution, and so they—and, by extension, their water/sewer systems—are not subject to state laws. Oddly, the major environmental laws of the 1970s ​made no explicit provision for regulation of tribal facilities. As a result, tribal water and sewer systems operated in a regulatory vacuum well into the 1980s.

Introducing tribal primacy

Beginning in the Reagan Administration, a series of amendments and executive orders extended federal environmental laws to tribal lands and gave EPA direct implementation authority over them. Some tribal officials successfully lobbied Congress to treat tribes as states for regulatory purposes. With these new rules, tribes may apply to take primary implementation responsibility, or “primacy,” under federal environmental laws. Tribes applying for primacy authority must demonstrate to EPA that they have the administrative capacity to handle regulatory enforcement.

More scenes like this one, please.

​What difference would implementation primacy make to tribal environmental regulation?

On one hand, tribes might engage in a “race to the bottom,” loosening or neglecting environmental rules in order to avoid regulatory costs and improve ​economic output. But a race‐to‐the‐bottom logic makes little sense for American Indian tribes with respect to environmental regulation. Already occupying the proverbial “bottom,” tribes have little reason to shirk regulatory compliance in a race there.

On the other hand, tribal primacy might lead to more rigorous enforcement, as tribes seek to improve health while maintaining their traditions and cultur​es. Federal regulators have few political incentives for devoting scarce resources to enforcement on tribal land, especially when tribes may lack the political strength to demand strict enforcement. At the same time, many tribal governments serve sparsely populated communities under poor economic conditions, leaving tribes with limited access to the human and financial capital necessary to maintain compliance. Regulatory neglect might be the unfortunate (though understandable) result. Tribes with primacy have more control over their own environmental fates. Moreover, primacy can give tribes an important lever in their environmental conflicts with neighboring firms and jurisdictions.*

What difference does primacy make?

To understand the impact of​ implementation primacy on tribal ​clean water enforcement, we analyzed CWA ​records for ​474 tribal wastewater treatment plants in the United States from 2016-2019. About 15% of these facilities operate under tribal regulatory primacy; the rest are regulated directly by the EPA.** After adjusting for facility size, we found that facilities operated by tribes with primacy were inspected more than twice as often as those regulated by the EPA.†

The enforcement gap between tribal and EPA enforcement ​is greatest for smaller facilities and declines as facility size grows. Over our three-year period of analysis, a very small facility (design capacity 5,000 gallons per day) received an average of 2.75 more inspections under tribal primacy than under EPA oversight. At a moderately large facility (2.5 million GPD), the difference fell to just 0.24, statistically indistinguishable from zero.

The fact that the biggest differences are in the smallest systems underscores the impact of tribal primacy as an administrative phenomenon: it stands to reason that EPA officials spend their limited resources on larger tribal facilities. But in the water sector, the greatest environmental​ injustices are often in the smallest, most isolated communities. ​Our evidence shows that tribal primacy has its greatest impact in those small, isolated communities that are otherwise easily neglected.

More of these people, please.

Effective sovereignty

Implementation authority over environmental regulation gives tribal governments effective sovereignty. Sovereignty turns from mere legal assertion to real, practical impact when tribal officials have greater control over their own destinies. Along with money for pipes and plants, efforts to improve tribal water systems must build human capital and organizational capacity to operate and regulate those facilities. Recognizing this reality, the EPA and the Indian Health Service, along with Indian organizations like Native American Water Association and Intertribal Council of Arizona, run programs aimed at building tribal capacity. In the long run, empowering and building ​tribal governance capacity offers perhaps the most promising avenue for improving the environment in Indian Country.



*In fact, we found that tribes with a history of frequent federal litigation were more likely to seek primacy. A history of litigation indicates tribal independence, nationalism, and other political factors related to assertions of sovereignty.

**At the time of our study, only one tribe (the Navajo Nation) held SDWA primacy, so we couldn’t analyze variation in drinking water regulation.

†Our analysis also adjusted for differences in the characteristics of tribes with and without primacy.