During a public health crisis, getting the research right is paramount
It started with a tweet.
A new peer-reviewed Utilities Policy article on water utility ownership, low-income households, and shutoffs? From a pair of professors at major research universities? This was right up my alley!
The paper’s title—Does public ownership of utilities matter for local government water policies?—is intriguing. Water system ownership, regulatory policy, and especially shutoffs are enormously important and notoriously difficult to analyze due to data difficulties. The tweet and top highlight finding were provocative: “Cities and towns with government-owned utilities shutoff customer drinking water less.” These are headline-grabbing claims, sure to draw the attention of water sector leaders and policymakers looking for ways to tackle an ongoing public health crisis. I dove into the paper immediately, excited about what I might find!
Unfortunately, the study is deeply flawed. Few readers—even within the academy—have the appetite to get into the methodological details necessary to understand what the data really show. My deep dive revealed that the article’s authors came to profoundly incorrect conclusions. Last week I emailed them with my concerns. This post is my attempt to clarify what the article obfuscates and to set the record straight.
From tweet to title to text, Homsy & Warner’s article promises to explore the role that “local government utility ownership play[s] in meeting equity and environmental goals.” The study’s literature review frames its goals as an inquiry on “Publicly owned versus privately owned water supply” (section 2.3), with interest in public/private differences in “equity” (2.1) and “environment: water resource management” (2.2). Specifically, they’re interested in whether publicly owned water systems provide more protections against shutoffs for low-income households and greater water conservation. The implied comparison is with private water utilities.The empirical analysis uses a 2015 ICMA survey sent to municipal chief administrative officers. The sample included all municipalities, towns, and counties with populations over 25,000 and a sample of smaller communities of populations between 2,500 and 25,000. The survey yielded 1,897 responses for a 22% response rate.* Respondents were asked whether their governments own water utilities. If the respondent said yes, the community was coded as having a “publicly owned utility” or “Government-owned utility”; if the respondent said no, then it was coded as having a “not publicly owned utility.”
The survey also asked questions about policy, including whether the city or county had taken action to "protect low-income households from water service shutoff.” The answer to this question is Homsy & Warner’s only measure of shutoff protection. They don't actually count shutoffs. About 8% responded that their governments provided shutoff protections.
These survey data were analyzed with a series of logistic regressions to predict the likelihood that a community has water shutoff protections as a function of public ownership and a series of other variables. Ownership emerges as a very strong predictor, leading Homsy & Warner to declare that: “if a utility is government-owned, the municipality is about two times more likely to have water shutoff protection policies.”
They conclude that “ownership matters, as communities with publicly owned utilities appear more inclined to protect residents from water service shutoffs.” That would be a big, important finding—if the data supported it. The trouble is that the data don’t show anything of the sort.
Where it all goes wrong: misleading measurement
The first hint that something was amiss was that just 55% of Homsey & Warner’s sample were publicly owned utilities, with the authors implying that the other 45% were served by private utilities. That struck me as extremely low, since about 85% of Americans get their drinking water from a local government, with about 15% served by private, investor-owned firms. How could the ICMA sample be so grossly skewed in favor of non-government utilities?**The answer is in the way that Homsy & Warner code “public ownership.” Recall that the ICMA survey was sent only to counties and municipalities and that respondents who reported that their governments own water systems were coded as “publicly owned.” What’s missing is special districts.
Special districts are local governments with narrow functions and limited powers; there are tens of thousands of such districts across the U.S.. The city and county officials responding to this survey who get their water from special districts would answer "no," and then Homsy & Warner would code the community's water system as "Not Publicly Owned."
Consider, for example, Central Arkansas Water. Headquartered in Little Rock, Central Arkansas Water serves a population of more than 450,000, including the cities of Little Rock, North Little Rock, Alexander, Sherwood, and Wrightsville, along with parts of unincorporated Pulaski County. Survey respondents from Pulaski County and all five of these cities would report that their governments did not own a water system. Homsy & Warner would code their water systems as "not publicly owned," even though a government--Central Arkansas Water--supplies their water.
Around 30% of the local government water systems in the United States are owned and operated by special districts. These special districts are definitely local governments. Without accounting for special district utilities, we cannot infer anything about public/private water system ownership from these data.
Sins of inference
But it gets worse. The study’s first highlighted claim is that “cities and towns with government-owned utilities shutoff customer drinking water less.”† Thing is, this study doesn’t measure shutoffs. It measures whether a municipal or county government has a policy to protect low-income households from shutoffs. Reliable shutoff data are notoriously hard to find, which is why serious research on the subject is rare. A reader who doesn’t get into the empirical weeds wouldn’t notice that data don’t support this claim.
Apart from problems with coding ownership and counting shutoffs is the question of what this all means for water governance. What should we infer from the fact that a municipality that owns a water system is more likely to have water shutoff protections than a municipality that does not own a water system? Homsy & Warner conclude that “communities with publicly owned utilities appear more inclined to protect residents from water service shutoffs.” That’s a bit like finding that people who own cars are more likely to have jumper cables than people who don’t own cars. Would we then infer that car owners care more about their families' transportation than transit riders?
Why it matters
Policymakers, advocates, regulators, and utility managers are looking for answers to a historically tough challenge in the COVID-19 pandemic. Researchers everywhere are working hard to find ways to help protect public health.
We certainly need to understand water shutoffs and how to prevent them. But the stakes are too high for policy researchers to play fast and loose with data and inferences. This isn’t an abstract, theoretical squabble over the literary interpretation of Hamlet. Real policies to protect real people in a moment of real crisis are on the line, and our communities need valid findings. The urgency of pandemic only heightens the need for rigorous, responsible policy research.
*It does not appear that Homsy & Warner adjusted their estimates to account for sample stratification or non-response bias.
**With just a 22% response rate, some of the issue is probably nonresponse bias. But it’s unlikely that nonresponse accounts for a 30% gap in share of publicly owned utilities.
†In this sentence and throughout the article, Homsy & Warner are vague about what publicly owned utilities are being compared with.
for a federal low-income water bill assistance program
The ink is barely dry on the $2 trillion coronavirus response law, but there are rumblings that a another relief bill will be at the top of the agenda when Congress reconvenes later this month. The latest noises out of Speaker Pelosi’s office indicate that the next bill will focus on immediate relief for families, small businesses, health systems, and local governments.
When it comes to household water affordability relief, the perennial favorite proposal is a federal means-tested assistance program for low-income families modeled after the Low Income Home Energy Assistance Program (LIHEAP). A $1.5 billion LIHEAP-style relief program for water was part of the House proposal for the last COVID-19 relief bill, but it was cut from the final bill and never enacted. The proposal is likely to be resurrected in the next bill.
Over the past week I’ve had several conversations with utility executives, policy experts, and government leaders about how Congress might best provide water relief in this ongoing and rapidly-moving pandemic. This post summarizes thoughts that have emerged from those conversations, and explain why I’m sympathetic but lukewarm on the idea of a federal LIHEAP-style program for water in this moment of crisis.
Redistributive programs come in two basic flavors: means-tested and entitlements.* Means-tested programs provide benefits to individuals and households who demonstrate need and whose resources (income, assets) fall below specific thresholds. People must apply for these benefits, and government bureaucrats evaluate applications to see that they meet program rules. Procedures for auditing and appeals accompany these processes. Those who receive benefits must reapply periodically in order to maintain eligibility. Benefits decline or disappear as incomes grow. Familiar means-tested assistance programs include TANF (“welfare”), SNAP (formerly Food Stamps), Section 8 housing, and LIHEAP.
Entitlement programs provide public benefits to qualifying individuals and households regardless of their need or resources—rich, middle-class, and poor households all may receive assistance. People are not required to demonstrate need or report income and assets to government agencies to get the benefits. K-12 education is a great example at the state/local level. School districts don’t require families to demonstrate financial need before enrolling their children, and millions of wealthy and middle-class kids attend school at the public expense across the country. Medicare and Social Security pensions are the two biggest federal examples: rich or poor, the government provides these programs whether or not their recipients “need” them.
It should come as little surprise that means-tested programs often carry a social stigma and entitlement programs are perennially popular.
LIHEAP for water?
Many local utilities provide some kind of means-tested assistance. With 50,000 community water systems operating across the country, these programs vary widely in design and administration.** No statewide water assistance programs exist, although California is building one. There is no federal low-income household assistance program for water or sewer bills. The closest analog is LIHEAP.
A LIHEAP-style water program is a fine idea in theory: it targets the needy population and helps pay for an essential but often expensive service. The program is familiar to the community advocacy crowd, and a network of state and local social service organizations already exists to help administer the program. But there are at least four big reasons to worry about federal LIHEAP-for-water as a cornerstone of affordability policy.
First, the extreme fragmentation of the water sector makes managing water bill assistance administratively costly in ways that it isn’t for energy. LIHEAP coordinates with the 3,200 electrical utilities and 1,400 gas utilities across the United States. There are 50,000 community water systems, and roughly 40,000 of those are very small, serving fewer than 3,300 people and employing just a handful of staff. Affordability is often most dire in these very small utilities in rural communities. Billing systems in these lightly-staffed utilities are often primitive and poorly-suited to coordinate with social service agencies. Making a LIHEAP-type program work for water will take months and significant investments in administrative systems and organizational capacity on the utility-side.
Second, like all means-tested programs, LIHEAP puts an administrative burden on the very people that it seeks to help. Learning about the program, applying, demonstrating eligibility, ensuring receipt, appealing decisions, and reapplying are time-consuming and sometimes humiliating processes. These costs may be especially significant for people with low literacy or limited English proficiency. Potentially eligible people may forego benefits if the application process is too burdensome, if they perceive a social stigma associated with public assistance, or if they do not trust government.
Third, forty years of experience with LIHEAP demonstrates the limits of the program. Historically, LIHEAP has reached an average of just 16% of eligible households. That’s not 16% of all households, that’s 16% of the population that qualifies for the program. The all-time high-water mark for LIHEAP outreach came during the 2009-2010 recession response, when the program helped 22% of eligible households. In other words, at its very best, LIHEAP failed to reach 78% of the people who needed it.
Finally, it is unclear that a LIHEAP-style program would address the immediate need to stop water shutoffs and reconnect every household during a public health crisis. Even assuming the most optimistic administrative scenario, LIHEAP-style assistance will take several weeks or months to work its way from the U.S. Treasury to state governments to social service organizations and finally into water billing systems. After all that, the program’s impact on shutoffs and reconnections will still depend on local practices.
I don’t hate the idea of federal low-income assistance for water. A LIHEAP-style program would surely help many people and could be an important part of a systemic strategy to improve the American water sector. But such a program would do little to alleviate the immediate COVID-19 crisis and could blunt political momentum for more comprehensive and meaningful reform.
Last week I blogged about how the federal government could move swiftly to help keep water and sewer services flowing everywhere during the COVID-19 crisis. My idea is a one-time conditional, formulaic grant program to support water utilities that agree to end residential shutoffs, restore service universally, forgive outstanding penalties, and structure prices to meet affordability standards. It’s an unorthodox and admittedly blunt instrument, designed to tackle a short-term crisis as quickly as possible, with the lowest management costs and least administrative burden on families. Sustainable solutions for the water sector will require more fundamental reforms to the way that we govern, finance, and manage these critical systems after the pandemic has passed.
*Tax expenditures are also redistributive, but I’m trying to keep this post short so I’m leaving them aside.
**To my knowledge, there has never been a systematic study of water assistance program effectiveness over a larger number of utilities.
How the federal government might end shutoffs & keep water flowing during the COVID-19 crisis
The COVID-19 crisis has escalated America’s water and sewer affordability challenge into a full-blown health emergency. Many low-income households struggle to pay for these essential services in the best of times, and the specter of shutoffs for non-payment now threatens to worsen the pandemic. It’s hard to wash hands, cook at home, and maintain adequate sanitation without water service.
In response to the fast-moving crisis, scores of utilities are suspending shutoffs and restoring service for the duration of the pandemic. That is a prudent move in this emergency, but suspending shutoffs and restoring service carries significant financial risks for utilities and does not fundamentally solve the affordability problem, even in the short-run. An end to shutoffs does not mean an end to high prices, late fees, or penalties. When the crisis passes, many customers will still have outstanding balances running into the thousands of dollars and once again face the threat of shut-offs. Meanwhile, in plenty of places shutoffs continue even as COVID-19 rages.
Federal water bill relief?
Last week Congress passed a monumental $2 trillion economic rescue package in response to the COVID-19 crisis sweeping the country. During the helter-skelter Capitol Hill negotiations over the COVID-19 bill, House members proposed $1.5 billion in water assistance relief for low-income households. Modeled after LIHEAP, the federal low-income energy assistance program, the proposal would have provided financial assistance to income-qualified households to help pay for water bills through existing LIHEAP administrative processes. The proposal didn’t make it into the bill that finally reached President Trump’s desk.
Although the water bill assistance would surely have helped many, it would likely have made little difference in the big picture. For starters, while $1.5 billion is a lot of money, means-tested assistance programs are costly to administer and burdensome for customers who need help. This sort of relief can help, but will take time to work its way through administrative processes and into consumers’ accounts to prevent shutoffs. Even at their best, means-tested programs help a small fraction of the eligible population—historically LIHEAP has reached only about 16% of those eligible for assistance. Complicating matters is the extreme fragmentation of the U.S. water sector, with 50,000 mostly small water systems operating across the country. Some of the poorest Americans live in small communities where utilities’ and social service organizations have limited capacity to administer assistance. The need for immediate relief in the face of a pandemic demands faster, farther-reaching action.
Bigger, bolder, faster action*
So what might work better? I’ve long argued that pricing, not assistance programs, is the best way to tackle water affordability. With the pandemic upon us and a massive, emergency need for universal in-home water and sanitation, it’s worth considering a similarly massive, emergency financial response. Here’s an outline of a scheme that could quickly end shutoffs and maximize short-term affordability relief with the lowest management cost to utilities and zero administrative burdens on customers.
The federal government should provide formulaic, conditional grants directly to water utilities. Grants would be awarded as a percentage of each utility’s budgeted 2020 annual rate revenue, with the percentage equal to the community’s poverty rate. For example, Seattle Public Utilities’ 2020 budget calls for $205 million in water revenue and about 12% of its population lives in poverty, so its grant would be $24.6 million. Detroit’s budgeted water rate revenue for 2019-2020 is $131 million and its poverty rate is 33%, so its grant would be $43.2 million.
In exchange for this cash injection, utilities would have to meet simple conditions on pricing and customer administration. Specifically, for the duration of the national COVID-19 pandemic, utilities would:
- End residential shutoffs for non-payment;
- Restore service to all occupied residences currently shut off;
- End residential foreclosures and financial penalties for non-payment or service restoration;
- Forgive all outstanding penalties, fees, and interest on residential water accounts;
- Structure prices so that 6,000 gallons of monthly residential water and sewer service costs less than $58 (eight hours of labor at federal minimum wage).
All community water systems that operate on a fee-for-service basis would qualify, including municipal, tribal, special district, and investor-owned systems. Utilities could use the money to offset revenue losses due to COVID-19 crisis, fund assistance programs, or maintain and improve capital.
Federal funds would be channeled from EPA through existing state Drinking Water Revolving Funds directly into utility coffers, requiring very little additional administrative capacity. There would be no administrative burden at all on customers. Administration for very small systems could be managed through state or county governments.
With annual water utility revenue totaling something like $70 billion and a national poverty rate of 11.8%, the program would end up costing around $8.5 billion dollars. For another $10 billion we could extend the program to cover sewer revenue, too. Until last week, those would seem like absurdly large sums, but they’re rounding errors in the $2 trillion-dollar package that Congress just approved.
Emergency & aftermath
To be clear, this isn’t a carefully considered, meticulously modeled plan—it’s an idea meant to get water flowing immediately in response to an urgent need. These are big, blunt policy instruments, but the proposal outlined here could be introduced on Monday, signed into law by Wednesday, and water service restored in communities across the country by Friday. In a pandemic every moment matters.
Lasting, sustainable solutions for the water sector will require more fundamental reforms to the way that we govern, finance, and manage these critical systems. I hope that once the COVID-19 storm fades, a renewed commitment to improving the American water sector is one of its silver linings.
*Thanks to Wendi Wilkes for prompting and helping me think this through via Twitter. She deserves a share of the credit if you like this idea, but no blame if you hate it.