An energy agency’s forecasting flaws

Just as Wall Street parses every word from the Federal Reserve to infer the state of the economy, energy traders scrutinize every release from the Energy Information Administration (EIA) to gauge trends in energy markets. The agency, part of the Department of Energy, aims to provide “independent and impartial energy information to promote sound policymaking.” Toward that aim, EIA issues weekly, monthly and annual reports compiling data on everything from prices to production to stockpiles.

{mosads}With its free and public data and analyses, EIA provides the best available gauge of current and historical conditions in energy markets. However, when it comes to forecasting the future, the EIA’s crystal ball appears to be clouded by persistent biases and errors. Despite the EIA’s special report last week rebutting critics of its renewable energy forecasts, we remain concerned that forecasting flaws may hinder the agency’s ability to reliably inform decision-making by policymakers and market participants.

The EIA issues forecasts in several forms: “Annual Energy Outlooks” projecting U.S. conditions through 2040; “International Energy Outlooks” for global markets; monthly “Short-Term Energy Outlooks” projecting U.S. conditions through next year; and various special forecasts in response to requests from Congress or other emerging policy needs. These forecasts are relied upon by businesses making investment decisions, utility commissions making regulatory decisions and by agencies like the Environmental Protection Agency (EPA) in predicting the impacts of policies pertaining to energy. Even many private-sector energy forecasts and academic analyses are based largely upon data and forecasts from EIA.

As with any effort to forecast the future, perfect foresight cannot be expected. This is especially true for complex energy markets, since human behavior can be more complicated to predict than the weather, and even the weather itself influences energy outcomes. However, just as meteorologists would reexamine a model that produced predictions that were too cold, so too must we be concerned if energy forecasts are persistently skewed in certain directions or are driven by flawed inputs or assumptions.

A recent peer-reviewed study led by the co-author of this column reviewed 630 projections made by the EIA between 2004 and 2014 that could be checked against actual data. The study found that most of EIA’s projections for renewables sharply under-projected generation or capacity, with especially pronounced under-projections of wind and solar in more recent years.

Both that study and the EIA’s special report found that unanticipated pro-renewables state and federal policies played a role. The EIA’s baseline forecasts assume only laws in place at the time of the outlook and thus become skewed when policies are subsequently changed, such as extensions of tax credits for renewables.

While it is beyond the EIA’s scope to predict policy changes, other issues identified by the peer-reviewed study were unforced errors. For one, the EIA assumed that many coal power plants would co-fire with biomass, despite little evidence for this trend in U.S. electricity markets. The study also found that the EIA neglects how volatile natural gas prices and environmental factors may influence investments in new generation capacity.

A tendency to over-predict the costs of wind and solar also played a role in forecast errors. For example, Lawrence Berkeley National Laboratory reported that utilities paid on average 2.5 cents/kilowatt hour (kWh) for wind and 5 cents/kWh for solar in contracts negotiated in 2014. Austin Energy now pays under 4 cents/kWh for solar. Meanwhile, the EIA forecasts that levelized costs of wind and solar will “fall” to 7.4 cents and 12.5 cents/kWh in 2020, respectively. These factor-of-three gaps between future cost forecasts and actual prices paid are too large to be explained by tax credits and run counter to ongoing cost declines for solar and wind reported by other Department of Energy offices.

Fossil fuel forecasts have been problematic as well. While recent plunges in fuel prices may have been challenging to forecast, it is worrisome that 95 percent confidence ranges from EIA forecasts have failed by substantial margins to capture recent oil and gas prices.

The EIA’s own retrospective analyses show its “Annual Energy Outlooks” have repeatedly over-predicted fossil fuel consumption and carbon dioxide emissions. It may be asked whether recent “Short-Term Energy Outlooks” are repeating those mistakes, projecting coal consumption to reverse ongoing trends and rise slightly next year.

While over-predictions of renewables costs and fossil fuel demand can in time be assessed with actual data, we can only speculate how errant forecasts have impacted investments and policy. Have over-predictions of wind and solar costs and coal consumption slowed the pursuit of renewables or contributed to over-investment in coal capacity? Did over-predictions of electricity demand lead to less stringent Clean Power Plan carbon dioxide targets than could have been pursued?

Looking ahead, our country will need the best possible data and forecasts to inform our pursuit of affordable, reliable and environmentally sustainable energy solutions. It is our hope that by recognizing some of the mistakes of past forecasts, future efforts can be informed more reliably.

Cohan is associate professor of civil and environmental engineering at Rice University. Gilbert is cofounder of Spark Library, an energy research platform.