Building Energy Efficiency for Climate Policy and Recovery Stimulus

Last April 17, Mateus Souza wrote this post on uc3nomics Blog, based on research with Peter Christensen, Paul Francisco, Erica Myers, and Hansen Shao.

Improving the energy efficiency of buildings is often viewed as one of the most promising strategies for climate policy. Retrofit and renovation programs have great potential to abate carbon emissions by lowering households’ energy consumption, which also translates into lower energy bills. These programs can also improve air quality within homes (Tonn, Rose, and Hawkins, 2018) and may even help create jobs (ORNL, 2014). Given all these potential benefits, renovation projects are taking a central role in economic stimulus packets for decarbonization, and for recovering from the recent energy and COVID-19 crises. For example, through the EU’s Recovery and Resilience Facility (EC, 2022), Spain intends to “(support) the green transition through investments of over €7.8 billion in the energy efficiency of public and private buildings.” Similarly, the U.S. Inflation Reduction Act (White House, 2023) projects investments of “$9 billion for states and Tribes for consumer home energy rebate programs, enabling communities to make homes more energy efficient, upgrade to electric appliances, and cut energy costs.”

However, economic evaluations have found that the energy savings from these programs often do not meet expectations. In some cases, the average savings may be as low as 30% of the expected savings (Fowlie, Greenstone, and Wolfram, 2018). This substantially lowers the cost-effectiveness of these programs and puts into question their role in climate policy. Mateus Souza and co-authors dig into this issue with two recent articles. The first helps to identify economic and behavioral explanations of why a “performance wedge” exists between the projected versus the realized energy savings of efficiency programs (Christensen et al., 2021). The second asks whether it is possible to use machine learning tools to improve projections of energy savings, with the objective of better targeting funds to homes that are more likely to benefit from the programs (Christensen et al., 2022).

Decomposing the wedge

To better understand the “performance wedge” in energy savings, we studied the Illinois Home Weatherization Assistance Program (IHWAP). The program provides fully subsidized improvements to the heating, ventilation, and air conditioning (HVAC) systems of low-income family homes in the state of Illinois. We analyzed detailed program information, including data on housing structure, demographics, and energy consumption for more than 9,800 homes. Using a novel machine learning-based approach (Souza, 2019), we investigate the importance of three channels that may explain the wedge: 1) systematic bias in engineering measurement and modeling of savings, 2) work quality during installation of the upgrades (workmanship), and 3) the rebound effect (savings may be offset in case households systematically increase their thermostats once the system becomes more energy efficient).

Results suggest that bias in model projections is one of the primary contributors to the wedge. Up to 41% of the wedge can be explained by discrepancies between projected and realized savings in five major retrofit categories: air sealing, furnace replacement, wall insulation, attic insulation, and windows.  Results are particularly striking for wall insulation, as shown in Figure 1. The red squares are point estimates of how the performance wedge increases depending on expenditures in that measure, compared to homes that received zero wall insulation spending. The whiskers represent 95% confidence intervals. The figure shows, for example, that the wedge is approximately 20 percentage points higher for homes with wall insulation expenditures between $1,501 and $1,800.

Figure 1: Increased Performance Wedge by Spending on Wall Insulation

Heterogeneity in workmanship is also an important factor in explaining the wedge.  Results suggest that the wedge could be reduced by up to 43% if all workers performed at top levels. This implies that there exist potential gains from changing worker incentives, investing in contractor training, etc.  On the other hand, only a modest portion of the wedge may be explained by behavioral factors such as the rebound effect. Using data on the realized relationship between outdoor air temperature and energy consumption, we find that households modestly increased their thermostats after weatherization, accounting for only up to 6% of the wedge.

We also analyze the program’s cost-effectiveness by comparing the energy and carbon abatement benefits versus the costs of the retrofits. We find that, on average, each home the program serves is associated with net benefits of -$325. Although average net benefits are close to zero, disaggregated estimates reveal substantial heterogeneity, such that approximately 42% of homes generate positive net benefits, as shown in Figure 2. Therefore, certain types of projects are highly cost-effective, suggesting a potential role for targeting in this context.

Figure 2: Net Present Benefits of Retrofitted Homes

Potential gains from targeting

Within this context of substantial heterogeneity in net benefits, a natural follow-up question is whether it is possible to identify the high-return projects before they are actually implemented. We conducted another analysis with data from the same program but now using information available only before the homes were retrofitted. The idea is to mimic the role of a program implementer who is trying to predict the magnitude of net benefits prior to performing the retrofits. To maximize the total predicted net benefits from the program, the implementer would then choose to treat only homes with positive expected returns. This consists of an ex-ante prediction/targeting exercise, which differs substantially from an ex-post evaluation performed with information available many months after the renovations.

As a first step, we show that it is possible to accurately predict home-specific energy savings from the program by using machine learning techniques. In fact, we find that our predictions are accurate even when using a subset of publicly available variables (such as the size and the age of the home, the number of rooms, and the presence of an attic). We then rank homes from highest to lowest net present benefits and calculate the cumulative monetary benefits from retrofitting homes in that order. These results are presented in Figure 3. We compare our machine learning rankings (in orange) to an engineering ranking (in green) that currently guides funding allocation decisions within the program. These are also compared to a ranking with perfect foresight (in blue). Results show that the machine learning strategy outperforms the engineering model and could drastically improve program cost-effectiveness.  Within this sample, targeting high-return interventions based on machine learning predictions can dramatically increase net benefits from $0.93 to $1.23 per dollar invested.

Figure 3: Potential Gains from Targeting

Conclusions

Thanks to recent advances in information and data technologies, retrofit programs can readily incorporate machine learning-based strategies to help select among candidate projects. Energy efficiency programs are often sponsored by utilities that have recently developed the data infrastructure to store, query, and serve household billing data. Integrating predictions from machine learning models into those infrastructures would be straightforward. Although these models may be computationally demanding, they only need occasional updates. Once the results are obtained, they can be fed into the backend of existing software that already help with funding allocation decisions.

The importance of considering and implementing these types of tools continues to grow as energy efficiency remains central to climate policy discussions. Optimal allocation of these funds may be crucial to achieve ambitious climate goals. Future work within this context has yet to explore, for example, the distributional implications of targeting investments based solely on energy or climate-related benefits. Analyses of the health and potential job creation impacts of these programs also seem mostly missing from the economic literature.

Further Reading:

Peter Christensen, Paul Francisco, Erica Myers, and Mateus Souza (2021). “Decomposing the Wedge between Projected and Realized Returns in Energy Efficiency Programs.” The Review of Economics and Statistics (Forthcoming); https://doi.org/10.1162/rest_a_01087

Peter Christensen, Paul Francisco, Erica Myers, Hansen Shao, and Mateus Souza (2022). “Energy Efficiency Can Deliver for Climate Policy: Evidence from Machine Learning-Based Targeting.” NBER Working Paper 30467; https://www.nber.org/papers/w30467

Mateus Souza (2019). “Predictive Counterfactuals for Treatment Effect Heterogeneity in Event Studies with Staggered Adoption.” SSRN Working Paper 3484635; EEL Discussion Paper 107; https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3484635

About the authors:

Mateus Souza is a Postdoctoral Researcher at EnergyEcoLab, Department of Economics, Universidad Carlos III de Madrid.

https://sites.google.com/view/mateussouza/home

Peter Christensen is an Associate Professor of Economics at the University of Illinois at Urbana-Champaign.

https://www.uiuc-bdeep.org/christensen

Paul Francisco is the Associate Director for Building Science at the ICRT Applied Research Institute, University of Illinois at Urbana-Champaign.

https://appliedresearch.illinois.edu/directory/profile/pwf

Erica Myers is an Associate Professor of Economics at the University of Calgary.

https://sites.google.com/site/ericacatherinemyers/home

Hansen Shao completed his PhD in Economics in 2021 at the University of Illinois at Urbana-Champaign, and is currently an economic consultant based in China.

References:

European Commission (2022). “Recovery and Resilience Facility: The key instrument at the heart of NextGenerationEU to help the EU emerge stronger and more resilient from the current crisis”. Available online: https://commission.europa.eu/business-economy-euro/economic-recovery/recovery-and-resilience-facility_en

Fowlie, Meredith, Michael Greenstone, and Catherine Wolfram (2018). “Do Energy Efficiency Investments Deliver? Evidence from the Weatherization Assistance Program”. The Quarterly Journal of Economics 133, 1597-1644; https://academic.oup.com/qje/article/133/3/1597/4828342

Tonn, B., E. Rose, and B. Hawkins (2018). “Evaluation of the U.S. Department of Energy’s Weatherization Assistance Program: Impact results”. Energy Policy 118, 279-290; https://www.sciencedirect.com/science/article/abs/pii/S0301421518301836

Oak Ridge National Laboratory (2014). “Weatherization Works – Summary of Findings from the Retrospective Evaluation of the U.S. Department of Energy’s Weatherization Assistance Program”. ORNL Technical Report 2014/338; https://nascsp.org/wp-content/uploads/2017/09/ORNL_TM-2014_338.pdf

U.S. White House (2023). “Building a Clean Energy Economy: A guidebook to the Inflation Reduction Act’s investments in clean energy and climate action.” Available online: https://www.whitehouse.gov/wp-content/uploads/2022/12/Inflation-Reduction-Act-Guidebook.pdf