Evaluating Projects and Policies

Learning Objectives

In this module, the following topics are covered: 1) the challenges of policy evaluation when costs and benefits accrue over time, 2) features of cost-benefit analysis, and 3) criteria for policy evaluation.

 

After reading this module, students should be able to

  • know five important features of how economists think about costs
  • understand why discounting is both important and controversial, and be able to calculate the net present value of a project or policy
  • know what cost-benefit analysis is, and be aware of some of its limitations
  • think about four criteria for evaluating a project that are not captured in a basic cost-benefit analysis

Introduction

Environmental valuation methods help analysts to evaluate the benefits society would gain from policies or cleanup and restoration projects that improve environmental quality or better steward our natural resources. Another set of tools can yield information about the costs of such actions (a brief description is below). But even if we have plausible estimates of the costs and benefits of something, more work needs to be done to put all that information together and make some rational choices about public policy and investments. This module discusses the challenges of policy evaluation when costs and benefits accrue over time, outlines the main features of cost-benefit analysis, and presents several other criteria for policy evaluation.

Net Present Value, Discounting, and Cost-benefit Analysis

Cost estimation has not generated the same amount of scholarly research as benefit valuation because the process of estimating the costs of environmental improvement is usually more straightforward than the process of estimating the benefits. Economists do think differently about costs than engineers or other physical scientists, and several key insights about the economics of cost evaluation are important for policy analysis. Viewed through an inverse lens, all these ideas are important for benefit estimation as well.

Opportunity Cost

Not all costs involve actual outlays of money. An opportunity cost is the foregone benefit of something that we choose (or are forced) not to do. The opportunity cost of a year of graduate school is the money you could have made if you had instead gotten a full-time job right after college. Endangered species protection has many opportunity costs: timber in old-growth forests can’t be cut and sold; critical habitat in urban areas can’t be developed into housing and sold to people who want to live in the area. Opportunity costs do not appear on firms’ or governments’ accounting sheets and are thus often overlooked in estimates of the costs of a policy. Studies of U.S. expenditures on endangered species’ recoveries have used only information about costs like direct government expenditures because opportunity costs are so challenging to measure (e.g. Dawson and Shogren, 2001).

Redwoods in Muir Woods

A Redwood Forest in California Forests can’t both be cut down and preserved for habitat. The dollar cost of lumber is straightforward to quantify, but it is more difficult to quantify the value of ecosystems. Cutting down the forest therefore has an opportunity cost that is hard to measure, and this can bias people and governments towards resource extraction. Source: Photo by Michael Barera

Transfers Are Not Costs

Cost totals should only include real changes in behavior or resource use, and not transfers of money from one party to another. For example, imagine a program in which a wastewater treatment plant can pay a farmer for the cost of taking land out of production and installing a wetland on the land that will soak up nutrients that would otherwise flow into a local river. The cost of those nutrient reductions is the cost of installing the wetland and the opportunity cost of the foregone farming activity. If payments for multiple services are permitted, the farmer might also be able to get paid by a conservation group for the wildlife benefit associated with the new wetland. However, that additional payment to the farmer is a pure transfer. The social cost of the wetland has not gone up just because the farmer was paid more for it.

Use the Correct Counterfactual

Many cursory analyses of the costs of a policy find the difference between the cost of something before and after the policy was put in place and claim that any increase was caused by the policy. For example, the U.S. government put temporary restrictions on offshore oil drilling after the Deepwater Horizon explosion and oil spill to consider new environmental regulations on such drilling. After those restrictions were put in place, the price of crude oil in the U.S. went up. A sloppy analysis would attribute all the costs of that price increase to the drilling restrictions. However, during the same period of 2010, the U.S. economy was beginning to pull out of a very deep recession; this caused increased manufacturing activity and consumer driving, and thus an increased call for fossil-fuel energy. Therefore, some of the increase in oil prices might have been driven by the increased demand for oil. A careful analysis would compare the price of oil with the restrictions in place to what the price of oil would have been during the same time period if the restrictions had not been implemented—that hypothetical scenario is the true counterfactual.

Additionality

A careful analysis of the costs of a program includes only costs that are additional, that is, new additions to costs that would have existed even in the absence of the program. For example, current regulations require developers to use temporary controls while constructing a new building to prevent large amounts of sediment from being washed into local rivers and lakes. Suppose EPA wants to estimate the costs of a new regulation that further requires new development to be designed such that stormwater doesn’t run off the site after the building is finished. A proper analysis would not include the costs of the temporary stormwater controls in the estimate of the cost of the new regulation, because those temporary controls would be required even in the absence of the new regulation (Braden and Ando, 2011). The concept of additionality has been made famous in the context of benefit estimation by a debate over whether programs that pay landowners not to deforest their lands have benefits that are additional; some of those lands might not have been deforested even without the payments, or the landowners may receive conservation payments from multiple sources for the same activity.

Control for Associated Market Changes

A careful cost analysis must pay attention to market changes associated with cost increases. To illustrate, suppose the government is thinking of passing a ban on agricultural use of methyl bromide. This ozone-depleting chemical is widely used as an agricultural fumigant, and is particularly important in strawberry production and shipping. A ban on methyl bromide might, therefore, increase the marginal cost of producing strawberries. A simple approach to estimating the cost of the proposed methyl bromide ban would be to find out how many strawberries were sold before the ban and calculate the increase in the total cost of producing that many strawberries. However, the increase in production costs will drive up the price of strawberries and lower the number of strawberries sold in the marketplace. There is a cost to society with two parts: (a) deadweight loss associated with the net benefits of the strawberries not sold, and (b) the increased cost of producing the strawberries that still are sold. That total social cost is lower, however, than the estimate yielded by the simple approach outlined above because the simple approach includes increased production costs for strawberries that are not sold. An accurate cost estimate must take into account market changes.

The concept of net benefits was introduced above; in the context of policy or project evaluation, net benefits are, quite simply, the difference between the benefits and the costs of a policy in a given year. However, environmental policies typically have benefits and costs that play out over a long period of time, and those flows are often not the same in every year. For example, wetland restoration in agricultural areas has a large fixed cost at the beginning of the project when the wetland is constructed and planted. Every year after that there is an opportunity cost associated with foregone farm income from the land in the wetland, but that annual cost is probably lower than the fixed construction cost. The wetland will yield benefits to society by preventing the flow of some nitrogen and phosphorus into nearby streams and by providing habitat for waterfowl and other animals. However, the wildlife benefits will be low in the early years, increasing over time as the restored wetland vegetation grows and matures. It is not too difficult to calculate the net benefits of the restoration project in each year, but a different methodology is needed to evaluate the net benefits of the project over its lifetime.

Some analysts simply add up all the costs and benefits for the years that they accrue. However, that approach assumes implicitly that we are indifferent between costs and benefits we experience now and those we experience in the future. That assumption is invalid for two reasons. First, empirical evidence has shown that humans are impatient and prefer benefits today over benefits tomorrow. One need only ask a child whether they want to eat a candy bar today or next week in order to see that behavior at work. Second, the world is full of investment opportunities (both financial and physical). Money today is worth more than money tomorrow because we could invest the money today and earn a rate of return. Thus, if there is a cost to environmental cleanup, we would rather pay those costs in the future than pay them now.

Economists have developed a tool for comparing net benefits at different points in time called discounting. Discounting converts a quantity of money received at some point in the future into a quantity that can be directly compared to money received today, controlling for the time preference described above. To do this, an analyst assumes a discount rate r, where r ranges commonly between zero and ten percent depending on the application. If we denote the net benefits t years from now as Vt(in the current year, t=0), then we say the present discounted value of Vt is
[latex]\text{PDV}\left({V}_{t}\right)=\frac{{V}_{t}}{\left(1+r\right){}^{t}\text{}}[/latex] Figure 6.4.2 shows how the present value of $10,000 declines with time, and how the rate of the decrease varies with the choice of discount rate r. If a project has costs and benefits every year for T years, then the net present value of the entire project is given by
[latex]\text{NPV}=\sum _{t=0}^{T}\frac{{V}_{t}}{{\left(1+r\right)}^{t}}[/latex] .

A graph of The Impact of a Discount Rate on Present Value Estimates

The Impact of a Discount Rate on Present Value Estimates Source: California Department of Transportation

A particular cost or benefit is worth less in present value terms the farther into the future it accrues and the higher the value of the discount rate. These fundamental features of discounting create controversy over the use of discounting because they make projects to deal with long-term environmental problems seem unappealing. The most pressing example of such controversy swirls around analysis of climate-change policy. Climate-change mitigation policies typically incur immediate economic costs (e.g. switching from fossil fuels to more expensive forms of energy) to prevent environmental damages from climate change several decades in the future. Discounting lowers the present value of the future improved environment while leaving the present value of current costs largely unchanged.

Cost-benefit analysis is just that: analysis of the costs and benefits of a proposed policy or project. To carry out a cost-benefit analysis, one carefully specifies the change to be evaluated, measures the costs and benefits of that change for all years that will be affected by the change, finds the totals of the presented discounted values of those costs and benefits, and compares them. Some studies look at the difference between the benefits and the costs (the net present value), while others look at the ratio of benefits to costs. A “good” project is one with a net present value greater than zero and a benefit/cost ratio greater than one.

The result of a cost-benefit analysis depends on a large number of choices and assumptions. What discount rate is assumed? What is the status quo counterfactual against which the policy is evaluated? How are the physical effects of the policy being modeled? Which costs and benefits are included in the analysis—are non-use benefits left out? Good cost-benefit analyses should make all their assumptions clear and transparent. Even better practice explores whether the results of the analysis are sensitive to assumptions about things like the discount rate (a practice called sensitivity analysis). Scandal erupted in 2000 when a whistle-blower revealed that the Army Corps of Engineers was pressuring its staff to alter assumptions to make sure a cost-benefit analysis yielded a particular result (EDV&CBN, 2000). Transparency and sensitivity analysis can help to prevent such abuses.

Efficiency, Cost Effectiveness, Innovation, and Equity

Cost-benefit analysis gives us a rough sense of whether or not a project is a good idea. However, it has many limitations. Here we discuss several other measures of whether a project is desirable. Economists use all these criteria and more when evaluating whether a policy is the right approach for solving a problem with externalities, public goods, and common-pool resources.

Efficiency

A policy is efficient if it maximizes the net benefits society could get from an action of that kind. Many projects and policies can pass a cost-benefit test but still not be efficient. Several levels of carbon dioxide emission reduction, for example, could have benefits exceeding costs, but only one will have the largest difference between benefits and costs possible. Such efficiency will occur when the marginal benefits of the policy are equal to its marginal costs. Sometimes a cost-benefit analysis will try to estimate the total costs and benefits for several policies with different degrees of stringency to try to see if one is better than the others. However, only information about the marginal benefit and marginal cost curves will ensure that the analyst has found the efficient policy. Unfortunately, such information is often very hard to find or estimate.

Cost Effectiveness

As we saw in the Module Environmental Valuation, it can be particularly difficult to estimate the benefits of environmental policy, and benefit estimates are necessary for finding efficient policies. Sometimes policy goals are just set through political processes—reducing sulfur dioxide emissions by 10 million tons below 1980 levels in the Clean Air Act acid rain provisions, cutting carbon dioxide emissions by 5% from 1990 levels in the Kyoto protocol—without being able to know whether those targets are efficient. However, we can still evaluate whether a policy will be cost effective and achieve its goal in the least expensive way possible. For example, for total pollution reduction to be distributed cost-effectively between all the sources that contribute pollution to an area (e.g. a lake or an urban airshed), it must be true that each of the sources is cleaning up such that they all face the same marginal costs of further abatement. If one source had a high marginal cost and another’s marginal cost was very low, total cost could be reduced by switching some of the cleanup from the first source to the second.

Incentives to Innovate:

At any one point in time, the cost of pollution control or resource recovery depends on the current state of technology and knowledge. For example, the cost of reducing carbon dioxide emissions from fossil fuels depends in part on how expensive solar and wind power are, and the cost of wetland restoration depends on how quickly ecologists are able to get new wetland plants to be established. Everyone in society benefits if those technologies improve and the marginal cost of any given level of environmental stewardship declines. Thus, economists think a lot about which kinds of policies do the best job of giving people incentives to develop cheaper ways to clean and steward the environment.

Fairness

A project can have very high aggregate net benefits, but distribute the costs and benefits very unevenly within society. We may have both ethical and practical reasons not to want a policy that is highly unfair. Some people have strong moral or philosophical preferences for policies that are equitable. In addition, if the costs of a policy are borne disproportionately by a single group of people or firms, that group is likely to fight against it in the political process. Simple cost-benefit analyses do not speak to issues of equity. However, it is common for policy analyses to break total costs and benefits down among subgroups to see if uneven patterns exist in their distribution. Studies can break down policy effects by income category to see if a policy helps or hurts people disproportionately depending on whether they are wealthy or poor. Other analyses carry out regional analyses of policy effects. . For example, climate-change mitigation policy increases costs disproportionately for poor households because of patterns in energy consumption across income groups. Furthermore, the benefits and costs of such policy are not uniform across space in the U.S. The benefits of reducing the severity of climate change will accrue largely to those areas that would be hurt most by global warming (coastal states hit by sea level rise and more hurricanes, Western states hit by severe water shortages) while the costs will fall most heavily on regions of the country with economies dependent on sales of oil and coal.

Some of our evaluative criteria are closely related to each other; a policy cannot be efficient if it is not cost-effective. However, other criteria have nothing to do with each other; a policy can be efficient but not equitable, and vice versa. Cost-benefit analyses provide crude litmus tests—we surely do not want to adopt policies that have costs exceeding their benefits. However, good policy development and evaluation considers a broader array of criteria.

Review Questions

What are some common mistakes people make in evaluating the costs of a policy or project, and what should you do to avoid them?

What is discounting, and how do we use it in calculating the costs and the benefits of a project that has effects over a long period of time?

Why is discounting controversial?

How does cost-benefit analysis complement some of the other measures people use to evaluate a policy or project?

References

Braden, J. B. & A. W. Ando. 2011. Economic costs, benefits, and achievability of low-impact development based stormwater regulations, in Economic Incentives for Stormwater Control, Hale W. Thurston, ed., Taylor & Francis, Boca Raton, FL.

Carson, R. T., Mitchell, R. C., Hanemann, M., Kopp, R. J., Presser, S., and Ruud, P. A. (2003).Contingent valuation and lost passive use: Damages from the Exxon Valdez oil spill. Environmental and Resource Economics, 25(3), 257-286. DOI: 10.1023/A:1024486702104.

Dawson, D. & Shogren, J. F. (2001). An update on priorities and expenditures under the Endangered Species Act. Land Economics, 77(4), 527-532.

EDV&CBN (2000). Environmental groups protest alteration of U.S. Army Corps cost benefit analysis. Environmental Damage Valuation and Cost Benefit News, 7(4), 1-3. http://www.costbenefitanalysis.org/newsletters/nws00apr.pdf .

Glossary

additionality
The extent to which a new action (policy, project etc.) adds to the benefits or costs associated with existing conditions.
cost-benefit analysis
Evaluation of how the overall benefits of a project compare to its costs.
cost effectiveness
The extent to which an outcome is achieved at the lowest cost possible.
counterfactual
The scenario against which a different scenario should be compared; in policy analysis, the way the world would have been in the absence of the policy.
discounting
The process of converting future values (costs or benefits) into an equivalent amount of money received today; controls for human time preference.
net present value
The present discounted value of a stream of net benefits.
opportunity cost
The cost of foregoing the next best choice when making a decision.
present discounted value
The value of something in present-day (rather than future) terms.
sensitivity analysis
Evaluation of how sensitive the results of an analysis are to changes in assumptions used in the analysis.