-. ' '■; ,.
}i-*.iiisns^
m^^^^
m
'■^■^-y ^^rr^'iSK!-^
s
333.556
i
l»i
f/.f
I ilf
V
''J*»
^
' '■■•'''i!i._"ii."'^Vii''^V
•
%einbef.
--:|^
;^^:v«
J
333.956 N17SS
SSUE PAPER
• S^-mON AND STEELHE AD RESEARCH AND MONITORING
88-23
NORTHWEST POWER PLANNING COUNCIL
September 21, 1988
Montana Slats Library
3 0864 1003 1165 6
MORRIS L BRUSETT
CHAIRMAN
Montana
George Turman Montana
;red HaUock Oregon
Norma Paulus Oregon
NORTHWEST POWER PLANNING COUNCIL
851 S.W. SIXTH AVENUE • SUITE 1100 PORTLAND, OREGON 97204-1348 • (503) 222-5161
Toll free number for Idaho, Montana & Washington; 1-800-222-3355 Toll free number for Oregon: 1-800-452-2324
TOM TRULQVE
VICE CHAIRMAN
Washington
R, Ted Botoger Washington
James A GoUer Idaho
Robert (Bob J Saxvik Idaho
Northwest Power Planning Council Staff Issue Paper- Salmon and Steelhead Research and Monitoring
September 21. 1988
Many important unceriainties remain about the biO'Dgy of Columbia River Basm salmon and steelhead and the success of the region s efforts to protect ana enhance them. Research and monitenng should provide information that will improve our understanding of this important resource. But the life cycle of these fish is complex. They migrate between river and marine environments, and are affected by a variety of human development activities withm these environments. They travel through many jurisdictional boundaries, and many entities are responsible for managing them. In addition, numerous parties have been interested m conducting research on a wide variety of topics related to salmon and steelhead. As a result, the development of a basinwide research and monitoring program has not been easy.
The significance of problems in research and monitoring and the urgency to find solutions were highlighted recently at the Council's Salmon and Steelhead Round Table, held m late June 1988. Although the Round Table was intended to be a general evaluation of the fish and wildlife program during the past five years, speaker after speaker focused on the need to solve research and monitoring problems. For example, Tim Wapato, executive director of the Columbia River Inter-Tribal Fish Commission, stated, ' Research and monitoring continue to operate m a very fragmented fashion, which results m technical and policy disputes. ... A process is needed to address such complex issues. ' Other participants echoed his call for coordinated research planning and monitoring. In developing this issue paper. Council staff has found that all major parties involved m salmon and steelhead research believe these issues must be addressed m the near future.
Therefore, the Council seeks comment on the following questions
1. What mechanism should be employed to provide policy guidance on research and monitoring questions, such as management needs and priorities, overall levels of research effort, and dispute resolution"^
2. How can cooperation m planning Corps of Engineers-funded research be improved while recognizing the Corps' authority and responsibility for its research program''
3. How might the technical quality of research and monitoring be unproved''
4. What method should be used to measure fish and wildlife program progress toward the doubling goaP
5. Who should fund the effort to monitor and evaluate the progress of the fish and wildlife program?
6. How can communication of research and monitoring results to resource managers and the interested public be improved?
The six issues identified for public comment are critical links in ensuring that necessary Information is developed and made available to refine the fish and wildlife program and to help achieve program goals. Their resolution should help in carrying out the Council's policy of adaptive management, which is intended to permit the pursuit of an action-oriented program in the face of considerable biological uncertainties. Adaptive management requires an assessment of the effectiveness of actions, the resolution of critical uncertainties, and an effective forum for communicating research and monitoring results to decision makers and other interested parties.
An additional problem that has been identified frequently by all parties in the basin is the formation of new groups to solve problems, a situation that often taxes available staff. Any solutions to research and monitoring problems need to make the most efficient use of limited staff time basinwide.
The purpose of this issue paper is to highlight the issues, identify potential options for solving the problems, and seek public comment on the options. Accordingly, the issue paper first describes the background of research and monitoring in the Columbia River Basin, how research and monitoring are provided for in the fish and wildlife program, and the problems in research and monitoring that still need to be solved. New developments that may contribute to solving these problems are described. Finally, the issue paper discusses the six issues in research and monitoring that require resolution, and describes options for resolving them.
Research planning and monitoring in the complicated jurisdictional environment of salmon and steelhead management are not simple matters. As a result, we have made no attempt to advocate a specific solution until public comment can be reviewed. The ultimate goal is to develop a cohesive research and monitoring process that addresses the identified concerns. Alternatives identified are not necessarily mutually exclusive, and commentors should feel free to propose additional alternatives as appropriate. We have developed this issue paper in close consultation with representatives of the Columbia Basin Fish and Wildlife Authority, Bonneville Power Administration, the Corps of Engineers, the Pacific Northwest Utilities Conference Committee, and other entities interested in salmon and steelhead research and monitoring.
The schedule for presentation and comment on the issue paper is as follows:
September 14-15 Staff presentation of issue paper at Council meeting in Boise
October 12-13 Public comment at Council meeting in Missoula
November 9-10 Public comment at Council meeting in Spokane
December 14-15 Council action at Portland Council meeting
Consultations will be held on request with interested parties during October and November. Public comment will be received in the Council's Portland office until 5 p.m., November 18, 1988.
Council action could result in a decision to enter rulemaking on a proposed fish and wildlife program amendment, which would involve another period of hearings in each state and public comment.
For further information or copies of reports mentioned in the issue paper, contact Judy Allender, Northwest Power Planning Council, 851 S.W. 6th, Suite 1 100, Portland, Oregon 97204, 503-222-5161 ;' 1-800-452-2324 (Oregon toll-free number); or 1-800-222-3355 (regional toll-free number).
BACKGROUND
Research, monitoring, and evaluation are all closely related activities aimed at increasing our understanding of the biology of salmon and steelhead. Ideally, research tests specific hypotheses that could explain a set of observations. Monitoring is the routine collection of information for the purpose of following the progress of actions or detecting changes from "normal" conditions. Evaluation is the use of monitoring or research information in an effort to determine the effectiveness of particular actions. These distinctions are not always clear in practice. The planning and collection of information for research, monitoring and evaluation frequently overlap.
Research
Research is a major component of fisheries activities in the Columbia River Basin. Salmon and steelhead research encompasses a wide diversity of subjects, reflecting the complexity of salmon and steelhead life cycles and their environment. Research subjects include habitat requirements, artificial production methods, supplementation (introduction of artificially-produced fish to augment natural production), disease, physiology, migration, passage at dams and through reservoirs, stock Identification, population dynamics, harvest management and genetics. This information is needed to refine and implement the fish and wildlife program, and to manage the salmon and steelhead resource.
About $12 to $16 million per year is spent on research in the Columbia River Basin. Bonneville funds about half of this effort, or $7 to $8 million per year, under the fish and wildlife program Bonneville expenditures for research have ranged from about one-quarter to one-third of Bonneville's annual fish and wildlife program expenditures. Bonneville's total program expenditures could increase to about $50 million per year in the 1990's, with about $13 to $16 million potentially available for research. Figures for program expenditures in the 1990's are for planning purposes only, and could change as new information becomes available.
In addition, the U.S. Army Corps of Engineers spends about $2 million per year on salmon and steelhead research in the basin. Three mid-Columbia Public Utility Districts, the National Marine Fisheries Service, the U.S. Fish and Wildlife Service, and other sources contribute up to $6 million per year. "^ ^
Related to this in-basin research are extensive salmon and steelhead research programs outside the Columbia River Basin. The largest of these are programs conducted by the Canada Department of Fisheries and Oceans, University of British Columbia, and Alaska Department of Fish and Game Extensive out-of-basin research is conducted on hatchery effectiveness, disease, supplementation, genetics, and harvest management.
There are good examples of research results that have clearly benefited the fisheries resource in the Columbia River Basin. Some notable highlights include:
1. The development of the Oregon Moist Pellet diet in the 1960s allowed consistent and convenient hatchery fish feeding without exposing fish to diseases previously present in fish foods.
2. New techniques have appreciably increased the success of diagnosing fish diseases. The capability to treat diseases through antibiotics and to prevent them through segregation and improved hatchery practices also has improved.
3. The development of the coded-wire tag allowed monitoring of specific groups of fish and has been an important tool in harvest management.
4. An internal fish tag termed the PIT (Passive Integrated Transponder) tag allows marl<ing of individual fish and offers considerable promise for mainstem passage and other research.
5. Mainstem dam operations have been changed in response to research results. For example, research led to changed spill procedures that reduced nitrogen supersaturation (gas bubble disease).
6. Size limits for upriver sturgeon harvest recently were changed as a result of research.
Monitoring
Monitoring is also an important component of fisheries management in the Columbia River Basin. The smolt monitoring program conducted by the fishery agencies and tribes' Fish Passage Center has been an important part of the fish and wildlife program water budget provisions for many years. Information from this program is used by the agencies and tribes annually to manage the water budget and other fish passage actions at mainstem projects. Bonneville spends some $1 to $2 million each year for mainstem smolt monitoring. Bonneville has also funded evaluation studies on the efficacy of fish and wildlife program habitat improvement projects in Idaho and Washington.
The management agencies conduct extensive monitoring outside the fish and wildlife program. For example, the Corps monitors the passage of adult fish past each of its projects, while the mid- Columbia Public Utility Districts conduct similar monitoring at some of their projects. The fishery agencies and tribes conduct a large scale program to monitor catch, escapements, and other fisheries data coastwide, primarily for harvest and other management decisions.
RESEARCH AND MONITORING IN THE FISH AND WILDLIFE PROGRAM
The 1987 Columbia River Basin Fish and Wildlife Program outlined a salmon and steelhead research and monitoring program (Section 206). The Council's objective was to end research fragmentation by establishing overall areas of emphasis, to provide a process for long-term research planning, and to improve the participation of interested parties in research planning.
For research funded by Bonneville, the program identified four areas of emphasis: studying reservoir mortality and water budget effectiveness, solving disease problems affecting spring and summer Chinook, increasing the effectiveness of hatchery production, and improving supplementation
-4-
#
techniques. The program established a research technical work group for each area of emphasis, composed of representatives of the fisheries agencies and tribes, Bonneville, the Corps, utilities, the Council and others. The technical work groups were given the responsibility to develop five-year research work plans, to assist Bonneville in developing, evaluating, and reviewing requests for proposals and project work statements, and to develop statistical design standards. The research work plans developed by the technical work groups were to be submitted to the Council for approval as a package.
In addition, the program established two areas of emphasis for research funded by the Corps: improving bypass at mainstem projects, and evaluating and improving the effectiveness of transportation. The program identified the Corps' Fish Passage Development and Evaluation Program as the mechanism for developing research work plans in these two areas of emphasis. The research planning process is advised by the Technical Coordinating Committee, which is composed of representatives of the state and federal fisheries agencies, Indian tribes, and the Corps. The recommendations of the Technical Coordinating Committee are submitted to the Corps' Fish Management Committee-the Corps' policy-level committee-for final approval.
The 1987 program also committed the Council to develop a system monitoring and evaluation program for measuring program progress toward the goal of doubling salmon and steelhead runs, to monitor consistency of actions with program policies, and to help unify data collection efforts within the basin. To help develop and implement the system monitoring and evaluation program, the Monitoring and Evaluation Group was established. It is composed of technical staff from the fisheries agencies and tribes, Bonneville, the Pacific Northwest Utilities Conference Committee, and private consultants.
The program states that the system monitoring and evaluation program should include:
1 . Development of alternative means to assess program progress and consistency with program policies;
2. evaluation of research results and implications for achieving program objectives;
3. development of a coordinated information system;
4. maintenance of the system planning model;
5. assistance in the integration of subbasin plans in system planning; and
6. development of methods to incorporate genetic conservation into system planning.
PROBLEMS IN RESEARCH AND MONITORING
The following have been raised frequently as the major concerns about current approaches to salmon and steelhead research and monitoring.
1.
Research planning and implementation lack a clear mechanism for policy guidance.
There is no established mechanism for ensuring that research is relevant to management problems and that overall levels of research effort are appropriate to management needs; nor is there a
defined procedure for referring research-related disputes to the policy level for resolution. This has caused many problems in implementing the program's research planning process in the past year and a half.
The major problems occurred in mainstem research areas, where policy conflicts have been most intense. For example, the lack of policy guidance contributed to the development of substantial and unresolved controversy in planning Bonneville-funded research on reservoir mortality and water budget effectiveness. Without a forum for resolution of these disputes, interested parties developed two competing research work plans for research in this area.
There have also been problems in achieving a coordinated program of Bonneville-funded and Corps-funded mainstem research, because a policy forum has not been available to develop overall mainstem management needs. As a result, some important research areas may be omitted, and there may not be adequate coordination of mainstem research funded by the two agencies. In short, the interested parties have not agreed on what needs to be learned and how it should be studied.
In addition, some are concerned that the Council's efforts to focus research through the areas of emphasis may exclude some important research topics such as evaluation of habitat improvements and survival of fish in the estuary and ocean. Some research spans two or more technical work groups and is not easily developed in the current planning framework. In the absence of a mechanism for policy guidance, work on some important topics that overlap areas of emphasis might not be pursued. For example, the role of hatchery practices in effects of supplementation is not clearly the purview of either the hatchery effectiveness or supplementation work groups. Research that addresses system- wide problems also may not be addressed by any one technical work group. Each of these deficiencies might be corrected with effective and continuing policy guidance.
2. Cooperation and coordination in planning Corps-funded research need to be improved.
In addition to the need to coordinate Bonneville-funded and Corps-funded research, disputes have arisen within the Corps research process because the fisheries agencies and tribes believe that their views have not been adequately represented or taken into account in the Corps decision-making structure. Some means to improve cooperation in Corps research planning may be needed.
3. Technical quality of research and monitorinq needs to be improved.
Some have argued that existing mechanisms may be inadequate to assure technical quality of research activities and products, including coordination of effort, development of statistical standards, and scientific review.
Coordination. Coordination should help avoid duplication of past and ongoing efforts, should enable researchers and resource managers to take advantage of opportunities to share resources, should provide for study designs that are complementary as appropriate, and should help assure that a systemwide research perspective is maintained.
As an example, coordination of projects within and between the six areas of research emphasis, and with hatchery production and supplementation programs, needs to occur. Currently, coordination does occur in an ad hoc fashion, but these efforts are limited by staffing and time limitations.
Coordination also needs to occur with other research programs in the basin, such as those associated with the Lower Snake River Compensation Plan hatcheries, Mitchell Act hatcheries, other hatchery programs, universities, utilities, and other research efforts. Opportunities may also exist for coordination with research efforts outside the basin, such as western Oregon and Washington, Alaska, Canada, and perhaps even farther afield.
Statistical standards. An important aspect of technical quality in research and monitoring is the use of sound statistical standards and experimental designs. Statistical standards provide a measure of objectivity in hypothesis testing, while a sound experimental design ensures that the research or monitoring program will adequately address the issue at hand. Flaws in the experimental design or failure to adopt adequate statistical standards can result in wasted effort and research that creates additional controversy instead of helping answer important questions. There is a need to provide guidance to researchers regarding the level of statistical precision required for different types of research and a process to review proposals for adequacy of experimental design.
Scientific review. The need also exists for peer review of research proposals and reports. Inviting unbiased scrutiny by recognized experts could improve the quality of research projects and the interpretation of results. It could help ensure that proposed research actually is needed and will contribute to answering major management questions. It could also decrease the appearance of conflict of interest that has been raised by some who are concerned when research planning is done by those whose organizations stand to obtain contracts as a result of the research plans.
"^^ It has not yet been determined how progress of the fish and wildlife program in achieving program goals should be measured.
The Council's fish and wildlife program represents a considerable investment on the part of the Council, fisheries managers, the implementing agencies, the region's ratepayers, and other interested parties. To date, there has been no systematic method available to determine the progress of the program to "protect, mitigate, and enhance" the fish and wildlife resource. The 1987 Fish and Wildlife Program provided the foundation for establishing a system monitoring and evaluation program to fill this gap. The system monitoring and evaluation program was intended to bring together research and monitoring results into a measure of progress toward the salmon and steelhead doubling goal. The units by which such progress should be measured-whether using smolts, adult fish, production surplus to spawning needs, or other method-need to be determined. The Monitoring and Evaluation Group has developed a series of recommendations for this effort.
^- There is a need to Identify who should fund the effort to monitor and evaluate the effectiveness of the fish and wildlife program. ' " ~ "~
As noted above, the 1987 Fish and Wildlife Program called for the development of proposed alternatives for a system monitoring and evaluation program. However, the program did not specify who should fund that effort and who should be involved in it. This issue needs to be addressed before the system monitoring and evaluation program can be fully implemented.
6- Communication of research and monitoring results needs to be improved.
There is no specific mechanism for summarizing and communicating research and monitoring results to decision makers and interested parties so that results can be used to improve management, mform policy decisions, and guide the adjustment of current actions and goals. Because
communication mechanisms are inadequate, there is the potential for duplication of past and ongoing research efforts. Many research and monitoring programs exist, but results are reported in varying formats at various times of the year. Often information is kept in files or Is reported In the "grey" literature that receives limited circulation. There is no central process for synthesizing the results of research into a form that is readily available to decision makers and interested parties. Existing communication mechanisms are not always effective for all relevant parties. As a result, Information gained from research and monitoring may not be contributing as effectively as it might to the achievement of program goals.
NEW DEVELOPMENTS IN RESEARCH AND MONITORING
During the past year, three major developments that could affect planning and Implementation of research and monitoring have occurred. These are- ongoing negotiation of a modified research planning process for mainstem passage research; agreement between Bonneville and the Columbia Basin Fish and Wildlife Authority on a process for implementing fish and wildlife program measures; and development by the Monitoring and Evaluation Group of alternative methods for measuring program progress. Each Is discussed briefly below.
"■ • Mainstem passage research planning process. As this issue paper goes to press, a long-term spill agreement Is being negotiated by the fisheries agencies and tribes, Bonneville, and the Pacific Northwest Utilities Conference Committee and others. This agreement may Include a research planning process for mainstem research. A four-stage process has been discussed. It includes the identification of research problems at the policy level, and scientific review by a panel of technical experts. The process is intended to address all research having to do with mainstem passage and hydroelectric project operations in the Columbia River Basin. It Includes the identification of major management research needs and the approval of an annual research plan. Until this process is put in place, a variety of ad hoc efforts have kept mainstem research and monitoring moving forward. If the process Is ultimately not incorporated in the long-term spill agreement, it is likely to be recommended as a modification of the Council's current research planning process.
2. Implementation Planning Process. Bonneville and the Columbia Basin Fish and Wildlife Authority are in final stages of lengthy negotiations on a process designed to formalize implementation of the fish and wildlife program. The goal of the process is to ensure close collaboration on program implementation between Bonneville, the fisheries agencies and tribes, the Council, and other parties. As is the case with the research planning process being negotiated under the spill agreement, the Implementation Planning Process provides for the formation of a policy group (Program Policy Review Group) to help set implementation priorities and Identify funding levels for major categories of expenditures. It creates a Scientific Review Group to review research proposals, evaluate Individual projects, and monitor the effectiveness of the fish and wildlife program. The Implementation Planning Process also creates several technical working groups to aid in project development and proposal review. The Scientific Review Group and technical working groups will review technical quality, schedules and long-term costs of proposed projects and will make recommendations to the Policy Review Group. The text of the Implementation Planning Process will be available in the near future.
A common thread in both of the above is an effort to separate policy and technical issues. This separation is viewed as useful because policy and technical issues frequently have become confused in the recent past, and the result has been that technical staff have been unable to pursue technical solutions to research issues. The intent of this structure is to allow the technical group to focus on technical issues by providing an explicit policy group to handle policy problems and mediate disputes.
3. Proposal for a System Monitoring and Evaluation Program. The 1987 Fish and Wildlife Program called for establishment of a Monitoring and Evaluation Group to assist in the design and implementation of a plan to monitor and evaluate progress toward the doubling goal and consistency with policies set forth in the program (Section 206(d)). The group has responded by examining alternatives and developing recommendations for the design of a system monitoring and evaluation program. A report describing their conclusions is attached. The group has also proceeded with development of several elements of the monitoring program described in the 1987 Fish and Wildlife Program, including a coordinated information system and a program to identify genetic impacts in production planning. A coordinated information system is intended to facilitate communication of research and monitoring data between the various agencies. It is also intended to make this information more accessible to all interested parties in a usable form.
The Monitoring and Evaluation Group identified several alternative approaches to measuring the progress of the fish and wildlife program. These are discussed below. The group also reached two conclusions: First, there was no single measure (smolt counts, adult counts or surplus production) that would totally reflect program progress. Each of the approaches they identified measures the progress of different aspects of the program. Second, the group felt that it was not possible to directly separate program effects from non-program effects such as changes in harvest rates or natural survival rates. This was because of the many conflicting factors that interact to determine the number of adult or juvenile fish produced from the Columbia River. The group concluded that it was possible to separate program effects from non-program effects only by using a computer model to simulate the salmon and steelhead life cycle.
ISSUES AND OPTIONS
As a result of the 1987 Fish and Wildlife Program provisions, a variety of ad hoc discussions, and the three initiatives described above, progress is being made in solving problems in research and monitoring. However, as noted above, a number of problems remain. With the problems of the past few years in mind, this is an opportune time to review the issues and decide whether more can be done to solve the remaining problems. Accordingly, the Council seeks comment on the following major research and monitoring questions and the options for resolving them. Commentors are especially urged to consider and provide comment on:
• The likely effectiveness of the options and opportunities for streamlining any proposed processes;
• the efficiency of the options in terms of relative levels of personnel time and attendant overhead costs; and
• appropriate levels of expenditure that may be required to implement the options.
The options are not necessarily mutually exclusive and may be considered in any appropriate combinations and variations. Commentors are also encouraged to propose any additional alternatives.
1. What mechanism should be employed to provide policy guidance on research and monitoring, such as management needs and priorities, overall levels of research effort, and dispute resolution?
Recent problems in research and monitoring have pointed to a need for early policy guidance so that resource managers can clearly identify the research questions they need to have answered and indicate the priority they place on the questions. Policy guidance also is needed to help deal with research-related disputes that arise.
There are several senior level policy groups currently working on different aspects of the fish and wildlife program. The System Planning Oversight Committee and the System Planning Forum were formed by the Council to address policy issues arising in system planning. The Mainstem Executive Committee was formed by the Council, the fishery agencies and tribes, Bonneville, the Pacific Northwest Utilities Conference Committee, and the Corps to address policy issues encountered in managing spill, water budget, and other aspects of mainstem passage. Under the Implementation Planning Process, a Program Policy Review Group would be formed. It would be a senior staff working group and would include somewhat fewer than the number of entities represented on the Mainstem Executive Committee and System Planning Oversight Committee. For example, as currently structured, its membership would not include representatives of fishing or conservation groups, or land and water managers.
Two options have been identified to respond to the need for policy guidance and dispute resolution on research issues:
Option A. Rely on the Implementation Planning Process to improve the current situation. This process creates a Program Policy Review Group, composed of Bonneville, the fisheries agencies and tribes, the Corps of Engineers and the Council. The Program Policy Review Group is slated to provide policy direction and an arena for resolving program implementation disputes. The group will assist Bonneville in developing an annual implementation work plan. The fish and wildlife program provides for Council review of the annual work plan once it is submitted by Bonneville.
Option B. Establish a single cooperative research and monitoring policy group. A policy group composed of senior-level staff could be formed to deal with Bonneville-funded and Corps-funded implementation. The group could augment the membership of the Implementation Planning Process Program Policy Review Group with some members of the System Planning Oversight Committee. The System Planning Oversight Committee has a membership that is somewhat broader than the Program Policy Review Group in that its membership includes sport fishing interests and land and water managers. The group could establish subcommittees to carry out specific functions now assigned to the individual policy groups. The group could reflect the interests of all entities involved and would follow the fish and wildlife program. With respect to research and monitoring the group could: 1) identify and prioritize systemwide management needs; 2) determine priorities based on identified management needs; 3) resolve disputes or provide mechanisms for doing so; and 4) oversee research and monitoring actions. The policy group could be supplemented by a senior policy appellate body as needed to resolve major disputes.
-10-
2. How can cooperation in planning Corps-funded research be improved while recognizing the Corps' authority and responsibility for its research progrann?
The Corps of Engineers established a research planning process in the early 1950s. This process, the Fish Passage Development and Evaluation Program, is composed of representatives from the Corps, fisheries agencies and tribes. The Corps believes it has the authority and responsibility to conduct project-specific research, not to address regionwide research needs. The Corps has stated an unwillingness to give up its discretion in managing and overseeing research.
The fisheries agencies and tribes have felt that their views have not been represented adequately or taken into account in Corps decisions on research. Agency, tribal and Corps representatives on the Fish Passage Development and Evaluation Program Technical Coordinating Committee have discussed ways to improve cooperation at the technical level, which could result in better representation of agency and tribal views to the Corps policy level. However, the fishery agency and tribal policy makers have felt that pursuing these discussions is not very useful. They feel that the overriding problem is that Corps policy decisions do not adequately take their views into account.
The following options describe potential solutions to this problem:
Option A. Implement a research planning process that recognizes the Corps' authority and responsibility but improves cooperation with the fisheries agencies and tribes. The following elements are based on the four-stage research planning process which may be developed as part of the long- term spill negotiations, and could include:
.1. A policy-level group could work with all affected parties to develop a mainstem research plan including objectives, criteria and guidelines for selection of mainstem research projects; and identification of priorities for mainstem research.
2. The Corps could agree not to implement projects inconsistent with the mainstem research plan unless it provides written justification that the proposal is consistent with the Northwest Power Act or is necessary to satisfy the Corps' authority and responsibility for its research program.
Qpt'on B. The Council could call on the Corps to work with other affected entities to review the current research process and develop a more cooperative approach.
3- How might the technical quality of research and monitoring be improved?
There may exist a number of mechanisms that could improve technical quality of research and monitoring. These include: better coordination of the many research, monitoring, and planning activities within and outside the basin; improved use of statistical techniques and experimental designs; and additional scientific review. Neither the research technical work groups nor the Monitoring and Evaluation Group is presently assigned to undertake these activities. The Scientific Review Group of the Implementation Planning Process, if implemented, could help provide the additional scientific review.
-11-
Four options for improving technical quality have been identified:
Option A. Rely on the Implementation Planning Process to improve technical quality.
This would require close cooperation between the Implementation Planning Process Scientific Review Group and the Council's Monitoring and Evaluation Group. The two groups would be responsible for 1) identifying opportunities for coordination among research and monitoring efforts; 2) providing scientific review of project statements of work, statistical standards, and results; 3) coordinating genetics monitoring and planning; and 4) implementing a program for monitoring and evaluating the effectiveness of program actions basin-wide.
Option B. Establish a cooperative integration and review group. This alternative would merge the existing Monitoring and Evaluation Group with the Scientific Review Group. It could also include representation from other responsible entities within the basin, e.g., the Corps of Engineers. With policy-level direction, the group could integrate and evaluate systemwide efforts to double salmon and steelhead runs. It could assist in identifying research and monitoring priorities, and in coordinating various research and production programs. In addition, it could improve research quality by establishing forums for scientific review and developing statistical standards. To accomplish these tasks the group could form subcommittees as needed to address such topics as system planning, genetics, system monitoring and evaluation, habitat classification, and scientific review of research proposals.
Option C. Incorporate a mechanism to increase independent scientific review by establishing technical peer review panels composed of independent scientists working inside'and outside the Columbia River Basin. These panels would be responsible for reviewing research proposals for scientific merit and could submit their views to the entity established under Option A or B above.
Option D. Establish an independent research planning foundation. The option would establish a scientific institution to plan and fund salmon and steelhead research in the basin. The goal of the foundation would be to provide independent planning and funding of program research. The organization could be completely independent, with an advisory board made up of representatives from the fisheries agencies and tribes. Alternately, it could be governed by a board of directors representing the major interests in the basin. Funds for research activities coming from Bonneville and other sources would be administered by the foundation. The foundation would be responsible for research planning, proposal solicitation, proposal review and evaluation, monitoring, and making available results concerning fish and wildlife related issues in the basin.
4. What method should be used to measure fish and wildlife program progress toward the doubling goal?
The 1987 Fish and Wildlife Program stated that the Council's system monitoring and evaluation program should include "[d]evelopment of alternative means to assess progress toward achieving the goal of doubling the runs of salmon and steelhead in the Columbia River Basin consistent with the policies stated in Section 204." The Monitoring and Evaluation Group has developed a set of alternatives and provided a recommendation to the Council. The report of the Monitoring and Evaluation Group is attached.
-12-
The discussion below sets out the alternatives provided by the Monitoring and Evaluation Group and provides a rough estimate of the initial cost of implementing each alternative. It should be made clear that these cost estimates only cover the cost of organizing existing information into an expression of program progress. They do not include the cost of all existing monitoring programs or related programs such as the coordinated information system. In addition, information beyond that collected now as part of existing monitoring and research efforts is likely to be needed to achieve an acceptable level of precision. The size of the overall effort may depend on the nature of subbasin plans currently being formulated in system planning. For this reason, it is not possible to provide the total cost of a monitoring program at this time. It will need to be reviewed in the future as more defined monitoring programs are proposed.
QP^'Q^ ^- Measure progress as the change in smolt output from the Columbia River. This would provide an early indication of the effect of the program on increasing smolt production and smolt survival through the hydroelectric system. However, it would not include those parts of the program designed to increase adult survival, nor would it provide a system-wide perspective. In addition, techniques do not presently exist for estimating the size of the smolt outmigration from the Columbia River, or for directly isolating the cause of changes in smolt numbers. Development of these techniques would likely be a long and expensive process.
An alternative that would address only fish migrating from above Bonneville Dam would be to use existing indices of smolt passage to provide an index of the change in smolt outmigration from year to year (as opposed to an estimate of the actual number). This would involve collecting and processing information now assembled by the Fish Passage Center and others. This is presently being done to some extent, although there is not universal agreement regarding the relationship between the indices and actual population size. The cost of implementing the smolt index as a measure of progress is estimated at about $50,000 annually.
QP^'Q" ^- Measure progress as the change in the number of adults produced, this approach would include the number of fish returning to the basin as well as those harvested in the ocean. An additional correction would have to be made to account for those fish harvested in the ocean that would have died in any event from natural causes. The adult count is the method implied in the Council doubling goal. However, like the previous option, the numbers of fish observed in any year are the product of many factors. Sorting out the effect of the program or of specific types of program actions would be very difficult. For this reason, it is likely that little knowledge would be gained that could be used to refine the program.
Above Bonneville Dam, computing the annual adult production would involve the use of adult counts made at the mainstem hydroelectric projects by the Corps of Engineers, as well as estimates of catch made by the fishery management agencies. Below Bonneville, fishery agency estimates of tributary return and catch could be used. Cost of reviewing and compiling this information into an annual index of progress is estimated at $75,000 annually.
Qp^'°" ^- Compute the change in salmon and steelhead surplus production as an index of program progress. This option would utilize a computer model of the salmon and steelhead life cycle which would incorporate the current understanding of the relationships between variables as well as real-time monitoring and research data. The principal result would be a calculation of the adults that are surplus to spawning needs. This surplus portion is available for harvest or to provide a buffer for unanticipated changes in mainstem mortality or natural fluctuations. As real-time monitoring and
-13-
research data are reflected in the calculation, the change in projected surplus production could be used as a measure of program progress.
This approach offers the opportunity to isolate the effects of the program within a computer model. The model can also be used to generate testable hypotheses about the program. As these are tested with specific program actions, important knowledge could be gained for refinement of the program. On the other hand, the modeling approach is focused on long-term change and accordingly would not provide real-time monitoring of changes in important parameters such as harvest or passage rates.
The cost of development, refinement, and operation of the model to generate an index of program progress is estimated at $150,000 annually.
Option P. Recommendation of the Monitoring and Evaluation Group. One of the principal conclusions of the group was that there was no single measure of program progress. Each of the above methods has certain features that are needed for a full expression of program effects. Smolt counts provide an early indication of program effects and adult counts measure the "bottom-line" of progress in restoring the fish runs. Analysis of the projected surplus production can increase our knowledge, and should aid in refinement of the program. For this reason, the Monitoring and Evaluation Group recommended that all three measures be used to evaluate the program.
The cost of using all three approaches is estimated at $200,000 annually.
5. Who should fund the effort to monitor and evaluate the progress of the fish and wildlife program?
While the 1987 program stated that ". . . the Council will develop a system monitoring and evaluation program ... to measure progress toward the doubling goal as well as consistency with program policies.", it did not specify how a monitoring program would be implemented or who should fund its implementation. A wide variety of groups in the region have an interest in monitoring the success of the fish and wildlife program. The Council has a fundamental interest in monitoring the program so that it can be refined to achieve the goals of the program. Bonneville and the utilities have an interest in determining the efficacy of investments of ratepayer funds. The fishery agencies and tribes want to ensure that the program is biologically sound and successful in achieving management goals. Many public interest and environmental groups are concerned that the implementation of the program adequately addresses their concerns as well.
Because of the many diverse groups that are interested in following the progress of the fish and wildlife program, it is clear that a monitoring program needs to be developed and implemented in a cooperative fashion. Two avenues appear to be available to accomplish this goal.
Option A. Council funding. Under this option, the Council would fund the further development and implementation of a program to monitor and evaluate the fish and wildlife program. This could be done through the existing Monitoring and Evaluation Group or an expanded group. As noted above, this group has spent considerable effort in developing alternative means to measure program progress and to develop the components of a monitoring program. The group currently utilizes expertise from the fish agencies, tribes. Council, the utilities, and consultants in a group that reports to the Council. This group could be charged to develop and implement a monitoring program and report periodically to the Council and the region regarding progress of the program and issues requiring particular management attention or possible modification of the program. The 1987 Fish and Wildlife Program
-14-
charged the Monitoring and Evaluation Group to "aid in development and implementation of a systemwide monitoring and evaluation program. . . ." Presently, the Council provides approximately $100,000 annually to fund the Monitoring and Evaluation Group. Further development and implementation of a monitoring program can be expected to require an expansion of the present group and its activities. This could increase costs significantly.
Option B. Bonneville funding. This option would call for Bonneville funding of a cooperative technical group that could represent all entities, including the Columbia Basin Fish and Wildlife Authority, Bonneville, the Pacific Northwest Utilities Conference Committee, the Corps, the Council and others. The group could be charged with conducting a system monitoring and evaluation program that built on the work to date of the Monitoring and Evaluation Group and included the elements listed in the Council's 1987 program. The group could report periodically to the Council and the region on progress of the program and identify areas of the program requiring particular management attention or possible modification of the program. Should a cooperative technical group be formed to address other aspects of research and monitoring (Issue 3 above), it might be the logical group to fulfill this function.
6- How can communication of research and monitoring results to resource managers and the interested public be improved?
It appears that existing mechanisms to communicate research and monitoring results are inadequate. The availability of policy guidance to articulate management needs could help solve this problem, because research and monitoring results that respond to clear management needs will very likely be communicated more effectively.' It is also important that research results be synthesized into a form that managers and others can use, and that an appropriate forum be developed for communicating results to all interested parties.
The following three options include a range of solutions, from implementing actions already planned (the coordinated information system) to establishing several mechanisms to institutionalize communication of results.
Option A. Take no additional action beyond existing mechanisms and the establishment of a coordinated information system. The 1987 Fish and Wildlife Program calls for the establishment of a coordinated information system as part of its system monitoring and evaluation program. The coordinated information system has been initiated. It is intended to facilitate effective exchange and dissemination of fisheries data, including coordination of systemwide data collection programs, identification of data collection needs, and coordination of system monitoring and evaluation program data with other data collection efforts.
Qp^'O" B. In addition to the coordinated information system, establish an annual research and monitoring seminar to promote interaction among scientists and decision makers, and to inform decision makers of research and monitoring results. This seminar could: 1) provide opportunities for scientists to discuss research and monitoring issues with others within and outside the basin; 2) inform decision makers and managers of research and monitoring results; and 3) provide opportunities for decision makers and managers to make scientists aware of management concerns and needs. The seminar could be held in two segments to accommodate both exchange among scientists and interaction between scientists and decision makers.
-15-
QP^'P'"' C- Establish a periodic publication for results of research and monitoring funded through the fish and wildlife program, and a compendium of ongoing salmon and steeihead research and monitoring in and out of the basin. These items could be developed in addition to, or as part of, the coordinated information system. Their purpose would be to increase the utility of research 'and monitoring results by furthering the circulation of information to researchers as well as decision makers and other interested parties. It could also serve to encourage the refined analysis and reporting of research and monitoring data.
mvax::larry:[wn]wjlla r&m ip
-16-
•
RECOMMENDATIONS FOR A PROGRAM TO MONITOR AND EVALUATE
THE FISH AND WILDLIFE PROGRAM
OF THE NORTHWEST POWER PLANNING COUNCIL
from The Monitoring and Evaluation Group
September 23, 1988
I. INTRODUCTION
In 1987, the Northwest Power Planning Council (Council) amended its fish and wildlife program (program). The program was developed in response to the Northwest Power Planning and Conservation Act of 1980 (the Act or Power Act), which called on the Council to develop a program to "protect, mitigate, and enhance" the fish and wildlife resource of the Columbia River Basin as affected by hydroelectric development and operation.
A significant feature of the 1987 program was the inclusion of a System Monitoring and Evaluation Program (SMEP) to track the progress of the program in achieving the Council's goals. To develop SMEP, the Council formed the Monitoring and Evaluation Group (MEG) in April 1987 composed of technical experts from the region.
One of the first charges of the group was to develop a set of alternative methods that could be used to measure the progress of the fish and wildlife program. This report summarizes MEG's findings. It also includes a recommendation for a measure of progress and provides a discussion of the elements that the group feels would be required to implement a system monitoring and evaluation program. Considerable work remains before a full scale monitoring program could be realized; the intent of the present effort is to further the discussion of a measure of progress and to initiate the development of the required elements of a monitoring program. Several important elements of the monitoring program specified in the 1987 Fish and Wildlife Program are now being implemented as part of the System Planning process.
Background
The Act called on the Council not only to develop a program to address the fish and wildlife losses in the basin, but to develop a program that would take significant, positive action despite acknowledged uncertainties both in the biological requirements and in the best methods for increasing fish production in the basin. The Act also called on the Council to treat the Columbia Basin as a system, necessitating an approach that transcends the jurisdictional complexities of the Columbia Basin.
The 1987 program represents a fundamental step in the development of the Council's efforts to meet this mandate. Measures in the new program are focused on achievement of an interim goal of doubling the runs of salmon and steelhead in the basin (Section 203). Efforts to reach this goal are to be consistent with the following set of policies (Section 204) that are intended to ensure equitable and sustainable benefits:
1 . The area above Bonneville Dam is accorded priority.
2. Genetic risks must be assessed.
3. Mainstem sun/ival must be improved expeditiously.
4. Increased production will result from mix of production methods.
-1-
5. Harvest management must support rebuilding.
6. System integration will be necessary to assure consistency.
7. Adaptive management should guide actions and improve knowledge.
To plan the actions needed to meet the interim goal, the Council established a basinwide process termed System Planning (Section 205). System Planning is an effort under the lead of the fish agencies and tribes to plan fisheries actions in 31 subbasins of the Columbia River. It includes, first of all, planning at the subbasin level to identify fish production objectives, constraints and opportunities. Secondly, it involves integrating these subbasin plans into an overall system plan to meet the doubling goal while maintaining consistency with the Council's policies.
Section 206(d) of the fish and wildlife program states that the System Monitoring and Evaluation Program is intended to follow the progress of the system plan in meeting the Council's goal and its consistency with the Council's policies. Such a program could fulfill two functions. First, SMEP could provide a link between the expenditure of ratepayer dollars and the Council's charge to "protect, mitigate and enhance" the fishery resource. The fish and wildlife program represents a sizeable expenditure by the Northwest ratepayers through the Bonneville Power Administration. SMEP could provide a measure of the progress achieved with these funds. Second, SMEP could provide a means of dealing with the uncertainty associated with the best means of achieving the mandate of the Power Act, The Act recognized the uncertainty but called on the Council to take positive action. A major funqtion of a system-wide monitoring program would be to provide the necessary feedback to the Council and the region so that, over time, the program could become a refined vehicle for achieving the Council's goals.
The program states that the Monitoring and Evaluation Program is to incorporate the following features (206(d) (2) (A-F)):
1 . "Development of alternative means to assess progress toward achieving the goal of doubling the runs of salmon and steelhead in the Columbia River Basin, consistent with the (program) policies
2. "Evaluation of research results and reporting on implications for program objectives";
3. "Development of a coordinated information system designed to facilitate effective exchange and dissemination of fisheries data";
4. "Maintenance of the system planning model, including documentation, recommended standards for use, and modifications'.';
5. "Integration with the system planning activities . . ."; and,
6. "Examination of quantitative methods to incorporate genetic conservation into production planning."
The initial issue in the development of the SMEP is the method of assessing progress. The choice of a measure will largely depend on the question asked of a monitoring program and the amount of information that the Council desires to obtain from a measure of progress; the ultimate size of the monitoring program will depend on the degree of resolution that is desired. The next section of this report will include a discussion of alternative means of measuring progress and MEG's recommendation regarding the best course of action. This will be followed by a description of the actions MEG feels are necessary for implementing this recommendation.
II. ALTERNATIVE APPROACHES TO MEASURING PROGRESS
Ideally, a measure of progress should not only determine progress, but should also provide information to increase our understanding, decrease uncertainty, and permit the program to be refined over time. The choice of a measure of progress largely depends on the question asked. If the question is "Have the number of salmon and steelhead produced in the Columbia Basin increased since the inception of the fish and wildlife program?" then a simple accounting of the adult production will suffice. However, little knowledge will be gained by such a method and it will not be possible to attribute positive or negative trends in production to the fish and wildlife program. A more complicated but useful question would be "Have the number of salmon and steelhead produced in the Columbia Basin increased as a result of the fish and wildlife program?" This requires the ability to separate out program effects from non-program effects such as changes in harvest rates or natural ocean survival rates. This presents considerable technical difficulties because of the numerous conflicting factors that determine the number of fish produced from the river. This question addresses the accountability for ratepayer expenditures in the whole program, but does not provide information that could be used to improve the program. A final refinement of the question might be "What types of measures in the fish and wildlife program are effective and which are not?" The answer to this provides a measure of progress, as well as information to refine the program over time.
MEG reviewed a number of options for a measure of progress and categorized them as observational or analytical in nature. The former refers to methods that rely on direct counts of fish, while the latter combines a host of monitoring and research data into mathematical expressions that attempt to explain trends in observational indices.
1. Observational Methods
Observational methods of measuring program progress would consist of enumerating the fish population at any of several points in the salmonid life cycle. These types of measurements have the advantage of being conceptually simple and could, in many cases, be derived using existing data sources.
While fish could be counted at any point in the life cycle, those in the following list would be likely observation points:
a. Smolts at the mouth of the Columbia River.
b. Adult returns to the mouth of the Columbia plus prior harvest.
c. Adult equivalent total production (explained below).
d. Adult returns to the mouth of the Columbia River.
e. Adults crossing Bonneville Dam.
Each of these points of measurement have various advantages and disadvantages relating to logistics involved in measurement, and their relevancy to the Council goal.
Juvenile Counts. The number of smolts from the Columbia system that reach the ocean has the appeal of being a direct measurement of the effects of many fish and wildlife program actions. Because the Council's jurisdiction is limited to actions relating to production and mainstem passage, and not ocean or river harvest rates, it could be said that the Council's obligation is simply to double the number of smolts reaching the ocean. The subsequent fate of the fish in regard to human intervention would be the responsibility of the state, federal, and tribal harvest managers.
Juvenile counts would provide a rapid index of the effect of program actions. The number of smolts would generally provide a measure of production and juvenile passage improvement within a
year or two. In contrast, the number of adults produced can only be measured after the return of up to five year classes in the case of Chinook salmon.
The Council, however, has consistently rejected using juvenile measurements in favor of adult production or returns as a final measure of program benefits. This is because an evaluation based solely on the production of juvenile fish fosters the fragmented approach to management that has plagued past efforts to restore the fishery and does not provide a systemwide perspective on program effectSc
Another difficulty with using juvenile counts to monitor the program is that they only tell part of the story in regard to the influence of the program on salmon and steelhead production. Several actions in the fish and wildlife program address the survival of adult fish. In addition, program actions can affect the survivability of smolts beyond the point where direct Council influence stops. Hatchery practices and mainstem passage conditions, for instance, can influence the quality and survival of smolts reaching the ocean. Doubling the number of smolts below Bonneville that are of poor quality and with little potential for return will not achieve the Council's goal.
Monitoring the smolt outmigration also has major logistical difficulties. Potentially, the population of smolts outmigrating from above Bonneville Dam could be determined by a population estimate at the dam itself. However, there is presently no reliable method for estimating the smolt population size at any of the mainstem hydroelectric projects. Also, there is presently no technique available to estimate smolt population size from tributaries below Bonneville Dam. Development of techniques for estimating the smolt population both above and below Bonneville Dam would be a difficult and expensive endeavor.
An alternative to estimating the size of the outmigrating population might be to rely on changes in various smolt migration indices as an index of program progress. A number of these indices are presently collected by the Fish Passage Center operated by the fishery agencies and tribes. Because the information is collected routinely, this could offer a low cost, and at the present time the only, possibility for measuring progress in terms of the size of the smolt outmigration. However, to be a suitable measure of progress, an index must have a consist, although probably unknown, relation to the size of the outmigrant population. At the present time, there is considerable doubt regarding the consistent relationship between existing smolt indices and the population size (see various annual reports from the Smolt Monitoring Program of the Fish Passage Center).
Adult counts. The primary advantage of using adult counts to evaluate the program is that they represent the "bottom line" in regard to the actual effect of the Council's efforts to restore the salmon and steelhead resource. Thus intuitively, adult counts may have the greatest appeal as a measure of program effects.
By far the easiest and least expensive method for monitoring the program in terms of adults would be to track the number of adult fish returning to the mouth of the Columbia River. For fish produced above Bonneville Dam, this would entail adding the number of fish counted at Bonneville to an estimate of the number caught or killed between the dam and the mouth of the Columbia River. Similarly, for fish produced below Bonneville, the estimate of loss below Bonneville would be added to the estimates of return to the individual subbasins. The Corps of Engineers and the fishery management agencies routinely collect this information, and no additional cost would be involved.
While simplicity is the greatest virtue of this approach, it is also its greatest drawback. An evaluation limited to adult returns to the river would have little ability to separate out program affects from all other factors affecting returns such as harvest and natural survival rates. Success or failure in achieving the doubling goal could not be attributed to the effect of the program. Very little would be
added to our understanding of the causes of trends in returns, and little contribution would be made toward the refinement of the program.
The program defines the doubling goal in terms of "the number of adults returning to the mouth of the Columbia River plus the number of adults caught in the ocean." Under this approach, adult returns to the river mouth would be estimated as described above, plus prior ocean catch of each stock. By adding the estimated ocean catch, the effect of fluctuations in harvest rate on returns would be taken into account, and would provide a truer measure of program progress.
However, it would still not be possible to totally isolate the contribution of the program from other factors, notably variation in the natural survival rate in the ocean. Even if the ocean survival rate could be assumed to be more or less constant (excepting dramatic and obvious changes such as an El Nino event), and progress toward the doubling goal could be attributed to the program, we would still not know how the program affected returns. No information would be provided as to the efficacy of different types of actions, our understanding of the system would not be increased, and no contribution would be made to the refinement of the program. In addition, adding the unadjusted catch to the estimate of return would consistently overestimate program production since some of the fish caught would have died in any event from natural causes prior to reaching the river mouth.
The measure could be improved by adjusting the estimates of ocean catch by the number of fish that would have died in the ocean from natural causes. This measure is termed the "adult equivalent run size" and is a true measure of the adult production from the basin or from a particular production scenario. Adult equivalent run size is used in the Pacific Salmon Treaty process, and by the federal courts in determining allocations between Indian and non-Indian fisheries. This measure would thus more accurately reflect actual adult production and would be consistent with other coastwide approaches.
However, the other shortcomings of the previous method would apply equally to adult equivalent run size as well. It is a fairly complex process that would not properly credit the program, and would contribute little to our knowledge or ability to refine the program.
2. Analytical Methods
Observational methods, while attractive because of their intuitive appeal, suffer as measures of progress because of their limited ability to increase our knowledge of the salmon and steelhead resource. Observational indices only address stock abundance and provide only limited amounts of information to isolate program effects or to refine the program. Analytical methods build on observational methods. They attempt to increase the information content by integrating environmental indices, research results, or monitoring data into mathematical expressions that are hypotheses explaining the trends evidenced by observational data. These expressions are refined by testing through research or monitoring programs.
Analytical methods can be divided into at least two general categories that will be termed here statistical and life-cycle approaches. Statistical methods can be used to discern relationships between variables such as run size and flow during the outmiqration, number of spawners, or the relationship between year-classes. A life cycle approach, on the other hand, uses a computer model as a conceptual basis for explaining trends displayed by observational indices. The degree of relationship between the model and the observed trends is the basis for refinement of knowledge using information gleaned from directed research and monitoring programs.
Statistical methods. A variety of statistical techniques are available to examine the relationship between variables or to partition observed variation into component parts. For instance, the Oregon Production Index (OP!) uses statistical correlation to provide harvest managers with an indication of the
size of the coho population off Oregon prior to setting ocean troll fishing seasons. In this case, the abundance of adult three-year-old coho salmon off the coast of Oregon over a series of years has a statistical relationship to the abundance of two-year-old fish that returned in the previous year from the same generation. By observing the number of two-year-old fish that return in a year, predictions can be made about the expected number of three-year-old fish the next year. Because it is a statistically based method, a measure of the probability and statistical confidence of the prediction can be made.
Statistical techniques offer a suite of powerful tools that can be used to gain insight regarding the sources of observed variation in returns, for instance, or for examining relationships within a limited set of variables. In most cases, they offer the ability to determine the statistical confidence that can be associated with a statement about the relationship between variables.
One difficulty with these techniques is that it is often necessary to concentrate on some small segment of the life cycle or a single correlative relationship such as the OPI. It is generally not possible to discern statistical relationships over the whole life cycle because of the many conflicting factors that contribute to the observed population size. These techniques, therefore, may not be applicable to the evaluation of the program as a whole.
Statistical relationships also do not necessarily imply cause and effect but may only be showing a relationship between two variables that are both responding to a change in another variable. In the case of the OPI, for instance, the abundance of fish that mature at age two may have no direct impact on the number that mature at age three. Instead, it is possible that both age classes have similar survival rates in the estuary and early ocean life stage, and that this survival rate is a critical factor affecting the ultimate abundance of both age classes. Observing a statistical correlation between the two year classes provides a useful management tool but does not provide insights that would lead to improved hatchery practices, for example.
Life cycle method. In a life cycle approach, the various discrete pieces of information that exist about salmon and steelhead are organized around a model of the life cycle. This can include information obtained from observational methods as well as information on the relationships between variables gleaned from a variety of statistical methods. The life cycle approach thus incorporates information from all the previously discussed methods as well as the results of research aimed at filling in critical data gaps. The model is itself a hypothesis about how the pieces fit together which can be tested by comparing its behavior to that of the real world.
This approach permits the available information to be examined in the context of the behavior of the total system, and provides a consistent framework for comparison and evaluation of actions. As the various components of the model are refined over time through monitoring and research, the model becomes a better and better expression of reality indicating an improved understanding of the system. In contrast to many statistical techniques, the life cycle technique does not attempt to predict year-to-year changes in fish abundance. Instead, it is used to examine the effect of various actions on the long-term trend in various types of production indices.
A number of output variables can be obtained from the life-cycle method including measures of stock abundance and stock productivity. Stock abundance is the estimated number of fish present, and is analogous to the observational estimates of abundance discussed above. Stock productivity refers to the ability of the population to produce fish surplus to the number that are required to spawn and maintain a stable population size. This is an important expression of the condition of a population since it reflects the number of fish available for harvest or to buffer the effect of environmental fluctuations. The amount of the population surplus to spawning needs is also an expression of the speed with which a population will respond to changes in the environment including mitigating measures. A population with a large surplus proportion will respond quickly to improvements in the environment, whereas in the reverse, the population may respond to improvements only over a very
long time period. This ability to provide a variety of outputs and to examine the important aspect of stock productivity is a major advantage of this method.
By providing a means to probe the relationships between variables and to examine the effect of hypotheses on stock abundance and productivity, the life cycle method could contribute toward an improved understanding of the system and the refinement of the program. The life cycle approach should make it possible to isolate, within the confines of the model, the effects of the program and allows them to be examined independent of real-world variation in non-program effects.
However, the method does have drawbacks. First, it lacks the intuitive appeal of the observational methods. The product of the life-cycle approach is knowledge about the system rather than simply the number of fish. While of obvious importance, knowledge can only be converted to fish production by an effective management structure that can deal with the information. The nature of the management structure is outside the purview of MEG and this discussion.
A second drawback to the life-cycle approach is that it lacks real-time application. This approach would not yield annual predictions of effects, nor would it attempt to explain year-to-year variation in returns. Instead, it would deal with the reasons for long-term trends in returns, efficacy of types of measures, and the state of our knowledge. Compliance with the Council's policies (Section 204) could not be easily assessed by the use of a model by itself. For instance, the Council's goal that harvest rates will be controlled to support rebuilding, will require a year-to-year monitoring of harvest rates. This would be more aptly addressed by compiling information collected from existing monitoring programs such as that being conducted under the Pacific Salmon Treaty process or various state and tribal management programs.
3. MEG Recommendation
MEG examined the above methods with the goal of designing a program that would permit the effect of program measures to be isolated from other effects, and would maximize the opportunity to learn from grogram implementation. MEG reached two conclusions:
1 . No single measure of the program progress was found. Different indices address different aspects of the problem of monitoring and evaluating the fish and wildlife program.
2. No method was found that would directly (e.g., experimentally) identify the effects of the program as distinct from non-program effects. It was MEG's conclusion that program effects would have to be isolated by analytical methods such as the life-cycle approach described above.
None of the methods examined would completely address the Council policy of assessing genetic risks in production planning, although many genetic aspects could be incorporated into a life cycle model. MEG suggests addressing genetic risks in part as a topic separate from the more quantitative measures discussed so far. The recommendation from MEG is that the Council utilize a four component measure of progress that consists of:
a. A measure of annual juvenile population.
b. An estimate of annual adult equivalent production.
c. A life cycle analysis of stock productivity.
d. A program to monitor the genetic impacts of management actions.
Juvenile population would be indexed annually to provide an initial indication of the effect of the program on the salmon and steelhead production. Juvenile population would first be indexed for subbasins selected according to an overall experimental design as discussed below. A second stage would be to estimate the size of the annual outmigration, probably at Bonneville Dam, and to estimate
the survival of migrants through the system. Much of this information would be obtained from the Smolt Monitoring Program conducted by the fishery agencies and tribes. Juvenile population data would: 1 ) provide annually an indication of the effect of program actions on the size of the juvenile outmigration, especially those dealing with fish production and the survival of juvenile fish through the hydroelectric system; 2) measure the progress of the program in expediting improvement in mainstem passage survival rates (program Section 204(c)); and 3) assess the geographical distribution of production programs within the basin (program Section 204(a)).
The adult equivalent production would index stock abundance. Progress of the program would be measured within the context of other non-program effects. The measure would be calculated as the estimated adult return to the mouth of the Columbia River corrected for natural ocean mortality and ocean harvest. The latter correction factors could probably be obtained from ongoing marking and monitoring being conducted by the fishery management agencies under the Pacific Salmon Treaty. The adult equivalent production would provide an annual assessment of the overall production from the Columbia system, and, in the course of its calculation, would provide the means to monitor consistency with the Council policy regarding harvest (program Section 2Q4(e)).
The life cycle analysis would increase our knowledge of the causes behind the trends seen in the previous two indices. A model would be used to organize information, identify and prioritize needed information, and compare alternative methods of achieving goals. As a means of measuring program progress, the model would be used to calculate the stock productivity.
MEG suggests using the model to determine the change in stock productivity that occurs as a result of program measures and for the program as a whole. In many ways the stock productivity is a more useful expression of stock condition than is stock abundance (e.g., the adult production). Stock abundance might be viewed as the easily observed tip of the iceberg while stock productivity is the more obscure but potentially more important underlying ice. Stock productivity provides insight into the capacity of the population to withstand harvest pressure, mainstem passage mortality rates, and environmental fluctuation while indicating how fast the population might respond to program measures. None of this can be found from simple expressions of stock abundance.
Genetics monitoring. In addition to measuring the progress of the fish and wildlife program, SMEP is charged with addressing means to assess genetic risks in production and to meaningfully incorporate genetics into production planning (production policy b and Section 206(d) (F)). MEG is responding to this through the System Planning process by providing guidelines for genetic impact assessment and a genetic conservation program. The latter will include measures to monitor genetic changes resulting from production programs and genetic research requirements.
III. ELEMENTS OF THE MEG RECOMMENDATION
The process for monitoring and evaluation proposed by MEG consists of collecting, organizing, and retrieving information about the biological system so that progress is followed and our increase in knowledge is maximized. Collection of information would occur through monitoring and research programs developed around an experimental design that seeks to maximize the cost effective collection of data. Organization of this information in a form that maximizes our learning and understanding would occur through the use of a life cycle model. This would summarize our understanding of the logical relationships between the collected data. Retrieval and reporting of the data collected through monitoring and research, and the knowledge that emerges from structuring this information, would occur through an information system that would also serve to coordinate data collection and facilitate communication between data bases. The genetics monitoring program forms an additional component of the SMEP which may not be fully encompassed by the other, more quantitative, aspects of the program. Each of these components will be discussed below. Some of
these components have been or are being implemented, primarily as a part of System Planning. Most of these elements are relevant to any of the measures of progress discussed in this report.
1. Experimental Design
The precision with which SMEP will monitor program progress will depend on the amount of information that is collected. Extremely intensive monitoring of all aspects of the salmonid life cycle in all subbasins would produce an extremely precise measure of program progress. However, the cost of such a program would likely be prohibitive. Thus a balance must be struck between the degree of precision desired, and the cost of obtaining the required information.
Because the desire would be to obtain the highest precision possible for the available funds, it is not possible at this time to set the degree of precision, or, therefore, to set the cost of the overall monitoring program. Some of the information pertinent to SMEP could be obtained from existing and planned monitoring programs the cost of which is known. However, much of the effort would be devoted to the monitoring of subbasin plans and from research to fill in specific data gaps. The number of new monitoring programs that will be needed to test the production hypotheses made in system planning cannot be known until the subbasin plans are available.
As subbasin plans are implemented, the monitoring program could begin to be focused by selecting a number of stocks and subbasins for intensive monitoring efforts. From a technical standpoint, the monitoring sites would be selected according to a preset experimental design. This design would consist of a stratification or categorization of information initially according to whether it is of a system-wide or subbasin nature. Subbasin data would be further categorized by habitat, biological, and management criteria.
System information refers to parameters such as harvest rates and mainstem passage rates that occur, for the most part, outside the subbasins. Subbasin data, on the other hand, is a function of environmental and biological characteristics specific to a subbasin. Subbasin data is divided into categories of information pertaining to production type, e.g., natural and hatchery production. This includes egg and smolt carrying capacities, juvenile survival rates and hatchery management scenarios.
Classification of habitat and watershed types will be done as a part of the design of the Coordinated Information System which will begin in fall of 1988. This will be discussed more fully below. Indicator stocks and subbasins would be chosen from each of the blocks defined by this categorization. These would be the focus of more intensive monitoring efforts to identify the efficacy of specific types of actions, and to provide needed life cycle parameters. This classification of the basin could also be used to guide the System Planning process in regard to selection of control subbasins and stocks and the adaptive implementation of subbasin plans.
Pertinent types of information that would be used to generate the components of the MEG recommendation are shown below. These are organized according to whether they are collected at the system or the subbasin level.
I. Juvenile information by species
a. System level by stock.
1 . Population size at Bonneville.
2. Passage survival rate.
3. Individual passage survival parameters.
b. Subbasin level by production type.
1 . Population size out of the subbasin.
2. Survival rates.
3. Carrying capacities.
4. Results of particular experiments and production programs.
II. Adult information by species
a. System level by stock.
1 . Adult returns to mouth of Columbia River or to Bonneville Dam.
2. Prior harvest.
3. Natural ocean survival rates.
b. Subbasin level by production type.
1 . Returns to the subbasin.
2. Terminal harvest rates.
Ill- Production potential by basin, species, stock, or measure
a. System level by stock.
1 . Ocean and river harvest rates.
2. Mainstem passage parameters.
3. Natural ocean survival rate.
b. Subbasin level by production type.
1 . Egg and smolt carrying capacities.
2. Survival rates.
3. Hatchery strategy.
4. Fecundity.
Some of this information could be obtained from existing sources. For instance, many of the juvenile passage parameters could come from the Smolt Monitoring Program presently operated by the fishery agencies and tribes. Similarly, the ocean harvest and survival rates could be obtained in whole or in part from marking and evaluation programs conducted by the fishery management agencies, especially those programs pertaining to the Pacific Salmon Treaty. Programs such as the Yakima-Klickitat Production Project and the Idaho habitat evaluation project would contribute to SMEP and would need to be coordinated with the CIS and the experimental design. Subbasin information is now being compiled through the System Planning process that will form the baseline for monitoring the implementation of production measures.
However, it can be anticipated that considerable additional information will be required to monitor the fish and wildlife program at an acceptable level of precision. Much of this would come from the monitoring of specific subbasin hypotheses generated as part of the subbasin plans being prepared through System Planning. Subbasin plans will include the identification of critical uncertainties and plans to monitor the progress of the plan and resolution of the uncertainties. SMEP would use the experimental design to organize, prioritize, and focus the numerous monitoring proposals that will arise
-10-
from System Planning, as well as those monitoring programs that are now underway or are being contemplated as part of other aspects of the fish and wildlife program.
It is also likely that SMEP and the system wide analysis of information proposed here would lead to insights that could affect the direction and prioritization of information needs in these programs or require additional effort outside the present capabilities. Examples of this are the need listed above for a juvenile population estimate at Bonneville Dam and the juvenile survival rate through the hydroelectric system. This need is also recognized by many other agencies and programs in the basin. Although the means to obtain this information are not presently known, the need remains and could spur development of the required techniques.
2. Life Cycle Model
Under the monitoring and evaluation scheme proposed here, a model would be used to organize the information collected from existing sources and from future monitoring and research activities into a logical and coherent picture of the life cycle of salmon and steelhead. MEG is proposing to use the System Planning Model as an initial tool to structure the existing data. This model is now being used in System Planning to compare production scenarios and eventually as a technical component of the integration of the subbasin plans into a system plan. For SMEP, the model would be used to compute the stock productivity of projects, subbasins, and ultimately the system.
The computation of stock productivity would occur at three points during project planning and implementation. First, it would be used to assist in the evaluation of production and management alternatives. This is now being done in System Planning, where the model is used to compare the increase in run size and productivity that can be expected from various alternatives given assumed harvest, passage, and natural survival rates. The model will also provide the technical basis for the system integration of the subbasin plans (Section 206(d) (E)). This will entail a process of balancing the productivity of the various stocks with the prevailing harvest and passage survival rates.
A second application of the model should be the identification of uncertainties that affect the achievement of goals. During the planning of production or management actions, much of the model input data will consist of assumptions and hypotheses, and the simulations will contain considerable uncertainty. Parameters identified as critically affecting the outcome of plans should be flagged as areas requiring special research or monitoring attention. The importance of these area should be communicated to the research technical work groups, if they fall within the existing research priorities, and to the Council if alternative or additional areas research emphasis are indicated.
Finally, after implementation of a set of actions, the model would be used in SMEP to compute a measure of progress of the program in doubling the capacity of the basin to produce salmon and steelhead. The simulations made during the planning would be updated over time as research indicates that modifications are warranted in the input data. These simulations would provide a baseline for measurement of progress. After implementation, simulations would continue and incorporate monitoring and research data. Progress of the program would then be measured by comparing these simulations to the baseline simulations. The process proposed here is thus becomes a continuous cycle of updating simulations made during the planning phase, to form a baseline, with real-time monitoring and research data as the plan is implemented.
3. Coordinated Information System
A program to monitor and evaluate a system as complex as the Columbia River will necessarily involve the coordination of large amounts of information. To a large degree, the monitoring and evaluation program proposed here can be characterized as a system to collect, organize, and evaluate information collected at all life stages, and to make this information available to decision makers in a
-11-
timely and effective manner. This will include information gathered at the system level, such as harvest rates, recovery of tagged fish in the fisheries, and the survival and abundance of juvenile and adult fish through the hydroelectric system; as well as subbasin data such as numbers of spawners, subbasin carrying capacities, juvenile survival rates, and hatchery and natural production levels. It will also depend heavily on research results that clarify the relationships between variables and the results of different management actions.
The Coordinated Information System (CIS), called for in Section 206{d)(C) of the program, will be used to bring together the data and information needed for SMEP. The CIS is a system that will facilitate exchange and dissemination of data within the Columbia Basin. Given the multi-jurisdictional nature of the basin, the need for such a system has been recognized for some time, and was one of the primary recommendations to emerge from the regional planning under the Enhancement Act.
MEG has prepared a workplan for the design of a Coordinated Information System that is expected to implemented in the fall of 1988. The workplan calls for the design and scoping of a CIS and the completion of demonstration projects in the John Day, Umatilla, Yakima, and Salmon River subbasins. As note above, the workplan also calls for the development of techniques for classification of habitat and watersheds which is relevant to the basin-wide experimental design of SMEP and the organization of the CIS. Finally, the workplan provides for the updating of the Stock Assessment Study as a periodic output from the CIS.
The CIS will consist primarily of standards and protocols that will enhance the communication between the various data bases that reside on systems maintained probably by the agencies responsible for collecting the data. Creation of the CIS originates with the system wide data collection conducted as part of System Planning. It will incorporate the natural and hatchery data bases maintained by the Council (Section 206 (e)(1) and (2)). The standards and protocols will address the following features of data collection:
• Quality and documentation of data. The effectiveness of the evaluation process in assisting decision making depends upon the quality of the data collection and on the analyses performed. Confidence in the SMEP requires accountability through documentation of data sources and procedures.
• Consistency in data collection. Consistencies in data collection, sampling and survey methods, units of measure, and consistency in the definitions of sampling units (time-area, fisheries, production types etc.) are needed to allow comparability between subbasins and projects throughout the Columbia Basin. Consistency is also necessary over time, since a primary function of SMEP is to evaluate change in production over time.
• Timeliness and quality of data reporting. Communication of data, analytical results and methods in a timely and clearly understandable way is necessary to perform the monitoring and evaluation steps on time and to convey results in a way that promotes trust. Communication of the SMEP process is an important step toward integrated system planning and research.
Genetics Monitoring
Despite the almost universal recognition of genetics as a factor contributing to the success of production programs, meaningful incorporation of genetic concerns into production planning has proven difficult. This stems from a lack of clear genetic objectives, the degree of uncertainty regarding how to incorporate genetics into production planning, and because of the gulf that separates the academic study of genetics from the practicalities of fisheries management. The genetics portion of SMEP deals with these last two problems to provide the tools to assist in the formation of genetic objectives.
-12-
The specifics of the SMEP genetics tasks are being developed by MEG as part of System Planning. To date, MEG has drafted two products designed to help subbasin planners assess possible genetic impacts of proposed actions. These are the Genetics Evaluation Tree (GET), which is a key to identify relative qualitative impacts of different actions, and the Genetics Evaluation Model (GEM), a computer model designed to determine a quantitative index of genetic impacts. These tools will be brought together in a procedure for identifying genetic impacts, and submitted to the System Planning Group on September 1, 1988.
The complete genetic conservation program will include the genetic impact assessment tools described above as well as: 1) technical documentation of the process; 2) identification of critical areas of uncertainty and needed research; and 3) identification of techniques for testing genetic hypotheses generated in system planning. As part of the System Planning Process, MEG will continue development of the Genetics Conservation Program. Input will be solicited through the Genetics Subcommittee of MEG and through independent peer review of program elements.
Conclusion
The intent of MEG's discussions to date and this paper has been to provide the Council with a range of options for measuring the progress of the fish and wildlife program, and to make recommendations regarding the elements that would make up a system monitoring and evaluation program. Further development of the program will require a resolution of the question of a measure of progress. Considerable work remains, and most would have to be completed by the end of System Planning. Because of the likely scope of SMEP, it is also important that a stable implementing mechanism and funding source be established.
mvax::larTy:[cm]chip smep discussion paper
-13-
Membership Monitoring and Evaluation Group
September 23, 1988
Name
Ron Boyce Mark Chilcote Tom Cooney Stan Detering Larry Everson Kathryn Kostow Chip McConnaha Lars Mobrand Phil Roger
Affiliation
Oregon Department of Fish and Wildlife
Oregon Department of Fish and Wildlife
Washington Department of Fisheries
Bonneville Power Administration
Bonneville Power Administration
Pacific Northwest Utilities Conference Committee
Northwest Power Planning Council
Consultant
Columbia River Inter-Tribal Fish Commission
fe"^-.vy-^
«
I
I
m.
';i:;'' _,'*.'
■' : J2l
.■ ;;;/■ >:^).^,^
■■::^>^-y'^
j>!i^',-.V;
''^^¥^i
■.-•'.
■ r :
!»fci'^i
■\ -"<;'. 1 a
'■'-I/
-,-?.-^.
^r":"-
1 »•■
|
■•1.: |
, V |
',■ |
||
|
:(u:^ |
• ;')'■ |
/ |
||
|
. i.-.>. ■ |
||||
|
: • .■ .v ■' - |
||||
|
■ v^>''^^■ |
; ' ' |
|||
|
"■'■ ■•'7i*" . |
■■■.t |
.v^ |
, ' |
|
|
M • |
*','>- ' |
|||
|
■ |
1 |
b |
r'^ |
-^ |