J

333.956
N17SS

SSUE PAPER

• S^-mON AND STEELHE AD
RESEARCH AND
MONITORING

88-23

NORTHWEST POWER PLANNING COUNCIL

September 21, 1988

Montana Slats Library

3 0864 1003 1165 6

MORRIS L BRUSETT

CHAIRMAN

Montana

George Turman
Montana

;red HaUock
Oregon

Norma Paulus
Oregon

NORTHWEST POWER PLANNING COUNCIL

851 S.W. SIXTH AVENUE • SUITE 1100
PORTLAND, OREGON 97204-1348 • (503) 222-5161

Toll free number for Idaho, Montana & Washington; 1-800-222-3355
Toll free number for Oregon: 1-800-452-2324

TOM TRULQVE

VICE CHAIRMAN

Washington

R, Ted Botoger
Washington

James A GoUer
Idaho

Robert (Bob J Saxvik
Idaho

Northwest Power Planning Council
Staff Issue Paper-
Salmon and Steelhead Research and Monitoring

September 21. 1988

Many important unceriainties remain about the biO'Dgy of Columbia River Basm salmon and
steelhead and the success of the region s efforts to protect ana enhance them. Research and
monitenng should provide information that will improve our understanding of this important resource.
But the life cycle of these fish is complex. They migrate between river and marine environments, and
are affected by a variety of human development activities withm these environments. They travel
through many jurisdictional boundaries, and many entities are responsible for managing them. In
addition, numerous parties have been interested m conducting research on a wide variety of topics
related to salmon and steelhead. As a result, the development of a basinwide research and monitoring
program has not been easy.

The significance of problems in research and monitoring and the urgency to find solutions were
highlighted recently at the Council's Salmon and Steelhead Round Table, held m late June 1988.
Although the Round Table was intended to be a general evaluation of the fish and wildlife program
during the past five years, speaker after speaker focused on the need to solve research and monitoring
problems. For example, Tim Wapato, executive director of the Columbia River Inter-Tribal Fish
Commission, stated, ' Research and monitoring continue to operate m a very fragmented fashion,
which results m technical and policy disputes. ... A process is needed to address such complex
issues. ' Other participants echoed his call for coordinated research planning and monitoring. In
developing this issue paper. Council staff has found that all major parties involved m salmon and
steelhead research believe these issues must be addressed m the near future.

Therefore, the Council seeks comment on the following questions

1. What mechanism should be employed to provide policy guidance on research and monitoring
questions, such as management needs and priorities, overall levels of research effort, and dispute
resolution"^

2. How can cooperation m planning Corps of Engineers-funded research be improved while
recognizing the Corps' authority and responsibility for its research program''

3. How might the technical quality of research and monitoring be unproved''

4. What method should be used to measure fish and wildlife program progress toward the doubling
goaP

5. Who should fund the effort to monitor and evaluate the progress of the fish and wildlife program?

6. How can communication of research and monitoring results to resource managers and the
interested public be improved?

The six issues identified for public comment are critical links in ensuring that necessary
Information is developed and made available to refine the fish and wildlife program and to help achieve
program goals. Their resolution should help in carrying out the Council's policy of adaptive
management, which is intended to permit the pursuit of an action-oriented program in the face of
considerable biological uncertainties. Adaptive management requires an assessment of the
effectiveness of actions, the resolution of critical uncertainties, and an effective forum for
communicating research and monitoring results to decision makers and other interested parties.

An additional problem that has been identified frequently by all parties in the basin is the
formation of new groups to solve problems, a situation that often taxes available staff. Any solutions to
research and monitoring problems need to make the most efficient use of limited staff time basinwide.

The purpose of this issue paper is to highlight the issues, identify potential options for solving the
problems, and seek public comment on the options. Accordingly, the issue paper first describes the
background of research and monitoring in the Columbia River Basin, how research and monitoring are
provided for in the fish and wildlife program, and the problems in research and monitoring that still
need to be solved. New developments that may contribute to solving these problems are described.
Finally, the issue paper discusses the six issues in research and monitoring that require resolution, and
describes options for resolving them.

Research planning and monitoring in the complicated jurisdictional environment of salmon and
steelhead management are not simple matters. As a result, we have made no attempt to advocate a
specific solution until public comment can be reviewed. The ultimate goal is to develop a cohesive
research and monitoring process that addresses the identified concerns. Alternatives identified are not
necessarily mutually exclusive, and commentors should feel free to propose additional alternatives as
appropriate. We have developed this issue paper in close consultation with representatives of the
Columbia Basin Fish and Wildlife Authority, Bonneville Power Administration, the Corps of Engineers,
the Pacific Northwest Utilities Conference Committee, and other entities interested in salmon and
steelhead research and monitoring.

The schedule for presentation and comment on the issue paper is as follows:

September 14-15 Staff presentation of issue paper at Council meeting in Boise

October 12-13 Public comment at Council meeting in Missoula

November 9-10 Public comment at Council meeting in Spokane

December 14-15 Council action at Portland Council meeting

Consultations will be held on request with interested parties during October and November.
Public comment will be received in the Council's Portland office until 5 p.m., November 18, 1988.

Council action could result in a decision to enter rulemaking on a proposed fish and wildlife
program amendment, which would involve another period of hearings in each state and public
comment.

For further information or copies of reports mentioned in the issue paper, contact Judy Allender,
Northwest Power Planning Council, 851 S.W. 6th, Suite 1 100, Portland, Oregon 97204, 503-222-5161 ;'
1-800-452-2324 (Oregon toll-free number); or 1-800-222-3355 (regional toll-free number).

BACKGROUND

Research, monitoring, and evaluation are all closely related activities aimed at increasing our
understanding of the biology of salmon and steelhead. Ideally, research tests specific hypotheses that
could explain a set of observations. Monitoring is the routine collection of information for the purpose
of following the progress of actions or detecting changes from "normal" conditions. Evaluation is the
use of monitoring or research information in an effort to determine the effectiveness of particular
actions. These distinctions are not always clear in practice. The planning and collection of information
for research, monitoring and evaluation frequently overlap.

Research

Research is a major component of fisheries activities in the Columbia River Basin. Salmon and
steelhead research encompasses a wide diversity of subjects, reflecting the complexity of salmon and
steelhead life cycles and their environment. Research subjects include habitat requirements, artificial
production methods, supplementation (introduction of artificially-produced fish to augment natural
production), disease, physiology, migration, passage at dams and through reservoirs, stock
Identification, population dynamics, harvest management and genetics. This information is needed to
refine and implement the fish and wildlife program, and to manage the salmon and steelhead resource.

About $12 to $16 million per year is spent on research in the Columbia River Basin. Bonneville
funds about half of this effort, or $7 to $8 million per year, under the fish and wildlife program
Bonneville expenditures for research have ranged from about one-quarter to one-third of Bonneville's
annual fish and wildlife program expenditures. Bonneville's total program expenditures could increase
to about $50 million per year in the 1990's, with about $13 to $16 million potentially available for
research. Figures for program expenditures in the 1990's are for planning purposes only, and could
change as new information becomes available.

In addition, the U.S. Army Corps of Engineers spends about $2 million per year on salmon and
steelhead research in the basin. Three mid-Columbia Public Utility Districts, the National Marine
Fisheries Service, the U.S. Fish and Wildlife Service, and other sources contribute up to $6 million per
year. "^ ^

Related to this in-basin research are extensive salmon and steelhead research programs outside
the Columbia River Basin. The largest of these are programs conducted by the Canada Department of
Fisheries and Oceans, University of British Columbia, and Alaska Department of Fish and Game
Extensive out-of-basin research is conducted on hatchery effectiveness, disease, supplementation,
genetics, and harvest management.

There are good examples of research results that have clearly benefited the fisheries resource in
the Columbia River Basin. Some notable highlights include:

1. The development of the Oregon Moist Pellet diet in the 1960s allowed consistent and convenient
hatchery fish feeding without exposing fish to diseases previously present in fish foods.

2. New techniques have appreciably increased the success of diagnosing fish diseases. The
capability to treat diseases through antibiotics and to prevent them through segregation and
improved hatchery practices also has improved.

3. The development of the coded-wire tag allowed monitoring of specific groups of fish and has
been an important tool in harvest management.

4. An internal fish tag termed the PIT (Passive Integrated Transponder) tag allows marl<ing of
individual fish and offers considerable promise for mainstem passage and other research.

5. Mainstem dam operations have been changed in response to research results. For example,
research led to changed spill procedures that reduced nitrogen supersaturation (gas bubble
disease).

6. Size limits for upriver sturgeon harvest recently were changed as a result of research.

Monitoring

Monitoring is also an important component of fisheries management in the Columbia River Basin.
The smolt monitoring program conducted by the fishery agencies and tribes' Fish Passage Center has
been an important part of the fish and wildlife program water budget provisions for many years.
Information from this program is used by the agencies and tribes annually to manage the water budget
and other fish passage actions at mainstem projects. Bonneville spends some $1 to $2 million each
year for mainstem smolt monitoring. Bonneville has also funded evaluation studies on the efficacy of
fish and wildlife program habitat improvement projects in Idaho and Washington.

The management agencies conduct extensive monitoring outside the fish and wildlife program.
For example, the Corps monitors the passage of adult fish past each of its projects, while the mid-
Columbia Public Utility Districts conduct similar monitoring at some of their projects. The fishery
agencies and tribes conduct a large scale program to monitor catch, escapements, and other fisheries
data coastwide, primarily for harvest and other management decisions.

RESEARCH AND MONITORING IN THE FISH AND WILDLIFE PROGRAM

The 1987 Columbia River Basin Fish and Wildlife Program outlined a salmon and steelhead
research and monitoring program (Section 206). The Council's objective was to end research
fragmentation by establishing overall areas of emphasis, to provide a process for long-term research
planning, and to improve the participation of interested parties in research planning.

For research funded by Bonneville, the program identified four areas of emphasis: studying
reservoir mortality and water budget effectiveness, solving disease problems affecting spring and
summer Chinook, increasing the effectiveness of hatchery production, and improving supplementation

-4-

#

techniques. The program established a research technical work group for each area of emphasis,
composed of representatives of the fisheries agencies and tribes, Bonneville, the Corps, utilities, the
Council and others. The technical work groups were given the responsibility to develop five-year
research work plans, to assist Bonneville in developing, evaluating, and reviewing requests for
proposals and project work statements, and to develop statistical design standards. The research work
plans developed by the technical work groups were to be submitted to the Council for approval as a
package.

In addition, the program established two areas of emphasis for research funded by the Corps:
improving bypass at mainstem projects, and evaluating and improving the effectiveness of
transportation. The program identified the Corps' Fish Passage Development and Evaluation Program
as the mechanism for developing research work plans in these two areas of emphasis. The research
planning process is advised by the Technical Coordinating Committee, which is composed of
representatives of the state and federal fisheries agencies, Indian tribes, and the Corps. The
recommendations of the Technical Coordinating Committee are submitted to the Corps' Fish
Management Committee-the Corps' policy-level committee-for final approval.

The 1987 program also committed the Council to develop a system monitoring and evaluation
program for measuring program progress toward the goal of doubling salmon and steelhead runs, to
monitor consistency of actions with program policies, and to help unify data collection efforts within the
basin. To help develop and implement the system monitoring and evaluation program, the Monitoring
and Evaluation Group was established. It is composed of technical staff from the fisheries agencies
and tribes, Bonneville, the Pacific Northwest Utilities Conference Committee, and private consultants.

The program states that the system monitoring and evaluation program should include:

1 . Development of alternative means to assess program progress and consistency with program
policies;

2. evaluation of research results and implications for achieving program objectives;

3. development of a coordinated information system;

4. maintenance of the system planning model;

5. assistance in the integration of subbasin plans in system planning; and

6. development of methods to incorporate genetic conservation into system planning.

PROBLEMS IN RESEARCH AND MONITORING

The following have been raised frequently as the major concerns about current approaches to
salmon and steelhead research and monitoring.

1.

Research planning and implementation lack a clear mechanism for policy guidance.

There is no established mechanism for ensuring that research is relevant to management
problems and that overall levels of research effort are appropriate to management needs; nor is there a

defined procedure for referring research-related disputes to the policy level for resolution. This has
caused many problems in implementing the program's research planning process in the past year and
a half.

The major problems occurred in mainstem research areas, where policy conflicts have been most
intense. For example, the lack of policy guidance contributed to the development of substantial and
unresolved controversy in planning Bonneville-funded research on reservoir mortality and water budget
effectiveness. Without a forum for resolution of these disputes, interested parties developed two
competing research work plans for research in this area.

There have also been problems in achieving a coordinated program of Bonneville-funded and
Corps-funded mainstem research, because a policy forum has not been available to develop overall
mainstem management needs. As a result, some important research areas may be omitted, and there
may not be adequate coordination of mainstem research funded by the two agencies. In short, the
interested parties have not agreed on what needs to be learned and how it should be studied.

In addition, some are concerned that the Council's efforts to focus research through the areas of
emphasis may exclude some important research topics such as evaluation of habitat improvements
and survival of fish in the estuary and ocean. Some research spans two or more technical work groups
and is not easily developed in the current planning framework. In the absence of a mechanism for
policy guidance, work on some important topics that overlap areas of emphasis might not be pursued.
For example, the role of hatchery practices in effects of supplementation is not clearly the purview of
either the hatchery effectiveness or supplementation work groups. Research that addresses system-
wide problems also may not be addressed by any one technical work group. Each of these
deficiencies might be corrected with effective and continuing policy guidance.

2. Cooperation and coordination in planning Corps-funded research need to be improved.

In addition to the need to coordinate Bonneville-funded and Corps-funded research, disputes
have arisen within the Corps research process because the fisheries agencies and tribes believe that
their views have not been adequately represented or taken into account in the Corps decision-making
structure. Some means to improve cooperation in Corps research planning may be needed.

3. Technical quality of research and monitorinq needs to be improved.

Some have argued that existing mechanisms may be inadequate to assure technical quality of
research activities and products, including coordination of effort, development of statistical standards,
and scientific review.

Coordination. Coordination should help avoid duplication of past and ongoing efforts, should
enable researchers and resource managers to take advantage of opportunities to share resources,
should provide for study designs that are complementary as appropriate, and should help assure that a
systemwide research perspective is maintained.

As an example, coordination of projects within and between the six areas of research emphasis,
and with hatchery production and supplementation programs, needs to occur. Currently, coordination
does occur in an ad hoc fashion, but these efforts are limited by staffing and time limitations.

Coordination also needs to occur with other research programs in the basin, such as those
associated with the Lower Snake River Compensation Plan hatcheries, Mitchell Act hatcheries, other
hatchery programs, universities, utilities, and other research efforts. Opportunities may also exist for
coordination with research efforts outside the basin, such as western Oregon and Washington, Alaska,
Canada, and perhaps even farther afield.

Statistical standards. An important aspect of technical quality in research and monitoring is the
use of sound statistical standards and experimental designs. Statistical standards provide a measure
of objectivity in hypothesis testing, while a sound experimental design ensures that the research or
monitoring program will adequately address the issue at hand. Flaws in the experimental design or
failure to adopt adequate statistical standards can result in wasted effort and research that creates
additional controversy instead of helping answer important questions. There is a need to provide
guidance to researchers regarding the level of statistical precision required for different types of
research and a process to review proposals for adequacy of experimental design.

Scientific review. The need also exists for peer review of research proposals and reports. Inviting
unbiased scrutiny by recognized experts could improve the quality of research projects and the
interpretation of results. It could help ensure that proposed research actually is needed and will
contribute to answering major management questions. It could also decrease the appearance of
conflict of interest that has been raised by some who are concerned when research planning is done
by those whose organizations stand to obtain contracts as a result of the research plans.

"^^ It has not yet been determined how progress of the fish and wildlife program in achieving
program goals should be measured.

The Council's fish and wildlife program represents a considerable investment on the part of the
Council, fisheries managers, the implementing agencies, the region's ratepayers, and other interested
parties. To date, there has been no systematic method available to determine the progress of the
program to "protect, mitigate, and enhance" the fish and wildlife resource. The 1987 Fish and Wildlife
Program provided the foundation for establishing a system monitoring and evaluation program to fill
this gap. The system monitoring and evaluation program was intended to bring together research and
monitoring results into a measure of progress toward the salmon and steelhead doubling goal. The
units by which such progress should be measured-whether using smolts, adult fish, production
surplus to spawning needs, or other method-need to be determined. The Monitoring and Evaluation
Group has developed a series of recommendations for this effort.

^- There is a need to Identify who should fund the effort to monitor and evaluate the effectiveness of
the fish and wildlife program. ' " ~ "~

As noted above, the 1987 Fish and Wildlife Program called for the development of proposed
alternatives for a system monitoring and evaluation program. However, the program did not specify
who should fund that effort and who should be involved in it. This issue needs to be addressed before
the system monitoring and evaluation program can be fully implemented.

6- Communication of research and monitoring results needs to be improved.

There is no specific mechanism for summarizing and communicating research and monitoring
results to decision makers and interested parties so that results can be used to improve management,
mform policy decisions, and guide the adjustment of current actions and goals. Because

communication mechanisms are inadequate, there is the potential for duplication of past and ongoing
research efforts. Many research and monitoring programs exist, but results are reported in varying
formats at various times of the year. Often information is kept in files or Is reported In the "grey"
literature that receives limited circulation. There is no central process for synthesizing the results of
research into a form that is readily available to decision makers and interested parties. Existing
communication mechanisms are not always effective for all relevant parties. As a result, Information
gained from research and monitoring may not be contributing as effectively as it might to the
achievement of program goals.

NEW DEVELOPMENTS IN RESEARCH AND MONITORING

During the past year, three major developments that could affect planning and Implementation of
research and monitoring have occurred. These are- ongoing negotiation of a modified research
planning process for mainstem passage research; agreement between Bonneville and the Columbia
Basin Fish and Wildlife Authority on a process for implementing fish and wildlife program measures;
and development by the Monitoring and Evaluation Group of alternative methods for measuring
program progress. Each Is discussed briefly below.

"■ • Mainstem passage research planning process. As this issue paper goes to press, a long-term
spill agreement Is being negotiated by the fisheries agencies and tribes, Bonneville, and the
Pacific Northwest Utilities Conference Committee and others. This agreement may Include a
research planning process for mainstem research. A four-stage process has been discussed. It
includes the identification of research problems at the policy level, and scientific review by a panel
of technical experts. The process is intended to address all research having to do with mainstem
passage and hydroelectric project operations in the Columbia River Basin. It Includes the
identification of major management research needs and the approval of an annual research plan.
Until this process is put in place, a variety of ad hoc efforts have kept mainstem research and
monitoring moving forward. If the process Is ultimately not incorporated in the long-term spill
agreement, it is likely to be recommended as a modification of the Council's current research
planning process.

2. Implementation Planning Process. Bonneville and the Columbia Basin Fish and Wildlife Authority
are in final stages of lengthy negotiations on a process designed to formalize implementation of
the fish and wildlife program. The goal of the process is to ensure close collaboration on
program implementation between Bonneville, the fisheries agencies and tribes, the Council, and
other parties. As is the case with the research planning process being negotiated under the spill
agreement, the Implementation Planning Process provides for the formation of a policy group
(Program Policy Review Group) to help set implementation priorities and Identify funding levels
for major categories of expenditures. It creates a Scientific Review Group to review research
proposals, evaluate Individual projects, and monitor the effectiveness of the fish and wildlife
program. The Implementation Planning Process also creates several technical working groups to
aid in project development and proposal review. The Scientific Review Group and technical
working groups will review technical quality, schedules and long-term costs of proposed projects
and will make recommendations to the Policy Review Group. The text of the Implementation
Planning Process will be available in the near future.

A common thread in both of the above is an effort to separate policy and technical issues. This
separation is viewed as useful because policy and technical issues frequently have become confused
in the recent past, and the result has been that technical staff have been unable to pursue technical
solutions to research issues. The intent of this structure is to allow the technical group to focus on
technical issues by providing an explicit policy group to handle policy problems and mediate disputes.

3. Proposal for a System Monitoring and Evaluation Program. The 1987 Fish and Wildlife Program
called for establishment of a Monitoring and Evaluation Group to assist in the design and
implementation of a plan to monitor and evaluate progress toward the doubling goal and
consistency with policies set forth in the program (Section 206(d)). The group has responded by
examining alternatives and developing recommendations for the design of a system monitoring
and evaluation program. A report describing their conclusions is attached. The group has also
proceeded with development of several elements of the monitoring program described in the
1987 Fish and Wildlife Program, including a coordinated information system and a program to
identify genetic impacts in production planning. A coordinated information system is intended to
facilitate communication of research and monitoring data between the various agencies. It is also
intended to make this information more accessible to all interested parties in a usable form.

The Monitoring and Evaluation Group identified several alternative approaches to measuring the
progress of the fish and wildlife program. These are discussed below. The group also reached two
conclusions: First, there was no single measure (smolt counts, adult counts or surplus production)
that would totally reflect program progress. Each of the approaches they identified measures the
progress of different aspects of the program. Second, the group felt that it was not possible to directly
separate program effects from non-program effects such as changes in harvest rates or natural survival
rates. This was because of the many conflicting factors that interact to determine the number of adult
or juvenile fish produced from the Columbia River. The group concluded that it was possible to
separate program effects from non-program effects only by using a computer model to simulate the
salmon and steelhead life cycle.

ISSUES AND OPTIONS

As a result of the 1987 Fish and Wildlife Program provisions, a variety of ad hoc discussions, and
the three initiatives described above, progress is being made in solving problems in research and
monitoring. However, as noted above, a number of problems remain. With the problems of the past
few years in mind, this is an opportune time to review the issues and decide whether more can be done
to solve the remaining problems. Accordingly, the Council seeks comment on the following major
research and monitoring questions and the options for resolving them. Commentors are especially
urged to consider and provide comment on:

• The likely effectiveness of the options and opportunities for streamlining any proposed processes;

• the efficiency of the options in terms of relative levels of personnel time and attendant overhead
costs; and

• appropriate levels of expenditure that may be required to implement the options.

The options are not necessarily mutually exclusive and may be considered in any appropriate
combinations and variations. Commentors are also encouraged to propose any additional alternatives.

1. What mechanism should be employed to provide policy guidance on research and monitoring,
such as management needs and priorities, overall levels of research effort, and dispute
resolution?

Recent problems in research and monitoring have pointed to a need for early policy guidance so
that resource managers can clearly identify the research questions they need to have answered and
indicate the priority they place on the questions. Policy guidance also is needed to help deal with
research-related disputes that arise.

There are several senior level policy groups currently working on different aspects of the fish and
wildlife program. The System Planning Oversight Committee and the System Planning Forum were
formed by the Council to address policy issues arising in system planning. The Mainstem Executive
Committee was formed by the Council, the fishery agencies and tribes, Bonneville, the Pacific
Northwest Utilities Conference Committee, and the Corps to address policy issues encountered in
managing spill, water budget, and other aspects of mainstem passage. Under the Implementation
Planning Process, a Program Policy Review Group would be formed. It would be a senior staff working
group and would include somewhat fewer than the number of entities represented on the Mainstem
Executive Committee and System Planning Oversight Committee. For example, as currently
structured, its membership would not include representatives of fishing or conservation groups, or land
and water managers.

Two options have been identified to respond to the need for policy guidance and dispute
resolution on research issues:

Option A. Rely on the Implementation Planning Process to improve the current situation. This
process creates a Program Policy Review Group, composed of Bonneville, the fisheries agencies and
tribes, the Corps of Engineers and the Council. The Program Policy Review Group is slated to provide
policy direction and an arena for resolving program implementation disputes. The group will assist
Bonneville in developing an annual implementation work plan. The fish and wildlife program provides
for Council review of the annual work plan once it is submitted by Bonneville.

Option B. Establish a single cooperative research and monitoring policy group. A policy group
composed of senior-level staff could be formed to deal with Bonneville-funded and Corps-funded
implementation. The group could augment the membership of the Implementation Planning Process
Program Policy Review Group with some members of the System Planning Oversight Committee. The
System Planning Oversight Committee has a membership that is somewhat broader than the Program
Policy Review Group in that its membership includes sport fishing interests and land and water
managers. The group could establish subcommittees to carry out specific functions now assigned to
the individual policy groups. The group could reflect the interests of all entities involved and would
follow the fish and wildlife program. With respect to research and monitoring the group could: 1)
identify and prioritize systemwide management needs; 2) determine priorities based on identified
management needs; 3) resolve disputes or provide mechanisms for doing so; and 4) oversee research
and monitoring actions. The policy group could be supplemented by a senior policy appellate body as
needed to resolve major disputes.

-10-

2. How can cooperation in planning Corps-funded research be improved while recognizing the
Corps' authority and responsibility for its research progrann?

The Corps of Engineers established a research planning process in the early 1950s. This
process, the Fish Passage Development and Evaluation Program, is composed of representatives from
the Corps, fisheries agencies and tribes. The Corps believes it has the authority and responsibility to
conduct project-specific research, not to address regionwide research needs. The Corps has stated an
unwillingness to give up its discretion in managing and overseeing research.

The fisheries agencies and tribes have felt that their views have not been represented adequately
or taken into account in Corps decisions on research. Agency, tribal and Corps representatives on the
Fish Passage Development and Evaluation Program Technical Coordinating Committee have
discussed ways to improve cooperation at the technical level, which could result in better
representation of agency and tribal views to the Corps policy level. However, the fishery agency and
tribal policy makers have felt that pursuing these discussions is not very useful. They feel that the
overriding problem is that Corps policy decisions do not adequately take their views into account.

The following options describe potential solutions to this problem:

Option A. Implement a research planning process that recognizes the Corps' authority and
responsibility but improves cooperation with the fisheries agencies and tribes. The following elements
are based on the four-stage research planning process which may be developed as part of the long-
term spill negotiations, and could include:

.1. A policy-level group could work with all affected parties to develop a mainstem research
plan including objectives, criteria and guidelines for selection of mainstem research
projects; and identification of priorities for mainstem research.

2. The Corps could agree not to implement projects inconsistent with the mainstem research
plan unless it provides written justification that the proposal is consistent with the
Northwest Power Act or is necessary to satisfy the Corps' authority and responsibility for
its research program.

Qpt'on B. The Council could call on the Corps to work with other affected entities to review the
current research process and develop a more cooperative approach.

3- How might the technical quality of research and monitoring be improved?

There may exist a number of mechanisms that could improve technical quality of research and
monitoring. These include: better coordination of the many research, monitoring, and planning
activities within and outside the basin; improved use of statistical techniques and experimental designs;
and additional scientific review. Neither the research technical work groups nor the Monitoring and
Evaluation Group is presently assigned to undertake these activities. The Scientific Review Group of
the Implementation Planning Process, if implemented, could help provide the additional scientific
review.

-11-

Four options for improving technical quality have been identified:

Option A. Rely on the Implementation Planning Process to improve technical quality.

This would require close cooperation between the Implementation Planning Process Scientific
Review Group and the Council's Monitoring and Evaluation Group. The two groups would be
responsible for 1) identifying opportunities for coordination among research and monitoring efforts; 2)
providing scientific review of project statements of work, statistical standards, and results; 3)
coordinating genetics monitoring and planning; and 4) implementing a program for monitoring and
evaluating the effectiveness of program actions basin-wide.

Option B. Establish a cooperative integration and review group. This alternative would merge the
existing Monitoring and Evaluation Group with the Scientific Review Group. It could also include
representation from other responsible entities within the basin, e.g., the Corps of Engineers. With
policy-level direction, the group could integrate and evaluate systemwide efforts to double salmon and
steelhead runs. It could assist in identifying research and monitoring priorities, and in coordinating
various research and production programs. In addition, it could improve research quality by
establishing forums for scientific review and developing statistical standards. To accomplish these
tasks the group could form subcommittees as needed to address such topics as system planning,
genetics, system monitoring and evaluation, habitat classification, and scientific review of research
proposals.

Option C. Incorporate a mechanism to increase independent scientific review by establishing
technical peer review panels composed of independent scientists working inside'and outside the
Columbia River Basin. These panels would be responsible for reviewing research proposals for
scientific merit and could submit their views to the entity established under Option A or B above.

Option D. Establish an independent research planning foundation. The option would establish a
scientific institution to plan and fund salmon and steelhead research in the basin. The goal of the
foundation would be to provide independent planning and funding of program research. The
organization could be completely independent, with an advisory board made up of representatives
from the fisheries agencies and tribes. Alternately, it could be governed by a board of directors
representing the major interests in the basin. Funds for research activities coming from Bonneville and
other sources would be administered by the foundation. The foundation would be responsible for
research planning, proposal solicitation, proposal review and evaluation, monitoring, and making
available results concerning fish and wildlife related issues in the basin.

4. What method should be used to measure fish and wildlife program progress toward the doubling
goal?

The 1987 Fish and Wildlife Program stated that the Council's system monitoring and evaluation
program should include "[d]evelopment of alternative means to assess progress toward achieving the
goal of doubling the runs of salmon and steelhead in the Columbia River Basin consistent with the
policies stated in Section 204." The Monitoring and Evaluation Group has developed a set of
alternatives and provided a recommendation to the Council. The report of the Monitoring and
Evaluation Group is attached.

-12-

The discussion below sets out the alternatives provided by the Monitoring and Evaluation Group
and provides a rough estimate of the initial cost of implementing each alternative. It should be made
clear that these cost estimates only cover the cost of organizing existing information into an expression
of program progress. They do not include the cost of all existing monitoring programs or related
programs such as the coordinated information system. In addition, information beyond that collected
now as part of existing monitoring and research efforts is likely to be needed to achieve an acceptable
level of precision. The size of the overall effort may depend on the nature of subbasin plans currently
being formulated in system planning. For this reason, it is not possible to provide the total cost of a
monitoring program at this time. It will need to be reviewed in the future as more defined monitoring
programs are proposed.

QP^'Q^ ^- Measure progress as the change in smolt output from the Columbia River. This would
provide an early indication of the effect of the program on increasing smolt production and smolt
survival through the hydroelectric system. However, it would not include those parts of the program
designed to increase adult survival, nor would it provide a system-wide perspective. In addition,
techniques do not presently exist for estimating the size of the smolt outmigration from the Columbia
River, or for directly isolating the cause of changes in smolt numbers. Development of these
techniques would likely be a long and expensive process.

An alternative that would address only fish migrating from above Bonneville Dam would be to use
existing indices of smolt passage to provide an index of the change in smolt outmigration from year to
year (as opposed to an estimate of the actual number). This would involve collecting and processing
information now assembled by the Fish Passage Center and others. This is presently being done to
some extent, although there is not universal agreement regarding the relationship between the indices
and actual population size. The cost of implementing the smolt index as a measure of progress is
estimated at about $50,000 annually.

QP^'Q" ^- Measure progress as the change in the number of adults produced, this approach
would include the number of fish returning to the basin as well as those harvested in the ocean. An
additional correction would have to be made to account for those fish harvested in the ocean that
would have died in any event from natural causes. The adult count is the method implied in the Council
doubling goal. However, like the previous option, the numbers of fish observed in any year are the
product of many factors. Sorting out the effect of the program or of specific types of program actions
would be very difficult. For this reason, it is likely that little knowledge would be gained that could be
used to refine the program.

Above Bonneville Dam, computing the annual adult production would involve the use of adult
counts made at the mainstem hydroelectric projects by the Corps of Engineers, as well as estimates of
catch made by the fishery management agencies. Below Bonneville, fishery agency estimates of
tributary return and catch could be used. Cost of reviewing and compiling this information into an
annual index of progress is estimated at $75,000 annually.

Qp^'°" ^- Compute the change in salmon and steelhead surplus production as an index of
program progress. This option would utilize a computer model of the salmon and steelhead life cycle
which would incorporate the current understanding of the relationships between variables as well as
real-time monitoring and research data. The principal result would be a calculation of the adults that
are surplus to spawning needs. This surplus portion is available for harvest or to provide a buffer for
unanticipated changes in mainstem mortality or natural fluctuations. As real-time monitoring and

-13-

research data are reflected in the calculation, the change in projected surplus production could be
used as a measure of program progress.

This approach offers the opportunity to isolate the effects of the program within a computer
model. The model can also be used to generate testable hypotheses about the program. As these are
tested with specific program actions, important knowledge could be gained for refinement of the
program. On the other hand, the modeling approach is focused on long-term change and accordingly
would not provide real-time monitoring of changes in important parameters such as harvest or passage
rates.

The cost of development, refinement, and operation of the model to generate an index of program
progress is estimated at $150,000 annually.

Option P. Recommendation of the Monitoring and Evaluation Group. One of the principal
conclusions of the group was that there was no single measure of program progress. Each of the
above methods has certain features that are needed for a full expression of program effects. Smolt
counts provide an early indication of program effects and adult counts measure the "bottom-line" of
progress in restoring the fish runs. Analysis of the projected surplus production can increase our
knowledge, and should aid in refinement of the program. For this reason, the Monitoring and
Evaluation Group recommended that all three measures be used to evaluate the program.

The cost of using all three approaches is estimated at $200,000 annually.

5. Who should fund the effort to monitor and evaluate the progress of the fish and wildlife program?

While the 1987 program stated that ". . . the Council will develop a system monitoring and
evaluation program ... to measure progress toward the doubling goal as well as consistency with
program policies.", it did not specify how a monitoring program would be implemented or who should
fund its implementation. A wide variety of groups in the region have an interest in monitoring the
success of the fish and wildlife program. The Council has a fundamental interest in monitoring the
program so that it can be refined to achieve the goals of the program. Bonneville and the utilities have
an interest in determining the efficacy of investments of ratepayer funds. The fishery agencies and
tribes want to ensure that the program is biologically sound and successful in achieving management
goals. Many public interest and environmental groups are concerned that the implementation of the
program adequately addresses their concerns as well.

Because of the many diverse groups that are interested in following the progress of the fish and
wildlife program, it is clear that a monitoring program needs to be developed and implemented in a
cooperative fashion. Two avenues appear to be available to accomplish this goal.

Option A. Council funding. Under this option, the Council would fund the further development
and implementation of a program to monitor and evaluate the fish and wildlife program. This could be
done through the existing Monitoring and Evaluation Group or an expanded group. As noted above,
this group has spent considerable effort in developing alternative means to measure program progress
and to develop the components of a monitoring program. The group currently utilizes expertise from
the fish agencies, tribes. Council, the utilities, and consultants in a group that reports to the Council.
This group could be charged to develop and implement a monitoring program and report periodically
to the Council and the region regarding progress of the program and issues requiring particular
management attention or possible modification of the program. The 1987 Fish and Wildlife Program

-14-

charged the Monitoring and Evaluation Group to "aid in development and implementation of a
systemwide monitoring and evaluation program. . . ." Presently, the Council provides approximately
$100,000 annually to fund the Monitoring and Evaluation Group. Further development and
implementation of a monitoring program can be expected to require an expansion of the present group
and its activities. This could increase costs significantly.

Option B. Bonneville funding. This option would call for Bonneville funding of a cooperative
technical group that could represent all entities, including the Columbia Basin Fish and Wildlife
Authority, Bonneville, the Pacific Northwest Utilities Conference Committee, the Corps, the Council and
others. The group could be charged with conducting a system monitoring and evaluation program that
built on the work to date of the Monitoring and Evaluation Group and included the elements listed in the
Council's 1987 program. The group could report periodically to the Council and the region on
progress of the program and identify areas of the program requiring particular management attention
or possible modification of the program. Should a cooperative technical group be formed to address
other aspects of research and monitoring (Issue 3 above), it might be the logical group to fulfill this
function.

6- How can communication of research and monitoring results to resource managers and the
interested public be improved?

It appears that existing mechanisms to communicate research and monitoring results are
inadequate. The availability of policy guidance to articulate management needs could help solve this
problem, because research and monitoring results that respond to clear management needs will very
likely be communicated more effectively.' It is also important that research results be synthesized into a
form that managers and others can use, and that an appropriate forum be developed for
communicating results to all interested parties.

The following three options include a range of solutions, from implementing actions already
planned (the coordinated information system) to establishing several mechanisms to institutionalize
communication of results.

Option A. Take no additional action beyond existing mechanisms and the establishment of a
coordinated information system. The 1987 Fish and Wildlife Program calls for the establishment of a
coordinated information system as part of its system monitoring and evaluation program. The
coordinated information system has been initiated. It is intended to facilitate effective exchange and
dissemination of fisheries data, including coordination of systemwide data collection programs,
identification of data collection needs, and coordination of system monitoring and evaluation program
data with other data collection efforts.

Qp^'O" B. In addition to the coordinated information system, establish an annual research and
monitoring seminar to promote interaction among scientists and decision makers, and to inform
decision makers of research and monitoring results. This seminar could: 1) provide opportunities for
scientists to discuss research and monitoring issues with others within and outside the basin; 2) inform
decision makers and managers of research and monitoring results; and 3) provide opportunities for
decision makers and managers to make scientists aware of management concerns and needs. The
seminar could be held in two segments to accommodate both exchange among scientists and
interaction between scientists and decision makers.

-15-

QP^'P'"' C- Establish a periodic publication for results of research and monitoring funded through
the fish and wildlife program, and a compendium of ongoing salmon and steeihead research and
monitoring in and out of the basin. These items could be developed in addition to, or as part of, the
coordinated information system. Their purpose would be to increase the utility of research 'and
monitoring results by furthering the circulation of information to researchers as well as decision makers
and other interested parties. It could also serve to encourage the refined analysis and reporting of
research and monitoring data.

mvax::larry:[wn]wjlla r&m ip

-16-

•

RECOMMENDATIONS FOR A PROGRAM TO MONITOR AND EVALUATE

THE FISH AND WILDLIFE PROGRAM

OF THE NORTHWEST POWER PLANNING COUNCIL

from
The Monitoring and Evaluation Group

September 23, 1988

I. INTRODUCTION

In 1987, the Northwest Power Planning Council (Council) amended its fish and wildlife program
(program). The program was developed in response to the Northwest Power Planning and
Conservation Act of 1980 (the Act or Power Act), which called on the Council to develop a program to
"protect, mitigate, and enhance" the fish and wildlife resource of the Columbia River Basin as affected
by hydroelectric development and operation.

A significant feature of the 1987 program was the inclusion of a System Monitoring and
Evaluation Program (SMEP) to track the progress of the program in achieving the Council's goals. To
develop SMEP, the Council formed the Monitoring and Evaluation Group (MEG) in April 1987
composed of technical experts from the region.

One of the first charges of the group was to develop a set of alternative methods that could be
used to measure the progress of the fish and wildlife program. This report summarizes MEG's findings.
It also includes a recommendation for a measure of progress and provides a discussion of the
elements that the group feels would be required to implement a system monitoring and evaluation
program. Considerable work remains before a full scale monitoring program could be realized; the
intent of the present effort is to further the discussion of a measure of progress and to initiate the
development of the required elements of a monitoring program. Several important elements of the
monitoring program specified in the 1987 Fish and Wildlife Program are now being implemented as
part of the System Planning process.

Background

The Act called on the Council not only to develop a program to address the fish and wildlife losses
in the basin, but to develop a program that would take significant, positive action despite
acknowledged uncertainties both in the biological requirements and in the best methods for increasing
fish production in the basin. The Act also called on the Council to treat the Columbia Basin as a
system, necessitating an approach that transcends the jurisdictional complexities of the Columbia
Basin.

The 1987 program represents a fundamental step in the development of the Council's efforts to
meet this mandate. Measures in the new program are focused on achievement of an interim goal of
doubling the runs of salmon and steelhead in the basin (Section 203). Efforts to reach this goal are to
be consistent with the following set of policies (Section 204) that are intended to ensure equitable and
sustainable benefits:

1 . The area above Bonneville Dam is accorded priority.

2. Genetic risks must be assessed.

3. Mainstem sun/ival must be improved expeditiously.

4. Increased production will result from mix of production methods.

-1-

5. Harvest management must support rebuilding.

6. System integration will be necessary to assure consistency.

7. Adaptive management should guide actions and improve knowledge.

To plan the actions needed to meet the interim goal, the Council established a basinwide process
termed System Planning (Section 205). System Planning is an effort under the lead of the fish agencies
and tribes to plan fisheries actions in 31 subbasins of the Columbia River. It includes, first of all,
planning at the subbasin level to identify fish production objectives, constraints and opportunities.
Secondly, it involves integrating these subbasin plans into an overall system plan to meet the doubling
goal while maintaining consistency with the Council's policies.

Section 206(d) of the fish and wildlife program states that the System Monitoring and Evaluation
Program is intended to follow the progress of the system plan in meeting the Council's goal and its
consistency with the Council's policies. Such a program could fulfill two functions. First, SMEP could
provide a link between the expenditure of ratepayer dollars and the Council's charge to "protect,
mitigate and enhance" the fishery resource. The fish and wildlife program represents a sizeable
expenditure by the Northwest ratepayers through the Bonneville Power Administration. SMEP could
provide a measure of the progress achieved with these funds. Second, SMEP could provide a means
of dealing with the uncertainty associated with the best means of achieving the mandate of the Power
Act, The Act recognized the uncertainty but called on the Council to take positive action. A major
funqtion of a system-wide monitoring program would be to provide the necessary feedback to the
Council and the region so that, over time, the program could become a refined vehicle for achieving
the Council's goals.

The program states that the Monitoring and Evaluation Program is to incorporate the following
features (206(d) (2) (A-F)):

1 . "Development of alternative means to assess progress toward achieving the goal of doubling the
runs of salmon and steelhead in the Columbia River Basin, consistent with the (program) policies

2. "Evaluation of research results and reporting on implications for program objectives";

3. "Development of a coordinated information system designed to facilitate effective exchange and
dissemination of fisheries data";

4. "Maintenance of the system planning model, including documentation, recommended standards
for use, and modifications'.';

5. "Integration with the system planning activities . . ."; and,

6. "Examination of quantitative methods to incorporate genetic conservation into production
planning."

The initial issue in the development of the SMEP is the method of assessing progress. The choice
of a measure will largely depend on the question asked of a monitoring program and the amount of
information that the Council desires to obtain from a measure of progress; the ultimate size of the
monitoring program will depend on the degree of resolution that is desired. The next section of this
report will include a discussion of alternative means of measuring progress and MEG's
recommendation regarding the best course of action. This will be followed by a description of the
actions MEG feels are necessary for implementing this recommendation.

II. ALTERNATIVE APPROACHES TO MEASURING PROGRESS

Ideally, a measure of progress should not only determine progress, but should also provide
information to increase our understanding, decrease uncertainty, and permit the program to be refined
over time. The choice of a measure of progress largely depends on the question asked. If the question
is "Have the number of salmon and steelhead produced in the Columbia Basin increased since the
inception of the fish and wildlife program?" then a simple accounting of the adult production will
suffice. However, little knowledge will be gained by such a method and it will not be possible to
attribute positive or negative trends in production to the fish and wildlife program. A more complicated
but useful question would be "Have the number of salmon and steelhead produced in the Columbia
Basin increased as a result of the fish and wildlife program?" This requires the ability to separate
out program effects from non-program effects such as changes in harvest rates or natural ocean
survival rates. This presents considerable technical difficulties because of the numerous conflicting
factors that determine the number of fish produced from the river. This question addresses the
accountability for ratepayer expenditures in the whole program, but does not provide information that
could be used to improve the program. A final refinement of the question might be "What types of
measures in the fish and wildlife program are effective and which are not?" The answer to this provides
a measure of progress, as well as information to refine the program over time.

MEG reviewed a number of options for a measure of progress and categorized them as
observational or analytical in nature. The former refers to methods that rely on direct counts of fish,
while the latter combines a host of monitoring and research data into mathematical expressions that
attempt to explain trends in observational indices.

1. Observational Methods

Observational methods of measuring program progress would consist of enumerating the fish
population at any of several points in the salmonid life cycle. These types of measurements have the
advantage of being conceptually simple and could, in many cases, be derived using existing data
sources.

While fish could be counted at any point in the life cycle, those in the following list would be likely
observation points:

a. Smolts at the mouth of the Columbia River.

b. Adult returns to the mouth of the Columbia plus prior harvest.

c. Adult equivalent total production (explained below).

d. Adult returns to the mouth of the Columbia River.

e. Adults crossing Bonneville Dam.

Each of these points of measurement have various advantages and disadvantages relating to
logistics involved in measurement, and their relevancy to the Council goal.

Juvenile Counts. The number of smolts from the Columbia system that reach the ocean has the
appeal of being a direct measurement of the effects of many fish and wildlife program actions.
Because the Council's jurisdiction is limited to actions relating to production and mainstem passage,
and not ocean or river harvest rates, it could be said that the Council's obligation is simply to double
the number of smolts reaching the ocean. The subsequent fate of the fish in regard to human
intervention would be the responsibility of the state, federal, and tribal harvest managers.

Juvenile counts would provide a rapid index of the effect of program actions. The number of
smolts would generally provide a measure of production and juvenile passage improvement within a

year or two. In contrast, the number of adults produced can only be measured after the return of up to
five year classes in the case of Chinook salmon.

The Council, however, has consistently rejected using juvenile measurements in favor of adult
production or returns as a final measure of program benefits. This is because an evaluation based
solely on the production of juvenile fish fosters the fragmented approach to management that has
plagued past efforts to restore the fishery and does not provide a systemwide perspective on program
effectSc

Another difficulty with using juvenile counts to monitor the program is that they only tell part of the
story in regard to the influence of the program on salmon and steelhead production. Several actions in
the fish and wildlife program address the survival of adult fish. In addition, program actions can affect
the survivability of smolts beyond the point where direct Council influence stops. Hatchery practices
and mainstem passage conditions, for instance, can influence the quality and survival of smolts
reaching the ocean. Doubling the number of smolts below Bonneville that are of poor quality and with
little potential for return will not achieve the Council's goal.

Monitoring the smolt outmigration also has major logistical difficulties. Potentially, the population
of smolts outmigrating from above Bonneville Dam could be determined by a population estimate at
the dam itself. However, there is presently no reliable method for estimating the smolt population size
at any of the mainstem hydroelectric projects. Also, there is presently no technique available to
estimate smolt population size from tributaries below Bonneville Dam. Development of techniques for
estimating the smolt population both above and below Bonneville Dam would be a difficult and
expensive endeavor.

An alternative to estimating the size of the outmigrating population might be to rely on changes in
various smolt migration indices as an index of program progress. A number of these indices are
presently collected by the Fish Passage Center operated by the fishery agencies and tribes. Because
the information is collected routinely, this could offer a low cost, and at the present time the only,
possibility for measuring progress in terms of the size of the smolt outmigration. However, to be a
suitable measure of progress, an index must have a consist, although probably unknown, relation to
the size of the outmigrant population. At the present time, there is considerable doubt regarding the
consistent relationship between existing smolt indices and the population size (see various annual
reports from the Smolt Monitoring Program of the Fish Passage Center).

Adult counts. The primary advantage of using adult counts to evaluate the program is that they
represent the "bottom line" in regard to the actual effect of the Council's efforts to restore the salmon
and steelhead resource. Thus intuitively, adult counts may have the greatest appeal as a measure of
program effects.

By far the easiest and least expensive method for monitoring the program in terms of adults
would be to track the number of adult fish returning to the mouth of the Columbia River. For fish
produced above Bonneville Dam, this would entail adding the number of fish counted at Bonneville to
an estimate of the number caught or killed between the dam and the mouth of the Columbia River.
Similarly, for fish produced below Bonneville, the estimate of loss below Bonneville would be added to
the estimates of return to the individual subbasins. The Corps of Engineers and the fishery
management agencies routinely collect this information, and no additional cost would be involved.

While simplicity is the greatest virtue of this approach, it is also its greatest drawback. An
evaluation limited to adult returns to the river would have little ability to separate out program affects
from all other factors affecting returns such as harvest and natural survival rates. Success or failure in
achieving the doubling goal could not be attributed to the effect of the program. Very little would be

added to our understanding of the causes of trends in returns, and little contribution would be made
toward the refinement of the program.

The program defines the doubling goal in terms of "the number of adults returning to the mouth
of the Columbia River plus the number of adults caught in the ocean." Under this approach, adult
returns to the river mouth would be estimated as described above, plus prior ocean catch of each
stock. By adding the estimated ocean catch, the effect of fluctuations in harvest rate on returns would
be taken into account, and would provide a truer measure of program progress.

However, it would still not be possible to totally isolate the contribution of the program from other
factors, notably variation in the natural survival rate in the ocean. Even if the ocean survival rate could
be assumed to be more or less constant (excepting dramatic and obvious changes such as an El Nino
event), and progress toward the doubling goal could be attributed to the program, we would still not
know how the program affected returns. No information would be provided as to the efficacy of
different types of actions, our understanding of the system would not be increased, and no contribution
would be made to the refinement of the program. In addition, adding the unadjusted catch to the
estimate of return would consistently overestimate program production since some of the fish caught
would have died in any event from natural causes prior to reaching the river mouth.

The measure could be improved by adjusting the estimates of ocean catch by the number of fish
that would have died in the ocean from natural causes. This measure is termed the "adult equivalent
run size" and is a true measure of the adult production from the basin or from a particular production
scenario. Adult equivalent run size is used in the Pacific Salmon Treaty process, and by the federal
courts in determining allocations between Indian and non-Indian fisheries. This measure would thus
more accurately reflect actual adult production and would be consistent with other coastwide
approaches.

However, the other shortcomings of the previous method would apply equally to adult equivalent
run size as well. It is a fairly complex process that would not properly credit the program, and would
contribute little to our knowledge or ability to refine the program.

2. Analytical Methods

Observational methods, while attractive because of their intuitive appeal, suffer as measures of
progress because of their limited ability to increase our knowledge of the salmon and steelhead
resource. Observational indices only address stock abundance and provide only limited amounts of
information to isolate program effects or to refine the program. Analytical methods build on
observational methods. They attempt to increase the information content by integrating environmental
indices, research results, or monitoring data into mathematical expressions that are hypotheses
explaining the trends evidenced by observational data. These expressions are refined by testing
through research or monitoring programs.

Analytical methods can be divided into at least two general categories that will be termed here
statistical and life-cycle approaches. Statistical methods can be used to discern relationships between
variables such as run size and flow during the outmiqration, number of spawners, or the relationship
between year-classes. A life cycle approach, on the other hand, uses a computer model as a
conceptual basis for explaining trends displayed by observational indices. The degree of relationship
between the model and the observed trends is the basis for refinement of knowledge using information
gleaned from directed research and monitoring programs.

Statistical methods. A variety of statistical techniques are available to examine the relationship
between variables or to partition observed variation into component parts. For instance, the Oregon
Production Index (OP!) uses statistical correlation to provide harvest managers with an indication of the

size of the coho population off Oregon prior to setting ocean troll fishing seasons. In this case, the
abundance of adult three-year-old coho salmon off the coast of Oregon over a series of years has a
statistical relationship to the abundance of two-year-old fish that returned in the previous year from the
same generation. By observing the number of two-year-old fish that return in a year, predictions can
be made about the expected number of three-year-old fish the next year. Because it is a statistically
based method, a measure of the probability and statistical confidence of the prediction can be made.

Statistical techniques offer a suite of powerful tools that can be used to gain insight regarding the
sources of observed variation in returns, for instance, or for examining relationships within a limited set
of variables. In most cases, they offer the ability to determine the statistical confidence that can be
associated with a statement about the relationship between variables.

One difficulty with these techniques is that it is often necessary to concentrate on some small
segment of the life cycle or a single correlative relationship such as the OPI. It is generally not possible
to discern statistical relationships over the whole life cycle because of the many conflicting factors that
contribute to the observed population size. These techniques, therefore, may not be applicable to the
evaluation of the program as a whole.

Statistical relationships also do not necessarily imply cause and effect but may only be showing a
relationship between two variables that are both responding to a change in another variable. In the
case of the OPI, for instance, the abundance of fish that mature at age two may have no direct impact
on the number that mature at age three. Instead, it is possible that both age classes have similar
survival rates in the estuary and early ocean life stage, and that this survival rate is a critical factor
affecting the ultimate abundance of both age classes. Observing a statistical correlation between the
two year classes provides a useful management tool but does not provide insights that would lead to
improved hatchery practices, for example.

Life cycle method. In a life cycle approach, the various discrete pieces of information that exist
about salmon and steelhead are organized around a model of the life cycle. This can include
information obtained from observational methods as well as information on the relationships between
variables gleaned from a variety of statistical methods. The life cycle approach thus incorporates
information from all the previously discussed methods as well as the results of research aimed at filling
in critical data gaps. The model is itself a hypothesis about how the pieces fit together which can be
tested by comparing its behavior to that of the real world.

This approach permits the available information to be examined in the context of the behavior of
the total system, and provides a consistent framework for comparison and evaluation of actions. As
the various components of the model are refined over time through monitoring and research, the
model becomes a better and better expression of reality indicating an improved understanding of the
system. In contrast to many statistical techniques, the life cycle technique does not attempt to predict
year-to-year changes in fish abundance. Instead, it is used to examine the effect of various actions on
the long-term trend in various types of production indices.

A number of output variables can be obtained from the life-cycle method including measures of
stock abundance and stock productivity. Stock abundance is the estimated number of fish present,
and is analogous to the observational estimates of abundance discussed above. Stock productivity
refers to the ability of the population to produce fish surplus to the number that are required to spawn
and maintain a stable population size. This is an important expression of the condition of a population
since it reflects the number of fish available for harvest or to buffer the effect of environmental
fluctuations. The amount of the population surplus to spawning needs is also an expression of the
speed with which a population will respond to changes in the environment including mitigating
measures. A population with a large surplus proportion will respond quickly to improvements in the
environment, whereas in the reverse, the population may respond to improvements only over a very

long time period. This ability to provide a variety of outputs and to examine the important aspect of
stock productivity is a major advantage of this method.

By providing a means to probe the relationships between variables and to examine the effect of
hypotheses on stock abundance and productivity, the life cycle method could contribute toward an
improved understanding of the system and the refinement of the program. The life cycle approach
should make it possible to isolate, within the confines of the model, the effects of the program and
allows them to be examined independent of real-world variation in non-program effects.

However, the method does have drawbacks. First, it lacks the intuitive appeal of the
observational methods. The product of the life-cycle approach is knowledge about the system rather
than simply the number of fish. While of obvious importance, knowledge can only be converted to fish
production by an effective management structure that can deal with the information. The nature of the
management structure is outside the purview of MEG and this discussion.

A second drawback to the life-cycle approach is that it lacks real-time application. This approach
would not yield annual predictions of effects, nor would it attempt to explain year-to-year variation in
returns. Instead, it would deal with the reasons for long-term trends in returns, efficacy of types of
measures, and the state of our knowledge. Compliance with the Council's policies (Section 204) could
not be easily assessed by the use of a model by itself. For instance, the Council's goal that harvest
rates will be controlled to support rebuilding, will require a year-to-year monitoring of harvest rates.
This would be more aptly addressed by compiling information collected from existing monitoring
programs such as that being conducted under the Pacific Salmon Treaty process or various state and
tribal management programs.

3. MEG Recommendation

MEG examined the above methods with the goal of designing a program that would permit the
effect of program measures to be isolated from other effects, and would maximize the opportunity to
learn from grogram implementation. MEG reached two conclusions:

1 . No single measure of the program progress was found. Different indices address different
aspects of the problem of monitoring and evaluating the fish and wildlife program.

2. No method was found that would directly (e.g., experimentally) identify the effects of the program
as distinct from non-program effects. It was MEG's conclusion that program effects would have
to be isolated by analytical methods such as the life-cycle approach described above.

None of the methods examined would completely address the Council policy of assessing
genetic risks in production planning, although many genetic aspects could be incorporated into a life
cycle model. MEG suggests addressing genetic risks in part as a topic separate from the more
quantitative measures discussed so far. The recommendation from MEG is that the Council utilize a
four component measure of progress that consists of:

a. A measure of annual juvenile population.

b. An estimate of annual adult equivalent production.

c. A life cycle analysis of stock productivity.

d. A program to monitor the genetic impacts of management actions.

Juvenile population would be indexed annually to provide an initial indication of the effect of the
program on the salmon and steelhead production. Juvenile population would first be indexed for
subbasins selected according to an overall experimental design as discussed below. A second stage
would be to estimate the size of the annual outmigration, probably at Bonneville Dam, and to estimate

the survival of migrants through the system. Much of this information would be obtained from the
Smolt Monitoring Program conducted by the fishery agencies and tribes. Juvenile population data
would: 1 ) provide annually an indication of the effect of program actions on the size of the juvenile
outmigration, especially those dealing with fish production and the survival of juvenile fish through the
hydroelectric system; 2) measure the progress of the program in expediting improvement in mainstem
passage survival rates (program Section 204(c)); and 3) assess the geographical distribution of
production programs within the basin (program Section 204(a)).

The adult equivalent production would index stock abundance. Progress of the program would
be measured within the context of other non-program effects. The measure would be calculated as the
estimated adult return to the mouth of the Columbia River corrected for natural ocean mortality and
ocean harvest. The latter correction factors could probably be obtained from ongoing marking and
monitoring being conducted by the fishery management agencies under the Pacific Salmon Treaty.
The adult equivalent production would provide an annual assessment of the overall production from the
Columbia system, and, in the course of its calculation, would provide the means to monitor
consistency with the Council policy regarding harvest (program Section 2Q4(e)).

The life cycle analysis would increase our knowledge of the causes behind the trends seen in the
previous two indices. A model would be used to organize information, identify and prioritize needed
information, and compare alternative methods of achieving goals. As a means of measuring program
progress, the model would be used to calculate the stock productivity.

MEG suggests using the model to determine the change in stock productivity that occurs as a
result of program measures and for the program as a whole. In many ways the stock productivity is a
more useful expression of stock condition than is stock abundance (e.g., the adult production). Stock
abundance might be viewed as the easily observed tip of the iceberg while stock productivity is the
more obscure but potentially more important underlying ice. Stock productivity provides insight into
the capacity of the population to withstand harvest pressure, mainstem passage mortality rates, and
environmental fluctuation while indicating how fast the population might respond to program
measures. None of this can be found from simple expressions of stock abundance.

Genetics monitoring. In addition to measuring the progress of the fish and wildlife program,
SMEP is charged with addressing means to assess genetic risks in production and to meaningfully
incorporate genetics into production planning (production policy b and Section 206(d) (F)). MEG is
responding to this through the System Planning process by providing guidelines for genetic impact
assessment and a genetic conservation program. The latter will include measures to monitor genetic
changes resulting from production programs and genetic research requirements.

III. ELEMENTS OF THE MEG RECOMMENDATION

The process for monitoring and evaluation proposed by MEG consists of collecting, organizing,
and retrieving information about the biological system so that progress is followed and our increase in
knowledge is maximized. Collection of information would occur through monitoring and research
programs developed around an experimental design that seeks to maximize the cost effective
collection of data. Organization of this information in a form that maximizes our learning and
understanding would occur through the use of a life cycle model. This would summarize our
understanding of the logical relationships between the collected data. Retrieval and reporting of the
data collected through monitoring and research, and the knowledge that emerges from structuring this
information, would occur through an information system that would also serve to coordinate data
collection and facilitate communication between data bases. The genetics monitoring program forms
an additional component of the SMEP which may not be fully encompassed by the other, more
quantitative, aspects of the program. Each of these components will be discussed below. Some of

these components have been or are being implemented, primarily as a part of System Planning. Most
of these elements are relevant to any of the measures of progress discussed in this report.

1. Experimental Design

The precision with which SMEP will monitor program progress will depend on the amount of
information that is collected. Extremely intensive monitoring of all aspects of the salmonid life cycle in
all subbasins would produce an extremely precise measure of program progress. However, the cost of
such a program would likely be prohibitive. Thus a balance must be struck between the degree of
precision desired, and the cost of obtaining the required information.

Because the desire would be to obtain the highest precision possible for the available funds, it is
not possible at this time to set the degree of precision, or, therefore, to set the cost of the overall
monitoring program. Some of the information pertinent to SMEP could be obtained from existing and
planned monitoring programs the cost of which is known. However, much of the effort would be
devoted to the monitoring of subbasin plans and from research to fill in specific data gaps. The
number of new monitoring programs that will be needed to test the production hypotheses made in
system planning cannot be known until the subbasin plans are available.

As subbasin plans are implemented, the monitoring program could begin to be focused by
selecting a number of stocks and subbasins for intensive monitoring efforts. From a technical
standpoint, the monitoring sites would be selected according to a preset experimental design. This
design would consist of a stratification or categorization of information initially according to whether it
is of a system-wide or subbasin nature. Subbasin data would be further categorized by habitat,
biological, and management criteria.

System information refers to parameters such as harvest rates and mainstem passage rates that
occur, for the most part, outside the subbasins. Subbasin data, on the other hand, is a function of
environmental and biological characteristics specific to a subbasin. Subbasin data is divided into
categories of information pertaining to production type, e.g., natural and hatchery production. This
includes egg and smolt carrying capacities, juvenile survival rates and hatchery management
scenarios.

Classification of habitat and watershed types will be done as a part of the design of the
Coordinated Information System which will begin in fall of 1988. This will be discussed more fully
below. Indicator stocks and subbasins would be chosen from each of the blocks defined by this
categorization. These would be the focus of more intensive monitoring efforts to identify the efficacy of
specific types of actions, and to provide needed life cycle parameters. This classification of the basin
could also be used to guide the System Planning process in regard to selection of control subbasins
and stocks and the adaptive implementation of subbasin plans.

Pertinent types of information that would be used to generate the components of the MEG
recommendation are shown below. These are organized according to whether they are collected at
the system or the subbasin level.

I. Juvenile information by species

a. System level by stock.

1 . Population size at Bonneville.

2. Passage survival rate.

3. Individual passage survival parameters.

b. Subbasin level by production type.

1 . Population size out of the subbasin.

2. Survival rates.

3. Carrying capacities.

4. Results of particular experiments and production programs.

II. Adult information by species

a. System level by stock.

1 . Adult returns to mouth of Columbia River or to Bonneville Dam.

2. Prior harvest.

3. Natural ocean survival rates.

b. Subbasin level by production type.

1 . Returns to the subbasin.

2. Terminal harvest rates.

Ill- Production potential by basin, species, stock, or measure

a. System level by stock.

1 . Ocean and river harvest rates.

2. Mainstem passage parameters.

3. Natural ocean survival rate.

b. Subbasin level by production type.

1 . Egg and smolt carrying capacities.

2. Survival rates.

3. Hatchery strategy.

4. Fecundity.

Some of this information could be obtained from existing sources. For instance, many of the
juvenile passage parameters could come from the Smolt Monitoring Program presently operated by
the fishery agencies and tribes. Similarly, the ocean harvest and survival rates could be obtained in
whole or in part from marking and evaluation programs conducted by the fishery management
agencies, especially those programs pertaining to the Pacific Salmon Treaty. Programs such as the
Yakima-Klickitat Production Project and the Idaho habitat evaluation project would contribute to SMEP
and would need to be coordinated with the CIS and the experimental design. Subbasin information is
now being compiled through the System Planning process that will form the baseline for monitoring the
implementation of production measures.

However, it can be anticipated that considerable additional information will be required to monitor
the fish and wildlife program at an acceptable level of precision. Much of this would come from the
monitoring of specific subbasin hypotheses generated as part of the subbasin plans being prepared
through System Planning. Subbasin plans will include the identification of critical uncertainties and
plans to monitor the progress of the plan and resolution of the uncertainties. SMEP would use the
experimental design to organize, prioritize, and focus the numerous monitoring proposals that will arise

-10-

from System Planning, as well as those monitoring programs that are now underway or are being
contemplated as part of other aspects of the fish and wildlife program.

It is also likely that SMEP and the system wide analysis of information proposed here would lead
to insights that could affect the direction and prioritization of information needs in these programs or
require additional effort outside the present capabilities. Examples of this are the need listed above for
a juvenile population estimate at Bonneville Dam and the juvenile survival rate through the hydroelectric
system. This need is also recognized by many other agencies and programs in the basin. Although
the means to obtain this information are not presently known, the need remains and could spur
development of the required techniques.

2. Life Cycle Model

Under the monitoring and evaluation scheme proposed here, a model would be used to organize
the information collected from existing sources and from future monitoring and research activities into
a logical and coherent picture of the life cycle of salmon and steelhead. MEG is proposing to use the
System Planning Model as an initial tool to structure the existing data. This model is now being used in
System Planning to compare production scenarios and eventually as a technical component of the
integration of the subbasin plans into a system plan. For SMEP, the model would be used to compute
the stock productivity of projects, subbasins, and ultimately the system.

The computation of stock productivity would occur at three points during project planning and
implementation. First, it would be used to assist in the evaluation of production and management
alternatives. This is now being done in System Planning, where the model is used to compare the
increase in run size and productivity that can be expected from various alternatives given assumed
harvest, passage, and natural survival rates. The model will also provide the technical basis for the
system integration of the subbasin plans (Section 206(d) (E)). This will entail a process of balancing the
productivity of the various stocks with the prevailing harvest and passage survival rates.

A second application of the model should be the identification of uncertainties that affect the
achievement of goals. During the planning of production or management actions, much of the model
input data will consist of assumptions and hypotheses, and the simulations will contain considerable
uncertainty. Parameters identified as critically affecting the outcome of plans should be flagged as
areas requiring special research or monitoring attention. The importance of these area should be
communicated to the research technical work groups, if they fall within the existing research priorities,
and to the Council if alternative or additional areas research emphasis are indicated.

Finally, after implementation of a set of actions, the model would be used in SMEP to compute a
measure of progress of the program in doubling the capacity of the basin to produce salmon and
steelhead. The simulations made during the planning would be updated over time as research
indicates that modifications are warranted in the input data. These simulations would provide a
baseline for measurement of progress. After implementation, simulations would continue and
incorporate monitoring and research data. Progress of the program would then be measured by
comparing these simulations to the baseline simulations. The process proposed here is thus becomes
a continuous cycle of updating simulations made during the planning phase, to form a baseline, with
real-time monitoring and research data as the plan is implemented.

3. Coordinated Information System

A program to monitor and evaluate a system as complex as the Columbia River will necessarily
involve the coordination of large amounts of information. To a large degree, the monitoring and
evaluation program proposed here can be characterized as a system to collect, organize, and evaluate
information collected at all life stages, and to make this information available to decision makers in a

-11-

timely and effective manner. This will include information gathered at the system level, such as harvest
rates, recovery of tagged fish in the fisheries, and the survival and abundance of juvenile and adult fish
through the hydroelectric system; as well as subbasin data such as numbers of spawners, subbasin
carrying capacities, juvenile survival rates, and hatchery and natural production levels. It will also
depend heavily on research results that clarify the relationships between variables and the results of
different management actions.

The Coordinated Information System (CIS), called for in Section 206{d)(C) of the program, will be
used to bring together the data and information needed for SMEP. The CIS is a system that will
facilitate exchange and dissemination of data within the Columbia Basin. Given the multi-jurisdictional
nature of the basin, the need for such a system has been recognized for some time, and was one of the
primary recommendations to emerge from the regional planning under the Enhancement Act.

MEG has prepared a workplan for the design of a Coordinated Information System that is
expected to implemented in the fall of 1988. The workplan calls for the design and scoping of a CIS
and the completion of demonstration projects in the John Day, Umatilla, Yakima, and Salmon River
subbasins. As note above, the workplan also calls for the development of techniques for classification
of habitat and watersheds which is relevant to the basin-wide experimental design of SMEP and the
organization of the CIS. Finally, the workplan provides for the updating of the Stock Assessment Study
as a periodic output from the CIS.

The CIS will consist primarily of standards and protocols that will enhance the communication
between the various data bases that reside on systems maintained probably by the agencies
responsible for collecting the data. Creation of the CIS originates with the system wide data collection
conducted as part of System Planning. It will incorporate the natural and hatchery data bases
maintained by the Council (Section 206 (e)(1) and (2)). The standards and protocols will address the
following features of data collection:

• Quality and documentation of data. The effectiveness of the evaluation process in assisting
decision making depends upon the quality of the data collection and on the analyses performed.
Confidence in the SMEP requires accountability through documentation of data sources and
procedures.

• Consistency in data collection. Consistencies in data collection, sampling and survey methods,
units of measure, and consistency in the definitions of sampling units (time-area, fisheries, production
types etc.) are needed to allow comparability between subbasins and projects throughout the
Columbia Basin. Consistency is also necessary over time, since a primary function of SMEP is to
evaluate change in production over time.

• Timeliness and quality of data reporting. Communication of data, analytical results and methods
in a timely and clearly understandable way is necessary to perform the monitoring and evaluation steps
on time and to convey results in a way that promotes trust. Communication of the SMEP process is an
important step toward integrated system planning and research.

Genetics Monitoring

Despite the almost universal recognition of genetics as a factor contributing to the success of
production programs, meaningful incorporation of genetic concerns into production planning has
proven difficult. This stems from a lack of clear genetic objectives, the degree of uncertainty regarding
how to incorporate genetics into production planning, and because of the gulf that separates the
academic study of genetics from the practicalities of fisheries management. The genetics portion of
SMEP deals with these last two problems to provide the tools to assist in the formation of genetic
objectives.

-12-

The specifics of the SMEP genetics tasks are being developed by MEG as part of System
Planning. To date, MEG has drafted two products designed to help subbasin planners assess possible
genetic impacts of proposed actions. These are the Genetics Evaluation Tree (GET), which is a key to
identify relative qualitative impacts of different actions, and the Genetics Evaluation Model (GEM), a
computer model designed to determine a quantitative index of genetic impacts. These tools will be
brought together in a procedure for identifying genetic impacts, and submitted to the System Planning
Group on September 1, 1988.

The complete genetic conservation program will include the genetic impact assessment tools
described above as well as: 1) technical documentation of the process; 2) identification of critical
areas of uncertainty and needed research; and 3) identification of techniques for testing genetic
hypotheses generated in system planning. As part of the System Planning Process, MEG will continue
development of the Genetics Conservation Program. Input will be solicited through the Genetics
Subcommittee of MEG and through independent peer review of program elements.

Conclusion

The intent of MEG's discussions to date and this paper has been to provide the Council with a
range of options for measuring the progress of the fish and wildlife program, and to make
recommendations regarding the elements that would make up a system monitoring and evaluation
program. Further development of the program will require a resolution of the question of a measure of
progress. Considerable work remains, and most would have to be completed by the end of System
Planning. Because of the likely scope of SMEP, it is also important that a stable implementing
mechanism and funding source be established.

mvax::larTy:[cm]chip smep discussion paper

-13-

Membership
Monitoring and Evaluation Group

September 23, 1988

Name

Ron Boyce
Mark Chilcote
Tom Cooney
Stan Detering
Larry Everson
Kathryn Kostow
Chip McConnaha
Lars Mobrand
Phil Roger

Affiliation

Oregon Department of Fish and Wildlife

Oregon Department of Fish and Wildlife

Washington Department of Fisheries

Bonneville Power Administration

Bonneville Power Administration

Pacific Northwest Utilities Conference Committee

Northwest Power Planning Council

Consultant

Columbia River Inter-Tribal Fish Commission