This effort attempts to assess the effectiveness of our population assistance
activities in the context of the programs of recipient governments. It
attempts to estimate the prevalence of contraceptive use as a measure of the
effect of a country’s overall population program in reducing birth rates,
and to estimate the dollar cost per contraceptive user as an indication of
the cost-effectiveness of the program. As the attached discussion of the
format emphasizes, however, any serious attempt to evaluate the
effectiveness of population programs must take into account the influence of
other socio-economic determinants of birth rates. For this reason, the
performance evaluation format calls for data on a wide range of
socio-economic variables potentially affecting fertility, and incorporates
this data in the final section of the format in which the overall
effectiveness of the program is analyzed. Additional work must be done to
improve our understanding of the relative contributions of family planning
services and these other determinants of fertility reduction; as this work
progresses the evaluation format will be refined by the Ad Hoc Group to
increase the usefulness of these criteria and to develop new and better
measures of program effectiveness.
The information provided by this evaluation effort will be used in deciding
how to allocate funds for our population assistance and will be included in
the required annual reports of the Ad Hoc Group. Priority attention will be
given to the key countries established as priorities for U.S. assistance by
NSDM–314.3
Attachment
Paper Prepared in the Department of State4
Washington, January 16, 1978
Description of Format for Evaluation of Population Program
Performance
The attached format for the evaluation, on a country-by-country basis, of
programs to reduce population growth5 has been developed by the
Department of State in consultation with the Agency for International
Development and the Department of the Treasury. The evaluation will be
used: (1) to assess the commitment and progress of countries in
controlling population growth and the effectiveness of existing
population programs; (2) to assist and support decisions about future
funding levels and the direction and priorities of our assistance
programs.
The format calls for information and analysis in the following
categories:
—basic demographic data over five-year periods or other useful intervals;
especially 1965–1975;
—social, economic, and political factors affecting population
programs;
—nature and extent of population programs;
—financial inputs, prevalence of contraceptive use, and cost per
contraceptive user; and
—critique evaluating the overall effectiveness of the programs.
Using data from a number of sources, including AID, the United Nations system, the World Bank, the Bureau
of the Census, the Popula[Page 1019]tion
Council, and the International Planned Parenthood Federation, the
performance evaluation effort attempts to:
—assess the impact and cost effectiveness of population programs, both
national efforts and external assistance, and
—assess the effect of economic, social, and political factors which may
have an influence on fertility and which provide a context for
population programs.
This will involve the development of data on past, present, and projected
resources dedicated to efforts to reduce population growth at country,
regional, and global levels from all sources. These data will be used to
examine the efforts not only of AID but
also of donors receiving AID support,
including the United Nations Fund for Population Activities, other
external donors, and recipient governments’ own national programs. The
data collected and evaluations made will be measured against specific
global priorities and objectives which have been established.
I. Demographic Data
Section I summarizes the demographic situation in the host country over
five-year periods (1965–1975). Most demographic data are not definitive
and the range and trends are indicated. Specifics are reflected in
Appendix A6
along with the source of the information.
II. Program Setting
Section II discusses the general setting in which the national program
functions, including the socio-economic phenomena that may relate to the
fertility reduction process and the various political, religious,
cultural, legal, administrative, managerial, and other constraints that
impede progress in developing a self-sustaining population program.
Information for this section is summarized from data contained in the
Development Assistance Program (DAP),
AID program documents, Sector
Reviews, Area Handbooks, Project Evaluation Summary, the World Bank, and
the UNFPA.
III. The Population Program
Section III describes the stage of population policy development, the
degree of commitment to the policy, and the establishment of demographic
goals.
The national program is summarized—its administration, the rationale
under which it operates, and the participation of other Govern[Page 1020]ment ministries and
organizations such as education, agriculture, and health in population
control activities.
Subsections cover the role of AID,
donors receiving AID support (including
the UNFPA), other donors, and the
host government in the national program (including the percentage of the
national budget designated for family planning activities).
IV. Financial Inputs, Impact, and
Cost-Effectiveness
Section IV consists of three subsections:
A. Annual per capita funding from 1965–70, and subsequent years through
1976, by the category of donors and the host government. Total funding,
per capita, and the percentage of financial input by each to the overall
national program are summarized. Details are reflected in Appendix
B.7
It is recognized that measuring resource allocations to family planning
(FP) programs by the host government
is difficult, because resources come from both direct and indirect
allocation of funds and in the form of infrastructure support. National
budgets do not reflect actual expenditures for FP in many cases. In addition, the quantity of resources
expended on FP activities does not
always reflect the quality of the services. Changes in the pattern of a
host government’s resource allocations can be attributed to a number of
factors which will differ between countries, such as demographic
factors, availability of domestic financial resources and external donor
support, level of development of the health and social service
infrastructure, existence of government policy, and the perceived
necessity by the government for direct FP activity.
Any conclusions on host government resource support are in most cases
tentative, and cross-country comparisons, while necessary, should be
made only with extreme caution. It is hoped that AID will be able to improve on the
uniformity and reliability of the data in the future.
B. The impact of the program is measured in part by prevalence of use—the
percentage of married women of reproductive age (MWRA) or couples using a reliable method
of contraception; the percentage of MWRA/couples with reasonable knowledge of a reliable method
of contraception; and the percentage of MWRA/couples with reasonable access to contraceptive
services.
Data on the above indicators were compiled by a number of the AID Missions last December for the AID Worldwide Population Of[Page 1021]ficers Conference. This performance
criteria effort relies on the ability of the USAIDs to continue to
collect these data on an annual basis.
Conceptually, the best measure of accomplishment of a FP program would be the difference between
the reduction in fertility produced by the program and what
theoretically would have been achieved in the given socio-economic
conditions without any FP program. Even
attempting to estimate this difference would require a much better
understanding than we now have of the relationship between fertility
reduction in a particular country and family planning services and other
socio-economic factors.
In the absence of such multivariate analyses of the determinants of birth
rates, one measure of fertility reduction is provided by the increase in
the use of contraceptives obtained from public and private programs
throughout the country. Based on field experience in selected countries,
a rough approximation of the relationship between the crude birth rate
and use of reliable methods of contraception (i.e., excluding
abstinence, coitus interruptus, and rhythm) is that a two percent
increase in the percentage of couples using contraception is associated
with a decrease of one birth per thousand population, and that a
contraceptive user rate of 60 to 70 percent is associated with a
replacement level birth rate of 20 per thousand.
In measuring rates of contraceptive use, there is considerable variation
among developing country FP programs in
the quality and reliability of FP
service statistics. Many systems generally collect only data on new
acceptors, from which current use rates must be estimated through the
application of method continuation rates to new acceptor data or through
analysis of contraceptive distribution over time.
An increasing number of the developing countries are beginning to use
prevalence of use as a measure of program impact, and AID has initiated programs to assist them
with this activity. Estimates of contraceptive use and births averted
can be improved by better data and more sophisticated techniques of
analysis.
The responsibility would rest with AID
in cooperation with the Office of the Coordinator of Population Affairs,
State Department, to establish guidelines for the missions to estimate
prevalence of use, and to continue efforts to develop new and better
estimates of program impact.
One measure of cost-effectiveness can be obtained by dividing annual
financial inputs from all sources by the number of users. These
cost-per-user statistics (in dollars) are calculated for the last year
in which data are available and as averages of the latest three, five,
and ten-year periods.
A crucial aspect of the analysis of cost-effectiveness is the explanation
of variations in cost-per-user rates. These rates tend to be much [Page 1022] higher in those countries
where social, economic and cultural factors encourage population growth.
For example, in Bangladesh, where rural societies are male-dominated,
village systems are weak, adequate numbers of administrators are
lacking, and rural health and educational systems are sadly deficient,
costs per user are almost certain to be higher than in a country such as
Thailand, where basic social, economic and cultural factors are more
favorable. Hence, any performance evaluation system must take these
essential factors into consideration in judging cost-effectiveness.
V. Critique
The purpose of this Section is to evaluate the overall effectiveness of a
country’s population program, bearing in mind the socio-economic factors
affecting the program, the leadership commitment to the program, the
encouragement, especially through person-to-person contacts, of smaller
families and the supply and delivery of family planning services. To the
extent possible, quantitative estimates are made of the effect of these
factors in reducing fertility. The critique also contains
recommendations for program action and emphasis.
In order to make this overall assessment of program effectiveness,
additional data must be obtained, through such means as better vital
registration systems, sample surveys, and research projects. There is
also the need, especially in key countries, for more sophisticated
multivariate studies to quantify, as far as possible, the impact of
socio-economic variables on population growth and identify more
accurately the degree to which a nation’s family planning program is
responsible for lowering fertility rates.
Global Policy and Program Analysis
These individual country analyses are aggregated into an overall
examination of the global effort to reduce population growth. This
global summary includes an evaluation of the efforts of recipient
nations, the U.S., other donor countries, and multilateral and private
organizations. Total funding for population programs on a regional and
global basis is measured against special priorities and objectives which
have been established and, where possible, progress toward specific
targets is estimated.
The performance evaluations, as improved by new data and analysis, are
applied on a regional and global basis to guide decision-makers with
respect to the performance of US-funded
bilateral, multilateral, and private population assistance. The global
summary will include a projection of demographic factors and estimated
impacts of alternative funding and program strategies.
