Impact Assessment Program

ACIAR has, from its beginning, placed significant emphasis on assessment of the impact of the research it funds, particularly focusing on quantifying the returns to research investments. It has used these assessments to account to stakeholders and to support improved decision making and management of its funds. ACIAR has not only applied the extensive body of existing literature to this area of investment analysis but also contributed to this literature through this process.

In ACIAR's early days, quantification of potential impacts were used to support aggregate priority setting and more effective project development, as well as enhancing the Centre's public accountability. As research efforts matured more attention has been focused on quantifying the returns on these investments by measuring impacts and adoption.

The Impact Assessment program currently commissions three types of finished project assessments. The first are primarily economic evaluations, which are published in ACIAR’s Impact Assessment Series (IAS). Most of the assessments are undertaken by independent economists with special expertise in measuring the impact of agricultural research. These involve an in-depth analysis of the adoption and impact of project outputs in the partner countries and Australia. The impact assessments provide estimates of the returns to the investment in the research area of interest. A qualitative assessment of social and environmental impacts is also sought. In recent years the focus has been on thematic evaluations of larger numbers of projects in similar areas and overviews of all past impact evaluation studies and an assessment of returns to total ACIAR funding.

As part of the evaluations, areas for practical methodology development have been identified and advances developed. Examples are methods for measuring the impact of agricultural research on poverty and the returns to capacity building and research in the area of natural resource management.

Recognising the complex contextual environment in which ACIAR operates, the second type of finished project evaluations are the set of adoption studies. The adoption studies are primarily undertaken to provide ACIAR, and our project leaders, with a greater understanding of the pathways to change. They are undertaken by the Australian project leader, 3 to 4 years after the completion of the project and provide ACIAR with information on the difference the project has made at the scientific and community levels in the partner countries and Australia. If uptake of the project results has not occurred, then the reasons why are sought. An increase understanding of the contextual environment in which we operate increases the likelihood that our research will have a positive impact.

In a similar vein, the third type of finished project evaluations is what could be termed 'impact pathway analysis'. Using an impact pathway framework as an evaluation tool primarily involves tracing the pathway to change from research outputs (the deliverables), to outcomes (use of the deliverables by the next and final users), to impact (the ultimate change in social, economic and/or environmental conditions that occurs with widespread adoption). Impact pathway studies provide an in-depth analysis of the role of the key stakeholders along the causal chain and the assumptions about what has to happen for the research outputs to be adopted and make a positive and sustainable difference. In addition, the perceived risks and other external factors influencing the expected outcomes and impacts are made explicit. Further, unintended effects and rival explanations should be assessed.

In addition, the experience gained through these activities has been used to provide training courses for research groups so that eventual impact and adoption is an integrated part of research project design and management. Emphasis is also placed on developing collaborative networks with Australian and partner country practitioners responsible for impact evaluations, and building capabilities to undertake robust analysis. These activities help improve the accuracy of the information used in assessing the impacts of the research and the effectiveness of the methodology used to quantify the returns on investment. ACIAR has also worked closely with institutions, such as IFPRI, to develop comprehensive, consistent software for undertaking these evaluations and has used these to train collaborating partners.