In such a brief summary, one cannot hope to capture all the significant issues that were addressed in these studies, but I do hope that what I say will serve as an appetizer, and I invite you to participate in the full course as published in the Monograph, edited by Woody Horton: Defining and Assessing the Impact of Information on Development, Building Research and Action Agendas.

I will begin with a brief description of each study, followed by a summary of lessons learned in the process; and end with some thoughts on priorities for future research.

I. Brief description of the eight studies

1. INFORD – Information Provision for Rural Development, Phase II (Africa)
2. SATCRIS – Regional Network Serving an International Research Community (India)
3. CABECA – Regional Information Infrastructure: Electronic Communication (Africa)
4. REDUC – Regional Network Serving Senior Policymakers (Latin America)
5. ALIDE – Assess the Impact of Information and Telecommunications on Policy Formulation (Latin America)
6. CARICOM – Information for Decision Making in the Caribbean Community (Caribbean)
7. ISER – Assessing the Impact of Information on Policy Formulation in the Caribbean Caribbean)
8. LISREL – Measuring the Impact of Information: A Path Analysis Approach (China)

- we have discovered many things, we know a lot more now than we did at the beginning. but the more we learn the more we discover that we need to probe further; and to incorporate other factors;

- we have learned many important lessons, on how to do the research, but we have found no fool-proof recipe, no one-size-fits-all model to determine impact, or to lead to development. “We now see that there is no straight-line cause and effect; things are connected by webs”. (Jane Jacobs)

- we know that just as we can quantify ‘economic growth’ but still argue about whether this means ‘development’, so we can identify inputs of information and inputs of expenditure on information systems, and see outputs in terms of number of computers per capita, or trained personnel, etc. but still be hesitant about the outcome of this activity in terms of benefit to the society.

I’ll now highlight some of the findings of the studies in so far as they give indications of what further work needs to be done in the future.

Findings

1. Despite the difficulties encountered, (and not yet overcome) the findings are an affirmation of the approach adopted in the Preliminary Framework, notably the use of indicators to assess performance and impact; the focus on outcomes, as opposed to just outputs; and the adoption of both quantitative and qualitative approaches. The value of using the concept of impact indicators to measure impact was modified to the extent that it was inadequate to use mainly subjective measures to derive the indicators.. The indicators developed in the measurement framework were based on perceived benefits, as declared by the users; but objective measures were also needed to accompany and verify the subjective assessment. A systematic search for objective indicators is a priority activity.

2. Responses to the use of information and subsequent action are related to internal, mental processes. In the studies we were trying to capture the outward and visible signs of internal processes taking place after information use.

Methodologies for assessing the impact of information on any outcomes therefore need to be capable of reflecting human behaviour.

3. Information and access to information was viewed by users as a very important resource in the development process and in the decision-making processes involved in development activities. This is not just the basic assumption of researchers, or the credo of information professionals in the various countries, in their on-going drive to make information a more potent force in development activities. This is the word from the field of users in those countries, for example

- decision makers themselves who identify specific bodies of information that are crucial for them; access to ideas that made a difference to their thinking;
- officials in agricultural extension; health care; mining, who tell how access to specific kinds of information was used and how the absence of specific information impeded their work, caused material loss, or led to guess work and less than optimal decisions;
- people in the political directorate who know they must now be informed and knowledgeable, even as they point out that formal information presented is not the only factor they have to take into consideration in making their decisions.

The studies showed information was considered a valued asset. The debate in assessing impact comes over what kinds of information are needed, or provided; the need for ‘intelligence information’ more often than raw data, factual ‘how-to’ information, analytical information, and ‘coping’ information. Reflections on content of information and at what cost.

4. A need for clearer definition of the concepts and the adoption of working definitions to allow for greater consistency among researchers when working with such concepts. Precisely because we are dealing with intangible variables, definitions are all the more important. In particular, there needs to be, at the outset, a clear identification of what is the nature of the ‘outcome’ being observed and definition of its components.

5. Qualitative assessments of impact were found to be more revealing than the sparse quantitative results achieved empirically.

But conceptually too, it was found that we need more insight into how people function with or without information. This is not readily obtainable by questionnaire . There are so many differences in how people think about information and how they use information that we feel that only extended or repeated personnel interviews can provide adequate data. (Meadow)

6. Was it possible to measure the impact of information on development?

The LISREL Path Analysis approach was successful in demonstrating that it is Feasible to measure quantitatively the impact of information on development, illustrated in this case by the success of small businesses in Shanghai, China. The methodology proved effective when tested in this particular environment, and the model has potential for broader application, where it may provide illumination of the extent of the contribution made by information in well-defined settings. (Such clearly delineated settings did not exist in the seven case studies.) But LISREL is a very demanding model; it requires clear specification of input and output variables (specification of desirable outcome variables was a major stumbling block in the seven case studies); and a large number of observation points. Adequate resources (time, personnel, technical backup, funds) are essential for ensuring these requirements can be fulfilled. LISREL Path Analysis is a research tool that can help advance our knowledge in the impact field. However, it is not an evaluation tool that can be applied routinely in assessing the impact of an information service.

III. Future work on Impact Assessment

As Martha Stone, the ‘guiding light’ for the study, said in the Foreword, we are still left with the original ‘So what’ question unanswered. So there is still work to be done. But what work? What is the nature of the further work needed?

1) Research 1. Developing and calculating indicators of impact. The impact of information on development cannot be directly observed or measured. Consequently, methodologies for assessment of impact are most likely to involve the use of indicators as surrogates for direct observation. Identification, definition and testing of indicators are critical tasks in furthering the impact assessment methodology. An imaginative search for objective indicators is required.. There is room for collaborative work with others both within the community of information professionals and without; and for drawing on the experience of other social scientists.

2. Address the need for inter-disciplinary approaches, and the need for a dynamic model of assessment. ( e.g. from the discipline of psychology, Jane Klobas and reference to the Theory of Planned Behaviour; Warren Thorngate and ‘affordances’.)

3. Production of a practical handbook or ‘how to’ manual that could be used as a guide for all impact study planners. Along with this, a comprehensive inventory of impact assessment should be prepared, comprising other efforts undertaken and results obtained.

4. Development of a training program. One imaginative way of building awareness of the possibilities of impact assessment would be to develop a training program for use in schools of Information Science. The program should include guidelines for applicants and donors to incorporate impact assessment in planning and funding.

2) Development Address information content needs vs. information technology needs.

• ­ In Latin America and the Caribbean local and regional information is needed; lack of information about their own societies hinders decision-making for development;
• ­ not enough cross-checking of information put in IT format with information that users really need;
• ­ address ways to fill these gaps in order to reap the benefits of IT in terms of greater access to information about the local and regional societies.
• ­ Two aspects of the gap
• ­ non-existing basic data: e.g country laws and regulations; taxes; prices
• ­ existing reports to government, white papers; and research documents
  (Refer: case study ISER and shortage of supply of input, ‘information’.)