Society Catching Up with the Technology: Assessing the Impact of Information and Telecommunications on Policy Formulation

Fay Durrant
Senior Program Specialist
International Development Research Centre (IDRC)
Ottawa, Ontario K1G 3H9
Canada

This project is particularly aimed at researching the issues and questions that relate to the impact of electronic networking on the access to and use of information. It is aimed at analyzing the role of information and the use of electronic networking in social policy, sustainable enterprise production, environmental aspects of sustainable development, and information policies for electronic networking. Project activities will contribute to the regional dialogue on information policy formulation and to increased capacity for information policy development.

The research focus responds to the identified need for the development of national and institutional policies relating to electronic networking. Currently in the Latin American and Caribbean region, there are increasing numbers of new Internet connections, and the potential for use by development institutions is significant. On the other hand, it is increasingly difficult for policymakers and managers of these institutions to determine the practical relevance of such wide-ranging developments or to determine the truly important uses and trends amidst all the commercial and media hype.

The region also needs to have the capacity to develop policies for universal access to the Internet and to similar facilities. This project's research will facilitate the definition and implementation of institutional policies for electronic networking and Internet access in the participating organizations. It will do this by demonstrating to policymakers and decisionmakers the role and significance of electronic networking in the process of access to information for policy formulation and decisionmaking in the region.

The methodology recognizes that the information society can be considered as having three components: the technology on which the systems are based; the products and services produced; and, most important, the benefits and social and organizational changes that will result. Advances in telecommunications can also be expected to provide users with the right infrastructure and the possibility to access, review, and use large amounts of data. This is recognized by the proponents as having the potential for creating new opportunities for individuals and institutions.

The potential number of applications of electronic networking is so great that it has tended to obscure some practical considerations. While the Internet has already shown that there is a vibrant demand for electronic content and services, the utility of many innovations is unclear to many potential users and policymakers. The utilization monitored and the impact measured by this research will be used as the basis for demonstrating to policymakers the utility of many of the innovations and to help identify the areas in which these services can result in maximum benefits to their institutions and to society.

The project is therefore designed to provide policymakers and decisionmakers with the basis for assessing and understanding the role of these technologies in the context of their own practical experiences in the processes of policy formulation and decisionmaking.

The methodology consists of four main elements:

The implementation of the methodology will benefit from the existence of the IDRC-supported global network on the impact of information on development and the Preliminary Framework for Impact Assessment. In particular, the methodology will use a modified version of the quantitative and qualitative analysis of the assessment indicators defined by Noel Boissière under the project, Assessing the Impact of Information on Policy Formulation in the Caribbean.

Although this methodology has been defined, it still needs to be tested in the environment of the participating user clientele. The assumption is that the basis of the methodology will require some adjustments at the outset and will be usable for the case studies and the other aspects of the research.

It is recognized that there may be technical difficulties for some participants to quickly access the information services, and these will need to be evaluated periodically by the network managers and the coordinator to ensure that the technical problems can be solved during the life of the project.

Research on the use of electronic communication raises some ethical questions that will affect the level of monitoring permitted. For example, while it is generally permitted to review participation in electronic conferences and in accessing the Internet, similar monitoring of the contents of electronic mail will not be permitted. Consequently, the evaluation will also depend on the users' consistency in recording their own use of the systems.

Research has shown that gender is a factor that can influence the optimal utilization of electronic networking and other information and communication technologies. The project will deal with this issue mainly by ensuring that there is fair representation in the target group and by incorporating the training materials used by the APC Project on Gender and Information Technology in the training programs for the target group.

The research design has been based on the identification of some specific problems, including:

The above problems signal that there is need for research and communication activities that can demonstrate the role of electronic networking in increasing access to information. In this project, the research will focus on identifying, with the participation of policymakers and decisionmakers, the impacts and potential impacts of these media. This research will demonstrate the role of electronic networking in increasing access to information and the subsequent use of this information for policy formulation. The process will also improve the linkages among the players in the information sector and those in other fields.

The Network of Networks--Latin America was convened in 1990 as a result of a series of consultations among representatives of 18 regional organizations that function as information providers. The project, which was supported by IDRC, was recently completed. It has been successful in capitalizing on the experiences and specializations of the regional networks. With the coordination of Asociación Latinoamericana de Instituciones Financieras de Desarrollo (ALIDE), four working modules were developed to guide the repositioning of the networks and adjustment to new challenges. By implementing the modules on a shared basis, four lead networks spearheaded use of computer-based communication (by AlterNex/Instituto Brasileiro de Análisis Sociales Económicas (IBASE)), training in new issues in information management (by Comisión Latinoamericana para América Latina (CEPAL) / Centro Latinoamericano de Documentación Económica y Social (CLADES), training in marketing of information services and products (by Centro de Investigación y Desarrollo de la Educación (CIDE/REDUC), and decentralized access to the databases of the networks (through the cooperative Red de Redes CD-ROM produced by Centro Latinoamericano y del Caribe de Información en Ciencias de la Salud (BIREME)).

As a result, the participating information specialists and users have been able to improve communication and access to information on a decentralized, multidisciplinary basis. Of particular note is the fact that the training and orientation provided by AlterNex stimulated the regional organizations to utilize computer-based communication and Internet services for communicating with users and for delivering information products and services. At present, all the participating networks use electronic mail and computer conferencing for inter- and intra-network communication; and in addition, six of the organizations have established points of Internet access with Gophers and Web sites.

The participating networks will, therefore, provide the framework for the proposed research activity. It will first demonstrate to the key users identified in the research the effects of using electronic networking for accessing information and, as a result, will measure the impacts on policy formulation in the regional priority areas of (1) social policies for participation in decentralization; (2) productive enterprise development; (3) sustainable aspects of environmental management; and (4) information policies for electronic networking. The framework for the proposed research consists of the following:

Objectives

The general objective of the project is to assess the impact of information and telecommunications on policy formulation in the following regional priority areas:

The specific objectives of the project are as follows:

Methodology

The participating networks will work under the coordination of ALIDE to implement the objectives of this research project in an integrated manner. ALIDE will be responsible for the management of the research and for the coordination of the inputs of the networks and the consultants.

It was agreed at the evaluation and project development meeting in Costa Rica in August 1995 that the focus would be on priority areas; the networks will, therefore, be working in groups according to their own program areas. Each network will undertake its own case study, and the results will be coordinated by ALIDE for productive enterprise development, in collaboration with Consejo Latinoamericano de Ciencias Sociales (CLACSO) for social policy, and Centro Panamericano de Ingeniería Sanitaria y Ciencias del Ambiente (CEPIS/REPIDISCA), for environmental management.

Consolidation of data on the information needs of the target group

Current users of the networks' products and services who are involved in policy formulation and decisionmaking at local, national, and regional levels will form the target group. Each network will identify 20 users to compose the target group and to participate in the research.

The network managers will initially organize and consolidate the information that they have gathered on their clientele and their information needs. The managers will undertake detailed analyses of the 20 selected users, their information needs, and their information-seeking behavior, to be updated at intervals over the life of the project.

These users, some of whom participated in the seminars on information management mounted by CEPAL and CLADES in the project, Network of Networks--Latin America, have been identified by the networks to participate in the research. They will be further trained in the use of electronic communication, conferencing, and Internet access; and they will be required to provide feedback on the impact of their information use on their activities of policy monitoring, policy formulation, and decisionmaking.

The methodology for the study of the users' needs will build on that used by Institute of Social and Economic Research of the University of the West Indies, in the IDRC supported project, Assessing the Impact of Information on Policy Formulation. A modified version will be prepared by a consultant and discussed on the electronic conference in preparation for its implementation by the member networks.

ALIDE and the networks, with guidance from the consultant, will analyze the survey results and will present this analysis to the participating organizations for incorporation into the strategic planning of each organization and network.

Training of the target group in electronic networking

To ensure that there will be adequate capacity for using and exploiting the benefits of electronic mail, computer conferencing, and specialized Internet services, training courses will be mounted for the members of the target group.

The courses will be mounted locally, at nodes of the Network of Networks--Latin America, the Association for Progressive Communications (APC), or at other local nodes. This will build on the experience of AlterNex and other nodes in training the members of the Network of Networks and other groups, such as the IDRC-sponsored computer conference on decentralization that was mounted on Chasque and other APC nodes.

During the training workshops and the follow-up online sessions, users will be advised of the most suitable connections in their own countries, appropriate modems, and the options of front-end software for efficient connections to the Internet. The computer conference redes.al on AlterNex will be used as the online meeting place for these consultations. This aspect of the methodology will be able to utilize the materials used in the training of the information specialists and some end users in the Network of Networks project. These materials will mainly be those used by the APC networks and CLACSO.

Definition of indicators: Benefits and outcomes and impact methodology

In this context, policymaking will be defined as the determination of broad issues resulting in laws, establishment of organizations or programs, taxation or tax relief, etc. Decisionmaking, on the other hand, applies to actions at a more specific level, relating to decisions on how to implement a policy. There is natural continuity between the two areas; therefore, while policy implementation is expected to be the more active aspect, decisionmaking is still necessary at each stage of moving the policy from a concept to a reality.

Measuring the impact of the information services used over time will enable the target group to record the effects of the access to electronic networking and the ways in which they have used the information acquired in the processes of policy formulation and decisionmaking. These records will enable the network managers to identify changes in the results. The focus will be on benefits or outcomes resulting from the use of electronic networking and use of the information services and products obtained from each network.

Focus groups of users will be held in conjunction with the assemblies and other general meetings of the participating regional institutions. In addition, the network managers and their representatives will interview members of the target group to discuss the project objectives and modus operandi and to receive feedback on the applicability of the indicators proposed. ALIDE, CLACSO, and CLADES have identified as possible meetings their annual assemblies where such interaction with users takes place and where consultations with or training of the target group can be accomplished.

Impact assessment indicators

The research will utilize the assessment approach developed with the support of a series of IDRC-supported projects, which resulted in a "Preliminary Framework for Measuring the Impact of Information on Development." The related consultations concluded that:

The preliminary framework (Menou 1993) suggested several types of indicators. Those related to performance and impact have been particularly selected as indicators, derived as a consequence of use of the information services and related usage (input factors) to outcomes (output benefits). On the basis of consultation with users of the University of the West Indies (UWI) Institute of Social and Economic Research (ISER) project in January 1995, Chambers and Boissière (1995) proposed the following seven impact indicators, which are currently being used in that project:

  1. User satisfaction. Chambers and Boissière (1995) propose that user satisfaction incorporates the degree of satisfaction relative to the investment of effort and money to acquire the information and whether or not the needs were met. This is seen initially as a qualitative measure and will depend on the users' judgments and the anecdotal evidence provided in the evaluation. While the impact of electronic networking will be an important aspect of the benefits and outcomes, the result will also be combined with the assessment of the content delivered and its use in policy formulation or decisionmaking.
  2. Status measures.
  3. Use per capita (frequency of use of services) in the target group.
  4. Number of users in the wider "target population."
  5. Measures related to the content and quality of information received: Improved analysis and decisionmaking; improvement in preparedness, skills, and effectiveness in negotiations; access to information and ideas, contacts with colleagues.
  6. Measures related to the channels by which the information is received: Increased flexibility of access, presentation in more user-friendly formats.
  7. Other measures: Time saved.

These measures will be used in implementing the current project as the basis for discussion with the members of the target group and for developing the set of indicators to be used in the research.

The measurement of impact will utilize a combination of quantitative and qualitative measures. Boissière prepared a matrix for the UWI/ISER project, which will provide the base of the methodology for defining the quantitative and qualitative measures for the evaluation and for translation of the indicators defined to measures.

Channels of feedback for users' responses exist in all the networks, but the channels need to be formalized by the network managers in collaboration with the national focal points of each network. The emphasis will also be on feedback relating to the indicators selected for final application. Each network manager will study the situation and adjust the current procedures as necessary. The proponents recognize the need for separate approaches to assessing impact in the different fields and, consequently, will be undertaking the case studies with a view to identifying the different characteristics of users and utilization in the different fields.

Development of electronic applications of current products and services

On the basis of the experience of the network managers and the consultation with the target group, each network will select among its present products and services and identify those to be delivered to users via the Internet and local networks. These will provide analyses of the state-of-the-art of the priority topics identified for the focus of the project. The products will be modified in accordance with the findings of the user survey and the requirements for electronic publishing as opposed to printed products. These products and services will provide the final definition of information services to be evaluated during the research.

This process will build on the experience of CLACSO in developing state-of-the-art analyses in the areas of governance and democracy, and of CLADES in analyzing the status of decentralization in the region. It is proposed to develop a Web site at IDRC that will provide links to the Web sites of the networks and access to the value-added products and services selected for the evaluation as well as the Red de Redes CD-ROM produced under the earlier project.

Information services are currently provided directly to the target group, and the national focal points ensure that there are local points of access to the regional outputs of the networks in each of the countries. This is the procedure that will continue during the project, but adjustments may be made according to feedback from the users.

Identifying areas of policy formulation and decisionmaking influenced

As part of the case study, each network manager will undertake an assessment to obtain feedback; to collect measurement data; and to identify the links between provision of the new information services and the impact on policy monitoring, policy formulation, and decisionmaking.

The data will be collected according to the set of outcomes, benefits, and measures selected, as outlined in the modified Boissière matrix that will be defined by the consultant in consultation with selected users.

Evaluation and feedback

End of Year 1

Review and adjustment of services. The results of the evaluation will be incorporated into the Internet discussion group that will be mounted to discuss the findings of the evaluation and the impact of the services. The network managers will also hold consultations with heads of organizations during the annual assemblies of each network. This will build on the experience of ALIDE, which has been incorporating sessions on the information network and services at the annual assemblies of the association.

Year 2

End of Year 2

Final evaluation and feedback. The results of the evaluation of the second year will be analyzed by the participants of the project--users and information providers--in an online conference that will also be aimed at disseminating the results of the project to a wider audience.

References

Chambers, Audrey, and N. Boissière. 1995. "Assessing the Impact of Information on Policy Formulation. In McConnell (ed), Making a Difference: Measuring the Impact of Information on Development. Ottawa: International Development Research Centre.

Menou, Michel. 1993. Measuring the Impact of Information on Development. Ottawa: International Development Research Centre.