Three-Dimensional Impact:
A case study of electronic government

Nick Moore, Acumen

The assessment of impact requires measurement of three dimensions: perspective, extent and intent. Electronic government services can be used to illustrate the scope and nature of these dimensions.

Perspective concerns the point of view of the actors who are affected by the activity. In the case of electronic government we can identify a number of such actors: individuals, corporate bodies, the bureaucracy and the politicians.

Extent relates to the timescales involved. A development like electronic government is likely to have an immediate impact on a small community of actors; a medium-term impact on a growing number of providers and users, and significant long-term behavioural consequences for all concerned.

Intent concerns the degree to which consequences were intended or accidental. This is clearly associated with the aims and objectives of the activity and with the identification and assessment of unintended consequences.

When designing studies to measure impact of a far-reaching activity like electronic government it is, therefore, important to be clear about which dimensions are being measured.

For the purposes of this paper, electronic government can be defined as a cluster of activities concerned with the use of information and communication technology to deliver public services of some kind. The providers could include central or local government or one of the agencies established as service-providers. The services themselves range from simple information provision through to interactive, transactional services.

The impact of a service or an activity can be measured in a number of different ways (See, for example Menou 1993). Whichever approach is used, a full assessment of impact needs to take account of three dimensions: perspective, extent and intent.

Perspective concerns the point of view of the actors who are affected by the activity. Extent relates to the timescales involved. Intent measures the intentional or accidental consequences and assesses the activity in relation to the originally expressed objectives.

The introduction of electronic government provides a case study that can be used to illustrate the scope and nature of these three dimensions.

Perspective

The assessment of the impact made by a service or an activity depends greatly on one's perspective. The view of a provider will be different from that of a recipient. Not only will they perceive the impact through different eyes, the issues that relate to impact will also differ.

In the case of electronic government, we can identify a wide range of actors, each of whom will have a different perspective and for whom different issues will need to be taken into account when measuring impact.

Individuals

Individuals, as recipients of electronic services, clearly have an important perspective on the impact the services make. Individuals will view such services as consumers. They will also have a more complex association with the services and their delivery in their role as citizens.

When making their assessment of the impact or effectiveness of the services, individuals are likely to adopt a comparative approach, making comparisons with electronic services provided by the private sector and others. They will also compare electronic services with other, non-electronic government services. When assessing impact, therefore, it will be necessary to bear these benchmarks in mind.

To assess impact from the individuals' perspective, it will be necessary to explore a number of issues. These are likely to include: convenience; perceived costs and benefits; privacy and data protection; security, and equity in the sense that all individuals are perceived to be able to access the electronic service equally.

Corporate bodies

Corporate bodies will have a slightly different view. They are also recipients of services but for them the activity is likely to be regarded more as a business transaction. They, therefore, are likely to make their comparisons with other business transactions and with their other dealings with government.

For this group, the critical issues are likely to focus around: speed and convenience; reliability; security; cost, and the compatibility of the government's electronic systems with those of the firm.

Civil servants

Civil servants bring the perspective of the service provider. Their benchmarks will be whatever has gone before in terms of service provision as well as their experience of other forms of electronic service delivery.

They will be assessing the ways in which the service affects costs, especially capital costs and the affect on other budgets They will be concerned with the organisational and structural impact of the new services and with the scope the new services offer for doing things differently. For them, relationships will also be an issue, particularly the affect on the relationship between governors and the governed and between the service providers and private sector suppliers. They will also be aware of data protection and security.

The government

The government itself will have a view as the manager of the government machine. Ministers will make comparisons with both the previous levels of service delivery and, increasingly with what is offered by the private sector.

The government will be concerned with a number of issues. First will be the impact on the wider political agenda. The present government will, for example, be interested in the impact on social exclusion and on the modernising government agenda. They may also question the potential offered for addressing the 'wicked issues' that transcend departmental boundaries. They will almost certainly be interested in costs and the potential for savings. And they will be aware of the issues surrounding openness and security. Finally, they should have regard for the impact of the electronic services on democratic structures.

The IT supply industry

In the present context it is inconceivable that the government would launch electronic services in anything other than a partnership with the private sector. It is, therefore, necessary to take the perspective of the IT supply industry into account. This is an industry looking at a big prospective market.

They will be making comparisons with other market sectors and, given the multinational nature of the players in the industry, with developments in other countries.

For the industry, the issues will be associated with the speed of market growth, their likely market share, the level of initial pump-priming investment they will be expected to make and with their ability to deliver the agenda set by government.

A plethora of perspectives

From this, it is clear that there are many players involved in this activity, each of whom brings a particular point of view. These perspectives will be shaped by experience and will be concerned with different issues.

For electronic government, as for most other services, it is possible to identify three main groups of player: the users or recipients of the service; the providers and the wider society or community within which the service is offered. Each of these, however, may have important sub-groups which need to be taken into account.

Before attempting to measure the impact of electronic government, it would, therefore, be necessary to identify the different players, to establish what they will use as benchmarks and points of comparison and to explore what, for them, are the important issues that need to be considered in any assessment of impact.

Extent

The second dimension concerns the extent of the impact, or the timescale over which it is to be measured. Here it is usual to consider the immediate, medium-term and long-term impacts. In our case study of electronic government this might reveal the following.

Immediate

The immediate impact on the user of the service will be associated with the time saved or lost as well as the convenience or inconvenience of using the new form of service.

For the provider, the immediate impact will be measured by the extent of any cost saving or any increase in costs. Associated with this may be a number of other measures, such as the accuracy of the service compared with what went before.

Medium term

In the medium-term, users may develop, through their use of the service, greater awareness or understanding of electronic delivery or government service. They may also become more or less receptive to electronic services.

The medium-term agenda for providers will concern cost reductions or increases; system developments and organisational change both within the providing department or agency and in its relationships with other providers.

At the societal level, the government may be concerned about the impact of a reduction in the number of fixed-point or physical services that results from the take-up of electronic services. This could be part of the wider agenda, focussing on issues of social inclusion and exclusion.

Long-term

As Keynes observed, in the long-term we are all dead. But before that, there may be time to arrive at an assessment of impact.

For users this will concern behavioural change and the relationship between them and the state. Providers will need to assess new cost structures; new organisational structures, and new development paths.

Societal impact will be associated with the changed relationship between citizens and the state, structural changes within the executive and, once again, with the wider political agenda.

Ripples in a pond

At one level, measuring impact over time is like watching the ripples that appear when a pebble is thrown into a pond: the longer the timescale the wider and the more diffuse the impact becomes.

At a deeper level, however, other changes take place. More actors are affected by the service. They may not experience the service first hand but this does not necessarily mean that the long-term causal effect is lessened - I have never eaten a McDonalds hamburger or bought a National Lottery ticket, yet my life has been changed by both institutions.

It is also the case that the issues for any particular actor change over time. What may be a pressing issue at the beginning of a service or a process may cease to be important as the service matures.

We, therefore have a two-dimensional matrix. On one axis are the different players each with their benchmarks and agenda of issues. On the other are the different timescales that can be used to measure the immediate, medium-term and long-term impacts of the service.

Intent

The third dimension concerns the degree to which the service outcomes were intended or accidental. The starting point has to be an assessment of whether or not the service achieved its objectives.

This focuses attention on the objectives. To assess impact it is necessary to know what they were; over what timescale they were to be achieved, and the extent to which they were, in fact, achieved.

The objectives of electronic government service are likely to be associated with efficiency and the opportunity offered to reduce the cost of provision; accuracy and speed; convenience for the user, and possibly, the opportunity to deliver better provision to those who are unable to access fixed-point services easily.

There may also be a second-level agenda associated with wider political objectives such as the reduction in social exclusion; the desire to modernise government or to break down departmental boundaries.

To leave things at that, however, would be to assume that all outcomes were intentional. Unintentional outcomes are almost inevitable and need to be taken fully into account. Some may be beneficial, others could be harmful. In sum, their effect could outweigh the effect of the intended outcomes.

The introduction of electronic government services, for example, could result in the withdrawal of fixed-point services, thus increasing the degree of social exclusion among those unable to use electronic services. Equally, they could build up popular acceptance of electronic service delivery thus opening the door to many other service providers.

It is, therefore, Important to be clear about the purpose and objectives of the service while bearing in mind the need to identify and assess the unintended outcomes.

Conclusion

We can therefore add a third dimension to our matrix. This would identify the intended and unintended outcomes for each player within each timescale.

The case study selected to illustrate these three dimensions is one which has far-reaching implications. It is not, however, atypical. The diffuse nature of information and its pervasive reach within modern society mean that there is usually a range of different players associated with any particular service. The full impact of information is seldom felt immediately. In most cases it is necessary to look for immediate and longer-term effects. And it would be naïve to assume that we should only concern ourselves with the intended outcomes of a service when measuring its effectiveness.

It does rather go to show why measuring the impact of information is such a devilish difficult thing to do.

References