AGGIS: Monitoring systems of good governance

The aim of this first draft report on the role of monitoring and indicators, prepared by the research team from the University of Ljubljana, Faculty of Social Sciences is three-fold:

• To give a general theoretical overview of the role of monitoring and monitoring systems on the basis  of which would be possible to tailor the monitoring alternative for the AGGIS project purposes;

• To present the so called soft-law tool of the EU, called the Open method of co-ordination in which the monitoring-like approaches as described through theoretical introduction of this report can be traced in practice;

• To present a general overview of the 3 already established and world-wide applied system
  broad  approaches  for  monitoring  governance  issues,  with  a  future  aim  to  select  those indicators/variables that could possibly be directly used for the AGGIS monitoring purposes.

THEORETICAL INSIGHT

WHY MONITOR

Each organisation, institution, as well as state and all the sub-systems that embodied them strive to get the feedback information about their making. This feedback information represents the basis for their own future  attitudes and orientations, as well as for the attitudes and orientations of the environments towards them. These reasons are the crucial ones for why public pursuit of the already existing  and  implemented  practices  and  patterns  in  democratic  societies  and  institutions  is  of fundamental importance.

Monitoring is a special analytical procedure used to produce information about the above stressed, about the results of the work of organisations or policies that they implement – either in private or public sector. As such monitoring is regarded to be one of the crucial procedures that is supposed to provide information about the performance of the work of the organisation or its policy, be it from the  perspective  of  the   organisation’s  resources,  processes  (actions  and  activities),  and  the perceptions of the wider environment in which it operates.

Based  on  the  described  broader  mission  monitoring  performs  at  least  four  major  functions: explanation, accounting, auditing and compliance (Dunn 2004: 355-356):

1. Explanatory   function   of   monitoring   yield   information   about   the   outcomes   of   the implementation, it can help explaining why the outcomes differ or are such as they are;
2. Accounting function of the monitoring process is important for delivering the information that can help in accounting various changes that follow the implementation of a process or policy (e.g. social, economic, environmental etc.);
3. Auditing function of monitoring enables to determine whether resources and services that have been targeted to the beneficiaries or certain target groups have actually reached them;
4. Compliance – monitoring in the case of the function of compliance helps to determine if the processes, activities and resources, staff, and other involved are in compliance with the standards and procedures that are defined in advance either by the organisation itself or external environment(1).

Due  to  the  exposed  functions  also  a  set  of  specific  aims  and  expectations  for  monitoring  the implementation of work of organisations and their policies can vary, and be as such synthesised into three  of   them that can have either 1) internal organisational motives, either 2) external environmental motives or 3) both of the motives:

• Monitoring  as  the  operational,  managerial  procedure  which  through  information  and evidences provides the feedbacks on the performance;

• Monitoring as a necessary prerequisite procedure that enables further assessment of the impacts  of implementation for the past and future state of the affairs, and further on the platforms for policy learning and potential introduction of policy changes;

• Monitoring  as  the  procedure  that  provides  the  information  about  the  impacts  that  the implementation  of one  organisation  and  its  making  have  on  system’s  wider governance practices, norms and values, such as democracy, transparency, human rights and well-being.

According to the exposed monitoring of organisations’ implementation is supposed to have two main missions:

1. To give the ex-post or feedback information about the characteristics of already or currently implemented work and activities that have been developed and undertaken in previous periods;
2. To give ex-ante platforms for the planning of the future implementation activities, which fundamentally refers to the need of evaluating past implementation practices with the aim to decide about their future destiny.

WHAT TO MONITOR

Parallel to the exposed it is especially important that a set of fundamental issues that need to be covered and monitored on the basis of the monitoring motives, mission and applied procedures is clearly  set.  Usually  each  framework  of  each  implementation  and  further  also  its  monitoring  is supposed to give the  answers to the following sets of questions (Chase 1979), which gives the information of the organisations’ democratic, transparent, accountable governance outlook:

• who are the people to be served and who are serving,

• what is the nature of the services to be delivered,

• what are the potential distortions and irregularities,

• is the implementation controllable (e.g. can implementation be measured).

Based on the exposed functions, expectations, motives and aims for the application of monitoring procedures, the next crucial challenge is to decide which type of data needs to be collected and which methods, procedures, apply according to those in-advance set needs for applying monitoring system. In this sense the crucial task is to decide which type of data is needed for those purposes and how  to  contextually   define  the  issues  that  need  to  be  monitored.  Mostly  the  authors  of implementation studies (see  Hogwood and Gunn 1984; Parsons 1999; Hill and Hupe 2002, Dunn 2004) classify the contents of the data that are needed for monitoring the implementation practices into two crucial categories which all relates to at least one of the four types of indicators according to their relevance:

1) input,

2) process,

3) output and

4) impact indicators.

Macro data category relates to the characteristics of wider system environment characteristics, e.g. to  the   broader  context  of  political,  social  and  economic  environment(s)  in  which  individual international sport  organizations are established and operates. This category partly covers/overlap with input and impact indicators and represents necessary precondition for in-depth monitoring of good governance. This category mainly relates to the data on

• Regime / Legal type and status of the state: type of democracy or type of legal status, legal basis/origins, elections and election rules,

• Economy of the state,

• Social welfare index,

•  Perception of corruption and transparency.

Micro category relates to the prevailing characteristics of individual organisation, its processes and work.  This  category again consists of a combination of all four types of indicators (input, process, output and impact) and relates to mainly the following:

• Institutional structure  characteristics: legal status, elections and election rules of organization’s leadership, structure of the leadership, structure of the membership, year budget, number of employees in the ISO etc.

• Processes  characteristics:  general  internal  decision-making  rules,  procedures  and practices

• Project  and  policy  characteristics:  data  on  the  implementation  of  the  concrete programs, projects 

• Cadre resources:  number and profiles of the employees (full-time, part-time, voluntary, gender)

•  Financial  resources:  data  that  relates  to  the  relevant  budget  aspects, including both operation of organization itself as well as implementation of concrete programs and activities

• Other relevant data: sources of knowledge etc.

HOW and WHO MONITORS

The data gathered for the purposes of monitoring performance mostly come from two sources:

1. Some data already exist, and are either: a) already available since they have been gathering for other  purposes (like the monitoring of the profiles of the states) and can thus be just extracted   from   the   existing   data-sets,   already   calculated   indexes   (like   Transparency International,  World   Governance  Index  and  Global  Reporting  Initiative);  b)  are  being gathered for the internal organisational purposes and are not publically available although they exist;
   
   
2. Data are not yet gathered. In this regard the data needs to be conducted mostly through
   applying the following methods:

• Review of the relevant already existing documentation and data: statistics, financial, policy documents

• Surveys

• Interviews

• Focus   groups,   panels   and   similar   methods   for   gathering   the   perceptions   on   the implementation practices

Further  on  mostly  the  data  that  are  relevant  for  the  implementation  and  which  performance supposed  to be monitored are defined in the so called codes of conducts, organisational/policy guidance, guidelines, standards etc. (see for example IFAC at  http://www.ifac.org/).

NOTE:

(1) Here we need to differentiate between policy and legal compliance, where the former relates to the question of how extensively the normative standards are being considered in the actual, day-to-day policy implementation, while the latter relates most often to the question of the formal acceptance of the agreements/standards.

Read the full report "MONITORING SYSTEMS OF GOOD GOVERNANCE"

---

This report was created as a part of the project 'Action for Good Governance in International Sports Organisations (AGGIS)', which was initiated by Play the Game/the Danish Institute for Sports Studies and awarded funding from the European Commission to contribute to the Commission’s so-called ‘Preparatory Actions’ initiative which will pave the way for the EU’s future strategies in the field of sport. Read more about AGGIS here