Wrap-up: August 2014 Discussion

Publish Date: 
Wednesday, August 27, 2014

Many thanks to everyone who participated in our lively discussion on the ‘pros and cons’ of indexes as a tool for monitoring. Indexes—composite scores, that are calculated by combining multiple indicators—are becoming an increasingly popular tool in rights advocacy, so it was a timely topic!  

There was wide agreement that promise of indexes lies in their potential for simplification, aggregation and comparability. However, several comments also noted the complexity that gets hidden behind the number.

Selecting the indicators that make up an index is the result of trade-offs. It’s not feasible to be 100% comprehensive and include everything. At the same time, prioritizing some indicators and leaving out others might distort the final index. The risk is that sometimes indicators get prioritized simply because they’re easier to measure, not because they really reflect the issue the best. In particular, qualitative information—which is important for giving context—is often not included in indexes.  

A couple of comments noted lack of uniformity with index methodologies and the actors who create them. For example, despite their popularity in the field of corporate accountability, there is little guidance on how to calculate reliable business and human rights indexes. Without sound and transparent methodologies, indexes run the risk of misinforming policy efforts.

Others raised issues around the politics of developing and using indexes. For example, what legitimacy do big international institutions have to calculate indexes? How does that impact the way governments respond to them? It was also pointed out that indexes can be used to praise, as well as critique governments. On the one hand, this means they can be a constructive, not only an antagonistic, way to engage with governments.  On the other hand, governments might be tempted to prioritize 'shortcut' policy changes that would improve their index score but not meaningfully improve things for people on the ground.

There were several suggestions for how to minimize cons and maximize pros of using indexes:

Some related to how indexes are calculated, such as: recognizing the multi-dimensionality of human rights abuses— severity, frequency, range; employing wider scales (e.g. 0-100 instead of 1-5) to capture changes in scores; or narrowing down an index to highlight a particular aspect of a right, rather than trying to capture it completely.

Others related to the process of developing indexes, such as ensuring that debates about selecting indicators and calculating indexes is inclusive. Without the participation of various stakeholders, including affected individuals, the validity and impact of the index will be compromised.

Others related to how indexes are presented. Their methodology should be clearly described to readers in a short, plain-language overview, for example. The ability to visualize an index online was another way to make it more accessible for readers.

A number of examples of indexes were also shared, which we’ve added to our resource library.

Allison Corkery
Working Group(s):