#CapacityParadox – The Serendipity Brain

The ‘Capacity’ Paradox – part 2

Editor’s note: Continuing the narrative from the previous part, the author delves in the depth of capacity crisis for sustainable development data. He is optimistic that the challenges are going to open up new vistas for national and international agencies to cooperate on mutually reinforcing efforts.

Not grabbing the opportunity for creating the appropriate capacity to address the data gaps, in the opinion of the author, could be as costly a mistake as failing the people of their rights. The narrative ends with a sigh for the dream data remaining elusive.

I presented some pieces of data and a chart in the last part related to some selected countries to demonstrate how the countries have progressed in developing the national capacity for production and dissemination of official statistics. Any intelligent person who knows that one must always look into the elements the data are made of would be reasonably justified to conclude that the countries of the developing world are doing well in gradually improving their statistical capacity in terms of a defined set of parameters.

At the same time, however, what message these impressive ‘score-lines’ give to a layman is a question that may have bothered none. Could it be simply a notion as being understood by a common man that some countries have already crossed the 90% mark and therefore, have not much to achieve further? If so, is it not an impression NSO’s would love to project in unreserved exhibitionism?

Now there lies a fallacy in this notion.

Let’s face it as a question that a common man could ask: Does the score in a true sense reflect a complete image of a county’s statistical capacity? Or, it’s just a fractured image?

One must not miss seeing what the SCI metadata has to say about this by way of explaining the structure, constituents, method, and rationale of the indicator. However, before we are done with the metadata thing, let’s try to understand the fallacy a bit more clearly.

Countries are severely constrained in statistical capacity to meet the challenges confronting the statistical systems as of 2018. Image credit Meghna Chakrabarti

Countries are in fact severely constrained in statistical capacity to meet the challenges confronting the statistical systems as of 2018. By 2016, the nations of the world were already seized with the 2030 Agenda of Sustainable Development Goals (SDGs) that their leaders had committed to achieving in the interest of the humanity and the planet earth.

Attaining the SDG targets under the goals by the year 2030, as they say, crucially depend on the countries’ ability to track the progress with good penetrative statistics. Knowing just the national count of the people to be reached with the good effects is not enough; a much stronger statistical capacity is required to know who they are, where they are located and what challenges they are facing.

A new statistical framework of 230+ indicators has been prescribed for tracking progress towards attaining the targets. An assessment of the IAEG-SDGs of the UN has revealed that as of 13 February 2019: The updated tier classification contains 101 Tier I indicators, 84 Tier II indicators, and 41 Tier III indicators.

This means, over 125 indicators (over 54%) belonging to Tier II, Tier III or multi-tier category have either no internationally established standards or methodology available, or data are not regularly published by the countries. This is a serious capacity deficit and a global syndrome.

Apart from these conceptual and methodological challenges, the overarching SDG principle of leaving no one behind has raised the bar (for the national statistical offices as well as for global monitoring agencies). UNICEF’s report ‘Progress for Every Child in the SDG Era‘ underscores the criticality:

It is no longer enough to monitor progress by global aggregates or national averages alone. Results need to be disaggregated to monitor progress among sub-national groups of people, especially those who are vulnerable such as the girls, children living in remote rural areas or informal urban settlements.

Children and Gender equality are central to the whole of the SDG agenda. 44 child-related indicators are situated under the 17 SDGs. Analyzing these indicators, UNICEF’s report maps them thematically into 5 dimensions of children’s rights:

the right to survive and thrive (17 indicators under SDG 2 and SDG 3)
the right to learn (5 indicators under SDG 4)
the right to be protected from violence (10 indicators under SDG 5, SDG 8 and SDG 16)
the right to live in a safe and clean environment (10 indicators under SDG 1, SDG 3, SDG 6, SDG 7 and SDG 13)
the right to a fair chance / to have an equal opportunity to succeed (4 indicators under SDG 1)

The report reveals that data are not available for each of these dimensions in substantial proportions (of the 202 countries covered): 22% missing the data for the dimension ‘survive & thrive’; 63% for the dimension ‘learning’; 64% for the dimension ‘protection’; 24% for the dimension ‘environment’; and 63% missing for the dimension ‘fair chance’.

If we look at the gender responsive indicators of the SDGs (there are 54 in number spread over 12 of the 17 SDGs), the story is no different – “only about 26% of the data necessary for global monitoring of the gender-specific indicators are currently available”.

When this is the picture at the national level, the stories of non-availability of data at the sub-national levels for the child- or gender-related indicators, or for any other sub-populations/groups of people are obviously all the more disquieting, not to speak of the untrodden areas of environment, climate, life below water and the like affecting livability on earth.

So, a person struggling for the SDG data would wonder whether the 2018 SCI score is any indication of the actual current statistical capacity of a country or what? As its structural constitution is defined for a pre-SDG framework and not redesigned post-2015 to account for the data on SDG indicators, SCI in the present form just reflects a statistical capacity for an incomplete basket of statistical deliverables.

The varying degrees of challenges confronting national statistical offices with the advent of the SDGs in not being able to produce the required data, especially disaggregated data, for the lack of capacity could be somewhat fathomed if realistically assessed statistical capacity were known overall for the countries and for the constituent dimensions. A warrant for action is already spelled out in what Target 18 under SDG 17 states:

By 2020, enhance capacity-building support to developing countries, including for least developed countries and small island developing States, to increase significantly the availability of high-quality, timely and reliable data disaggregated by income, gender, age, race, ethnicity, migratory status, disability, geographic location and other characteristics relevant in national contexts.

The target essentially calls for action on the part of development agencies which will eagerly seek information on the intensity and depth of capacity deficit in the face of the SDGs across the world to assess how much effort and energy they may have to apply and where.

The SCI, if re-engineered taking into account the SDG indicators, could provide much of that all-important insight in terms of the score-levels of statistical capacity. Undoubtedly, a new score for 2018 by the hypothetical SDG-laden SCI should invariably be much lower than the existing 2018 score for even the very-high-scoring countries in the World Bank database. Now these hypothetically discordant scores as compared to the existing scorelines could potentially instigate debate within the number cognoscenti.

If it were so happening, anyone, in the same way as what happened to Mark Twain, might easily be tempted to ascribe the denigrated numbers to incredibility of statistics. Save probably those who studied the metadata and appreciated the plausibility of how scores could dwindle on loading the construction framework of SCI with additional parameters as of SDGs; they might easily argue, ‘a student of the fourth standard scoring 90 % overall in the fourth grade examination will in all probability do extremely badly if allowed to sit for an eighth grade test.’ Then that’s a matter of capacity gap – a gap due to the difference in the frame of reference as it would be the case if the SDGs were embedded in the SCI framework.

Statistical capacity indicator scores from 2004-18 in 8 developing nations. Data compiled by the author, source World Bank, SCI database.

In the chart depicted in the previous section (also see above), the temporal ups and downs of the lines, however, are happening in spite of no change in the frame of reference of the SCI. These are somewhat akin to fluctuations in productivity (I spoke of in part 1).Change in the frame of reference by placing the SDGs on the SCI framework could materialize if and only when the data would be forthcoming on the majority of SDG indicators.

Since 2004, when Marrakesh Action Plan for Statistics was developed, strategic planning has been recognised to be a powerful engine for guiding the national statistics development programmes (NSDP), increasing political and financial support for statistics, and ensuring that countries are able to produce the data and statistics needed for monitoring and evaluating their development outcomes. A Global Action Plan for Sustainable Development Data has come into being at Cape Town in January 2017 for coordinated action on capacity development for sustainable development data.

Is it still too soon to expect the governments and statistics agencies to be investing resources and energies for augmenting the capacity to produce sustainable development data?

The governments and statistics agencies investing resources and energies for augmenting the capacity to produce sustainable development.
Image creditMeghna Chakrabarti

Dreaming a dream of the spring setting in when the agencies of change go on waving the magic wands (to transform the data eco-system), the ‘goose’ of golden data lies in slumber. And thus, SCI keeps on serving with the scores as they are – the only presentable measures of statistical capacity that does not lay the ‘golden eggs’.

This blog originated out of a dinner table conversation at the Chakrabarti household, where Satyabrata Chakrabarti (the dad and former Deputy Director General at Central Statistics Office, Government of India), tries to convince his two daughters of the impact and current assessment of a statistical tool for social and economic sectors. While Meghna and I go in a trajectory to assess, its impacts in our fields.

Cover image by Meghna Chakrabarti (L), Author Satyabrata Chakrabarti (M) and editing/blog design by Rituparna Chakrabarti (R)

We publish using the Creative Commons Attribution (CC-BY) license so that users can read, download and reuse text and data for free – provided the authors, illustrators, and the primary sources are given appropriate credit.

The ‘Capacity’ Paradox

Editor’s note: In this two-part blog series, the author explores the issue of the statistical capacity deficit in the current global context. In part 1, his main aim is to uphold the interplay of the system’s capacity, it’s efficacy and productivity, impacting the socio-economic sphere globally, especially the developing nations.

I have been tickling my imagination to mentally visualize the ‘golden eggs’ laid by a goose with ‘statistical capacity’. One day, no sooner I closed my eyes than my thought process veered off the track and entered the dull territory of numbers. The detour thereafter is the story here – passing on a road lined with numerical landmarks shining gloriously with images of statistics atop in some semblance of ‘jewels in the crown’.

Indeed, in the beginning, as I kept on stirring my imagination for a while, I did make an effort to identify something royal about numbers. It’s then I realized the connection that ‘statistical system’ has with the business of the king. The journey, therefore, started with me being captivated by the ideas and concepts of a system’s capacity that enables rulers to indulge in statistical stockpiling, although they turned out to be harmlessly mundane and theoretical as in the descriptions to follow.

The golden eggs laid by the statistical capacity. Image credit Meghna Chakrabarti

A system, as the wise men say, has to perform a given set of tasks routinelyy. So, it must have some capacity to do so. So a system, or for that matter a person or a machine which can produce something has a capacity to create a certain quantity of something in a given period. This is quite trivial. However, what’s not so nugatory is that the capacity does not necessarily remain equally productive all the time. When capacity is not equally productive, i.e., sometimes less productive and sometimes more, then the wise men say that it’s a matter of efficiency of the system or the person.

So, how do I know the efficiency of the ‘goose that laid golden eggs’? Is that conceivable?

For a moment I’m inclined to believe: yes, it is. For there is a measure called productivity to quantify the efficiency of a person, machine, factory, or a system. Who doesn’t know that inputs (i.e., labor, material, energy, cost, time, etc.) are used by the system and converted into useful outputs? So this notion about the effectiveness of any productive effort termed productivity is measured in terms of the rate of output per unit of input.

A trick, however, lies here in the fact that even when you keep the inputs fixed, the productivity can vary.

That’s because there are factors which, uncontrollably, unnoticeably or otherwise, influence the output to change from time to time. These factors, even when known for the role they play, often remain unrecognized or unexplained. If they are not explainable or perceptible, variation in productivity may appear strange, or even intriguing. And you know it very well that Mark Twain was intrigued; intrigued by his own productivity figures. Mark Twain wrote in 1906,

Figures often beguile me, particularly when I have the arranging of them myself; in which case the remark attributed to Disraeli would often apply with justice and force: ‘There are three kinds of lies: lies, damned lies, and statistics….

What puzzled Mark Twain was the output-to-time ratio as a measure of his ability to write words (i.e., writing productivity) since the ratio when measured on two separate occasions differed violently in magnitude. This is quite an analogy to the cases where we observe variation in human performance (nay, productivity) due to factors not easily visible/ discernible at the time of measuring it. As for example, weather, time of the day or, level of comfort in the room or something of that sort having an effect on the mood or psychological/mental condition of the performer.

However, the great author in the pleasure of his own creative inspiration perhaps made himself imperceptive to the factors playing on his own productive capacity. And as a result, that remark of Disraeli that he immortalized in his auto-biography made statistics damnable forever.

Nevertheless, in a contrasting scene of reality, statistical numbers indicating the state of development often fail to reveal the vital things when they do matter a lot. Even those produced in the mills of highly reputed institutions or by the national statistical offices acclaimed for their high statistical standards are frequently damned for such reasons.

Statistical capacity building enables statistical practitioners in the public and private sectors to use methods for data collection, analysis and interpretation; and contributing to the development of statistical infrastructure and human resources in official, survey, business, education and research. Image credit: http://www.pixabay.com

Heaps of literature on concepts and methods for collection of data, a compilation of statistics and dissemination of results including analytical tools have been produced and promoted under the aegis of the United Nations Statistical Commission (UNSC) over the last 70 years just to make official data all over the world sounding credible and dependable. These works covering almost all fields of economic and social relevance in development have helped to strengthen national statistical systems, especially the statistical capacity building.

By ‘statistical capacity’ they mean a nation’s ability to collect, analyze and disseminate high-quality data about its population and economy. In this sense, however, countries have attained varying levels of capacity at the national level, the most significant difference being in the ability to collect the data. The collection of data for calculating recommended statistical measures at regular intervals involves systemic rigor and requires considerable resources for conducting large scale operations. The situation becomes all the more challenging when the system gets to respond to new realities, e.g., evolution of the measurement paradigm necessitated by new statistics-based evaluation.

No system, in fact, can very quickly adapt to significant reforms in the statistical framework that may evolve as a necessity. The national policies which are able to quickly respond and make structural/procedural transformation are really robust in statistical capacity.

The World Bank developed a Statistical Capacity Indicator (SCI) for assessing the capacity of a country’s statistical system. It is a composite score based on a diagnostic framework assessing the areas: methodology, data sources, and periodicity and timeliness. Countries are scored against 25 criteria in these areas, using publicly available information and/or country input. The overall statistical capacity score is then calculated as a simple average of all the area scores on a scale of 0 – 100. From the scores of nearly 140 developing countries of the world in the last 15 years, what is evident is a clear trend of improvement for most of the nations, though there are times for falls after rises.

Afghanistan, for example, rose from almost a state of void (24.4 in 2004) to 50.0 in 2018; it’s a success story of a devastated nation. The period covered coincides with the countries being engaged with the Millenium Development Goals (MDGs) and therefore, the third dimension: ‘periodicity and timeliness’ looks at the availability of key socio-economic indicators, of which 9 are MDG indicators.

It’s interesting to see how some of the major developing countries have fared. Mexico, a high performing country with a scoreline of 74.4 in 2004, 85.6 in 2010, and 92.2 in 2015, finished at 96.7 in 2018 – a story of steady improvement; especially against an ordinary scenario for its own region (Latin America & the Caribbean) with the average score hovering in the range of 74 – 78 during the period, it’s remarkable.

Against South Asia’s regional score ranging between 65 and 76, India finished at 91.1 in 2018 moving through 78.9 in 2004, 81.1 in 2010 and 77.8 in 2015; whereas Bangladesh moves from 70.0 in 2004 to 72.2 in 2018 after a rise to 76.6 in 2015; and Pakistan moved from 73.3 in 2004 to 78.9 in 2018.

Indonesia, another major country in East Asia and the Pacific region having a regional score of 77.5 in 2018, moved from 86.7 in 2004 to 90.0 in 2018. On the other hand, the Philippines and Thailand of this region remained almost static during the period scoring in the range 81 – 88. In sub-Saharan Africa, Rwanda did remarkably well moving from 61.1 in 2004 to 78.9 in 2018 and exceeding the regional score of 62.4 in 2018 by a considerable margin, whereas Ghana, not so worse a beginner moved up by 20 points from 51.1 in 2004 to 71.1 in 2018; and Tanzania moved from 67.8 to 71.1 during the corresponding period.

But is this enough? In the coming blog, we will discuss the fallacy in the SCI system and how it fails to give a credible picture of the statistical capacity of the developing countries. I will try to answer the following question – is SCI a proper tool in the present context to measure with? Also, I will talk about how the Sustainable Development Agenda requires a new measurement paradigm and offers opportunities for national systems to evolve into core entities of country-specific data eco-systems for the same.