Numbers as Indicators

http://fridayreflections.typepad.com/weblog/2007/10/not-everything-.html

Not everything that counts can be counted, and not everything that can be counted counts --- Albert Einstein

Timor-Leste is one of the countries in South East Asia with the highest incidence of malaria. It is one of the leading causes of maternal mortality in the country. My organization is implementing a project on Maternal and Child Health in two districts in Timor-Leste. It is on the last stages of project implementation and at the moment we are just consolidating the activities and now preparing for end-of-project evaluation. At this stage, stories of most significant change are being collected.

The focus of the story this time is the health staff of the community health center but I won’t be dwelling in details about their story. Though one thing for sure, the project has created impact to the health services of the sub-district. This was based on their personal account of how the project enabled them to deliver health services to the remotest villages through the trained community health volunteers. A major focus of intervention was decreasing malaria incidence in the communities. During the visit to the health center, I collected data on statistics of malaria cases to support my story. Based on the humble records of the laboratory technician, yes, I call it ‘humble records’ because patients who underwent malaria tests are recorded in a notebook improvised by the laboratory technician which was admirable because of the ingenuity in marking positive  and negative results. It was only in 2011 that the Ministry of Health developed a log book for the laboratory.

Counting manually, recorded cases of malaria in 2009 was 27 and in 2011 it was 702. Judging by the figures, immediately the conclusion would be: the project failed because instead of decreasing the prevalence, it had increased. The number shows a different story. When asked why the number increased, the health staff cited a number of reasons: 1.) increased awareness of people about detection and management of malaria; 2.) information about the availability of laboratory for malaria testing at the community health center; 3.) regular information dissemination by the community health volunteers. Therefore, detection of malaria has increased as more people go to the clinic at the onset of malaria symptoms.

I realized that increasing or decreasing figure does not always show the real picture. Since I started focusing my development work in Monitoring and Evaluation, I have never been a strong advocate of ‘quantitative indicators’ alone. The results you are measuring always matter.  That is why I always believe in mixed indicators which is balancing quantitative indicator with qualitative indicator so that numbers are properly described and analyzed. I also believe in collecting significant change stories because it’s a testimony of the changes in people's lives or the community as a result of the project's intervention. It’s a story of change which comes from the people who are the main actors of the development process.

After completing the interview, it reminded me of what Albert Einstein said: not everything that counts can be counted and not everything that can be counted counts. Indeed!

On a side note, this was the road on the way to our project area. It's a National Road. We waited for few minutes while the road was being cleared for vehicles to pass through.