People are obsessed with measurements. Each one of us wants to constantly measure everything around us. Whether it is our own personal wealth, or our academic performance or a movie that we want to see or just saw, or the service that we received in a recent flight that we took; we want to constantly measure and rate. It is now become such an integral part of our lives that we do not pause to ask why are we measuring and what are the metrics that we are using. Being in the Development sector, I have been fascinated by the evolving obsession of practitioners, donors, academia and the community in measuring and evaluating. The tools, methodologies and the people involved in this activity are getting better and better. Everyone seems to be so preoccupied and engaged that many consider a program a failure or bad, if some acceptable form of measurement is not undertaken. I have written numerous proposals and implemented many projects in different sectors of health, education and community development that I have found myself questioning not just the validity but also some of the metrics and the fundamental premise that drives these measurements. I am not saying that measurements by themselves are wrong. All that I am trying to point out is that we need to understand the program being measured, the competence of the people measuring, the tools deployed and the metrics of measurement and more importantly the context – before one indulges in this activity.
I would like to explain this a little more clearly with a personal example. I first came to the tribal area in the forests of Heggadadevanakote in Mysore district of South India in 1987 and began running a small dispensary. Obstetrics was one of my favorite subjects, and it was only natural that my focus was on maternal health and mortality. Being concerned by the high maternal mortality in the area, I started exploring how I could bring it down. Public health knowledge and practice has established maternal mortality to be an important indicator of the health of a community. It was also the time when everyone (including the WHO and Government of India) focused heavily on maternal health and safe motherhood initiatives. The whole health sector was being pushed towards Institutional deliveries and we also got caught up in this excitement. Over the next many years, we campaigned for Institutional deliveries and ensured that we created adequate facilities for safe childbirth at our hospitals. We even had a World Bank funded project in 2001-02 to ensure improved maternal & child health outcomes amongst the tribals in the area.
A few weeks ago, my wife Bindu, an obstetrician who has been long associated with the program was remarking how she was seeing a huge positive change w.r.t. the health-seeking behaviors of tribal women. She told me how challenging it was 20 years ago to motivate the tribal women to have Institutional deliveries but now the women sought the hospital on their own. Institutional deliveries, which were non-existent when I first came to the area, had grown to 40% by the end of the World Bank funded project in 2002. Today, virtually every tribal woman in the area comes to our hospitals to deliver their babies. Our Reproductive and Child Health (RCH) programs have been written about and studied; World Bank considers the RCH project that they funded us as one of their best; Public Health practitioners and academicians are impressed with the falling maternal mortality and improved health outcomes. It is indeed very reassuring when so many people and institutions with their sophisticated tools and methods call us a public health success.
A year away from our projects has given me the space, the energy and the willingness to re-look critically at all that we have been doing. I was assessing what it was that I first came to the tribal areas to do and how I had done it. Using the metric of improved health outcomes and the falling maternal mortality and morbidity rates, we were definitely an unqualified success. But was this the right metric to measure our work and intent? Can this metric capture everything that exists in this ecosystem? How honest would it be if we did not try to engage ourselves in outcomes that unintentionally emerged because of our programs, but were not given any attention, as they were not readily visible or worthy of measurement? Or that we were ignorant of the metrics that one needed to deploy?
In our intent to reduce the maternal mortality by increasing institutional deliveries, had we not unintentionally taken away the community’s ability to cope and manage this natural phenomenon without any dependence on people or a system outside their community and tradition? We today have a generation of young women who have mostly delivered their babies in our hospitals but who have neither the knowledge, the attitude nor practice to ensure that they can continue their century old tradition of delivering children at home. What if we changed the metric to building the capacity and competence of the community to have cost-effective and rational health practices that did not need an expensive health care system that they could neither afford not sustain with their resources. Isn’t building the capacity and competence of communities to ensure a workable health system that they can run and sustain with their own resources and abilities more important than running a sophisticated health care program that needs doctors, nurses and managers to come from faraway cities?
Well, the metric of measuring what is important for communities is what the development sector needs to focus on rather than what is easily measurable or merely the programmatic and managerial aspects. And the metric needs to be something that attempts to capture what is happening in the whole ecosystem rather than the just the piece that is the most evident.