- Prerna Mukharya
Ceteris Paribus does not work
As someone who has led over 150 field evaluations, ranging from a few hundred to a few thousand surveys in each exercise across a lakh plus locations in India, one thing is certain - Ceteris Paribus does not apply to fieldwork in India i.e. nothing is constant when you are out on a field mission. Economics 101 teaches us many theorems under the assumption, “all things constant” or Ceteris Paribus (as it is called in Latin). What happens though, when we venture out to test first hand, our assumptions of what policies or social programs might work? What happens when we talk to people, survey them, try and irk out their actual beliefs, try and figure why they act the way they do?
Through this piece, I will highlight how each research evaluation or data collection exercise is still an eye-opener. Every time we go on a field mission, we are surprised at the evolving nature of human behaviour and how our assumptions are thwarted, owing to reasons so diverse, so perfunctory on occasions, that we aren’t surprised to be surprised any more.
The concept of over-surveyed populations
For a moment let us think about locations whether it is a tier 3, tier 4 or any rural part. For instance, a place like Osmanabad situated in the outskirts of Nashik, which is a few hundred kilometers from Pune, a site not too far from Gaya, Ajmer at a five hour driving distance from Delhi perhaps. What is common to all these locations? Well, the fact that while they aren’t the most talked about areas yet getting to these sites is not all that difficult. This is perhaps why there has been a deluge of social interventions in these locations.
When one conducts survey exercises or research in some of these parts, one is met with respondents, whose minds are pre-programmed to answer a certain way. Ask them- the legal age of marriage for a girl and almost everyone will say 18. Even though Rajasthan and its child brides in certain districts are known to all. Question people on the concept of child labour, and you would be surprised at how parents and children are aware of how it is illegal and will offer their two cents on why education is critical. Yet, as you pay a visit unannounced you will see many children skipping school, or working in mines, brick kilns, as labourers day after day.
As organizations try and figure where to roll out interventions, often owing to ease of accessibility of certain “peri-urban” areas or owing to airports 3-5 hours away, or good highways connecting certain districts, we will see a concentration of work in certain parts. And yet many not so accessible regions miss out.
Next, consider the concept of stakeholders or communities that are important for the position they hold in the day to day functioning of our life, for the power they exert as a vote bank, or for the influence they have on trade bodies. When we plan a research evaluation, we are often surprised at why certain people, groups of people or entire communities are acting in a way that is detrimental to their growth, or well-being. We often find stakeholders doing the exact opposite of what we would assume they should be doing ‘under normal circumstances’. For example, one might assume, that parents would want their children
This could mean farmers or truckers who deliver food to our cities from farms or the migrant workers involved in construction work. They are an influential group, and what they think about agricultural policies matters significantly. Similarly, the city could not function if truckers were to go on strike. A community of low skilled workers, similarly, care about the number of days they get to work under MNREGA. Any small shift or tweak to policies concerning them, without their consent or having them on board can mean toppled governments, agitations, and therefore big financial losses (among other negative externalities) to the economy. What is interesting to note is that on many occasions these policy changes could be beneficial to these sub-groups, groups at large or in the medium to long term.
This may create chasms in the short run for stakeholders such as land owners, or providers of logistical services for example, who might in turn fund some of these agitations or be instrumental in creating clouts of misinformation. It is therefore imperative that as a policy maker, one evaluates how a policy which is largely meant to create social value especially for the lower echelons affects some of the mid and upper sections of people i.e. figure how your social plan or policy may rock status quo and how the ‘well-to-do losers ‘will react and influence the ‘low-skilled potential gainers’.
My last example stems from the concept of over surveyed populations. Yes, there is such a thing! Much like fashion in dressing, vied-for movie stars, trending food items, each year or season may have its favourite. For example, if in a particular term, the central government is focusing on sanitation then a lot of money from philanthropists, CSRs, Indian and international donors will move to that particular sector. For five years under Swachh Bharat, we saw agencies, not for profits do all things concerning installing toilets, encouraging the use of toilets, ODF villages. Good in one way and yes, super important in a lot of ways but it also led to respondent fatigue. We saw schools dressed up with freshly painted school toilet walls on our arrival. It’s as if schools, households knew who was coming, what they were coming to ask, and they had heard it all, seen it all. Consequence - our evaluations will result in solutions that may not be sustainable or suggest models that may not work when the funding begins to taper down or when the sub sector isn’t as attention-worthy or fashionable any more.
It is therefore important that funders, donors and their research/data partners do their due diligence, before finalising their intervention location; undertake stakeholder mapping exercises to ensure the right audience is targeted in the right manner. Conversations with local organisations, individual champions can lend deep insight into behavioural traits, influencing variables, external factors et al.
Inspite of our diligence it is imperative one goes in with a fresh head, each time one takes up ground level work. This is in the interest of better understanding what our ‘beneficiary’ or the ‘recipient of social program’, or ‘consumer of our social intervention’ truly desires and needs. As economists or data practitioners, our duty to do right by our classroom learning therefore implying that we leave our assumptions behind, and accept that ceteris paribus does not exist outside or a classroom.