The Illusion of Predictability in Development: The Problem with Measurement

-Dibyendu Chaudhuri, Integrator, PRADAN

Development rarely follows a predictable path. As communities evolve through complex social interactions, conventional monitoring and evaluation frameworks often fail to capture the realities of change. This article explores why measuring development requires moving beyond rigid indicators towards learning, adaptation and context.

A few months ago, I was discussing my article on M&E (Monitoring and Evaluation) failure, titled ‘Six Reasons Why the Monitoring and Evaluation System is Failing in the Social Sector’ ((https://indiacsr.in/six-reasons-why-the-monitoring-and-evaluation-system-is-failing-in-social-sector/), with Ajit.

Ajit Kumar, the MIS Coordinator for PRADAN's Madhya Pradesh team, has spent nearly fifteen years designing, implementing and refining monitoring and evaluation systems across diverse rural development programmes. During our conversation, he remarked that I had overlooked what he believed was the central issue. "Development," he said, "is not fixed and repetitive like a machine."

That one observation stayed with me long after our conversation ended. It became the starting point for this article.

Anyone who has spent significant time in the development sector knows this feeling. We design an intervention, roll it out across different villages and communities, and then discover that the same activity produces very different results in different places. Sometimes the outcomes are better than expected, sometimes worse, and sometimes simply different in ways nobody anticipated. The M&E framework we built, the logframe, the theory of change, the indicators we carefully selected begins to lose its explanatory power. It was designed to measure progress along one predicted pathway, but the village before us has taken an entirely different route.

Where exactly does the problem lie?

How M&E frameworks are usually built

Most development programmes begin with a vision of change. Based on previous experience, organisations formulate a strategy that they believe will help achieve that vision. The strategy generates a hypothesis, a pathway describing how a particular set of interventions is expected to lead, step by step, to the desired outcomes. Depending on the methodology being used, this pathway is documented as a logframe, a theory of change, or a similar planning framework.

The M&E system is then built around this predicted sequence of results. It assumes that if activity X is carried out, it will produce output Y, which will eventually contribute to outcome Z. In practice, multiple activities may produce several outputs, which together contribute to multiple outcomes. Indicators are selected, targets are fixed, and progress is measured against this expected sequence.

Hidden beneath this approach, however, lies a powerful assumption: that the same intervention, implemented with different people in different places, will broadly follow the same pathway and produce similar outcomes.

It is a convenient assumption but also a problematic one.

Why does the same intervention not behave the same way twice?

To understand why, we may borrow an idea from complexity science: emergent properties.

Consider water. A single water molecule is not "wet." Wetness only emerges when countless molecules interact with one another. The same is true of the human brain. An individual neuron does not think. Thinking emerges from billions of neurons interacting as a network. In other words, the behaviour of the whole cannot always be predicted simply by examining its individual parts.

Communities function in much the same way.

A village is not a machine with mechanical parts that predictably produce identical outputs. It is a complex adaptive system, a web of interacting households, institutions, customs, social norms, relationships of trust and rivalry, local leadership, markets and governance structures. These elements continuously influence one another, adapting and evolving over time.

The moment an intervention enters such a system, it becomes part of that web of interactions. People respond. Institutions adapt. Relationships shift. New opportunities emerge while unforeseen constraints surface. These responses, in turn, influence how the intervention itself unfolds. Six months later, we are no longer observing the same village plus an intervention; we are observing a system that has evolved because of the intervention.

Complex adaptive systems are also highly sensitive to initial conditions, an idea often described through the butterfly effect in chaos theory. Two systems that appear almost identical at the outset can gradually evolve in remarkably different directions because small differences become amplified over time.

So, two villages subjected to exactly the same intervention will not necessarily follow the same developmental pathway. And since no two villages are ever truly identical, not in their history, their social structure, their leadership, or their context, it stands to reason that the same intervention is likely to produce different outcomes in different places. Not because the implementation was flawed, but because that's simply how complex systems behave.

So what do we do instead?

If complexity is an inherent feature of development rather than an exception, then perhaps the question is not how to eliminate uncertainty but how to design evaluation systems that can learn from it.

This requires shifting the role of M&E from merely verifying whether reality followed a predetermined plan to understanding how change is actually unfolding across different contexts.

Several approaches already move in this direction.

Developmental Evaluation works alongside programmes in real time instead of measuring them solely against fixed indicators. For example, if a livelihoods intervention is implemented in two villages, a developmental evaluator would not simply compare whether both villages achieved identical income targets. Instead, they would examine how each village's social dynamics, leadership and market access shaped different trajectories. As these patterns emerge, programme teams can adapt their strategies to suit each context rather than forcing every village into a common implementation model.

Outcome Harvesting reverses the traditional sequence of evaluation. Rather than beginning with predetermined outcomes and measuring progress towards them, it starts by identifying the outcomes that have actually occurred, whether intended or unintended, positive or negative and then works backwards to understand how the intervention may have contributed to those changes.

Adaptive M&E frameworks treat the theory of change itself as a living hypothesis rather than a fixed blueprint. As feedback emerges from implementation, the theory evolves accordingly. A natural resource management programme working across multiple villages, for instance, may discover that one village is experiencing elite capture while another demonstrates strong community participation. Instead of treating these as deviations from the original theory, the programme revises its understanding separately for each context and adapts its strategy accordingly.

What unites these approaches is a fundamental shift in perspective. Instead of measuring how far reality has deviated from one predicted pathway, they seek to understand and learn from the multiple pathways through which change actually unfolds.

Why is this so difficult to put into practice?

Part of the answer lies with implementing organisations themselves.

Most organisations build their credibility on interventions that have previously delivered positive results. These successful experiences naturally become the basis for designing future programmes. Past success provides confidence, helps attract partnerships and informs programme design. Yet it can also create an implicit expectation that similar interventions will produce similar outcomes elsewhere, even though every new context is fundamentally different.

Donors, understandably, operate within their own accountability frameworks. They are responsible for demonstrating that public or philanthropic resources have been used effectively, which often requires clearly defined objectives, measurable indicators and evidence of results. These expectations are entirely legitimate and play an important role in ensuring transparency, accountability and responsible stewardship of resources.

At the same time, these requirements can unintentionally encourage programme designs that privilege predictability and standardisation over adaptation. This is not because donors fail to recognise complexity, but because accountability systems have traditionally evolved around demonstrating planned results that can be measured and attributed.

The result is not a conflict between donors and implementing organisations, but a genuine tension between two equally important objectives: maintaining accountability while acknowledging that development rarely unfolds in a neat, linear fashion.

Consequently, organisations often end up designing rigid measurement frameworks even when years of field experience tell them that real change is far more dynamic. What gets reported is a tidy sequence of planned outputs and outcomes, while what is actually happening on the ground is a messy, iterative and deeply contextual process of adaptation and emergence.

We're only fooling ourselves

None of this suggests that measurement has no place in development. On the contrary, evidence remains essential for accountability, for learning and for improving programmes. The question is not whether we should measure, but what we choose to measure and whether our measurement systems are capable of evolving alongside the realities they seek to understand.

If we continue relying on rigid frameworks despite knowing their limitations, we are not really fooling anyone but ourselves. Village societies behave as complex adaptive systems, not mechanical ones. Outcomes emerge through countless interactions between people, institutions and contexts rather than through a fixed, predictable chain of cause and effect.

Ajit was right. Development is not a machine. It is a living, evolving process shaped by people, relationships and context. Perhaps it is time our monitoring and evaluation systems reflected that reality instead of expecting the world to conform to our models.