The social and economic development budget of India is well into hundreds of billions of dollars. Yet, when asked about the yardsticks used to measure success, policymakers often discuss developmental programmes – so many health camps held, so much concrete bought and so on. Rarely do they venture into the measurement and assessment of outputs, surely a critical exercise to understand the effectiveness of government programmes and policies. Such understanding, often termed impact assessment, is in itself an input into future policy decisions – should we continue with current programmes or discard them in favour of other programmes? Even if programmes are continued, can they be modified to maximise their effectiveness? And in case of completed programmes, what can we learn to carry over to new programmes?
There are as many evaluation methodologies as there are experts and there is no single method that is identified as a universal yardstick. In this cover feature on Impact Assessment we explore four – each on an important, large-scale programme operated by the Indian government. The assessment are differentiated by the techniques used and demonstrate what one can learn from each case. In the first article, Professor Shilpa Aggarwal writes about the Pradhan Mantri Gram Sadak Yojana, a decade long road-building programme that connected thousands of villages to markets in rural India. This article relies heavily on data downloaded directly from the programme’s website and marries it with national surveys on household welfare. Professor Aggarwal uncovers surprising effects of the market access afforded by rural roads – consumption of many goods, including agricultural commodities such as fertiliser increased, but simultaneously teenagers dropping out from their school to exploit employment opportunities in nearby towns.
Professor Sisir Debnath evaluates Janani Suraksha Yojana (JSY), a central government scheme aimed at decreasing maternal and infant mortality by encouraging institutional deliveries. This evaluation uses the District Level Health Surveys which report detailed household level information on the health behaviour and health status of a large number of households in various districts in India. The evaluation finds that the JSY was effective both in increasing institutional deliveries as well as drastically decreasing mortality, partly because of cash incentives for health workers to motivate them to steer women to hospitals. So this evaluation has implications not only for the JSY scheme, but also for a wide variety of new and existing programmes that use conditional cash incentives to motivate a policy objective.
Assessment of outputs is a critical exercise to understand the effectiveness of government programmes and policies. Such understanding, often termed impact assessment, is in itself an input into future policy decisions.
In contrast, Professor Ashwini Chhatre and Research Associate Arnab Dutta ask whether extrinsic motivation, such as cash incentives, might displace intrinsic motivation to participate in public programmes. Using the Mid-Himalayan Watershed Project as a case study, they show that the displacement is complete, and that cash incentives can actually have a regressive impact! It may serve as a cautionary tale for designers of public programmes.
Finally, my article on JEEViKA, or the Bihar Rural Livelihoods Project, used primary data collection to understand JEEViKA’s role in trust building in the villages where it was implemented. At the community level programmes, mutual trust is a key to getting along and implementing the programme successfully. Yet, no survey collects systematic information on variables such as trust. So we devised new experimental techniques to find out about trust and trustworthiness in the context of rural Bihar. We found the programme indeed increased trust, something that might lead to long term social and economic benefits even when the programme is no longer operational. These assessments have lessons for both the government and the private sector since the techniques translate readily to programme evaluation by private sector firms. For example, a manager might evaluate the effectiveness of a training programme for new employees, or that of an IT system for tracking orders. The combination of managerial insights and rigorous data analysis can produce profitable results!