Before we, as a society, try to change how we solve problems and plan new initiatives we need to know what has worked already and what hasn’t. Over the past few decades there have been renewed calls to measure the impact of societal initiatives in order to allocate resources wisely and understand what to continue, scale up or cancel. With government and philanthropic funding shrinking, both government agencies and charities see outcome-based evaluation as an urgent need. There has also been an increased demand to understand with more precision and nuance whether programs are successful, how they are successful and what factors have led to that success.
Within that context the Office of Management and Budget (OMB) issued yesterday a new memorandum (M-13-17) encourages government agencies to both: “(1) draw on existing credible evidence in formulating their budget proposals and performance plans and (2) propose new strategies to develop additional evidence relevant to addressing important policy challenges.” Interesting incentive: “Agency requests are more likely to be fully funded if they show a widespread commitment to evidence and innovation”.
The memorandum also encourages agencies the use of the following “cross-cutting strategies”:
- Harnessing data to improve agency results (leveraging open data sets)
- High-quality, low-cost evaluations and rapid, iterative experimentation (including testing behavioral insights to improve quality and lower costs)
- Using innovative outcome-focused grant designs (including prizes and challenges)
- Strengthening agency capacity to use evidence (through the development of common metrics and evidence frameworks).
Given GovLab’s work on developing new measurement techniques, we are honored and delighted that the memorandum considers also “Technical assistance in designing evaluations and improving tools. This may include connecting your agency with Intergovernmental Personnel Act (IPA) assignments or consultation from outside experts to help design and implement your proposals. For example, a number of external organizations, such as the NYU Governance Lab, J-PAL North America, the Pew-MacArthur Results First initiative and the Coalition for Evidence-Based Policy are seeking Federal partners for evidence and innovation initiatives designed to improve results at the Federal, State, and local levels.”
Building a new culture of learning and experimentation, envisaged by the new OMB memorandum, is imperative to make governance more agile and innovative. While there is good reason to believe that breakthroughs may come from recent innovations such as community-based problem solving, behavioral economic insights about human behavior, or predictive analytic experiments, there are limited studies measuring exactly how productive it is to use these kinds of new governance techniques. Without a deeper understanding of whether, when, why and to what extent an intervention has made an impact, any initiative we design will be sub-optimal and will produce less than the desired results. If we are going to accelerate the rate of experimentation in governance and create more agile institutions capable of piloting new techniques and getting rid of ineffectual programs, we need research that will enable us to move away from “faith-based” engagement initiatives toward “evidence-based” ones. Among our next projects are plans to build a next generation, open peer review system to assist to evaluate governance projects and innovation”.