Issue 8: Show me the evidence

Plus, why do we ignore solved policy questions?

WELCOME

I have a bee in my bonnet this month, prompted by a specific op-ed published in a major newspaper – although it could have been prompted by many op-eds I’ve read recently. This piece spoke to a policy challenge on which I am currently working, and suggested that the first step towards a solution is…

A task force.

Once I’d gotten my uncharitable reaction out in the group chat, I started to think about how this point of view is a great example of a major challenge in Canadian policymaking: the choice to pretend that many policy questions have not been solved.

From traffic congestion to housing to clean energy and beyond, we not only know the answers but have seen them successfully addressed either in the past or in other jurisdictions. Yet when asked to act, Canadian political and policy leadership reverts to the mean of studying or discussing the problem as though the universe has yet to reveal its mysteries.

What causes this behaviour? Genuine ignorance of the answers? Our cultural allegiance to the status quo? Low expectations of the voting public? The kneejerk “Canada is simply too unique” response to foreign case studies? Insufficient policymaking capacity and sophistication? A frustrating soup of all of the above?

To paraphrase a meme, we need a full-scale moratorium on task forces until we can figure out what is going on. Perhaps with a Royal Commission…?

This month’s issue has a heavy emphasis on evidence-based policy, both how to do it and the criticisms and complications that arise from assuming “an engineer’s view” of the sophisticated machines that are society, the economy, and politics. Below, you’ll find practical guides as well as commentaries that can help level up your use of evidence.

THIS MONTH’S SYLLABUS

🎓 Designing and delivering randomised trials for social policy – King's College London

🎓 What Works, Faster – King’s College London

Interested in applying the ‘gold standard’ of research to your policy design? King’s College London has published two practical pieces on randomized control trials for social policy, including how to reduce the amount of time necessary for evidence evaluation to deliver policy answers faster. Interested in how reality complicates this kind of research? Read on…

💻 Vital City – Issue 7: Does Evidence Matter? When research meets the real world

Bookmark this online urban policy magazine based in New York for your next rainy weekend (or beach getaway, if you’re a truly committed wonk). This issue on evidence in policymaking collects short essays from 19(!) thinkers that approach the problem by drawing on both academic research and hands-on experience in government. (Many of the pieces are in conversation with this article on the “engineer’s view” in policymaking). Two highlights include Elizabeth Glazer’s “What Really Drives Policy?” and Jennifer Pahlka’s “The Limits of Death Star Thinking”.

⚙️ Toolkits – The Institutional Architecture Lab

A hot topic in governance today is the collapse of longstanding public institutions, and the downward spiral of low capacity delivering poor outcomes leading to public distrust. The Institutional Architecture Lab has developed a trio of toolkits dedicated to helping policymakers both renew existing, and build new, institutions that are more resilient to this challenge. While their Playbook is the most practical document, the guide to building legitimacy for your institution is a must-read.

🏛️ Universal Credit Programme Final Lessons Learned – UK Department for Work & Pensions

Recommended by friend of the newsletter Noah Zon, this lessons learned document from a 15-year universal credit initiative contains clear-headed and universally-applicable advice for public servants who design or run public-facing programs. The most refreshing takeaway? How important taking care of your team is to success.

💻 Exploring Policy Innovation: Tools, Techniques + Approaches – Brookfield Institute for Innovation + Entrepreneurship

In a piece from 2018, the former iteration of the Brookfield Institute collected expert thinking on the question, “What are the factors that impact successful policy innovation and development?”. Their answers are both academic and practical, and include some insight into the differences and overlap between policy innovation and public sector innovation. The report has a lot of great graphics, including this one illustrating common flaws within the traditional policy development cycle:

📰 "I Randomly Decided To Pay Off A School’s Lunch Debt. Then Something Incredible Happened." by DJ Bracken, Huffpost

This first-person piece about how the author accidentally became an expert on Utah school lunch debt policy is a wonderful illustration of the value of different perspectives in policymaking and advocacy: “I still remember sitting in a meeting with actual policy analysts and education officials, feeling like a child who had wandered into the wrong classroom, while simultaneously realizing that I somehow knew more about certain aspects of the lunch debt situation than these lifelong professionals did. It wasn’t because I was smarter or more dedicated, but simply because I’d been looking directly at a specific problem they only encountered as part of a much larger institutional landscape.” (This article is also a great lesson in “you can just do stuff”, one of the fundamental principles for a satisfying and impactful life.)

📣 "We need to change the emphasis of 'evidence based policies' – social post by Steven Klaiber-Noble, Twitter/X

Policycraft’s Hot Take of the Month is back! While we’re discussing how to design, generate, speed up, and integrate evidence into policymaking, software designer Steven Klaiber-Noble is thinking of what comes next: generating evidence from in-play policies and programs. However, he pits experimentation against observational analysis, while previous authors in this issue would argue we can’t do policymaking well without both.