Prize-backed Challenges

From MediaWiki
Jump to: navigation, search

Prize-backed Challenges are a crowdsourcing and open innovation technique that generally relies on monetary prizes to inspire wide participation to help solve particular problems.

Description

To help solve really big problems, two ideas are increasingly considered by the United States government and others: prize-induced contests and grand challenges. As budgets tighten and information and communication technologies continue to advance, leveraging the public’s expertise through contests and challenges is becoming more attractive to government agencies. Both contests and grand challenges offer many advantages for government—including paying only for results, establishing an ambitious goal without having to predict which team or approach is most likely to succeed, bringing out-of-discipline perspectives to bear, stimulating private sector investment that is much greater than the prize value[1]—but U.S. Chief Technology Officer Todd Park and others believe that the greatest advantage comes from widening the pool of potential problem solvers beyond the “usual suspects.”

To gain a greater understanding of prize-induced contests and grand challenges, it is important to recognize that, though they each shift the locus of innovation from inside a government agency to the public and channel motivations in untraditional ways, there are central differences between the two methods. Prize-induced contests, as the name suggests, are largely defined by the monetary incentive that drives the public’s engagement with the contest. Grand challenges, on the other hand, rely on a less-concrete system of incentives, one based on different stakeholders working together to tackle the big, audacious public challenges of our time. As such, these notable differences between contests and grand challenges make separate research agendas essential. While additional insight is needed on both methods, the research questions regarding prize-induced contests are mainly tactical and aimed at developing best practices, and those regarding the less-structured grand challenge space are more fundamental and strategic.

Prize-induced contests generally rely on monetary prizes to inspire private-sector engagement with large societal problems. In contests, Park notes that, “You draw in unusual suspects along with the more usual suspects. Studies suggest people that win these contests were not the usual suspects, but their aptitude seems obvious in retrospect.”[2] Of course, to draw in these unusual suspects, incentives are essential. While civic-mindedness could feasibly influence citizens and businesses to engage with public-sector contests, monetary prizes can help spur private-sector action by offering a lump sum to the winner(s), which, depending on the contest, could represent a profit or, at least, some return on investment. As gamification and similar competition-based systems of influence are gaining popularity in unexpected areas, prize-induced contests rely on both monetary prizes in and participants’ competitive spirit to incentivize previously untapped innovators to engage with difficult problems.

The U.S. government contest portal, Challenge.gov, features prize-induced projects from many agencies, including the Department of Health and Human Services’ “Mobilizing Data for Pressure Ulcer Prevention Challenge,”[3] the Department of Veterans Affairs’ “VA Medical Appointment Scheduling Contest,”[4] and the Department of Energy’s “Apps for Energy”[5] contest. On a more local scale, New York City recently introduced the BigApps 3.0 challenge, which “offers $50,000 in cash and other prizes to software developers for the best new apps that utilize NYC Open Data to help city residents, visitors and businesses.”[6]

Literature on Use in Government

Beyond anecdotal success stories, little research or systematic evidence currently exists on the efficacy of prize-induced innovation.

One of the most in-depth discussions to date occurred at the Collaborative Innovation Summit in June 2012, which examined the use of public sector prizes to spur innovation in the federal government and was hosted by the White House Office of Science and Technology Policy, NASA Center of Excellence for Collaborative Innovation, Case Foundation and Joyce Foundation.[7] This conference featured speakers from the public, private and NGO sectors, who discussed topics including prize design,[8] legal authority issues[9] and big data challenges.[10]

The most cited study originating outside of the White House on the use of prizes and contests to spur innovation and engagement is McKinsey’s “And the winner is…” report.[11] This study focuses on the philanthropic realm, though most of its findings can easily be applied to government programs. The McKinsey report finds that prizes deliver change in seven central ways: identifying excellence, influencing public perception, focusing communities on specific problems, mobilizing new talent, strengthening problem-solving communities, educating individuals and mobilizing capital.

MIT professor Heidi Williams’ study, “Innovation Inducement Prizes: Connecting Research to Policy” focuses on two key design issues that are important to the impact of prize-induced contests: setting the size of the innovation inducement prize, and spurring development of technologies that are desirable to consumers.[12] While this paper largely offers a literature review of the field, Williams argues that additional empirical research is essential to fully understand the effects and best practices across these initiatives.

Kevin C. Desouza of IBM’s Center for the Business of Government produced a report specifically focused on Challenge.gov.[13] While Desouza’s report contains a number of relevant findings and recommendations for improving the site, his methodology—based completely on analysis of the user-facing site and interviews with contest participants and site managers—reinforces Williams’ claim that the field is in need of more empirical research. Another report from the IBM Center is exclusively focused on best practices in creating and managing prizes for use in government innovation contests.[14]

The Center of Excellence for Collaborative Innovation (CoECI), a NASA office, is also working with OSTP to study open and collaborative innovation programs, both within NASA and in the private sector InnoCentive platform. CoECI’s publications primarily examine NASA-focused initiatives, including “Open Collaboration: A Problem Solving Strategy That Is Redefining NASA’s Innovative Spirit,”[15] and “Investment in Open Innovation Service Providers: NASA’s Innovative Strategy for Solving Space Exploration Challenges.”[16]

More recently, the Knight Foundation produced a list of seven central issues[17] that they have grappled with in their prize-induced “News Challenge.”[18] In this media-based contest, many of the issues being confronted by Knight are ones that government agencies attempting to tap public innovation also face. These include determining the correct scale for contests, fostering a network of innovators, convening a panel to assess proposals, and post-contest support and funding.

Karim R. Lakhani and Michael L. Tushman’s report, “Open Innovation and Organizational Boundaries,” contains a number of insights regarding the shifting of innovation from private firms to members of the public working as individuals or in informal groups, and the ability of private firms to leverage that public innovation.[19] Lakhani and Tushman note that, “We do not yet have a theory of the firm, either for incumbents or new entrants, which takes into account community innovation.” While many of its findings could be applied to the public sector, the report does not specifically focus on the use of prizes or contests, and instead examines open innovation more generally. However, Lakhani has produced research on “distributed innovation” that does take contests into account.[20] Like many others, he argues that such a system is the best way to overcome Joy’s Law—“No matter who you are, most of the smartest people work for someone else.”

Innovations and Tools

Case Studies

People and Organizations

Quotes

“As President Kennedy observed, ‘By defining our goal more clearly, by making it seem more manageable and less remote, we can help all peoples to see it, to draw hope from it, and to move irresistibly towards it.’”

Research Questions

In order to develop deeper tactical insights into both the limits and full potential of prize-induced contests, we need to define and systematize a research agenda that distinguishes these contests from other types of citizen engagement and explores the opportunities and challenges unique to such initiatives. Research is desperately needed to understand what works and in what circumstances. A greater understanding of the field will help to guide the continued evolution and optimization of innovative citizen engagement projects.

Considering the limited research to date, we need a more comprehensive understanding of such initiatives that possibly can be turned into a toolkit for further implementation. The following questions remain unanswered, hampering policymakers from realizing the full potential of prizes and contests to advance the public good:

Big Questions

  • What is the role of government in solving really big problems?
  • How can we facilitate open and transparent sharing of ideas and solutions between the individuals, organizations and broad sectors engaged with grand challenges?
  • Can we develop a simple test or set of metrics to determine if a given problem would be a strong candidate for a prize-induced contest?
  • How do we sequence community building around these research questions?

Design and Metrics

  • How should government agencies decide whether to use a prize-induced model instead of a more traditional form of R&D?
  • What unique features differentiate prize-induced contests from grand challenges and other forms of citizen engagement?
  • Do new or overlapping prizes dilute the effectiveness of others?[11]
  • When is the best time to create or discontinue a prize?[11]
  • What are appropriate objectives for a prize? And what is it that makes a prize effective at achieving them? [11]
  • Are there certain types of prize structures that are better able to create a community of problem-solvers—both in the interest of winning the prize and beyond?[11]
  • Contests often provide large-scale goals, but often lack more gradual milestones. Could they be more effective if they gave participants more guidance on how to progress through the contest?
  • Should prize-induced projects incorporate any best practices from other non-competitive collaborative innovation programs, like hackathons and idea “boot camps?”
  • Many prize-induced contests feature both public and private sponsors. How do different types of sponsorship affect results and engagement?
  • Do contests sponsored by both the public and private sector address broader issues like the multi-stakeholder grand challenges?
  • What design best practices are most effective in eliciting contest submissions with value to consumers?

Participation and incentives

  • The stereotypical public participant in prize-induced contests is the retired or under-employed person working in a garage. Is this actually the case? How involved are start-ups and established businesses in contests? Are universities seeking to increase student and faculty engagement in the programs?
  • Can we determine who does not actually participate in a contest by looking at who enters for the first round and doesn't advance?
  • Do prizes create incentive for individual effort rather than teamwork within or across institutions?
  • How do government prize-induced initiatives compare—in terms of success and participant engagement—to similar public-innovation programs in the private and NGO sectors?
  • Challenge.gov features many contests that are focused on science and technology, as well as many addressing social and public policy. Are more people getting involved in the technical challenges even though many social and policy issues challenges likely have a lower barrier to entry and require less-advanced knowledge and skills? And if so, why?
  • Are people involved in lower levels of government engaging with policy issues challenges the way people in tech industries are likely engaging with the technical challenges?
  • Recently, New York Mayor Michael Bloomberg’s foundation “launched a $9 million contest inviting mayors to come up with the best ideas to tackle urban problems.”[21] Can prize-induced innovation contests be successful when targeted exclusively to people within government?

Public and Private Impact

  • Is there any guarantee that a prize for new ideas will stimulate commercialization and widespread adoption critical to improving lives on a large scale?[11]
  • What effects have prize-induced contests had on workforce development? Are individuals learning valuable new job skills through engaging with the contests? Are new businesses gaining traction following their participation? *Are participating individuals and companies gaining government employment or contracts following the contests?

Funding

  • While prize-induced projects have clear ties to other forms of crowdsourcing, is there a place for crowdfunding in these projects—both in terms of supplying the prize money and stimulating private engagement on expensive projects?
  • What makes prize-induced projects preferable to the use of social impact bonds, which have many similarities but rely on private sector financing rather than taxpayer money?
  • Is the pay-for-performance system currently used by some healthcare providers another possible guide for government agencies to draw on public innovation without upfront costs? Could such a program be instituted without contracting the pool of “unusual suspects” working toward solving problems?

Evaluation

  • How should grand challenge submissions be judged? Assuming that a panel is used, who should sit on it? Government workers from relevant? Influential members of the private sector? NGO workers with a vested interest in the contest?
  • How can the evaluation process be developed to ensure that innovation—rather than ease of deployment, cost and other practical concerns—is the central metric upon which submissions are judged?
  • Assuming a winner is not found, can the evaluation process act as an opportunity for feedback for participants, rather than as the final judgment as to whether a proposal is satisfactory or not?

Post-Contest Concerns

  • Should contest participants—including individuals and private businesses—or government agencies hold the intellectual property rights to innovations created as part of government contests?
  • What are the best practices for data sharing, both between participants and after the contest ends?
  • A common complaint regarding prize-induced contests is the lack of post-contest engagement with developers.[22] How can government agencies ensure that new innovations do not become obsolete due to lack of attention, especially considering one of the central reasons for such contests is lessening the workload of government employees?

Further Reading

Karim R. Lakhani, “The Case for Enabling Distributed Innovation,” http://bit.ly/1aNR8kW

“And the winner is…’ Capturing the promise of philanthropic prizes,” McKinsey & Company, July 2009, http://bit.ly/1ai8ZxU

Kevin C. Desouza, “Challenge.gov: Using Competitions and Awards to Spur Innovation,” IBM Center for The Business of Government Using Technology Series, 2012, http://bit.ly/1aDYBD3

References

  1. OSTP Memo on Prizes and Challenges, Department of Health and Human Services, http://www.hhs.gov/open/initiatives/challenges/ostp_memo.html
  2. Alex Fitzpatrick, Is Crowdsourcing the Secret to Creating Innovation in Government,” Mashable, June 13, 2012, http://mashable.com/2012/06/13/crowdsourcing-government/
  3. http://challenge.gov/ONC/457-mobilizing-data-for-pressure-ulcer-prevention-challenge
  4. http://vascheduling.challenge.gov/
  5. http://appsforenergy.challenge.gov/
  6. Erin Durkin, "Mayor Michael Bloomberg employs innovative contest style to bring bright, new ideas to meet city challenges," New York Daily News, February 10, 2013, http://www.nydailynews.com/new-york/bloomy-offers-big-sandy-money-best-ideas-article-1.1260504
  7. http://www.casefoundation.org/collaborative-innovation
  8. “Collaborative Innovation: Challenges of Designing & Administering Prizes,” http://www.casefoundation.org/videos/httpthetimeistudiedabroadtumblrcompo st25534308678when-i-drink-2-euro-wine
  9. “Collaborative Innovation: Legal Authorities Discussion,” http://www.casefoundation.org/videos/collaborative-innovation-legal-authorities-discussion
  10. “Collaborative Innovation: Algorithm/ Big Data Challenges Panel Discussion,” http://www.casefoundation.org/videos/collaborative-innovation-algorithm-big-data-challenges-panel-discussion
  11. 11.0 11.1 11.2 11.3 11.4 11.5 “‘And the winner is…’ Capturing the promise of philanthropic prizes,” McKinsey & Company, July 2009, http://mckinseyonsociety.com/downloads/reports/Social-Innovation/And_the_winner_is.pdf
  12. Heidi Williams, “Innovation Inducement Prizes: Connecting Research to Policy,” Journal of Policy Analysis and Management, vol. 31, no. 3 (2012), http://economics.mit.edu/files/7823
  13. Kevin C. Desouza, “Challenge.gov: Using Competitions and Awards to Spur Innovation,” IBM Center for The Business of Government Using Technology Series, 2012, http://www.businessofgovernment.org/sites/default/files/Challenge.gov_.pdf
  14. Luciano Kay, “Managing Innovation Prizes in Government,” IBM Center for the Business of Government, Collaborating Across Boundaries Series, 2011, http://www.businessofgovernment.org/sites/default/files/Managing%20Innovation%20Prizes%20in%20Government.pdf
  15. C.M. Rando, et al, “Open Collaboration: A Problem Solving Strategy That Is Redefining NASA’s Innovative Spirit,” 62nd International Astronautical Congress, 2011, http://www.nasa.gov/pdf/649977main_IAC-11E629x11483OpenInnovStrat.pdf
  16. C.M. Rando, et al, “Investment in Open Innovation Service Providers: NASA’s Innovative Strategy for Solving Space Exploration Challenges,” 61st International Astronautical Congress, 2010, http://www.nasa.gov/pdf/649979main_IAC-10.D3.4.3OpenInnov.pdf
  17. John Bracken, “#newschallenge: seven issues for the future,” Knight Blog, January 25, 2013, http://www.knightfoundation.org/blogs/knightblog/2013/1/25/newschallenge-seven-issues-for-the-future/
  18. http://newschallenge.tumblr.com/
  19. Karim R. Lakhani and Michael L. Tushman, “Open Innovation and Organizational Boundaries: The Impact of Task Decomposition and Knowledge Distribution on the Locus of Innovation,” Harvard Business School Working Paper, January 5, 2012, http://www.hbs.edu/faculty/Publication%20Files/12-057.pdf
  20. Karim R. Lakhani, “The Case for Enabling Distributed Innovation,” http://www.iom.edu/~/media/Files/Activity%20Files/Disease/NCPF/2009-FEB-Wkshp-on-Precompetitive-Collaboration/Lakhani.pdf
  21. Erin Durkin, "Mayor Michael Bloomberg employs innovative contest style to bring bright, new ideas to meet city challenges," New York Daily News, February 10, 2013, http://www.nydailynews.com/new-york/bloomy-offers-big-sandy-money-best-ideas-article-1.1260504
  22. Mark Headd, “A glass half full view of government app contests,” govfresh, June 8, 2010, http://govfresh.com/2010/06/a-%E2%80%98glass-half-full%E2%80%99-view-of-government-app-contests/