ExpertNet: Disclosing Expertise in the City Government of Buenos Aires

Asking city government employees to disclose their expertise to increase engagement and improve municipal problem-solving

logo-expertnet

Spanish version

Professionals in municipal government have a hard time identifying who among their colleagues has the experience and expertise to improve how a city serves its citizens. The person sitting at the next desk with the ambiguous title of “manager” or “director” could have the know-how to make the difference between success and failure in an important public project. To make city government more effective, we want to be able to identify the talents, skills and capacity of the municipal workforce.

Ancient Athens governed itself by tapping the intelligence and expertise of its citizenry in making decisions. Historians believe that the participation of citizen experts was instrumental in making Athens the most successful city-state of its time. In the same way that ancient Athens tapped the expertise of its citizens, modern cities could benefit from better finding and engaging the expertise within the ranks of their citizens, beginning with their own municipal employees. Working with partner cities, the ExpertNet project at the Governance Lab is building open source tools to enable municipal government employees to identify each other’s areas of expertise. We will also test the efficacy of knowing more about the skills and abilities of employees for improving collaboration and problem solving.

ExpertNet builds on earlier work in connection with the White House Open Government Initiative on designing a platform for asking people to share more about their expertise – their experience, expertise, skills and passions. Using a collaborative wiki, the United States General Services Administration (GSA) and Open Government Initiative solicited feedback from the public on the development of a platform that 1) enables government officials to circulate notice of opportunities to participate in public consultations to members of the public with expertise on a topic; and 2) provides those volunteer experts with a mechanism to provide useful, relevant and manageable feedback back to government officials.

While, at least initially, the targeted population for ExpertNet has shifted from experts within the general public to experts within city governments, the central objective remains essentially the same: the creation of a purpose-built question-asking tool that will enable government employees to tap dormant, existing expertise to help solve problems.

Today the City of Buenos Aires makes the following information about a municipal employee publicly searchable:Rudi Profile

Rather than a multi-faceted view of an employee’s knowledge and skills, the existing system, which is typical of most HR directories, provides simple business card information such as Name and Contact Email, and basic professional affiliation like Department and Title.

ExpertNet seeks to expand what we know by going beyond titles to capture a broad picture of people’s expertise. Useful expertise goes beyond what shows up in formal credentials. It can include a wide range of skills and past experience, encompassing what we know how to do and our ability to explain it to others.

What We Seek to Build

In collaboration with partner cities, the GovLab will build an open source, linked data system for capturing municipal government expertise.

We want to understand:

  1. what kinds of expertise are most helpful to identify;
  2. what are the best ways to collect that information;
  3. how expertise impacts people’s willingness to collaborate;
  4. whether identifying employee expertise helps cities to be more effective at solving problems; and
  5. the resulting impact on citizen and employee perceptions.

The ExpertNet project will progress through four experimental stages:

  • Phase 1. In the first phase, we are focusing on employee self-reporting. We will test different categories of information people might supply (credentials, skills, interests, past projects, partnerships, work experience and future plans) and how that correlates to a willingness to collaborate and to collaborate well. The goal is to run three pilot implementations of 300-500 people each in two cities.
  • Phase 2. In the second phase, we will explore strategies for creating a linked data infrastructure, to connect and make searchable the skills and expertise of employees across cities. The goal is to connect people across all the pilot implementations and to test how we can match people to problems for better problem-solving.
  • Phase 3. In the third phase, we will explore integration with LinkedIn, VIVO and other popular international, regional and local sources of expertise as well as open datasets on publications and grants. The goal is to test whether tapping into existing databases is effective in supplementing and vetting self-reported data collected in Phase 1.
  • Phase 4. In the final phase, we will explore integration with existing HR systems. The goal is to transform ExpertNet from a stand-alone expertise platform into the central employee database for partner cities, which will allow us to explore what happens when a city’s HR system is populated with robust data on skills and experience, rather than exclusively featuring simple biographical information.

Rollout Plan for Phase 1

In the interest of undertaking an iterative approach, consistent with the GovLab’s action research methodology described below, we will begin our work with city government partners by rolling out a low-tech first phase of the project. Phase 1 will be a useful pilot project that will help populate initial databases, optimize categories of information to be collected (and test different categories), identify constraints and inform further platform development.

The central objectives for Phase 1 include:

  • Simultaneously initiating three pilot projects in each of two cities;
  • Undertaking in-depth background research on the partner cities, governments, departments and any other stakeholders within the pilot project’s scope;
  • Doing a survey with questionnaires to obtain self-reported information in a diversity of topic areas;
  • Collecting and maintaining data about government-held information on employees taking part in the pilot and the projects they worked on; and
  • Studying this data to optimize the categories of information the platform will collect.

The role of partner cities in Phase 1 will be:

  • Identifying departments and employees to take part in the pilot;
  • Submitting relevant government-held data on employees, departments, and projects into the ExpertNet database;
  • Educating employees about the project, its goals and the ways that it could help improve their ability to do their jobs;
  • Disseminating the questionnaire and encouraging participation; and
  • Identifying projects that can act as initial testbeds for ExpertNet.

The role of the GovLab in Phase 1 will be:

  • Undertaking in-depth background research related both to project partners and expertise networking;
  • Crafting the questionnaire with clear rationales for each category of information to be collected;
  • Collaborating on the development of the ExpertNet technical platform;
  • Guiding the roll out of ExpertNet in each pilot city and providing any needed support to partner cities as they disseminate the questionnaire and educate employees as to its purpose; and
  • Maintaining and analyzing the evolving expertise database to support the work needed to move to Phase 2.

Initial background questions for partner cities:

  • Which departments should be included in an initial pilot project?
  • What are departmental hierarchical structures?
  • Are there any existing communications or social media platforms for city employees that the GovLab should be aware of before advancing work on the ExpertNet project?
  • Are there any specific legal barriers that could affect development?
  • Are there any political concerns that could act as a barrier to development?
  • Is there existing data on projects undertaken in different departments?
  • What is the outside contracting policy for the city?
  • What are the cities technical resources, including available budget and number of internal developers?
  • Which projects are best suited to act as initial testbeds for ExpertNet?

Why This Matters: Use Cases

A searchable database of city employees’ skills, experience and interests can be put to a number of uses.

Identifying Expertise to Help Solve a Specific Problem

First and foremost, ExpertNet will allow city employees to easily find and engage expertise within and across departments that can help them solve specific problems. For instance, if an employee in Buenos Aires’ Department of Education is struggling to squash a bug in their payroll system and the internal IT support cannot help, the employee can search ExpertNet for “Database Management” and find an employee in the Department of Health who can help address the problem. Instead of wasting time searching for answers or wasting money to contract outside help, employees can leverage the expertise already available within their government to reach the same beneficial end.

Inspiring and Incentivizing Collaboration Within and Across Departments

Both through its system of endorsements and badges and intrinsic motivations made manifest by the system, ExpertNet will seek to instill a spirit of collaboration and employee engagement within cities. While employees will be able to use the system to solve important problems, like the database management example described above, ExpertNet will also provide an avenue for skills and interest sharing in a less formal way. For example, an employee might use ExpertNet to find willing help from a colleague who can provide Photoshop tips and tricks or good songs for beginners on the guitar. Whether to serve formal or informal ends, ExpertNet will seek to inspire individuals to collaborate with their colleagues and view themselves as a part of a large, department-spanning team.

 Avoiding Redundant Work

In addition to creating a searchable database of who knows what, ExpertNet will seek to create a searchable database of who’s doing what. By providing information on the types of projects that employees in different departments are pursuing, ExpertNet can help employees discover who has potentially useful knowledge and experience while also avoiding redundant work. For example, if an employee in the Department of Social Development is preparing to draft a report advocating for a new youth community center, she could use ExpertNet to discover that a similar report was just drafted six months prior in the Department of Culture.

Tools & Technologies

In performing and designing these experiments, we will leverage and learn from a variety of innovative tools and technologies already in existence. For instance:

  • Expert networking platforms, like VIVO and Stack Exchange, will provide useful data to draw from as well as best practices to emulate for creating searchable platforms of individuals’ skills, interests and experiences.
  • Ratings and endorsement systems, as found on LinkedIn and eBay, will allow users to vet skills, gain demonstrable recognition for good work, and make it possible to create badges and leaderboards to incentivize participation.
  • Linked Data technologies, as used in CKAN and Google Now, will allow us to consolidate data on individuals’ skills, preferences, and experience from a diversity of sources.
  • Predictive search functionality, as deployed by Google and Bing, will ensure consistency of self-reported data and make expert discovery easier.
  • High-Performance databases, like Oracle DB2 and MSSQL, will satisfy our data management and storage needs.

Central Question

Experiment

Output

Phase 1

What types of information on individuals are most useful for determining expertise and willingness to collaborate on projects?

Deploying two similar but different questionnaires in partner cities:

  • One version giving the employee more freedom to self-report skills, interests and experiences
  • The other using pre-defined information categories and available responses to select from.

An initial expertise platform and underlying database;

Qualitative insights on different techniques for obtaining employee information.

Phase 2

Does the ability to tap expertise across city governments, not just across departments, result in a more useful platform?

Creating a linked database of employee expertise across pilot implementations, and testing use and engagement compared to Phase 1.

A cross-city expertise platform and underlying database.

Observable changes in use based on expanding the platform beyond city government borders

Phase 3

Does tapping into existing data sources provide a fuller and more accurate view of individuals’ expertise and inspire more participation?

Supplementing and vetting self-reported data using existing data sources like LinkedIn, VIVO, Facebook and publications databases.

Deploying a cross-checking engine to compare self-reported skills on ExpertNet to relevant data from other sources.

More robust and user-friendly profile creation options.

Improved data quality and expertise matching capabilities.

Phase 4

Can an expertise engagement platform fully replace traditional city government HR systems?

Integrating ExpertNet with existing city HR systems and studying the impact and changes in employee collaboration and problem-solving trends.

An HR system incorporating both simple biographical and professional information and multi-faceted expertise information.

Observable data on usage changes following reform of HR system.

What is Smarter Governance?

ExpertNet is part of the GovLab’s ongoing Smarter Governance efforts, which are based on the belief that people who are not traditionally included in certain governance decisions have expertise, experience, and ideas that can offer new insights and solutions for public problems.

Advances in science and technology have provided new tools for identifying who knows what and targeting requests to participate to those most likely to contribute. With the ability to identify and target engagement to a small number of relevant audiences, such as city government employees with a specific set of skills and real-world experience, and to offer them ways to participate meaningfully, city governments have a better chance of crafting a solution that is better suited to the material reality.

Opening an organization to the use of what Harvard Business School professor Henry W. Chesbrough calls “purposive inflows and outflows of knowledge” is well known in the business literature as a driver of innovation.[I] This “open innovation” paradigm is the “antithesis of the traditional vertical integration model where internal [research and development] activities lead to internally developed products that are then distributed by the firm.”[II] In many (but by no means all) businesses, the traditional assumption that working in secrecy leads to innovation is becoming obsolete.[III] In the contemporary environment businesses are finding expertise in every corner of their organizations and outside of them.[IV] Open innovation not only expands opportunities for new solutions based on previously unavailable or disconnected silos of knowledge, but also allows external actors the benefits of leveraging an institution’s resources, such as its investment in research and development.[V] The narrative of a lone entrepreneur bringing his or her innovation to the market has been replaced with the concept of networks and communities coming together in an interactive process of discovery.[VI] New businesses are thriving by supplying expertise to organizations such as banks looking to inform their investment decisions[VII] or law firms seeking expert witnesses.[VIII] Relying exclusively on internal research and development is seen as limiting the potential scope of innovation in many sectors.

Open innovation techniques can help the public sector as well as private enterprise. Consider this early example. In 2005, before Facebook and Twitter, New York Law School students and Beth Noveck designed and later launched Peer to Patent, the first expert network in government, which connected volunteer scientists and technologists to national patent offices to better inform how they decide who gets the twenty year grant of the monopoly right called a patent.[IX] They invented the process, technology and the policy by which a closed bureaucracy could open its doors and collaborate with knowledgeable people to do its work better. Time after time, the public was able to dig up quickly the expertise the official couldn’t find, enabling her to make the final determination informed by citizen participation but subject to the independent law and rules of the patent process. Despite the fact that patent practice hadn’t changed much since Thomas Jefferson began as the first examiner in 1791, under the new Patent Act, passed with near unanimity – Peer to Patent citizen engagement, once an experimental pilot undertaken only with the applying inventor’s consent – is now enshrined in law.

While Peer to Patent, as well as much of the GovLab’s Smarter Governance work, is premised on bringing expertise from the general public into government, it is important to recognize that city governments possess large, dispersed populations of individuals with a diversity of skills and experience. The lessons learned from open innovation in business and expert networking in a variety of sectors and industries have not yet been put to use in city governments. With these insights as inspiration and the skills and experiences of city employees as fuel, ExpertNet will seek to make city governments across the world smarter.

In municipal government, different departments often operate in silos, essentially acting as their own organizations-within-an-organization – and suffering from the same kinds of limits as companies that rely exclusively on their internal research and development. For city governments, “open innovation” begins by having staffers in one department find and reach out to experts in other parts of the government. Government professionals possess tremendous experience, but no single employee or department can possibly possess the entirety of skills and experience necessary to address every problem encountered. There is no reason to think that people who have worked inside a bureaucracy for a long time will have access to all the best know-how. Getting better expertise within and across departments can improve how city government departments solve problems by bringing new ideas, perspectives and insights to bear.

What Are the Methods for Identifying and Measuring Expertise?

Credential-based 

Traditionally, credentials bestowed on individuals by widely recognized institutions have been the central means for demonstrating and identifying expertise. From doctorates earned at Ivy League universities to notary public licenses to plumbing certifications, people generally associate expertise with pre-defined systems of demonstrated aptitude administered and validated, through official documentation, by an authoritative institutional structure. While credentials remain the dominant means for demonstrating expertise, they don’t give a complete picture: people now often acquire skills through channels that exist outside traditional institutions.

Experience-based

For employees in most environments, experience-based expertise is the most important means for career advancement – following an initial reliance on credentials. The demonstrated ability to do good work, which may or may not be formally quantified, is a key barometer of who knows what in a workplace setting. Online, platforms like GitHub and Stack Exchange are using experience-based systems for quantifying and vetting expertise in a more useful and agile way than relying simply on credentials.

Reputation-based

Recommendation and endorsement systems for expertise are becoming popular in response to the growing disconnect between individuals’ skills and traditionally recognized systems of credentials. The paradigmatic examples of LinkedIn, Yelp and eBay demonstrate how community-sourced ratings, recommendations and endorsements can provide useful metrics in addition to or in place of traditional certifications. For example, an individual who has completed online training with CodeAcademy can list that credential on her LinkedIn profile page and have it endorsed by colleagues and clients, whether or not she has any academic degrees or credentials in web development.

What ExpertNet Will Capture

ExpertNet will collect, categorize, and manage structured information on expertise and experience. With an eye toward platform iteration and linking existing databases, ExpertNet will initially capture information in the following broad categories: Biographical, Education and Credentials, Skills, Interests, Projects, Partnerships, Work Experience, and Future Plans.

Whether expertise rooted in credentials, experience or reputation, information can be gleaned automatically from mining online biosketches and data sources such as public and grants records or self reported. At least in its early phases, self-reporting will be the central method we test for identifying expertise. As the platform seeks to develop and populate databases with information that can be used to identify and measure expertise, gaps in available data will need to be filled by employees self-reporting a variety of information on their backgrounds, skills, interests and experience.

Biographical

Inputs traditionally associated with HR systems, like name, position, specialty, department and geographic area, will play an important role in establishing a person’s identity. In addition to providing basic searchable information, the bio section can be seen as a type of e-business card with both personal and professional aspects. Such a system has been implemented by, for instance, LinkedIn as a way to identify professionals in a searchable way.

Education and Credentials

Questions related to educational attainment and other traditional credentials and certifications are one way to estimate a person’s capability. While the ExpertNet system will seek to look beyond such traditional metrics of expertise, the information found here will still provide potentially useful data. However, ExpertNet will seek to emulate platforms like Stack Exchange in the sense that demonstrated proficiency is valued much more highly than traditional forms of certification.

Skills

Beginning with a self-reporting system, and eventually linking other data sources in later phases, the centerpiece of the ExpertNet system focuses on employees’ skills. Two important features will ensure that skills are accurately listed in an employee’s profile: an automated, predictive system for skills input to ensure data consistency, and an endorsement system allowing other users to vet skills and recognize satisfactory work.

Interests

ExpertNet will ask users to self-report their professional and personal interests to help identify the types of expertise they want to leverage and the types of projects they want to contribute to. In later phases, ExpertNet will link that data with information drawn from existing social networks, like Facebook, to develop a more fully formed image of an individual’s interests.

Projects

ExpertNet will collect information on major projects the individual has completed, both to identify her relevant experience and to ensure that new work is not redundant. This focus on project experience is one of the factors that sets ExpertNet apart from traditional HR systems. Just as other users will have the ability to endorse an individual’s skills on her profile page, collaborators on a listed project will be able to give a colleague a “+1” for a job well done.

Partnerships

Collecting information on partnerships serves a dual purpose. First, it helps identify whether an individual has worked in a collaborative capacity across departments in the past, whether she felt those experiences were positive and if she is willing to partner with other departments in the future. Second, by giving users the ability to list people that they’ve collaborated with (both within the same department and otherwise) and note whether that experience was beneficial, ExpertNet will supplement the endorsement/reputation system in place for the Skills and Projects section. Over time, this form of questioning will show which employees are consistently taking part in fruitful collaborations.

Work Experience

Information on work experience will provide context for self-reported skills and help identify any dormant professional expertise not being leveraged in an individual’s current position. Even more than the skills and interests sections, the ability to link self-reported data with existing professional data stores such as LinkedIn will enable the continued development of a rich and useful database on work experience, while simultaneously making it easier for the user to create her profile.

Future Plans

Information on an employee’s future plans can help identify those who will be willing to work on a given project. For example, if an administrative employee lists “Web Developer” as her desired future position, perhaps that’s a sign that she would be less willing to use her free time to help work on an outside project in an administrative capacity and more likely to help out with a web project.

Potential Risks and Barriers

While the GovLab is confident that ExpertNet will benefit city governments and their employees in many ways, the project is not without risks and barriers, including:

  • Data Availability. The biggest initial concern for the project is likely to be the availability of useful government data. While other sources can provide many types of information, project data must either be self-reported by employees or obtained from official government data sources. If such data is not available, it will take time and manual entry to input data in a usable form.
  • Low-Quality Self-Reported Data. Low-quality self-reported data could also pose challenges to the usefulness of the system. While it will not be easy to obtain accurate, uniform data through self-reporting, the project will take steps to mitigate concerns – including the use of a predictive entry system.
  • Cultural Resistance. Traditional institutional culture will have to be transformed to break down departmental siloes and increase collaboration among city employees. While the potential benefits and uses of ExpertNet will likely inspire many employees to experiment with it, we expect some resistance at different organizational levels.
  • Political Concerns. ExpertNet and its underlying principles are wholly apolitical. However, given the oft-partisan climate of many governments, political issues are still likely to arise.
  • Constraints and Challenges Unique to Certain Contexts. Many constraints and challenges will be impossible to predict until work on the project begins once a partnership is formalized.   

The GovLab’s Approach: Living Labs and Action Research

The late MIT Professor Kurt Lewin, a pioneer of modern social psychology who coined the term action research, said, “research that produces nothing but books will not suffice.” This notion is central to the GovLab’s Living Labs projects, which are undertaken using an action research methodology.

Action research involves experimenting in real-world settings with a partner organization. In this process, the research team and partners work together to collect and analyze data, evaluate performance based on various metrics, and change practices based on the feedback or evidence gathered.

In contrast to traditional research, which poses a question, collects and analyzes data, interprets the results and publishes findings – leaving to others the task of determining what can be done with the results – action research combines research and its application seamlessly. Action research is both a research process and an approach to affecting social or organizational change: It actually implements potential solutions and studies them as part of the research process. The outcomes of action research are both solutions to social and organizational problems and contributions to scientific knowledge and theory. 

The Living Labs are intended to have a broad impact beyond their benefits to the GovLab’s partner institutions. The lessons learned in our Living Labs will inform overriding efforts to transform dysfunctional institutions and help create blueprints for effective, legitimate governance.

The GovLab’s Living Labs action research methodology is defined by the following objectives:

  • Developing practical designs for collaborative democracy;
  • Putting these designs into practice with institutional partners;
  • Doing research on the outcomes to advance our understanding of different approaches; and
  • Using that understanding to design subsequent implementations.

With these organizing objectives and this general methodology in mind, the GovLab’s Living Labs will follow a well-defined process, tailored to the needs, abilities and resources of our partner institutions.

The GovLab’s Living Labs Methodology:

Methodology

ExpertNet Central Research Questions and Variables

The GovLab’s work on ExpertNet will be organized around key research questions, both to ensure a focused approach to developing the project itself and to help develop a blueprint for broader innovation of institutional governance. Our initial research questions, divided by experiment phase and variable, are the following:

 Phase 1

  • Expertise: Identification
    • How do we know who knows what and target requests to participate?
    • Can we supplement the use of the three paradigmatic techniques for identifying expertise – reputation, credential and experience – with self-reported information to obtain a full understanding of individuals’ expertise and experience?
    • Through strategic data collection and analysis, is it possible to determine not only who is capable of helping to address a problem, but also who is willing to do so?
  • Expertise: Matching
    • What can we learn from experiences in online search and advertising, academic discovery and other contexts regarding how to match people’s skills to work in different contexts, and apply those lessons to city governments?
  • Expertise: Incentives and Motivation
    • What incentives cause people to share their knowledge and expertise?
    • How do incentives and motivations to participate in Smarter Governance initiatives differ when the expertise being leveraged exists in a constrained, clearly defined workplace environment, rather than existing outside of institutional borders within the general public?
  • Barriers: Cultural and Legal Challenges and Readiness
    • What are the legal and cultural impediments for cities to be open to expertise?
    • How can intentionally siloed city departments be pushed to open themselves to leveraging outside expertise and sharing their own?

Phase 2

  • Expertise: Identification
    • How can we identify the types of problems that are best-suited to getting expertise in?
  • Expertise: Matching
    • How do we match skills and expertise to problems?
  • Expertise: Incentives and Motivation
    • Are gamification techniques – such as point systems, badges and leaderboards – effective in incentivizing individuals to engage in city government expert networking?
  • Framing: Problem Definition
    • How can city employees define requests to expert colleagues so as to:
      • Articulate the problem in a language that captures interest and expertise otherwise untapped;
      • Clarify the urgency and need to find a solution;
      • Develop sufficiently granular problem statements for productive ideation;
      • Promote and embrace diverse and novel responses that expand the existing solution space; and
      • Entice the relevant experts to respond?
    • How does ambiguity in the problem statement or targeted request hinder success?
    • How can ambiguity be avoided when defining the problem?
    • How can problems be framed in a way that helps engage experts in unrelated fields who could contribute based on solutions encountered in their own work?

Phase 3

  • Expertise: Identification
    • How can different data sources – including internal government databases and existing social networks like LinkedIn – be linked together to provide a multi-faceted view of individuals’ expertise and experience?
  • Expertise: Matching
    • Can predictive information input and search functionalities improve expert matching?
  • Expertise: Incentives and Motivation
    • What are the extrinsic (like prizes) and intrinsic (such as ego, altruism, autonomy, curiosity, empathy, etc.) motivations for participation, and how do they compare within the context of problems within city governments?
  • Barriers: Cultural and Legal Challenges and Readiness
    • What absorptive capabilities must departments develop to assimilate the diverse knowledge of city employees?

Phase 4

  • Expertise: Incentives and Motivation
    • How can a single engagement platform cater to the heterogeneous motivations that must be taken into account to engage a diversity of expert individuals?
  • Barriers: Cultural and Legal Challenges and Readiness
    • Can an expertise identification and matching platform replace existing city HR systems?
    • How can we ensure that older employees and those with little tech-savvy are engaged in a modern, technology-based expertise discovery system?
    • Can internal expert networking help solve larger problems within city governments, or will use remain relegated to more constrained, individualized requests?

Metrics of Success

The development of meaningful systems of metrics and evaluation is a key facet of the Planning stage of the Living Labs methodology. While stakeholder engagement and targeted due diligence will be required to craft a targeted evaluation system, the GovLab has identified the following initial metrics of success:

Quantitative Measures

  • Number of Active Users. In time, ExpertNet aims to become an ingrained piece of partner cities’ HR frameworks, with profile creation used as a part of the onboarding process. Initially, however, the number of employees taking part in the system will provide a useful measure of interest and engagement. Changes in use over time will help determine if the system is experiencing a network effect, where use inspires more use.
  • Total Endorsements, Recommendations and Ratings. Similar to the Active Users metric, the total number of endorsements, recommendations and ratings will provide a useful measure of employees’ continued use of and engagement with the system.
  • Number of Unique Users Earning Badges and Endorsements. Studying the number of unique users earning badges and endorsements, especially focusing on changes over time, will provide a means for measuring the success of the ExpertNet incentive and motivation system.
  • Number Listed Projects. Considering the central objective of identifying which projects different city employees are working on or have recently worked on, the number of projects listed in the system, whether self-reported or drawn from an existing database, will provide a means for determining the platform’s success in helping employees avoid redundant work.
  • Number Successful and Unsuccessful Expertise Search Queries. While also providing a view of which types of skills and experience do and don’t exist within city governments, the number of successful and unsuccessful search queries will also help identify any gaps in collected data.

Social Science Measures

  • User Satisfaction. ExpertNet is meant to improve the problem-solving capacity of city employees. As such, likely the best metric of success will come from surveys of participating employees asking about their satisfaction with the platform.
  • Employee Engagement and Tendency Toward Collaboration. Obtaining insight into ExpertNet’s role in increasing employee engagement and interest in collaboration will prove more difficult, but will be essential for determining the platform’s success in relation to its objectives.
  • Perceived Improvement in Problem-Solving. In theory, increasing employee collaboration and the capacity to identify and engage expertise will improve decision-making across departments. However, testing this assumption by measuring the problem-solving success of people utilizing the platform will be essential.
  • Assessment by Directors and Supervisors. In addition to surveying city employees in general, studying the assessments of directors and supervisors will help determine whether the system has been useful for their employees. Perhaps more importantly, engaging directors and supervisors will help determine if the platform is successfully navigating cultural barriers within city government.

[I] Henry Chesbrough, “Open Innovation: A New Paradigm for Understanding Industrial Innovation,” in Open Innovation: Researching a New Paradigm, eds. H. Chesbrough and W. Vanhaverbeke and J. West, (New York: Oxford University Press, 2006).

[II] Ibid.

[III] Henry Chesbrough, Open innovation: The new imperative for creating and profiting from technology, (Boston: Harvard Business School Press, 2003).

[IV] David Weinberger, Too Big to Know: Rethinking Knowledge Now That the Facts Aren’t the Fact, Experts Are Everywhere, and the Smartest Person in the Room Is the Room, (New York: Basic Books, 2012).

[V] Linus Dahlander and David M. Gann, “How open is innovation?” Research Policy 29, no. 6 (July 2010): 699-709.

[VI] Frank Piller, Christoph Ihl and Alexander Vossen, “A typology of customer co-creation in the innovation process,” available at SSRN, December 29, 2010, http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1732127

[VII] See Dawn Cowie, “Noble launches expert network,” Financial News, February 10, 2009, http://www.efinancialnews.com/story/2009-02-10/noble-launches-expert- network?ea9c8a2de0ee111045601ab04d673622

[VIII] See Andrew Lu, “When Should Attorneys Hire Expert Witnesses?” FindLaw, December 13, 2012, http://blogs.findlaw.com/strategist/2012/12/when-should-attorneys-hire-expert-witnesses.html

[IX] Beth Simone Noveck, “Peer to Patent: Collective intelligence, open review, and patent reform,” Harvard Journal of Law & Technology 20 (2006): 123.

Trackbacks/Pingbacks

  1. An ExpertNet Pilot at the FDA? - December 7, 2013

    […] GovLab workshop earlier this year had ExpertNet on the agenda. They appear to have since been exploring opportunities at the local […]

Leave a Reply