Quality Framework for Integrated Data Systems

Governance: Overview

Data governance is the people, policies, and procedures that support how data are used and protected. Data governance for a cross-sector data sharing effort can draw upon one agency’s existing data governance practices, involve a separate set of policies and procedures, or be a hybrid of the two. Regardless of the approach taken, cross-sector governance policies and practices should be explicit and collaboratively agreed upon, rather than implicit and driven by any one partner.

Remember – data flow at the speed of trust. Cross-agency data governance is a way of institutionalizing trust to ensure that data use is legal, ethical, and a good idea.

Purpose, Mission, and Vision Driven

The mission and vision of a data integration effort should guide all other data governance decisions and practices.
Learn More

Practical and Strategic

In order to determine the optimal organizational roles and legal framework for data sharing, start by considering three key activities
Learn More

Collaborative

Governance policies and procedures should be developed cooperatively, and focus on building trust and strong relationships among partner organizations.
Learn More

Iterative and Flexible

Governance is an ongoing process, not a product or a project. It should be refined as a data sharing and integration effort evolves.
Learn More

Transparent

Data are a public good, and most integration efforts are largely funded with taxpayer dollars.
Learn More
Download the Quality Framework

Introduction to Data Sharing

Legal Report

Sample mission & governance documents from Hartford Data Collaborative

Learn more and engage with us to help state and local governments collaborate and responsibly use data to improve lives.

Our Services

Examples and Models

Self Assessment Tool

Purpose, Mission, and Vision Driven

The mission and vision of a data integration effort should guide all other data governance decisions and practices. Strong mission and vision statements include a clear articulation of mutual benefit for data partners and the broader community, as well as guiding principles. While the principles of each effort will be unique, they should include an acknowledgement of the risks of sharing data and clear commitments to mitigate those risks and ensure privacy is protected.

Different data sharing purposes – indicators and reporting; analytics, research, and evaluation; and operations and service delivery – will fulfill different visions, and have different legal and technical, and privacy implications. It’s important to define your specific purpose(s) for data sharing and work with partner organizations to build consensus and determine shared priorities.

Key question(s) to ask:

Do you have a mission or vision to guide why, how, what, and with whom you share and integrate data?
Do you acknowledge the risks and benefits of data sharing?
Do governance documents reflect a shared understanding of which purpose(s) will drive design?

Practical and Strategic

In order to determine the optimal organizational roles and legal framework for data sharing, start by considering three key activities:

  1. Hosting governance (including data and access procedures, managing legal agreements, stakeholder engagement and procedural oversight)
  2. Managing technology (including data transfer, security, storage, integration, and access)
  3. Conducting analysis (including methods, tools, visualizations, and insights)

An effort that is strategically positioned will assign governance roles that match the expertise and capacity of partner organizations.

Consider whether partners or organizations have:
– legal authority to use the data as intended by the identified purpose
– relationships with data partners
– staff capacity, and
– technical capacity for data management.

Efforts that carefully consider these practical and strategic questions early on are well poised to execute clear agreements that support strategic data flow.

Key question(s) to ask:

Who is managing your key activities (governance, technology, and analysis)?
Why was a particular organization chosen to manage these functions?
What relationships and resources are they well-positioned to receive and leverage?

Collaborative

Governance policies and procedures should be developed cooperatively, and focus on building trust and strong relationships among partner organizations.

At least three groups should support governance:

  • Deciders: An executive group that supports strategic decision-making, and includes leadership representatives from all data contributing agencies
  • Recommenders: A data or research subcommittee that supports agenda setting and project oversight, and includes content experts from data contributing agencies, relevant stakeholders, and academic partners (where applicable)
  • Doers: Staff who are charged with executing projects and key activities

Representatives of the communities reflected in the data should also be engaged throughout the governance process.

Key question(s) to ask:

How were governance policies and procedures developed?
Who is currently involved in the governance process? Who decides? Makes recommendations? Does the day to day work? How are community members represented?

Iterative and Flexible

Governance is an ongoing process, not a product or a project. It should be refined as a data sharing and integration effort evolves. Governance documents should be flexible enough to withstand inevitable staffing changes. They should also be revisited regularly, especially when major shifts in partners, purposes, or priorities occur. Since data integration is a developing field, keeping pace with national best practices (e.g., data security standards) is essential.

Key question(s) to ask:

When were governance documents and procedures created?
When were they last reviewed and updated?
Do they reflect current national best practices?
Do they reflect the priorities of partners and the community?

Transparent

Data are a public good, and most integration efforts are largely funded with taxpayer dollars. As such, transparency around what data are being shared and for what purpose is essential to creating accountability and ethical use.

Demonstrating and communicating the value of integrated data to diverse stakeholders is essential in order to build trust and sustain momentum for data sharing. Policies, protocols, and documentation of the data integration effort – as well as any specific projects the effort is engaged in – should be readily available to the public in understandable and accessible formats.

Key question(s) to ask:

What governance documents and protocols are publicly available?
How accessible are these documents?
How are data and analytics from the integration effort made publicly available?

Tiered

We recommend a tiered structure of legal agreements. While each will have different signatories and purposes, all agreements should use consistent terminology.

Executing a high-level, non-binding agreement among partners, such as a Letter of Intent (LOI), can be a helpful, optional first step towards clarifying the purpose of collaboration. This simple exercise can also signal executive support to data owners, data stewards, and legal counsel as they work towards agreement on the broader legal framework.

The Enterprise Memorandum of Understanding (EMOU) is an essential document where partners agree to the rules of the data sharing agreement. The EMOU governs the responsibilities of data partners and is signed by all participating agencies.

Next, a Data Sharing Agreement (DSA) with each partner operationalizes the use of identifiable data for linkage. Finally, a Data Use License (DUL) or Data Use Agreement (DUA) governs the release of data that have been de-identified for analysis, and is signed by the data user for each approved project.

Key question(s) to ask:

What legal agreement(s) or document(s) are used to facilitate data sharing and integration?
How do these documents work together?

Standardized but Flexible

Many sites operate with hundreds of data sharing agreements across agencies, each with different terms, structures, and signatories. Standardizing terms and conditions of access can improve work-flow, support insights, and reduce costs.

We recommend starting with a review of the agreements already used in your jurisdiction before selecting exemplars to template and use routinely across agencies. While this process requires an investment of time up front, it should make each subsequent negotiation faster and more predictable.

Using standard but modular documents can also increase the flexibility of legal agreements. For example, some sites have created a “terms” document that is linked in their agreements and updated regularly. This allows for inevitable changes to occur, without the need to renegotiate agreements in full.

Key question(s) to ask:

Has a scan been conducted of commonly used legal agreements for data sharing?
Is there a standard template/model for agreements?
If so, is it modular or malleable to potential project-specific needs?
How often are agreements renegotiated or amended?

Transparent and Comprehensible

Legal agreements – in particular those operating at higher levels of the tiered structure – should be written so that non-lawyers can follow along. We recommend the use of appendices to separate out things like security requirements and data elements from the main text of agreements.

In addition, if legal agreements themselves, or at least the existence of the agreements, can be made public, this can help establish trust with the public and earn social license for data sharing.

Key question(s) to ask:

How accessible are the language, length, and organization of legal agreements?
Can non-lawyers understand the content?
Are the agreements publicly available?

Technical Approach Driven by Purpose

There are many technical approaches to sharing and integrating data, often operating simultaneously in the same jurisdiction – or within the same agency – for different purposes.

For example, a Department of Health and Human Services might integrate client-level information across several programs into a dashboard to inform case management or care coordination. This is an operational use of data. The same department might also contribute de-identified or anonymized data to a cross-agency effort to understand the education and workforce trajectories of their clients. This is an analytic use of data. The two efforts may draw upon some of the same datasets, but they require different technical approaches.

Purpose(s) should always drive design when building or procuring IT infrastructure. Collaboration between executive leadership, technologists, practitioners, and data users is essential.

Key question(s) to ask:

Which purpose for data sharing will drive your technical approach?
If partners have identified more than one purpose for sharing, is everyone clear on the different technical approaches that will be required to fulfill those purposes?
Have the needs of decision-makers, practitioners, and other data users been clearly articulated to technologists?

Data are Relevant and High Quality

With the right technical approach, design, and infrastructure in place, the next step is to ensure that the data are relevant to the mission of the effort and question at hand, and that they are of appropriate quality to inform the answer.

We refer to these data concepts as “data relevancy” and “data quality.” Data relevancy considers whether the usefulness of the information meets the intended audience’s anticipated purposes. Data quality refers to whether disseminated information is accurate, reliable, and unbiased. Both relevance and quality should be assessed before data are used, and preferably before data are shared or linked at all.

Key question(s) to ask:

Are the data relevant to your mission, as well as the current research question or project?
How is data quality assessed and documented?

Transparent and Secure Data Access Procedures

While there is inherent risk in providing data access, there is also risk in under-utilizing data assets. Finding a balance between risk and benefit is important. We recommend that data sharing efforts make their data access procedures standard and transparent, differentiating internal vs. external access. This typically requires substantial staff and partner training, clear public communication, and multiple points of review and oversight.

Part of that oversight should focus on data security. Data security plans should include technical details regarding data encryption methods and expectations regarding routine maintenance, as well as the legal, procedural, and physical components aimed at preventing a data breach or security incident. While these events are uncommon, how a site prepares for and responds to such threats is essential for building and maintaining trust among data partners and the broader community.

Key question(s) to ask:

What data access procedures are in place for internal users? External users?
Who is in charge of ensuring data security?
Is there a multi-layered data security plan that involves technical, legal, procedural, and physical components? Is this information publicly available?

Strong Staffing and Planning

Staff of a data integration effort are the “doers” who carry out daily operations. This team should include diverse competencies to support both the relational and technical aspects of data sharing. Key responsibilities such as managing governance activities and communicating with data partners should be formally designated (i.e., stated in the staff member’s job description rather than listed as “other duties as assigned”). As efforts develop, the daily operations change. Planning is important to address staffing gaps in core competencies as they arise.

Key question(s) to ask:

Who staffs key roles (i.e., full- or part-time employees, contractors, students)?
Are key activities established in job descriptions, rather than “other duties as assigned”?
Are there diverse competencies among staff? What competencies are missing?
Do you have a strategic plan that addresses staffing?

Sustained investment & diverse funding sources

Different approaches to data sharing vary in terms of cost, but all require sustained investment over time in order to establish and grow a data sharing effort. Sustained investment enables partners to iteratively improve practice, build trust, and demonstrate impact.

Most efforts rely on multiple funding sources, including state and federal dollars, philanthropic grants, and fees for data access. Diverse funding sources enable data sharing efforts to manage and sustain operations in the event of a revenue loss or as priorities shift.

Key question(s) to ask:

How many sources of funding support data sharing?
Is funding project-specific, or is there core, recurring funding for staffing and infrastructure? 
Have public and private funding opportunities been explored? Federal and local opportunities?

Flexibility to Respond to New Needs & Opportunities

Policy priorities change. Sometimes emergencies create urgent information needs. We recommend regularly revisiting partner priorities in governance meetings in order to quickly identify and respond to new needs. Legal agreements and data access procedures should also be periodically reviewed to ensure that they allow for appropriate flexibility of data use.

Sites with flexible capacity are well-positioned to grow quickly as new opportunities arise. For example, data sharing efforts that were prepared ahead of time to adjust their focus, agreements, and access procedures in times of crisis have stepped in to support COVID-19 research and emergency response. In doing so, many have proved themselves essential and gained new data partners and resources.

Key question(s) to ask:

How flexible are staff, resources, governance, and legal structures?
Are partner priorities and information needs revisited often?
What are the current barriers to flexible response (e.g., strained relationships, complex legal process, restrictive access procedures)?

Data-driven Decision-making

Integrated data can inform decision-making in a variety of ways. Data can be used to better understand community needs and assets, evaluate what works, streamline work across agencies serving similar populations, and better allocate resources.

The extent to which an effort achieves these desired impacts depends on both how actionable their initial research questions are and how well they communicate findings to those who can take action. As you build the governance, legal, and technical capacities to share and integrate data more routinely, you will also need to ensure data insights are also shared and integrated on a regular basis.

Key question(s) to ask:

Are research questions clearly aligned to policy levers (i.e., if we knew the answer,
is there something we could do better)?
How are findings communicated to those with decision-making power?
How are findings communicated to participating agencies, partners,
and community stakeholders?

Timeliness

Timing is everything. Data and data insights are only useful if they are available when decisions need to be made. It is important to be able to leverage existing infrastructure to answer new questions in a matter of weeks or months, not years. If capacity is limited, plan work in phases: start with the simple, descriptive questions you can answer with the data you have, and use the resulting insights to build buy-in for additional data access and more complex analysis.

Key question(s) to ask:

Are data collected, accessed, analyzed, and results communicated in time for key decisions?
Do early results build support for additional data access and analysis?

Social License

Data sharing efforts must develop public approval – the “social license” to operate – in order to drive change. Social license comes from an effort’s perceived legitimacy, credibility, compliance with legal and privacy rules, and overall public trust. Earning it requires dedicating time and resources to develop relationships, source and incorporate feedback, and engage with diverse stakeholders on an ongoing basis.

It is particularly important to take your time building relationships and social license with BIPOC and other historically marginalized groups disproportionately harmed by government systems. For a detailed discussion of these issues and examples of strategies for building social license with a racial equity lens, see our Toolkit for Centering Racial Equity Throughout Data Integration.

Key question(s) to ask:

What evidence is there that social license has been built?
How have you changed relationships between agencies, communities,
staff, leadership, the public, and other stakeholders?
Are there key relationships missing?

This site is registered on Toolset.com as a development site.