What is an inclusive content analysis?

An inclusive content analysis is a way to understand the nature of content—meaning what perspectives are included in stories, what language is used, what images are used, and more—at a high level through quantitative and qualitative data. It's a way to test assumptions about content, set baselines, and demonstrate whether anecdotes are representative of trends or outliers.

How is an inclusive content analysis different from a source diversity audit?

A source diversity audit looks at individual sources and their demographic traits, and the purpose is to help newsrooms answer questions about source representation vis-à-vis the population of their intended audience. An inclusive content analysis looks at pieces of journalism as a whole, and the purpose is to gain a broader understanding about newsroom choices as they pertain to framing, language, and representation.

Why is an inclusive content analysis useful, and who is it useful for?

Our analyses answer questions that newsrooms have about their content with reliable data. This information can be used to set organizational strategies and agendas and help newsrooms make informed decisions. Beyond that, inclusive content analyses can be cited as reliable research to share with stakeholders, including general audiences and philanthropic funders.

Use cases include refining existing style guides to be more equitable, providing clearer guidance and resources for image usage, and publishing public reports in an effort to be more transparent and building trust with audiences.

How do you do it?

First and foremost, we ask our newsroom partners what their priority questions are. Everything flows from what the newsroom wants to know about their content, whether it's testing an assumption or wanting to have hard numbers to back up claims. From there, we build a coding framework that will answer those questions. The framework includes explicit definitions of terms to ensure reliable application of the framework. Most questions can be answered either yes or no, but some require more nuance. Sample questions include:

• Does any image depict a stereotype, especially but not exclusively when depicting people belonging to historically excluded groups?

• What is the primary point of view of the article? ("Primary" meaning more than half of the paragraphs, voices, and/or images.)

1. Institutional (voices are those that speak on behalf of an entity)

2. Community (voices are those that speak about an individual or collective experience)

3. Perspective (unsourced articles whose point of view is that of the author)

From there, we develop a representative, randomized sample of content (supplied by the newsroom), apply the framework through content coding, and deliver results in the form of a report, presentation, and a customized Looker Studio dashboard.

How did IA develop its approach?

The foundations of this work go back to 2016, when Director of Research and Evaluation Eric Garcia McKinley began developing a source diversity tracking system for the Minnesota Public Radio newsroom, an initiative that continued after he departed the organization in 2018. After conducting multiple source diversity analyses at Impact Architects since 2019, based on what we learned we evolved the approach to focus more on content than source demographics. In source audits, in addition to source demographics we always included details about the pieces of journalism they appeared in, and some of our greatest insights came from these details, such as considering in what context Black sources are most likely to appear. This led us to consider source representation more broadly, for example identifying whether institutional or community voices are prioritized more frequently, rather than identifying individual sources in a story. 

Where can I find more information?

While we at Impact Architects have developed our unique approach, we use a lot of excellent public information about inclusive content. A few of our favorite resources that inform our work include:

• Resolve Philly's Modifier, especially its resources and Revisions newsletter

• Vox Media's Language, Please

• LAist's Dialogue Style Guide

Still want to learn more? Schedule a 30-minute meeting with Impact Architects' Director of Research and Evaluation Eric Garcia McKinley, PhD.