Last year, the Advertising team at The New York Times asked a question: could we accurately predict the emotions that are evoked by Times articles? If so, we could empower advertisers to place ads more relevant to the context in which they are shown. To explore this idea, The Times’s Data Science team launched Project Feels, a project to understand and predict the emotional impact of Times articles.

In a nutshell, we built prediction algorithms with large amounts of data collected via crowdsourcing. Our predictions made sense qualitatively, and we ran successful experiments demonstrating that readers’ emotional response positively correlated with engagement on articles. This approach, called perspective targeting, was one of the first data products launched by nytDEMO, a new initiative aimed at helping advertisers place the right marketer stories with the right articles.

To be clear: this is an advertising project and was done without coordination with the newsroom; its findings will never impact our news report or other editorial decisions.

Data Collection

To learn to predict emotions from articles, we first needed the right data. We surveyed over 1,200 readers who participated voluntarily to create our initial dataset. This was the first time The New York Times ever systematically crowdsourced data for machine learning.

We asked respondents how they felt while reading a series of articles and asked them to choose from a number of different emotion categories (which were learned from earlier experiments), as well as a No Emotion category.