Data Stories · Psychology of Text

Communities Have Emotional Fingerprints

Google paid trained raters to read 58,000 Reddit comments and mark 27 emotions. The labels reveal two things: every community runs on its own emotional fuel, and humans profoundly disagree about feelings — which should worry anyone shipping an "emotion AI."

Who this data represents English Reddit comments from January 2019, drawn from popular subreddits, each labeled by 3–5 paid raters. The labels are third-party perceptions of text — not the commenter's felt emotion, not Reddit overall, not any offline population.

211,225

individual rater judgments across 58,011 comments

483

subreddits, each with its own emotional signature

99.1%

of the time raters split on grief — the least agreeable emotion

3.3×

gratitude rate in r/Divorce vs. Reddit overall

Ask five people what emotion a sentence carries and you will often get three answers. GoEmotions is the rare dataset built to expose that fact rather than sand it away: instead of one “ground truth” label per comment, it keeps every rater's judgment. That design choice turns a machine-learning benchmark into a small social-science instrument — one that can measure communities and raters at the same time.

What communities run on

Aggregate the judgments by subreddit and each community develops a legible fingerprint. Support communities — r/SuicideWatch, r/Divorce, r/depression — run on caring and gratitude: people arrive in crisis and thank the ones who show up. Political subs concentrate fear, anger and disapproval. Sports fandoms swing between pride and disappointment — the single most distinctive emotion of r/Mavericks fans is disappointment. None of this is surprising once seen; the point is that it is measurable from twenty-word comments.

Figure 1 · Emotional fingerprints of 25 communities

How much each subreddit over- or under-indexes on each emotion, relative to the average rate (color = lift; hover for details)

Rows are 25 communities curated to span the space — support, confession, humor, politics, dating, sports — each with ≥500 annotations. Columns group by valence: positive, ambiguous, negative, neutral. Blue = the community over-indexes on that emotion relative to the all-Reddit rate (up to 3×).

The emotions that travel together

Because raters could pick several labels per comment, the data also records which emotions co-occur in a single reading. The result is a map of emotional adjacency: fear ↔ nervousness is by far the tightest pair (18× more likely together than chance), followed by excitement ↔ joy and anger ↔ annoyance. Note what the strong pairs share — they are intensity gradations of the same underlying state, which is precisely what makes them hard to tell apart.

Figure 2 · The co-occurrence network

Emotions linked when one rater applies both to the same comment · edge weight = Jaccard overlap · drag the nodes

Node size = how often the emotion is selected; color = valence group in the GoEmotions taxonomy (positive, negative, ambiguous, neutral). The positive cluster (admiration–approval–gratitude–love) is dense; surprise and realization bridge the hemispheres.

Disagreement is the signal

Now the uncomfortable part. For every comment where at least one rater saw a given emotion, how often did the other raters see it too? For gratitude — with its unmistakable surface form, the words thank you — raters split only 56% of the time. For grief, they split 99.1% of the time. Almost never, in 560 comments touching grief, did all raters agree it was there.

Figure 3 · How often humans disagree, by emotion

Share of comments (≥2 raters, ≥1 selection) where raters split on the label

Every bar is a lower bound on how hard the concept is: raters were trained, paid, and reading the same words. The emotions models are most often asked to detect in the wild — grief, nervousness, disappointment — are the ones humans agree on least.

This is the epistemic-humility chart every “emotion recognition” product page should carry. A classifier reporting 92% accuracy on grief is being graded against labels that humans themselves reproduce less than half the time. The ceiling isn't the model; it's the concept. GoEmotions, to its credit, ships the disagreement — most benchmarks quietly vote it away.