Trip-Report Classification Methodology
Every community experience report on this site carries a tag — positive, neutral, or cautionary — summarising the author's own verdict on their experience. This page documents exactly how that tag is computed: the signals, the weights, the thresholds, and the measured accuracy. The system is deterministic and runs entirely locally; every tag can be reproduced and explained from the report text. (For the effect-profile scores, see the Effect Profile Scoring Methodology.)
Why the system was rebuilt
The original classifier matched keyword patterns against a report's title and its first 500 characters only. Trip reports are narratives: they open with scene-setting and deliver their verdict at the end. The failure that prompted this rebuild was Erowid exp #79150 (fluoxetine), which opens “I was always an outgoing, extroverted kid… I loved life” and ends “a massive snowball of mental issues that might take years to recover from.” The old classifier saw only the opening and tagged it positive — while Erowid's own reviewers had filed it under Health Problems, a tag the pipeline never read. The rebuilt system reads the whole report and the editorial tags, and re-tagging the corpus changed roughly half of all labels.
Signals
Each Erowid report is scored on a continuous scale from −1 (unambiguous harm story) to +1 (unambiguously valued experience) by combining five signals. The tables below are rendered live from the exact constants the scorer uses, so this page always matches the code.
Erowid's human reviewers attach topic categories to ~99% of reports, including outcome categories. A report tagged Health Problems or Train Wrecks & Trip Disasters was judged by a person to describe harm; that judgement outweighs any single keyword. The first strong negative tag contributes −0.45 and each additional one −0.15; positive tags contribute the weights listed below.
Authors put verdicts in titles (“Never Again”, “Best Night of My Life”). A strong-negative title pattern contributes −0.40, a strong-positive +0.35, and mild vocabulary ±0.08 per hit, capped at ±0.16.
The final fifth of the report (at least 400 and at most 2,500 characters) is scored separately, because the author's summing-up lives there. Strong vocabulary counts double; the net count is scaled by 0.09 per hit. This is the signal that fixes the #79150 failure class.
The whole text is scanned with the same lexicon, and the net count is divided by the report's length in thousands of words — so a long report does not accumulate spurious hits, and a short furious one is not diluted.
Unambiguous emergency markers — hospitalisation, ambulance, seizures, overdose, psychiatric admission — contribute −0.12 each wherever they appear, unless negated (“didn't need the hospital” does not count). A report containing an un-negated harm event can never be tagged positive.
Negation handling
Every lexicon hit is checked against the 40 characters before it. “No regrets” does not count as regret, “never felt better” is recognised as a positive idiom, and “avoided the emergency room” is not a harm event. This voiding applies to all four lexicons and the event detector alike.
Editorial topic weights
The Erowid category ids below were verified against the full 41,408-report corpus (topic and substance ids share one namespace; only outcome topics are mapped).
| Erowid category | id | Contribution |
|---|---|---|
| Bad Trips | 6 | −0.45 first / −0.15 each further |
| Train Wrecks & Trip Disasters | 7 | −0.45 first / −0.15 each further |
| Post Trip Problems | 8 | −0.45 first / −0.15 each further |
| Addiction & Habituation | 10 | −0.45 first / −0.15 each further |
| Health Problems | 27 | −0.45 first / −0.15 each further |
| Overdose | 29 | −0.45 first / −0.15 each further |
| Hospital | 36 | −0.45 first / −0.15 each further |
| Difficult Experiences | 5 | −0.18 |
| Loss of Magic | 34 | −0.18 |
| Glowing Experiences | 4 | +0.45 |
| Mystical Experiences | 9 | +0.20 |
| Health Benefits | 32 | +0.20 |
From score to label
The signal sum is clamped to [−1, +1] and mapped to the three display labels with deliberately asymmetric thresholds:
cautionary ≤ -0.15
< neutral < +0.22 ≤ positive
The cautionary boundary sits closer to zero than the positive one because the two mistakes are not symmetrical on a harm-reduction site: a harm story mislabelled “positive” misleads in the dangerous direction, while a good story mislabelled “neutral” merely undersells. Two further guards enforce the same asymmetry: a report with a strong negative editorial tag, or an un-negated harm event, is never labelled positive regardless of its score; and if such a tag is present and the text does not actively push back, the label stays cautionary.
Alongside the label, each report stores its raw score, a confidence grade (high / medium / low — driven by the score's magnitude and whether an editorial tag agrees), and a mixed flag set when a report carries substantial evidence in both directions — a genuinely great night followed by a hospital visit is mixed, not an average.
What each source can support
Measured accuracy
Before the re-tagged corpus shipped, a stratified sample of 54 Erowid reports (six from every old → new label transition, so the sample is deliberately hard) was independently labelled by two blinded raters reading the full text, with a third rater breaking ties. Against that reference:
| Old classifier | This system | |
|---|---|---|
| Agreement with human raters | 22/54 (41%) | 39/54 (72%) |
| Harm stories labelled “positive” | 4 of 16 | 0 of 16 |
The sample deliberately over-weights the hard cases: it draws equally from every old-to-new label transition, including the classes where the two systems disagree, so corpus-wide accuracy is higher than the sampled figure. The raters themselves agreed with each other on 49 of 54 reports (91%), which bounds what any classifier can score on this reference. The remaining disagreements are dominated by genuinely contested reports — authors who valued an experience that Erowid’s reviewers filed under a harm category; this system sides with the reviewer in those cases, by design.
Known limitations
The scorer reads English only, cannot detect sarcasm, and can be misled by long quoted passages describing someone else's experience. On Bluelight and Reddit it sees only an excerpt, so a thread whose excerpt is scene-setting can still be under-tagged — the same failure the Erowid pipeline used to have, disclosed here rather than hidden. Reports with little affect vocabulary in either direction default to neutral at low confidence; neutral is the honest answer when the text does not argue.