A three-class hate speech classifier — hate speech, offensive, neither — with a case study that says plainly what it does and what it misses.
This is a ground-up rebuild of Hate-Classify: A Service Framework for Hate Speech Identification on Social Media, a Django/MySQL project. The machine learning is ported faithfully — same corpus, same preprocessing rule, same five algorithms, same three-model hard vote. Everything around it is new.
npm install
npm run dev # http://localhost:3000
The trained model is committed, so it runs immediately. npm run train re-fits it from the corpus in
about 1.5 seconds.
Every submission runs through the same six stages, server-side, in well under a millisecond:
raw text
│
├─ 1. preprocess lowercase · erase @mentions · non-alphanumerics → space
│ "you people don't belong here" → "you people don t belong here"
│
├─ 2. tokenise \b\w\w+\b — words of two or more characters
│
├─ 3. vectorise count each token's column in a 7,729-term vocabulary
│ unknown terms have no column, so they carry no weight at all
│
├─ 4. score five fitted models read the same sparse vector
│ Naive Bayes · Linear SVM · Logistic Regression · Decision Tree · SGD
│
├─ 5. vote three of them — NB + SVM + Logistic — cast a hard majority vote
│ the decision tree is scored but benched, as in the original
│
└─ 6. attribute Logistic Regression coefficients × term counts, ranked
→ the verdict, the spread, and the terms it rests on
The clip above shows the interesting failure first. Three voters split 2–1 for hate speech, while
the averaged posterior across those same three voters puts offensive ahead at 45.0% against 39.5%.
A hard vote reports the majority and discards how close the call was — so the analyser shows both
rather than picking the flattering one. It also shows the evidence: here, people and you are
carrying the decision, which is exactly the shallowness the limits section describes.
| Route | |
|---|---|
/ |
The case study — the corpus, the live reader, four decisions, the model bench, the full classification reports and confusion matrix, the limits |
/analyze |
Classify any sentence: all five models, the ensemble vote, and the terms each decision rests on |
There are no accounts and nothing is logged. The original's user registration and service-provider console were dropped: the classifier is the point, and every number they displayed comes from the trained model, so it lives in the case study instead.
Measured on a stratified 20% holdout (3,965 rows), seed 42.
| Model | Accuracy | Macro-F1 | Hate recall |
|---|---|---|---|
| SGD Classifier | 90.39% | 0.734 | 0.317 |
| Logistic Regression | 90.06% | 0.730 | 0.322 |
| Linear SVM | 89.03% | 0.717 | 0.330 |
| Decision Tree | 88.05% | 0.686 | 0.248 |
| Naive Bayes | 88.17% | 0.651 | 0.152 |
| Ensemble (hard vote) | 89.84% | 0.724 | 0.313 |
The gap between columns two and four is the whole finding. 77.6% of the corpus is a single class, so accuracy largely measures the corpus. The ensemble is ~90% accurate and finds roughly one hate post in three. Refitting Logistic Regression with balanced class weights moves hate recall 0.322 → 0.530 and costs 5.4 points of accuracy; both arms, swept across every decision threshold, are in the case study's fourth panel.
Verify any claim on the site:
npm run probe # the sentences the demo uses
npm run probe -- "your own sentence here"
npm test
70 assertions over the ported pipeline, run on Node's built-in test runner. They exist because a port that claims fidelity to scikit-learn has to be checkable, so they pin the things that would otherwise drift silently:
- the preprocessing regex, character for character, including the URL-shredding flaw
- the CSV reader against quoted commas, escaped quotes, embedded newlines and a BOM
- the token pattern and the alphabetically sorted vocabulary — the shipped weight matrices are indexed by column position, so a vocabulary that sorts differently pairs every weight with the wrong term and still produces plausible-looking output
- precision/recall/F1 against a hand-checked confusion matrix, and sklearn's
zero_division=0 - the hard vote's tie-break — numpy's argmax, lowest class id, which on this label map means a three-way split resolves to hate speech
- every figure published in the table above, asserted against the shipped model card
That last one is the point of the suite: retraining in a way that moves the numbers fails npm test
instead of quietly turning this README into fiction.
Kept, because changing it would make the reported metrics meaningless:
- The preprocessing rule, character for character —
re.sub("(@[A-Za-z0-9]+)|([^0-9A-Za-z \t])", " ", tweet.lower()) CountVectorizer's default token pattern\b\w\w+\band alphabetically sorted vocabulary- The label map (0 hate, 1 offensive, 2 neither) and the 80/20 split
- Multinomial Naive Bayes (α=1.0), LinearSVC, Logistic Regression, a gini decision tree, hinge-loss SGD
- The ensemble's exact membership: Naive Bayes + SVM + Logistic Regression, hard vote. The decision tree is trained and scored but never votes — as in the original
Changed, with reasons:
| Original | Here | |
|---|---|---|
| Training | Refit all five models from a 20k-row CSV on every prediction request | Fitted once offline; weights ship as JSON. ~40s → 0.2ms |
| Reproducibility | train_test_split unseeded — every run reported different accuracy |
Seeded, stratified |
| Vocabulary | Fit on the full corpus before splitting | Fit on the training split only; the original leaked test terms into training |
| Vocabulary size | Unbounded | min_df=2, capped at 20k — 19,473 terms seen, 7,729 kept |
| Accounts | Registration and login, passwords in plaintext CharField(max_length=10), raw user id in request.session |
Removed — nothing here needs to know who you are |
| Console | Gated on if admin == "Admin" and password == "Admin" inline in the view; users table, prediction log, .xls export via xlwt |
Removed. The model reports and confusion matrix it displayed are in the case study, ungated |
| Class ratio | Deleted and rebuilt a DB table per page view; divided by zero on a fresh install | Removed with the log it counted |
The one thing deliberately not fixed: the preprocessing rule strips punctuation before
tokenising, so URLs are not removed but shredded — t.co links leave co behind as a feature in
1,871 training documents. It is preserved so the metrics stay honest, and surfaced in the case study
as a finding.
data/tweets.csv 19,826 labelled tweets
scripts/train.ts offline training → src/model/
scripts/probe.ts classify from the terminal
tests/ the suite described above
src/lib/ml/ preprocessing, vectoriser, the five models, metrics
src/lib/classifier.ts runtime inference
src/model/bundle.gen.ts generated — the fitted weights the server imports
src/model/card.json generated — every number the UI renders
The model is embedded as a generated TypeScript module rather than an imported .json: a 1 MB JSON
import makes tsc infer a 7,729-element tuple type, and it removes any question of file tracing at
deploy time.
Training and tests run on Node's native TypeScript support (Node 22.18+), so there is no build step
and no ts-node.
It reads bag-of-words unigrams. It has no access to sarcasm, negation, or who is being addressed — "you people" scores as hate on the strength of "you" and "people". It inherits its annotators' judgements wholesale, including this corpus's documented tendency to mark African-American English as offensive more often than the same sentence in standard English. That bias is in the labels, not the architecture, so more training does not fix it.
A classifier is a suggestion. A person decides.