Solutions · Retail
Predict a review's star rating from its text
Score every product review 1–5 from its text and verified-purchase flag, so a moderation queue can be ordered by where the predicted rating and the given rating disagree. A worked example of why accuracy is the wrong metric when 63.8% of your labels are the same label.
- 63.8%
- Of reviews are 5-star — the majority class
- 45.8%
- Macro-F1, vs 15.6% for always guessing 5
- 616
- 1- and 2-star reviews in 10,000 rows
The problem
For each row in the reviews table, predict `rating` — the 1–5 stars the reviewer gave — from just two columns: the free-text `review_text` and the boolean `verified` purchase flag. The output is a predicted rating plus a probability for each of the five classes, so reviews can be ranked by how far the text sits from the stars attached to it.
Ask a question in plain English
You ask
“How are the star ratings distributed across these reviews?”
| Rating | Reviews | Share |
|---|---|---|
| 5 star | 6,378 | 63.8% |
| 4 star | 2,146 | 21.5% |
| 3 star | 860 | 8.6% |
| 2 star | 327 | 3.3% |
| 1 star | 289 | 2.9% |
This is the whole problem in one table. A rule with no model in it — always answer 5 — is right 63.8% of the time and surfaces none of the 616 one- and two-star reviews. Any scoring system whose headline number rewards that rule is measuring the wrong thing.
Explore the model dashboard
The same result, as an auto-generated dashboard you can share with a link — no login required for viewers.
What this model is for
Every marketplace has the same queue problem. Reviews arrive faster than anyone can read them, so the question is never “read everything” — it’s “read these first”.
This model gives every review a predicted 1–5 rating from its text alone, plus a probability for each of the five stars. That output is not interesting on its own. What’s interesting is the disagreement: a review whose text reads like a 2 while carrying 5 stars is a different object from a review where text and stars agree, and it is the kind of thing a human should look at.
But before any of that is worth building, this dataset makes a demand about how you’re allowed to measure it.
1. Why this is worth solving
Moderation doesn’t scale by reading more. It scales by ordering the queue well — and the ordering problem here is unusually stark.
Across these 10,000 reviews, 6,378 (63.8%) are 5-star and 2,146 (21.5%) are 4-star. The reviews most likely to matter to a moderator — the ones alleging a defect, a scam, a product that never arrived — are the 616 rated 1 or 2 stars. That is 6.2% of the queue holding most of what you’re looking for.
That shape is exactly where the lazy answer wins on paper:
- A rule that always answers “5 stars” is 63.8% accurate on this data.
- It surfaces none of those 616 reviews. It never predicts a 1, a 2, a 3 or a 4 even once.
- Its macro-F1 — the score that weights each rating equally — is 15.6%.
A rule that is right 63.8% of the time and flags nothing has not helped a moderation queue. Macro-F1 is the number that notices.
So the value of a model here isn’t “be accurate.” It’s “be useful about the 6.2%” — and you cannot tell whether you’ve achieved that by looking at accuracy at all.
2. Two numbers that disagree
The trained model reports accuracy 60.9% and macro-F1 45.8%. Those point in opposite directions, and both are true.
| Always answer “5” | This model | |
|---|---|---|
| Accuracy | 63.8% | 60.9% |
| Macro-F1 | 15.6% | 45.8% |
| Ratings it ever predicts | 1 of 5 | all 5 |
Accuracy asks how often the exact star was right. On a pile that is 63.8% five-star, that question is mostly about the five-star pile, and it can be answered well by ignoring the problem entirely.
Macro-F1 scores the model on each rating separately — how it handles 1-star, then 2-star, and so on — then averages the five evenly. A rating with 289 examples counts as much as one with 6,378. The always-5 rule scores zero on four of five ratings, so it lands at 15.6%.
The model is genuinely worse at naming the exact star across the whole pile, and genuinely better — roughly 2.9× on macro-F1 — at handling all five ratings instead of one. Which of those matters is decided by what you’re doing with it, and for a moderation queue it is the second.
One honest caveat: the baseline figures are computed on the full 10,000 rows, while the model’s are from its own evaluation split. The direction is not in doubt; the exact point gap is approximate.
3. How it actually fails
Five real calls against the deployed model, one per true rating, using reviews from the dataset:
| Actual | Predicted | Confidence |
|---|---|---|
| 1 ★ | 1 | 0.90 |
| 2 ★ | 3 | 0.54 |
| 3 ★ | 3 | 0.73 |
| 4 ★ | 5 | 0.48 (4 close behind at 0.43) |
| 5 ★ | 4 | 0.39 |
Two of five exact. Every miss is an adjacent star — and the one review a moderator would most want surfaced, the 1-star, came back correct at 0.90, the model’s most confident call of the five.
That is a shape worth understanding rather than averaging away. The model reads sentiment strength reasonably and struggles to split 4 from 5 — a distinction that matters enormously for a product page’s average and hardly at all for deciding what a human should read next.
These are five hand-picked illustrations, not an evaluation. Five calls support no rate, no error bar, and no claim that misses are always adjacent.
4. Try it
The panel below calls the live model. Paste any review — the strongest signal comes from strongly-worded text, and you can watch the probability spread rather than just the winning star. Near-ties between adjacent stars are the normal case, not a malfunction.
Note what the verified flag does. It carries 88.6% importance against the text’s 100.0%, so flipping it on the same text can move the prediction — a large share of this model’s signal sits on one boolean, which is a finding about the model rather than a feature of it.
Try it yourself
Every trained model gets a REST endpoint. Paste any review and run a real prediction.
Try the model
Paste any review and run a real prediction.
Review
Result
recordedpredicted rating
5
- prediction
- 5
- task_type
- multiclass_classification
- mode
- inductive
- version_label
- v1
A real response from this model, recorded on 2026-08-13. This review was given 4 stars by its author; the model puts 5 narrowly ahead of 4 (0.476 vs 0.434) — a near-tie, which is the model's most common failure shape. The model ID is redacted; the demo proxy resolves it server-side.
Frequently asked questions
Does this detect fake reviews?
No, and it is important to be exact about that. Nothing in this dataset labels a review as fake, paid, incentivised or bot-written — the columns are review_time, customer_id, product_id, rating, verified, review_text and summary. What was built and measured is star-rating prediction. It can point a human at reviews whose text does not match the stars attached to it, which is a useful signal to triage on, but a disagreement is not evidence of fraud and this model was never trained or evaluated against a fraud label.
The accuracy is 60.9%. Is that good?
It is worse than doing nothing, and that is the most useful thing on this page. Because 63.8% of these reviews are 5-star, a one-line rule that always answers 5 scores 63.8% accuracy — 2.9 points better than the model. Accuracy on a distribution this skewed mostly measures how big the majority class is. The metric that tracks the actual job is macro-F1, which scores each of the five ratings separately and averages them evenly: the always-5 rule scores 15.6% there, the model 45.8%.
So is macro-F1 45.8% good?
Better, not good. Macro-precision is 0.4391 and macro-recall is 0.4950 — both under half. Averaged across the five ratings, fewer than half of the model's calls for a given rating are correct, and it finds about half of the ones present. Treat it as a way to order a queue in front of a human, not as a decision. Per-class precision and recall for the 1- and 2-star classes — the ones this use case actually depends on — were not computed, so this page does not claim the model surfaces bad reviews well.
Are the two baselines measured on the same rows?
No, and the comparison is approximate because of it. The 63.8% accuracy and 15.6% macro-F1 for the always-5 rule are computed on the full 10,000-row sample. The model's 60.9% and 45.8% are the figures its training run reported, on its own evaluation split, and no holdout size or cross-validation detail was recorded for that run. The gap between the model and the baseline is large enough on macro-F1 that the direction is not in doubt; the exact point differences are not precise.
Why does `verified` matter so much?
It is the model's second feature at 88.6% importance against the text's 100.0%, which is worth flagging rather than celebrating: a large share of the reported signal sits on one boolean (6,913 verified against 3,087 not). Feature importance is a ranking from the dashboard, not evidence of causation. If you rebuild this on your own data, check whether that flag is carrying more weight than you want before relying on the output.
What data does it need?
One reviews table with the review text and a rating column to learn from. This build used only two of the seven available columns — review_time, customer_id, product_id and summary are present in the data and play no part in the result. Related tables can be linked in as-is; you don't pre-join them.
Does it work in languages other than English?
It will return a prediction for any language, and the demo accepts any script. It was trained on English reviews and no measurement was made of how it performs on anything else, so treat non-English output as unvalidated.