Assistive AI Accuracy Dashboard

The Assistive AI Accuracy dashboard shows how well Magic AI’s suggestions are landing on your service desk — and, where they aren’t, exactly who or what changed them. Use it to tune AI, defend accuracy numbers in a review, and find the boards where technicians disagree with the AI most. Open it from Thread admin → General → Analytics.

​

What this dashboard answers

• How accurate are Magic AI’s priority, category, and title suggestions this month, and is that improving?
• Which boards have the lowest accuracy?
• When a suggestion is changed, who or what changed it — a technician, an automation, or a change made in the PSA?
• Which individual tickets sit behind an override, and what did the value change from and to?

​

Filters and time windows

A filter bar at the top of the dashboard controls both tabs: Changing a filter re-runs the underlying queries rather than re-slicing what’s already on screen, so results can take a moment to come back. A Clear filters button appears as soon as anything differs from the defaults, and resets everything — including the date range.

​

What’s on the dashboard

​

Accuracy tab

KPI tiles show the latest month’s accuracy for each feature, each with a small up or down badge comparing it to the month before, plus a tile for total exceptions and a tile for total AI suggestions. Hover any tile for a tooltip explaining precisely what it measures. Trend chart plots monthly accuracy as one line per feature, colored consistently throughout the dashboard — teal for Prioritization, blue for Categorization, orange for Title. Magic insights is a collapsible AI-written narrative interpreting the trend for a leadership audience. It is built from a fixed trailing three-month window, not from your current filter selection, so it won’t follow you if you widen the date range.

​

Exceptions tab

Override patterns table groups exceptions by board and override source, and shows per-feature accuracy alongside the volume — so you can read accuracy and the exceptions driving it together in one row. Override source classifies who or what changed the AI’s suggestion: Clicking a row opens a drawer listing the underlying tickets, each showing the AI-suggested value next to the final value, with search, sort, and export. Both the table and the drawer export to CSV.

​

Metric definitions

​

Things to watch

• The earliest date shown is the earliest accuracy data recorded for your workspace — not a display limit. If your history is shorter than 12 months, All time and Last 12 months legitimately show the same range.
• The header scope note — for example “Aug 2025 – Jul 2026 · 7 boards · 412 overrides” — is computed from the same filtered and grouped data as the table beneath it, so the two are meant to agree.
• Changes made directly in the PSA cannot be credited to a member. This is a PSA API limitation around impersonation, so those overrides land in the PSA-attributed sources rather than naming a technician. See Things to know & gotchas.
• Every figure is computed upstream — the dashboard only formats what it receives, so a number here matches the same metric elsewhere in Magic Analytics.
• Data refreshes daily. Today’s suggestions and overrides generally appear tomorrow.

​

Next steps