Meta Global AI Developer Hackathon · Startup track

Screenshot to Tokens

A design system from a screenshot in seconds. A more accessible app from the tokens in one change.

Point it at a live web app with no design tokens. SAM 3.1 reads the interface from pixels, Muse Spark names a tiered token set, a person reviews it, the hardcoded values in the real source become var(--token, original), and four checks prove nothing moved before a pull request opens.

The demo video

Title card of the demo video: Screenshot to Tokens, Meta Global AI Developer Hackathon, Startup track.
The 2:57 demo video. Watch it in the repo README (the repo is public from Nov 3; the video gets its own player here then).

What it does

Two Meta model steps (SAM 3.1, Muse Spark); everything else is deterministic code you can check.

  1. Capture

    Playwright opens the live web app and takes a full-page screenshot plus the computed style of every element.

  2. Segment

    SAM 3.1 finds buttons, inputs and icons from the pixels, even where the markup describes nothing.

  3. Name

    Muse Spark names a three-tier token set, primitive → semantic → component, with a reason for each token. Code checks every proposal.

  4. Review

    A person accepts, renames, rejects or merges tokens in the HTML report.

  5. Rewrite

    Hardcoded values in the real source become var(--token, original), so the original stays as the fallback.

  6. Verify, then open a pull request

    Four checks: fallbacks only must be 0 px changed, merged colors must be invisible, the full cleanup is shown for review, and no text may get harder to read (text contrast, WCAG AA).

Three tiers, one name each

Real lines from the Argon Dashboard run: tokens.css and the rewritten main.css.

  1. --color-blue-500: #5e72e4;Primitive · the raw value
  2. --color-action-primary: var(--color-blue-500);Semantic · what it means
  3. --button-bg: var(--color-action-primary);Component · where it’s used
  4. background-color: var(--button-bg, #5e72e4);Source · the original kept as the fallback
Change color.action.primary once and every button, badge and link follows. A name a person, or a model, can act on.

Try it

Argon Dashboard React (Creative Tim, MIT) on our fork's GitHub Pages: the original, the tokenized build, a proposed change previewed live, and the token playground.

  • Original

    The app as published, every color typed in place.

  • Tokenized

    Same pixels, now driven by tokens.css.

  • A proposed change, previewed

    color.action.primary set to orange: every button, badge and link follows.

  • Token playground

    Change any token, check text contrast, build a dark theme (static, no model calls).

The review reports

Evidence

All numbers from real runs, reproducible from the repo (as in docs/SUBMISSION.md).

Each claim, its measured result, and the file in the repo that holds it.
ClaimResultWhere
Rewrite is safe on real sites28 public, permissively licensed sites we didn't build: 3,687 values rewritten, 0 px changed on 23 of 23 stable sites with edits; a second run gave the same verdict on all 27 reachabledocs/breadth.md
Segmentation on real sitesHand-labeled first screens of 3 sites: the combined segmenter passes all threedocs/real-site-segeval.md
End to end, Argon Dashboard17 to 23 s, naming 4.2 to 4.9 s; checks 0 px · 0.000% · 0.919%docs/latency.md
Human reviewAccept, rename and merge take the fixture's answer-key score from 20/21 to 21/21, still 0 pxdemo/L5-review-take.md
Text contrast vs the W3C ruleACT rule afw4f7 (text has minimum contrast): 32 of 32 approved test cases consistent, 31 exactdocs/a11y.md
Text contrast vs axe-core 4.1428 sites: agrees on 308 of 309 violations and 2,165 of 2,174 passes, 99.9% agreement (1 opposite verdict in 2,483, on a page that animates); of the items axe-core can't decide (text on gradients and images), it settles 97% and flags the rest for reviewdocs/a11y.md
Icons and controls (SAM)On the div page, SAM found 9 controls the code doesn't describe; each measured for contrast and sizedocs/a11y-nontext.md
Visible keyboard focus30 real sites, 1,779 Tab stops: 52% show a focus indicator that meets the bar, 17 sites under half. One focus-ring token set takes Argon from 0 of 25 to 25 of 25, still 0 px when nothing is focused. SAM also finds controls a keyboard can't reach: on the div page, 7 of 7 clickable controlsdocs/keyboard.md
Dark mode as a token fileLight Bootstrap Dashboard: 83 of 84 text pairs pass in dark (14 of 84 in its original light theme)docs/dark-mode.md
Point and say: contrastArgon, click "Since last month", "This label is too light. Make it readable, keep our gray.": color.text.muted moves, 21 elements 2.95:1 → 4.53:1, whole page 41 → 20 failing text pairs, 0 regressions, 3 of 3 runs. Light Bootstrap: 12 elements 2.81:1 → 4.54:1docs/point-and-say.md
Point and say: keyboard focusArgon, "Give every control a visible focus ring in our blue.": visible focus on 0 → 25 of 25 controls, 0 worse, 3 of 3 runsdocs/point-and-say.md
Point and say: blocked on regressionArgon, "Make these buttons red.": blocked and reverted, naming "Month" (4.20:1 → 3.77:1) and 2 moredocs/point-and-say.md
Tests291 passingpnpm test
Run outputsthe verify and timing summaries behind every row above are committedruns/

Dark mode as a token file

Light Bootstrap Dashboard: 83 of 84 text pairs pass text contrast in dark (14 of 84 in its original light theme). One token file, every text pair checked; the top navbar is not a token, so it stays light.

Light Bootstrap Dashboard in its original light theme: white cards, a line chart and a pie chart, dark sidebar.
Before: the original light theme.
The same dashboard with the dark token file applied: black cards and page, light text, the same charts; the top navbar stays light.
After: the proposed dark token file.

28 public sites we didn't build

Mirrored, tokenized and verified end to end: 3,687 values rewritten, 0 px changed on 23 of 23 stable sites with edits. Two sites animate on their own and are reported, not counted as passes. From docs/breadth.md.

Per site, 2026-10-28 re-run. Safety checks: fallbacks only must be 0 px; merged colors must be invisible.
SiteFrameworkLicenceStatusTokensEditsSafety checks
wonyong-jang.github.iojekyllMITok70460fallback-only 0 px (pass); colors 0.000% (pass)
devbattery.github.iojekyllMITok106358fallback-only 0 px (pass); colors 0.000% (pass)
ragu-manjegowda.github.iojekyllMITok53125fallback-only 0 px (pass); colors 0.000% (pass)
tw93.github.iojekyllMITok421fallback-only 0 px (pass); colors 0.000% (pass)
josh-wong.github.iodocusaurusMITok6685fallback-only 0 px (pass); colors 0.000% (pass)
diogoribeiro7.github.iojekyllMITok75294fallback-only 0 px (pass); colors 0.000% (pass)
geuni620.github.iogatsbyISCok432fallback-only 0 px (pass); colors 0.000% (pass)
utzcoz.github.iojekyllApache-2.0ok3284fallback-only 0 px (pass); colors 0.000% (pass)
andra-sun.github.iojekyllMITok7450fallback-only 0 px (pass); colors 0.000% (pass)
awfp1314.github.ioastroMITok3617fallback-only 0 px (pass); colors 0.000% (pass)
proinsias.github.iojekyllMITok52243fallback-only 0 px (pass); colors 0.000% (pass)
gaotianchi.github.iojekyllMITok41213fallback-only 0 px (pass); colors 0.000% (pass)
burning-cost.github.iojekyllMITok1500fallback-only 43 px (FAIL); colors 0.000% (pass)
cgoldberg.github.iojekyllMITok7242fallback-only 0 px (pass); colors 0.000% (pass)
mavrikant.github.iojekyllMITok111305fallback-only 1100 px (FAIL); colors 0.010% (pass)
zzy979.github.iojekyllMITok7450fallback-only 0 px (pass); colors 0.000% (pass)
ethmarks.github.iohugoApache-2.0mirror-failed--site returns 404
nithinbekal.github.iojekyllMITok2942fallback-only 0 px (pass); colors 0.000% (pass)
infi-defi.github.iohugoMITok2716fallback-only 0 px (pass); colors 0.000% (pass)
cwwhitney.github.iohugoMITok36514fallback-only 0 px (pass); colors 0.331% (FAIL)
c-gyeltshen.github.iojekyllMITok00fallback-only 0 px (pass); colors 0.000% (pass)
ruzickap.github.iojekyllApache-2.0ok7643fallback-only 0 px (pass); colors 0.000% (pass)
stuocs.github.iojekyllMITok7251fallback-only 0 px (pass); colors 0.000% (pass)
cheonjoosung.github.iojekyllMITok2623fallback-only 0 px (pass); colors 0.000% (pass)
dieghernan.github.iojekyllMITok76502fallback-only 0 px (pass); colors 0.000% (pass)
hacksocnotts.github.iohugoMITok5065fallback-only 0 px (pass); colors 0.000% (pass)
linkarzu.github.iojekyllMITok7459fallback-only 0 px (pass); colors 0.000% (pass)
zeikar.github.iojekyllMITok6943fallback-only 29719 px (FAIL); colors 0.350% (FAIL)

Point and say

Try it live On Argon Dashboard, with a shared demo budget that counts down on the page.

Click a control in the live web app, then say or type the change. Meta's speech model transcribes it, SAM 3.1 outlines what was clicked, contrast and focus fixes are computed (no model), and every change is re-checked on the whole page and blocked if any text gets harder to read. Screenshots from the recorded runs on Argon:

Token playground beside Argon Dashboard. Request: this label is too light, make it readable, keep our gray. Result: text contrast on 21 elements goes from 2.95:1 to 4.53:1.
“Make it readable, keep our gray.” Text contrast: 21 elements 2.95:1 → 4.53:1, whole page 41 → 20 failing text pairs.
Token playground beside Argon Dashboard with focus rings drawn on the sidebar links. Result: visible keyboard focus on 0 of 25 controls before, 25 of 25 after.
“A visible focus ring in our blue.” Keyboard focus: 0 → 25 of 25 controls.
Token playground beside Argon Dashboard. Request: make these buttons red. Result: blocked, text contrast would regress on the Month button from 4.20:1 to 3.77:1, reverted automatically.
“Make these buttons red.” Blocked and reverted: “Month” would drop from 4.20:1 to 3.77:1.

Code and pull request