ReAct vision loop
The agent does not answer in one shot. It Reasons, Acts with tools, and observes results — up to 6 steps. Figma context comes from example.json; the SSE replay is from agent-output-2.txt.
Run stats (from mock capture)
Step-by-step replay
Click each step to see what the agent did and why. Data is parsed through the same vision-stream.ts pipeline the wizard uses.
Macro scan (screenshot)
Initial observation
Agent reasoning (explained)
Gemini receives the rendered frame PNG from example.json (Google Pixel 2 - 1) plus a allowlist of valid node IDs. It scans for hierarchy clashes, typos, clipping, and spacing issues before calling tools.
Final structured output
After tool turns, the agent streams JSON. The client parser extracts enrichments and maps them to real node IDs from the investigation.
The descriptive text 'This will search vaccine availibility in your pincode area for today.' displays an orphan word ('today.') on the last line. This creates an awkward visual break and impacts readability.
Fix: Adjust the text container width or line-height to prevent orphan words, ensuring better text flow and visual balance. The `absoluteRenderBounds` width of 289.94px within a `absoluteBoundingBox` width of 320px indicates potential for better text reflow. While no specific guideline on orphan words was found, general typography principles suggest avoiding them. Refer to 'Typography System' in the design manual for overall text consistency principles.
The input fields within the 'Pincode input' component are too densely packed horizontally, leaving minimal spacing between individual input boxes. This creates a cramped visual appearance and deviates from consistent spacing.
Fix: Increase the horizontal spacing between the individual pincode input fields to improve readability and visual comfort. Ensure that spacing values are deliberate and follow the 8pt grid system. Refer to '8pt Grid System' and 'Deliberate Spacing' in the design manual: 'All spacing values must be divisible by 8 or 4' and 'Spacing between elements should follow the grid system' to create visual harmony.
How the stream becomes UI
connecting
Stream opens
investigating
Tool turns (ReAct loop)
synthesizing
JSON critique written
complete
Findings parsed
SSE chunks like start-step, tool-output-available, and text-delta are consumed by consumeVisionStreamChunks — the same code path as the live wizard, just fed from a saved file here.