For about a year, "give your AI coding agent visual context" has meant one thing: you, sending pixels to the model. Paste a screenshot. Crop the bug. Wire up an MCP screenshot tool so it can grab a window on its own. All of it flows one direction — from your screen into the agent.
That is half a loop. Bidirectional visual feedback is the other half: the agent draws its understanding back onto your screen, and you confirm or correct it before it acts. The clip above shows the whole loop on a real page.
What "bidirectional" actually means
Two directions, both visual:
- You → agent (the half we already have): screenshots, crops, an annotated capture — a screen the agent can read. Covered in how to give your AI coding agent visual context.
- Agent → you (the missing half): the agent places labeled shapes — boxes, arrows, notes — directly on your live screen to show what it understood or what it intends to do. You correct those marks, and it reads your response.
The result is a conversation in pixels, not a one-way upload.
Why one direction isn't enough
When feedback only flows toward the agent, its intent stays invisible until it produces an output. You ask for a change, it goes quiet, and the next thing you see is a diff — which is the first moment you can tell whether it understood you. If it didn't, you are already in the expensive correction loop ("no, the other one"). As covered in why screenshots aren't enough, we made agents good at receiving visual context and did almost nothing to make their intent visible before they act.
The loop, in practice
The demo above runs the canonical case — an agent planning a layout:
- The agent draws its plan. Instead of silently generating a page, it sketches labeled regions on your screen: nav here, hero full-width, this is the primary CTA, a 3-up feature grid.
- You review it visually. The plan is right there, on the real page, as editable marks.
- You correct it. You annotate feedback — "hero should be full-width," "make the CTA bigger" — right on the screen.
- The agent reads your feedback and adjusts — then builds the corrected version.
The misunderstanding gets caught at the plan stage, where it is cheap to fix, instead of in the diff, where it is expensive.
How it works (over MCP)
Two tools do the work:
annotate_understanding— the agent draws labeled shapes onto a live overlay on your screen.get_annotation_feedback— the agent reads your review (your comments, what you changed) and acts on it.
Screentack hosts both over MCP, so any agent that speaks MCP — Claude Code, and others — can draw on your screen and read your corrections. Both tools are free.
What your agent can draw
The loop isn't only for page layouts. Anything an agent can express as labeled shapes, it can sketch on your screen for you to correct before it commits to it:
- UI & layout plans — where the nav, hero, CTA, and sections go (the demo above), or restyling an existing page.
- System & architecture diagrams — services, boundaries, and how data moves between them, before it scaffolds anything.
- Workflows & flowcharts — the steps it intends to take, branch by branch, before it runs them.
- Data models & dependency graphs — tables and relations, or what-depends-on-what, drawn over your code or canvas.
- Debugging — it marks where it thinks the bug is, on the actual UI, and you confirm or redirect.
The win is the same every time: the agent's thinking becomes a visible artifact you can correct, instead of a description you have to imagine or a build you have to undo. The highest-leverage version is validating a plan before any code exists — see how to validate your agent's plan before it writes code.
It is also the cheaper loop
Catching a misunderstanding at the intent stage costs a fraction of catching it in a generated artifact. We put numbers on that in annotate, don't regenerate: validating a lightweight plan instead of regenerating a real page to check it can cost roughly 2x fewer tokens, and the gap grows with artifact size.
The takeaway
Screenshots taught your agent to see. The next step is letting it show you what it sees — and letting you fix it before it builds. That two-way visual loop is what turns an agent from a black box into a collaborator.
Want a coding agent that shows you its plan before it writes the code? Download Screentack — a free 7-day trial, then $29 once.