Where pure Playwright still falls over

Both Chart.js and Apache ECharts render bar charts by drawing pixels into an HTML <canvas> element. The axis labels ("Monday", "Wednesday") and data labels ("60") are part of that picture — they are not text nodes anywhere in the DOM. A human reading the chart sees them; Playwright querying the DOM does not. The page snapshot shown to Playwright contains only the <h1>, because that is everything the DOM actually has.

A visual agent looks at the rendered screenshot the same way a human does. Optical character recognition (or a multimodal model) extracts "Wednesday" and "60" from pixels. The assertion can be expressed in user terms — "the Wednesday bar shows 60 hours" — and verified against what is on screen.

Switching the same chart to Highcharts (which renders to SVG) makes Playwright work normally — SVG <text> nodes are real DOM. The split is canvas vs SVG, not "charts in general".

The natural way to confirm a 3D viewer is rendering is to read the canvas pixels. Three.js — and almost every other WebGL library — uses preserveDrawingBuffer: false by default. The browser is allowed to wipe the buffer between frames, so gl.readPixels() from outside the render loop returns all zeros even when the scene is visibly drawn on screen. The cube is there for a human; from Playwright's vantage point the canvas is black.

A screenshot of the actual rendered page does not go through the WebGL context — it captures the compositor output, the same thing your monitor displays. A visual agent has the rendered cube to work with directly, in colour.

Mapbox GL JS (and similar libraries like Cesium and Deck.gl) paint streets, place names and POI labels directly into a WebGL canvas from vector-tile data. None of the human-readable text on the map exists in the DOM. The same flow on a non-WebGL library — say, Leaflet with raster tiles and HTML markers — would work fine, because labels there are real DOM nodes.

Vision sees the map the way the user does. "Verify the city name 'Brno' appears near the centre" becomes a screenshot crop plus text recognition — exactly how a tester would describe the check verbally.

This is the same family of detections Bot Arena demonstrates — navigator.webdriver, missing chrome.runtime, HeadlessChrome in the user agent, software-only WebGL renderer, and so on. Public test pages like sannysoft and CreepJS list them out openly: thirteen of them fail for plain Playwright. The same arithmetic plays out silently inside Cloudflare, Akamai or PerimeterX in front of real customer portals.

A VNC-driven real Chrome is a real Chrome — none of the thirteen flags fire because the browser is not under automation control from the inside. The Bot Arena failure report shows this on five layered scenarios.

reCAPTCHA's silent path is exactly the same trust check as Cloudflare Turnstile (Bot Arena Level 5). When the checkbox click comes from a browser that has tripped the automation tells, Google falls back to an image challenge — pick the bicycles, etc. — that Playwright cannot solve. The token is never issued and the demo form never reports success.

Same reason as Turnstile: a real Chrome session driven via VNC has a real fingerprint, real mouse trajectory, and real history. reCAPTCHA hands it the silent pass and the form submits without an image challenge ever appearing.

The HTML <video> element exposes currentTime, duration and paused — nothing about what is being decoded into the visible frame. A test author who wants to verify "the right training video is playing", "the speaker has appeared", or "the closing logo is shown" cannot get that from the DOM. The same applies to live-streaming dashboards, video-conferencing tiles, screen-share previews and CCTV grids.

A frame screenshot plus a vision model answers the question directly: "describe what is on screen". The assertion can be the same plain English the customer used in the bug report.

The Ctrl+S download actually succeeds — bpmn-js exports valid XML. The failure is more surprising: only "Proces" was recorded as the task name, not "Process order". page.keyboard.type fires synthetic events as fast as Chromium will accept them, with no inter-key delay. bpmn-js's text-input overlay debounces its label commit against the next blur/Escape — and Playwright's Escape arrives before the last few characters of "Process order" have made it through the debounce. A human typing at human speed never has this race.

OS-level key events are paced by the real input pipeline. The same flow driven through VNC arrives at the application at human cadence, so debounce-style commits get the full string and the XML reflects what the user typed.

Odoo's web client is built on Owl, a reactive framework that mutates DOM around generated IDs and per-customer module configuration. Selectors that work today break on the next release; selectors that work on demo tenant A do not work on customer tenant B; the customer autocomplete in the invoice form is fed asynchronously and a synthetic fill() often does not trigger the surface of the matching menuitem in time. The same trained accountant who finishes this flow in thirty seconds never hits any of this.

A visual driver targets what the user sees — the field labelled "Customer", the dropdown row whose visible text is "ACME Corp", the button captioned "Confirm". Those labels stay constant across Odoo releases and across customer customisations. The same recording transfers to a different tenant whose underlying DOM looks completely different.