Bot Arena

Out of arena

Where Playwright runs out of road in public SaaS

Modern web spreadsheets and enterprise data grids render to <canvas> for performance, putting the working surface beyond the reach of selector-based automation. Below: the canvas pattern at scale, why ERP + spreadsheets is the worst-affected combination, and a live Playwright-vs-AIVA demo against Odoo Spreadsheet as the exemplar.

The pattern, at scale

900M+

monthly users on a canvas-rendered spreadsheet

Google Sheets alone; Microsoft Excel adds another 750M+ (Microsoft, 2017 baseline), Smartsheet runs at 85% of Fortune 500. Google Docs migrated to canvas in May 2021 and broke a swathe of Chrome extensions in the process — selector tools only see the chrome around the grid.

The category

71%

of FP&A teams own EPM tools — and still run on spreadsheets

2025 AFP FP&A Benchmarking Survey. A separate 700-leader survey: 40% of businesses still manage half their financial data manually; 26% manage the majority manually (FinTech Strategy, 2025). Spreadsheets stay the de facto ERP surface — and vendors ship canvas spreadsheets to plug the gap.

Our exemplar

13M+

Odoo users — the canvas-ERP pattern with a public trial

40% YoY growth, 7,000 new clients/month, €5 billion valuation (Odoo SA, Nov 2024). ~15% of the SMB ERP market, projected 25% by 2027. Their Spreadsheet ships in every Enterprise edition — and the free trial lets us demonstrate the failure publicly, against a real grid, without violating anyone's terms.

Exemplar · live demo

Add a purchase order to Odoo Spreadsheet

Open-source ERP

https://testforme.odoo.com/odoo

Goal

A procurement clerk needs to record a new incoming order — PO-2026-3123, 700 units of RM-3002 (Stainless 316L bar) from NorthSteel Foundry, due 2026-05-22, status Draft — in the company's Odoo Spreadsheet ERP workbook.

Steps

  1. Log in to Odoo and open the ERP workbook in the Documents app.
  2. Navigate to the Purchase Orders register.
  3. Open a new row above the TOTAL line for the incoming order.
  4. Fill in PO #, order date, supplier, SKU, description, qty, unit cost, net total, expected receipt and status.
  5. Save and verify the order was recorded.

Problem

The spreadsheet renders to a single <canvas>, and the chrome around it (Name Box, menus, formula bar) uses class names Odoo wraps differently from the o-spreadsheet library docs. Playwright never reaches the canvas — it can't get past the chrome.

Where Playwright bounces off

  1. ✓ Reaches Outer Odoo DOM — login form, Documents app, file card

    Standard locators work here: input[name="login"], .o_kanban_record, the Documents nav link. This is what the recording shows succeeding for the first ~10 seconds.

  2. ✗ Stops here o-spreadsheet chrome — Name Box, top-bar menus, sheet tabs, formula bar

    The library docs claim .o-name-box, .o-topbar-menu, .o-sheet, .o-formula-bar. Odoo's wrapped build exposes none of them — every documented selector resolves to zero elements. The test stops here, timing out on the first Name-Box click.

  3. · No DOM <canvas> grid — every cell, gridline, total, conditional fill

    Painted into a single canvas element with no DOM children. Even if a future test author reverse-engineers the chrome selectors, no per-cell locator exists; verification has no DOM surface to assert against.

AIVA reads all three layers as rendered pixels — the layer separation doesn't apply. Login form, sheet tabs, and grid cells are visually identical inputs to the same vision model.

✗ Fails

Playwright recording (23 s)

✓ Passes

AIVA recording (same task — 2× speed)

AIVA driving the same Odoo Spreadsheet task end-to-end — selecting the TOTAL row, inserting the row above, typing the ten Purchase Order cells across, and verifying the result — by looking at the rendered pixels the same way a human operator does.

Show AIVA's step-by-step run log
AIVA test-runs viewer screenshot showing each step the agent executed against Odoo Spreadsheet (login credentials redacted).

The AIVA test-run log records every step it took on the canvas — clicking "TOTAL", opening Insert → Row above, typing each value, pressing Tab. The screenshot is exported from AIVA's test-runs viewer. Login credentials have been redacted for publication.

Show the Playwright test expand 
import { test, expect, type Page } from '@playwright/test';

const START_URL = 'https://testforme.odoo.com/odoo';
const EMAIL = process.env.ODOO_EMAIL!;
const PASSWORD = process.env.ODOO_PASSWORD!;
const FILE_NAME = 'odoo-erp-mock';
const TOTAL_ROW_REF = 'A15';
const NEW_ROW_VALUES = [
  'PO-2026-3123', '2026-05-02', 'NorthSteel Foundry', 'RM-3002',
  'Stainless 316L bar — 60mm', '700', '14.20', '9940.00',
  '2026-05-22', 'Draft',
] as const;

test('insert a row in Purchase Orders, fill it, verify', async ({ page }) => {
  test.setTimeout(180_000);

  // 1. Log in
  await page.goto(START_URL);
  if (page.url().includes('/web/login')) {
    await page.locator('input[name="login"]').fill(EMAIL);
    await page.locator('input[name="password"]').fill(PASSWORD);
    await page.getByRole('button', { name: /log in/i }).click();
    await page.waitForURL((u) => !u.pathname.startsWith('/web/login'));
  }

  // 2. Open Documents → odoo-erp-mock.xlsx
  await page.getByRole('link', { name: /^Documents$/ }).click();
  await page
    .locator('.o_kanban_record, .o_data_row')
    .filter({ hasText: FILE_NAME })
    .first()
    .dblclick();
  await page.locator('.o-grid canvas').first().waitFor();

  // 3. Switch to Purchase Orders sheet (sheet tabs ARE real DOM)
  await page.locator('.o-sheet').filter({ hasText: /^Purchase Orders$/ }).first().click();

  // 4. Select A15 (TOTAL) and Insert → Row → Above
  await selectCellByReference(page, TOTAL_ROW_REF);
  await page.locator('.o-topbar-menu').filter({ hasText: /^Insert$/ }).first().click();
  await page.locator('.o-menu-item').filter({ hasText: /^Row(s)?$/i }).first().click();
  await page.locator('.o-menu-item').filter({ hasText: /Row\s+above/i }).first().click();

  // 5. Re-select A15 (now empty) and type ten cells across
  await selectCellByReference(page, TOTAL_ROW_REF);
  for (let i = 0; i < NEW_ROW_VALUES.length; i++) {
    await page.keyboard.type(NEW_ROW_VALUES[i], { delay: 20 });
    await page.keyboard.press(i < NEW_ROW_VALUES.length - 1 ? 'Tab' : 'Enter');
  }

  // 6. Verify: re-select A15 and read the formula bar
  await selectCellByReference(page, TOTAL_ROW_REF);
  const composer = page
    .locator('.o-spreadsheet-topbar .o-composer, .o-formula-bar')
    .first();
  const value = (await composer.textContent())?.trim() ?? '';
  expect(value).toBe(NEW_ROW_VALUES[0]); // ← fails: returns "" under headless timing
});

// Name Box: real-DOM <input> at top-left of the grid that selects a cell
// by A1-reference. The only navigation path that works without per-cell DOM.
async function selectCellByReference(page: Page, ref: string) {
  const nameBox = page.locator('.o-name-box input, [class*="name-box"] input').first();
  await nameBox.click();
  await page.keyboard.press('ControlOrMeta+A');
  await page.keyboard.type(ref);
  await page.keyboard.press('Enter');
  await page.waitForTimeout(80);
}

TimeoutError: locator.click: Timeout 15000ms exceeded. waiting for locator('.o-name-box input, [class*="name-box"] input, …').first()