-
Notifications
You must be signed in to change notification settings - Fork 224
keyboard:type action does not trigger Vue component's filter/search in filterable Select components #154
Description
Title: keyboard:type action does not trigger Vue component's filter/search in filterable Select components
Body:
Bug Description
When using agent.act() with keyboard:type to type into a filterable <select>-like component (e.g., a custom Vue 2 component with a searchable dropdown list), the list does not filter as expected. The text appears visually in the input field, but the underlying component's filter logic is never triggered.
Environment
magnitude-coreversion:0.3.1- Browser: Chromium (via Playwright/Patchright)
- Frontend framework of target app: Vue 2 (production build,
__vue__not exposed) - Component type: Custom filterable select component (
lls-select) — similar to Element UI's<el-select filterable>
Steps to Reproduce
- Start a browser agent targeting a page with a filterable select component
- Use
agent.act()to instruct the AI to type a keyword into the select's search input - The AI executes
keyboard:typeaction with the keyword - Observe: the keyword appears in the input visually, but the dropdown list remains unfiltered (shows all options)
Root Cause Analysis
The keyboard:type action in magnitude-core (via Playwright's page.keyboard.type()) simulates keydown/keyup events but does not dispatch the native input event or compositionend event that Vue/custom component frameworks rely on to trigger their internal filter logic.
We also tried:
page.keyboard.insertText(keyword)— same result, list not filteredelement.value = keyword+dispatchEvent(new Event('input'))viapage.evaluate— works in browser console manually, but when called afteragent.act()opens the dropdown, the dropdown state gets reset because the AI may re-click the trigger, causing a race condition- Direct Playwright
locator.fill()— the target input isreadonly, fill is rejected
The core issue is there is no way to hook into the agent.act() lifecycle between "click opens dropdown" and "type into search box" — by the time page.evaluate runs after act() completes, the dropdown may have already closed or reset.
Expected Behavior
Either:
keyboard:typeshould dispatch a nativeinputevent (and optionallycompositionstart/compositionendfor CJK input) so that framework components respond correctly- Or, provide a way to execute custom Playwright code between AI actions within a single
act()call (e.g., a hook or callback)
Workaround Attempted
None that fully works. The closest approach was using page.evaluate to directly set the value and dispatch input event, but it's unreliable due to the dropdown lifecycle race condition.
Additional Context
This is a very common scenario in enterprise web apps built with Vue/React that use virtual-scroll filterable select components (Element UI, Ant Design, custom components). The keyboard:type limitation makes it practically impossible to automate these components reliably.