Skip to content

bbox定位不准 #70

@betacatZ

Description

@betacatZ

task: "打开淘宝,将阿迪达斯运动鞋加入购物车"
model: IPADS-SAI/MobiMind-Reasoning-4B-1208
问题:定位不准导致循环卡在确认步骤

2026-02-27 10:22:08,600 - INFO - HTTP Request: POST http://localhost:8000/v1/chat/completions "HTTP/1.1 200 OK"
2026-02-27 10:22:08,602 - INFO - [evaluation] Decider time taken: 0.730078935623169 seconds
2026-02-27 10:22:08,603 - INFO - Decider response:
{"reasoning": "当前位于商品规格选择页面,已成功选择颜色为“黑色/碳黑色”,鞋码为“38 235mm”。为了将选中的商品加入购物车,我需要点击底部的“确认”按钮。我将点击“确认”按钮以完成选择并加入购物车。", "action": "click", "parameters": {"target_element": "确认按钮"}}
2026-02-27 10:22:08,625 - INFO - Dumping UI hierarchy...
2026-02-27 10:22:09,017 - INFO - HTTP Request: POST http://localhost:8000/v1/chat/completions "HTTP/1.1 200 OK"
2026-02-27 10:22:09,018 - INFO - [evaluation] Grounder time taken: 0.24073338508605957 seconds
2026-02-27 10:22:09,019 - INFO - Grounder response:[
    {"bbox_2d": [23, 905, 976, 948], "label": "确认按钮"}
]
[
    {"bbox_2d": [23, 905, 976, 948], "label": "确认按钮"}
]
Clicking on bbox: [33, 2896, 1405, 3033]
Image size: width=1440, height=3200
Adjusted bbox: [33, 2896, 1405, 3033]
2026-02-27 10:22:10,676 - INFO - HTTP Request: POST http://localhost:8000/v1/chat/completions "HTTP/1.1 200 OK"
2026-02-27 10:22:10,677 - INFO - [evaluation] Decider time taken: 0.7415294647216797 seconds
2026-02-27 10:22:10,677 - INFO - Decider response:
{"reasoning": "当前位于商品规格选择页面,已成功选择颜色为“黑色/碳黑色”,鞋码为“38 235mm”。为了将选中的商品加入购物车,我需要点击底部的“确认”按钮。我将点击“确认”按钮以完成选择并加入购物车。", "action": "click", "parameters": {"target_element": "确认按钮"}}
2026-02-27 10:22:10,699 - INFO - Dumping UI hierarchy...
2026-02-27 10:22:11,031 - INFO - HTTP Request: POST http://localhost:8000/v1/chat/completions "HTTP/1.1 200 OK"
2026-02-27 10:22:11,032 - INFO - [evaluation] Grounder time taken: 0.24379897117614746 seconds
2026-02-27 10:22:11,033 - INFO - Grounder response:[
    {"bbox_2d": [23, 905, 977, 948], "label": "确认按钮"}
]
[
    {"bbox_2d": [23, 905, 977, 948], "label": "确认按钮"}
]
Clicking on bbox: [33, 2896, 1406, 3033]
Image size: width=1440, height=3200
Adjusted bbox: [33, 2896, 1406, 3033]
2026-02-27 10:22:12,714 - INFO - HTTP Request: POST http://localhost:8000/v1/chat/completions "HTTP/1.1 200 OK"
2026-02-27 10:22:12,716 - INFO - [evaluation] Decider time taken: 0.743877649307251 seconds
2026-02-27 10:22:12,716 - INFO - Decider response:
{"reasoning": "当前位于商品规格选择页面,已成功选择颜色为“黑色/碳黑色”,鞋码为“38 235mm”。为了将选中的商品加入购物车,我需要点击底部的“确认”按钮。我将点击“确认”按钮以完成选择并加入购物车。", "action": "click", "parameters": {"target_element": "确认按钮"}}
2026-02-27 10:22:12,733 - INFO - Dumping UI hierarchy...
2026-02-27 10:22:13,058 - INFO - HTTP Request: POST http://localhost:8000/v1/chat/completions "HTTP/1.1 200 OK"
2026-02-27 10:22:13,060 - INFO - [evaluation] Grounder time taken: 0.24343657493591309 seconds
2026-02-27 10:22:13,061 - INFO - Grounder response:[
    {"bbox_2d": [23, 905, 977, 948], "label": "确认按钮"}
]
[
    {"bbox_2d": [23, 905, 977, 948], "label": "确认按钮"}
]
Clicking on bbox: [33, 2896, 1406, 3033]
Image size: width=1440, height=3200
Adjusted bbox: [33, 2896, 1406, 3033]

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions