MCPcopy Create free account

hub / github.com/IPADS-SAI/MobiAgent / functions

Functions923 in github.com/IPADS-SAI/MobiAgent

⨍ Functions 923◇ Types & classes 124↳ Endpoints 16
↓ 1 callersMethodget_logger
获取或创建日志器
MobiFlow/avdag/logger.py:306
↓ 1 callersMethodget_num_tasks
(self)
agent_rr/action_cache/tree.py:372
↓ 1 callersMethodget_ocr_config
获取OCR配置
utils/config.py:193
↓ 1 callersFunctionget_optimal_label_pos
check overlap of text and background detection box, and get_optimal_label_pos, pos: str, position of the text, must be one of 'top left', 't
utils/box_annotator.py:189
↓ 1 callersFunctionget_optimal_label_pos
check overlap of text and background detection box, and get_optimal_label_pos, pos: str, position of the text, must be one of 'top left', 't
agent_rr/omniparser/box_annotator.py:189
↓ 1 callersMethodget_package_name
根据应用名称获取包名，如果失败则使用AI分析任务描述并改写任务 Args: app_name: 应用名称 task_description: 任务描述（可选，用于AI分析） R
runner/UI-TARS-agent/batch_task_executor.py:247
↓ 1 callersFunctionget_parsed_content_icon
(filtered_boxes, starting_idx, image_source, caption_model_processor, prompt=None, batch_size=128)
utils/omni_utils.py:63
↓ 1 callersFunctionget_parsed_content_icon
(filtered_boxes, starting_idx, image_source, caption_model_processor, prompt=None, batch_size=128)
agent_rr/omniparser/utils.py:64
↓ 1 callersFunctionget_parsed_content_icon_phi3v
(filtered_boxes, ocr_bbox, image_source, caption_model_processor)
utils/omni_utils.py:110
↓ 1 callersFunctionget_parsed_content_icon_phi3v
(filtered_boxes, ocr_bbox, image_source, caption_model_processor)
agent_rr/omniparser/utils.py:111
↓ 1 callersFunctionget_preference_aspects
获取指定任务类型的偏好提取方面 Args: task_type: 任务类型 Returns: 偏好提取方面列表
runner/mobiagent/preference_templates.py:78
↓ 1 callersFunctionget_screenshot
获取设备截图并编码为base64 Args: device: 设备对象 device_type: 设备类型，"Android" 或 "Harmony" Returns: Base64编码的截图字符串
runner/mobiagent/mobiagent.py:368
↓ 1 callersFunctionget_screenshot
获取设备截图并转换为base64
runner/mobiagent/multi_task/executor.py:70
↓ 1 callersMethodget_screenshot
(self)
agent_rr/agent/env.py:59
↓ 1 callersFunctionget_som_labeled_img
Process either an image path or Image object Args: image_source: Either a file path (str) or PIL Image object ...
utils/omni_utils.py:391
↓ 1 callersFunctionget_som_labeled_img
Process either an image path or Image object Args: image_source: Either a file path (str) or PIL Image object ...
agent_rr/omniparser/utils.py:392
↓ 1 callersMethodget_task_directory
获取任务目录路径
runner/UI-TARS-agent/ui_tars_automation/data_manager.py:177
↓ 1 callersFunctionget_trajectory
(trajectory_template, fmt)
agent_rr/train/task_template.py:42
↓ 1 callersMethodget_weights_dir
获取权重文件目录
utils/config.py:201
↓ 1 callersFunctionget_xyxy
(input)
utils/omni_utils.py:478
↓ 1 callersFunctionget_xyxy
(input)
agent_rr/omniparser/utils.py:457
↓ 1 callersFunctionget_yolo_model
(model_path)
utils/omni_utils.py:55
↓ 1 callersFunctionget_yolo_model
(model_path)
agent_rr/omniparser/utils.py:56
↓ 1 callersMethodgroup_by_app
按应用名称分组任务描述 Args: task_descriptions: 任务描述列表 Returns: 按app_name分组的字典
MobiFlow/auto_rules/task_extractor.py:85
↓ 1 callersFunctionhandle_click
处理点击操作
collect/auto/server.py:64
↓ 1 callersFunctionhandle_input
(text)
collect/auto/server.py:75
↓ 1 callersFunctionhandle_swipe
(direction)
collect/auto/server.py:94
↓ 1 callersMethodhasStructuredData
()
app/app/src/main/java/com/mobi/agent/Message.java:56
↓ 1 callersFunctionhideAppInfoModal
()
collect/manual/static/js/script.js:1695
↓ 1 callersFunctionhideTaskDescriptionModal
()
collect/manual/static/js/script.js:1026
↓ 1 callersFunctioninit
(service_ip, decider_port, grounder_port, planner_port, enable_user_profile=False, use_graphrag=False)
runner/mobiagent/mobiagent.py:269
↓ 1 callersFunctioninit
初始化客户端连接
runner/mobiagent/multi_task/mobiagent_refactored.py:166
↓ 1 callersMethodinitializeMediaProjection
(int resultCode, Intent resultData)
app/app/src/main/java/com/mobi/agent/ScreenCaptureService.java:142
↓ 1 callersMethodinitializeScreenshotManagerAndRequestPermission
初始化截图管理器并在首次启动时主动获取权限 使用新的MediaProjection前台服务架构
app/app/src/main/java/com/mobi/agent/MainActivity.java:1816
↓ 1 callersMethodinput
(self, text)
runner/mobiagent/mobiagent.py:64
↓ 1 callersMethodinput
(self, text)
runner/mobiagent/multi_task/mobiagent_refactored.py:64
↓ 1 callersMethodinputChineseViaClipboard
(String text)
app/app/src/main/java/com/mobi/agent/MyAccessibilityService.java:1312
↓ 1 callersMethodinputTextByClipboard
使用剪贴板粘贴的方式输入文本 @param text 要输入的文本 @return 是否成功
app/app/src/main/java/com/mobi/agent/MyAccessibilityService.java:651
↓ 1 callersFunctionintersection_area
(box1, box2)
utils/box_annotator.py:168
↓ 1 callersFunctionintersection_area
(box1, box2)
agent_rr/omniparser/box_annotator.py:168
↓ 1 callersMethodisAppRunning
检查指定应用是否正在运行 @param context 上下文对象 @param packageName 应用包名 @return 是否正在运行
app/app/src/main/java/com/mobi/agent/monitoru2.java:625
↓ 1 callersMethodisImportantEvent
判断是否为需要记录的重要事件
app/app/src/main/java/com/mobi/agent/MyAccessibilityService.java:58
↓ 1 callersMethodisLikelyInputElement
判断节点是否可能是输入元素
app/app/src/main/java/com/mobi/agent/MyAccessibilityService.java:1193
↓ 1 callersMethodisPermissionError
判断是否为权限相关错误
app/app/src/main/java/com/mobi/agent/ScreenshotManager.java:204
↓ 1 callersFunctionis_point_in_bounds
检查点(x,y)是否在bounds范围内
utils/parse_xml.py:16
↓ 1 callersMethodkeyevent
(self, key)
runner/mobiagent/mobiagent.py:72
↓ 1 callersMethodkeyevent
(self, key)
runner/mobiagent/multi_task/mobiagent_refactored.py:72
↓ 1 callersMethodkeyevent
Send a key event
collect/manual/device.py:50
↓ 1 callersMethodlist_available_icons
列出可用的图标 Args: app_id: 应用ID，用于筛选特定应用的图标 Returns: 可用图标名称列表
utils/icon_detection.py:323
↓ 1 callersMethodlist_available_models
列出所有可用的模型 Returns: 模型名称到路径的映射
utils/weights_manager.py:95
↓ 1 callersFunctionload_frames_from_dir
将包含 images/xml/actions/react 的目录转换为帧序列。 规则： - 帧索引从 1 开始（按文件名 N.xml/N.jpg 推断），依次排序。 - 每帧字段： - image: `<folder>/<i>.jpg`（存在时） -
MobiFlow/avdag/trace_loader.py:18
↓ 1 callersMethodload_icon_template
加载图标模板 Args: icon_path: 图标文件路径 Returns: 图标模板的numpy数组，加载失败返回None
utils/icon_detection.py:42
↓ 1 callersFunctionload_preference_templates
从JSON文件加载偏好模板配置 Returns: 偏好模板配置字典
runner/mobiagent/preference_templates.py:14
↓ 1 callersMethodload_tasks
加载任务文件 Args: task_file: 任务文件路径 Returns: 任务列表
runner/UI-TARS-agent/batch_task_executor.py:229
↓ 1 callersFunctionload_template_yaml
加载模版 YAML 文件内容
MobiFlow/auto_rules/main.py:99
↓ 1 callersMethodlog_possible_paths
输出配置中存在的可能路径到日志
MobiFlow/avdag/dag.py:152
↓ 1 callersFunctionmain
主函数
runner/UI-TARS-agent/batch_task_executor.py:731
↓ 1 callersFunctionmain
主函数
collect/manual/start.py:31
↓ 1 callersFunctionmain
(args)
agent_rr/run_experiment.py:127
↓ 1 callersFunctionmain
主函数
MobiFlow/universal_test_runner.py:597
↓ 1 callersFunctionmain
主函数
MobiFlow/test_model_connectivity.py:300
↓ 1 callersFunctionmain
主程序入口
MobiFlow/auto_rules/main.py:122
↓ 1 callersFunctionmake_llm_options
构造带 LLM 回调的 VerifierOptions，使用 LangChain OpenAI 兼容接口。 注意：不在库内硬编码 key；由调用方传入。 Args: api_key: API密钥 base_url: API基础URL
MobiFlow/avdag/verifier.py:503
↓ 1 callersMethodmatch_keyword_in_frame
在帧中搜索关键词 Args: frame: 帧数据 keyword: 要搜索的关键词 enable_ocr: 是否启用OCR识别 Re
utils/advanced_ocr.py:389
↓ 1 callersMethodmatch_template_multiscale
多尺度模板匹配 Args: image: 目标图像（灰度图） template: 模板图像（灰度图） threshold: 相似度阈值
utils/icon_detection.py:75
↓ 1 callersMethodmiuiNotificationScreenshot
()
app/app/src/main/java/com/mobi/agent/MyAccessibilityService.java:3111
↓ 1 callersMethodnodeContainsText
(AccessibilityNodeInfo node, String text)
app/app/src/main/java/com/mobi/agent/MyAccessibilityService.java:2924
↓ 1 callersMethodnon_maximum_suppression
非极大值抑制，去除重叠的检测框 Args: matches: 匹配结果列表 overlap_threshold: 重叠阈值 Returns:
utils/icon_detection.py:127
↓ 1 callersMethodonSuccess
(String filePath)
app/app/src/main/java/com/mobi/agent/ScreenCaptureService.java:56
↓ 1 callersMethodonSuccess
(String filePath)
app/app/src/main/java/com/mobi/agent/ScreenshotManager.java:35
↓ 1 callersMethodparse
(self, image)
agent_rr/omniparser/omniparser.py:14
↓ 1 callersFunctionparse_bounds
解析bounds字符串，返回(left, top, right, bottom)
utils/parse_xml.py:4
↓ 1 callersFunctionparse_json_response
解析 JSON 响应，支持 guided decoding 和普通模式 Args: response_str: 模型返回的响应字符串 is_guided_decoding: 是否启用了 guided decoding（默认 True）
runner/mobiagent/mobiagent.py:299
↓ 1 callersFunctionparse_json_response
解析JSON响应
runner/mobiagent/multi_task/executor.py:46
↓ 1 callersFunctionparse_planner_response
(response_str: str)
runner/mobiagent/mobiagent.py:869
↓ 1 callersFunctionparse_planner_response
解析planner的响应JSON
collect/auto/server.py:120
↓ 1 callersMethodparse_planner_response
解析planner的响应JSON（参考mobiagent） Args: response_str: 模型返回的响应字符串 Returns: 解析后的JSON对象
runner/UI-TARS-agent/batch_task_executor.py:133
↓ 1 callersMethodperformThreeFingerSwipeScreenshot
()
app/app/src/main/java/com/mobi/agent/MyAccessibilityService.java:2738
↓ 1 callersFunctionposition_num_repeat
(index, total_length)
collect/construct_sft.py:79
↓ 1 callersMethodpowerVolumeScreenshot
()
app/app/src/main/java/com/mobi/agent/MyAccessibilityService.java:3407
↓ 1 callersFunctionpredict_yolo
Use huggingface model to replace the original model
utils/omni_utils.py:362
↓ 1 callersFunctionpredict_yolo
Use huggingface model to replace the original model
agent_rr/omniparser/utils.py:363
↓ 1 callersMethodprint_cnt
(self)
agent_rr/run_experiment.py:62
↓ 1 callersMethodprint_cnt
(self)
agent_rr/run_experiment.py:92
↓ 1 callersMethodprint_final_summary
打印最终汇总
MobiFlow/universal_test_runner.py:546
↓ 1 callersMethodprint_summary
打印测试结果摘要
MobiFlow/test_model_connectivity.py:276
↓ 1 callersMethodprocessImage
(Image image)
app/app/src/main/java/com/mobi/agent/ScreenCaptureService.java:279
↓ 1 callersFunctionprocess_folder
处理单个文件夹
collect/auto/draw_bounds.py:105
↓ 1 callersMethodprocess_frame_text
处理帧中的所有文本信息 Args: frame: 包含图像、XML等信息的帧数据 Returns: ProcessedText: 处理后的综合文本
utils/advanced_ocr.py:352
↓ 1 callersMethodprocess_inputs
(self, input_texts)
agent_rr/action_cache/reranker.py:30
↓ 1 callersFunctionquick_start
快速启动示例 - 淘宝购物任务
runner/UI-TARS-agent/quick_start.py:9
↓ 1 callersFunctionrefine_next_subtask_description
根据已有artifacts重写下一个子任务描述 Args: next_subtask_description: 下一个子任务描述 artifacts: 已有的artifacts字典 planner_client: Plann
runner/mobiagent/multi_task/planner.py:475
↓ 1 callersFunctionrelative_point
(point, width, height)
collect/construct_sft.py:185
↓ 1 callersFunctionremove_overlap_new
ocr_bbox format: [{'type': 'text', 'bbox':[x,y], 'interactivity':False, 'content':str }, ...] boxes format: [{'type': 'icon', 'bbox':[x,y], '
utils/omni_utils.py:215
↓ 1 callersFunctionremove_overlap_new
ocr_bbox format: [{'type': 'text', 'bbox':[x,y], 'interactivity':False, 'content':str }, ...] boxes format: [{'type': 'icon', 'bbox':[x,y], '
agent_rr/omniparser/utils.py:216
↓ 1 callersMethodremove_task
(self, task_idx)
agent_rr/action_cache/tree.py:48
↓ 1 callersMethodremove_task_trace
(self, task)
agent_rr/action_cache/tree.py:174
↓ 1 callersMethodrequestInitialScreenshotPermission
主动申请截图权限（首次启动时调用）
app/app/src/main/java/com/mobi/agent/MainActivity.java:1850
↓ 1 callersFunctionrequest_screenshot
(url)
agent_rr/agent/env.py:23
↓ 1 callersMethodrerank
(self, query_tasks, document_task, step)
agent_rr/action_cache/reranker.py:24
← previousnext →401–500 of 923, ranked by callers