MCPcopy Create free account

hub / github.com/OthersideAI/self-operating-computer / functions

Functions55 in github.com/OthersideAI/self-operating-computer

↓ 8 callersFunctioncapture_screen_with_cursor
(file_path)
operate/utils/screenshot.py:11
↓ 8 callersFunctionclean_json
(content)
operate/models/apis.py:1008
↓ 7 callersFunctionget_user_first_message_prompt
()
operate/models/prompts.py:265
↓ 7 callersFunctionget_user_prompt
()
operate/models/prompts.py:260
↓ 7 callersFunctionsupports_ansi
Check if the terminal supports ANSI escape codes
operate/utils/style.py:19
↓ 5 callersFunctioncall_gpt_4o
(messages)
operate/models/apis.py:65
↓ 5 callersFunctionconfirm_system_prompt
On `Exception` we default to `call_gpt_4_vision_preview` so we have this function to reassign system prompt in case of a previous failure
operate/models/apis.py:984
↓ 4 callersFunctionget_system_prompt
Format the vision prompt more efficiently and print the name of the prompt used
operate/models/prompts.py:210
↓ 4 callersFunctionget_text_coordinates
Gets the coordinates of the text element at the specified index as a percentage of screen width and height. Args: result (list): The
operate/utils/ocr.py:66
↓ 4 callersFunctionget_text_element
Searches for a text element in the OCR results and returns its index. Also draws bounding boxes on the image. Args: result (list): Th
operate/utils/ocr.py:10
↓ 4 callersFunctiongpt_4_fallback
(messages, objective, model)
operate/models/apis.py:968
↓ 4 callersMethodinitialize_openai
(self)
operate/config.py:50
↓ 4 callersMethodrequire_api_key
(self, key_name, key_description, is_required)
operate/config.py:152
↓ 2 callersFunctionconvert_percent_to_decimal
(percent)
operate/utils/misc.py:5
↓ 2 callersMethodwrite
(self, content)
operate/utils/operating_system.py:10
↓ 1 callersFunctionadd_labels
(base64_data, yolo_model)
operate/utils/label.py:55
↓ 1 callersFunctioncall_claude_3_with_ocr
(messages, objective, model)
operate/models/apis.py:759
↓ 1 callersFunctioncall_gemini_pro_vision
Get the next action for Self-Operating Computer using Gemini Pro Vision
operate/models/apis.py:259
↓ 1 callersFunctioncall_gpt_4o_labeled
(messages, objective, model)
operate/models/apis.py:537
↓ 1 callersFunctioncall_gpt_4o_with_ocr
(messages, objective, model)
operate/models/apis.py:311
↓ 1 callersFunctioncall_o1_with_ocr
(messages, objective, model)
operate/models/apis.py:424
↓ 1 callersFunctioncall_ollama_llava
(messages)
operate/models/apis.py:681
↓ 1 callersFunctioncall_qwen_vl_with_ocr
(messages, objective, model)
operate/models/apis.py:142
↓ 1 callersMethodclick_at_percentage
( self, x_percentage, y_percentage, duration=0.2, circle_radius=50,
operate/utils/operating_system.py:39
↓ 1 callersFunctioncompress_screenshot
(raw_screenshot_filename, screenshot_filename)
operate/utils/screenshot.py:30
↓ 1 callersFunctionevaluate_final_screenshot
Load the final screenshot and return True or False if it meets the given guideline.
evaluate.py:88
↓ 1 callersFunctionformat_evaluation_prompt
(guideline)
evaluate.py:69
↓ 1 callersFunctionget_click_position_in_percent
Calculates the click position at the center of the bounding box and converts it to percentages. :param coordinates: A tuple of the bounding
operate/utils/label.py:141
↓ 1 callersFunctionget_label_coordinates
Retrieves the coordinates for a given label. :param label: The label to find coordinates for (e.g., "~1"). :param label_coordinates: Dic
operate/utils/label.py:29
↓ 1 callersFunctionget_next_action
(model, messages, objective, session_id)
operate/models/apis.py:34
↓ 1 callersFunctionget_test_model
()
evaluate.py:136
↓ 1 callersMethodinitialize_anthropic
(self)
operate/config.py:124
↓ 1 callersMethodinitialize_google
(self)
operate/config.py:95
↓ 1 callersMethodinitialize_ollama
(self)
operate/config.py:111
↓ 1 callersMethodinitialize_qwen
(self)
operate/config.py:72
↓ 1 callersFunctionis_overlapping
(box1, box2)
operate/utils/label.py:40
↓ 1 callersFunctionmain
()
evaluate.py:152
↓ 1 callersFunctionmain
Main function for the Self-Operating Computer. Parameters: - model: The model used for generating responses. - terminal_prompt: A st
operate/operate.py:33
↓ 1 callersFunctionmain_entry
()
operate/main.py:9
↓ 1 callersMethodmouse
(self, click_detail)
operate/utils/operating_system.py:28
↓ 1 callersFunctionoperate
(operations, model)
operate/operate.py:134
↓ 1 callersFunctionparse_eval_content
(content)
evaluate.py:74
↓ 1 callersMethodpress
(self, keys)
operate/utils/operating_system.py:18
↓ 1 callersMethodprompt_and_save_api_key
(self, key_name, key_description)
operate/config.py:162
↓ 1 callersFunctionrun_test_case
Returns True if the result of the test with the given prompt meets the given guideline for the given model.
evaluate.py:119
↓ 1 callersMethodsave_api_key_to_env
(key_name, key_value)
operate/config.py:184
↓ 1 callersFunctionsupports_ansi
Check if the terminal supports ANSI escape codes
evaluate.py:32
↓ 1 callersMethodvalidation
Validate the input parameters for the dialog operation.
operate/config.py:131
Method__init__
(self)
operate/config.py:31
Method__init__
(self, model, message="Model not recognized")
operate/exceptions.py:9
Method__new__
(cls)
operate/config.py:25
Method__str__
(self)
operate/exceptions.py:14
Functionget_last_assistant_message
Retrieve the last message from the assistant in the messages array. If the last assistant message is the first message in the array, return N
operate/models/apis.py:954
Functionparse_operations
(response)
operate/utils/misc.py:17
Functionvalidate_and_extract_image_data
(data)
operate/utils/label.py:10