MCPcopy Create free account

hub / github.com/ScrapeGraphAI/Scrapegraph-ai / functions

Functions904 in github.com/ScrapeGraphAI/Scrapegraph-ai

Functioncapture_function_usage
(call_fn: Callable)
scrapegraphai/telemetry/telemetry.py:195
Methodcapture_screenshot
(scroll_position, counter)
scrapegraphai/nodes/fetch_screen_node.py:44
Functioncompare_results
Compare two scraping results, optionally ignoring certain keys. Args: result1: First result result2: Second result ignore
tests/fixtures/helpers.py:292
Methodcontent
(self, file_id)
tests/test_batch_api.py:210
Methodconvert_json_to_graphviz
Converts a JSON graph configuration to a Graphviz object for visualization. Args: json_data (dict): A JSON representatio
scrapegraphai/builders/graph_builder.py:137
Functioncount_tokens
(text)
scrapegraphai/utils/split_text_into_chunks.py:25
Functioncreate_mock_graph_result
Create a mock graph execution result. Args: answer: The answer/result exec_info: Execution info dictionary error: Optiona
tests/fixtures/helpers.py:102
Functioncreate_mock_llm_response
Create a mock LLM response. Args: content: Response content **kwargs: Additional response attributes Returns: Mock r
tests/fixtures/helpers.py:85
Functiondecorator
(func)
scrapegraphai/utils/research_web.py:106
Functiondelayed_scraper
(url)
tests/test_chromium.py:951
Functiondisable_telemetry
()
scrapegraphai/telemetry/telemetry.py:65
Methoddo_GET
Handle GET requests.
tests/fixtures/mock_server/server.py:30
Functiondraw_selection_rectanlge
(event, x, y, flags, param)
scrapegraphai/utils/screenshot_scraping/screenshot_preparation.py:67
Functiondummy
(url, browser_name="chromium")
tests/test_chromium.py:476
Functiondummy_apply_stealth
(context)
tests/test_chromium.py:1409
Functiondummy_ascrape
(url, browser_name="chromium")
tests/test_chromium.py:2013
Functiondummy_delay
(url)
tests/test_chromium.py:1888
Functiondummy_failure
(url)
tests/test_chromium.py:322
Functiondummy_js
(url)
tests/test_chromium.py:457
Functiondummy_node
Fixture for a GenerateAnswerNode instance using DummyLLM. Uses a valid input keys string ("dummy_input & doc") to avoid parsing errors.
tests/test_generate_answer_node.py:33
Functiondummy_node_for_process
Fixture for creating a GenerateAnswerNode instance for testing the process() method success case.
tests/test_generate_answer_node.py:161
Functiondummy_node_with_pipe
Fixture for a GenerateAnswerNode instance using DummyLLMWithPipe. Uses a valid input keys string ("dummy_input & doc") to avoid parsing error
tests/test_generate_answer_node.py:72
Functiondummy_non_async_scraper
A dummy scraper function that is not asynchronous.
tests/test_chromium.py:1584
Functiondummy_non_string
(url)
tests/test_chromium.py:1468
Functiondummy_none
(url)
tests/test_chromium.py:1110
Functiondummy_playwright
(url, browser_name="chromium")
tests/test_chromium.py:1393
Functiondummy_scraper
A dummy scraping function that returns dummy HTML content for the URL.
tests/test_chromium.py:56
Functiondummy_scraper_with_delay
(url, browser_name="chromium")
tests/test_chromium.py:2152
Functiondummy_selenium
(url)
tests/test_chromium.py:105
Methoderror
(self, msg)
tests/test_batch_api.py:334
Methodevaluate_expression
Evaluate an expression with parentheses.
scrapegraphai/nodes/base_node.py:206
Methodevaluate_simple_expression
Evaluate an expression without parentheses.
scrapegraphai/nodes/base_node.py:195
Methodexecute
Execute the markdownify graph. Args: initial_state: A dictionary containing either: - "url": The URL to
scrapegraphai/graphs/markdownify_graph.py:67
Methodexecute
Generate text from an image using an image-to-text model. The method retrieves the image from the list of URLs provided in the state
scrapegraphai/nodes/image_to_text_node.py:43
Methodexecute
Executes the node's logic to concatenate the answers from multiple graph instances into a single answer. Args: s
scrapegraphai/nodes/concat_answers_node.py:41
Methodexecute
Generates an answer by constructing a prompt from the user's input and the scraped content, querying the language model, and parsing
scrapegraphai/nodes/search_node_with_context.py:50
Methodexecute
Generates an answer by constructing a prompt from the user's input and the scraped content, querying the language model, and parsing
scrapegraphai/nodes/search_internet_node.py:60
Methodexecute
(self, state: dict)
scrapegraphai/nodes/generate_answer_node_k_level.py:73
Methodexecute
Generates Python code for a function that extracts data from HTML based on a output schema. Args: state (dict): The curr
scrapegraphai/nodes/generate_code_node.py:93
Methodexecute
Generate a refined prompt using the user's prompt, the schema, and additional context. Args: state (dict): The current s
scrapegraphai/nodes/prompt_refiner_node.py:62
Methodexecute
Checks if a website is scrapeable based on the robots.txt file and updates the state with the scrapeability status. The method constr
scrapegraphai/nodes/robots_node.py:57
Methodexecute
Executes the node's logic to parse the HTML documents content. Args: state (dict): The current state of the graph. The i
scrapegraphai/nodes/parse_node_depth_k_node.py:42
Methodexecute
Executes the node's logic to fetch the HTML content of a specified URL and its sub-links recursively, then updates the graph's state
scrapegraphai/nodes/fetch_node_level_k.py:72
Methodexecute
Checks if the specified key is present in the state and decides the next node accordingly. Args: state (dict): The curre
scrapegraphai/nodes/conditional_node.py:61
Methodexecute
Executes the node's logic to merge the answers from multiple graph instances into a single answer. Args: state (
scrapegraphai/nodes/merge_answers_node.py:57
Methodexecute
Generate a refined prompt for the reasoning task based on the user's input and the JSON schema. Args: state (dic
scrapegraphai/nodes/reasoning_node.py:56
Methodexecute
Executes the node's logic to instantiate and run multiple graph instances in parallel. Args: state (dict): The current s
scrapegraphai/nodes/graph_iterator_node.py:46
Methodexecute
Filter out relevant links from the webpage that are relavant to prompt. Out of the filtered links, also ensure that all links are nav
scrapegraphai/nodes/search_link_node.py:93
Methodexecute
Executes the node's logic to parse the HTML document content and split it into chunks. Args: state (dict): The current s
scrapegraphai/nodes/parse_node.py:62
Methodexecute
Generates a list of probable HTML tags based on the user's input and updates the state with this list. The method constructs a prompt
scrapegraphai/nodes/get_probable_tags_node.py:45
Methodexecute
Generates an answer by constructing a prompt from the user's input and the scraped content, querying the language model, and parsing
scrapegraphai/nodes/generate_answer_csv_node.py:70
Methodexecute
(self, state: dict)
scrapegraphai/nodes/description_node.py:47
Methodexecute
Generates an answer by constructing a prompt from the user's input and the scraped content, querying the language model, and parsing
scrapegraphai/nodes/generate_answer_omni_node.py:64
Methodexecute
Execute the node's logic based on the current state and update it accordingly. Args: state (dict): The current state of
scrapegraphai/nodes/base_node.py:71
Methodexecute
Executes the node's logic to merge the answers from multiple graph instances into a single answer. Args: state (d
scrapegraphai/nodes/merge_generated_scripts_node.py:41
Methodexecute
Converts text to speech using the specified text-to-speech model. Args: state (dict): The current state of the graph. Th
scrapegraphai/nodes/text_to_speech_node.py:39
Methodexecute
Generates an analysis of the provided HTML code based on the wanted infromations to be extracted. Args: state (dict): Th
scrapegraphai/nodes/html_analyzer_node.py:58
Methodexecute
Generates a python script for scraping a website using the specified library. Args: state (dict): The current state of t
scrapegraphai/nodes/generate_scraper_node.py:55
Methodexecute
Captures screenshots from the input URL and stores them in the state dictionary as bytes.
scrapegraphai/nodes/fetch_screen_node.py:27
Methodexecute
Wrapper to run the asynchronous execute_async function in a synchronous context.
scrapegraphai/nodes/generate_answer_from_image_node.py:112
Methodexecute
Executes the node's logic to convert HTML content to Markdown. Args: state (dict): The current state of the graph. The i
scrapegraphai/nodes/markdownify_node.py:41
Methodexecute
Executes the node's logic to index the content present in the state. Args: state (dict): The current state of the graph.
scrapegraphai/integrations/indexify_node.py:37
Methodexecute
(self, inputs)
tests/test_csv_scraper_multi_graph.py:18
Methodexecute
(self, inputs)
tests/test_csv_scraper_multi_graph.py:26
Methodexecute
(self, inputs)
tests/test_csv_scraper_multi_graph.py:127
Methodexecute
(self, inputs)
tests/test_csv_scraper_multi_graph.py:150
Methodexecute
(self, inputs)
tests/test_script_creator_multi_graph.py:22
Methodexecute
(self, inputs)
tests/test_script_creator_multi_graph.py:174
Functionexport_to_xml
Export data to an XML file. :param data: List of dictionaries containing the data to export :param filename: Name of the file to save th
scrapegraphai/utils/data_export.py:47
Functionextract
Call the scrapegraph-py extract endpoint across SDK versions.
scrapegraphai/integrations/scrapegraph_py_compat.py:55
Functionfail_apply_stealth
(context)
tests/test_chromium.py:1647
Functionfailing_scraper
(url)
tests/test_chromium.py:1250
Functionfake_dynamic_import
(backend, message)
tests/test_chromium.py:565
Functionfake_invoke_timeout
(chain, inputs, timeout)
tests/test_generate_answer_node.py:196
Functionfake_invoke_with_timeout
(chain, inputs, timeout)
tests/test_generate_answer_node.py:94
Functionfake_parse_or_search_proxy
(proxy)
tests/test_chromium.py:1925
Functionfuzzy_match_strings
Check if two strings are similar enough. Args: str1: First string str2: Second string threshold: Similarity threshold (0-
tests/fixtures/helpers.py:312
Functiongemini_config
Google Gemini configuration for testing.
tests/conftest.py:139
Methodgenerate_report
Generate a human-readable performance report. Returns: Formatted report string
tests/fixtures/benchmarking.py:126
Functiongenerate_test_csv
Generate test CSV data. Args: num_rows: Number of data rows to generate Returns: CSV string
tests/fixtures/helpers.py:187
Functiongenerate_test_html
Generate test HTML with customizable content. Args: title: Page title num_items: Number of list items to generate item_te
tests/fixtures/helpers.py:133
Functiongenerate_test_json
Generate test JSON data. Args: num_records: Number of records to generate Returns: Dictionary with test data
tests/fixtures/helpers.py:164
Methodget_source
(self)
scrapegraphai/integrations/burr_bridge.py:72
Functionget_verbosity
Get the current verbosity level of the root logger for the library. Returns: int: The current verbosity level of the root logger for
scrapegraphai/utils/logging.py:79
Functiongraph_config
Configuration of the graph
tests/graphs/smart_scraper_fireworks_test.py:16
Functiongraph_config
Configuration of the graph
tests/graphs/script_generator_test.py:11
Functiongraph_config
Configuration for the CodeGeneratorGraph
tests/graphs/code_generator_graph_openai_test.py:31
Functiongraph_config
Configuration for the SearchGraph
tests/graphs/search_graph_openai_test.py:20
Functiongraph_config
Configuration of the graph
tests/graphs/smart_scraper_ollama_test.py:11
Functiongraph_config
Configuration of the graph
tests/graphs/smart_scraper_multi_lite_graph_openai_test.py:16
Functiongraph_config
Configuration of the graph
tests/graphs/smart_scraper_openai_test.py:17
Functiongraph_config
Configuration for the XMLScraperGraph
tests/graphs/xml_scraper_openai_test.py:21
Functiongraph_config
Configuration for the DepthSearchGraph
tests/graphs/depth_search_graph_openai_test.py:16
Functiongraph_config
Configuration of the graph
tests/graphs/scrape_xml_ollama_test.py:28
Functiongraph_config
Creation of the graph
tests/graphs/screenshot_scraper_test.py:15
Functiongraph_config
Configuration of the graph
tests/graphs/smart_scraper_ernie_test.py:11
Functiongraph_config
Configuration of the graph
tests/graphs/smart_scraper_clod_test.py:16
Functiongraph_config
Configuration of the graph fixture.
tests/graphs/scrape_plain_text_mistral_test.py:28
Functiongroq_config
Groq configuration for testing.
tests/conftest.py:108
Methodhandle_directory
Handles the directory by compressing the source document and updating the state. Parameters: state (dict): The current state
scrapegraphai/nodes/fetch_node.py:128
← previousnext →401–500 of 904, ranked by callers