Code
Hub
Workspaces
Following
Trending
Connect
MCP
copy
Create free account
hub
/
github.com/ScrapeGraphAI/Scrapegraph-ai
/ functions
Functions
904 in github.com/ScrapeGraphAI/Scrapegraph-ai
⨍
Functions
904
◇
Types & classes
195
↳
Endpoints
27
Function
capture_function_usage
(call_fn: Callable)
scrapegraphai/telemetry/telemetry.py:195
Method
capture_screenshot
(scroll_position, counter)
scrapegraphai/nodes/fetch_screen_node.py:44
Function
compare_results
Compare two scraping results, optionally ignoring certain keys. Args: result1: First result result2: Second result ignore
tests/fixtures/helpers.py:292
Method
content
(self, file_id)
tests/test_batch_api.py:210
Method
convert_json_to_graphviz
Converts a JSON graph configuration to a Graphviz object for visualization. Args: json_data (dict): A JSON representatio
scrapegraphai/builders/graph_builder.py:137
Function
count_tokens
(text)
scrapegraphai/utils/split_text_into_chunks.py:25
Function
create_mock_graph_result
Create a mock graph execution result. Args: answer: The answer/result exec_info: Execution info dictionary error: Optiona
tests/fixtures/helpers.py:102
Function
create_mock_llm_response
Create a mock LLM response. Args: content: Response content **kwargs: Additional response attributes Returns: Mock r
tests/fixtures/helpers.py:85
Function
decorator
(func)
scrapegraphai/utils/research_web.py:106
Function
delayed_scraper
(url)
tests/test_chromium.py:951
Function
disable_telemetry
()
scrapegraphai/telemetry/telemetry.py:65
Method
do_GET
Handle GET requests.
tests/fixtures/mock_server/server.py:30
Function
draw_selection_rectanlge
(event, x, y, flags, param)
scrapegraphai/utils/screenshot_scraping/screenshot_preparation.py:67
Function
dummy
(url, browser_name="chromium")
tests/test_chromium.py:476
Function
dummy_apply_stealth
(context)
tests/test_chromium.py:1409
Function
dummy_ascrape
(url, browser_name="chromium")
tests/test_chromium.py:2013
Function
dummy_delay
(url)
tests/test_chromium.py:1888
Function
dummy_failure
(url)
tests/test_chromium.py:322
Function
dummy_js
(url)
tests/test_chromium.py:457
Function
dummy_node
Fixture for a GenerateAnswerNode instance using DummyLLM. Uses a valid input keys string ("dummy_input & doc") to avoid parsing errors.
tests/test_generate_answer_node.py:33
Function
dummy_node_for_process
Fixture for creating a GenerateAnswerNode instance for testing the process() method success case.
tests/test_generate_answer_node.py:161
Function
dummy_node_with_pipe
Fixture for a GenerateAnswerNode instance using DummyLLMWithPipe. Uses a valid input keys string ("dummy_input & doc") to avoid parsing error
tests/test_generate_answer_node.py:72
Function
dummy_non_async_scraper
A dummy scraper function that is not asynchronous.
tests/test_chromium.py:1584
Function
dummy_non_string
(url)
tests/test_chromium.py:1468
Function
dummy_none
(url)
tests/test_chromium.py:1110
Function
dummy_playwright
(url, browser_name="chromium")
tests/test_chromium.py:1393
Function
dummy_scraper
A dummy scraping function that returns dummy HTML content for the URL.
tests/test_chromium.py:56
Function
dummy_scraper_with_delay
(url, browser_name="chromium")
tests/test_chromium.py:2152
Function
dummy_selenium
(url)
tests/test_chromium.py:105
Method
error
(self, msg)
tests/test_batch_api.py:334
Method
evaluate_expression
Evaluate an expression with parentheses.
scrapegraphai/nodes/base_node.py:206
Method
evaluate_simple_expression
Evaluate an expression without parentheses.
scrapegraphai/nodes/base_node.py:195
Method
execute
Execute the markdownify graph. Args: initial_state: A dictionary containing either: - "url": The URL to
scrapegraphai/graphs/markdownify_graph.py:67
Method
execute
Generate text from an image using an image-to-text model. The method retrieves the image from the list of URLs provided in the state
scrapegraphai/nodes/image_to_text_node.py:43
Method
execute
Executes the node's logic to concatenate the answers from multiple graph instances into a single answer. Args: s
scrapegraphai/nodes/concat_answers_node.py:41
Method
execute
Generates an answer by constructing a prompt from the user's input and the scraped content, querying the language model, and parsing
scrapegraphai/nodes/search_node_with_context.py:50
Method
execute
Generates an answer by constructing a prompt from the user's input and the scraped content, querying the language model, and parsing
scrapegraphai/nodes/search_internet_node.py:60
Method
execute
(self, state: dict)
scrapegraphai/nodes/generate_answer_node_k_level.py:73
Method
execute
Generates Python code for a function that extracts data from HTML based on a output schema. Args: state (dict): The curr
scrapegraphai/nodes/generate_code_node.py:93
Method
execute
Generate a refined prompt using the user's prompt, the schema, and additional context. Args: state (dict): The current s
scrapegraphai/nodes/prompt_refiner_node.py:62
Method
execute
Checks if a website is scrapeable based on the robots.txt file and updates the state with the scrapeability status. The method constr
scrapegraphai/nodes/robots_node.py:57
Method
execute
Executes the node's logic to parse the HTML documents content. Args: state (dict): The current state of the graph. The i
scrapegraphai/nodes/parse_node_depth_k_node.py:42
Method
execute
Executes the node's logic to fetch the HTML content of a specified URL and its sub-links recursively, then updates the graph's state
scrapegraphai/nodes/fetch_node_level_k.py:72
Method
execute
Checks if the specified key is present in the state and decides the next node accordingly. Args: state (dict): The curre
scrapegraphai/nodes/conditional_node.py:61
Method
execute
Executes the node's logic to merge the answers from multiple graph instances into a single answer. Args: state (
scrapegraphai/nodes/merge_answers_node.py:57
Method
execute
Generate a refined prompt for the reasoning task based on the user's input and the JSON schema. Args: state (dic
scrapegraphai/nodes/reasoning_node.py:56
Method
execute
Executes the node's logic to instantiate and run multiple graph instances in parallel. Args: state (dict): The current s
scrapegraphai/nodes/graph_iterator_node.py:46
Method
execute
Filter out relevant links from the webpage that are relavant to prompt. Out of the filtered links, also ensure that all links are nav
scrapegraphai/nodes/search_link_node.py:93
Method
execute
Executes the node's logic to parse the HTML document content and split it into chunks. Args: state (dict): The current s
scrapegraphai/nodes/parse_node.py:62
Method
execute
Generates a list of probable HTML tags based on the user's input and updates the state with this list. The method constructs a prompt
scrapegraphai/nodes/get_probable_tags_node.py:45
Method
execute
Generates an answer by constructing a prompt from the user's input and the scraped content, querying the language model, and parsing
scrapegraphai/nodes/generate_answer_csv_node.py:70
Method
execute
(self, state: dict)
scrapegraphai/nodes/description_node.py:47
Method
execute
Generates an answer by constructing a prompt from the user's input and the scraped content, querying the language model, and parsing
scrapegraphai/nodes/generate_answer_omni_node.py:64
Method
execute
Execute the node's logic based on the current state and update it accordingly. Args: state (dict): The current state of
scrapegraphai/nodes/base_node.py:71
Method
execute
Executes the node's logic to merge the answers from multiple graph instances into a single answer. Args: state (d
scrapegraphai/nodes/merge_generated_scripts_node.py:41
Method
execute
Converts text to speech using the specified text-to-speech model. Args: state (dict): The current state of the graph. Th
scrapegraphai/nodes/text_to_speech_node.py:39
Method
execute
Generates an analysis of the provided HTML code based on the wanted infromations to be extracted. Args: state (dict): Th
scrapegraphai/nodes/html_analyzer_node.py:58
Method
execute
Generates a python script for scraping a website using the specified library. Args: state (dict): The current state of t
scrapegraphai/nodes/generate_scraper_node.py:55
Method
execute
Captures screenshots from the input URL and stores them in the state dictionary as bytes.
scrapegraphai/nodes/fetch_screen_node.py:27
Method
execute
Wrapper to run the asynchronous execute_async function in a synchronous context.
scrapegraphai/nodes/generate_answer_from_image_node.py:112
Method
execute
Executes the node's logic to convert HTML content to Markdown. Args: state (dict): The current state of the graph. The i
scrapegraphai/nodes/markdownify_node.py:41
Method
execute
Executes the node's logic to index the content present in the state. Args: state (dict): The current state of the graph.
scrapegraphai/integrations/indexify_node.py:37
Method
execute
(self, inputs)
tests/test_csv_scraper_multi_graph.py:18
Method
execute
(self, inputs)
tests/test_csv_scraper_multi_graph.py:26
Method
execute
(self, inputs)
tests/test_csv_scraper_multi_graph.py:127
Method
execute
(self, inputs)
tests/test_csv_scraper_multi_graph.py:150
Method
execute
(self, inputs)
tests/test_script_creator_multi_graph.py:22
Method
execute
(self, inputs)
tests/test_script_creator_multi_graph.py:174
Function
export_to_xml
Export data to an XML file. :param data: List of dictionaries containing the data to export :param filename: Name of the file to save th
scrapegraphai/utils/data_export.py:47
Function
extract
Call the scrapegraph-py extract endpoint across SDK versions.
scrapegraphai/integrations/scrapegraph_py_compat.py:55
Function
fail_apply_stealth
(context)
tests/test_chromium.py:1647
Function
failing_scraper
(url)
tests/test_chromium.py:1250
Function
fake_dynamic_import
(backend, message)
tests/test_chromium.py:565
Function
fake_invoke_timeout
(chain, inputs, timeout)
tests/test_generate_answer_node.py:196
Function
fake_invoke_with_timeout
(chain, inputs, timeout)
tests/test_generate_answer_node.py:94
Function
fake_parse_or_search_proxy
(proxy)
tests/test_chromium.py:1925
Function
fuzzy_match_strings
Check if two strings are similar enough. Args: str1: First string str2: Second string threshold: Similarity threshold (0-
tests/fixtures/helpers.py:312
Function
gemini_config
Google Gemini configuration for testing.
tests/conftest.py:139
Method
generate_report
Generate a human-readable performance report. Returns: Formatted report string
tests/fixtures/benchmarking.py:126
Function
generate_test_csv
Generate test CSV data. Args: num_rows: Number of data rows to generate Returns: CSV string
tests/fixtures/helpers.py:187
Function
generate_test_html
Generate test HTML with customizable content. Args: title: Page title num_items: Number of list items to generate item_te
tests/fixtures/helpers.py:133
Function
generate_test_json
Generate test JSON data. Args: num_records: Number of records to generate Returns: Dictionary with test data
tests/fixtures/helpers.py:164
Method
get_source
(self)
scrapegraphai/integrations/burr_bridge.py:72
Function
get_verbosity
Get the current verbosity level of the root logger for the library. Returns: int: The current verbosity level of the root logger for
scrapegraphai/utils/logging.py:79
Function
graph_config
Configuration of the graph
tests/graphs/smart_scraper_fireworks_test.py:16
Function
graph_config
Configuration of the graph
tests/graphs/script_generator_test.py:11
Function
graph_config
Configuration for the CodeGeneratorGraph
tests/graphs/code_generator_graph_openai_test.py:31
Function
graph_config
Configuration for the SearchGraph
tests/graphs/search_graph_openai_test.py:20
Function
graph_config
Configuration of the graph
tests/graphs/smart_scraper_ollama_test.py:11
Function
graph_config
Configuration of the graph
tests/graphs/smart_scraper_multi_lite_graph_openai_test.py:16
Function
graph_config
Configuration of the graph
tests/graphs/smart_scraper_openai_test.py:17
Function
graph_config
Configuration for the XMLScraperGraph
tests/graphs/xml_scraper_openai_test.py:21
Function
graph_config
Configuration for the DepthSearchGraph
tests/graphs/depth_search_graph_openai_test.py:16
Function
graph_config
Configuration of the graph
tests/graphs/scrape_xml_ollama_test.py:28
Function
graph_config
Creation of the graph
tests/graphs/screenshot_scraper_test.py:15
Function
graph_config
Configuration of the graph
tests/graphs/smart_scraper_ernie_test.py:11
Function
graph_config
Configuration of the graph
tests/graphs/smart_scraper_clod_test.py:16
Function
graph_config
Configuration of the graph fixture.
tests/graphs/scrape_plain_text_mistral_test.py:28
Function
groq_config
Groq configuration for testing.
tests/conftest.py:108
Method
handle_directory
Handles the directory by compressing the source document and updating the state. Parameters: state (dict): The current state
scrapegraphai/nodes/fetch_node.py:128
← previous
next →
401–500 of 904, ranked by callers