Code
Hub
Workspaces
Following
Trending
Connect
MCP
copy
Create free account
hub
/
github.com/ScrapeGraphAI/Scrapegraph-ai
/ functions
Functions
904 in github.com/ScrapeGraphAI/Scrapegraph-ai
⨍
Functions
904
◇
Types & classes
195
↳
Endpoints
27
Method
test_run_safe_async
(self)
tests/graphs/abstract_graph_test.py:213
Method
test_run_success
Test run() returns the merged script when execution is successful.
tests/test_script_creator_multi_graph.py:33
Method
test_run_with_answer
Test that the run() method returns the correct answer when present.
tests/test_omni_search_graph.py:25
Method
test_run_with_empty_prompt
Test run() method with an empty prompt.
tests/test_script_creator_multi_graph.py:181
Function
test_run_with_exception_in_execute
Test that CSVScraperMultiGraph.run propagates exceptions from the graph's execute method.
tests/test_csv_scraper_multi_graph.py:146
Method
test_run_without_answer
Test that the run() method returns the default message when no answer is found.
tests/test_omni_search_graph.py:42
Method
test_schema_deepcopy
Test that the schema is deep copied correctly so external changes do not affect it.
tests/test_omni_search_graph.py:96
Method
test_scrape_csv_file
Test scraping a CSV file.
tests/integration/test_file_formats_integration.py:89
Method
test_scrape_csv_url
Test scraping CSV from a URL.
tests/integration/test_file_formats_integration.py:101
Function
test_scrape_do_fetch_api_encoding_special_characters
Test scrape_do_fetch function in API mode with a target URL that includes query parameters and special characters. This test verifies that th
tests/test_scrape_do.py:265
Function
test_scrape_do_fetch_empty_token_without_proxy
Test scrape_do_fetch in API mode with an empty token. This verifies that even when the token is an empty string, the URL is constructed as ex
tests/test_scrape_do.py:208
Function
test_scrape_do_fetch_exception_propagation
Test that scrape_do_fetch properly propagates exceptions raised by requests.get.
tests/test_scrape_do.py:161
Function
test_scrape_do_fetch_with_proxy_custom_env
Test scrape_do_fetch using proxy mode with a custom PROXY_SCRAPE_DO_URL environment variable.
tests/test_scrape_do.py:131
Function
test_scrape_do_fetch_with_proxy_no_geo
Test scrape_do_fetch function using proxy mode without geoCode. This test verifies that: - The function constructs the correct proxy
tests/test_scrape_do.py:36
Function
test_scrape_do_fetch_with_proxy_with_empty_geo
Test scrape_do_fetch function using proxy mode with an empty geoCode string. Even though geoCode is provided (as an empty string), it should
tests/test_scrape_do.py:230
Function
test_scrape_do_fetch_with_proxy_with_geo
Test scrape_do_fetch function using proxy mode with geoCode and super_proxy enabled. This test verifies that: - The function construc
tests/test_scrape_do.py:70
Function
test_scrape_do_fetch_with_proxy_with_geo_and_super_false
Test scrape_do_fetch function using proxy mode with geoCode provided and super_proxy set to False. This test verifies that the correct proxy
tests/test_scrape_do.py:174
Function
test_scrape_do_fetch_without_proxy
Test scrape_do_fetch function using API mode (without proxy). This test verifies that: 1. The function correctly uses the API mode when
tests/test_scrape_do.py:9
Function
test_scrape_do_fetch_without_proxy_custom_env
Test scrape_do_fetch using API mode with a custom API_SCRAPE_DO_URL environment variable.
tests/test_scrape_do.py:107
Method
test_scrape_json_file
Test scraping a JSON file.
tests/integration/test_file_formats_integration.py:25
Method
test_scrape_json_url
Test scraping JSON from a URL.
tests/integration/test_file_formats_integration.py:37
Function
test_scrape_method_js_support
Test that scrape method calls ascrape_with_js_support when requires_js_support is True.
tests/test_chromium.py:868
Function
test_scrape_method_playwright
Test the scrape method with playwright backend.
tests/test_chromium.py:171
Function
test_scrape_method_retry_logic
Test the retry logic in the scrape method.
tests/test_chromium.py:189
Function
test_scrape_method_selenium
Test that the scrape method works correctly for selenium by returning the dummy selenium content.
tests/test_chromium.py:102
Function
test_scrape_method_selenium_firefox
Test that the scrape method works correctly for selenium with firefox backend.
tests/test_chromium.py:650
Function
test_scrape_method_unsupported_backend
Test that the scrape method raises a ValueError when an unsupported backend is provided.
tests/test_chromium.py:94
Method
test_scrape_multiple_pages
Test scraping multiple pages simultaneously.
tests/integration/test_multi_graph_integration.py:21
Function
test_scrape_playwright_value_error_retry_failure
Test that ascrape_playwright retries on ValueError and ultimately raises RuntimeError after exhausting retries.
tests/test_chromium.py:1905
Method
test_scrape_products_page
Test scraping a products page.
tests/integration/test_smart_scraper_integration.py:75
Function
test_scrape_selenium_exception
Test that the scrape method for selenium backend raises a ValueError when ascrape_undetected_chromedriver fails.
tests/test_chromium.py:1247
Method
test_scrape_test_website
Test scraping the official test website.
tests/integration/test_smart_scraper_integration.py:161
Function
test_scrape_uses_js_support_flag
Test that the scrape method uses ascrape_with_js_support when requires_js_support is True.
tests/test_chromium.py:1387
Method
test_scrape_with_openai
Test scraping with OpenAI using mock server.
tests/integration/test_smart_scraper_integration.py:38
Method
test_scrape_with_schema
Test scraping with a Pydantic schema.
tests/integration/test_smart_scraper_integration.py:54
Method
test_scrape_with_timeout
Test scraping with a slow-loading page.
tests/integration/test_smart_scraper_integration.py:90
Method
test_scrape_xml_file
Test scraping an XML file.
tests/integration/test_file_formats_integration.py:57
Method
test_scrape_xml_url
Test scraping XML from a URL.
tests/integration/test_file_formats_integration.py:69
Method
test_scraping_performance
Benchmark scraping performance.
tests/integration/test_smart_scraper_integration.py:181
Function
test_scraping_pipeline
Start of the scraping pipeline
tests/graphs/smart_scraper_fireworks_test.py:29
Function
test_scraping_pipeline
Start of the scraping pipeline
tests/graphs/smart_scraper_ollama_test.py:25
Function
test_scraping_pipeline
Start of the scraping pipeline
tests/graphs/smart_scraper_multi_lite_graph_openai_test.py:30
Function
test_scraping_pipeline
Start of the scraping pipeline
tests/graphs/smart_scraper_openai_test.py:30
Function
test_scraping_pipeline
Start of the scraping pipeline
tests/graphs/scrape_xml_ollama_test.py:42
Function
test_scraping_pipeline
Start of the scraping pipeline
tests/graphs/smart_scraper_ernie_test.py:25
Function
test_scraping_pipeline
Start of the scraping pipeline
tests/graphs/smart_scraper_clod_test.py:29
Function
test_scraping_pipeline
Test the SmartScraperGraph scraping pipeline.
tests/graphs/scrape_plain_text_mistral_test.py:42
Function
test_screenshot_scraper_graph
test
tests/graphs/screenshot_scraper_test.py:29
Function
test_script_creator_graph
Test the ScriptCreatorGraph
tests/graphs/script_generator_test.py:27
Method
test_search_and_scrape
Test searching and scraping results.
tests/integration/test_multi_graph_integration.py:85
Function
test_search_graph
Test the SearchGraph functionality
tests/graphs/search_graph_openai_test.py:40
Function
test_search_graph_execution_info
Test getting the execution info of SearchGraph
tests/graphs/search_graph_openai_test.py:54
Function
test_search_link_node
Test the SearchLinkNode execution.
tests/nodes/search_link_node_test.py:42
Function
test_search_proxy_servers_exception
()
tests/utils/test_proxy_rotation.py:28
Function
test_search_proxy_servers_success
()
tests/utils/test_proxy_rotation.py:14
Function
test_search_proxy_success
()
tests/utils/test_proxy_rotation.py:61
Function
test_set_common_params
Test that the set_common_params method correctly updates the configuration of all nodes in the graph.
tests/graphs/abstract_graph_test.py:51
Method
test_set_common_params
Test that the set_common_params method correctly updates the configuration of all nodes in the graph.
tests/graphs/abstract_graph_test.py:249
Function
test_shallow_copy_fallback
()
tests/utils/copy_utils_test.py:98
Function
test_smart_scraper_pipeline
()
tests/graphs/search_link_ollama.py:4
Method
test_specific_model_token_values
Test specific expected token values for selected models from various providers.
tests/test_models_tokens.py:88
Method
test_specific_models_additional
Test specific token values for additional models across various providers.
tests/test_models_tokens.py:130
Method
test_specific_token_value
Test specific expected token value for a known model.
tests/test_models_tokens.py:52
Function
test_srcfile_import_missing_spec
()
tests/utils/test_sys_dynamic_import.py:36
Function
test_srcfile_import_missing_spec_loader
(mocker)
tests/utils/test_sys_dynamic_import.py:44
Function
test_srcfile_import_success
()
tests/utils/test_sys_dynamic_import.py:21
Method
test_successful_result
Test creating a successful batch result.
tests/test_batch_api.py:96
Method
test_timeout_custom_value
Test that custom timeout value is properly stored.
tests/test_fetch_node_timeout.py:74
Method
test_timeout_default_value
Test that default timeout is set to 30 seconds.
tests/test_fetch_node_timeout.py:65
Method
test_timeout_no_config
Test that timeout defaults to 30 when no node_config provided.
tests/test_fetch_node_timeout.py:92
Method
test_timeout_none_value
Test that timeout can be disabled by setting to None.
tests/test_fetch_node_timeout.py:83
Method
test_timeout_not_overridden_in_loader_kwargs
Test that existing timeout in loader_kwargs is not overridden.
tests/test_fetch_node_timeout.py:213
Method
test_timeout_propagated_to_chromium_loader
Test that timeout is propagated to ChromiumLoader via loader_kwargs.
tests/test_fetch_node_timeout.py:189
Method
test_to_jsonl_line_basic
Test basic JSONL line generation.
tests/test_batch_api.py:28
Method
test_to_jsonl_line_custom_temperature
Test custom temperature in JSONL output.
tests/test_batch_api.py:78
Method
test_to_jsonl_line_with_max_tokens
Test JSONL line with max_tokens specified.
tests/test_batch_api.py:45
Method
test_to_jsonl_line_with_response_format
Test JSONL line with response_format specified.
tests/test_batch_api.py:56
Method
test_to_jsonl_line_without_optional_fields
Test that optional fields are excluded when None.
tests/test_batch_api.py:67
Method
test_togetherai_specific
Test specific token value for 'meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo' in the toghetherai provider.
tests/test_models_tokens.py:177
Method
test_token_limits_range
Test that token limits for all models fall within a plausible range (e.g., 1 to 300000).
tests/test_models_tokens.py:66
Method
test_total_model_keys
Test that the total number of models across all providers is above an expected count.
tests/test_models_tokens.py:47
Function
test_unhandled_type
()
tests/utils/copy_utils_test.py:152
Function
test_with_boto3
()
tests/utils/copy_utils_test.py:192
Function
test_with_pydantic
()
tests/utils/copy_utils_test.py:185
Function
test_xml_scraper_execution_info
Test getting the execution info of XMLScraperGraph
tests/graphs/xml_scraper_openai_test.py:68
Function
test_xml_scraper_graph
Test the XMLScraperGraph scraping pipeline
tests/graphs/xml_scraper_openai_test.py:53
Function
test_xml_scraper_save_results
Test saving the results of XMLScraperGraph to CSV and JSON
tests/graphs/xml_scraper_openai_test.py:86
Function
unsetDEFAULT_HANDLER
Remove the default handler from the root logger for the library.
scrapegraphai/utils/logging.py:171
Function
unset_formatting
Remove formatting for all handlers bound to the root logger for the library.
scrapegraphai/utils/logging.py:206
Function
unset_propagation
Disable propagation of the root logger for the library.
scrapegraphai/utils/logging.py:185
Function
update_plot
(top_bottom, left_right, image_size)
scrapegraphai/utils/screenshot_scraping/screenshot_preparation.py:164
Method
validate_execution_errors
Ensure errors dictionary contains execution key.
scrapegraphai/utils/code_error_analysis.py:64
Function
validate_extracted_fields
Validate that required fields were extracted with minimum values. Args: result: Extraction result required_fields: List of requir
tests/fixtures/helpers.py:223
Method
validate_max_results
Validate max results.
scrapegraphai/utils/research_web.py:80
Method
validate_query
Validate search query.
scrapegraphai/utils/research_web.py:73
Function
validate_schema_match
Validate that data matches a Pydantic schema. Args: data: Data to validate schema_class: Pydantic schema class Returns:
tests/fixtures/helpers.py:206
Method
validate_search_engine
Validate search engine.
scrapegraphai/utils/research_web.py:63
Method
validate_validation_errors
Ensure errors dictionary contains validation key.
scrapegraphai/utils/code_error_analysis.py:79
Function
verify_request_signature
Verify the signature of an incoming request. Args: request_data (Dict): Request data to verify signature (str): Provided sig
scrapegraphai/utils/research_web.py:456
Function
warning_once
Emit a warning log with the same message only once. This function is added as a method to the logging.Logger class. It emits a warning l
scrapegraphai/utils/logging.py:215
← previous
next →
801–900 of 904, ranked by callers