Code
Hub
Workspaces
Following
Trending
Connect
MCP
copy
Create free account
hub
/
github.com/ScrapeGraphAI/Scrapegraph-ai
/ functions
Functions
904 in github.com/ScrapeGraphAI/Scrapegraph-ai
⨍
Functions
904
◇
Types & classes
195
↳
Endpoints
27
Function
test_ascrape_playwright_scroll_constant_height
Test that ascrape_playwright_scroll exits the scroll loop when page height remains constant.
tests/test_chromium.py:1070
Function
test_ascrape_playwright_scroll_exception_cleanup
Test that ascrape_playwright_scroll calls browser.close() when an exception occurs during page navigation.
tests/test_chromium.py:1264
Function
test_ascrape_playwright_scroll_invalid_params
Test that ascrape_playwright_scroll raises ValueError for invalid scroll parameters.
tests/test_chromium.py:216
Function
test_ascrape_playwright_scroll_invalid_type
Test that ascrape_playwright_scroll raises TypeError when invalid types are passed for scroll or sleep.
tests/test_chromium.py:1974
Function
test_ascrape_playwright_scroll_load_state
Test that ascrape_playwright_scroll waits for the custom load_state value.
tests/test_chromium.py:1855
Function
test_ascrape_playwright_scroll_retry_failure
Test that ascrape_playwright_scroll retries on failure and returns an error message after retry_limit attempts.
tests/test_chromium.py:882
Function
test_ascrape_playwright_scroll_scroll_to_bottom_false
Test ascrape_playwright_scroll with scroll_to_bottom=False. Simulate a page whose scroll height increases initially then remains constant; wit
tests/test_chromium.py:1775
Function
test_ascrape_playwright_scroll_timeout_none
Test ascrape_playwright_scroll when timeout is None and scroll_to_bottom is True. The test uses a dummy page.evaluate sequence to simulate increas
tests/test_chromium.py:1506
Function
test_ascrape_playwright_stealth_exception_cleanup
Test that ascrape_playwright calls browser.close() even if Malenia.apply_stealth fails.
tests/test_chromium.py:1603
Function
test_ascrape_undetected_chromedriver_failure
Test that ascrape_undetected_chromedriver returns an error message after all retry attempts when driver.get always fails.
tests/test_chromium.py:1034
Function
test_ascrape_undetected_chromedriver_missing_import
Test that ascrape_undetected_chromedriver raises ImportError when undetected_chromedriver is not installed.
tests/test_chromium.py:501
Function
test_ascrape_undetected_chromedriver_quit_called
Test that ascrape_undetected_chromedriver calls driver.quit() on every attempt even when get() fails.
tests/test_chromium.py:518
Function
test_ascrape_undetected_chromedriver_success
Test that ascrape_undetected_chromedriver successfully returns content using the selenium backend.
tests/test_chromium.py:293
Function
test_ascrape_undetected_chromedriver_unsupported_browser
Test ascrape_undetected_chromedriver raises an error when an unsupported browser is provided.
tests/test_chromium.py:333
Function
test_ascrape_undetected_chromedriver_zero_retry
Test that ascrape_undetected_chromedriver returns empty result when retry_limit is set to 0.
tests/test_chromium.py:1228
Function
test_ascrape_with_js_support
Test the ascrape_with_js_support method with different browser configurations.
tests/test_chromium.py:146
Function
test_ascrape_with_js_support_browser_error_cleanup
Test ascrape_with_js_support to ensure that browser.close() is always called even if an exception occurs. This simulates a navigation error and ch
tests/test_chromium.py:1527
Function
test_ascrape_with_js_support_browser_name_override_new
Test that ascrape_with_js_support calls the firefox branch correctly when browser_name is set to "firefox". This simulates a dummy playwright that
tests/test_chromium.py:1800
Function
test_ascrape_with_js_support_calls_close
Test that ascrape_with_js_support calls browser.close() after scraping.
tests/test_chromium.py:969
Function
test_ascrape_with_js_support_context_error_cleanup
Test that ascrape_with_js_support calls browser.close() even if new_context fails.
tests/test_chromium.py:2096
Function
test_ascrape_with_js_support_exception_cleanup
Test that ascrape_with_js_support calls browser.close() after an exception occurs.
tests/test_chromium.py:1146
Function
test_ascrape_with_js_support_negative_retry
Test that ascrape_with_js_support returns None when retry_limit is negative (loop not executed).
tests/test_chromium.py:697
Function
test_ascrape_with_js_support_non_timeout_retry
Test that ascrape_with_js_support retries on a non-timeout exception and eventually succeeds.
tests/test_chromium.py:1330
Function
test_ascrape_with_js_support_retry_failure
Test that ascrape_with_js_support retries and ultimately fails when page.goto always times out.
tests/test_chromium.py:236
Function
test_ascrape_with_js_support_retry_success
Test that ascrape_with_js_support retries on failure and returns content on a subsequent successful attempt.
tests/test_chromium.py:576
Function
test_ascrape_with_js_support_storage_state
Test that ascrape_with_js_support passes the storage_state to the new_context call.
tests/test_chromium.py:726
Function
test_ascrape_with_js_support_value_error_failure
Test that ascrape_with_js_support raises RuntimeError after exhausting retries on persistent ValueError.
tests/test_chromium.py:1721
Function
test_ascrape_with_js_support_value_error_success
Test that ascrape_with_js_support retries on ValueError and eventually succeeds.
tests/test_chromium.py:1663
Method
test_azure_openai_tokens
Test that the 'azure_openai' provider exists and its tokens are valid.
tests/test_models_tokens.py:18
Function
test_basic_html_to_md
()
tests/utils/convert_to_md_test.py:4
Method
test_batch_model_override
Test that batch_config model overrides the LLM model name.
tests/test_batch_api.py:366
Function
test_bing_search
Tests search_on_web with Bing search engine.
tests/utils/research_web_test.py:15
Function
test_build_cmd_defaults
()
tests/test_plasmate.py:69
Function
test_build_cmd_with_headers
()
tests/test_plasmate.py:89
Function
test_build_cmd_with_selector
()
tests/test_plasmate.py:81
Function
test_burr_kwargs
Test that burr_kwargs configuration correctly sets use_burr and burr_config on the graph.
tests/graphs/abstract_graph_test.py:35
Function
test_circular_reference
()
tests/utils/copy_utils_test.py:105
Function
test_circular_reference_in_dict
()
tests/utils/copy_utils_test.py:174
Function
test_cleanup_html_no_body
Test cleanup_html raises ValueError when no <body> tag is present.
tests/test_cleanup_html.py:61
Function
test_cleanup_html_success
Test cleanup_html with valid HTML containing title, body, links, images, and scripts.
tests/test_cleanup_html.py:32
Function
test_client
()
tests/utils/copy_utils_test.py:158
Function
test_code_generator_execution_info
Test getting the execution info of CodeGeneratorGraph
tests/graphs/code_generator_graph_openai_test.py:71
Function
test_code_generator_graph
Test the CodeGeneratorGraph scraping pipeline
tests/graphs/code_generator_graph_openai_test.py:55
Method
test_completed_batch
Test a completed batch job info.
tests/test_batch_api.py:125
Function
test_complex_html_structure
()
tests/utils/convert_to_md_test.py:30
Method
test_concurrent_scraping_performance
Test performance of concurrent scraping.
tests/integration/test_multi_graph_integration.py:39
Function
test_config_deepcopy
Test that the configuration dictionary is deep-copied. Modifying the original config after instantiation should not affect the multi_graph copy.
tests/test_csv_scraper_multi_graph.py:103
Method
test_config_deepcopy
Test that the config passed to OmniSearchGraph is deep copied properly.
tests/test_omni_search_graph.py:81
Method
test_config_deepcopy
Test that the configuration is deep copied during initialization.
tests/test_script_creator_multi_graph.py:84
Method
test_consistent_results_across_providers
Test that different providers produce consistent results.
tests/integration/test_smart_scraper_integration.py:134
Method
test_create_graph_node_configs
Test that _create_graph() sets correct node configurations for its nodes.
tests/test_script_creator_multi_graph.py:130
Function
test_create_graph_structure
Test that _create_graph constructs a graph with the expected structure.
tests/test_csv_scraper_multi_graph.py:74
Method
test_create_graph_structure
Test that the _create_graph() method returns a graph with the expected structure.
tests/test_omni_search_graph.py:58
Method
test_create_graph_structure
Test _create_graph() returns a BaseGraph with the correct graph name and structure.
tests/test_script_creator_multi_graph.py:65
Method
test_create_llm
(self, llm_config, expected_model)
tests/graphs/abstract_graph_test.py:148
Method
test_create_llm_unknown_provider
(self)
tests/graphs/abstract_graph_test.py:152
Method
test_create_llm_with_custom_model_instance
Test that the _create_llm method correctly uses a custom model instance when provided in the configuration.
tests/graphs/abstract_graph_test.py:228
Method
test_create_llm_with_rate_limit
(self, llm_config, expected_model)
tests/graphs/abstract_graph_test.py:208
Method
test_custom_search_engine_config
Test that the custom search_engine parameter from the config is correctly passed to the SearchInternetNode.
tests/test_search_graph.py:97
Function
test_deepcopy_object_without_dict
()
tests/utils/copy_utils_test.py:113
Function
test_deepcopy_simple_dict
()
tests/utils/copy_utils_test.py:52
Function
test_deepcopy_simple_list
()
tests/utils/copy_utils_test.py:60
Function
test_deepcopy_with_custom_deepcopy_fallback
()
tests/utils/copy_utils_test.py:91
Function
test_deepcopy_with_frozenset
()
tests/utils/copy_utils_test.py:76
Function
test_deepcopy_with_object
()
tests/utils/copy_utils_test.py:83
Function
test_deepcopy_with_tuple
()
tests/utils/copy_utils_test.py:68
Function
test_depth_search_execution_info
Test getting the execution info of DepthSearchGraph
tests/graphs/depth_search_graph_openai_test.py:48
Function
test_depth_search_graph
Test the DepthSearchGraph scraping pipeline
tests/graphs/depth_search_graph_openai_test.py:33
Method
test_depth_search_graph_initialization
Test that DepthSearchGraph initializes correctly with different source types. This test verifies that the input_key is set to 'url' f
tests/test_depth_search_graph.py:19
Function
test_dynamic_import_failure
Test that ChromiumLoader raises ImportError when dynamic_import fails.
tests/test_chromium.py:562
Function
test_dynamic_import_import_error_with_custom_message
()
tests/utils/test_sys_dynamic_import.py:86
Function
test_dynamic_import_module_already_imported
()
tests/utils/test_sys_dynamic_import.py:74
Function
test_dynamic_import_success
()
tests/utils/test_sys_dynamic_import.py:62
Function
test_empty_html
()
tests/utils/convert_to_md_test.py:25
Method
test_empty_parsed_docs_raises
Test that empty parsed_docs raises ValueError.
tests/test_batch_api.py:313
Method
test_entry_point_node
Test that the graph entry point is the GraphIteratorNode (the first node).
tests/test_script_creator_multi_graph.py:150
Method
test_ernie_all_values
Test that all models in the 'ernie' provider have token values exactly 4096.
tests/test_models_tokens.py:187
Method
test_error_handling_404
Test handling of 404 errors.
tests/integration/test_smart_scraper_integration.py:107
Function
test_execute_merge_json_decode_error
Test that execute() handles a JSONDecodeError in the merge chain properly.
tests/test_generate_answer_node.py:125
Function
test_execute_multiple_chunks
Test the execute() method for a scenario with multiple document chunks. It simulates parallel processing of chunks and then merges them.
tests/test_generate_answer_node.py:84
Function
test_execute_script_creator_single_chunk
Test the execute() method for the scenario when script_creator mode is enabled. This verifies that the non-markdown prompt templates branch i
tests/test_generate_answer_node.py:206
Method
test_execute_search_node
(self)
tests/nodes/search_internet_node_test.py:33
Function
test_execute_single_chunk
Test the execute() method for a single document chunk.
tests/test_generate_answer_node.py:109
Function
test_execute_timeout_single_chunk
Test that execute() properly handles a Timeout exception in the single chunk branch.
tests/test_generate_answer_node.py:190
Function
test_extract_from_script_tags
Test extracting JSON and dynamic data from script tags.
tests/test_cleanup_html.py:13
Method
test_failed_result
Test creating a failed batch result.
tests/test_batch_api.py:108
Function
test_fallback_to_chrome_on_empty
(mock_run, mock_which)
tests/test_plasmate.py:185
Function
test_fetch_csv
()
tests/nodes/fetch_node_test.py:60
Function
test_fetch_html
(mocker)
tests/nodes/fetch_node_test.py:6
Function
test_fetch_json
()
tests/nodes/fetch_node_test.py:42
Function
test_fetch_txt
()
tests/nodes/fetch_node_test.py:69
Function
test_fetch_xml
()
tests/nodes/fetch_node_test.py:51
Method
test_format_instructions_without_schema
Test default format instructions when no schema is provided.
tests/test_batch_api.py:386
Method
test_get_considered_urls
Test that get_considered_urls returns the correct list of URLs considered during the search process.
tests/test_search_graph.py:17
Function
test_get_execution_info
Get the execution info
tests/graphs/smart_scraper_fireworks_test.py:43
Function
test_get_execution_info
Get the execution info
tests/graphs/smart_scraper_ollama_test.py:40
Function
test_get_execution_info
Get the execution info
tests/graphs/smart_scraper_multi_lite_graph_openai_test.py:44
Function
test_get_execution_info
Get the execution info
tests/graphs/smart_scraper_openai_test.py:44
Function
test_get_execution_info
Get the execution info
tests/graphs/smart_scraper_ernie_test.py:40
Function
test_get_execution_info
Get the execution info
tests/graphs/smart_scraper_clod_test.py:43
← previous
next →
601–700 of 904, ranked by callers