Code
Hub
Workspaces
Following
Trending
Connect
MCP
copy
Create free account
hub
/
github.com/ScrapeGraphAI/Scrapegraph-ai
/ functions
Functions
904 in github.com/ScrapeGraphAI/Scrapegraph-ai
⨍
Functions
904
◇
Types & classes
195
↳
Endpoints
27
Method
test_get_execution_info
Test that get_execution_info returns the execution info stored in the graph.
tests/graphs/abstract_graph_test.py:307
Function
test_get_execution_info_with_schema
Get the execution info with schema
tests/graphs/smart_scraper_openai_test.py:59
Method
test_get_state
Test that get_state returns the correct final state with or without a provided key, and raises KeyError for missing keys.
tests/graphs/abstract_graph_test.py:276
Method
test_google_providers
Test that Google provider dictionaries ('google_genai' and 'google_vertexai') contain expected entries.
tests/test_models_tokens.py:26
Function
test_google_search
Tests search_on_web with Google search engine.
tests/utils/research_web_test.py:8
Method
test_groq_specific
Test specific token value for 'claude-3-haiku-20240307\'' in the groq provider.
tests/test_models_tokens.py:168
Method
test_handles_partial_failures
Test that partial failures in batch results are handled correctly.
tests/test_batch_api.py:230
Function
test_html_with_links_and_images
()
tests/utils/convert_to_md_test.py:9
Function
test_html_with_tables
()
tests/utils/convert_to_md_test.py:14
Method
test_in_progress_batch
Test an in-progress batch job info.
tests/test_batch_api.py:139
Method
test_init_attributes
Test that initial attributes are set correctly upon initialization.
tests/test_script_creator_multi_graph.py:98
Function
test_init_chat_ollama_format
Test that the __init__ method of GenerateAnswerNode sets the format attribute of a ChatOllama LLM correctly.
tests/test_generate_answer_node.py:249
Function
test_init_custom_params
()
tests/test_plasmate.py:45
Function
test_init_defaults
()
tests/test_plasmate.py:36
Function
test_init_dynamic_import_called
Test that dynamic_import is called during initialization.
tests/test_chromium.py:1204
Function
test_init_invalid_format
()
tests/test_plasmate.py:60
Function
test_init_overrides
Test that ChromiumLoader picks up and overrides attributes using kwargs.
tests/test_chromium.py:420
Function
test_init_with_no_proxy
Test that initializing ChromiumLoader with proxy=None results in loader.proxy being None.
tests/test_chromium.py:664
Function
test_invalid_proxy_raises_error
Test that providing an invalid proxy causes a ValueError during initialization (via parse_or_search_proxy).
tests/test_chromium.py:1922
Function
test_invalid_search_engine
Tests search_on_web with invalid search engine.
tests/utils/research_web_test.py:22
Function
test_is_ipv4_address
()
tests/utils/test_proxy_rotation.py:69
Method
test_json_scraper_graph_no_answer_found
Test JSONScraperGraph when no answer is found. This test checks if the graph correctly handles the scenario where no answer is genera
tests/test_json_scraper_graph.py:135
Method
test_json_scraper_graph_with_custom_schema
Test JSONScraperGraph with a custom schema. This test checks if the graph correctly handles a custom schema input and passes
tests/test_json_scraper_graph.py:189
Method
test_json_scraper_graph_with_directory
Test JSONScraperGraph with a directory of JSON files. This test checks if the graph correctly handles multiple JSON files input
tests/test_json_scraper_graph.py:21
Method
test_json_scraper_graph_with_single_file
Test JSONScraperGraph with a single JSON file. This test checks if the graph correctly handles a single JSON file input and p
tests/test_json_scraper_graph.py:78
Method
test_json_scraping_performance
Benchmark JSON scraping performance.
tests/integration/test_file_formats_integration.py:122
Function
test_lazy_load
Test that lazy_load yields Document objects with the correct dummy content and metadata.
tests/test_chromium.py:72
Function
test_lazy_load_empty_content
Test that lazy_load yields a Document with empty content if the scraper returns an empty string.
tests/test_chromium.py:1085
Function
test_lazy_load_empty_urls
Test that lazy_load yields no documents when the urls list is empty.
tests/test_chromium.py:493
Function
test_lazy_load_empty_urls
(mock_which)
tests/test_plasmate.py:262
Function
test_lazy_load_exception
Test that lazy_load propagates exception if the scraping function fails.
tests/test_chromium.py:319
Function
test_lazy_load_invalid_backend
Test that lazy_load raises AttributeError if the scraping method for an invalid backend is missing.
tests/test_chromium.py:1024
Function
test_lazy_load_markdown_format
(mock_run, mock_which)
tests/test_plasmate.py:131
Function
test_lazy_load_multiple_urls
(mock_run, mock_which)
tests/test_plasmate.py:117
Function
test_lazy_load_no_binary_raises
(mock_which)
tests/test_plasmate.py:173
Function
test_lazy_load_non_iterable_urls
Test that lazy_load raises TypeError when urls is not an iterable (e.g., integer).
tests/test_chromium.py:1995
Function
test_lazy_load_non_string_scraper
Test that lazy_load yields Document objects even if the scraping function returns a non‐string value.
tests/test_chromium.py:1463
Function
test_lazy_load_nonzero_returncode_skips
(mock_run, mock_which, caplog)
tests/test_plasmate.py:155
Function
test_lazy_load_scraper_returns_none
Test that lazy_load yields Document objects with page_content as None when the scraper returns None.
tests/test_chromium.py:1105
Function
test_lazy_load_sequential_timing
Test that lazy_load runs scraping sequentially rather than concurrently.
tests/test_chromium.py:2147
Function
test_lazy_load_skips_empty_content
(mock_run, mock_which, caplog)
tests/test_plasmate.py:145
Function
test_lazy_load_timeout_skips
(mock_run, mock_which, caplog)
tests/test_plasmate.py:164
Function
test_lazy_load_with_js_support
Test that lazy_load uses ascrape_with_js_support when requires_js_support is True.
tests/test_chromium.py:452
Function
test_lazy_load_with_non_async_scraper
Test that lazy_load raises a ValueError when a non-async function is used as the scraper. In this case, using a non-async function in place of an
tests/test_chromium.py:1589
Function
test_lazy_load_with_none_urls
Test that lazy_load raises TypeError when urls is None.
tests/test_chromium.py:2139
Function
test_lazy_load_with_single_url_string
Test that lazy_load yields Document objects when urls is a string (iterating over characters).
tests/test_chromium.py:1956
Function
test_lazy_load_with_tuple_urls
Test that lazy_load yields Document objects correctly when urls is provided as a tuple.
tests/test_chromium.py:2170
Function
test_lazy_load_yields_document
(mock_run, mock_which)
tests/test_plasmate.py:103
Function
test_llm_missing_tokens
Test that missing model tokens causes default to 8192 with an appropriate warning printed.
tests/graphs/abstract_graph_test.py:17
Function
test_max_results
Tests search_on_web with different max_results values.
tests/utils/research_web_test.py:28
Method
test_max_results_config
Test that the max_results parameter from the config is correctly passed to the SearchInternetNode.
tests/test_search_graph.py:68
Function
test_minify_html
Test minify_html function to remove comments and unnecessary whitespace.
tests/test_cleanup_html.py:70
Function
test_minimax_deprecated_models_removed
Older deprecated MiniMax models should be removed from the list.
tests/test_minimax_models.py:47
Function
test_minimax_m27_still_available
MiniMax-M2.7 and its highspeed variant should remain as legacy options.
tests/test_minimax_models.py:40
Function
test_minimax_m3_in_model_list
MiniMax-M3 should be in the model list.
tests/test_minimax_models.py:28
Function
test_minimax_m3_listed_first
MiniMax-M3 should be the first (default) model in the minimax dict.
tests/test_minimax_models.py:34
Function
test_minimax_token_limits
MiniMax model token limits should match upstream documentation.
tests/test_minimax_models.py:57
Method
test_model_name_extraction
Test model name is correctly extracted from LLM instance.
tests/test_batch_api.py:349
Function
test_no_fallback_when_content_present
When Plasmate returns content, Chrome fallback should not be called.
tests/test_plasmate.py:208
Function
test_no_retry_returns_none
Test that ascrape_playwright returns None if retry_limit is set to 0.
tests/test_chromium.py:470
Method
test_no_whitespace_in_model_names
Test that model names do not contain leading or trailing whitespace.
tests/test_models_tokens.py:121
Method
test_non_empty_model_keys
Ensure that model token names are non-empty strings.
tests/test_models_tokens.py:58
Method
test_non_empty_provider
Test that each provider dictionary is not empty.
tests/test_models_tokens.py:81
Method
test_non_existent_model
Test that a non-existent model within a valid provider returns None.
tests/test_models_tokens.py:114
Method
test_non_existent_provider
Test that a non-existent provider returns None.
tests/test_models_tokens.py:41
Method
test_nvidia_specific
Test specific token value for 'meta/codellama-70b' in the nvidia provider.
tests/test_models_tokens.py:159
Method
test_openai_tokens
Test that the 'openai' provider exists and its tokens are valid positive integers.
tests/test_models_tokens.py:7
Function
test_parse_expression
Test parse_expression function.
tests/utils/parse_state_keys_test.py:8
Function
test_parse_or_search_proxy_exception
()
tests/utils/test_proxy_rotation.py:101
Function
test_parse_or_search_proxy_success
()
tests/utils/test_proxy_rotation.py:75
Function
test_parse_or_search_proxy_unknown_server
()
tests/utils/test_proxy_rotation.py:113
Function
test_parse_proxy_exception
()
tests/utils/test_proxy_rotation.py:52
Function
test_parse_proxy_success
()
tests/utils/test_proxy_rotation.py:40
Method
test_pdf_parsing_timeout_exceeded
Test that PDF parsing raises TimeoutError when timeout is exceeded.
tests/test_fetch_node_timeout.py:163
Method
test_pdf_parsing_with_timeout
Test that PDF parsing completes within timeout.
tests/test_fetch_node_timeout.py:147
Function
test_process_missing_content_and_user_prompt
Test that process() raises a ValueError when either the content or the user prompt is missing.
tests/test_generate_answer_node.py:45
Function
test_process_success
Test that process() successfully generates an answer when both user prompt and content are provided.
tests/test_generate_answer_node.py:174
Method
test_provider_structure
Test that every provider in models_tokens has a dictionary as its value.
tests/test_models_tokens.py:74
Method
test_providers_count
Test that the total number of providers is as expected (at least 15).
tests/test_models_tokens.py:108
Function
test_proxy_parsing_in_init
Test that providing a proxy triggers the use of parse_or_search_proxy and sets loader.proxy correctly.
tests/test_chromium.py:636
Function
test_reduce_html_no_body
Test reduce_html returns specific message when no <body> tag is present.
tests/test_cleanup_html.py:142
Function
test_reduce_html_reduction_0
Test reduce_html at reduction level 0 (minification only).
tests/test_cleanup_html.py:86
Function
test_reduce_html_reduction_1
Test reduce_html at reduction level 1 (remove unnecessary attributes and empty style tags).
tests/test_cleanup_html.py:101
Function
test_reduce_html_reduction_2
Test reduce_html at reduction level 2 (further reducing text content and decomposing style tags).
tests/test_cleanup_html.py:120
Method
test_rejects_groq_provider
Test that Groq provider is rejected.
tests/test_batch_api.py:293
Method
test_rejects_non_openai_provider
Test that non-OpenAI providers are rejected.
tests/test_batch_api.py:280
Method
test_requests_get_with_timeout
Test that requests.get is called with timeout when use_soup=True.
tests/test_fetch_node_timeout.py:102
Method
test_requests_get_without_timeout_when_none
Test that requests.get is called without timeout argument when timeout=None.
tests/test_fetch_node_timeout.py:125
Method
test_results_sorted_by_custom_id
Test that results are sorted by custom_id for consistent ordering.
tests/test_batch_api.py:173
Method
test_retrieve_no_output_file
Test that retrieval fails when no output file is available.
tests/test_batch_api.py:159
Function
test_robots_node_force_scrapable
(robots_node)
tests/nodes/robot_node_test.py:59
Function
test_robots_node_not_scrapable
(robots_node)
tests/nodes/robot_node_test.py:41
Function
test_robots_node_scrapable
(robots_node)
tests/nodes/robot_node_test.py:25
Function
test_run_argument_passing
Test that CSVScraperMultiGraph.run passes the correct input arguments to the graph's execute method and returns the expected answer.
tests/test_csv_scraper_multi_graph.py:119
Method
test_run_called_twice
Test that running run() twice returns consistent and updated results.
tests/test_script_creator_multi_graph.py:196
Method
test_run_exception
Test that run() propagates exceptions raised by graph.execute.
tests/test_script_creator_multi_graph.py:162
Method
test_run_failure
Test run() returns failure message when merged_script is missing.
tests/test_script_creator_multi_graph.py:50
Function
test_run_no_answer
Test that CSVScraperMultiGraph.run returns a fallback message when no answer is provided.
tests/test_csv_scraper_multi_graph.py:58
Method
test_run_no_answer_found
Test that the run() method returns "No answer found." when the final state doesn't contain an "answer" key.
tests/test_search_graph.py:41
Method
test_run_no_schema
Test run() when schema is None.
tests/test_script_creator_multi_graph.py:116
← previous
next →
701–800 of 904, ranked by callers