Fetches the HTML content of a given source URL. Args: source (str): The URL to fetch content from. loader_kw

Returns a random user agent from the list. Returns: str: Random user agent string.

Get summary statistics for a specific test. Args: test_name: Name of the test Returns: BenchmarkSummary if r

Get the cost in USD for a given model and number of tokens. Args: model_name: Name of the model num_tokens: Number of tokens

Executes the web scraping and searching process. Returns: str: The answer to the prompt.

Executes the web scraping and searching process. Returns: str: The answer to the prompt.

Executes the scraping process and returns the generated code. Returns: str: The generated code.

Executes the scraping process and returns the generated code. Returns: str: The generated code.

Executes the web scraping process and returns the answer to the prompt.

Executes the web scraping and parsing process first and then concatenate the content and generates answers to a given prompt.

Set the verbosity level of the root logger for the library to INFO.

Stop the mock server.

Ensure errors dictionary has expected structure.

Asynchronously fetch a URL by running the plasmate binary in a thread pool.

Build the full prompt text for a single document. Args: user_prompt: The user's question/prompt. content: The scraped

Build telemetry payload dict. Returns None if required fields are missing.

Return the path to the plasmate binary, or raise ImportError.

Cleans the URLs extracted from the text. Args: urls (List[str]): The list of URLs to clean. Returns:

Convert a Burr state to a dictionary state. Args: burr_state (State): The Burr state to convert. Returns:

Create Burr actions from the base graph nodes. Returns: dict: A dictionary of Burr actions with the node name

Creates an extraction chain for processing the user prompt and generating the graph configuration. Returns: LLMC

Creates the graph of nodes representing the workflow for web scraping and searching. Returns: BaseGraph: A graph instanc

Abstract method to create a graph representation.

Creates the graph of nodes representing the workflow for web scraping and searching. Returns: BaseGraph: A graph instanc

Creates an instance of the OpenAI class with the provided language model configuration. Returns: OpenAI: An instance of

Create a large language model instance based on the configuration provided. Args: llm_config (dict): Configuration param

Create Burr transitions from the base graph edges. Returns: list: A list of tuples representing the transitions between

Parses and evaluates the condition expression against the state. Args: state (dict): The current state of the graph.

Executes a single node and returns execution information.

Executes the graph by traversing nodes starting from the entry point using the standard method.

Synchronously fetch a URL via the plasmate binary.

Generates a string description of all available nodes and their arguments. Returns: str: A string description of all ava

Extracts LLM and embedder model information from the node.

Determines the next node to execute based on current node type and result.

Returns a node instance by its name.

Extracts schema information from the node configuration.

Initialize a Burr application from the base graph. Args: initial_state (dict): The initial state of the Burr application

CHecks if the URL format is valid. Args: url (str): The URL to check. Returns: bool: True if the UR

Parses the input keys expression to extract relevant keys from the state based on logical conditions. The expression can cont

Helper function for Bing search with improved error handling. Args: query (str): Search query max_results (int): Maximum num

Helper function for DuckDuckGo search using the ``ddgs`` package. The ``duckduckgo-search`` package was renamed to ``ddgs``; recent ``la

Helper function for SearXNG search. Args: query (str): Search query max_results (int): Maximum number of results to return

Helper function for Serper search. Args: query (str): Search query max_results (int): Maximum number of results to return

Send CSV response.

Send JSON response.

Send telemetry in a background daemon thread.

Send XML response.

Serve 500 error.

Serve CSV endpoint.

Serve dynamically generated content.

Serve home page.

Serve JSON endpoint.

Serve paginated content.

Serve products page.

Serve projects page.

Simulate rate limiting.

Simulate a slow response.

Serve XML endpoint.

Sets the true_node_name and false_node_name for each ConditionalNode.

Updates source type and source information from FetchNode.

Validates if the provided input keys meet the minimum length requirement. Args: input_keys (List[str]): The list of inpu

Analyze scraped content using ScrapegraphAI. Args: content (str): The scraped HTML or text content. Returns: dict: The

Adds a node to the graph. Args: node (BaseNode): The node instance to add to the graph.

Compares two dictionaries for semantic equality after normalization. Args: generated_result (Dict[str, Any]): The generated result d

Create and submit an OpenAI Batch API job. Args: client: An initialized OpenAI client. requests: List of BatchRequest objects to

Creates a sandbox environment and executes the provided function code. Args: function_code (str): The code to be execute

Crop an image using the specified coordinates. Parameters: image (PIL.Image): The image to be cropped. LEFT (int, optional):

Detects and extracts text from a given image. Parameters: image (PIL Image): The input image to extract text from. la

Provides an exclusive callback for the LLM model in a thread-safe manner. Args: llm_model: The LLM model instance (e.g.,

Execute the batch generation node. Takes multiple parsed documents and a user prompt, builds prompts for each document, and submits t

Executes the GenerateAnswerNode. Args: state (dict): The current state of the graph. The input keys will be used

Analyzes the execution errors in the generated code and HTML code. Args: state (dict): Contains the 'generated_code', 'errors', 'htm

Generates corrected code based on execution error analysis. Args: state (dict): Contains the 'generated_code'. analysis (str

Executes the execution reasoning loop to ensure the generated code runs without errors. Args: state (dict): The current

Export data to a CSV file. :param data: List of dictionaries containing the data to export :param filename: Name of the file to save the

Export data to a JSON file. :param data: List of dictionaries containing the data to export :param filename: Name of the file to save th

Filter out PDF links from search results. Args: urls (List[str]): List of URLs Returns: List[str]: Filtered list of URL

Format proxy configuration into a string. Args: proxy_config: Proxy configuration as string, dict, or ProxyConfig Returns:

Generates the initial code based on the provided state. Args: state (dict): The current state of the reasoning process.

Get the current status of a batch job. Args: client: An initialized OpenAI client. batch_id: The batch ID returned by create_batc

Returns the list of URLs considered during the search. Returns: List[str]: A list of URLs considered during the search.

Function to get custom callback for LLM token usage statistics.

Converts relative URLs to full URLs based on the base URL. Filters out non-web links (mailto:, tel:, javascript:, etc.). Arg