Scrapes HTML pages from URLs using a (headless) instance of the Chromium web driver with proxy protection. Attributes: backend: The w

SmartScraper is a scraping pipeline that automates the process of extracting information from web pages using a natural language model to

BaseGraph manages the execution flow of a graph composed of interconnected nodes. Attributes: nodes (list): A dictionary mapping eac

A node responsible for fetching the HTML content of a specified URL and updating the graph's state with this content. It uses ChromiumLoader

Initializes the GenerateAnswerNode class. Args: input (str): The input data type for the node. output (List[str]): The outpu

ScriptCreatorMultiGraph is a scraping pipeline that scrapes a list of URLs generating web scraping scripts. It only requires a user promp

A node responsible for parsing HTML content from a document. The parsed content is split into chunks for further processing. This node e

SearchGraph is a scraping pipeline that searches the internet for answers to a given prompt. It only requires a user prompt to search the int

A node responsible for instantiating and running multiple graph instances in parallel. It creates as many graph instances as the number of el

A node responsible for merging the answers from multiple graph instances into a single answer. Attributes: llm_model: An instance of

JSONScraperGraph defines a scraping pipeline for JSON files. Attributes: prompt (str): The prompt for the graph. source (str

CSVScraperMultiGraph is a scraping pipeline that scrapes a list of URLs and generates answers to a given prompt. It only requires a user

Exception raised when state dictionary is missing required keys.

XMLScraperGraph is a scraping pipeline that extracts information from XML files using a natural language model to interpret and answer prompt

Exception raised when state dictionary is missing required keys.

Status information about a batch job.

A single request within a batch submission.

OmniSearchGraph is a scraping pipeline that searches the internet for answers to a given prompt. It only requires a user prompt to search the

A node that generates answers using the OpenAI Batch API. Instead of making individual LLM calls for each document, this node collects all pr

The result of a single request within a completed batch.

CodeGeneratorGraph is a script generator pipeline that generates the function extract_data(html: str) -> dict() for extracting the wanted

Exception raised when search request fails.

SmartScraperMultiGraph is a scraping pipeline that scrapes a list of URLs and generates answers to a given prompt. It only requires a use

Base exception for code analysis errors.

Results from a single benchmark run.

A class representing a graph for extracting information from CSV files. Attributes: prompt (str): The prompt used to generate an ans

Base exception for code generation errors.

CodeGeneratorGraph is a script generator pipeline that generates the function extract_data(html: str) -> dict() for extracting the wanted

ScriptCreatorGraph defines a scraping pipeline for generating web scraping scripts. Attributes: prompt (str): The prompt for the gra

SmartScraperMultiLiteGraph is a scraping pipeline that scrapes a list of URLs and merge the content first and finally generates answers to a

DocumentScraperGraph is a scraping pipeline that automates the process of extracting information from web pages using a natural language mode

A node responsible for checking if a website is scrapeable or not based on the robots.txt file. It uses a language model to determine if the

A node that generates a search query based on the user's input and searches the internet for relevant information. The node constructs a prom

Track and analyze benchmark results.

Base model for code analysis state validation.

A node that determines the next step in the graph's execution flow based on the presence and content of a specified key in the graph's state.

Base model for code correction state validation.

A dummy chain for simulating a chain's invoke behavior. Returns a successful answer in the expected format.

A dummy ChatOllama class to simulate ChatOllama behavior.

DummyLLM that supports the pipe '|' operator. When used in a chain with a PromptTemplate, the pipe operator returns self, simulating chain com

JSONScraperMultiGraph is a scraping pipeline that scrapes a list of URLs and generates answers to a given prompt. It only requires a user

OmniScraper is a scraping pipeline that automates the process of extracting information from web pages using a natural language model to

Fetches pages using Plasmate — a lightweight Rust browser engine that outputs Structured Object Model (SOM) instead of raw HTML. Advantages o

A node responsible for compressing the input tokens and storing the document in a vector database for retrieval. Relevant chunks are stored i

Exception raised when search configuration is invalid.

SearchLinkGraph is a scraping pipeline that automates the process of extracting information from web pages using a natural language model

A node that can filter out the relevant links in the webpage content for the user prompt. Node expects the already scrapped links on the webp

SmartScraperLiteGraph is a scraping pipeline that automates the process of extracting information from web pages. Attributes: pr

A scraping pipeline that uses OpenAI Batch API for cost savings. Similar to SmartScraperMultiGraph, but instead of making individual LLM call

SmartScraperMultiConditionalGraph is a scraping pipeline that scrapes a list of URLs and generates answers to a given prompt. Attributes

XMLScraperMultiGraph is a scraping pipeline that scrapes a list of URLs and generates answers to a given prompt. It only requires a user

Summary statistics for multiple benchmark runs.

Bridge class to integrate Burr into ScrapeGraphAI graphs. Args: base_graph (BaseGraph): The base graph to convert to a Burr applicat

Bridge class to convert a base graph node to a Burr action. This is nice because we can dynamically declare the inputs/outputs (and not rely o

A wrapper for the ChatOpenAI class (CLōD uses an OpenAI-like API) that provides default configuration and could be extended with additional m

A node responsible for concatenating the answers from multiple graph instances into a single answer. Attributes: verbose (bool):

Callback Handler that tracks LLMs info.

CustomLLMCallbackManager class provides a mechanism to acquire a callback for LLM models in an exclusive, thread-safe manner. Attributes

Custom exception raised when an object cannot be deep-copied.

A wrapper for the ChatOpenAI class (DeepSeek uses an OpenAI-like API) that provides default configuration and could be extended with addition

A node responsible for compressing the input tokens and storing the document in a vector database for retrieval. Relevant chunks are stored i

Dummy BaseGraph to test _create_graph method without side effects.

Dummy graph that returns a predefined answer.

Dummy graph that simulates absence of answer in final_state.

A dummy schema class with a model_json_schema method.

Model for execution analysis state validation.

A node responsible for fetching the HTML content of a specified URL and all its sub-links recursively up to a certain level of hyperlink the

FetchScreenNode captures screenshots from a given URL and stores the image data as bytes.

A node that generates an answer using a language model (LLM) based on the user's input and the content extracted from a webpage. It construct

GenerateAnswerFromImageNode analyzes images from the state dictionary using the OpenAI API and updates the state with the consolidated answer

A node responsible for compressing the input tokens and storing the document in a vector database for retrieval. Relevant chunks are stored i

A node that generates an answer using a large language model (LLM) based on the user's input and the content extracted from a webpage. It con

A node that generates Python code for a function that extracts data from HTML based on a output schema. Attributes: llm_model: A

Generates a python script for scraping a website using the specified library. It takes the user's prompt and the scraped content as input and

A node that generates an analysis of the provided HTML code based on the wanted infromations to be extracted. Attributes: llm_model:

Retrieve images from a list of URLs and return a description of the images using an image-to-text model. Attributes: llm_model:

A node responsible for converting HTML content to Markdown format. This node takes HTML content from the state and converts it to clean, rea

A node responsible for merging scripts generated. Attributes: llm_model: An instance of a language model client, configured for gener

A wrapper for the ChatOpenAI class (MiniMax uses an OpenAI-compatible API) that provides default configuration and could be extended with add