XMLScraperGraph is a scraping pipeline that extracts information from XML files using a natural language model to interpret and answer prompts. Attributes: prompt (str): The prompt for the graph. source (str): The source of the graph. config (dict): Configuratio
| 12 | |
| 13 | |
| 14 | class XMLScraperGraph(AbstractGraph): |
| 15 | """ |
| 16 | XMLScraperGraph is a scraping pipeline that extracts information from XML files using a natural |
| 17 | language model to interpret and answer prompts. |
| 18 | |
| 19 | Attributes: |
| 20 | prompt (str): The prompt for the graph. |
| 21 | source (str): The source of the graph. |
| 22 | config (dict): Configuration parameters for the graph. |
| 23 | schema (BaseModel): The schema for the graph output. |
| 24 | llm_model: An instance of a language model client, configured for generating answers. |
| 25 | embedder_model: An instance of an embedding model client, |
| 26 | configured for generating embeddings. |
| 27 | verbose (bool): A flag indicating whether to show print statements during execution. |
| 28 | headless (bool): A flag indicating whether to run the graph in headless mode. |
| 29 | model_token (int): The token limit for the language model. |
| 30 | |
| 31 | Args: |
| 32 | prompt (str): The prompt for the graph. |
| 33 | source (str): The source of the graph. |
| 34 | config (dict): Configuration parameters for the graph. |
| 35 | schema (BaseModel): The schema for the graph output. |
| 36 | |
| 37 | Example: |
| 38 | >>> xml_scraper = XMLScraperGraph( |
| 39 | ... "List me all the attractions in Chioggia.", |
| 40 | ... "data/chioggia.xml", |
| 41 | ... {"llm": {"model": "openai/gpt-3.5-turbo"}} |
| 42 | ... ) |
| 43 | >>> result = xml_scraper.run() |
| 44 | """ |
| 45 | |
| 46 | def __init__( |
| 47 | self, |
| 48 | prompt: str, |
| 49 | source: str, |
| 50 | config: dict, |
| 51 | schema: Optional[Type[BaseModel]] = None, |
| 52 | ): |
| 53 | super().__init__(prompt, config, source, schema) |
| 54 | |
| 55 | self.input_key = "xml" if source.endswith("xml") else "xml_dir" |
| 56 | |
| 57 | def _create_graph(self) -> BaseGraph: |
| 58 | """ |
| 59 | Creates the graph of nodes representing the workflow for web scraping. |
| 60 | |
| 61 | Returns: |
| 62 | BaseGraph: A graph instance representing the web scraping workflow. |
| 63 | """ |
| 64 | |
| 65 | fetch_node = FetchNode(input="xml | xml_dir", output=["doc"]) |
| 66 | |
| 67 | generate_answer_node = GenerateAnswerNode( |
| 68 | input="user_prompt & (relevant_chunks | doc)", |
| 69 | output=["answer"], |
| 70 | node_config={ |
| 71 | "llm_model": self.llm_model, |
no outgoing calls