Functions — github.com/Lightning-AI/LitServe

hub / github.com/Lightning-AI/LitServe / functions

Functions984 in github.com/Lightning-AI/LitServe

⨍ Functions 984◇ Types & classes 226↳ Endpoints 30

↓ 85 callersFunctionwrap_litserve_start

Pytest utility to start the server in a context manager.

src/litserve/utils.py:90

↓ 84 callersMethodjson

(self)

tests/unit/test_request_handlers.py:55

↓ 45 callersMethodtrigger_event

Triggers an event, invoking all registered callbacks for that event.

src/litserve/callbacks/base.py:78

↓ 35 callersMethodmodel

(x)

tests/e2e/openai_embedding_with_batching.py:8

↓ 35 callersMethodput

(self, *args, block=True, timeout=None)

tests/unit/test_batch.py:193

↓ 33 callersMethodget

(self, timeout=None)

tests/unit/test_loops.py:87

↓ 32 callersMethodput_response

( self, transport: MessageTransport, response_queue_id: int, uid: str,

src/litserve/loops/base.py:351

↓ 26 callersMethodareceive

(self, timeout: Optional[int] = None, consumer_id: Optional[int] = None)

tests/unit/test_loops.py:127

↓ 25 callersMethodrun

( self, lit_api: LitAPI, device: str, worker_id: int, request_queue: Q

tests/unit/test_loops.py:696

↓ 20 callersMethodpre_setup

(self, spec: Optional[LitSpec] = None)

src/litserve/api.py:370

↓ 18 callersMethodget

Get an item from the queue asynchronously.

src/litserve/transport/zmq_queue.py:173

↓ 17 callersMethodkill

(self)

src/litserve/loops/base.py:311

↓ 16 callersFunction_python_type_to_json_schema

Convert Python type annotation to JSON schema type string or dict.

src/litserve/mcp.py:196

↓ 16 callersMethodstart

Start the broker in a background thread or process.

src/litserve/transport/zmq_queue.py:41

↓ 14 callersMethodinfo

(request: Request)

src/litserve/server.py:1001

↓ 14 callersMethodrun

Start the LitServer to serve AI model requests with production-ready performance. This method launches the complete serving infrastructure: i

src/litserve/server.py:1186

↓ 12 callersFunction_inject_context

(context: Union[list[dict], dict], func, *args, **kwargs)

src/litserve/loops/base.py:44

↓ 12 callersMethodmount

Mount an ASGI app endpoint to LitServer. Use this method when you want to add an additional endpoint to the server such as /metrics endpoint f

src/litserve/loggers.py:33

↓ 11 callersMethodclose

Clean up resources.

src/litserve/transport/zmq_queue.py:125

↓ 11 callersMethodput

Send an item to a specific consumer.

src/litserve/transport/zmq_queue.py:112

↓ 9 callersMethodput_error_response

( self, transport: MessageTransport, response_queue_id: int, uid: str,

src/litserve/loops/base.py:366

↓ 7 callersFunction_async_inject_context

(context: Union[list[dict], dict], func, *args, **kwargs)

src/litserve/loops/base.py:83

↓ 7 callersFunction_ensure_lightning_installed

Ensure lightning-sdk is installed, attempting auto-installation if needed.

src/litserve/cli.py:9

↓ 7 callersFunctionextract_input_schema

Extract JSON schema for function input parameters from a Python function. Supports regular type annotations, Pydantic Fields, and Pydantic BaseMod

src/litserve/mcp.py:54

↓ 7 callersMethodformat_encoded_response

(self, data)

src/litserve/api.py:347

↓ 7 callersMethodfrom_exception

(exc: HTTPException)

src/litserve/utils.py:65

↓ 7 callersFunctionis_package_installed

(package_name: str)

src/litserve/utils.py:251

↓ 7 callersMethodset

(self)

tests/conftest.py:369

↓ 6 callersMethodareceive

(self, **kwargs)

tests/unit/test_batch.py:201

↓ 6 callersMethodbatch

(self, inputs)

tests/unit/test_litapi.py:42

↓ 6 callersMethodencode_response

(self, output_stream)

tests/unit/test_litapi.py:61

↓ 6 callersMethodlog

Log a key-value pair to the server.

src/litserve/api.py:387

↓ 6 callersMethodpopulate_context

(self, lit_spec: LitSpec, request: Any)

src/litserve/loops/base.py:340

↓ 5 callersMethodas_tool

(self)

src/litserve/mcp.py:386

↓ 5 callersFunctionconfigure_logging

Configure logging for the entire library with sensible defaults. Args: level (int): Logging level (default: logging.INFO) format

src/litserve/utils.py:152

↓ 5 callersMethodunbatch

(self, output)

tests/unit/test_litapi.py:45

↓ 4 callersMethod_get_request_queue

(self, api_path: str)

src/litserve/server.py:1043

↓ 4 callersFunction_handle_async_function

(func, *args, **kwargs)

src/litserve/loops/base.py:64

↓ 4 callersFunction_migration_warning

(feature_name)

src/litserve/server.py:193

↓ 4 callersFunction_param_name_to_title

Convert parameter name to a readable title.

src/litserve/mcp.py:241

↓ 4 callersMethod_sanitize_accelerator

(accelerator: Optional[str])

src/litserve/connector.py:61

↓ 4 callersMethod_start_worker_monitoring

( self, manager: mp.Manager, uvicorn_workers: dict[str, Union[mp.Process, threading.Th

src/litserve/server.py:1509

↓ 4 callersMethodadd_endpoint

src/litserve/specs/base.py:53

↓ 4 callersMethodadd_logger

(self, logger: Logger)

src/litserve/loggers.py:114

↓ 4 callersFunctionadd_ssl_context_from_env

Loads SSL context from base64-encoded environment variables. This function checks for the presence of `LIGHTNING_CERT_PEM` and `LIGHTNING_KEY

src/litserve/utils.py:296

↓ 4 callersFunctioncheck_cuda_with_nvidia_smi

Checks if CUDA is installed using the `nvidia-smi` command-line tool. Returns count of visible devices.

src/litserve/connector.py:100

↓ 4 callersFunctioncollate_requests

( loop: "LitLoop", lit_api: LitAPI, request_queue: Queue, transport: MessageTransport, )

src/litserve/loops/base.py:99

↓ 4 callersFunctiondump_exception

(exception)

src/litserve/utils.py:74

↓ 4 callersFunctiongenerate_random_zmq_address

Generate a random IPC address in the /tmp directory. Ensures the address is unique. Returns: str: A random IPC address suitable for Z

src/litserve/utils.py:211

↓ 4 callersMethodrun

(self, worker_id: int, sockets: Union[list[socket.socket], None] = None)

src/litserve/server.py:400

↓ 4 callersMethodwait

(self)

tests/conftest.py:372

↓ 3 callersMethod_add_callbacks

(self, callbacks: Union[Callback, list[Callback]])

src/litserve/callbacks/base.py:70

↓ 3 callersMethod_handle_error_response

Raise HTTPException as is and rest as 500 after logging the error.

src/litserve/server.py:352

↓ 3 callersMethod_init_manager

(self, num_api_servers: int)

src/litserve/server.py:1113

↓ 3 callersMethod_prepare_app_run

(self, app: FastAPI)

src/litserve/server.py:1434

↓ 3 callersMethod_prepare_request

Common request preparation logic.

src/litserve/server.py:285

↓ 3 callersMethod_submit_request

Submit request to worker queue.

src/litserve/server.py:294

↓ 3 callersFunction_sync_fn_to_async_fn

(func, *args, **kwargs)

src/litserve/loops/base.py:51

↓ 3 callersMethodany_stream

(self)

src/litserve/server.py:265

↓ 3 callersMethodareceive

Receive a message from model workers or any publisher.

src/litserve/transport/base.py:12

↓ 3 callersFunctioncreate_transport_from_config

(config: TransportConfig)

src/litserve/transport/factory.py:33

↓ 3 callersMethoddevice_identifiers

(self, accelerator, device)

src/litserve/server.py:940

↓ 3 callersMethodgenerate_client_file

(port: Union[str, int] = 8000)

src/litserve/server.py:1091

↓ 3 callersMethodhas_active_requests

(self)

src/litserve/api.py:398

↓ 3 callersMethodlaunch_inference_worker

(self, lit_api: LitAPI)

src/litserve/server.py:831

↓ 3 callersMethodlist_tools

(self)

src/litserve/mcp.py:501

↓ 3 callersFunctionmain

()

src/litserve/__main__.py:22

↓ 3 callersMethodpredict

(self, x)

src/litserve/test_examples/simple_example.py:31

↓ 3 callersMethodsend

Send a message to a consumer in the main process.

src/litserve/transport/base.py:7

↓ 3 callersMethodsend

(self, response, consumer_id: int)

tests/unit/test_batch.py:204

↓ 3 callersMethodsetup

(self, device)

tests/unit/test_auth.py:59

↓ 3 callersMethodsetup

(self, device)

tests/e2e/default_batched_streaming.py:20

↓ 3 callersMethodshutdown

(self)

tests/conftest.py:392

↓ 3 callersMethodunbatch

(self, output)

tests/unit/test_batch.py:66

↓ 3 callersMethodwait_for_subscribers

Wait for at least one subscriber to be ready. Args: timeout: Maximum time to wait in seconds Returns: bool:

src/litserve/transport/zmq_queue.py:91

↓ 2 callersMethod__init__

(self, *args, **kwargs)

tests/unit/test_loops.py:83

↓ 2 callersMethod_choose_gpu_accelerator_backend

()

src/litserve/connector.py:84

↓ 2 callersMethod_encode_response

(self, output: Union[dict[str, str], list[dict[str, str]]])

src/litserve/specs/openai.py:460

↓ 2 callersFunction_get_default_handler

(stream, format)

src/litserve/utils.py:145

↓ 2 callersMethod_perform_graceful_shutdown

Encapsulates the graceful shutdown logic.

src/litserve/server.py:1130

↓ 2 callersMethod_process_single_request

( self, request, lit_api: LitAPI, transport: MessageTransport, callbac

src/litserve/loops/simple_loops.py:146

↓ 2 callersMethod_process_streaming_request

( self, request, lit_api: LitAPI, transport: MessageTransport, callbac

src/litserve/loops/streaming_loops.py:137

↓ 2 callersMethod_run_single_loop_with_async

( self, lit_api: LitAPI, request_queue: Queue, transport: MessageTransport,

src/litserve/loops/simple_loops.py:217

↓ 2 callersMethod_start_server

(self, port, num_uvicorn_servers, log_level, sockets, uvicorn_worker_type, **kwargs)

src/litserve/server.py:1444

↓ 2 callersMethodadd_request

Add a new sequence to active sequences and perform any action before prediction such as filling the cache.

src/litserve/loops/continuous_batching_loop.py:110

↓ 2 callersMethodadd_tool

(self, tool: ToolEndpointType)

src/litserve/mcp.py:497

↓ 2 callersMethodbatch

(self, inputs)

src/litserve/specs/openai.py:441

↓ 2 callersFunctionbenchmark

Benchmark the ML server.

tests/perf_test/bert/utils.py:53

↓ 2 callersFunctioncall_after_stream

(streamer: AsyncIterator, callback, *args, **kwargs)

src/litserve/utils.py:127

↓ 2 callersMethodconnect_mcp_server

LitServer calls this method to connect MCP server to the FastAPI app. Args: mcp_tools: List of MCP tools to connect to the MCP se

src/litserve/mcp.py:556

↓ 2 callersFunctioncreate_session

Create a session object with custom connection pool settings.

tests/perf_test/bert/utils.py:25

↓ 2 callersMethoddata_streamer

(q: deque, data_available: asyncio.Event, send_status: bool = False)

src/litserve/server.py:946

↓ 2 callersMethoddecode_request

Convert the request payload to your model input.

src/litserve/specs/base.py:62

↓ 2 callersMethodencode_response

Convert the model output to a response payload. To enable streaming, it should yield the output.

src/litserve/specs/base.py:67

↓ 2 callersMethodget_batch_requests

( self, lit_api: LitAPI, request_queue: Queue, transport: MessageTransport,

src/litserve/loops/base.py:320

↓ 2 callersFunctionget_default_loop

Get the default loop based on the stream flag, batch size, and async support. Args: stream: Whether streaming is enabled max_batc

src/litserve/loops/loops.py:29

↓ 2 callersMethodget_image

Decode the Base64 string and return a PIL Image object.

src/litserve/schema/image.py:26

↓ 2 callersMethodget_request

(self, request_queue: Queue, block: bool = True, timeout: Optional[float] = None)

src/litserve/loops/base.py:334

↓ 2 callersMethodhandle_request

(self, request, request_type)

src/litserve/server.py:312

↓ 2 callersMethodhas_capacity

Check if we can add more sequences based on current batch.

src/litserve/loops/continuous_batching_loop.py:128

next →1–100 of 984, ranked by callers