Functions — github.com/Lightning-AI/LitServe

hub / github.com/Lightning-AI/LitServe / functions

Functions984 in github.com/Lightning-AI/LitServe

⨍ Functions 984◇ Types & classes 226↳ Endpoints 30

↓ 2 callersFunctioninference_worker

( lit_api: LitAPI, device: str, worker_id: int, request_queue: Queue, transport: MessageTr

src/litserve/loops/loops.py:61

↓ 2 callersMethodmark_completed

Mark a request as completed and remove it from the tracked state.

src/litserve/loops/continuous_batching_loop.py:122

↓ 2 callersFunctionmean

(lst)

tests/parity_fastapi/main.py:77

↓ 2 callersFunctionpredict

()

tests/unit/test_litapi.py:169

↓ 2 callersMethodrun

( self, lit_api: LitAPI, device: str, worker_id: int, request_queue: Q

src/litserve/loops/base.py:288

↓ 2 callersMethodrun_batched_loop

( self, lit_api: LitAPI, request_queue: Queue, transport: MessageTransport,

src/litserve/loops/simple_loops.py:312

↓ 2 callersMethodrun_batched_streaming_loop

( self, lit_api: LitAPI, request_queue: Queue, transport: MessageTransport,

src/litserve/loops/streaming_loops.py:298

↓ 2 callersFunctionrun_bench

(conf: dict, num_samples: int, port: int)

tests/parity_fastapi/benchmark.py:100

↓ 2 callersFunctionrun_simple_request

(server, num_requests=1)

tests/unit/test_callbacks.py:17

↓ 2 callersMethodrun_single_loop

( self, lit_api: LitAPI, request_queue: Queue, transport: MessageTransport,

src/litserve/loops/simple_loops.py:33

↓ 2 callersMethodrun_streaming_loop

( self, lit_api: LitAPI, request_queue: Queue, transport: MessageTransport,

src/litserve/loops/streaming_loops.py:33

↓ 2 callersMethodrun_streaming_loop_async

( self, lit_api: LitAPI, request_queue: Queue, transport: MessageTransport,

src/litserve/loops/streaming_loops.py:206

↓ 2 callersFunctionset_trace_if_debug

Set a tracepoint in the code if the environment variable LITSERVE_DEBUG is set.

src/litserve/utils.py:245

↓ 2 callersMethodsetup

Setup the model so it can be called in `predict`.

src/litserve/api.py:245

↓ 2 callersMethodsetup

(self, device)

src/litserve/test_examples/simple_example.py:23

↓ 2 callersMethodsetup

Must be called in the subprocess to setup the ZMQ transport.

src/litserve/transport/zmq_transport.py:15

↓ 2 callersMethodsetup_auth

(self)

src/litserve/server.py:1494

↓ 2 callersFunctionshortuuid

()

src/litserve/specs/openai.py:42

↓ 2 callersMethodstop

Stop the broker.

src/litserve/transport/zmq_queue.py:76

↓ 2 callersFunctiontry_health

(port)

tests/parity_fastapi/main.py:53

↓ 2 callersMethodunbatch

(self, output)

src/litserve/specs/openai.py:444

↓ 2 callersMethodvalidate_chat_message

(self, obj)

src/litserve/specs/openai.py:457

↓ 1 callersMethod__add__

(self, other: "UsageInfo")

src/litserve/specs/openai.py:51

↓ 1 callersMethod__init__

(self)

src/litserve/loops/base.py:305

↓ 1 callersMethod__init__

(self, lit_api)

tests/unit/test_request_handlers.py:34

↓ 1 callersMethod__init__

(self, path="test_logger_temp.txt")

tests/unit/test_logger.py:95

↓ 1 callersMethod__reduce__

(self)

tests/unit/test_zmq_queue.py:84

↓ 1 callersMethod_accelerator_device_count

(self)

src/litserve/connector.py:78

↓ 1 callersFunction_call_handler

(handler, **kwargs)

src/litserve/mcp.py:248

↓ 1 callersMethod_choose_auto_accelerator

(self)

src/litserve/connector.py:72

↓ 1 callersMethod_connect

(self, lit_api: "LitAPI")

src/litserve/mcp.py:381

↓ 1 callersFunction_create_mp_transport

(config: TransportConfig)

src/litserve/transport/factory.py:28

↓ 1 callersFunction_create_zmq_transport

(config: TransportConfig)

src/litserve/transport/factory.py:20

↓ 1 callersMethod_detect_path_collision

(self)

src/litserve/server.py:240

↓ 1 callersMethod_handle_embedding_response

( self, embeddings: Union[list, "np.ndarray", "torch.Tensor", "list[list[float]]"], num_items: int = 1

src/litserve/specs/openai_embedding.py:206

↓ 1 callersMethod_is_picklable

(obj)

src/litserve/loggers.py:120

↓ 1 callersFunction_mixed_response_to_buffer

Handle both regular and streaming responses. Detect streaming responses by checking if the response is for streaming.

src/litserve/server.py:97

↓ 1 callersMethod_mount

(self, path: str, app: ASGIApp)

src/litserve/loggers.py:111

↓ 1 callersMethod_mount_with_fastapi

Mounts MCP server's Starlette app to the FastAPI app. Args: app: LitServer's FastAPI app to mount the MCP server to.

src/litserve/mcp.py:518

↓ 1 callersFunction_openai_format_error

(error: Exception)

src/litserve/specs/openai.py:334

↓ 1 callersMethod_parse_message

Parse a message received from ZMQ.

src/litserve/transport/zmq_queue.py:147

↓ 1 callersMethod_register_api_endpoints

src/litserve/server.py:1046

↓ 1 callersMethod_register_internal_endpoints

(self)

src/litserve/server.py:974

↓ 1 callersMethod_register_middleware

(self)

src/litserve/server.py:1082

↓ 1 callersMethod_register_spec_endpoints

(self, lit_api: LitAPI)

src/litserve/server.py:1070

↓ 1 callersMethod_setup_socket

Setup ZMQ socket - to be implemented by subclasses

src/litserve/transport/zmq_queue.py:143

↓ 1 callersMethod_validate_async_methods

Validate that async methods are properly implemented when enable_async is True.

src/litserve/api.py:195

↓ 1 callersMethod_validate_response

(self, response: Union[dict, list[Embedding], Any])

src/litserve/specs/openai_embedding.py:185

↓ 1 callersFunctionadd_log_handler

Allow users to add custom log handlers. Example usage: file_handler = logging.FileHandler('library_logs.log') add_log_handler(file_handle

src/litserve/utils.py:200

↓ 1 callersMethodadd_request

(self, uid: str, request)

tests/unit/test_loops.py:878

↓ 1 callersMethodas_async

(self)

src/litserve/specs/base.py:75

↓ 1 callersFunctionasync_fn

(*args, **kwargs)

src/litserve/loops/base.py:54

↓ 1 callersFunctionazip

(*async_iterables)

src/litserve/utils.py:80

↓ 1 callersMethodbatch

(self, inputs)

src/litserve/test_examples/openai_spec_example.py:60

↓ 1 callersMethodbatch

(self, inputs)

src/litserve/test_examples/openai_spec_example.py:134

↓ 1 callersFunctionbenchmark

Benchmark the ML server.

tests/parity_fastapi/benchmark.py:53

↓ 1 callersMethodcheck_devices_and_accelerators

Check if the devices are in a valid fomra and raise an error if they are not.

src/litserve/connector.py:39

↓ 1 callersMethodclose

Clean up resources asynchronously.

src/litserve/transport/zmq_queue.py:187

↓ 1 callersMethodcreate_logger

(self)

src/litserve/loggers.py:82

↓ 1 callersFunctioncreate_session

Create a session object with custom connection pool settings.

tests/parity_fastapi/benchmark.py:29

↓ 1 callersFunctioncreate_session

Create a session object with custom connection pool settings.

tests/perf_test/bert/benchmark.py:18

↓ 1 callersFunctioncreate_session

Create a session object with custom connection pool settings.

tests/perf_test/stream/stream_speed/benchmark.py:21

↓ 1 callersMethoddecode_request

(self, request)

tests/unit/test_loops.py:102

↓ 1 callersMethoddecode_request

(self, request)

tests/unit/test_litapi.py:53

↓ 1 callersMethoddecode_request

(self, request: Request)

tests/unit/test_auth.py:62

↓ 1 callersMethoddecode_request

(self, request: Request)

tests/unit/test_batch.py:79

↓ 1 callersMethoddecode_request

(self, request)

tests/e2e/default_batched_streaming.py:23

↓ 1 callersFunctiondockerize

Generate a Dockerfile for the given server code. Example usage: litserve dockerize server.py --port 8000 --gpu Args: server_

src/litserve/docker_builder.py:122

↓ 1 callersFunctiondummy_streamer

()

tests/unit/test_utils.py:34

↓ 1 callersMethodencode_response

(self, output_stream, context)

src/litserve/test_examples/openai_spec_example.py:97

↓ 1 callersMethodencode_response

(self, output)

src/litserve/test_examples/openai_spec_example.py:126

↓ 1 callersMethodencode_response

(self, output_stream_batch, context)

src/litserve/test_examples/openai_spec_example.py:141

↓ 1 callersMethodencode_response

(self, output)

tests/unit/test_loops.py:108

↓ 1 callersMethodencode_response

(self, output)

tests/unit/test_auth.py:68

↓ 1 callersMethodencode_response

(self, output)

tests/unit/test_batch.py:87

↓ 1 callersMethodencode_response

(self, output_stream)

tests/e2e/default_batched_streaming.py:30

↓ 1 callersFunctionextract_code_blocks

(lines: list[str])

tests/unit/test_readme.py:27

↓ 1 callersMethodextract_usage_info

(self, output: dict)

src/litserve/specs/openai.py:447

↓ 1 callersMethodform

(self)

tests/unit/test_request_handlers.py:60

↓ 1 callersFunctionget_average_throughput

(num_requests=100, num_samples=10)

tests/perf_test/bert/benchmark.py:35

↓ 1 callersFunctionget_code_blocks

(file: str)

tests/unit/test_readme.py:37

↓ 1 callersFunctionget_extra_time

(content: str)

tests/unit/test_readme.py:43

↓ 1 callersMethodget_from_queues

(self, uids)

src/litserve/specs/openai.py:490

↓ 1 callersMethodget_mcp_tools

(self)

src/litserve/server.py:272

↓ 1 callersMethodget_num_items

Return the number of sentences or tokens in the input.

src/litserve/specs/openai_embedding.py:48

↓ 1 callersMethodhandle_request

(self, request, request_type)

src/litserve/server.py:317

↓ 1 callersMethodhandle_request

(self, request, request_type)

tests/unit/test_request_handlers.py:69

↓ 1 callersFunctionkillall

()

tests/conftest.py:142

↓ 1 callersMethodlaunch_single_inference_worker

(self, lit_api: LitAPI, worker_id: int)

src/litserve/server.py:867

↓ 1 callersMethodload_cache

(self, x)

tests/unit/test_loops.py:739

↓ 1 callersFunctionmain

()

tests/minimal_run.py:22

↓ 1 callersFunctionmain

(batch_size: int, workers_per_device: int)

tests/parity_fastapi/ls-server.py:58

↓ 1 callersFunctionmain

()

tests/parity_fastapi/main.py:81

↓ 1 callersFunctionmain

()

tests/perf_test/bert/benchmark.py:60

↓ 1 callersFunctionmain

()

tests/perf_test/stream/stream_speed/benchmark.py:61

↓ 1 callersMethodnon_streaming_completion

(self, request: ChatCompletionRequest, generator_list: list[AsyncGenerator])

src/litserve/specs/openai.py:586

↓ 1 callersFunctionnotify_timed_out_requests

( response_queues: list[Queue], timed_out_uids: list[tuple[int, str]], )

src/litserve/loops/continuous_batching_loop.py:33

↓ 1 callersMethodon_schedule_task_done

(self, schedule_task: asyncio.Task)

src/litserve/loops/base.py:300

↓ 1 callersMethodpre_setup

(self)

src/litserve/server.py:247

↓ 1 callersMethodpre_setup

(self, lit_api: LitAPI, spec: Optional[LitSpec] = None)

src/litserve/loops/base.py:379

← previous next →101–200 of 984, ranked by callers