MCPcopy Create free account
hub / github.com/NVIDIA/TensorRT-LLM / _wait_for_response

Method _wait_for_response

tensorrt_llm/executor/rpc/rpc_client.py:318–336  ·  view source on GitHub ↗

Wait for a response from the socket. Returns: RPCResponse from the server

(self)

Source from the content-addressed store, hash-verified

316 self._broadcast_streaming_error(exception)
317
318 async def _wait_for_response(self) -> RPCResponse:
319 """Wait for a response from the socket.
320
321 Returns:
322 RPCResponse from the server
323 """
324 # Use timeout-based recv to handle cancellation gracefully
325 # This prevents the CancelledError from being logged as an exception
326 while True:
327 try:
328 # Short timeout allows periodic checks for cancellation
329 return await self._client_socket.get_async_noblock(timeout=2)
330 except asyncio.TimeoutError:
331 # Check if we should exit due to cancellation
332 if self._closed or (self._reader_asyncio_task
333 and self._reader_asyncio_task.cancelled()):
334 raise asyncio.CancelledError("Reader task cancelled")
335 # Otherwise continue polling
336 continue
337
338 async def _response_reader(self):
339 """Task to read responses from the socket and set results on futures."""

Callers 1

_response_readerMethod · 0.95

Calls 1

get_async_noblockMethod · 0.80

Tested by

no test coverage detected