MCPcopy Create free account
hub / github.com/NVIDIA/TensorRT-LLM / process_res

Method process_res

tensorrt_llm/executor/rpc_proxy_mixin.py:105–125  ·  view source on GitHub ↗
(res: list)

Source from the content-addressed store, hash-verified

103 event_loop = None
104
105 def process_res(res: list):
106 for r in res:
107 client_id = r.client_id
108 nonlocal event_loop
109 nonlocal async_queues
110
111 if client_id not in self._results:
112 logger.warning(f"Received response for unknown client_id: {client_id}")
113 continue
114
115 queue = self._results[client_id].queue
116 if isinstance(queue, _SyncQueue):
117 queue.put_nowait(r)
118 async_queues.append(queue)
119 # all the loops are identical
120 event_loop = event_loop or queue.loop
121 else:
122 queue.put(r)
123
124 if (is_llm_response(r) and r.result.is_final) or isinstance(r, ErrorResponse):
125 self._results.pop(client_id)
126
127 # Handle the case where responses might not be a list of lists
128 if responses and not isinstance(responses[0], list):

Callers

nothing calls this directly

Calls 6

is_llm_responseFunction · 0.85
popMethod · 0.80
warningMethod · 0.45
put_nowaitMethod · 0.45
appendMethod · 0.45
putMethod · 0.45

Tested by

no test coverage detected