MCPcopy
hub / github.com/scrapy/scrapy / _get_process_spider_output

Method _get_process_spider_output

scrapy/core/spidermw.py:251–287  ·  view source on GitHub ↗
(mw: Any)

Source from the content-addressed store, hash-verified

249 # This method is only needed until _async compatibility methods are removed.
250 @staticmethod
251 def _get_process_spider_output(mw: Any) -> Callable[..., Any] | None:
252 normal_method: Callable[..., Any] | None = getattr(
253 mw, "process_spider_output", None
254 )
255 async_method: Callable[..., Any] | None = getattr(
256 mw, "process_spider_output_async", None
257 )
258 if not async_method:
259 if normal_method and not isasyncgenfunction(normal_method):
260 raise TypeError(
261 f"Middleware {global_object_name(mw.__class__)} doesn't support"
262 f" asynchronous spider output. Its process_spider_output() method"
263 f" should be an async generator function or it should additionally"
264 f" define a process_spider_output_async() method."
265 )
266 return normal_method
267 if not normal_method:
268 logger.error(
269 f"Middleware {global_object_name(mw.__class__)} has"
270 f" process_spider_output_async() without process_spider_output(),"
271 f" skipping this method. Please rename it to process_spider_output()."
272 )
273 return None
274 if not isasyncgenfunction(async_method):
275 logger.error(
276 f"{global_object_name(async_method)} is not "
277 f"an async generator function, skipping this method."
278 )
279 return normal_method
280 if isasyncgenfunction(normal_method):
281 logger.error(
282 f"{global_object_name(normal_method)} is an async "
283 f"generator function while process_spider_output_async() exists, "
284 f"skipping both methods. Please remove process_spider_output_async()."
285 )
286 return None
287 return async_method

Callers 1

_add_middlewareMethod · 0.95

Calls 2

global_object_nameFunction · 0.90
errorMethod · 0.80

Tested by

no test coverage detected