MCPcopy
hub / github.com/NanmiCoder/MediaCrawler / load_all_ip

Method load_all_ip

proxy/base_proxy.py:68–84  ·  view source on GitHub ↗

Load all unexpired IP information from Redis :param proxy_brand_name: Proxy provider name :return:

(self, proxy_brand_name: str)

Source from the content-addressed store, hash-verified

66 self.cache_client.set(key=ip_key, value=ip_value_info, expire_time=ex)
67
68 def load_all_ip(self, proxy_brand_name: str) -> List[IpInfoModel]:
69 """
70 Load all unexpired IP information from Redis
71 :param proxy_brand_name: Proxy provider name
72 :return:
73 """
74 all_ip_list: List[IpInfoModel] = []
75 all_ip_keys: List[str] = self.cache_client.keys(pattern=f"{proxy_brand_name}_*")
76 try:
77 for ip_key in all_ip_keys:
78 ip_value = self.cache_client.get(ip_key)
79 if not ip_value:
80 continue
81 all_ip_list.append(IpInfoModel(**json.loads(ip_value)))
82 except Exception as e:
83 utils.logger.error("[IpCache.load_all_ip] get ip err from redis db", e)
84 return all_ip_list

Callers 3

get_proxyMethod · 0.80
get_proxyMethod · 0.80
get_proxyMethod · 0.80

Calls 3

IpInfoModelClass · 0.85
keysMethod · 0.45
getMethod · 0.45

Tested by

no test coverage detected