If the filepath_or_buffer is a url, translate and return the buffer. Otherwise passthrough. Parameters ---------- filepath_or_buffer : a url, filepath (str or pathlib.Path), or buffer compression : str or dict, default 'infer' For on-the-fl
(
filepath_or_buffer: FilePath | BaseBuffer,
encoding: str = "utf-8",
compression: CompressionOptions | None = None,
mode: str = "r",
storage_options: StorageOptions | None = None,
)
| 295 | |
| 296 | |
| 297 | def _get_filepath_or_buffer( |
| 298 | filepath_or_buffer: FilePath | BaseBuffer, |
| 299 | encoding: str = "utf-8", |
| 300 | compression: CompressionOptions | None = None, |
| 301 | mode: str = "r", |
| 302 | storage_options: StorageOptions | None = None, |
| 303 | ) -> IOArgs: |
| 304 | """ |
| 305 | If the filepath_or_buffer is a url, translate and return the buffer. |
| 306 | Otherwise passthrough. |
| 307 | |
| 308 | Parameters |
| 309 | ---------- |
| 310 | filepath_or_buffer : a url, filepath (str or pathlib.Path), |
| 311 | or buffer |
| 312 | |
| 313 | compression : str or dict, default 'infer' |
| 314 | For on-the-fly compression of the output data. If 'infer' and |
| 315 | 'filepath_or_buffer' is path-like, then detect compression from the |
| 316 | following extensions: '.gz', |
| 317 | '.bz2', '.zip', '.xz', '.zst', '.tar', '.tar.gz', '.tar.xz' or '.tar.bz2' |
| 318 | (otherwise no compression). |
| 319 | Set to ``None`` for no compression. |
| 320 | Can also be a dict with key ``'method'`` set |
| 321 | to one of {``'zip'``, ``'gzip'``, ``'bz2'``, ``'zstd'``, ``'xz'``, ``'tar'``} |
| 322 | and other key-value pairs are forwarded to |
| 323 | ``zipfile.ZipFile``, ``gzip.GzipFile``, |
| 324 | ``bz2.BZ2File``, ``zstandard.ZstdCompressor``, ``lzma.LZMAFile`` or |
| 325 | ``tarfile.TarFile``, respectively. |
| 326 | As an example, the following could be passed for faster compression and to |
| 327 | create a reproducible gzip archive: |
| 328 | ``compression={'method': 'gzip', 'compresslevel': 1, 'mtime': 1}``. |
| 329 | |
| 330 | encoding : the encoding to use to decode bytes, default is 'utf-8' |
| 331 | mode : str, optional |
| 332 | |
| 333 | storage_options : dict, optional |
| 334 | Extra options that make sense for a particular storage connection, e.g. |
| 335 | host, port, username, password, etc. For HTTP(S) URLs the key-value pairs |
| 336 | are forwarded to ``urllib.request.Request`` as header options. For other |
| 337 | URLs (e.g. starting with "s3://", and "gcs://") the key-value pairs are |
| 338 | forwarded to ``fsspec.open``. Please see ``fsspec`` and ``urllib`` for more |
| 339 | details, and for more examples on storage options refer `here |
| 340 | <https://pandas.pydata.org/docs/user_guide/io.html? |
| 341 | highlight=storage_options#reading-writing-remote-files>`_. |
| 342 | |
| 343 | |
| 344 | Returns the dataclass IOArgs. |
| 345 | """ |
| 346 | filepath_or_buffer = stringify_path(filepath_or_buffer) |
| 347 | |
| 348 | # handle compression dict |
| 349 | compression_method, compression = get_compression_method(compression) |
| 350 | compression_method = infer_compression(filepath_or_buffer, compression_method) |
| 351 | |
| 352 | # GH21227 internal compression is not used for non-binary handles. |
| 353 | if compression_method and hasattr(filepath_or_buffer, "write") and "b" not in mode: |
| 354 | warnings.warn( |
no test coverage detected