Build a PyArrow array from the passed buffer. Parameters ---------- buffer : ColumnBuffers Dictionary containing tuples of underlying buffers and their associated dtype. data_type : Tuple[DtypeKind, int, str, str], Dtype description of the column as a tu
(
buffers: ColumnBuffers,
data_type: Tuple[DtypeKind, int, str, str],
length: int,
describe_null: ColumnNullType,
offset: int = 0,
allow_copy: bool = True,
)
| 318 | |
| 319 | |
| 320 | def buffers_to_array( |
| 321 | buffers: ColumnBuffers, |
| 322 | data_type: Tuple[DtypeKind, int, str, str], |
| 323 | length: int, |
| 324 | describe_null: ColumnNullType, |
| 325 | offset: int = 0, |
| 326 | allow_copy: bool = True, |
| 327 | ) -> pa.Array: |
| 328 | """ |
| 329 | Build a PyArrow array from the passed buffer. |
| 330 | |
| 331 | Parameters |
| 332 | ---------- |
| 333 | buffer : ColumnBuffers |
| 334 | Dictionary containing tuples of underlying buffers and |
| 335 | their associated dtype. |
| 336 | data_type : Tuple[DtypeKind, int, str, str], |
| 337 | Dtype description of the column as a tuple ``(kind, bit-width, format string, |
| 338 | endianness)``. |
| 339 | length : int |
| 340 | The number of values in the array. |
| 341 | describe_null: ColumnNullType |
| 342 | Null representation the column dtype uses, |
| 343 | as a tuple ``(kind, value)`` |
| 344 | offset : int, default: 0 |
| 345 | Number of elements to offset from the start of the buffer. |
| 346 | allow_copy : bool, default: True |
| 347 | Whether to allow copying the memory to perform the conversion |
| 348 | (if false then zero-copy approach is requested). |
| 349 | |
| 350 | Returns |
| 351 | ------- |
| 352 | pa.Array |
| 353 | |
| 354 | Notes |
| 355 | ----- |
| 356 | The returned array doesn't own the memory. The caller of this function |
| 357 | is responsible for keeping the memory owner object alive as long as |
| 358 | the returned PyArrow array is being used. |
| 359 | """ |
| 360 | data_buff, _ = buffers["data"] |
| 361 | try: |
| 362 | validity_buff, validity_dtype = buffers["validity"] |
| 363 | except TypeError: |
| 364 | validity_buff = None |
| 365 | try: |
| 366 | offset_buff, offset_dtype = buffers["offsets"] |
| 367 | except TypeError: |
| 368 | offset_buff = None |
| 369 | |
| 370 | # Construct a pyarrow Buffer |
| 371 | data_pa_buffer = pa.foreign_buffer(data_buff.ptr, data_buff.bufsize, |
| 372 | base=data_buff) |
| 373 | |
| 374 | # Construct a validity pyarrow Buffer, if applicable |
| 375 | if validity_buff: |
| 376 | validity_pa_buff = validity_buffer_from_mask(validity_buff, |
| 377 | validity_dtype, |
no test coverage detected