Functions — github.com/deepseek-ai/smallpond

hub / github.com/deepseek-ai/smallpond / functions

Functions992 in github.com/deepseek-ai/smallpond

⨍ Functions 992◇ Types & classes 154↳ Endpoints 2

Method__str__

(self)

smallpond/execution/task.py:3006

Method__str__

(self)

smallpond/execution/task.py:3524

Method__str__

(self)

smallpond/execution/executor.py:114

Method__str__

(self)

smallpond/execution/scheduler.py:73

Method__str__

(self)

smallpond/execution/workqueue.py:212

Method__str__

(self)

smallpond/execution/workqueue.py:301

Method_call_process

( self, runtime_ctx: RuntimeContext, input_readers: List[arrow.RecordBatchReader] )

smallpond/execution/task.py:2466

Method_call_process

( self, runtime_ctx: RuntimeContext, input_tables: List[arrow.Table] )

smallpond/execution/task.py:2504

Method_column_str

A column string used in SQL select clause.

smallpond/logical/dataset.py:109

Method_column_str

(self)

smallpond/logical/dataset.py:597

Method_dump_periodically

Dump the graph and timeline every minute. Set `self._stop_event` to have a final dump and stop this thread.

smallpond/session.py:302

Method_final_output_root

(self)

smallpond/execution/task.py:751

Method_final_results

(self)

smallpond/execution/task.py:3539

Method_pop_unbuffered

(self, count: int)

smallpond/execution/workqueue.py:273

Method_pop_unbuffered

(self, count: int)

smallpond/execution/workqueue.py:336

Method_pristine_attrs

All attributes in __slots__.

smallpond/execution/task.py:664

Method_process_batches

( self, runtime_ctx: RuntimeContext, input_tables: List[arrow.Table] )

smallpond/execution/task.py:2522

Method_producer

(self)

smallpond/utility.py:152

Method_push_unbuffered

(self, item: WorkItem)

smallpond/execution/workqueue.py:279

Method_push_unbuffered

(self, item: WorkItem)

smallpond/execution/workqueue.py:365

Method_resolved_path_str

(self)

smallpond/logical/dataset.py:105

Method_runtime_output_root

(self)

smallpond/execution/task.py:759

Method_summarize_task

Return the total number of tasks and the number of tasks that are finished.

smallpond/dataframe.py:176

Method_write_partitioned_tables

(self, partitioned_tables)

smallpond/execution/task.py:2908

Methodabandoned_tasks

(self)

smallpond/execution/scheduler.py:513

Methodabsolute_paths

An ordered list of absolute paths of the given file patterns. Example:: >>> DataSet(['data/100.parquet', '/datasetA/*.parque

smallpond/logical/dataset.py:238

Methodalive

(self)

smallpond/execution/scheduler.py:131

Methodalive_executors

(self)

smallpond/execution/scheduler.py:540

Methodallocated_cpus

(self)

smallpond/execution/scheduler.py:151

Methodallocated_gpus

(self)

smallpond/execution/scheduler.py:155

Methodallocated_memory

(self)

smallpond/execution/scheduler.py:159

Methodallow_speculative_exec

Whether the task is allowed to be executed by speculative execution.

smallpond/execution/task.py:807

Methodallow_speculative_exec

(self)

smallpond/execution/task.py:3365

Methodanalyzed_logical_plan

(self)

smallpond/execution/task.py:3596

Methodany_input_empty

(self)

smallpond/execution/task.py:714

Methodavailable_cpus

(self)

smallpond/execution/scheduler.py:163

Methodavailable_gpu_quota

(self)

smallpond/execution/executor.py:122

Methodavailable_memory

(self)

smallpond/execution/task.py:271

Methodavailable_memory

(self)

smallpond/execution/scheduler.py:167

Methodbind

(self, conn: duckdb.DuckDBPyConnection)

smallpond/logical/udf.py:181

Methodbind

(self, conn: duckdb.DuckDBPyConnection)

smallpond/logical/udf.py:206

Methodbind

(self, conn: duckdb.DuckDBPyConnection)

smallpond/logical/udf.py:225

Methodbroken_iterator

(iter: Iterable[int], sleep_ms: int)

tests/test_utility.py:24

Methodbusy

(self)

smallpond/execution/executor.py:118

Methodbusy

(self)

smallpond/execution/scheduler.py:94

Functionchoose_random_urls

( runtime_ctx: RuntimeContext, input_tables: List[arrow.Table], k: int = 5 )

tests/test_execution.py:234

Methodclean_complex_attrs

(self)

smallpond/execution/task.py:1949

Methodclean_complex_attrs

(self)

smallpond/execution/task.py:2199

Methodclean_output

(self, force=False)

smallpond/execution/task.py:3385

Methodcleanup

Clean up the runtime directory. This will be called when the job is finished.

smallpond/execution/task.py:450

Methodcleanup

(self)

smallpond/execution/task.py:2620

Methodcleanup

(self)

smallpond/execution/task.py:2959

Methodcleanup_root

(self)

smallpond/execution/task.py:337

Methodcollect_file_sizes

(file_paths)

smallpond/execution/task.py:976

Methodcollect_nodes

(node: Node)

smallpond/logical/node.py:2128

Methodcompression_level_str

(self)

smallpond/execution/task.py:1236

Methodcompression_options

(self)

smallpond/execution/task.py:1245

Methodcompression_type_str

(self)

smallpond/execution/task.py:1228

Functionconsumer

(wq: WorkQueue, id: int)

tests/test_workqueue.py:42

Methodcopy_file

(src_path: Path, dst_path: Path)

smallpond/execution/task.py:3428

Functioncopy_input_batch

( runtime_ctx: RuntimeContext, input_batches: List[arrow.Table], msg: str )

tests/test_execution.py:149

Functioncopy_input_data_frame

( runtime_ctx: RuntimeContext, input_dfs: List[DataFrame] )

tests/test_execution.py:158

Functioncopy_input_data_frame_batch

( runtime_ctx: RuntimeContext, input_dfs: List[DataFrame] )

tests/test_execution.py:165

Methodcpu_limit

(self)

smallpond/execution/workqueue.py:82

Methodcreate

( engine_type: Literal["duckdb", "arrow"], *args, **kwargs )

smallpond/execution/task.py:2804

Methodcreate

( ctx: RuntimeContext, id: str, queue_dir: str, init_epoch=0 )

smallpond/execution/scheduler.py:82

Methodcreate_dataset

(row_ranges: List[RowRange])

smallpond/logical/dataset.py:823

Methodcreate_link_or_copy

(src_path: Path, dst_path: Path)

smallpond/execution/task.py:3434

Methodcreate_merge_task

( self, runtime_ctx: RuntimeContext, input_deps: List[Task], partition_infos:

smallpond/logical/node.py:1329

Methodcreate_producer_task

( self, runtime_ctx: RuntimeContext, input_deps: List[Task], partition_infos:

smallpond/logical/node.py:1376

Methodcreate_producer_task

( self, runtime_ctx: RuntimeContext, input_deps: List[Task], partition_infos:

smallpond/logical/node.py:1420

Methodcreate_producer_task

( self, runtime_ctx: RuntimeContext, input_deps: List[Task], partition_infos:

smallpond/logical/node.py:1516

Methodcreate_producer_task

( self, runtime_ctx: RuntimeContext, input_deps: List[Task], partition_infos:

smallpond/logical/node.py:1576

Methodcreate_producer_task

( self, runtime_ctx: RuntimeContext, input_deps: List[Task], partition_infos:

smallpond/logical/node.py:1773

Methodcreate_task

( self, runtime_ctx: RuntimeContext, input_deps: List[Task], partition_infos:

smallpond/logical/node.py:325

Methodcreate_task

( self, runtime_ctx: RuntimeContext, input_deps: List[Task], partition_infos:

smallpond/logical/node.py:391

Methodcreate_task

( self, runtime_ctx: RuntimeContext, input_deps: List[Task], partition_infos:

smallpond/logical/node.py:504

Methodcreate_task

( self, runtime_ctx: RuntimeContext, input_deps: List[Task], partition_infos:

smallpond/logical/node.py:662

Methodcreate_task

( self, runtime_ctx: RuntimeContext, input_deps: List[Task], partition_infos:

smallpond/logical/node.py:838

Methodcreate_task

( self, runtime_ctx: RuntimeContext, input_deps: List[Task], partition_infos:

smallpond/logical/node.py:1146

Methodcreate_task

( self, runtime_ctx: RuntimeContext, input_deps: List[Task], partition_infos:

smallpond/logical/node.py:1240

Methodcreate_task

( self, runtime_ctx: RuntimeContext, input_deps: List[Task], partition_infos:

smallpond/logical/node.py:1950

Methoddata_root

(self)

smallpond/execution/driver.py:312

Methoddefault_output_name

(self)

smallpond/execution/task.py:726

Methodelapsed_time

(self)

smallpond/execution/scheduler.py:447

Methodelapsed_time

(self)

smallpond/execution/workqueue.py:98

Methodemit

(self, record: logging.LogRecord)

smallpond/utility.py:183

Methodempty

Whether the dataset is empty.

smallpond/logical/dataset.py:170

Methodempty

(self)

smallpond/logical/dataset.py:687

Functionempty_file

( runtime_ctx: RuntimeContext, input_datasets: List[DataSet], output_path: str )

tests/test_execution.py:213

Methodestimated_data_size

Return the estimated data size in bytes.

smallpond/logical/dataset.py:694

Methodestimated_data_size

The estimated uncompressed data size in bytes.

smallpond/io/arrow.py:45

Methodexec_on_scheduler

(self)

smallpond/contrib/log_dataset.py:13

Methodexec_on_scheduler

(self)

smallpond/execution/task.py:1370

Methodexec_on_scheduler

(self)

smallpond/execution/task.py:1384

Methodexec_on_scheduler

(self)

smallpond/execution/task.py:1429

Methodexec_on_scheduler

(self)

smallpond/execution/task.py:1620

Methodexec_on_scheduler

(self)

smallpond/execution/task.py:1705

Methodexec_on_scheduler

(self)

smallpond/execution/task.py:1788

Methodexec_on_scheduler

(self)

smallpond/execution/task.py:3303

← previous next →501–600 of 992, ranked by callers