API: merge_duckdb¶
Merge all pixel-level parquet tables for one flightline.
Parameters¶
flightline_dir : Path
Directory containing the flightline's parquet outputs.
out_name : str, optional
Custom name for the merged parquet. If None, defaults to:
None which allows the Ray helper to choose the configured default.
merge_memory_limit_gb : float or str, optional
Upper bound on DuckDB's memory usage. Floats are interpreted as GiB.
Provide None to leave the default DuckDB behaviour unchanged.
merge_threads : int, optional
Number of threads DuckDB should use for the merge. None keeps the
engine default (usually os.cpu_count()).
merge_row_group_size : int or None, optional
Target number of rows per Parquet row group in the merged output.
If None, DuckDB will auto-determine the optimal size (better streaming performance).
merge_temp_directory : Path, optional
Directory where DuckDB should spill temporary data. Defaults to a
.duckdb_tmp subdirectory inside flightline_dir.
Returns¶
Path Path to the merged parquet file.