Feature | Benefit |
---|---|
Lazy Execution with IR Compilation | Delays execution until results are needed, enabling cross-operation optimizations |
Projection/Predicate Pushdown | Reduces data processing by selecting only required columns and applying early filters |
Multi-threaded Apache Arrow Backend | Accelerates operations (groupby, join, sort) through parallel processing |
Memory-Optimized Data Types | Uses efficient dtypes (int16, bool[pyarrow]) and Arrow format for reduced memory consumption |
Query Plan Optimization | Reorders operations automatically (e.g., column selection before sorting) for optimal execution |
Batch Processing | Processes accumulated operati |