Skip to content

Expose SessionContext.add_optimizer_rule for Python-defined logical optimizer rules #1574

Description

@timsaucer

Background

SessionContext::add_optimizer_rule registers an Arc<dyn OptimizerRule> that runs during logical plan optimization. The Python bindings only expose remove_optimizer_rule, never the additive side. PR #1557 ultimately landed add_physical_optimizer_rule for the physical pipeline via FFI but did not address the logical pipeline.

Upstream signature

pub fn add_optimizer_rule(&self, optimizer_rule: Arc<dyn OptimizerRule + Send + Sync>)

User value

Lets users add domain-specific logical rewrites (predicate normalization, redundant join elimination, scan pruning informed by external metadata) without forking DataFusion. Complements the existing remove_optimizer_rule to round out the surface.

Why deferred

Blocked upstream. As documented in PR #1557, DataFusion does not currently expose an FFI bridge for the logical OptimizerRule / AnalyzerRule traits, and there are no Python constructors for LogicalPlan node variants -- a pure-Python rule could observe plans but not transform them. The PR #1557 commit history shows an initial attempt at a Python-defined logical rule that was abandoned for this reason. This is filed for tracking; it should be revisited once upstream lands an FFI_OptimizerRule (mirroring FFI_PhysicalOptimizerRule).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions