Skip to content

Conversation

@ARF1
Copy link
Owner

@ARF1 ARF1 commented Mar 21, 2015

Based on visualfabriq#27 but can be rebased on master.

This introduces the infrastructure for plug-in query transformers. Included are three sample query transformers:

  • InOperatorTransformer: my_col in ['ABC', 'DEF'] is transformed into (my_col == 'ABC') | (my_col == 'DEF'). The operation not in is similarly transformed.
  • TrivialBooleanExpressionsOptimizer: (my_col == 'ABC') | (False) is transformed into False (limited usefulness without an intelligent query optimizer)
  • CachedFactorOptimizer: converts comparisons containing columns with cached factors into comparisons using the factor instead. (Naive implementation, currently only useful for edge-cases.)

By default this PR does not change the behaviour or dependencies of bquery. Query transformers have to be explicitly enabled by configuring them, e.g.:

from transformers import InOperatorTransformer, TrivialBooleanExpressionsOptimizer
b.transformers = [InOperatorTransformer(), TrivialBooleanExpressionsOptimizer()]

For convenience, a shortcut is provided for these (currently) most useful transformers with transformers.standard_transformers:

from transformers import standard_transformers
b.transformers = standard_transformers

The overhead for queries is negligible for reasonably sized databases: For the query db["my_col=='AB1234567890'"] bquery without query transformers requires 362 ms, with all query transformers configured (including CachedFactorOptimizer) 367 ms.

With a non-compressed database the CachedFactorOptimizer shows some minor positive effects: 547 ms vs. 296 ms

@ARF1 ARF1 closed this Mar 21, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants