Enable system database by default for queries

[aalexfvk]

Enable system database by default for some select queries (listing databases, tables, partitions. parts, etc)

Summary by Sourcery

Enable system database visibility in admin listing commands while adding flexible database exclusion filters.

New Features:

• Add an --exclude-database option to partition and part group CLI commands to filter out databases by name pattern or list.

Enhancements:

• Allow system database data to be included by default in parts, detached parts, part log, partitions, databases, and tables listings while still excluding information_schema.
• Propagate an exclude_database_pattern parameter through internal listing functions for parts, detached parts, part log, and partitions to support pattern-based database filtering.
• Extend database exclusion in list_databases to accept pattern-based matching instead of a single exact database name.

[sourcery-ai]

Reviewer's Guide

Enables inclusion of ClickHouse system databases by default in various listing commands while adding a flexible pattern-based way to exclude databases, wiring the new filter through internal query builders and CLI options for parts, detached parts, partitions, and databases/tables.

Sequence diagram for CLI partition listing with exclude_database_pattern

sequenceDiagram
  actor User
  participant CLI_partition_group
  participant CLI_get_partitions
  participant Internal_part as Internal_part_module
  participant ClickHouse as ClickHouse_server

  User->>CLI_partition_group: chadmin partition-group --exclude-database pattern
  CLI_partition_group->>CLI_get_partitions: get_partitions(exclude_database_pattern=pattern)

  CLI_get_partitions->>CLI_get_partitions: Render SQL template
  CLI_get_partitions->>Internal_part: list_parts(ctx, database=None, exclude_database_pattern=pattern, ...)

  Internal_part->>Internal_part: Build query with
  Internal_part->>Internal_part: WHERE database NOT IN (information_schema, INFORMATION_SCHEMA)
  Internal_part->>Internal_part: AND database NOT format_str_match(pattern)

  Internal_part->>ClickHouse: Execute filtered listing query
  ClickHouse-->>Internal_part: Result rows
  Internal_part-->>CLI_get_partitions: Filtered parts list
  CLI_get_partitions-->>CLI_partition_group: Aggregated partition groups
  CLI_partition_group-->>User: Display partitions excluding pattern-matched databases

Loading

File-Level Changes

Change	Details	Files
Allow pattern-based exclusion of databases for parts, detached parts, and part log listings while no longer hard-excluding the system database.	Add optional exclude_database_pattern parameter to list_parts, list_detached_parts, and list_part_log functions. Adjust SQL templates to only exclude information_schema/INFORMATION_SCHEMA by default and conditionally append a NOT match on exclude_database_pattern when provided. Propagate exclude_database_pattern into query execution calls so it is available in the templating context.	ch_tools/chadmin/internal/part.py
Expose new exclude-database CLI option for partition- and part-group commands and feed it into partition discovery logic.	Add --exclude-database/ exclude_database_pattern option to partition_group and part_group Click CLI commands with descriptive help text. Extend get_partitions signature with exclude_database_pattern and propagate it down to the query rendering call. Update partition listing queries to stop hard-excluding system while still excluding information_schema and to apply exclude_database_pattern when set.	ch_tools/chadmin/cli/partition_group.py ch_tools/chadmin/cli/part_group.py
Make database exclusion in list_databases pattern-based and stop excluding the system database by default.	Change default database filter to only exclude information_schema/INFORMATION_SCHEMA instead of system as well. Switch exclude_database handling from a single equality check to using format_str_match so patterns and lists are supported.	ch_tools/chadmin/internal/database.py
Stop hard-excluding the system database in table listings while retaining existing pattern-based exclusion support.	Update list_tables query to only exclude information_schema/INFORMATION_SCHEMA when no explicit database filters are provided. Keep existing exclude_database_pattern support that adds a NOT match clause on t.database.	ch_tools/chadmin/internal/table.py

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[sourcery-ai]

Hey - I've found 1 issue, and left some high level feedback:

• In internal/database.list_databases the exclude_database argument now goes through format_str_match, effectively changing it from an exact match to a pattern match; if this is intentional, consider renaming it to exclude_database_pattern (and/or updating any user-facing help) to avoid confusing callers that expect exact semantics.
• The new exclude_database_pattern option is added to several CLIs and internals; it would be good to double-check that its behavior is clearly differentiated from existing database/database_pattern filters (e.g., how they combine) and that help strings consistently describe that it accepts LIKE-style patterns and comma-separated lists.

Prompt for AI Agents

Please address the comments from this code review:

## Overall Comments
- In `internal/database.list_databases` the `exclude_database` argument now goes through `format_str_match`, effectively changing it from an exact match to a pattern match; if this is intentional, consider renaming it to `exclude_database_pattern` (and/or updating any user-facing help) to avoid confusing callers that expect exact semantics.
- The new `exclude_database_pattern` option is added to several CLIs and internals; it would be good to double-check that its behavior is clearly differentiated from existing `database`/`database_pattern` filters (e.g., how they combine) and that help strings consistently describe that it accepts LIKE-style patterns and comma-separated lists.

## Individual Comments

### Comment 1
<location> `ch_tools/chadmin/internal/database.py:69` </location>
<code_context>
-        WHERE database NOT IN ('system', 'information_schema', 'INFORMATION_SCHEMA')
+        WHERE database NOT IN ('information_schema', 'INFORMATION_SCHEMA')
         {% endif %}
         {% if exclude_database %}
-          AND database != '{{ exclude_database }}'
+          AND database NOT {{ format_str_match(exclude_database) }}
</code_context>

<issue_to_address>
**issue (bug_risk):** Changing `exclude_database` from a strict inequality to a pattern-based `NOT` may widen the exclusion scope.

Previously this was an exact inequality:

```sql
AND database != '{{ exclude_database }}'
```
Now it uses pattern-based matching:

```sql
AND database NOT {{ format_str_match(exclude_database) }}
```
If `format_str_match` allows wildcards or comma-separated lists, callers expecting a single exact DB exclusion might now unintentionally exclude additional databases (e.g., `foo` also matching `foo_backup`). Please confirm that all current callers either expect this broader behavior or are updated accordingly, especially if the goal is to align with `exclude_database_pattern` semantics elsewhere.
</issue_to_address>

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}