Skip to content

show consumer groups in Topics view#3264

Draft
Dave Shoup (shouples) wants to merge 29 commits intomainfrom
djs/consumer-groups-in-view
Draft

show consumer groups in Topics view#3264
Dave Shoup (shouples) wants to merge 29 commits intomainfrom
djs/consumer-groups-in-view

Conversation

@shouples
Copy link
Contributor

@shouples Dave Shoup (shouples) commented Feb 7, 2026

Summary of Changes

Adds consumer groups as a visible resource in the Topics view (pending rename in a future branch):
image
image

The Consumer Groups container is shown at the top in a collapsed state, while the (also new) Topics container is auto-expanded to be as minimally disruptive as possible for users.

Closes #3300

(Still more work to be done as part of the #3227 parent issue)

Associated PRs

Click-testing instructions

  1. Connect to a Confluent Cloud environment with an active Kafka cluster
  2. Expand the Topics view for that cluster
  3. Verify a "Consumer Groups" container appears alongside the "Topics" container
  4. Expand the Consumer Groups container and confirm groups are listed with their state (if consumers have been active recently)
  5. Expand a consumer group to verify its members (consumers) are shown

Optional: Any additional details or context that should be provided?

  • The downstream commands branch (placeholder in the stack) will add context menu actions for consumer groups (e.g. per-container refreshing, copying IDs)
  • Consumer group members are fetched per-group using Promise.allSettled, so one failed member fetch won't block the rest

Pull request checklist

Please check if your PR fulfills the following (if applicable):

Tests

  • Added new
  • Updated existing
  • Deleted existing

Release notes

  • Does anything in this PR need to be mentioned in the user-facing CHANGELOG?

@airlock-confluentinc airlock-confluentinc bot force-pushed the djs/consumer-group-models branch 6 times, most recently from d5c9efc to 9b4450d Compare February 24, 2026 16:15
@shouples Dave Shoup (shouples) changed the base branch from djs/consumer-group-models to djs/resource-container-refactor February 24, 2026 16:20
@airlock-confluentinc airlock-confluentinc bot force-pushed the djs/resource-container-refactor branch from 4384847 to 2d0df9f Compare February 24, 2026 22:01
@airlock-confluentinc airlock-confluentinc bot force-pushed the djs/consumer-groups-in-view branch from 099e61c to 8246ea3 Compare February 24, 2026 22:01
@airlock-confluentinc airlock-confluentinc bot force-pushed the djs/resource-container-refactor branch from 2d0df9f to 2daefec Compare February 26, 2026 16:49
@airlock-confluentinc airlock-confluentinc bot force-pushed the djs/consumer-groups-in-view branch 2 times, most recently from 25f8ed1 to 0b24f3d Compare February 27, 2026 14:41
@airlock-confluentinc airlock-confluentinc bot force-pushed the djs/resource-container-refactor branch 2 times, most recently from 6b8d9b8 to ec16f3d Compare February 27, 2026 14:48
@airlock-confluentinc airlock-confluentinc bot force-pushed the djs/consumer-groups-in-view branch 3 times, most recently from 3b61491 to a585b8b Compare February 27, 2026 22:39
@sonarqube-confluent
Copy link

Quality Gate failed Quality Gate failed

Failed conditions
76.9% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube

Base automatically changed from djs/resource-container-refactor to main March 10, 2026 20:34
Copilot AI review requested due to automatic review settings March 10, 2026 20:38
@airlock-confluentinc airlock-confluentinc bot force-pushed the djs/consumer-groups-in-view branch from 9b6234e to 27c744f Compare March 10, 2026 20:38
@shouples Dave Shoup (shouples) review requested due to automatic review settings March 10, 2026 20:38
Copilot AI review requested due to automatic review settings March 12, 2026 12:52
@shouples Dave Shoup (shouples) review requested due to automatic review settings March 12, 2026 12:52
Copilot AI review requested due to automatic review settings March 12, 2026 15:34
@shouples Dave Shoup (shouples) review requested due to automatic review settings March 12, 2026 15:34
Comment on lines -1849 to -1853
{
"command": "confluent.topics.create",
"when": "view == confluent-topics && confluent.kafkaClusterSelected",
"group": "navigation@2"
},
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved from the view/title section to view/item/context section since this is now shown inline on the "Topics" container item

*
* Raises an error if the cluster ID of any consumer group does not match the given cluster's ID.
*/
async setConsumerGroupsForCluster(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really tempted to abstract this away into another variation of [set|get]KafkaClusterResource<T> like here:

async setFlinkDatabaseResources<T extends { databaseId: string }>(

@shouples Dave Shoup (shouples) marked this pull request as ready for review March 13, 2026 01:22
@shouples Dave Shoup (shouples) requested a review from a team as a code owner March 13, 2026 01:22
Copilot AI review requested due to automatic review settings March 13, 2026 01:22
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds Kafka consumer groups (and their member consumers) as a first-class, visible resource in the Topics view, backed by new loader + storage caching and updated tests to reflect the new containerized tree structure.

Changes:

  • Introduces root-level “Consumer Groups” and “Topics” containers in the Topics view provider and wires up tree item rendering/parenting for consumer groups and consumers.
  • Adds loader + ResourceManager support for fetching/caching consumer groups (including member hydration) and a small URL-parsing helper (parseCoordinatorId).
  • Updates unit/E2E tests, command placement (Create Topic inline action), and CHANGELOG entry.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
tests/e2e/objects/views/TopicsView.ts Updates E2E view object to use the new Topics container and updated tree levels.
src/viewProviders/topics.ts Adds containerized tree structure and consumer group/member support in the Topics view.
src/viewProviders/topics.test.ts Expands unit coverage for new containers, consumer group behavior, and targeted tree refresh events.
src/storage/resourceManager.ts Adds workspace-state caching for consumer groups with rehydration of groups and members.
src/storage/resourceManager.test.ts Adds tests for consumer group cache set/get and rehydration behavior.
src/loaders/utils/loaderUtils.ts Adds parseCoordinatorId() helper.
src/loaders/utils/loaderUtils.test.ts Adds tests for consumer group/member error propagation and coordinator ID parsing.
src/loaders/resourceLoader.ts Extends the ResourceLoader API with getConsumerGroupsForCluster().
src/loaders/resourceLoader.test.ts Adds tests for consumer group loading/caching behavior.
src/loaders/cachingResourceLoader.ts Implements getConsumerGroupsForCluster() with caching + per-group member fetch.
package.json Moves “Create Topic” from view title navigation to an inline action on the Topics container.
CHANGELOG.md Documents the new consumer group visibility in the Topics view.

You can also share your feedback on Copilot code review. Take the survey.

Comment on lines +243 to 250
// clear stale entries before repopulating
this.topicsInTreeView.clear();
this.subjectsInTreeView.clear();
this.subjectToTopicMap.clear();

const loader = ResourceLoader.getInstance(cluster.connectionId);
try {
const topics = await loader.getTopicsForCluster(cluster, forceDeepRefresh);
Comment on lines +290 to +295
// clear stale entries before repopulating
this.consumerGroupsInTreeView.clear();

const loader = ResourceLoader.getInstance(cluster.connectionId);
try {
const consumerGroups = await loader.getConsumerGroupsForCluster(cluster, forceDeepRefresh);
Comment on lines 262 to 273
this.logger.error("Error fetching topics for cluster", cluster, err);
const message = err instanceof Error ? err.message : String(err);
this.topicsContainer.setError(
new CustomMarkdownString()
.addWarning(`Failed to load topics for **${cluster.name}**:`)
.addCodeBlock(message),
);
if (err instanceof TopicFetchError) {
window.showErrorMessage(
`Failed to list topics for cluster "${cluster.name}": ${err.message}`,
);
}
new CustomMarkdownString()
.addWarning(`Failed to load consumer groups for **${cluster.name}**:`)
.addCodeBlock(message),
);
Comment on lines +289 to +295
// Convert API response to ConsumerGroup models, fetching members for each group.
const consumerGroups: ConsumerGroup[] = await Promise.all(
responseConsumerGroups.map(async (data) => {
let members: Consumer[] = [];
try {
const memberData = await fetchConsumerGroupMembers(cluster, data.consumer_group_id);
members = memberData.map(
Comment on lines +318 to +319
state: data.state as ConsumerGroupState,
isSimple: data.is_simple,
/** Get all topic items in the view (nested under the Topics container). */
get topics(): Locator {
return this.body.locator("[role='treeitem'][aria-level='1']");
return this.body.locator("[role='treeitem'][aria-level='2']");
@sonarqube-confluent
Copy link

if (!related) return null;
const lastSegment = related.split("/").pop();
if (!lastSegment) return null;
const parsed = parseInt(lastSegment, 10);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess the SQ feedback makes sense here since we're already using Number anyhow

@shouples Dave Shoup (shouples) marked this pull request as draft March 14, 2026 00:24
@shouples Dave Shoup (shouples) marked this pull request as draft March 14, 2026 00:24
@shouples
Copy link
Contributor Author

Back to draft mode to address E2E test issues

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Show Consumer Groups container through Topics view provider

3 participants