show consumer groups in Topics view#3264
show consumer groups in Topics view#3264Dave Shoup (shouples) wants to merge 29 commits intomainfrom
Conversation
d5c9efc to
9b4450d
Compare
4384847 to
2d0df9f
Compare
099e61c to
8246ea3
Compare
2d0df9f to
2daefec
Compare
25f8ed1 to
0b24f3d
Compare
6b8d9b8 to
ec16f3d
Compare
3b61491 to
a585b8b
Compare
|
…r in CachingResourceLoader
…iew provider; update tests
…nsion doesn't support consumer group management
9b6234e to
27c744f
Compare
…ld group state; add tests
| { | ||
| "command": "confluent.topics.create", | ||
| "when": "view == confluent-topics && confluent.kafkaClusterSelected", | ||
| "group": "navigation@2" | ||
| }, |
There was a problem hiding this comment.
Moved from the view/title section to view/item/context section since this is now shown inline on the "Topics" container item
| * | ||
| * Raises an error if the cluster ID of any consumer group does not match the given cluster's ID. | ||
| */ | ||
| async setConsumerGroupsForCluster( |
There was a problem hiding this comment.
Really tempted to abstract this away into another variation of [set|get]KafkaClusterResource<T> like here:
vscode/src/storage/resourceManager.ts
Line 721 in 066658e
There was a problem hiding this comment.
Pull request overview
Adds Kafka consumer groups (and their member consumers) as a first-class, visible resource in the Topics view, backed by new loader + storage caching and updated tests to reflect the new containerized tree structure.
Changes:
- Introduces root-level “Consumer Groups” and “Topics” containers in the Topics view provider and wires up tree item rendering/parenting for consumer groups and consumers.
- Adds loader + ResourceManager support for fetching/caching consumer groups (including member hydration) and a small URL-parsing helper (
parseCoordinatorId). - Updates unit/E2E tests, command placement (Create Topic inline action), and CHANGELOG entry.
Reviewed changes
Copilot reviewed 12 out of 12 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/e2e/objects/views/TopicsView.ts | Updates E2E view object to use the new Topics container and updated tree levels. |
| src/viewProviders/topics.ts | Adds containerized tree structure and consumer group/member support in the Topics view. |
| src/viewProviders/topics.test.ts | Expands unit coverage for new containers, consumer group behavior, and targeted tree refresh events. |
| src/storage/resourceManager.ts | Adds workspace-state caching for consumer groups with rehydration of groups and members. |
| src/storage/resourceManager.test.ts | Adds tests for consumer group cache set/get and rehydration behavior. |
| src/loaders/utils/loaderUtils.ts | Adds parseCoordinatorId() helper. |
| src/loaders/utils/loaderUtils.test.ts | Adds tests for consumer group/member error propagation and coordinator ID parsing. |
| src/loaders/resourceLoader.ts | Extends the ResourceLoader API with getConsumerGroupsForCluster(). |
| src/loaders/resourceLoader.test.ts | Adds tests for consumer group loading/caching behavior. |
| src/loaders/cachingResourceLoader.ts | Implements getConsumerGroupsForCluster() with caching + per-group member fetch. |
| package.json | Moves “Create Topic” from view title navigation to an inline action on the Topics container. |
| CHANGELOG.md | Documents the new consumer group visibility in the Topics view. |
You can also share your feedback on Copilot code review. Take the survey.
| // clear stale entries before repopulating | ||
| this.topicsInTreeView.clear(); | ||
| this.subjectsInTreeView.clear(); | ||
| this.subjectToTopicMap.clear(); | ||
|
|
||
| const loader = ResourceLoader.getInstance(cluster.connectionId); | ||
| try { | ||
| const topics = await loader.getTopicsForCluster(cluster, forceDeepRefresh); |
| // clear stale entries before repopulating | ||
| this.consumerGroupsInTreeView.clear(); | ||
|
|
||
| const loader = ResourceLoader.getInstance(cluster.connectionId); | ||
| try { | ||
| const consumerGroups = await loader.getConsumerGroupsForCluster(cluster, forceDeepRefresh); |
| this.logger.error("Error fetching topics for cluster", cluster, err); | ||
| const message = err instanceof Error ? err.message : String(err); | ||
| this.topicsContainer.setError( | ||
| new CustomMarkdownString() | ||
| .addWarning(`Failed to load topics for **${cluster.name}**:`) | ||
| .addCodeBlock(message), | ||
| ); | ||
| if (err instanceof TopicFetchError) { | ||
| window.showErrorMessage( | ||
| `Failed to list topics for cluster "${cluster.name}": ${err.message}`, | ||
| ); | ||
| } |
| new CustomMarkdownString() | ||
| .addWarning(`Failed to load consumer groups for **${cluster.name}**:`) | ||
| .addCodeBlock(message), | ||
| ); |
src/loaders/cachingResourceLoader.ts
Outdated
| // Convert API response to ConsumerGroup models, fetching members for each group. | ||
| const consumerGroups: ConsumerGroup[] = await Promise.all( | ||
| responseConsumerGroups.map(async (data) => { | ||
| let members: Consumer[] = []; | ||
| try { | ||
| const memberData = await fetchConsumerGroupMembers(cluster, data.consumer_group_id); | ||
| members = memberData.map( |
src/loaders/cachingResourceLoader.ts
Outdated
| state: data.state as ConsumerGroupState, | ||
| isSimple: data.is_simple, |
| /** Get all topic items in the view (nested under the Topics container). */ | ||
| get topics(): Locator { | ||
| return this.body.locator("[role='treeitem'][aria-level='1']"); | ||
| return this.body.locator("[role='treeitem'][aria-level='2']"); |
… item locators in the Topics view
|
| if (!related) return null; | ||
| const lastSegment = related.split("/").pop(); | ||
| if (!lastSegment) return null; | ||
| const parsed = parseInt(lastSegment, 10); |
There was a problem hiding this comment.
I guess the SQ feedback makes sense here since we're already using Number anyhow
|
Back to draft mode to address E2E test issues |






Summary of Changes
Adds consumer groups as a visible resource in the Topics view (pending rename in a future branch):


The Consumer Groups container is shown at the top in a collapsed state, while the (also new) Topics container is auto-expanded to be as minimally disruptive as possible for users.
Closes #3300
(Still more work to be done as part of the #3227 parent issue)
Associated PRs
ResourceContainerfor Flink database and Kafka cluster child resources #3292Click-testing instructions
Optional: Any additional details or context that should be provided?
Promise.allSettled, so one failed member fetch won't block the restPull request checklist
Please check if your PR fulfills the following (if applicable):
Tests
Release notes