Skip to content

SCRUM-5949: Fix facet sort, routing distribution, reduce BP threads#1549

Merged
oblodgett merged 2 commits intostagefrom
SCRUM-5949-fix-facet-sort-order
Apr 3, 2026
Merged

SCRUM-5949: Fix facet sort, routing distribution, reduce BP threads#1549
oblodgett merged 2 commits intostagefrom
SCRUM-5949-fix-facet-sort-order

Conversation

@cmpich
Copy link
Copy Markdown
Contributor

@cmpich cmpich commented Apr 3, 2026

Summary

  • Fix category facet sort order to sort by count descending
  • Fix variant indexer shard routing: use large random int instead of 0-15 range to ensure even distribution across all shards (was leaving 5 of 16 shards empty)
  • Reduce RoutedBulkIndexer threads from shardCount4 to shardCount2 (32 per species instead of 64) to reduce ES client count and memory pressure
  • Add ES client identity logging to diagnose unclosed client leak that prevents JVM exit

Test plan

  • Verified empty shards caused by limited routing range on stage cluster
  • Verify even shard distribution after next variant indexer run
  • Verify client create/close logs match (no leaked clients)
  • Verify JVM exits cleanly after indexing + snapshot completes

Replace hardcoded CATEGORY_ORDER sort with count-based sort to restore
production behavior where facets appear from most to fewest results.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@cmpich cmpich requested a review from a team as a code owner April 3, 2026 11:56
@claude
Copy link
Copy Markdown

claude bot commented Apr 3, 2026

Code Review

The change looks correct. It replaces a hardcoded category display order with a simple descending sort by count (Long.compare(b.getTotal(), a.getTotal())), which matches the ticket intent (SCRUM-5949).

No bugs, no performance concerns, no security issues. The removed CATEGORY_ORDER list and associated index-lookup logic are cleanly eliminated. The sort comparator correctly uses Long.compare to avoid overflow pitfalls of subtraction-based comparison.

LGTM.

- Use Integer.MAX_VALUE for routing to fix empty shard problem (5 of 16 shards empty)
- Reduce BP threads from shardCount*4 to shardCount*2 (32 vs 64 per species)
- Add ES client identity logging to track unclosed client leak
- Remove shardCount from RoutedBulkIndexer constructor
@oblodgett oblodgett changed the title fix: sort category facets by count descending (SCRUM-5949) SCRUM-5949: Fix facet sort, routing distribution, reduce BP threads Apr 3, 2026
@oblodgett oblodgett enabled auto-merge April 3, 2026 12:21
@oblodgett oblodgett merged commit 53f76d0 into stage Apr 3, 2026
7 checks passed
@oblodgett oblodgett deleted the SCRUM-5949-fix-facet-sort-order branch April 3, 2026 12:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants