Skip to content

Conversation

@ghukill
Copy link
Contributor

@ghukill ghukill commented Dec 22, 2025

Purpose and background context

This PR bumps TDA to version 3.10, which allows reindex-source to complete successfully against a dataset that doesn't yet have embeddings.

How can a reviewer manually see the effects of these changes?

1- Set AWS Dev1 TIMDEX credentials

2- Set env vars:

WORKSPACE=dev
TIMDEX_OPENSEARCH_ENDPOINT=search-timdex-dev-fgby3dckzlfmni2len2wbdf444.us-east-1.es.amazonaws.com
WARNING_ONLY_LOGGERS=botocore,opensearch,urllib3

3- Reindex researchdatabases, which does not have embeddings

pipenv run tim --verbose \
reindex-source \
-s researchdatabases \
s3://timdex-extract-dev-222053980223/dataset

Note the following log line embedded in the overall success:

Table 'current_embeddings' not found in DuckDB context. Embeddings may not yet exist or TIMDEXDataset.refresh() may be required.

Includes new or updated dependencies?

YES

Changes expectations for external applications?

YES: reindex-source works

What are the relevant tickets?

Code review

  • Code review best practices are documented here and you are encouraged to have a constructive dialogue with your reviewers about their preferences and expectations.

@ghukill ghukill marked this pull request as ready for review December 22, 2025 16:39
@ghukill ghukill requested a review from a team as a code owner December 22, 2025 16:39
@coveralls
Copy link

Pull Request Test Coverage Report for Build 20438052803

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage remained the same at 95.732%

Totals Coverage Status
Change from base Build 20348552949: 0.0%
Covered Lines: 471
Relevant Lines: 492

💛 - Coveralls

@ghukill ghukill merged commit 0a5d7a9 into main Dec 22, 2025
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants