Migrate Twitter adapter to official X API v2#191
Merged
user1303836 merged 2 commits intomainfrom Feb 6, 2026
Merged
Conversation
Replace the third-party twitterapi.io proxy with direct X API v2
integration using Bearer Token authentication. Key changes:
- Rewrite TwitterAdapter to use GET /2/users/by/username and
GET /2/users/{id}/tweets endpoints with field expansions
- Add in-memory user ID caching to minimize API credit usage
- Use server-side retweet filtering via exclude=retweets param
- Parse X API v2 response format (data/includes/meta structure)
- Extract quoted tweet text, media thumbnails, and author info
from the includes expansion objects
- Rename config field twitter_api_key -> twitter_bearer_token
- Update pipeline, source management cog, and .env.example
- Rewrite test suite for v2 API format (21 tests, up from 17)
- Add tests for user ID caching, media thumbnails, empty
timelines, and tweets without user expansion
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
There was a problem hiding this comment.
Your free trial has ended. If you'd like to continue receiving code reviews, you can add a payment method here.
…te_tweet test - Fix README cost calculation (4,800 -> 28,800 reads/month for 10 sources at 15-min interval) - Remove unused entities field from TWEET_FIELDS to reduce response payload - Add explicit test for note_tweet long-form text preference over truncated text
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces the third-party twitterapi.io proxy with the official X API v2 (pay-per-use). This eliminates the dependency on a third-party service and uses Twitter's native endpoints with Bearer Token authentication.
Key changes:
TwitterAdapterto use X API v2 endpoints (/2/users/by/username/{username}and/2/users/{id}/tweets)TWITTER_BEARER_TOKENenv var)note_tweet)exclude=retweets,repliesandmax_results=5Changes
src/intelstream/adapters/twitter.pysrc/intelstream/config.pytwitter_api_key->twitter_bearer_tokensrc/intelstream/services/pipeline.pysrc/intelstream/discord/cogs/source_management.pysrc/intelstream/adapters/__init__.py.env.exampleREADME.mdtests/test_adapters/test_twitter.pytests/test_discord/test_source_management.pytests/test_services/test_pipeline.pyCost Profile
Architect Review Notes
Four issues found and fixed during code review:
max_resultswas 10, changed to 5 (cost optimization per design)excludewas missingreplies, changed toretweets,replies(server-side filtering)note_tweetfield was missing fromTWEET_FIELDSand_parse_tweet(long tweets >280 chars)adapters/__init__.pywas not updated to exportTwitterAdapterTest Plan
ruff check .passesruff format --check .passesmypy src/passes