diff --git a/CLAUDE.md b/CLAUDE.md index d27eaa0..73ac010 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -579,11 +579,15 @@ Résultats avec JWT token (Pro Premium) : - [x] ✅ Tests unitaires datafeed.py (71 tests) - [x] ✅ Tests exceptions et config (100% couverture) -### Phase 5 : UX & Documentation ✅ PARTIELLEMENT COMPLÉTÉ +### Phase 5 : UX & Documentation ✅ COMPLÉTÉ (Nov 2025) - [x] ✅ Exemples complets (2FA, date range, quiet mode, CAPTCHA) - [x] ✅ Guide de troubleshooting (README.md) - [x] ✅ Messages d'erreur clairs (exceptions personnalisées) -- [ ] Documentation API complète (Sphinx/MkDocs) +- [x] ✅ Documentation API complète (MkDocs Material) + - 19 fichiers de documentation + - API Reference complete (TvDatafeed, TvDatafeedLive, Seis, Consumer, Exceptions, Config) + - Getting Started guides (installation, quickstart, authentication) + - Examples (basic, 2fa, live-feed, date-range) --- @@ -619,14 +623,34 @@ Résultats avec JWT token (Pro Premium) : --- -**Version** : 1.5 +**Version** : 1.6 **Dernière mise à jour** : 2025-11-22 -**Statut** : ✅ Phase 1, Phase 2, Phase 3 et Phase 4 complétées | ⚠️ reCAPTCHA bloque auth username/password +**Statut** : ✅ Phase 1, Phase 2, Phase 3, Phase 4 et Phase 5 complétées | ⚠️ reCAPTCHA bloque auth username/password --- ## Historique des mises à jour +### Version 1.6 (2025-11-22) +- ✅ Phase 5 complétée : UX & Documentation +- ✅ Documentation API complète avec MkDocs Material +- ✅ Configuration mkdocs.yml avec Material theme, navigation tabs, search, code highlighting +- ✅ 19 fichiers de documentation créés : + - `docs/index.md` : Page d'accueil avec features et quick start + - `docs/api/index.md` : Vue d'ensemble API avec liens vers toutes les classes + - `docs/api/tvdatafeed.md` : Documentation complète TvDatafeed (paramètres, méthodes, exemples) + - `docs/api/tvdatafeedlive.md` : Documentation TvDatafeedLive avec architecture threading + - `docs/api/helpers.md` : Seis, Consumer, utils, validators + - `docs/api/exceptions.md` : Hiérarchie complète des exceptions avec patterns de handling + - `docs/api/configuration.md` : NetworkConfig, AuthConfig, DataConfig, ThreadingConfig + - `docs/getting-started/installation.md`, `quickstart.md`, `authentication.md` + - `docs/examples/basic.md`, `2fa.md`, `live-feed.md`, `date-range.md` +- ✅ Tables de paramètres avec types, defaults, descriptions +- ✅ Exemples de code pour chaque méthode (tabs pour différentes options) +- ✅ Documentation complète des variables d'environnement +- ✅ Patterns avancés : keyring, AWS Secrets Manager, retry patterns +- ✅ Revue architecturale Phase 5 : 9/10 - APPROUVÉ + ### Version 1.5 (2025-11-22) - 🔴 **DÉCOUVERTE CRITIQUE** : reCAPTCHA invisible bloque l'authentification username/password - ✅ Documenté la solution de contournement via JWT auth_token diff --git a/docs/api/configuration.md b/docs/api/configuration.md new file mode 100644 index 0000000..f5feaac --- /dev/null +++ b/docs/api/configuration.md @@ -0,0 +1,395 @@ +# Configuration + +TvDatafeed provides dataclass-based configuration for flexible customization. + +## Overview + +Configuration is organized into specialized classes: + +| Class | Purpose | +|-------|---------| +| `NetworkConfig` | WebSocket and HTTP settings | +| `AuthConfig` | Authentication settings | +| `DataConfig` | Data processing settings | +| `ThreadingConfig` | Threading settings | +| `TvDatafeedConfig` | Aggregate configuration | + +All configuration can be loaded from environment variables. + +--- + +## NetworkConfig + +Configuration for WebSocket and HTTP connections. + +```python +@dataclass +class NetworkConfig: + ws_url: str = "wss://data.tradingview.com/socket.io/websocket" + ws_headers: Dict[str, str] = field(default_factory=lambda: { + "Origin": "https://data.tradingview.com" + }) + connect_timeout: float = 10.0 + send_timeout: float = 5.0 + recv_timeout: float = 30.0 + max_retries: int = 3 + base_retry_delay: float = 2.0 + max_retry_delay: float = 60.0 + requests_per_minute: int = 60 +``` + +### Attributes + +| Attribute | Type | Default | Description | +|-----------|------|---------|-------------| +| `ws_url` | str | wss://data.tradingview.com/... | WebSocket URL | +| `ws_headers` | dict | {"Origin": ...} | WebSocket headers | +| `connect_timeout` | float | 10.0 | Connection timeout (seconds) | +| `send_timeout` | float | 5.0 | Send timeout (seconds) | +| `recv_timeout` | float | 30.0 | Receive timeout (seconds) | +| `max_retries` | int | 3 | Maximum retry attempts | +| `base_retry_delay` | float | 2.0 | Initial retry delay (seconds) | +| `max_retry_delay` | float | 60.0 | Maximum retry delay (seconds) | +| `requests_per_minute` | int | 60 | Rate limit | + +### Environment Variables + +| Variable | Maps To | +|----------|---------| +| `TV_WS_URL` | `ws_url` | +| `TV_CONNECT_TIMEOUT` | `connect_timeout` | +| `TV_SEND_TIMEOUT` | `send_timeout` | +| `TV_RECV_TIMEOUT` | `recv_timeout` | +| `TV_MAX_RETRIES` | `max_retries` | +| `TV_BASE_RETRY_DELAY` | `base_retry_delay` | +| `TV_MAX_RETRY_DELAY` | `max_retry_delay` | +| `TV_REQUESTS_PER_MINUTE` | `requests_per_minute` | + +### Usage + +```python +from tvDatafeed.config import NetworkConfig + +# Default configuration +config = NetworkConfig() + +# From environment variables +config = NetworkConfig.from_env() + +# Custom configuration +config = NetworkConfig( + recv_timeout=60.0, + max_retries=5, + base_retry_delay=5.0 +) +``` + +--- + +## AuthConfig + +Configuration for authentication. + +```python +@dataclass +class AuthConfig: + sign_in_url: str = 'https://www.tradingview.com/accounts/signin/' + signin_headers: Dict[str, str] = field(default_factory=lambda: { + 'Referer': 'https://www.tradingview.com' + }) + username: Optional[str] = None + password: Optional[str] = None + two_factor_code: Optional[str] = None +``` + +### Attributes + +| Attribute | Type | Default | Description | +|-----------|------|---------|-------------| +| `sign_in_url` | str | https://...signin/ | Sign-in endpoint | +| `signin_headers` | dict | {"Referer": ...} | Auth request headers | +| `username` | str | None | TradingView username | +| `password` | str | None | TradingView password | +| `two_factor_code` | str | None | 2FA code | + +### Environment Variables + +| Variable | Maps To | +|----------|---------| +| `TV_USERNAME` | `username` | +| `TV_PASSWORD` | `password` | +| `TV_2FA_CODE` | `two_factor_code` | + +### Usage + +```python +from tvDatafeed.config import AuthConfig + +# From environment variables +config = AuthConfig.from_env() + +# Check if credentials are provided +if config.username and config.password: + print("Authenticated mode") +else: + print("Unauthenticated mode") +``` + +--- + +## DataConfig + +Configuration for data processing. + +```python +@dataclass +class DataConfig: + max_bars: int = 5000 + default_bars: int = 10 + validate_data: bool = True + fill_missing_volume: str = 'zero' + timezone: str = 'UTC' +``` + +### Attributes + +| Attribute | Type | Default | Description | +|-----------|------|---------|-------------| +| `max_bars` | int | 5000 | Maximum bars per request | +| `default_bars` | int | 10 | Default bars if not specified | +| `validate_data` | bool | True | Validate OHLCV data | +| `fill_missing_volume` | str | 'zero' | How to fill missing volume | +| `timezone` | str | 'UTC' | Default timezone | + +### Environment Variables + +| Variable | Maps To | +|----------|---------| +| `TV_MAX_BARS` | `max_bars` | +| `TV_DEFAULT_BARS` | `default_bars` | +| `TV_VALIDATE_DATA` | `validate_data` | +| `TV_FILL_MISSING_VOLUME` | `fill_missing_volume` | +| `TV_TIMEZONE` | `timezone` | + +### Usage + +```python +from tvDatafeed.config import DataConfig + +# From environment variables +config = DataConfig.from_env() + +print(f"Max bars: {config.max_bars}") +print(f"Validation: {'enabled' if config.validate_data else 'disabled'}") +``` + +--- + +## ThreadingConfig + +Configuration for TvDatafeedLive threading. + +```python +@dataclass +class ThreadingConfig: + retry_limit: int = 50 + retry_sleep: float = 0.1 + shutdown_timeout: float = 10.0 +``` + +### Attributes + +| Attribute | Type | Default | Description | +|-----------|------|---------|-------------| +| `retry_limit` | int | 50 | Max data fetch retries | +| `retry_sleep` | float | 0.1 | Sleep between retries (seconds) | +| `shutdown_timeout` | float | 10.0 | Graceful shutdown timeout (seconds) | + +### Environment Variables + +| Variable | Maps To | +|----------|---------| +| `TV_RETRY_LIMIT` | `retry_limit` | +| `TV_RETRY_SLEEP` | `retry_sleep` | +| `TV_SHUTDOWN_TIMEOUT` | `shutdown_timeout` | + +### Usage + +```python +from tvDatafeed.config import ThreadingConfig + +# From environment variables +config = ThreadingConfig.from_env() + +print(f"Retry limit: {config.retry_limit}") +print(f"Shutdown timeout: {config.shutdown_timeout}s") +``` + +--- + +## TvDatafeedConfig + +Aggregate configuration containing all sub-configurations. + +```python +@dataclass +class TvDatafeedConfig: + network: NetworkConfig = field(default_factory=NetworkConfig) + auth: AuthConfig = field(default_factory=AuthConfig) + data: DataConfig = field(default_factory=DataConfig) + threading: ThreadingConfig = field(default_factory=ThreadingConfig) + debug: bool = False +``` + +### Attributes + +| Attribute | Type | Description | +|-----------|------|-------------| +| `network` | NetworkConfig | Network configuration | +| `auth` | AuthConfig | Auth configuration | +| `data` | DataConfig | Data configuration | +| `threading` | ThreadingConfig | Threading configuration | +| `debug` | bool | Enable debug mode | + +### Class Methods + +#### from_env() + +Create configuration from all environment variables. + +```python +@classmethod +def from_env(cls) -> 'TvDatafeedConfig': + return cls( + network=NetworkConfig.from_env(), + auth=AuthConfig.from_env(), + data=DataConfig.from_env(), + threading=ThreadingConfig.from_env(), + debug=os.getenv('TV_DEBUG', 'false').lower() == 'true', + ) +``` + +#### default() + +Create configuration with default values. + +```python +@classmethod +def default(cls) -> 'TvDatafeedConfig': + return cls() +``` + +### Usage + +```python +from tvDatafeed.config import TvDatafeedConfig + +# Load all configuration from environment +config = TvDatafeedConfig.from_env() + +# Access sub-configurations +print(f"WebSocket timeout: {config.network.recv_timeout}s") +print(f"Username: {config.auth.username}") +print(f"Max bars: {config.data.max_bars}") +print(f"Debug mode: {config.debug}") + +# Or use defaults +config = TvDatafeedConfig.default() +``` + +--- + +## Global Default Configuration + +A pre-created default configuration instance is available: + +```python +from tvDatafeed.config import DEFAULT_CONFIG + +# Use default values +print(DEFAULT_CONFIG.network.recv_timeout) # 30.0 +print(DEFAULT_CONFIG.data.max_bars) # 5000 +``` + +--- + +## Environment Variable Reference + +### Complete List + +```bash +# Network +TV_WS_URL=wss://data.tradingview.com/socket.io/websocket +TV_CONNECT_TIMEOUT=10.0 +TV_SEND_TIMEOUT=5.0 +TV_RECV_TIMEOUT=30.0 +TV_MAX_RETRIES=3 +TV_BASE_RETRY_DELAY=2.0 +TV_MAX_RETRY_DELAY=60.0 +TV_REQUESTS_PER_MINUTE=60 + +# Authentication +TV_USERNAME=your_username +TV_PASSWORD=your_password +TV_2FA_CODE=123456 +TV_TOTP_SECRET=YOUR_BASE32_SECRET + +# Data +TV_MAX_BARS=5000 +TV_DEFAULT_BARS=10 +TV_VALIDATE_DATA=true +TV_FILL_MISSING_VOLUME=zero +TV_TIMEZONE=UTC + +# Threading +TV_RETRY_LIMIT=50 +TV_RETRY_SLEEP=0.1 +TV_SHUTDOWN_TIMEOUT=10.0 + +# Misc +TV_DEBUG=false +TV_VERBOSE=true +TV_WS_TIMEOUT=5.0 +TV_MAX_RESPONSE_TIME=60.0 +``` + +### Example .env File + +```bash +# .env file for TvDatafeed + +# Authentication (required for full access) +TV_USERNAME=your_email@example.com +TV_PASSWORD=your_secure_password +TV_TOTP_SECRET=JBSWY3DPEHPK3PXP + +# Timeouts (increase for slow connections) +TV_WS_TIMEOUT=30.0 +TV_MAX_RESPONSE_TIME=120.0 + +# Logging +TV_VERBOSE=false +TV_DEBUG=false +``` + +### Loading .env File + +```python +from dotenv import load_dotenv +from tvDatafeed import TvDatafeed +from tvDatafeed.config import TvDatafeedConfig +import os + +# Load .env file +load_dotenv() + +# Configuration is now loaded from environment +config = TvDatafeedConfig.from_env() + +# TvDatafeed also reads from environment +tv = TvDatafeed( + username=os.getenv('TV_USERNAME'), + password=os.getenv('TV_PASSWORD'), + totp_secret=os.getenv('TV_TOTP_SECRET') +) +``` diff --git a/docs/api/exceptions.md b/docs/api/exceptions.md new file mode 100644 index 0000000..9dc8b23 --- /dev/null +++ b/docs/api/exceptions.md @@ -0,0 +1,568 @@ +# Exceptions + +TvDatafeed uses a hierarchy of custom exceptions for precise error handling. + +## Exception Hierarchy + +``` +TvDatafeedError (base) + | + +-- AuthenticationError + | +-- TwoFactorRequiredError + | +-- CaptchaRequiredError + | + +-- NetworkError + | +-- WebSocketError + | | +-- WebSocketTimeoutError + | +-- ConnectionError + | +-- RateLimitError + | + +-- DataError + | +-- DataNotFoundError + | +-- DataValidationError + | | +-- InvalidOHLCError + | +-- SymbolNotFoundError + | +-- InvalidIntervalError + | + +-- ThreadingError + | +-- LockTimeoutError + | +-- ConsumerError + | + +-- ConfigurationError +``` + +--- + +## Base Exception + +### TvDatafeedError + +Base exception for all TvDatafeed errors. + +```python +class TvDatafeedError(Exception): + """Base exception for all TvDatafeed errors""" + pass +``` + +#### Usage + +```python +from tvDatafeed.exceptions import TvDatafeedError + +try: + df = tv.get_hist('BTCUSDT', 'BINANCE', Interval.in_daily) +except TvDatafeedError as e: + # Catches any TvDatafeed-related error + print(f"TvDatafeed error: {e}") +``` + +--- + +## Authentication Exceptions + +### AuthenticationError + +Raised when authentication fails. + +```python +class AuthenticationError(TvDatafeedError): + def __init__( + self, + message: str = "Authentication failed", + username: Optional[str] = None + ) +``` + +#### Attributes + +| Attribute | Type | Description | +|-----------|------|-------------| +| `username` | str | Username that failed authentication | + +#### Common Causes + +- Invalid credentials +- Account locked +- Network timeout during auth + +#### Example + +```python +from tvDatafeed.exceptions import AuthenticationError + +try: + tv = TvDatafeed(username='bad', password='wrong') +except AuthenticationError as e: + print(f"Login failed: {e}") + if e.username: + print(f"Username: {e.username}") +``` + +--- + +### TwoFactorRequiredError + +Raised when 2FA is required but not provided. + +```python +class TwoFactorRequiredError(AuthenticationError): + def __init__(self, method: str = "totp") +``` + +#### Attributes + +| Attribute | Type | Description | +|-----------|------|-------------| +| `method` | str | 2FA method required (e.g., "totp") | + +#### Example + +```python +from tvDatafeed.exceptions import TwoFactorRequiredError + +try: + tv = TvDatafeed(username='user', password='pass') +except TwoFactorRequiredError as e: + print(f"2FA required: {e.method}") + # Retry with 2FA code + tv = TvDatafeed( + username='user', + password='pass', + totp_code=input("Enter 2FA code: ") + ) +``` + +--- + +### CaptchaRequiredError + +Raised when TradingView requires CAPTCHA verification. + +```python +class CaptchaRequiredError(AuthenticationError): + def __init__(self, username: Optional[str] = None) +``` + +#### Attributes + +| Attribute | Type | Description | +|-----------|------|-------------| +| `username` | str | Username that triggered CAPTCHA | + +#### Workaround + +The exception message includes detailed instructions: + +```python +from tvDatafeed.exceptions import CaptchaRequiredError + +try: + tv = TvDatafeed(username='user', password='pass') +except CaptchaRequiredError as e: + print(e) # Contains workaround instructions + # Use auth_token instead + # See examples/captcha_workaround.py +``` + +--- + +## Network Exceptions + +### NetworkError + +Base class for network-related errors. + +```python +class NetworkError(TvDatafeedError): + pass +``` + +--- + +### WebSocketError + +Raised when WebSocket connection fails. + +```python +class WebSocketError(NetworkError): + def __init__( + self, + message: str = "WebSocket error", + url: Optional[str] = None + ) +``` + +#### Example + +```python +from tvDatafeed.exceptions import WebSocketError + +try: + df = tv.get_hist('BTCUSDT', 'BINANCE', Interval.in_daily) +except WebSocketError as e: + print(f"Connection failed: {e}") + # Retry or fallback +``` + +--- + +### WebSocketTimeoutError + +Raised when WebSocket operation times out. + +```python +class WebSocketTimeoutError(WebSocketError): + def __init__( + self, + operation: str = "operation", + timeout: float = 0 + ) +``` + +#### Example + +```python +from tvDatafeed.exceptions import WebSocketTimeoutError + +try: + df = tv.get_hist('BTCUSDT', 'BINANCE', Interval.in_daily, n_bars=5000) +except WebSocketTimeoutError as e: + print(f"Timeout: {e}") + # Try with longer timeout + tv = TvDatafeed(ws_timeout=60.0) +``` + +--- + +### ConnectionError + +Raised when connection cannot be established. + +```python +class ConnectionError(NetworkError): + def __init__( + self, + message: str = "Connection failed", + retry_count: int = 0 + ) +``` + +--- + +### RateLimitError + +Raised when rate limit is exceeded. + +```python +class RateLimitError(NetworkError): + def __init__( + self, + limit: int, + window: int = 60 + ) +``` + +#### Attributes + +| Attribute | Type | Description | +|-----------|------|-------------| +| `limit` | int | Requests limit | +| `window` | int | Time window in seconds | + +--- + +## Data Exceptions + +### DataError + +Base class for data-related errors. + +```python +class DataError(TvDatafeedError): + pass +``` + +--- + +### DataNotFoundError + +Raised when no data is returned. + +```python +class DataNotFoundError(DataError): + def __init__(self, symbol: str, exchange: str) +``` + +#### Attributes + +| Attribute | Type | Description | +|-----------|------|-------------| +| `symbol` | str | Symbol that was not found | +| `exchange` | str | Exchange searched | + +#### Example + +```python +from tvDatafeed.exceptions import DataNotFoundError + +try: + df = tv.get_hist('INVALID', 'BINANCE', Interval.in_daily) +except DataNotFoundError as e: + print(f"Symbol {e.symbol} not found on {e.exchange}") + # Use search_symbol to find correct name + results = tv.search_symbol(e.symbol) +``` + +--- + +### DataValidationError + +Raised when data validation fails. + +```python +class DataValidationError(DataError): + def __init__( + self, + field: str, + value: any, + reason: str + ) +``` + +#### Attributes + +| Attribute | Type | Description | +|-----------|------|-------------| +| `field` | str | Field that failed validation | +| `value` | any | Invalid value | +| `reason` | str | Reason for failure | + +#### Example + +```python +from tvDatafeed.exceptions import DataValidationError + +try: + df = tv.get_hist('BTCUSDT', 'BINANCE', Interval.in_daily, n_bars=-1) +except DataValidationError as e: + print(f"Invalid {e.field}: {e.value}") + print(f"Reason: {e.reason}") +``` + +--- + +### InvalidOHLCError + +Raised when OHLC data violates expected relationships. + +```python +class InvalidOHLCError(DataValidationError): + def __init__( + self, + open_price: float, + high: float, + low: float, + close: float + ) +``` + +--- + +### SymbolNotFoundError + +Raised when symbol is not found. + +```python +class SymbolNotFoundError(DataError): + def __init__( + self, + symbol: str, + exchange: Optional[str] = None + ) +``` + +--- + +### InvalidIntervalError + +Raised when interval is invalid. + +```python +class InvalidIntervalError(DataError): + def __init__( + self, + interval: str, + supported: list = None + ) +``` + +#### Example + +```python +from tvDatafeed.exceptions import InvalidIntervalError + +try: + df = tv.get_hist('BTCUSDT', 'BINANCE', 'invalid') +except InvalidIntervalError as e: + print(f"Invalid interval: {e.interval}") + # Use Interval enum instead +``` + +--- + +## Threading Exceptions + +### ThreadingError + +Base class for threading-related errors. + +```python +class ThreadingError(TvDatafeedError): + pass +``` + +--- + +### LockTimeoutError + +Raised when lock acquisition times out. + +```python +class LockTimeoutError(ThreadingError): + def __init__( + self, + timeout: float, + resource: str = "resource" + ) +``` + +--- + +### ConsumerError + +Raised when consumer encounters an error. + +```python +class ConsumerError(ThreadingError): + def __init__( + self, + message: str, + seis: Optional[str] = None + ) +``` + +--- + +## Configuration Exceptions + +### ConfigurationError + +Raised when configuration is invalid. + +```python +class ConfigurationError(TvDatafeedError): + def __init__( + self, + parameter: str, + value: any, + reason: str + ) +``` + +#### Attributes + +| Attribute | Type | Description | +|-----------|------|-------------| +| `parameter` | str | Parameter name | +| `value` | any | Invalid value | +| `reason` | str | Reason for failure | + +#### Example + +```python +from tvDatafeed.exceptions import ConfigurationError + +try: + tv = TvDatafeed(ws_timeout=-100) +except ConfigurationError as e: + print(f"Invalid config: {e.parameter}={e.value}") + print(f"Reason: {e.reason}") +``` + +--- + +## Error Handling Patterns + +### Comprehensive Error Handling + +```python +from tvDatafeed import TvDatafeed, Interval +from tvDatafeed.exceptions import ( + AuthenticationError, + TwoFactorRequiredError, + CaptchaRequiredError, + WebSocketError, + WebSocketTimeoutError, + DataNotFoundError, + DataValidationError, + ConfigurationError +) + +def fetch_data_safely(symbol, exchange, interval, n_bars): + try: + tv = TvDatafeed() + return tv.get_hist(symbol, exchange, interval, n_bars=n_bars) + + except CaptchaRequiredError: + print("CAPTCHA required - see workaround in docs") + return None + + except TwoFactorRequiredError: + print("2FA required - provide totp_secret or totp_code") + return None + + except AuthenticationError as e: + print(f"Authentication failed: {e}") + return None + + except WebSocketTimeoutError as e: + print(f"Timeout: {e}") + print("Try increasing TV_WS_TIMEOUT environment variable") + return None + + except WebSocketError as e: + print(f"Connection error: {e}") + return None + + except DataNotFoundError as e: + print(f"No data for {e.symbol} on {e.exchange}") + return None + + except DataValidationError as e: + print(f"Invalid input: {e.field}={e.value}") + return None + + except ConfigurationError as e: + print(f"Config error: {e.parameter}={e.value}") + return None +``` + +### Retry Pattern + +```python +import time +from tvDatafeed.exceptions import WebSocketError, WebSocketTimeoutError + +def fetch_with_retry(tv, symbol, exchange, interval, max_retries=3): + for attempt in range(max_retries): + try: + return tv.get_hist(symbol, exchange, interval, n_bars=100) + except (WebSocketError, WebSocketTimeoutError) as e: + if attempt < max_retries - 1: + wait = 2 ** attempt # Exponential backoff + print(f"Retry {attempt + 1}/{max_retries} in {wait}s...") + time.sleep(wait) + else: + raise +``` diff --git a/docs/api/helpers.md b/docs/api/helpers.md new file mode 100644 index 0000000..827edf5 --- /dev/null +++ b/docs/api/helpers.md @@ -0,0 +1,663 @@ +# Helper Classes + +This page documents the helper classes used by TvDatafeed for live data processing. + +## Seis + +**S**ymbol-**E**xchange-**I**nterval **S**et - A container for a unique combination of symbol, exchange, and interval. + +### Class Definition + +```python +class Seis: + def __init__( + self, + symbol: str, + exchange: str, + interval: Interval + ) +``` + +### Constructor Parameters + +| Parameter | Type | Description | +|-----------|------|-------------| +| `symbol` | str | Ticker symbol (e.g., 'BTCUSDT') | +| `exchange` | str | Exchange name (e.g., 'BINANCE') | +| `interval` | Interval | Chart interval | + +### Properties + +| Property | Type | Access | Description | +|----------|------|--------|-------------| +| `symbol` | str | read-only | Ticker symbol | +| `exchange` | str | read-only | Exchange name | +| `interval` | Interval | read-only | Chart interval | +| `tvdatafeed` | TvDatafeedLive | read/write | Parent TvDatafeedLive reference | + +### Methods + +#### new_consumer + +Create a new consumer for this Seis. + +```python +def new_consumer( + self, + callback: Callable, + timeout: int = -1 +) -> Union[Consumer, bool] +``` + +##### Parameters + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `callback` | Callable | **required** | Callback function | +| `timeout` | int | -1 | Max wait time | + +##### Returns + +`Consumer` object or `False` if timeout. + +##### Raises + +| Exception | Condition | +|-----------|-----------| +| `NameError` | No TvDatafeedLive reference | + +##### Example + +```python +def my_callback(seis, data): + print(f"New data for {seis.symbol}") + +consumer = seis.new_consumer(my_callback) +``` + +--- + +#### del_consumer + +Remove a consumer from this Seis. + +```python +def del_consumer( + self, + consumer: Consumer, + timeout: int = -1 +) -> bool +``` + +##### Parameters + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `consumer` | Consumer | **required** | Consumer to remove | +| `timeout` | int | -1 | Max wait time | + +##### Returns + +`True` if successful, `False` if timeout. + +--- + +#### get_hist + +Get historical data for this Seis. + +```python +def get_hist( + self, + n_bars: int = 10, + timeout: int = -1 +) -> Union[pd.DataFrame, bool] +``` + +##### Parameters + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `n_bars` | int | 10 | Number of bars | +| `timeout` | int | -1 | Max wait time | + +##### Returns + +`pd.DataFrame` with OHLCV data or `False` if timeout. + +##### Example + +```python +# Get 100 bars for this Seis +df = seis.get_hist(n_bars=100) +``` + +--- + +#### del_seis + +Remove this Seis from TvDatafeedLive. + +```python +def del_seis( + self, + timeout: int = -1 +) -> bool +``` + +##### Returns + +`True` if successful, `False` if timeout. + +##### Example + +```python +# Remove seis from live feed +seis.del_seis() +``` + +--- + +#### get_consumers + +Get list of consumers for this Seis. + +```python +def get_consumers(self) -> list +``` + +##### Returns + +Thread-safe copy of consumer list. + +--- + +#### is_new_data + +Check if data is newer than last update. + +```python +def is_new_data(self, data: pd.DataFrame) -> bool +``` + +##### Returns + +`True` if data is new, `False` otherwise. + +!!! note "Internal Method" + This method is primarily used internally by TvDatafeedLive. + +--- + +### String Representations + +```python +seis = Seis('BTCUSDT', 'BINANCE', Interval.in_1_hour) + +repr(seis) # Seis("BTCUSDT","BINANCE",Interval.in_1_hour) +str(seis) # symbol='BTCUSDT',exchange='BINANCE',interval='in_1_hour' +``` + +### Equality Comparison + +Two Seis instances are equal if they have the same symbol, exchange, and interval: + +```python +seis1 = Seis('BTCUSDT', 'BINANCE', Interval.in_1_hour) +seis2 = Seis('BTCUSDT', 'BINANCE', Interval.in_1_hour) +seis3 = Seis('ETHUSDT', 'BINANCE', Interval.in_1_hour) + +seis1 == seis2 # True +seis1 == seis3 # False +``` + +### Thread Safety + +Seis uses internal locks for thread-safe operations: + +| Lock | Protects | +|------|----------| +| `_consumers_lock` | Consumer list modifications | +| `_updated_lock` | Last update timestamp | + +--- + +## Consumer + +A threaded callback handler for processing live data. + +### Class Definition + +```python +class Consumer(threading.Thread): + def __init__( + self, + seis: Seis, + callback: Callable[[Seis, pd.DataFrame], None] + ) +``` + +### Constructor Parameters + +| Parameter | Type | Description | +|-----------|------|-------------| +| `seis` | Seis | The Seis to receive data from | +| `callback` | Callable | Function to call with new data | + +### Callback Signature + +```python +def callback(seis: Seis, data: pd.DataFrame) -> None: + """ + Called when new data is available. + + Parameters + ---------- + seis : Seis + The Seis that received data + data : pd.DataFrame + Single row DataFrame with OHLCV data + """ + pass +``` + +### Properties + +| Property | Type | Access | Description | +|----------|------|--------|-------------| +| `seis` | Seis | read/write | Seis reference (thread-safe) | +| `callback` | Callable | read/write | Callback function (thread-safe) | +| `name` | str | read-only | Thread name (auto-generated) | + +### Methods + +#### start + +Start the consumer thread. + +```python +def start(self) -> None +``` + +!!! note "Called Automatically" + `TvDatafeedLive.new_consumer()` calls this automatically. + +--- + +#### stop + +Stop the consumer thread. + +```python +def stop(self) -> None +``` + +##### Example + +```python +consumer.stop() +``` + +--- + +#### put + +Add data to the processing queue. + +```python +def put(self, data: pd.DataFrame) -> None +``` + +##### Parameters + +| Parameter | Type | Description | +|-----------|------|-------------| +| `data` | pd.DataFrame | Data to process | + +!!! note "Internal Method" + This is called by TvDatafeedLive when new data arrives. + +--- + +#### del_consumer + +Stop and remove from Seis. + +```python +def del_consumer( + self, + timeout: int = -1 +) -> bool +``` + +##### Returns + +`True` if successful, `False` if timeout. + +--- + +### String Representations + +```python +consumer = Consumer(seis, my_callback) + +repr(consumer) # Consumer(Seis("BTCUSDT","BINANCE",Interval.in_1_hour),my_callback) +str(consumer) # Seis("BTCUSDT","BINANCE",Interval.in_1_hour),callback=my_callback +``` + +### Thread Model + +``` +Consumer Thread + | + +-- Queue.get(timeout=1.0) # Wait for data + | | + | +-- data = None -> Exit thread + | +-- data = DataFrame -> callback(seis, data) + | + +-- Check _stopped flag periodically + | + +-- On exception in callback: + +-- Log error + +-- Remove from Seis + +-- Exit thread +``` + +### Thread Safety + +| Lock | Protects | +|------|----------| +| `_lock` | `_seis`, `_callback`, `_stopped` | + +### Lifecycle + +```python +# 1. Create consumer +consumer = Consumer(seis, callback) + +# 2. Start thread (done by new_consumer) +consumer.start() + +# 3. Consumer processes data in background +# ... time passes ... + +# 4. Stop consumer +consumer.stop() + +# 5. Wait for thread to finish +consumer.join() +``` + +--- + +## Utility Functions + +The `utils` module provides helper functions used throughout TvDatafeed. + +### generate_session_id + +Generate a random session ID. + +```python +def generate_session_id( + prefix: str = "qs", + length: int = 12 +) -> str +``` + +#### Example + +```python +from tvDatafeed.utils import generate_session_id + +session = generate_session_id() # "qs_abcdefghijkl" +``` + +--- + +### generate_chart_session_id + +Generate a chart session ID. + +```python +def generate_chart_session_id(length: int = 12) -> str +``` + +#### Example + +```python +from tvDatafeed.utils import generate_chart_session_id + +chart_session = generate_chart_session_id() # "cs_abcdefghijkl" +``` + +--- + +### retry_with_backoff + +Retry a function with exponential backoff. + +```python +def retry_with_backoff( + func: Callable[..., T], + max_retries: int = 3, + base_delay: float = 1.0, + max_delay: float = 60.0, + exceptions: tuple = (Exception,), + on_retry: Optional[Callable[[int, Exception], None]] = None +) -> T +``` + +#### Parameters + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `func` | Callable | **required** | Function to retry | +| `max_retries` | int | 3 | Maximum attempts | +| `base_delay` | float | 1.0 | Initial delay (seconds) | +| `max_delay` | float | 60.0 | Maximum delay (seconds) | +| `exceptions` | tuple | (Exception,) | Exceptions to catch | +| `on_retry` | Callable | None | Callback on each retry | + +#### Example + +```python +from tvDatafeed.utils import retry_with_backoff + +def fetch_data(): + # May fail temporarily + return api.get_data() + +# Retry up to 3 times with exponential backoff +data = retry_with_backoff( + fetch_data, + max_retries=3, + base_delay=2.0, + exceptions=(ConnectionError, TimeoutError) +) +``` + +--- + +### mask_sensitive_data + +Mask sensitive data for logging. + +```python +def mask_sensitive_data( + data: str, + visible_chars: int = 4, + mask_char: str = '*' +) -> str +``` + +#### Example + +```python +from tvDatafeed.utils import mask_sensitive_data + +token = "my_secret_token_12345" +masked = mask_sensitive_data(token) # "***************12345" +``` + +--- + +### ContextTimer + +Context manager for timing code blocks. + +```python +class ContextTimer: + def __init__( + self, + name: str = "Operation", + logger_instance: Optional[logging.Logger] = None + ) +``` + +#### Example + +```python +from tvDatafeed.utils import ContextTimer + +with ContextTimer("Data fetch"): + df = tv.get_hist('BTCUSDT', 'BINANCE', Interval.in_daily) +# Logs: "Data fetch took 1.23s" +``` + +--- + +## Validators + +The `validators` module provides input validation functions. + +### Validators.validate_symbol + +Validate a symbol name. + +```python +@staticmethod +def validate_symbol( + symbol: str, + allow_formatted: bool = True +) -> str +``` + +#### Example + +```python +from tvDatafeed.validators import Validators + +# Valid symbols +Validators.validate_symbol('BTCUSDT') # 'BTCUSDT' +Validators.validate_symbol('BINANCE:BTCUSDT') # 'BINANCE:BTCUSDT' + +# Invalid - raises DataValidationError +Validators.validate_symbol('') +``` + +--- + +### Validators.validate_exchange + +Validate an exchange name. + +```python +@staticmethod +def validate_exchange(exchange: str) -> str +``` + +--- + +### Validators.validate_n_bars + +Validate number of bars. + +```python +@staticmethod +def validate_n_bars( + n_bars: int, + max_bars: int = 5000 +) -> int +``` + +--- + +### Validators.validate_timeout + +Validate timeout value. + +```python +@staticmethod +def validate_timeout(timeout: Union[int, float]) -> float +``` + +--- + +### Validators.validate_ohlc + +Validate OHLC relationships. + +```python +@staticmethod +def validate_ohlc( + open_price: float, + high: float, + low: float, + close: float, + raise_on_error: bool = True +) -> bool +``` + +#### Example + +```python +from tvDatafeed.validators import Validators + +# Valid OHLC +Validators.validate_ohlc(100, 110, 95, 105) # True + +# Invalid - high < open +Validators.validate_ohlc(100, 90, 85, 95) # Raises InvalidOHLCError +``` + +--- + +### Validators.validate_credentials + +Validate authentication credentials. + +```python +@staticmethod +def validate_credentials( + username: Optional[str], + password: Optional[str] +) -> bool +``` + +#### Returns + +- `True` if both provided +- `False` if neither provided +- Raises `ConfigurationError` if only one provided + +--- + +### Validators.validate_date_range + +Validate date range for queries. + +```python +@staticmethod +def validate_date_range( + start_date: Optional[datetime], + end_date: Optional[datetime] +) -> tuple +``` + +#### Validation Rules + +- Both must be provided together (or both None) +- Start must be before end +- Neither can be in the future +- Both must be after 2000-01-01 diff --git a/docs/api/index.md b/docs/api/index.md new file mode 100644 index 0000000..d66fa26 --- /dev/null +++ b/docs/api/index.md @@ -0,0 +1,113 @@ +# API Reference + +This section provides detailed documentation for all public classes, methods, and functions in TvDatafeed. + +## Core Classes + +### Data Retrieval + +| Class | Description | +|-------|-------------| +| [TvDatafeed](tvdatafeed.md) | Main class for fetching historical data from TradingView | +| [TvDatafeedLive](tvdatafeedlive.md) | Extended class with real-time data monitoring | + +### Helper Classes + +| Class | Description | +|-------|-------------| +| [Seis](helpers.md#seis) | Symbol-Exchange-Interval Set container | +| [Consumer](helpers.md#consumer) | Callback handler for live data processing | +| [Interval](tvdatafeed.md#interval) | Enum for chart intervals | + +### Configuration + +| Class | Description | +|-------|-------------| +| [NetworkConfig](configuration.md#networkconfig) | Network and WebSocket configuration | +| [AuthConfig](configuration.md#authconfig) | Authentication configuration | +| [DataConfig](configuration.md#dataconfig) | Data processing configuration | +| [ThreadingConfig](configuration.md#threadingconfig) | Threading configuration | + +### Exceptions + +| Exception | Description | +|-----------|-------------| +| [TvDatafeedError](exceptions.md#tvdatafeederror) | Base exception | +| [AuthenticationError](exceptions.md#authenticationerror) | Authentication failures | +| [WebSocketError](exceptions.md#websocketerror) | WebSocket issues | +| [DataNotFoundError](exceptions.md#datanotfounderror) | Data not available | + +## Module Structure + +``` +tvDatafeed/ + __init__.py # Public exports + main.py # TvDatafeed class + datafeed.py # TvDatafeedLive class + seis.py # Seis class + consumer.py # Consumer class + exceptions.py # Custom exceptions + config.py # Configuration classes + utils.py # Utility functions + validators.py # Input validators +``` + +## Quick Links + +- [TvDatafeed.get_hist()](tvdatafeed.md#get_hist) - Get historical data +- [TvDatafeed.search_symbol()](tvdatafeed.md#search_symbol) - Search for symbols +- [TvDatafeedLive.new_seis()](tvdatafeedlive.md#new_seis) - Add symbol to live feed +- [TvDatafeedLive.new_consumer()](tvdatafeedlive.md#new_consumer) - Add callback for live data + +## Usage Patterns + +### Historical Data Only + +Use `TvDatafeed` for one-time data fetching: + +```python +from tvDatafeed import TvDatafeed, Interval + +tv = TvDatafeed() +df = tv.get_hist('BTCUSDT', 'BINANCE', Interval.in_1_hour, n_bars=100) +``` + +### Live Data Monitoring + +Use `TvDatafeedLive` for continuous monitoring: + +```python +from tvDatafeed import TvDatafeedLive, Interval + +def on_new_data(seis, data): + print(f"New data for {seis.symbol}: {data}") + +tv = TvDatafeedLive() +seis = tv.new_seis('BTCUSDT', 'BINANCE', Interval.in_1_minute) +consumer = tv.new_consumer(seis, on_new_data) + +# Later, cleanup +tv.del_tvdatafeed() +``` + +### Error Handling + +```python +from tvDatafeed import TvDatafeed, Interval +from tvDatafeed.exceptions import ( + AuthenticationError, + WebSocketError, + DataNotFoundError +) + +tv = TvDatafeed() + +try: + df = tv.get_hist('INVALID', 'EXCHANGE', Interval.in_daily) +except DataNotFoundError as e: + print(f"Symbol not found: {e}") +except WebSocketError as e: + print(f"Connection error: {e}") +except AuthenticationError as e: + print(f"Auth error: {e}") +``` diff --git a/docs/api/tvdatafeed.md b/docs/api/tvdatafeed.md new file mode 100644 index 0000000..fa010f4 --- /dev/null +++ b/docs/api/tvdatafeed.md @@ -0,0 +1,418 @@ +# TvDatafeed + +The main class for fetching historical data from TradingView. + +## Class Definition + +```python +class TvDatafeed: + def __init__( + self, + username: Optional[str] = None, + password: Optional[str] = None, + auth_token: Optional[str] = None, + totp_secret: Optional[str] = None, + totp_code: Optional[str] = None, + ws_timeout: Optional[float] = None, + verbose: Optional[bool] = None, + ) -> None +``` + +## Constructor Parameters + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `username` | str | None | TradingView username | +| `password` | str | None | TradingView password | +| `auth_token` | str | None | Pre-obtained authentication token (bypasses login) | +| `totp_secret` | str | None | TOTP secret for automatic 2FA code generation | +| `totp_code` | str | None | Manual 6-digit 2FA code | +| `ws_timeout` | float | None | WebSocket timeout in seconds | +| `verbose` | bool | None | Enable verbose logging | + +### Authentication Options + +There are several ways to authenticate: + +=== "Username/Password" + + ```python + tv = TvDatafeed( + username='your_username', + password='your_password' + ) + ``` + +=== "With 2FA (TOTP Secret)" + + ```python + tv = TvDatafeed( + username='your_username', + password='your_password', + totp_secret='YOUR_BASE32_SECRET' + ) + ``` + +=== "With 2FA (Manual Code)" + + ```python + tv = TvDatafeed( + username='your_username', + password='your_password', + totp_code='123456' + ) + ``` + +=== "Pre-obtained Token" + + ```python + # Use when CAPTCHA is required + tv = TvDatafeed(auth_token='your_auth_token') + ``` + +=== "Environment Variables" + + ```python + import os + from dotenv import load_dotenv + + load_dotenv() + + tv = TvDatafeed( + username=os.getenv('TV_USERNAME'), + password=os.getenv('TV_PASSWORD'), + totp_secret=os.getenv('TV_TOTP_SECRET') + ) + ``` + +### Timeout Configuration + +Priority for WebSocket timeout: + +1. `ws_timeout` parameter (highest priority) +2. `TV_WS_TIMEOUT` environment variable +3. `NetworkConfig.recv_timeout` +4. Default: 5 seconds + +```python +# Custom timeout +tv = TvDatafeed(ws_timeout=30.0) + +# Or via environment +# export TV_WS_TIMEOUT=30.0 +tv = TvDatafeed() +``` + +### Verbose Logging + +Control logging verbosity: + +```python +# Quiet mode (production) +tv = TvDatafeed(verbose=False) + +# Verbose mode (debugging) +tv = TvDatafeed(verbose=True) + +# Or via environment +# export TV_VERBOSE=false +tv = TvDatafeed() +``` + +## Methods + +### get_hist + +Fetch historical OHLCV data from TradingView. + +```python +def get_hist( + self, + symbol: str, + exchange: str = "NSE", + interval: Interval = Interval.in_daily, + n_bars: Optional[int] = None, + fut_contract: Optional[int] = None, + extended_session: bool = False, + start_date: Optional[datetime] = None, + end_date: Optional[datetime] = None, +) -> Optional[pd.DataFrame] +``` + +#### Parameters + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `symbol` | str | **required** | Symbol name (e.g., 'BTCUSDT', 'AAPL') | +| `exchange` | str | "NSE" | Exchange name (e.g., 'BINANCE', 'NASDAQ') | +| `interval` | Interval | in_daily | Chart interval | +| `n_bars` | int | None | Number of bars (max 5000) | +| `fut_contract` | int | None | Futures contract number | +| `extended_session` | bool | False | Include extended trading hours | +| `start_date` | datetime | None | Start date for date range query | +| `end_date` | datetime | None | End date for date range query | + +!!! warning "Mutually Exclusive Parameters" + `n_bars` and `start_date`/`end_date` are mutually exclusive. Use one or the other. + +#### Returns + +| Type | Description | +|------|-------------| +| `pd.DataFrame` | DataFrame with columns: symbol, datetime (index), open, high, low, close, volume | +| `None` | If no data is available | + +#### Examples + +=== "Basic Usage" + + ```python + from tvDatafeed import TvDatafeed, Interval + + tv = TvDatafeed() + + # Get 100 daily bars + df = tv.get_hist( + symbol='BTCUSDT', + exchange='BINANCE', + interval=Interval.in_daily, + n_bars=100 + ) + + print(df.head()) + # symbol open high low close volume + # datetime + # 2024-01-01 BINANCE:BTCUSDT 42000.0 42500.0 41500.0 42200.0 1234567890.0 + ``` + +=== "Date Range Query" + + ```python + from datetime import datetime + from tvDatafeed import TvDatafeed, Interval + + tv = TvDatafeed() + + df = tv.get_hist( + symbol='AAPL', + exchange='NASDAQ', + interval=Interval.in_1_hour, + start_date=datetime(2024, 1, 1), + end_date=datetime(2024, 1, 31) + ) + + # Access timezone metadata + print(f"Timezone: {df.attrs.get('timezone', 'Not set')}") + ``` + +=== "Futures Contract" + + ```python + tv = TvDatafeed() + + # Front month continuous contract + df = tv.get_hist( + symbol='CRUDEOIL', + exchange='MCX', + interval=Interval.in_daily, + fut_contract=1, + n_bars=100 + ) + ``` + +=== "Extended Session" + + ```python + tv = TvDatafeed() + + # Include pre/post market data + df = tv.get_hist( + symbol='AAPL', + exchange='NASDAQ', + interval=Interval.in_15_minute, + extended_session=True, + n_bars=200 + ) + ``` + +#### Exceptions + +| Exception | Condition | +|-----------|-----------| +| `DataValidationError` | Invalid symbol, exchange, n_bars, or date range | +| `InvalidIntervalError` | Invalid interval type | +| `WebSocketError` | Connection failure | +| `WebSocketTimeoutError` | Operation timeout | +| `DataNotFoundError` | No data available | + +--- + +### search_symbol + +Search for symbols on TradingView. + +```python +def search_symbol( + self, + text: str, + exchange: str = '' +) -> list +``` + +#### Parameters + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `text` | str | **required** | Search query (e.g., 'BTC', 'AAPL') | +| `exchange` | str | '' | Limit to specific exchange | + +#### Returns + +| Type | Description | +|------|-------------| +| `list` | List of dictionaries with symbol information | + +Each dictionary contains: + +- `symbol`: Symbol name +- `exchange`: Exchange name +- `description`: Symbol description +- `type`: Asset type (stock, crypto, etc.) + +#### Examples + +```python +tv = TvDatafeed() + +# Search all exchanges +results = tv.search_symbol('BTC') + +# Search specific exchange +results = tv.search_symbol('BTC', 'BINANCE') + +# Use results +for r in results[:5]: + full_symbol = f"{r['exchange']}:{r['symbol']}" + print(f"{full_symbol} - {r['description']}") +``` + +--- + +### format_search_results + +Format search results for display. + +```python +@staticmethod +def format_search_results( + results: list, + max_results: int = 10 +) -> str +``` + +#### Parameters + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `results` | list | **required** | Results from search_symbol() | +| `max_results` | int | 10 | Maximum results to format | + +#### Example + +```python +tv = TvDatafeed() +results = tv.search_symbol('ETH', 'BINANCE') +print(tv.format_search_results(results)) +``` + +--- + +### is_valid_date_range + +Validate a date range for historical queries. + +```python +@staticmethod +def is_valid_date_range( + start: datetime, + end: datetime +) -> bool +``` + +#### Parameters + +| Parameter | Type | Description | +|-----------|------|-------------| +| `start` | datetime | Start date | +| `end` | datetime | End date | + +#### Returns + +`True` if the date range is valid, `False` otherwise. + +#### Validation Rules + +- Start date must be before end date +- Neither date can be in the future +- Both dates must be after 2000-01-01 + +--- + +## Interval + +Enum class for chart intervals. + +```python +class Interval(enum.Enum): + in_1_minute = "1" + in_3_minute = "3" + in_5_minute = "5" + in_15_minute = "15" + in_30_minute = "30" + in_45_minute = "45" + in_1_hour = "1H" + in_2_hour = "2H" + in_3_hour = "3H" + in_4_hour = "4H" + in_daily = "1D" + in_weekly = "1W" + in_monthly = "1M" +``` + +### Usage + +```python +from tvDatafeed import Interval + +# Use enum values +df = tv.get_hist('BTCUSDT', 'BINANCE', Interval.in_1_hour, n_bars=100) + +# Available intervals +for i in Interval: + print(f"{i.name}: {i.value}") +``` + +--- + +## Attributes + +| Attribute | Type | Description | +|-----------|------|-------------| +| `token` | str | Authentication token | +| `ws_timeout` | float | WebSocket timeout in seconds | +| `verbose` | bool | Verbose logging enabled | +| `max_response_time` | float | Maximum cumulative response time | +| `session` | str | Session ID | +| `chart_session` | str | Chart session ID | + +--- + +## Class Constants + +| Constant | Value | Description | +|----------|-------|-------------| +| `__ws_timeout` | 5 | Default WebSocket timeout | +| `__ws_max_retries` | 3 | Max WebSocket connection retries | +| `__ws_retry_base_delay` | 2.0 | Base retry delay (seconds) | +| `__ws_retry_max_delay` | 10.0 | Max retry delay (seconds) | +| `__max_response_time` | 60.0 | Default max response time | diff --git a/docs/api/tvdatafeedlive.md b/docs/api/tvdatafeedlive.md new file mode 100644 index 0000000..4efe4fb --- /dev/null +++ b/docs/api/tvdatafeedlive.md @@ -0,0 +1,416 @@ +# TvDatafeedLive + +Extended class for real-time data monitoring with callback support. + +`TvDatafeedLive` inherits from `TvDatafeed` and adds live data feed capabilities using threading. + +## Class Definition + +```python +class TvDatafeedLive(TvDatafeed): + def __init__( + self, + username: Optional[str] = None, + password: Optional[str] = None + ) +``` + +## Constructor Parameters + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `username` | str | None | TradingView username | +| `password` | str | None | TradingView password | + +!!! note "Inherited Parameters" + `TvDatafeedLive` inherits all authentication options from `TvDatafeed`, including 2FA support. + +## Architecture + +``` +TvDatafeedLive + | + +-- Main Thread (monitors intervals) + | | + | +-- Fetches new data when interval expires + | +-- Pushes data to consumers + | + +-- Consumer Threads (one per callback) + | + +-- Receives data from queue + +-- Executes callback function +``` + +## Methods + +### new_seis + +Create and add a new Seis (Symbol-Exchange-Interval Set) to the live feed. + +```python +def new_seis( + self, + symbol: str, + exchange: str, + interval: Interval, + timeout: int = -1 +) -> Union[Seis, bool] +``` + +#### Parameters + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `symbol` | str | **required** | Ticker symbol | +| `exchange` | str | **required** | Exchange name | +| `interval` | Interval | **required** | Chart interval | +| `timeout` | int | -1 | Max wait time (-1 = blocking) | + +#### Returns + +| Type | Description | +|------|-------------| +| `Seis` | The created or existing Seis object | +| `False` | If timeout expired | + +#### Exceptions + +| Exception | Condition | +|-----------|-----------| +| `ValueError` | Symbol/exchange not found on TradingView | + +#### Example + +```python +from tvDatafeed import TvDatafeedLive, Interval + +tv = TvDatafeedLive() + +# Add symbol to live feed +seis = tv.new_seis('BTCUSDT', 'BINANCE', Interval.in_1_minute) + +# If Seis already exists, same instance is returned +seis2 = tv.new_seis('BTCUSDT', 'BINANCE', Interval.in_1_minute) +assert seis is seis2 +``` + +--- + +### del_seis + +Remove a Seis from the live feed. + +```python +def del_seis( + self, + seis: Seis, + timeout: int = -1 +) -> bool +``` + +#### Parameters + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `seis` | Seis | **required** | Seis to remove | +| `timeout` | int | -1 | Max wait time (-1 = blocking) | + +#### Returns + +`True` if successful, `False` if timeout expired. + +#### Exceptions + +| Exception | Condition | +|-----------|-----------| +| `ValueError` | Seis not in live feed | + +#### Example + +```python +seis = tv.new_seis('BTCUSDT', 'BINANCE', Interval.in_1_minute) + +# Later, remove it +tv.del_seis(seis) +``` + +--- + +### new_consumer + +Create a new consumer with a callback function for a Seis. + +```python +def new_consumer( + self, + seis: Seis, + callback: Callable[[Seis, pd.DataFrame], None], + timeout: int = -1 +) -> Union[Consumer, bool] +``` + +#### Parameters + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `seis` | Seis | **required** | Seis to monitor | +| `callback` | Callable | **required** | Function to call with new data | +| `timeout` | int | -1 | Max wait time (-1 = blocking) | + +#### Callback Signature + +```python +def callback(seis: Seis, data: pd.DataFrame) -> None: + """ + Parameters + ---------- + seis : Seis + The Seis that received new data + data : pd.DataFrame + DataFrame with single row of OHLCV data + """ + pass +``` + +#### Returns + +| Type | Description | +|------|-------------| +| `Consumer` | The created Consumer object | +| `False` | If timeout expired | + +#### Exceptions + +| Exception | Condition | +|-----------|-----------| +| `ValueError` | Seis not in live feed | + +#### Example + +```python +def on_new_data(seis, data): + """Called when new bar is available""" + print(f"[{seis.symbol}] New data at {data.index[0]}") + print(f" Open: {data['open'].iloc[0]}") + print(f" High: {data['high'].iloc[0]}") + print(f" Low: {data['low'].iloc[0]}") + print(f" Close: {data['close'].iloc[0]}") + print(f" Volume: {data['volume'].iloc[0]}") + +tv = TvDatafeedLive() +seis = tv.new_seis('BTCUSDT', 'BINANCE', Interval.in_1_minute) +consumer = tv.new_consumer(seis, on_new_data) +``` + +--- + +### del_consumer + +Remove a consumer from its Seis. + +```python +def del_consumer( + self, + consumer: Consumer, + timeout: int = -1 +) -> bool +``` + +#### Parameters + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `consumer` | Consumer | **required** | Consumer to remove | +| `timeout` | int | -1 | Max wait time (-1 = blocking) | + +#### Returns + +`True` if successful, `False` if timeout expired. + +#### Example + +```python +consumer = tv.new_consumer(seis, callback) + +# Later, remove the consumer +tv.del_consumer(consumer) +``` + +--- + +### get_hist + +Get historical data (inherited from TvDatafeed with thread-safe locking). + +```python +def get_hist( + self, + symbol: str, + exchange: str = "NSE", + interval: Interval = Interval.in_daily, + n_bars: int = 10, + fut_contract: Optional[int] = None, + extended_session: bool = False, + timeout: int = -1 +) -> Union[pd.DataFrame, bool] +``` + +#### Parameters + +Same as `TvDatafeed.get_hist()` plus: + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `timeout` | int | -1 | Max wait time (-1 = blocking) | + +#### Returns + +`pd.DataFrame` with OHLCV data, or `False` if timeout expired. + +#### Example + +```python +tv = TvDatafeedLive() + +# Safe to call while live feed is running +df = tv.get_hist('BTCUSDT', 'BINANCE', Interval.in_1_hour, n_bars=100) +``` + +--- + +### del_tvdatafeed + +Stop all threads and cleanup resources. + +```python +def del_tvdatafeed(self) -> None +``` + +#### Example + +```python +tv = TvDatafeedLive() +seis = tv.new_seis('BTCUSDT', 'BINANCE', Interval.in_1_minute) +consumer = tv.new_consumer(seis, callback) + +# ... use the live feed ... + +# Cleanup when done +tv.del_tvdatafeed() +``` + +--- + +## Complete Example + +```python +from tvDatafeed import TvDatafeedLive, Interval +import time + +def btc_handler(seis, data): + """Handle BTC updates""" + close = data['close'].iloc[0] + print(f"BTC: ${close:,.2f}") + +def eth_handler(seis, data): + """Handle ETH updates""" + close = data['close'].iloc[0] + print(f"ETH: ${close:,.2f}") + +# Create live feed +tv = TvDatafeedLive() + +try: + # Add symbols to monitor + btc_seis = tv.new_seis('BTCUSDT', 'BINANCE', Interval.in_1_minute) + eth_seis = tv.new_seis('ETHUSDT', 'BINANCE', Interval.in_1_minute) + + # Add callbacks + btc_consumer = tv.new_consumer(btc_seis, btc_handler) + eth_consumer = tv.new_consumer(eth_seis, eth_handler) + + # Let it run for a while + print("Monitoring BTC and ETH... Press Ctrl+C to stop") + while True: + time.sleep(1) + +except KeyboardInterrupt: + print("\nShutting down...") + +finally: + # Always cleanup + tv.del_tvdatafeed() + print("Done") +``` + +--- + +## Threading Model + +### Locks + +| Lock | Protects | Description | +|------|----------|-------------| +| `_lock` | `_sat`, public operations | Main lock for all operations | +| `_thread_lock` | `_main_thread` | Access to main thread reference | + +### Thread Safety + +- All public methods acquire locks before modifying state +- Consumer callbacks run in separate threads +- Exception in callback stops that consumer only +- Graceful shutdown with configurable timeouts + +### Constants + +| Constant | Value | Description | +|----------|-------|-------------| +| `RETRY_LIMIT` | 50 | Max retries for data fetch | +| `SHUTDOWN_TIMEOUT` | 10.0 | Main thread shutdown timeout | +| `CONSUMER_STOP_TIMEOUT` | 5.0 | Consumer thread stop timeout | + +--- + +## Best Practices + +### 1. Always Cleanup + +```python +tv = TvDatafeedLive() +try: + # ... use live feed ... +finally: + tv.del_tvdatafeed() +``` + +### 2. Handle Callback Exceptions + +```python +def safe_callback(seis, data): + try: + # Your logic here + process_data(data) + except Exception as e: + logging.error(f"Error processing {seis.symbol}: {e}") + # Don't re-raise - keeps consumer alive +``` + +### 3. Use Timeout for Non-Blocking + +```python +# Non-blocking with 5 second timeout +seis = tv.new_seis('BTCUSDT', 'BINANCE', Interval.in_1_minute, timeout=5) +if seis is False: + print("Timeout - operation didn't complete in 5 seconds") +``` + +### 4. Multiple Callbacks per Symbol + +```python +seis = tv.new_seis('BTCUSDT', 'BINANCE', Interval.in_1_minute) + +# Multiple consumers for same seis +consumer1 = tv.new_consumer(seis, log_to_file) +consumer2 = tv.new_consumer(seis, send_alert) +consumer3 = tv.new_consumer(seis, update_database) +``` diff --git a/docs/examples/2fa.md b/docs/examples/2fa.md new file mode 100644 index 0000000..04dea82 --- /dev/null +++ b/docs/examples/2fa.md @@ -0,0 +1,279 @@ +# 2FA Authentication Examples + +This guide shows how to use TvDatafeed with two-factor authentication. + +## Prerequisites + +For automatic 2FA code generation, install pyotp: + +```bash +pip install pyotp +``` + +## Using TOTP Secret (Recommended) + +The TOTP secret is shown when you first set up 2FA in TradingView. + +```python +from tvDatafeed import TvDatafeed, Interval + +# With TOTP secret - codes are generated automatically +tv = TvDatafeed( + username='your_email@example.com', + password='your_password', + totp_secret='JBSWY3DPEHPK3PXP' # Your base32 secret +) + +# Now use normally +df = tv.get_hist('BTCUSDT', 'BINANCE', Interval.in_daily, n_bars=100) +print(df.head()) +``` + +## Using Manual Code + +If you don't have the TOTP secret: + +```python +from tvDatafeed import TvDatafeed, Interval + +# Ask user for current code +code = input("Enter your 6-digit 2FA code: ") + +tv = TvDatafeed( + username='your_email@example.com', + password='your_password', + totp_code=code +) + +df = tv.get_hist('BTCUSDT', 'BINANCE', Interval.in_daily, n_bars=100) +``` + +## Using Environment Variables + +### Setup + +Create a `.env` file: + +```bash +# .env - Add to .gitignore! +TV_USERNAME=your_email@example.com +TV_PASSWORD=your_secure_password +TV_TOTP_SECRET=JBSWY3DPEHPK3PXP +``` + +### Usage + +```python +import os +from dotenv import load_dotenv +from tvDatafeed import TvDatafeed, Interval + +# Load environment variables +load_dotenv() + +# Create instance - reads from environment +tv = TvDatafeed( + username=os.getenv('TV_USERNAME'), + password=os.getenv('TV_PASSWORD'), + totp_secret=os.getenv('TV_TOTP_SECRET') +) + +df = tv.get_hist('BTCUSDT', 'BINANCE', Interval.in_daily, n_bars=100) +print(df.head()) +``` + +## Handling 2FA Errors + +### TwoFactorRequiredError + +```python +from tvDatafeed import TvDatafeed, Interval +from tvDatafeed.exceptions import TwoFactorRequiredError + +try: + # Try without 2FA first + tv = TvDatafeed( + username='your_email@example.com', + password='your_password' + ) +except TwoFactorRequiredError: + print("2FA is required for this account!") + + # Option 1: Use TOTP secret + tv = TvDatafeed( + username='your_email@example.com', + password='your_password', + totp_secret='YOUR_SECRET_HERE' + ) + + # Option 2: Ask for manual code + # code = input("Enter 2FA code: ") + # tv = TvDatafeed( + # username='your_email@example.com', + # password='your_password', + # totp_code=code + # ) + +# Now use normally +df = tv.get_hist('BTCUSDT', 'BINANCE', Interval.in_daily, n_bars=100) +``` + +### Invalid Code Error + +```python +from tvDatafeed import TvDatafeed, Interval +from tvDatafeed.exceptions import AuthenticationError + +try: + tv = TvDatafeed( + username='your_email@example.com', + password='your_password', + totp_code='wrong_code' + ) +except AuthenticationError as e: + if 'Invalid 2FA code' in str(e): + print("The 2FA code was invalid or expired!") + print("Tips:") + print(" - Ensure your device time is synchronized") + print(" - TOTP codes expire every 30 seconds") + print(" - Double-check your TOTP secret") + else: + raise +``` + +## Generating TOTP Codes Programmatically + +If you have pyotp installed: + +```python +import pyotp + +# Your TOTP secret from TradingView +secret = 'JBSWY3DPEHPK3PXP' + +# Create TOTP object +totp = pyotp.TOTP(secret) + +# Get current code +code = totp.now() +print(f"Current 2FA code: {code}") + +# Verify a code +is_valid = totp.verify('123456') +print(f"Code valid: {is_valid}") + +# Get remaining time for current code +import time +remaining = 30 - (int(time.time()) % 30) +print(f"Code expires in: {remaining} seconds") +``` + +## Complete Example with Error Handling + +```python +import os +import sys +from dotenv import load_dotenv +from tvDatafeed import TvDatafeed, Interval +from tvDatafeed.exceptions import ( + AuthenticationError, + TwoFactorRequiredError, + CaptchaRequiredError, + ConfigurationError +) + +def create_authenticated_client(): + """Create an authenticated TvDatafeed client.""" + + # Load environment variables + load_dotenv() + + username = os.getenv('TV_USERNAME') + password = os.getenv('TV_PASSWORD') + totp_secret = os.getenv('TV_TOTP_SECRET') + + # Validate credentials + if not username or not password: + print("Error: TV_USERNAME and TV_PASSWORD must be set") + sys.exit(1) + + try: + tv = TvDatafeed( + username=username, + password=password, + totp_secret=totp_secret + ) + print("Authentication successful!") + return tv + + except TwoFactorRequiredError: + if not totp_secret: + print("Error: 2FA required but TV_TOTP_SECRET not set") + print("Please set TV_TOTP_SECRET in your .env file") + sys.exit(1) + raise + + except CaptchaRequiredError as e: + print("CAPTCHA verification required!") + print(str(e)) # Shows workaround instructions + sys.exit(1) + + except AuthenticationError as e: + print(f"Authentication failed: {e}") + sys.exit(1) + + except ConfigurationError as e: + print(f"Configuration error: {e}") + sys.exit(1) + +# Usage +tv = create_authenticated_client() +df = tv.get_hist('BTCUSDT', 'BINANCE', Interval.in_daily, n_bars=100) +print(f"Got {len(df)} bars of data") +``` + +## Secure Storage Best Practices + +### Using keyring (System Keychain) + +```python +import keyring +from tvDatafeed import TvDatafeed, Interval + +# Store credentials (do once) +# keyring.set_password('tvdatafeed', 'username', 'your_email@example.com') +# keyring.set_password('tvdatafeed', 'password', 'your_password') +# keyring.set_password('tvdatafeed', 'totp_secret', 'YOUR_SECRET') + +# Retrieve credentials +username = keyring.get_password('tvdatafeed', 'username') +password = keyring.get_password('tvdatafeed', 'password') +totp_secret = keyring.get_password('tvdatafeed', 'totp_secret') + +tv = TvDatafeed( + username=username, + password=password, + totp_secret=totp_secret +) +``` + +### Using AWS Secrets Manager + +```python +import boto3 +import json +from tvDatafeed import TvDatafeed, Interval + +def get_tv_credentials(): + """Get credentials from AWS Secrets Manager.""" + client = boto3.client('secretsmanager') + response = client.get_secret_value(SecretId='tvdatafeed-credentials') + return json.loads(response['SecretString']) + +creds = get_tv_credentials() +tv = TvDatafeed( + username=creds['username'], + password=creds['password'], + totp_secret=creds['totp_secret'] +) +``` diff --git a/docs/examples/basic.md b/docs/examples/basic.md new file mode 100644 index 0000000..916dfff --- /dev/null +++ b/docs/examples/basic.md @@ -0,0 +1,309 @@ +# Basic Usage Examples + +## Fetching Historical Data + +### Simple Example + +```python +from tvDatafeed import TvDatafeed, Interval + +# Create instance +tv = TvDatafeed() + +# Get daily data +df = tv.get_hist( + symbol='BTCUSDT', + exchange='BINANCE', + interval=Interval.in_daily, + n_bars=100 +) + +print(df.head()) +print(f"Columns: {df.columns.tolist()}") +print(f"Shape: {df.shape}") +``` + +### Multiple Symbols + +```python +from tvDatafeed import TvDatafeed, Interval + +tv = TvDatafeed() + +symbols = [ + ('BTCUSDT', 'BINANCE'), + ('ETHUSDT', 'BINANCE'), + ('AAPL', 'NASDAQ'), + ('MSFT', 'NASDAQ'), +] + +for symbol, exchange in symbols: + df = tv.get_hist(symbol, exchange, Interval.in_daily, n_bars=100) + print(f"{exchange}:{symbol}: {len(df)} bars") +``` + +### Different Intervals + +```python +from tvDatafeed import TvDatafeed, Interval + +tv = TvDatafeed() + +intervals = [ + Interval.in_1_minute, + Interval.in_5_minute, + Interval.in_15_minute, + Interval.in_1_hour, + Interval.in_4_hour, + Interval.in_daily, + Interval.in_weekly, +] + +for interval in intervals: + df = tv.get_hist('BTCUSDT', 'BINANCE', interval, n_bars=10) + print(f"{interval.name}: {len(df)} bars") +``` + +## Working with DataFrames + +### Basic Analysis + +```python +from tvDatafeed import TvDatafeed, Interval +import pandas as pd + +tv = TvDatafeed() +df = tv.get_hist('BTCUSDT', 'BINANCE', Interval.in_daily, n_bars=100) + +# Basic statistics +print("Statistics:") +print(df[['open', 'high', 'low', 'close', 'volume']].describe()) + +# Daily returns +df['returns'] = df['close'].pct_change() +print(f"\nAverage daily return: {df['returns'].mean():.4%}") +print(f"Volatility: {df['returns'].std():.4%}") +``` + +### Technical Indicators + +```python +from tvDatafeed import TvDatafeed, Interval +import pandas as pd + +tv = TvDatafeed() +df = tv.get_hist('BTCUSDT', 'BINANCE', Interval.in_daily, n_bars=100) + +# Simple Moving Averages +df['SMA_20'] = df['close'].rolling(window=20).mean() +df['SMA_50'] = df['close'].rolling(window=50).mean() + +# Exponential Moving Average +df['EMA_20'] = df['close'].ewm(span=20, adjust=False).mean() + +# RSI (Relative Strength Index) +delta = df['close'].diff() +gain = (delta.where(delta > 0, 0)).rolling(window=14).mean() +loss = (-delta.where(delta < 0, 0)).rolling(window=14).mean() +rs = gain / loss +df['RSI'] = 100 - (100 / (1 + rs)) + +print(df[['close', 'SMA_20', 'SMA_50', 'RSI']].tail(10)) +``` + +### Export Data + +```python +from tvDatafeed import TvDatafeed, Interval + +tv = TvDatafeed() +df = tv.get_hist('BTCUSDT', 'BINANCE', Interval.in_daily, n_bars=100) + +# To CSV +df.to_csv('btcusdt_daily.csv') + +# To Excel +df.to_excel('btcusdt_daily.xlsx') + +# To JSON +df.to_json('btcusdt_daily.json', orient='records', date_format='iso') + +print("Data exported successfully!") +``` + +## Symbol Search + +### Search Examples + +```python +from tvDatafeed import TvDatafeed + +tv = TvDatafeed() + +# Search all exchanges +results = tv.search_symbol('BTC') +print(f"Found {len(results)} results for 'BTC'") + +# Search specific exchange +results = tv.search_symbol('BTC', 'BINANCE') +print(f"Found {len(results)} results for 'BTC' on BINANCE") + +# Display formatted results +print(tv.format_search_results(results)) +``` + +### Find Correct Symbol + +```python +from tvDatafeed import TvDatafeed, Interval + +tv = TvDatafeed() + +# Don't know the exact symbol? Search first! +results = tv.search_symbol('NVIDIA') + +for r in results[:5]: + full_symbol = f"{r['exchange']}:{r['symbol']}" + print(f"{full_symbol} - {r['description']} ({r['type']})") + +# Then use the correct symbol +df = tv.get_hist('NVDA', 'NASDAQ', Interval.in_daily, n_bars=100) +``` + +## Error Handling + +### Comprehensive Error Handling + +```python +from tvDatafeed import TvDatafeed, Interval +from tvDatafeed.exceptions import ( + TvDatafeedError, + AuthenticationError, + WebSocketError, + WebSocketTimeoutError, + DataNotFoundError, + DataValidationError +) + +def safe_get_hist(tv, symbol, exchange, interval, n_bars): + """Safely fetch historical data with error handling.""" + try: + df = tv.get_hist(symbol, exchange, interval, n_bars=n_bars) + return df + + except DataNotFoundError as e: + print(f"[ERROR] Symbol not found: {e.symbol} on {e.exchange}") + return None + + except DataValidationError as e: + print(f"[ERROR] Invalid input: {e.field}={e.value}") + print(f" Reason: {e.reason}") + return None + + except WebSocketTimeoutError as e: + print(f"[ERROR] Connection timed out: {e}") + return None + + except WebSocketError as e: + print(f"[ERROR] Connection failed: {e}") + return None + + except TvDatafeedError as e: + print(f"[ERROR] TvDatafeed error: {e}") + return None + +# Usage +tv = TvDatafeed() + +# Valid symbol +df = safe_get_hist(tv, 'BTCUSDT', 'BINANCE', Interval.in_daily, 100) +if df is not None: + print(f"Got {len(df)} bars") + +# Invalid symbol +df = safe_get_hist(tv, 'INVALID', 'BINANCE', Interval.in_daily, 100) +``` + +### Retry Logic + +```python +import time +from tvDatafeed import TvDatafeed, Interval +from tvDatafeed.exceptions import WebSocketError, WebSocketTimeoutError + +def fetch_with_retry(tv, symbol, exchange, interval, n_bars, max_retries=3): + """Fetch data with automatic retry on network errors.""" + for attempt in range(max_retries): + try: + return tv.get_hist(symbol, exchange, interval, n_bars=n_bars) + except (WebSocketError, WebSocketTimeoutError) as e: + if attempt < max_retries - 1: + wait_time = 2 ** attempt # Exponential backoff + print(f"Attempt {attempt + 1} failed: {e}") + print(f"Retrying in {wait_time} seconds...") + time.sleep(wait_time) + else: + print(f"All {max_retries} attempts failed") + raise + +tv = TvDatafeed() +df = fetch_with_retry(tv, 'BTCUSDT', 'BINANCE', Interval.in_daily, 100) +``` + +## Logging Configuration + +### Enable Debug Logging + +```python +import logging +from tvDatafeed import TvDatafeed, Interval + +# Configure logging +logging.basicConfig( + level=logging.DEBUG, + format='%(asctime)s - %(name)s - %(levelname)s - %(message)s' +) + +# Create instance - will show debug info +tv = TvDatafeed() +df = tv.get_hist('BTCUSDT', 'BINANCE', Interval.in_daily, n_bars=10) +``` + +### Quiet Mode + +```python +from tvDatafeed import TvDatafeed, Interval + +# Suppress info messages, only show warnings and errors +tv = TvDatafeed(verbose=False) +df = tv.get_hist('BTCUSDT', 'BINANCE', Interval.in_daily, n_bars=100) +``` + +## Timeout Configuration + +### Custom Timeouts + +```python +from tvDatafeed import TvDatafeed, Interval + +# Longer timeout for slow connections +tv = TvDatafeed(ws_timeout=60.0) + +# Fetch large amount of data +df = tv.get_hist('BTCUSDT', 'BINANCE', Interval.in_1_minute, n_bars=5000) +print(f"Got {len(df)} bars") +``` + +### Environment Variable + +```python +import os +os.environ['TV_WS_TIMEOUT'] = '60.0' +os.environ['TV_MAX_RESPONSE_TIME'] = '120.0' + +from tvDatafeed import TvDatafeed, Interval + +# Uses environment variable values +tv = TvDatafeed() +df = tv.get_hist('BTCUSDT', 'BINANCE', Interval.in_1_minute, n_bars=5000) +``` diff --git a/docs/examples/date-range.md b/docs/examples/date-range.md new file mode 100644 index 0000000..1079f3c --- /dev/null +++ b/docs/examples/date-range.md @@ -0,0 +1,425 @@ +# Date Range Query Examples + +This guide shows how to fetch historical data for specific date ranges. + +## Basic Date Range Query + +### Simple Example + +```python +from datetime import datetime +from tvDatafeed import TvDatafeed, Interval + +tv = TvDatafeed() + +# Get data for January 2024 +df = tv.get_hist( + symbol='BTCUSDT', + exchange='BINANCE', + interval=Interval.in_1_hour, + start_date=datetime(2024, 1, 1), + end_date=datetime(2024, 1, 31) +) + +print(f"Got {len(df)} bars from {df.index[0]} to {df.index[-1]}") +print(df.head()) +``` + +## Date Range vs n_bars + +### When to Use Each + +```python +from datetime import datetime, timedelta +from tvDatafeed import TvDatafeed, Interval + +tv = TvDatafeed() + +# Use n_bars when you need a specific number of recent bars +df_nbars = tv.get_hist( + symbol='BTCUSDT', + exchange='BINANCE', + interval=Interval.in_daily, + n_bars=100 # Last 100 bars +) +print(f"n_bars: {len(df_nbars)} bars") + +# Use date range when you need a specific time period +df_range = tv.get_hist( + symbol='BTCUSDT', + exchange='BINANCE', + interval=Interval.in_daily, + start_date=datetime(2024, 1, 1), + end_date=datetime(2024, 3, 31) +) +print(f"Date range: {len(df_range)} bars") +``` + +!!! warning "Mutually Exclusive" + You cannot use both `n_bars` and date range. Choose one or the other. + +## Dynamic Date Ranges + +### Last N Days + +```python +from datetime import datetime, timedelta +from tvDatafeed import TvDatafeed, Interval + +tv = TvDatafeed() + +# Last 7 days +end_date = datetime.now() +start_date = end_date - timedelta(days=7) + +df = tv.get_hist( + symbol='BTCUSDT', + exchange='BINANCE', + interval=Interval.in_1_hour, + start_date=start_date, + end_date=end_date +) + +print(f"Last 7 days: {len(df)} hourly bars") +``` + +### Last N Months + +```python +from datetime import datetime +from dateutil.relativedelta import relativedelta +from tvDatafeed import TvDatafeed, Interval + +tv = TvDatafeed() + +# Last 3 months +end_date = datetime.now() +start_date = end_date - relativedelta(months=3) + +df = tv.get_hist( + symbol='BTCUSDT', + exchange='BINANCE', + interval=Interval.in_daily, + start_date=start_date, + end_date=end_date +) + +print(f"Last 3 months: {len(df)} daily bars") +``` + +### Year to Date + +```python +from datetime import datetime +from tvDatafeed import TvDatafeed, Interval + +tv = TvDatafeed() + +# YTD (Year to Date) +year = datetime.now().year +start_date = datetime(year, 1, 1) +end_date = datetime.now() + +df = tv.get_hist( + symbol='AAPL', + exchange='NASDAQ', + interval=Interval.in_daily, + start_date=start_date, + end_date=end_date +) + +print(f"YTD {year}: {len(df)} daily bars") +``` + +## Working with Different Timeframes + +### Intraday Data + +```python +from datetime import datetime +from tvDatafeed import TvDatafeed, Interval + +tv = TvDatafeed() + +# Get 1-minute data for a single day +df = tv.get_hist( + symbol='BTCUSDT', + exchange='BINANCE', + interval=Interval.in_1_minute, + start_date=datetime(2024, 1, 15, 0, 0), + end_date=datetime(2024, 1, 15, 23, 59) +) + +print(f"1-minute bars for Jan 15: {len(df)} bars") +print(f"Expected: ~1440 bars (24 hours * 60 minutes)") +``` + +### Hourly Data + +```python +from datetime import datetime +from tvDatafeed import TvDatafeed, Interval + +tv = TvDatafeed() + +# Get hourly data for a week +df = tv.get_hist( + symbol='BTCUSDT', + exchange='BINANCE', + interval=Interval.in_1_hour, + start_date=datetime(2024, 1, 1), + end_date=datetime(2024, 1, 7) +) + +print(f"Hourly bars for first week: {len(df)} bars") +print(f"Expected: ~168 bars (7 days * 24 hours)") +``` + +## Date Range Validation + +### Valid Date Range + +```python +from datetime import datetime +from tvDatafeed import TvDatafeed, Interval +from tvDatafeed.exceptions import DataValidationError + +tv = TvDatafeed() + +# Check validity before querying +start = datetime(2024, 1, 1) +end = datetime(2024, 1, 31) + +if tv.is_valid_date_range(start, end): + df = tv.get_hist( + symbol='BTCUSDT', + exchange='BINANCE', + interval=Interval.in_daily, + start_date=start, + end_date=end + ) + print(f"Got {len(df)} bars") +else: + print("Invalid date range!") +``` + +### Handling Validation Errors + +```python +from datetime import datetime +from tvDatafeed import TvDatafeed, Interval +from tvDatafeed.exceptions import DataValidationError + +tv = TvDatafeed() + +try: + # Invalid: start > end + df = tv.get_hist( + symbol='BTCUSDT', + exchange='BINANCE', + interval=Interval.in_daily, + start_date=datetime(2024, 12, 31), + end_date=datetime(2024, 1, 1) # Before start! + ) +except DataValidationError as e: + print(f"Validation error: {e}") + print(f"Field: {e.field}") + print(f"Value: {e.value}") + print(f"Reason: {e.reason}") +``` + +## Timezone Handling + +### Timezone Metadata + +```python +from datetime import datetime +from tvDatafeed import TvDatafeed, Interval + +tv = TvDatafeed() + +df = tv.get_hist( + symbol='BTCUSDT', + exchange='BINANCE', + interval=Interval.in_1_hour, + start_date=datetime(2024, 1, 1), + end_date=datetime(2024, 1, 7) +) + +# Access timezone metadata +timezone = df.attrs.get('timezone', 'Not set') +print(f"Timezone: {timezone}") +``` + +### Working with pytz + +```python +from datetime import datetime +import pytz +from tvDatafeed import TvDatafeed, Interval + +tv = TvDatafeed() + +# Create timezone-aware datetimes +eastern = pytz.timezone('US/Eastern') +start = eastern.localize(datetime(2024, 1, 1, 9, 30)) # Market open +end = eastern.localize(datetime(2024, 1, 5, 16, 0)) # Market close + +# Convert to UTC for API +start_utc = start.astimezone(pytz.UTC).replace(tzinfo=None) +end_utc = end.astimezone(pytz.UTC).replace(tzinfo=None) + +df = tv.get_hist( + symbol='AAPL', + exchange='NASDAQ', + interval=Interval.in_15_minute, + start_date=start_utc, + end_date=end_utc +) + +print(f"Got {len(df)} bars") +``` + +## Batch Queries + +### Multiple Date Ranges + +```python +from datetime import datetime +from dateutil.relativedelta import relativedelta +from tvDatafeed import TvDatafeed, Interval +import pandas as pd + +tv = TvDatafeed() + +# Get data for each quarter of 2023 +quarters = [ + (datetime(2023, 1, 1), datetime(2023, 3, 31), "Q1"), + (datetime(2023, 4, 1), datetime(2023, 6, 30), "Q2"), + (datetime(2023, 7, 1), datetime(2023, 9, 30), "Q3"), + (datetime(2023, 10, 1), datetime(2023, 12, 31), "Q4"), +] + +all_data = [] + +for start, end, name in quarters: + df = tv.get_hist( + symbol='BTCUSDT', + exchange='BINANCE', + interval=Interval.in_daily, + start_date=start, + end_date=end + ) + df['quarter'] = name + all_data.append(df) + print(f"{name}: {len(df)} bars") + +# Combine all quarters +full_year = pd.concat(all_data) +print(f"\nTotal 2023: {len(full_year)} bars") +``` + +### Monthly Data Collection + +```python +from datetime import datetime +from dateutil.relativedelta import relativedelta +from tvDatafeed import TvDatafeed, Interval +import pandas as pd + +tv = TvDatafeed() + +# Collect monthly data for a year +start_month = datetime(2023, 1, 1) +all_data = [] + +for i in range(12): + month_start = start_month + relativedelta(months=i) + month_end = month_start + relativedelta(months=1) - relativedelta(days=1) + + df = tv.get_hist( + symbol='AAPL', + exchange='NASDAQ', + interval=Interval.in_daily, + start_date=month_start, + end_date=month_end + ) + + all_data.append(df) + print(f"{month_start.strftime('%B %Y')}: {len(df)} bars") + +# Combine +full_year = pd.concat(all_data) +full_year = full_year[~full_year.index.duplicated()] # Remove duplicates +print(f"\nTotal: {len(full_year)} bars") +``` + +## Analysis Examples + +### Compare Periods + +```python +from datetime import datetime +from tvDatafeed import TvDatafeed, Interval + +tv = TvDatafeed() + +# Compare Q1 2023 vs Q1 2024 +q1_2023 = tv.get_hist( + symbol='BTCUSDT', + exchange='BINANCE', + interval=Interval.in_daily, + start_date=datetime(2023, 1, 1), + end_date=datetime(2023, 3, 31) +) + +q1_2024 = tv.get_hist( + symbol='BTCUSDT', + exchange='BINANCE', + interval=Interval.in_daily, + start_date=datetime(2024, 1, 1), + end_date=datetime(2024, 3, 31) +) + +# Calculate returns +ret_2023 = (q1_2023['close'].iloc[-1] / q1_2023['close'].iloc[0] - 1) * 100 +ret_2024 = (q1_2024['close'].iloc[-1] / q1_2024['close'].iloc[0] - 1) * 100 + +print("Q1 Performance Comparison:") +print(f" 2023: {ret_2023:+.2f}%") +print(f" 2024: {ret_2024:+.2f}%") +``` + +### Specific Event Analysis + +```python +from datetime import datetime, timedelta +from tvDatafeed import TvDatafeed, Interval + +tv = TvDatafeed() + +# Analyze price action around Bitcoin halving (approx April 2024) +event_date = datetime(2024, 4, 20) +days_before = 30 +days_after = 30 + +df = tv.get_hist( + symbol='BTCUSDT', + exchange='BINANCE', + interval=Interval.in_daily, + start_date=event_date - timedelta(days=days_before), + end_date=event_date + timedelta(days=days_after) +) + +# Find event position +event_idx = df.index.get_indexer([event_date], method='nearest')[0] + +# Before event +before = df.iloc[:event_idx] +# After event +after = df.iloc[event_idx:] + +print("Halving Analysis:") +print(f" 30 days before: ${before['close'].iloc[0]:,.0f} -> ${before['close'].iloc[-1]:,.0f}") +print(f" 30 days after: ${after['close'].iloc[0]:,.0f} -> ${after['close'].iloc[-1]:,.0f}") +``` diff --git a/docs/examples/live-feed.md b/docs/examples/live-feed.md new file mode 100644 index 0000000..8b3695c --- /dev/null +++ b/docs/examples/live-feed.md @@ -0,0 +1,416 @@ +# Live Data Feed Examples + +This guide shows how to use `TvDatafeedLive` for real-time data monitoring. + +## Basic Live Feed + +### Simple Callback + +```python +from tvDatafeed import TvDatafeedLive, Interval +import time + +def on_new_data(seis, data): + """Called when new bar is available.""" + close = data['close'].iloc[0] + volume = data['volume'].iloc[0] + print(f"[{seis.symbol}] Close: ${close:,.2f}, Volume: {volume:,.0f}") + +# Create live feed +tv = TvDatafeedLive() + +try: + # Add symbol to monitor + seis = tv.new_seis('BTCUSDT', 'BINANCE', Interval.in_1_minute) + + # Add callback + consumer = tv.new_consumer(seis, on_new_data) + + print(f"Monitoring {seis.symbol}... Press Ctrl+C to stop") + + # Keep running + while True: + time.sleep(1) + +except KeyboardInterrupt: + print("\nStopping...") + +finally: + # Always cleanup + tv.del_tvdatafeed() + print("Done") +``` + +## Multiple Symbols + +### Monitor Multiple Assets + +```python +from tvDatafeed import TvDatafeedLive, Interval +import time + +def create_handler(name): + """Create a handler with custom name for logging.""" + def handler(seis, data): + close = data['close'].iloc[0] + change = data['close'].iloc[0] - data['open'].iloc[0] + pct = (change / data['open'].iloc[0]) * 100 + print(f"[{name}] ${close:,.2f} ({pct:+.2f}%)") + return handler + +tv = TvDatafeedLive() + +try: + # Define symbols to monitor + symbols = [ + ('BTCUSDT', 'BINANCE', 'BTC'), + ('ETHUSDT', 'BINANCE', 'ETH'), + ('SOLUSDT', 'BINANCE', 'SOL'), + ] + + # Add each symbol + seises = [] + consumers = [] + + for symbol, exchange, name in symbols: + seis = tv.new_seis(symbol, exchange, Interval.in_1_minute) + consumer = tv.new_consumer(seis, create_handler(name)) + seises.append(seis) + consumers.append(consumer) + print(f"Added {name}") + + print("\nMonitoring... Press Ctrl+C to stop\n") + + while True: + time.sleep(1) + +except KeyboardInterrupt: + print("\nStopping...") + +finally: + tv.del_tvdatafeed() + print("Done") +``` + +## Multiple Callbacks per Symbol + +### Different Processing for Same Data + +```python +from tvDatafeed import TvDatafeedLive, Interval +import time +import logging + +logging.basicConfig(level=logging.INFO) + +def log_to_console(seis, data): + """Log to console.""" + print(f"[{seis.symbol}] {data.index[0]}: ${data['close'].iloc[0]:,.2f}") + +def check_alerts(seis, data): + """Check for price alerts.""" + close = data['close'].iloc[0] + + # Example alerts + if close > 70000: + print(f"*** ALERT: {seis.symbol} above $70,000! ***") + elif close < 60000: + print(f"*** ALERT: {seis.symbol} below $60,000! ***") + +def calculate_indicators(seis, data): + """Calculate technical indicators.""" + # This is simplified - in production, you'd maintain history + print(f"[{seis.symbol}] Processing for indicators...") + +tv = TvDatafeedLive() + +try: + seis = tv.new_seis('BTCUSDT', 'BINANCE', Interval.in_1_minute) + + # Multiple consumers for same seis + consumer1 = tv.new_consumer(seis, log_to_console) + consumer2 = tv.new_consumer(seis, check_alerts) + consumer3 = tv.new_consumer(seis, calculate_indicators) + + print("Monitoring with 3 callbacks... Press Ctrl+C to stop") + + while True: + time.sleep(1) + +except KeyboardInterrupt: + pass + +finally: + tv.del_tvdatafeed() +``` + +## Data Accumulation + +### Build Historical Data + +```python +from tvDatafeed import TvDatafeedLive, Interval +import pandas as pd +import time +import threading + +class DataAccumulator: + """Accumulate live data into a DataFrame.""" + + def __init__(self, max_rows=1000): + self.data = pd.DataFrame() + self.max_rows = max_rows + self.lock = threading.Lock() + + def callback(self, seis, data): + """Callback to accumulate data.""" + with self.lock: + self.data = pd.concat([self.data, data]) + + # Keep only last max_rows + if len(self.data) > self.max_rows: + self.data = self.data.iloc[-self.max_rows:] + + def get_data(self): + """Get accumulated data (thread-safe).""" + with self.lock: + return self.data.copy() + +# Create accumulator +accumulator = DataAccumulator(max_rows=100) + +tv = TvDatafeedLive() + +try: + seis = tv.new_seis('BTCUSDT', 'BINANCE', Interval.in_1_minute) + consumer = tv.new_consumer(seis, accumulator.callback) + + print("Accumulating data... Press Ctrl+C to stop") + + while True: + time.sleep(10) # Check every 10 seconds + df = accumulator.get_data() + if len(df) > 0: + print(f"\n--- Accumulated {len(df)} bars ---") + print(f"Latest: {df.iloc[-1]['close']:.2f}") + print(f"High: {df['high'].max():.2f}") + print(f"Low: {df['low'].min():.2f}") + +except KeyboardInterrupt: + pass + +finally: + tv.del_tvdatafeed() + + # Final data + final_df = accumulator.get_data() + print(f"\nFinal dataset: {len(final_df)} bars") + print(final_df.tail()) +``` + +## Fetching Historical Data While Live + +### Combined Usage + +```python +from tvDatafeed import TvDatafeedLive, Interval +import time + +def on_new_data(seis, data): + print(f"[LIVE] {seis.symbol}: ${data['close'].iloc[0]:,.2f}") + +tv = TvDatafeedLive() + +try: + # Start live monitoring + seis = tv.new_seis('BTCUSDT', 'BINANCE', Interval.in_1_minute) + consumer = tv.new_consumer(seis, on_new_data) + + print("Live feed started") + + # Can still fetch historical data while live feed is running + print("\nFetching historical data...") + df_hourly = tv.get_hist('BTCUSDT', 'BINANCE', Interval.in_1_hour, n_bars=24) + print(f"Got {len(df_hourly)} hourly bars") + + df_daily = tv.get_hist('BTCUSDT', 'BINANCE', Interval.in_daily, n_bars=30) + print(f"Got {len(df_daily)} daily bars") + + print("\nHistorical + Live data working together!") + + while True: + time.sleep(1) + +except KeyboardInterrupt: + pass + +finally: + tv.del_tvdatafeed() +``` + +## Error Handling in Callbacks + +### Safe Callback Pattern + +```python +from tvDatafeed import TvDatafeedLive, Interval +import logging +import time + +logging.basicConfig(level=logging.INFO) +logger = logging.getLogger(__name__) + +def safe_callback(seis, data): + """Callback with error handling.""" + try: + # Your logic here + close = data['close'].iloc[0] + + # Simulate potential error + if close > 70000: + # Do something that might fail + result = process_data(data) + save_to_database(result) + + print(f"[{seis.symbol}] Processed: ${close:,.2f}") + + except Exception as e: + # Log error but don't crash + logger.error(f"Error in callback for {seis.symbol}: {e}") + # Optionally send alert + # send_alert(f"Callback error: {e}") + +def process_data(data): + """Process data (may raise exception).""" + return data + +def save_to_database(data): + """Save to database (may raise exception).""" + pass + +tv = TvDatafeedLive() + +try: + seis = tv.new_seis('BTCUSDT', 'BINANCE', Interval.in_1_minute) + consumer = tv.new_consumer(seis, safe_callback) + + while True: + time.sleep(1) + +except KeyboardInterrupt: + pass + +finally: + tv.del_tvdatafeed() +``` + +## Using Timeouts + +### Non-Blocking Operations + +```python +from tvDatafeed import TvDatafeedLive, Interval +import time + +tv = TvDatafeedLive() + +def my_callback(seis, data): + print(f"New data: {data['close'].iloc[0]}") + +try: + # With 5-second timeout + seis = tv.new_seis('BTCUSDT', 'BINANCE', Interval.in_1_minute, timeout=5) + + if seis is False: + print("Failed to create seis within 5 seconds") + else: + consumer = tv.new_consumer(seis, my_callback, timeout=5) + + if consumer is False: + print("Failed to create consumer within 5 seconds") + else: + print("Success!") + + while True: + time.sleep(1) + +except KeyboardInterrupt: + pass + +finally: + tv.del_tvdatafeed() +``` + +## Cleanup Patterns + +### Removing Individual Components + +```python +from tvDatafeed import TvDatafeedLive, Interval +import time + +tv = TvDatafeedLive() + +def callback(seis, data): + print(f"{seis.symbol}: ${data['close'].iloc[0]:,.2f}") + +try: + # Create + seis = tv.new_seis('BTCUSDT', 'BINANCE', Interval.in_1_minute) + consumer = tv.new_consumer(seis, callback) + + time.sleep(30) # Monitor for 30 seconds + + # Remove just the consumer (seis keeps running) + tv.del_consumer(consumer) + print("Consumer removed") + + time.sleep(10) + + # Remove the seis (stops all monitoring for this symbol) + tv.del_seis(seis) + print("Seis removed") + +except KeyboardInterrupt: + pass + +finally: + # Full cleanup + tv.del_tvdatafeed() +``` + +### Using Seis Methods + +```python +from tvDatafeed import TvDatafeedLive, Interval +import time + +tv = TvDatafeedLive() + +def callback(seis, data): + print(f"{seis.symbol}: ${data['close'].iloc[0]:,.2f}") + +try: + seis = tv.new_seis('BTCUSDT', 'BINANCE', Interval.in_1_minute) + + # Create consumer via seis + consumer = seis.new_consumer(callback) + + time.sleep(30) + + # Get historical data via seis + df = seis.get_hist(n_bars=100) + print(f"Historical: {len(df)} bars") + + # Remove consumer via seis + seis.del_consumer(consumer) + + # Remove seis via seis + seis.del_seis() + +except KeyboardInterrupt: + pass + +finally: + tv.del_tvdatafeed() +``` diff --git a/docs/getting-started/authentication.md b/docs/getting-started/authentication.md new file mode 100644 index 0000000..f7361e5 --- /dev/null +++ b/docs/getting-started/authentication.md @@ -0,0 +1,238 @@ +# Authentication + +Authentication with TradingView provides access to more data and higher rate limits. + +## Authentication Methods + +### 1. Username/Password + +Basic authentication with TradingView credentials: + +```python +from tvDatafeed import TvDatafeed + +tv = TvDatafeed( + username='your_username', + password='your_password' +) +``` + +### 2. With 2FA (TOTP Secret) - Recommended + +If your account has 2FA enabled, use your TOTP secret for automatic code generation: + +```python +from tvDatafeed import TvDatafeed + +tv = TvDatafeed( + username='your_username', + password='your_password', + totp_secret='YOUR_BASE32_SECRET' # From authenticator setup +) +``` + +!!! tip "Finding Your TOTP Secret" + The TOTP secret is shown when you first set up 2FA in TradingView. + It's usually a base32-encoded string like `JBSWY3DPEHPK3PXP`. + +### 3. With 2FA (Manual Code) + +If you don't have the TOTP secret, you can provide the current code: + +```python +from tvDatafeed import TvDatafeed + +tv = TvDatafeed( + username='your_username', + password='your_password', + totp_code='123456' # Current 6-digit code +) +``` + +!!! warning "Code Expiration" + Manual codes expire quickly (usually 30 seconds). + Use `totp_secret` instead for automation. + +### 4. Pre-obtained Auth Token + +Use when CAPTCHA is required or for advanced use cases: + +```python +from tvDatafeed import TvDatafeed + +tv = TvDatafeed(auth_token='your_auth_token') +``` + +## Using Environment Variables + +### Recommended Setup + +Create a `.env` file (add to `.gitignore`!): + +```bash +# .env +TV_USERNAME=your_email@example.com +TV_PASSWORD=your_secure_password +TV_TOTP_SECRET=YOUR_BASE32_SECRET +``` + +Load and use: + +```python +import os +from dotenv import load_dotenv +from tvDatafeed import TvDatafeed + +load_dotenv() + +tv = TvDatafeed( + username=os.getenv('TV_USERNAME'), + password=os.getenv('TV_PASSWORD'), + totp_secret=os.getenv('TV_TOTP_SECRET') +) +``` + +### Available Environment Variables + +| Variable | Description | +|----------|-------------| +| `TV_USERNAME` | TradingView username | +| `TV_PASSWORD` | TradingView password | +| `TV_TOTP_SECRET` | TOTP secret for 2FA | +| `TV_2FA_CODE` | Manual 2FA code | + +## Handling CAPTCHA + +If TradingView requires CAPTCHA, you'll get a `CaptchaRequiredError`. + +### Workaround + +1. Log in to TradingView manually in your browser +2. Complete the CAPTCHA +3. Open DevTools (F12) > Application > Cookies +4. Find `authToken` cookie for tradingview.com +5. Copy the token value +6. Use it with TvDatafeed: + +```python +from tvDatafeed import TvDatafeed + +tv = TvDatafeed(auth_token='your_copied_auth_token') +``` + +## Handling 2FA Required Error + +```python +from tvDatafeed import TvDatafeed +from tvDatafeed.exceptions import TwoFactorRequiredError + +try: + tv = TvDatafeed(username='user', password='pass') +except TwoFactorRequiredError: + print("2FA required!") + + # Option 1: Provide TOTP secret + tv = TvDatafeed( + username='user', + password='pass', + totp_secret='YOUR_SECRET' + ) + + # Option 2: Ask for code + code = input("Enter 2FA code: ") + tv = TvDatafeed( + username='user', + password='pass', + totp_code=code + ) +``` + +## Security Best Practices + +### Do + +- Store credentials in environment variables +- Use `.env` files (add to `.gitignore`) +- Use `totp_secret` instead of `totp_code` for automation +- Regularly rotate passwords + +### Don't + +- Hardcode credentials in source code +- Commit `.env` files to version control +- Share TOTP secrets +- Log sensitive data + +### Example Secure Setup + +```python +import os +import logging +from dotenv import load_dotenv +from tvDatafeed import TvDatafeed + +# Configure logging to NOT show credentials +logging.basicConfig(level=logging.INFO) + +# Load from environment +load_dotenv() + +# Validate environment +required_vars = ['TV_USERNAME', 'TV_PASSWORD'] +missing = [v for v in required_vars if not os.getenv(v)] +if missing: + raise EnvironmentError(f"Missing: {missing}") + +# Create instance +tv = TvDatafeed( + username=os.getenv('TV_USERNAME'), + password=os.getenv('TV_PASSWORD'), + totp_secret=os.getenv('TV_TOTP_SECRET'), + verbose=False # Quiet mode for production +) + +# Use it +df = tv.get_hist('BTCUSDT', 'BINANCE', Interval.in_daily, n_bars=100) +``` + +## Troubleshooting + +### Invalid Credentials + +``` +AuthenticationError: Authentication failed: Invalid username or password +``` + +- Double-check username/password +- Try logging in manually on tradingview.com +- Check for typos in environment variables + +### 2FA Code Invalid + +``` +AuthenticationError: Invalid 2FA code +``` + +- Ensure your device time is synchronized +- TOTP codes are time-sensitive (30-second window) +- Verify the TOTP secret is correct + +### CAPTCHA Required + +``` +CaptchaRequiredError: TradingView requires CAPTCHA verification +``` + +- Log in manually and complete CAPTCHA +- Use auth_token workaround +- Wait and try again later + +### Account Locked + +``` +AuthenticationError: Account locked +``` + +- Wait 15-30 minutes +- Reset password if necessary +- Contact TradingView support diff --git a/docs/getting-started/installation.md b/docs/getting-started/installation.md new file mode 100644 index 0000000..adaa950 --- /dev/null +++ b/docs/getting-started/installation.md @@ -0,0 +1,98 @@ +# Installation + +## Requirements + +- Python 3.8 or higher +- pip package manager + +## Install from PyPI + +```bash +pip install tvdatafeed +``` + +## Install from Source + +```bash +git clone https://github.com/semoi/tvdatafeed.git +cd tvdatafeed +pip install -e . +``` + +## Optional Dependencies + +### 2FA Support (TOTP) + +For automatic 2FA code generation: + +```bash +pip install pyotp +``` + +### Environment Variables + +For loading credentials from `.env` files: + +```bash +pip install python-dotenv +``` + +### Development Dependencies + +For contributing to TvDatafeed: + +```bash +pip install -e ".[dev]" +``` + +This includes: +- pytest for testing +- pytest-cov for coverage +- flake8 for linting +- mypy for type checking + +## Verify Installation + +```python +from tvDatafeed import TvDatafeed, Interval + +# Should print the version +print(TvDatafeed.__module__) + +# Quick test +tv = TvDatafeed() +df = tv.get_hist('BTCUSDT', 'BINANCE', Interval.in_daily, n_bars=5) +print(df) +``` + +## Troubleshooting + +### ImportError + +If you get `ImportError: No module named 'tvDatafeed'`: + +```bash +# Make sure you're using the correct pip +python -m pip install tvdatafeed +``` + +### WebSocket Connection Issues + +If WebSocket connections fail: + +```bash +# Install websocket-client +pip install websocket-client +``` + +### SSL Certificate Errors + +On some systems, you may need to update certificates: + +```bash +# macOS +brew install openssl + +# Ubuntu/Debian +sudo apt-get install ca-certificates +``` diff --git a/docs/getting-started/quickstart.md b/docs/getting-started/quickstart.md new file mode 100644 index 0000000..534aed3 --- /dev/null +++ b/docs/getting-started/quickstart.md @@ -0,0 +1,154 @@ +# Quick Start + +This guide will get you up and running with TvDatafeed in minutes. + +## Basic Usage + +### 1. Create TvDatafeed Instance + +```python +from tvDatafeed import TvDatafeed, Interval + +# Unauthenticated (limited data) +tv = TvDatafeed() +``` + +### 2. Fetch Historical Data + +```python +# Get 100 daily bars for Bitcoin +df = tv.get_hist( + symbol='BTCUSDT', + exchange='BINANCE', + interval=Interval.in_daily, + n_bars=100 +) + +print(df.head()) +``` + +Output: +``` + symbol open high low close volume +datetime +2024-01-01 BINANCE:BTCUSDT 42000.0 42500.0 41500.0 42200.0 1234567890.0 +2024-01-02 BINANCE:BTCUSDT 42200.0 43000.0 42000.0 42800.0 1345678901.0 +... +``` + +### 3. Search for Symbols + +```python +# Find Bitcoin symbols on Binance +results = tv.search_symbol('BTC', 'BINANCE') + +for r in results[:5]: + print(f"{r['exchange']}:{r['symbol']} - {r['description']}") +``` + +## Different Timeframes + +```python +# 1 minute bars +df_1m = tv.get_hist('BTCUSDT', 'BINANCE', Interval.in_1_minute, n_bars=100) + +# 1 hour bars +df_1h = tv.get_hist('BTCUSDT', 'BINANCE', Interval.in_1_hour, n_bars=100) + +# Daily bars +df_1d = tv.get_hist('BTCUSDT', 'BINANCE', Interval.in_daily, n_bars=100) + +# Weekly bars +df_1w = tv.get_hist('BTCUSDT', 'BINANCE', Interval.in_weekly, n_bars=100) +``` + +## Different Asset Classes + +### Stocks + +```python +# Apple on NASDAQ +df = tv.get_hist('AAPL', 'NASDAQ', Interval.in_daily, n_bars=100) + +# Microsoft +df = tv.get_hist('MSFT', 'NASDAQ', Interval.in_daily, n_bars=100) +``` + +### Crypto + +```python +# Bitcoin +df = tv.get_hist('BTCUSDT', 'BINANCE', Interval.in_1_hour, n_bars=100) + +# Ethereum +df = tv.get_hist('ETHUSDT', 'BINANCE', Interval.in_1_hour, n_bars=100) +``` + +### Forex + +```python +# EUR/USD +df = tv.get_hist('EURUSD', 'FX', Interval.in_daily, n_bars=100) + +# GBP/USD +df = tv.get_hist('GBPUSD', 'FX', Interval.in_daily, n_bars=100) +``` + +### Futures + +```python +# Crude Oil continuous front month +df = tv.get_hist('CRUDEOIL', 'MCX', Interval.in_daily, n_bars=100, fut_contract=1) + +# Nifty futures +df = tv.get_hist('NIFTY', 'NSE', Interval.in_daily, n_bars=100, fut_contract=1) +``` + +## Date Range Queries + +```python +from datetime import datetime + +# Get data for specific date range +df = tv.get_hist( + symbol='BTCUSDT', + exchange='BINANCE', + interval=Interval.in_1_hour, + start_date=datetime(2024, 1, 1), + end_date=datetime(2024, 1, 31) +) + +print(f"Got {len(df)} bars from {df.index[0]} to {df.index[-1]}") +``` + +## Extended Trading Hours + +```python +# Include pre-market and after-hours data +df = tv.get_hist( + symbol='AAPL', + exchange='NASDAQ', + interval=Interval.in_15_minute, + n_bars=100, + extended_session=True +) +``` + +## Error Handling + +```python +from tvDatafeed.exceptions import DataNotFoundError, WebSocketError + +try: + df = tv.get_hist('INVALID', 'BINANCE', Interval.in_daily) +except DataNotFoundError as e: + print(f"Symbol not found: {e}") +except WebSocketError as e: + print(f"Connection error: {e}") +``` + +## Next Steps + +- [Authentication Guide](authentication.md) - Get more data with a TradingView account +- [API Reference](../api/index.md) - Complete API documentation +- [Examples](../examples/basic.md) - More code examples diff --git a/docs/index.md b/docs/index.md new file mode 100644 index 0000000..c89a56e --- /dev/null +++ b/docs/index.md @@ -0,0 +1,113 @@ +# TvDatafeed Documentation + +**TvDatafeed** is a Python library for fetching historical and real-time market data from TradingView. + +## Features + +- **Historical Data**: Retrieve up to 5000 bars of OHLCV data +- **Real-time Data**: Monitor multiple symbols with live updates +- **2FA Support**: Full support for two-factor authentication (TOTP) +- **Date Range Queries**: Fetch data for specific time periods +- **Multiple Timeframes**: Minutes, hours, days, weeks, and months +- **Threading Support**: Process live data with callback functions + +## Quick Start + +### Installation + +```bash +pip install tvdatafeed +``` + +### Basic Usage + +```python +from tvDatafeed import TvDatafeed, Interval + +# Create instance (unauthenticated - limited data) +tv = TvDatafeed() + +# Get historical data +df = tv.get_hist( + symbol='BTCUSDT', + exchange='BINANCE', + interval=Interval.in_1_hour, + n_bars=100 +) + +print(df.head()) +``` + +### With Authentication + +```python +from tvDatafeed import TvDatafeed, Interval + +# Authenticated access +tv = TvDatafeed( + username='your_username', + password='your_password' +) + +# Get more data with authenticated access +df = tv.get_hist('AAPL', 'NASDAQ', Interval.in_daily, n_bars=1000) +``` + +### With 2FA + +```python +from tvDatafeed import TvDatafeed, Interval + +# With TOTP secret (recommended) +tv = TvDatafeed( + username='your_username', + password='your_password', + totp_secret='YOUR_TOTP_SECRET' # From authenticator setup +) + +# Or with manual code +tv = TvDatafeed( + username='your_username', + password='your_password', + totp_code='123456' # Current 6-digit code +) +``` + +## Supported Intervals + +| Interval | Description | +|----------|-------------| +| `Interval.in_1_minute` | 1 minute | +| `Interval.in_3_minute` | 3 minutes | +| `Interval.in_5_minute` | 5 minutes | +| `Interval.in_15_minute` | 15 minutes | +| `Interval.in_30_minute` | 30 minutes | +| `Interval.in_45_minute` | 45 minutes | +| `Interval.in_1_hour` | 1 hour | +| `Interval.in_2_hour` | 2 hours | +| `Interval.in_3_hour` | 3 hours | +| `Interval.in_4_hour` | 4 hours | +| `Interval.in_daily` | Daily | +| `Interval.in_weekly` | Weekly | +| `Interval.in_monthly` | Monthly | + +## Environment Variables + +TvDatafeed supports configuration via environment variables: + +| Variable | Description | Default | +|----------|-------------|---------| +| `TV_USERNAME` | TradingView username | None | +| `TV_PASSWORD` | TradingView password | None | +| `TV_TOTP_SECRET` | TOTP secret for 2FA | None | +| `TV_2FA_CODE` | Manual 2FA code | None | +| `TV_WS_TIMEOUT` | WebSocket timeout (seconds) | 5 | +| `TV_MAX_RESPONSE_TIME` | Max response wait time | 60 | +| `TV_VERBOSE` | Enable verbose logging | true | + +## Next Steps + +- [Installation Guide](getting-started/installation.md) +- [Quick Start Tutorial](getting-started/quickstart.md) +- [API Reference](api/index.md) +- [Examples](examples/basic.md) diff --git a/mkdocs.yml b/mkdocs.yml new file mode 100644 index 0000000..03098c4 --- /dev/null +++ b/mkdocs.yml @@ -0,0 +1,110 @@ +site_name: TvDatafeed Documentation +site_description: Python library to fetch historical and real-time data from TradingView +site_author: TvDatafeed Contributors +site_url: https://tvdatafeed.readthedocs.io/ + +repo_name: tvdatafeed +repo_url: https://github.com/semoi/tvdatafeed + +theme: + name: material + language: en + palette: + - scheme: default + primary: blue + accent: blue + toggle: + icon: material/brightness-7 + name: Switch to dark mode + - scheme: slate + primary: blue + accent: blue + toggle: + icon: material/brightness-4 + name: Switch to light mode + features: + - navigation.tabs + - navigation.sections + - navigation.top + - navigation.expand + - search.suggest + - search.highlight + - content.tabs.link + - content.code.copy + - content.code.annotate + icon: + repo: fontawesome/brands/github + +plugins: + - search: + separator: '[\s\-,:!=\[\]()"`/]+|\.(?!\d)|&[lg]t;|(?!\b)(?=[A-Z][a-z])' + - minify: + minify_html: true + +markdown_extensions: + - abbr + - admonition + - attr_list + - def_list + - footnotes + - md_in_html + - toc: + permalink: true + - pymdownx.arithmatex: + generic: true + - pymdownx.betterem: + smart_enable: all + - pymdownx.caret + - pymdownx.details + - pymdownx.emoji: + emoji_index: !!python/name:material.extensions.emoji.twemoji + emoji_generator: !!python/name:material.extensions.emoji.to_svg + - pymdownx.highlight: + anchor_linenums: true + line_spans: __span + pygments_lang_class: true + - pymdownx.inlinehilite + - pymdownx.keys + - pymdownx.magiclink: + repo_url_shorthand: true + user: semoi + repo: tvdatafeed + - pymdownx.mark + - pymdownx.smartsymbols + - pymdownx.superfences: + custom_fences: + - name: mermaid + class: mermaid + format: !!python/name:pymdownx.superfences.fence_code_format + - pymdownx.tabbed: + alternate_style: true + - pymdownx.tasklist: + custom_checkbox: true + - pymdownx.tilde + +nav: + - Home: index.md + - Getting Started: + - Installation: getting-started/installation.md + - Quick Start: getting-started/quickstart.md + - Authentication: getting-started/authentication.md + - API Reference: + - Overview: api/index.md + - TvDatafeed: api/tvdatafeed.md + - TvDatafeedLive: api/tvdatafeedlive.md + - Helper Classes: api/helpers.md + - Exceptions: api/exceptions.md + - Configuration: api/configuration.md + - Examples: + - Basic Usage: examples/basic.md + - 2FA Authentication: examples/2fa.md + - Live Data Feed: examples/live-feed.md + - Date Range Queries: examples/date-range.md + +extra: + social: + - icon: fontawesome/brands/github + link: https://github.com/semoi/tvdatafeed + generator: false + +copyright: Copyright © 2024 TvDatafeed Contributors