diff --git a/docs/plans/2026-01-22-tywrapped-ecosystem-vision.md b/docs/plans/2026-01-22-tywrapped-ecosystem-vision.md new file mode 100644 index 0000000..b0a9bf1 --- /dev/null +++ b/docs/plans/2026-01-22-tywrapped-ecosystem-vision.md @@ -0,0 +1,378 @@ +# RFC: @tywrapped Ecosystem + +| | | +|---|---| +| **Status** | Draft | +| **Created** | 2026-01-22 | +| **Author** | @bbopen | + +## Summary + +Create a **curated ecosystem of pre-wrapped Python libraries** published to npm under the `@tywrapped` organization, making world-class Python libraries accessible to JavaScript/TypeScript developers who don't know Python. + +## Motivation + +JavaScript/TypeScript developers often need access to Python libraries that are best-in-class for their domain (PyTorch, NumPy, SciPy, scikit-learn). Currently, they must either: +1. Learn Python and manage a Python environment +2. Use inferior JS alternatives +3. Set up tywrap themselves + +This RFC proposes a community-governed ecosystem of pre-built, tested, and versioned wrappers that "just work" via `npm install`. + +## Goals + +1. **Adoption friction** - Lower the barrier by providing ready-to-use npm packages instead of requiring users to run tywrap themselves +2. **Quality assurance** - Ensure high-quality, tested wrappers through governance and CI gates +3. **Discovery/Marketing** - Make tywrap visible in the npm ecosystem through packages like `@tywrapped/pytorch` +4. **Ecosystem sustainability** - Create a network effect where these become the "blessed" way to use Python libs from JS + +## Target Audience + +**JS/TS developers who don't know Python** and need access to libraries that are: +- Best-in-class for their domain (not just "best in Python") +- Have no well-maintained JS equivalent +- The primary project doesn't provide a JS SDK + +## Unique Value Proposition + +Tywrap is unique in the ecosystem: + +| Project | Approach | Types | Existing Libraries | +|---------|----------|-------|-------------------| +| **tywrap** | Analyzes Python source → generates TS wrappers | Auto-generated | Yes, directly wraps numpy/pandas/etc. | +| **PyBridge** | Manual bridge | Manual TS interfaces | Requires wrapper functions | +| **pymport** | Embeds Python in-process | Minimal | Weak typing | +| **node-calls-python** | In-process execution | Basic conversions | No real TS types | + +**Tywrap's unique combination:** +1. Auto-generates TypeScript types from Python source analysis +2. Wraps existing libraries directly (no wrapper functions needed) +3. Multi-runtime (Node.js subprocess AND browser via Pyodide) +4. Rich data science type mappings (numpy, pandas, torch, scipy, sklearn) + +--- + +## Governance Model + +### Organization Structure + +- **Central branded org**: `@tywrapped` on npm, `tywrapped` on GitHub +- **Nonprofit/community-driven** structure that can receive sponsorship +- **Technical committee** provides governance and quality oversight +- Trust established via the central org, not individual maintainers + +### Maintainer Model + +- Trusted maintainers assigned to specific libraries +- Maintainers approve releases for their libraries +- Outreach to upstream Python projects welcomed but not required + +### Upstream Relationship + +| Level | Description | +|-------|-------------| +| **Awareness** | Default - notify upstream, they know we exist | +| **Collaboration** | Optional - they contribute to wrapper design | +| **Ownership** | Optional - upstream joins tywrap community as maintainer | + +Upstream can "come to us" rather than requiring us to convince them. + +--- + +## Package Naming + +**Pattern**: `@tywrapped/` + +Examples: +- `@tywrapped/numpy` +- `@tywrapped/pytorch` +- `@tywrapped/scipy` +- `@tywrapped/beautifulsoup4` + +**Rationale**: +- Scoped under org we control - prevents squatting +- Follows `@types/*` precedent (DefinitelyTyped) +- Clear this is community wrapper, not official +- Nominative fair use for library names + +**Required disclaimer in each package**: +> This is a community-maintained wrapper and is not affiliated with or endorsed by [upstream project]. + +--- + +## Versioning Strategy + +**Pattern**: Mirror upstream by default, compound version for tywrap patches. + +| Scenario | Version | +|----------|---------| +| Wraps numpy 1.26.0 | `@tywrapped/numpy@1.26.0` | +| Tywrap fix for same numpy | `@tywrapped/numpy@1.26.0-tywrap.1` | +| New numpy release | `@tywrapped/numpy@1.26.1` (resets suffix) | + +**Republish policy**: Every tywrap core release triggers regeneration and republish of all active packages with compound version. Users on any upstream version benefit from improvements. + +**Version support breadth**: Separate indicator from quality tiers. Packages aim to match upstream's official support matrix (e.g., if numpy supports 1.24+, so does `@tywrapped/numpy`). + +**Breaking changes**: Mirror upstream directly. No buffering, no parallel support periods. The wrapper is a transparent pass-through - breaking changes in numpy mean breaking changes in `@tywrapped/numpy`. Document this clearly so users understand they're accepting upstream's upgrade path. + +--- + +## Template Repository Structure + +### Standardized Core (Fixed) + +- GitHub Actions workflows (CI/CD) +- npm publish pipeline +- Health badge generation +- Dependabot configuration +- `tywrap.config.ts` structure +- CONTRIBUTING guidelines +- License (MIT) +- Disclaimer template + +### Flexible Edges (Customizable) + +- Library-specific integration tests +- Usage examples +- Extended documentation +- Custom TypeScript utilities on top of generated code + +--- + +## CI/CD Pipeline + +### Triggers + +Releases triggered by **either**: +1. **Upstream Python release** - Dependabot detects new version +2. **Tywrap release** - New tywrap improves codegen + +### Pipeline Flow + +``` +1. Trigger (Dependabot or tywrap release) + ↓ +2. CI regenerates wrappers + ↓ +3. Run test suite (based on health tier) + ↓ +4. If tests pass → Create PR / request approval + ↓ +5. Library maintainer reviews/approves + ↓ +6. Publish to npm +``` + +### Human Gate + +Library maintainer must approve before publish - automation does the heavy lifting, humans ensure quality. + +--- + +## Package Health Indicators + +Two separate badge systems for orthogonal concerns: + +### Quality Tier (testing depth) + +| Tier | Badge | Requirement | +|------|-------|-------------| +| 🥉 Bronze | Generation + types | Wrappers generate, TypeScript compiles | +| 🥈 Silver | Smoke tests | Basic "does it call Python and return" tests | +| 🥇 Gold | Integration tests | Extensive API surface coverage | +| 💎 Platinum | Upstream parity | Ported subset of Python test suite | + +### Support Breadth (version coverage) + +| Level | Indicator | Meaning | +|-------|-----------|---------| +| Basic | `latest` | Only current upstream version | +| Standard | `latest+1` | Latest + previous major | +| Full | `upstream` | Matches upstream's support matrix | +| Extended | `upstream+LTS` | Upstream support + extended LTS | + +### Display Format + +README example: +``` +@tywrapped/numpy +Quality: 🥇 Gold | Support: upstream (1.24+) | Runtimes: node, browser +``` + +### Progression Path + +- All packages start at Bronze quality, Basic support +- Maintainers level up through added tests and version support +- LLM-assisted tooling can help port Python tests to TypeScript +- Health indicators create clear upgrade path and contribution opportunities + +**Visibility**: Prominent in README, npm description, and searchable in package registry. + +--- + +## Runtime Selection & Browser Support + +### Auto-detection with Explicit Override + +**Default behavior (zero config):** +- Browser environment → Pyodide runtime +- Node.js environment → Subprocess runtime (safest, no native deps) + +**Override for power users:** +```typescript +import { setRuntime, InProcessBridge } from '@tywrapped/numpy'; +setRuntime(new InProcessBridge()); // Use node-calls-python for speed +``` + +### Runtime Packages + +- `@tywrap/runtime-subprocess` - Bundled by default, no native dependencies +- `@tywrap/runtime-inprocess` - Optional, requires node-calls-python native addon +- `@tywrap/runtime-pyodide` - Auto-loaded in browser environments + +### Browser Support + +Opt-in per package, clearly marked. + +Packages declare supported runtimes in metadata: +- `runtimes: ["node"]` - Node.js only (heavy libraries like PyTorch) +- `runtimes: ["node", "browser"]` - Both supported + +Visible in README badge, npm package metadata, and package documentation. + +--- + +## Initial Launch Libraries + +Criteria for selection: +- High demand from JS developers +- No good existing JS alternative +- Reasonable API surface +- Good upstream test coverage + +**Candidates**: + +| Library | Domain | JS Alternative? | Priority | +|---------|--------|-----------------|----------| +| NumPy | Numerical computing | Limited (ndarray-js) | High | +| SciPy | Scientific computing | None comprehensive | High | +| PyTorch | ML/Deep learning | TensorFlow.js (different) | High | +| scikit-learn | Classical ML | None good | Medium | +| BeautifulSoup4 | HTML parsing | Cheerio (different API) | Medium | +| Pandas | Data manipulation | Danfo.js (less mature) | Medium | + +--- + +## Funding & Brand Protection + +### Funding Model: Open Collective + +- Transparent finances - all income and expenses visible to community +- Fiscal host handles legal and tax compliance +- Proven model used by webpack, Babel, Vue, and similar projects +- Funds distributed to maintainers based on contribution +- Sponsors get visibility and community goodwill + +**Fund allocation** (suggested starting point): +- 60% - Maintainer stipends (proportional to package maintenance) +- 20% - Infrastructure costs (CI, hosting, tooling) +- 15% - Bounties for new packages, test improvements, documentation +- 5% - Reserve for legal/unexpected expenses + +### Trademark: Register Early + +- File trademark application before public launch +- Estimated cost: $250-400 per class +- Protects brand from squatting +- Establishes legitimacy for enterprise adoption +- Required classes: software development tools, software distribution + +**Brand assets to protect**: +- "tywrapped" name +- `@tywrapped` npm scope (already protected by npm org ownership) +- Logo (when created) +- Domain name (tywrapped.org or similar) + +--- + +## Runtime Abstraction Feasibility + +### Current Architecture (Already Pluggable!) + +The tywrap codebase already has a clean transport abstraction: + +``` +┌─────────────────────────────────────────┐ +│ Transport Interface │ +│ init() | send() | dispose() | isReady │ +└─────────────────┬───────────────────────┘ + │ implements + ┌───────────┼───────────┐ + ▼ ▼ ▼ + ProcessIO HttpIO PyodideIO + (subprocess) (HTTP) (WASM) +``` + +**Key interface** (`src/runtime/transport.ts`): +```typescript +interface Transport extends Disposable { + init(): Promise; + send(message: string, timeoutMs: number, signal?: AbortSignal): Promise; + readonly isReady: boolean; +} +``` + +`BridgeProtocol` is already transport-agnostic - it accepts any `Transport` implementation. + +### Adding node-calls-python Support + +**Recommended approach: JSON passthrough** + +1. Create `NodeCallsPythonIO implements Transport` +2. Keep the JSON protocol (parse incoming JSON, call Python via node-calls-python, serialize result) +3. Estimated ~200-300 lines of code +4. Small JSON overhead, but simpler integration and maintains consistency + +### Implementation Estimate + +| Task | Effort | +|------|--------| +| `NodeCallsPythonIO` transport class | 2-3 days | +| Bridge adapter for node-calls-python | 1 day | +| Integration tests | 1-2 days | +| Documentation | 0.5 day | +| **Total** | **~1 week** | + +### Conclusion + +**No architectural changes needed.** The runtime abstraction already exists. Adding new runtimes is a matter of implementing the `Transport` interface. + +--- + +## Next Steps + +### Phase 1: Foundation (before launch) +1. Register "tywrapped" trademark +2. Set up GitHub org (`tywrapped`) and npm org (`@tywrapped`) +3. Create Open Collective account +4. Secure domain (tywrapped.org or similar) + +### Phase 2: Template & Tooling +5. Create template repository with standardized CI/CD +6. Build badge generation tooling for health indicators +7. Document governance charter and contribution guidelines + +### Phase 3: Proof of Concept +8. Build `@tywrapped/numpy` as first package +9. Validate CI/CD pipeline with real upstream updates +10. Gather feedback, iterate on template + +### Phase 4: Expansion +11. Add 2-3 more high-priority packages (scipy, scikit-learn) +12. Recruit initial maintainers +13. Announce publicly, begin community building + +### Optional (parallel track) +- Prototype `NodeCallsPythonIO` transport for performance optimization +- Explore LLM-assisted test porting from Python to TypeScript