Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions connector-registry/this-is-a-test/_meta/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# this-is-a-test (Registry)

This is the top-level registry entry for `this-is-a-test`.

- Versions live under `this-is-a-test/v1`
- See author implementations under `this-is-a-test/v1/514-labs`
9 changes: 9 additions & 0 deletions connector-registry/this-is-a-test/_meta/connector.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
{
"$schema": "https://schemas.connector-factory.dev/connector-root.schema.json",
"identifier": "this-is-a-test",
"name": "DFSFSDFSDF",
"category": "api",
"tags": ["DSDFSF"],
"description": "FSDFSDFSDFSFSDFSDFSDF",
"homepage": "https://openweathermap.org"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Changelog

All notable changes to this connector will be documented in this file.

## 0.1.0 - Scaffold created

- Initial scaffold for `this-is-a-test` (TypeScript) added.
18 changes: 18 additions & 0 deletions connector-registry/this-is-a-test/v1/514-labs/_meta/LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
MIT License

Copyright (c) 514-labs

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
10 changes: 10 additions & 0 deletions connector-registry/this-is-a-test/v1/514-labs/_meta/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# this-is-a-test Connector (by 514-labs)

This directory contains language-agnostic metadata and documentation for the `this-is-a-test` connector.

- Name: `this-is-a-test`
- Author: `514-labs`
- Category: `api`
- Languages: `typescript`

See `_meta/connector.json` for connector metadata and implementation folders for language-specific code.
22 changes: 22 additions & 0 deletions connector-registry/this-is-a-test/v1/514-labs/_meta/connector.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
{
"name": "this-is-a-test",
"author": "514-labs",
"version": "v1",
"languages": [
"typescript"
],
"category": "api",
"capabilities": {
"extract": true,
"transform": false,
"load": false
},
"source": {
"type": "api",
"spec": "",
"homepage": ""
},
"tags": [],
"maintainers": [],
"issues": { "typescript": { "default": "" } }
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{
"identifier": "this-is-a-test",
"name": "this-is-a-test",
"author": "514-labs",
"version": "v1",
"language": "typescript",
"implementations": ["default"]
}
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
API_KEY=
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
node_modules/
dist/
.env
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
# Connector Implementation Guide

This guide walks you through implementing your `this-is-a-test` connector from scaffold to production.

**📖 For detailed step-by-step instructions, see:**
`connector-registry/_scaffold/CONNECTOR_IMPLEMENTATION_GUIDE.md`

Or view online: [Connector Implementation Guide](../../../../../../_scaffold/CONNECTOR_IMPLEMENTATION_GUIDE.md)

## Quick Checklist

### Phase 1: Understand Your API ✓
- [ ] Read the `this-is-a-test` API documentation
- [ ] Identify authentication method
- [ ] Understand pagination pattern
- [ ] Note rate limits
- [ ] Identify key endpoints

### Phase 2: Scaffold ✓
- [x] Connector scaffolded

### Phase 3: Configure Authentication
- [ ] Update `ConnectorConfig` type in `src/client/connector.ts`
- [ ] Implement `init()` method to set up auth
- [ ] Update `.env.example` with required credentials

### Phase 4: Implement Pagination
- [ ] Choose pagination pattern (offset/cursor/page)
- [ ] Implement in `src/lib/paginate.ts`
- [ ] Test with API to verify it works

### Phase 5: Implement Resources
- [ ] Define resource types in `src/resources/data.ts`
- [ ] Implement list/get/create methods as needed
- [ ] Add resource getters to connector class

### Phase 6: Add Schemas
- [ ] Create JSON schemas in `schemas/raw/json/`
- [ ] Add documentation `.md` files
- [ ] Register in `schemas/index.json`

### Phase 7: Update Documentation
- [ ] Update `README.md` with examples
- [ ] Document config in `docs/configuration.md`
- [ ] Update `.env.example`

### Phase 8: Write Tests
- [ ] Update `tests/resource.test.ts`
- [ ] Add observability tests if using logging/metrics
- [ ] Test pagination edge cases

### Phase 9: Build & Test
- [ ] Run `pnpm run build` - should complete without errors
- [ ] Run `pnpm test` - all tests should pass
- [ ] Try `examples/basic-usage.ts` with real API

### Phase 10: Real-World Testing
- [ ] Test with real credentials
- [ ] Verify pagination with large datasets
- [ ] Test error handling
- [ ] Test rate limiting (if applicable)

## Common Patterns

See the detailed guide for code examples of:
- Simple API key authentication
- Bearer token authentication
- OAuth2 authentication
- Offset-based pagination
- Cursor-based pagination
- Page number pagination

## Reference Connectors

Look at these for examples:
- **Socrata** (`connector-registry/socrata/`): Simple API with offset pagination
- **Meta Ads** (`connector-registry/meta-ads/`): OAuth2 with cursor pagination
- **Google Analytics** (`connector-registry/google-analytics/`): Service account auth

## Need Help?

Refer to the full implementation guide at:
`connector-registry/_scaffold/CONNECTOR_IMPLEMENTATION_GUIDE.md`
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# this-is-a-test (TypeScript)

TypeScript implementation for `this-is-a-test` by `514-labs`.

> **📖 Implementation Guide**: See `CONNECTOR_GUIDE.md` for step-by-step instructions on implementing this connector.

Schemas: see `schemas/index.json` for machine-readable definitions and accompanying Markdown docs.
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Configuration

- baseUrl, timeoutMs, userAgent
- auth: api key, basic, bearer, oauth2 (per core)
- retry: maxAttempts, initialDelayMs, maxDelayMs, backoffMultiplier, respectRetryAfter
- rateLimit: requestsPerSecond, burstCapacity, adaptiveFromHeaders
- hooks: arrays for beforeRequest/afterResponse/onError/onRetry
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Getting started

```ts
import { createConnector } from './src'

const conn = createConnector()
conn.initialize({
baseUrl: 'https://api.example.com',
auth: { type: 'bearer', bearer: { token: process.env.API_TOKEN! } },
logging: { enabled: true, level: 'info' },
})

for await (const page of conn.myResource.getAll({ pageSize: 100, maxItems: 500 })) {
console.log(page.length)
}
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Limits

Describe API limits, quotas, and rate limiting behavior.
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Observability

## Logging
```ts
conn.initialize({
logging: { enabled: true, level: 'info', includeQueryParams: true, includeHeaders: true, includeBody: false }
})
```
Emits http_request and http_response with optional fields.

## Metrics
```ts
import { createMetricsHooks, InMemoryMetricsSink } from '../src/observability/metrics-hooks'
const sink = new InMemoryMetricsSink()
conn.initialize({ hooks: createMetricsHooks(sink) })
```
Inspect sink.events for telemetry.
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Schema

Refer to `schemas/index.json` and related files.

## Organization

Schemas support nested folder structures for better organization:

- **raw/** - Raw API schemas
- `endpoints/` - API endpoint request/response schemas
- `types/` - Shared type definitions for endpoints (not shown as Files)
- `events/` - Event payloads
- **extracted/** - Normalized schemas
- `entities/` - Business entities
- `metrics/` - Aggregated data
- **files/** - File-based schemas (CSV/JSON/Parquet/Avro/NDJSON). Only items under `schemas/files` appear in the Files tab.

## Adding Schemas

1. Create schema files in appropriate nested folders
2. Update `schemas/index.json` with correct paths for endpoints and tables
3. Place file schemas under `schemas/files` (no index entries needed)
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
import { createConnector } from '../src'

const conn = createConnector()
conn.initialize({
baseUrl: 'https://api.example.com',
auth: { type: 'bearer', bearer: { token: 'YOUR_TOKEN' } },
})

console.log('Connector initialized')

// Example usage:
// for await (const page of conn.data.getAll({ pageSize: 100, maxItems: 200 })) {
// console.log(page)
// }
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
language = "typescript"
description = "TypeScript connector for this-is-a-test"
post_install_print = """
🚀 Next steps to get your this-is-a-test connector running:

📖 IMPORTANT: Read the implementation guide first!
$ cat CONNECTOR_GUIDE.md
Or open: connector-registry/_scaffold/CONNECTOR_IMPLEMENTATION_GUIDE.md

📂 Go to your connector directory:
$ cd

📦 Install dependencies:
$ npm install

🔧 Build the connector:
$ npm run build

🧪 Run tests:
$ npm test

📋 Follow the checklist in CONNECTOR_GUIDE.md:
Phase 3: Configure authentication in src/client/connector.ts
Phase 4: Implement pagination in src/lib/paginate.ts
Phase 5: Implement resources in src/resources/
Phase 6-10: Add schemas, docs, tests

💡 Reference connectors for examples:
• connector-registry/socrata/ (simple API key + offset pagination)
• connector-registry/meta-ads/ (OAuth2 + cursor pagination)
• connector-registry/google-analytics/ (service account auth)
"""
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
module.exports = {
testEnvironment: 'node',
transform: { '^.+\.tsx?$': ['ts-jest', { tsconfig: 'tsconfig.test.json' }] },
};
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
{
"name": "@workspace/connector-this-is-a-test",
"version": "0.0.0",
"private": true,
"main": "dist/index.js",
"types": "dist/index.d.ts",
"scripts": {
"build": "tsc -p tsconfig.json",
"test": "jest --runInBand"
},
"engines": {"node": ">=20"},
"dependencies": {
"@connector-factory/core": "^0.1.0"
},
"devDependencies": {
"typescript": "^5.4.0",
"jest": "^29.7.0",
"ts-jest": "^29.1.2",
"@types/jest": "^29.5.12",
"@types/node": "^20.11.30",
"nock": "^13.5.0"
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Extracted JSON Schemas

This directory contains normalized/transformed JSON schemas.

## Organization

Like raw schemas, extracted schemas can be organized in nested folders:
- `entities/` - Normalized business entities
- `metrics/` - Calculated metrics or aggregations
- `reports/` - Structured report schemas
- Or any logical grouping for your transformed data

## Naming Convention

Use the same folder structure as in `raw/endpoints/` when there's a 1:1 mapping, or create new logical groupings for transformed data.
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Extracted events (JSON)

Describe the normalized event shape.
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{\n \"$schema\": \"http://json-schema.org/draft-07/schema#\",\n \"title\": \"Extracted Event\",\n \"type\": \"object\",\n \"additionalProperties\": false,\n \"properties\": {}\n}\n
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Extracted relational schema

Describe normalized tables and relationships.
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{\n \"tables\": []\n}\n
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
-- DDL placeholders for extracted tables
Loading