Implement Shopify Admin API connector with cursor-based pagination

connector-registry/shopify/_meta/README.md

@@ -0,0 +1,6 @@
+# Shopify (Registry)
+
+This is the top-level registry entry for `shopify`.
+
+- Versions live under `shopify/{version}`
+- See author implementations under `shopify/{version}/{author}`

connector-registry/shopify/_meta/connector.json

@@ -0,0 +1,9 @@
+{
+  "$schema": "https://schemas.connector-factory.dev/connector-root.schema.json",
+  "identifier": "shopify",
+  "name": "Shopify",
+  "category": "api",
+  "tags": ["ecommerce", "retail", "sales"],
+  "description": "Shopify is a leading e-commerce platform for online stores and retail point-of-sale systems",
+  "homepage": "https://www.shopify.com"
+}

connector-registry/shopify/v1/514-labs/_meta/CHANGELOG.md

@@ -0,0 +1,11 @@
+# Changelog
+
+All notable changes to this connector will be documented in this file.
+
+## 0.1.0 - Initial Implementation
+
+- Initial scaffold for `shopify` (TypeScript) connector
+- Support for Shopify Admin API authentication
+- Cursor-based pagination support
+- Core resources: products, orders, customers
+- GraphQL and REST API support

connector-registry/shopify/v1/514-labs/_meta/LICENSE

@@ -0,0 +1,18 @@
+MIT License
+
+Copyright (c) 514-labs
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

connector-registry/shopify/v1/514-labs/_meta/README.md

@@ -0,0 +1,10 @@
+# Shopify Connector (by 514-labs)
+
+This directory contains language-agnostic metadata and documentation for the `shopify` connector.
+
+- Name: `shopify`
+- Author: `514-labs`
+- Category: `api`
+- Languages: `typescript`
+
+See `_meta/connector.json` for connector metadata and implementation folders for language-specific code.

connector-registry/shopify/v1/514-labs/_meta/connector.json

@@ -0,0 +1,22 @@
+{
+  "name": "shopify",
+  "author": "514-labs",
+  "version": "v1",
+  "languages": [
+    "typescript"
+  ],
+  "category": "api",
+  "capabilities": {
+    "extract": true,
+    "transform": false,
+    "load": false
+  },
+  "source": {
+    "type": "api",
+    "spec": "https://shopify.dev/docs/api/admin-rest",
+    "homepage": "https://www.shopify.com"
+  },
+  "tags": ["ecommerce", "retail", "sales", "inventory"],
+  "maintainers": [],
+  "issues": { "typescript": { "default": "" } }
+}

connector-registry/shopify/v1/514-labs/typescript/_meta/connector.json

@@ -0,0 +1,8 @@
+{
+  "identifier": "shopify",
+  "name": "shopify",
+  "author": "514-labs",
+  "version": "v1",
+  "language": "typescript",
+  "implementations": ["default"]
+}

connector-registry/shopify/v1/514-labs/typescript/default/.env.example

@@ -0,0 +1,4 @@
+# Shopify API credentials
+SHOPIFY_SHOP_NAME=your-shop-name
+SHOPIFY_ACCESS_TOKEN=your-admin-api-access-token
+SHOPIFY_API_VERSION=2024-10

connector-registry/shopify/v1/514-labs/typescript/default/.gitignore

@@ -0,0 +1,3 @@
+node_modules/
+dist/
+.env

connector-registry/shopify/v1/514-labs/typescript/default/README.md

@@ -0,0 +1,37 @@
+# Shopify (TypeScript)
+
+TypeScript implementation for `shopify` by `514-labs`.
+
+## Quick Start
+
+```typescript
+import { createConnector } from '@workspace/connector-shopify'
+
+const connector = createConnector()
+connector.init({
+  shopName: 'your-shop-name',
+  accessToken: process.env.SHOPIFY_ACCESS_TOKEN!,
+  apiVersion: '2024-10',
+})
+
+// Fetch products
+for await (const page of connector.products.list({ pageSize: 50 })) {
+  console.log(`Fetched ${page.length} products`)
+}
+```
+
+## Authentication
+
+Shopify uses custom access tokens for authentication. You'll need:
+- A Shopify store name (e.g., `your-shop-name`)
+- An Admin API access token
+
+See [Shopify Admin API documentation](https://shopify.dev/docs/api/admin-rest) for details on obtaining access tokens.
+
+## Available Resources
+
+- `products` - Product management
+- `orders` - Order management
+- `customers` - Customer management
+
+See `schemas/index.json` for machine-readable definitions and accompanying Markdown docs.

connector-registry/shopify/v1/514-labs/typescript/default/docs/configuration.md

@@ -0,0 +1,65 @@
+# Configuration
+
+## Required Configuration
+
+- **shopName**: Your Shopify store name (without `.myshopify.com`)
+  - Example: If your store URL is `https://my-store.myshopify.com`, use `my-store`
+- **accessToken**: Your Shopify Admin API access token
+  - Obtain from: Shopify Admin → Apps → Develop apps → Create an app → Install → Admin API access token
+
+## Optional Configuration
+
+- **apiVersion**: Shopify API version to use (default: `2024-10`)
+  - Format: `YYYY-MM` (e.g., `2024-10`, `2024-07`)
+  - See [Shopify API versioning](https://shopify.dev/docs/api/usage/versioning)
+- **logging**: Request/response logging configuration
+  - `enabled`: Enable logging (default: `false`)
+  - `level`: Log level - `debug`, `info`, `warn`, `error` (default: `info`)
+  - `includeQueryParams`: Include query parameters in logs (default: `false`)
+  - `includeHeaders`: Include headers in logs (default: `false`)
+  - `includeBody`: Include request/response bodies in logs (default: `false`)
+  - `logger`: Custom logger function (default: `console.log`)
+- **metrics**: Metrics collection configuration
+  - `enabled`: Enable metrics collection (default: `false`)
+
+## Authentication
+
+This connector uses Shopify's Admin API access token authentication. The token is sent via the `X-Shopify-Access-Token` header.
+
+### Obtaining an Access Token
+
+1. Go to your Shopify Admin
+2. Navigate to Apps → Develop apps
+3. Click "Create an app"
+4. Give your app a name and configure the scopes you need
+5. Click "Install app"
+6. Copy the Admin API access token
+
+### Required Scopes
+
+Depending on which resources you want to access, configure the following scopes:
+
+- `read_products` - For products resource
+- `read_orders` - For orders resource
+- `read_customers` - For customers resource
+
+## Example
+
+```typescript
+import { createConnector } from '@workspace/connector-shopify'
+
+const connector = createConnector()
+connector.init({
+  shopName: 'my-store',
+  accessToken: 'shpat_xxxxxxxxxxxx',
+  apiVersion: '2024-10',
+  logging: {
+    enabled: true,
+    level: 'info',
+    includeQueryParams: true,
+  },
+  metrics: {
+    enabled: true,
+  },
+})
+```

connector-registry/shopify/v1/514-labs/typescript/default/docs/getting-started.md

@@ -0,0 +1,78 @@
+# Getting Started
+
+## Installation
+
+```bash
+npm install @workspace/connector-shopify
+# or
+pnpm add @workspace/connector-shopify
+```
+
+## Basic Usage
+
+```typescript
+import { createConnector } from '@workspace/connector-shopify'
+
+// Initialize the connector
+const connector = createConnector()
+connector.init({
+  shopName: 'your-shop-name',
+  accessToken: process.env.SHOPIFY_ACCESS_TOKEN!,
+  apiVersion: '2024-10',
+})
+
+// Fetch products
+for await (const page of connector.products.list({ pageSize: 50 })) {
+  for (const product of page) {
+    console.log(`Product: ${product.title}`)
+  }
+}
+
+// Get a single product
+const product = await connector.products.get(123456789)
+console.log(product)
+
+// Fetch orders
+for await (const page of connector.orders.list({ status: 'open' })) {
+  for (const order of page) {
+    console.log(`Order ${order.name}: ${order.email}`)
+  }
+}
+
+// Search customers
+const customers = await connector.customers.search({
+  query: 'email:john@example.com'
+})
+console.log(customers)
+```
+
+## Authentication
+
+See [Configuration](./configuration.md) for details on obtaining and using Shopify API credentials.
+
+## Available Resources
+
+- **Products** - List, get, and count products
+- **Orders** - List, get, and count orders
+- **Customers** - List, get, search, and count customers
+
+## Pagination
+
+All list methods support cursor-based pagination with the following parameters:
+
+- `pageSize`: Number of items per page (default: 50, max: 250)
+- `maxItems`: Maximum total items to fetch across all pages
+
+Example:
+```typescript
+// Fetch 100 products, 50 at a time
+for await (const page of connector.products.list({ pageSize: 50, maxItems: 100 })) {
+  console.log(`Page size: ${page.length}`)
+}
+```
+
+## Rate Limiting
+
+Shopify enforces rate limits on API requests. The connector does not automatically handle rate limiting, so you may want to implement retry logic or use the built-in retry configuration from `@connector-factory/core`.
+
+See [Shopify Rate Limits](https://shopify.dev/docs/api/usage/rate-limits) for more information.

connector-registry/shopify/v1/514-labs/typescript/default/docs/limits.md

@@ -0,0 +1,70 @@
+# Limits
+
+## Rate Limits
+
+Shopify implements rate limiting for API requests to ensure fair usage and system stability.
+
+### REST Admin API Rate Limits
+
+- **Standard Plan**: 2 requests per second
+- **Shopify Plus**: 4 requests per second (can be increased)
+- **GraphQL Admin API**: 1000 points per second (cost-based)
+
+### Leaky Bucket Algorithm
+
+Shopify uses a "leaky bucket" algorithm for rate limiting:
+- Each app gets a bucket that can hold a certain number of requests
+- The bucket "leaks" at a constant rate (requests per second)
+- When the bucket is full, additional requests are throttled with a 429 status code
+
+### Response Headers
+
+Shopify includes rate limit information in response headers:
+- `X-Shopify-Shop-Api-Call-Limit`: Current usage / Total limit (e.g., "32/40")
+- `Retry-After`: Time in seconds to wait before retrying (on 429 responses)
+
+### Handling Rate Limits
+
+The connector doesn't automatically retry on rate limit errors. To handle rate limits:
+
+1. **Monitor response headers** to track your usage
+2. **Implement retry logic** with exponential backoff
+3. **Use the core retry configuration**:
+
+```typescript
+connector.initialize({
+  shopName: 'your-shop',
+  accessToken: 'token',
+  retry: {
+    maxAttempts: 3,
+    initialDelayMs: 1000,
+    maxDelayMs: 10000,
+    backoffMultiplier: 2,
+    respectRetryAfter: true,
+  },
+})
+```
+
+### Best Practices
+
+- Batch requests when possible
+- Use cursor-based pagination efficiently
+- Monitor the `X-Shopify-Shop-Api-Call-Limit` header
+- Implement exponential backoff for retries
+- Consider using webhooks instead of polling
+
+## Data Limits
+
+### Query Parameters
+
+- **limit**: Maximum 250 items per page
+- **fields**: Limit returned fields to reduce response size
+
+### Bulk Operations
+
+For large data exports, consider using Shopify's Bulk Operations API instead of paginating through REST endpoints.
+
+## References
+
+- [Shopify API Rate Limits](https://shopify.dev/docs/api/usage/rate-limits)
+- [Bulk Operations](https://shopify.dev/docs/api/usage/bulk-operations/queries)