Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/young-bikes-battle.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@electric-sql/pglite-sync': patch
---

Bump the version of the Electric sync client
42 changes: 9 additions & 33 deletions docs/docs/sync.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,12 @@ You can then use the `syncShapeToTable` method to sync a table from Electric:

```ts
const shape = await pg.electric.syncShapeToTable({
shape: { url: 'http://localhost:3000/v1/shape', table: 'todo' },
shape: {
url: 'http://localhost:3000/v1/shape',
params: {
table: 'todo',
},
},
table: 'todo',
primaryKey: ['id'],
})
Expand Down Expand Up @@ -69,7 +74,7 @@ The `syncShapeToTable` is a relatively thin wrapper around the Electric [ShapeSt
It takes the following options as an object:

- `shape: ShapeStreamOptions`<br>
The shape stream specification to sync, described by [`ShapeStreamOptions`](#shapestreamoptions).
The shape stream specification to sync, described by the Electric [ShapeStream API](https://electric-sql.com/docs/api/clients/typescript#shapestream) options, see the [ShapeStream API](https://electric-sql.com/docs/api/clients/typescript#options) for more details.

- `table: string`<br>
The name of the table to sync into.
Expand Down Expand Up @@ -100,40 +105,11 @@ The returned `shape` object from the `syncShapeToTable` call has the following m
- `subscribe(cb: () => void, error: (err: FetchError | Error) => void)`<br>
A callback to indicate that the shape caught up to the main Postgres.

- `subscribeMustRefresh(cb: () => void)`<br>
A callback that is called when the stream emits a `must-refresh` message.

- `unsubscribeMustRefresh(cb: () => void)`<br>
Unsubscribe from the `mustRefresh` notification.

- `unsubscribe()`<br>
Unsubscribe from the shape. Note that this does not clear the state that has been synced into the table.

### `ShapeStreamOptions`

- `url: string`<br>
The full URL to where the Shape is hosted. This can either be the Electric server directly, or a proxy. E.g. for a local Electric instance, you might set `http://localhost:3000/v1/shape`

- `table: string`<br>
The name of the table in the remote database to sync from

- `where?: string`<br>
Where clauses for the shape.

- `offset?: Offset`<br>
The "offset" on the shape log. This is typically not set as the ShapeStream will handle this automatically. A common scenario where you might pass an offset is if you're maintaining a local cache of the log. If you've gone offline and are re-starting a ShapeStream to catch-up to the latest state of the Shape, you'd pass in the last offset and shapeId you'd seen from the Electric server so it knows at what point in the shape to catch you up from.

- `shapeId?: string`<br>
The server side `shapeId`, similar to `offset`, this isn't typically used unless you're maintaining a cache of the shape log.

- `backoffOptions`<br>
Options to configure the backoff rules on failure

- `subscribe?: boolean`<br>
Automatically fetch updates to the Shape. If you just want to sync the current shape and stop, pass false.

- `signal?: AbortSignal`<br>
A `AbortSignal` instance to use to abort the sync.
- `stream: ShapeStream`<br>
The underlying `ShapeStream` instance, see the [ShapeStream API](https://electric-sql.com/docs/api/clients/typescript#shapestream) for more details.

## Limitations

Expand Down
2 changes: 1 addition & 1 deletion packages/pglite-sync/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@
"dist"
],
"dependencies": {
"@electric-sql/client": "~0.8.0"
"@electric-sql/client": "~0.9.0"
},
"devDependencies": {
"@electric-sql/pglite": "workspace:*",
Expand Down
65 changes: 52 additions & 13 deletions packages/pglite-sync/test/sync.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,10 @@ describe('pglite-sync', () => {
}))

const shape = await pg.electric.syncShapeToTable({
shape: { url: 'http://localhost:3000/v1/shape', table: 'todo' },
shape: {
url: 'http://localhost:3000/v1/shape',
params: { table: 'todo' },
},
table: 'todo',
primaryKey: ['id'],
})
Expand Down Expand Up @@ -121,7 +124,10 @@ describe('pglite-sync', () => {
}))

const shape = await pg.electric.syncShapeToTable({
shape: { url: 'http://localhost:3000/v1/shape', table: 'todo' },
shape: {
url: 'http://localhost:3000/v1/shape',
params: { table: 'todo' },
},
table: 'todo',
primaryKey: ['id'],
})
Expand Down Expand Up @@ -201,7 +207,10 @@ describe('pglite-sync', () => {
const numResumes = 3
for (let i = 0; i < numResumes; i++) {
const shape = await pg.electric.syncShapeToTable({
shape: { url: 'http://localhost:3000/v1/shape', table: 'todo' },
shape: {
url: 'http://localhost:3000/v1/shape',
params: { table: 'todo' },
},
table: 'todo',
primaryKey: ['id'],
shapeKey: 'foo',
Expand Down Expand Up @@ -270,7 +279,10 @@ describe('pglite-sync', () => {

const numInserts = 100
const shape = await pg.electric.syncShapeToTable({
shape: { url: 'http://localhost:3000/v1/shape', table: 'todo' },
shape: {
url: 'http://localhost:3000/v1/shape',
params: { table: 'todo' },
},
table: 'todo',
primaryKey: ['id'],
shapeKey: 'foo',
Expand Down Expand Up @@ -334,7 +346,10 @@ describe('pglite-sync', () => {

// resuming should
const resumedShape = await pg.electric.syncShapeToTable({
shape: { url: 'http://localhost:3000/v1/shape', table: 'todo' },
shape: {
url: 'http://localhost:3000/v1/shape',
params: { table: 'todo' },
},
table: 'todo',
primaryKey: ['id'],
shapeKey: 'foo',
Expand Down Expand Up @@ -375,7 +390,10 @@ describe('pglite-sync', () => {
const altTable = 'bar'

const shape1 = await pg.electric.syncShapeToTable({
shape: { url: 'http://localhost:3000/v1/shape', table: 'todo' },
shape: {
url: 'http://localhost:3000/v1/shape',
params: { table: 'todo' },
},
table: table,
primaryKey: ['id'],
})
Expand All @@ -384,15 +402,21 @@ describe('pglite-sync', () => {
await expect(
async () =>
await pg.electric.syncShapeToTable({
shape: { url: 'http://localhost:3000/v1/shape', table: 'todo_alt' },
shape: {
url: 'http://localhost:3000/v1/shape',
params: { table: 'todo_alt' },
},
table: table,
primaryKey: ['id'],
}),
).rejects.toThrowError(`Already syncing shape for table ${table}`)

// should be able to sync shape into other table
const altShape = await pg.electric.syncShapeToTable({
shape: { url: 'http://localhost:3000/v1/shape', table: 'bar' },
shape: {
url: 'http://localhost:3000/v1/shape',
params: { table: 'bar' },
},
table: altTable,
primaryKey: ['id'],
})
Expand All @@ -403,7 +427,10 @@ describe('pglite-sync', () => {
shape1.unsubscribe()

const shape2 = await pg.electric.syncShapeToTable({
shape: { url: 'http://localhost:3000/v1/shape', table: 'todo_alt' },
shape: {
url: 'http://localhost:3000/v1/shape',
params: { table: 'todo_alt' },
},
table: table,
primaryKey: ['id'],
})
Expand All @@ -420,7 +447,10 @@ describe('pglite-sync', () => {
}))

const shape = await pg.electric.syncShapeToTable({
shape: { url: 'http://localhost:3000/v1/shape', table: 'todo' },
shape: {
url: 'http://localhost:3000/v1/shape',
params: { table: 'todo' },
},
table: 'todo',
primaryKey: ['id'],
})
Expand Down Expand Up @@ -506,7 +536,10 @@ describe('pglite-sync', () => {
expect(result0.rows[0]).toEqual({ current_setting: 'false' })

const shape = await pg.electric.syncShapeToTable({
shape: { url: 'http://localhost:3000/v1/shape', table: 'test_syncing' },
shape: {
url: 'http://localhost:3000/v1/shape',
params: { table: 'test_syncing' },
},
table: 'test_syncing',
primaryKey: ['id'],
})
Expand Down Expand Up @@ -548,7 +581,10 @@ describe('pglite-sync', () => {
}))

const shape = await pg.electric.syncShapeToTable({
shape: { url: 'http://localhost:3000/v1/shape', table: 'todo' },
shape: {
url: 'http://localhost:3000/v1/shape',
params: { table: 'todo' },
},
table: 'todo',
primaryKey: ['id'],
useCopy: true,
Expand Down Expand Up @@ -624,7 +660,10 @@ describe('pglite-sync', () => {
}))

const shape = await pg.electric.syncShapeToTable({
shape: { url: 'http://localhost:3000/v1/shape', table: 'todo' },
shape: {
url: 'http://localhost:3000/v1/shape',
params: { table: 'todo' },
},
table: 'todo',
primaryKey: ['id'],
useCopy: true,
Expand Down
11 changes: 6 additions & 5 deletions pnpm-lock.yaml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.