-
Notifications
You must be signed in to change notification settings - Fork 93
Add Duplicate Writes documentation #1103
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
e5c00bb
f4bb1c5
5d6b97c
5ec0c9d
ab18b95
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,243 @@ | ||
| --- | ||
| sidebar_position: 3 | ||
| slug: /interacting/duplicate-writes | ||
| description: Handle duplicate writes and missing deletes in write operations | ||
| --- | ||
|
|
||
| import { | ||
| AuthzModelSnippetViewer, | ||
| CardBox, | ||
| DocumentationNotice, | ||
| ProductConcept, | ||
| ProductName, | ||
| ProductNameFormat, | ||
| RelatedSection, | ||
| RelationshipTuplesViewer, | ||
| WriteRequestViewer, | ||
| SupportedLanguage, | ||
| } from "@components/Docs"; | ||
|
|
||
|
|
||
| # Duplicate Writes | ||
|
|
||
| <DocumentationNotice /> | ||
|
|
||
| Handle duplicate writes and missing deletes gracefully using the Write API's optional `on_duplicate` and `on_missing` parameters. | ||
|
|
||
| <CardBox title="When to use" appearance="filled"> | ||
|
|
||
| Common scenarios include high-volume data imports into <ProductName format={ProductNameFormat.ShortForm}/> or migrations between distributed systems where you need to ensure data synchronization without causing transaction failures on duplicate writes or missing deletes. | ||
|
|
||
| </CardBox> | ||
|
|
||
| ## Understanding the Problem | ||
|
|
||
| The Write API is the primary method for adding and removing <ProductConcept section="what-is-a-relationship-tuple" linkName="relationship tuples" /> in your <ProductName format={ProductNameFormat.ShortForm}/> store. | ||
|
|
||
| By default, the Write API operates with strict validation ("all-or-nothing"): | ||
|
|
||
| - Writing a tuple that already exists causes the entire request to fail (even if only 1 tuple out of 40 possible tuples is a duplicate) | ||
| - Deleting a tuple that doesn't exist causes the entire request to fail | ||
|
|
||
| This strict behavior requires applications to check the current state of the tuple in <ProductName format={ProductNameFormat.ShortForm}/> before making changes, implement complex retry logic, or simply ignore the errors altogether. | ||
|
|
||
| ## How to Handle Duplicate Operations | ||
|
|
||
| To improve import and migration scenarios, the Write API provides optional parameters to control how duplicate writes and missing deletes are handled. | ||
|
|
||
| The `on_duplicate` and `on_missing` parameters change this behavior, allowing you to instruct the API to ignore these cases and process the rest of the request successfully. | ||
|
|
||
| ### Response: Default Behavior | ||
|
|
||
| Without the `on_duplicate` parameter, attempting to write an existing tuple returns a `400 Bad Request`. | ||
|
|
||
| ```http | ||
| HTTP/1.1 400 Bad Request | ||
|
|
||
| { | ||
| "code": "write_failed_due_to_invalid_input", | ||
| "message": "cannot write a tuple which already exists: user: 'user:anne', relation: 'writer', object: 'document:2025-budget': tuple to be written already existed or the tuple to be deleted did not exist" | ||
| } | ||
| ``` | ||
|
|
||
| ### Response: Ignoring Duplicates | ||
|
|
||
| Setting `on_duplicate: "ignore"` allows the duplicate to be ignored and the API returns a `200 OK`. | ||
|
|
||
| ```http | ||
| HTTP/1.1 200 OK | ||
|
|
||
| {} | ||
| ``` | ||
|
|
||
| ## Optional API Parameters | ||
|
|
||
| The parameters are added within the `writes` and `deletes` objects in the body of a Write request. | ||
|
|
||
| ### Writes | ||
|
|
||
| The `on_duplicate` parameter controls the behavior when writing tuples. | ||
|
|
||
| - **"error" (Default)**: The request fails if any tuple in the writes array already exists. This maintains backward compatibility. | ||
| - **"ignore"**: The API ignores any attempt to write a tuple that already exists and proceeds to write the new ones in the same transaction. The request succeeds. | ||
|
|
||
| <WriteRequestViewer | ||
| skipSetup={true} | ||
| relationshipTuples={[ | ||
| { | ||
| user: 'user:anne', | ||
| relation: 'writer', | ||
| object: 'document:2025-budget', | ||
| }, | ||
| ]} | ||
| writeOptions={{ | ||
| on_duplicate: 'ignore', | ||
| }} | ||
| expectedResponse={{ | ||
| data: {}, | ||
| }} | ||
| allowedLanguages={[ | ||
| SupportedLanguage.CURL, | ||
| ]} | ||
|
|
||
| /> | ||
|
|
||
| :::caution | ||
| At the moment, this feature is only available on the API. Supported SDKs will follow shortly after. | ||
| ::: | ||
|
|
||
| ### Deletes | ||
|
|
||
| The `on_missing` parameter controls behavior when deleting tuples. | ||
|
|
||
| - **"error" (Default)**: The request fails if any tuple in the deletes array does not exist. | ||
| - **"ignore"**: The API ignores any attempt to delete a tuple that does not exist and proceeds to delete the ones that do exist. The request succeeds. | ||
|
|
||
| <WriteRequestViewer | ||
| skipSetup={true} | ||
| deleteRelationshipTuples={[ | ||
| { | ||
| user: 'user:anne', | ||
| relation: 'writer', | ||
| object: 'document:2025-budget', | ||
| }, | ||
| ]} | ||
| deleteOptions={{ | ||
| on_missing: 'ignore', | ||
| }} | ||
| expectedResponse={{ | ||
| data: {}, | ||
| }} | ||
| allowedLanguages={[ | ||
| SupportedLanguage.CURL, | ||
| ]} | ||
|
|
||
| /> | ||
|
|
||
| ## Using Both Parameters Together | ||
|
|
||
| The decision to have separate `on_duplicate` and `on_missing` parameters is intentional. This design gives you granular, independent control over the behavior of writes and deletes within a single atomic transaction. You can mix and match strict and permissive behaviors to suit your exact needs. | ||
|
|
||
| For example, you might perform a strict delete (`on_missing: "error"`) to confirm that a specific permission has been successfully removed before making a permissive write (`on_duplicate: "ignore"`) that guarantees the new permission exists. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't follow the example, if I perform a delete with 'on_missing:error' and the relation does not exist, it's fine, I can add the new permission safelty. |
||
|
|
||
| :::tip | ||
| This flexibility is particularly useful when you are trying to synchronize state between your application's database and <ProductName format={ProductNameFormat.ShortForm}/>. You might need a strict operation to fail so you can roll back corresponding changes in your own database, ensuring overall system consistency. | ||
| ::: | ||
|
|
||
| <WriteRequestViewer | ||
| skipSetup={true} | ||
| relationshipTuples={[ | ||
| { | ||
| user: 'user:anne', | ||
| relation: 'reader', | ||
| object: 'document:2025-budget', | ||
| }, | ||
| ]} | ||
| writeOptions={{ | ||
| on_duplicate: 'ignore', | ||
| }} | ||
| deleteRelationshipTuples={[ | ||
| { | ||
| user: 'user:anne', | ||
| relation: 'writer', | ||
| object: 'document:2025-budget', | ||
| }, | ||
| ]} | ||
| deleteOptions={{ | ||
| on_missing: 'error', | ||
| }} | ||
| expectedResponse={{ | ||
| data: {}, | ||
| }} | ||
| allowedLanguages={[ | ||
| SupportedLanguage.CURL, | ||
| ]} | ||
|
|
||
| /> | ||
|
|
||
| In this example: | ||
| - If the permission to delete doesn't exist, the entire request will fail as intended. | ||
| - If the tuple exists and is successfully deleted, the write permission will succeed whether the tuple already exists or is written in that moment. | ||
|
|
||
| ## Important Concepts | ||
|
|
||
| ### The Write Request Remains Atomic | ||
|
|
||
| All tuples in a request will still be processed as a single atomic unit. The `ignore` option only changes the success criteria for individual operations in the request. | ||
|
|
||
| ### "Best effort" ignore | ||
|
|
||
| For writes: An `on_duplicate: 'ignore'` operation uses a "best effort" approach. We will attempt *once* to ignore duplicates and write non-duplicates, but if there is ever a conflict writing to the database (i.e. write a tuple we don’t think exists but it suddenly exists, probably due to a parallel request), we will abort the race condition immediately, and just return a `409 Conflict` error. These errors are rare, but can happen. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Let's verify if this happens in SQL databases |
||
|
|
||
| For deletes: An `on_missing: 'ignore'` operation is immune to race conditions due to database-level locks. It is not possible for another request to interfere since a delete operation will always succeed if the tuple exists or not. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We don't need to mention this |
||
|
|
||
| ### "Ignore" is Not an "Upsert" | ||
|
|
||
| It is critical to understand that `on_duplicate: "ignore"` will not update an existing tuple, only ignore an identical tuple. This is why we do not call the operation an "idempotent" operation. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do we need to explain this? We don't need to mention idempotent or explain why we don't use it |
||
|
|
||
| The behavior of `on_duplicate: "ignore"` is more nuanced for tuples with conditions. | ||
| - **Identical Tuples**: If a tuple in the request is 100% identical to an existing tuple (same user, relation, object, condition name, and condition context), it will be safely ignored. | ||
| - **Conflicting Tuples**: If a tuple key (user, relation, object) matches an existing tuple, but the condition is different, this is a conflict. The write attempt will be rejected, and the entire transaction will fail with a `409 Conflict` error. **The correct pattern to safely update a tuple's condition requires explicitly deleting the old tuple and writing the new one within the same atomic Write request.** | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @tylernix can you confirm adding/deleting the same in a single request works well? |
||
|
|
||
|
|
||
|
|
||
| ```http | ||
| HTTP/1.1 409 Conflict | ||
| { | ||
| "code": "Aborted", | ||
| "message": "transactional write failed due to conflict: attempted to write a tuple which already exists with a different condition: user: 'user:anne', relation: 'writer', object: 'document:2025-budget'" | ||
| } | ||
| ``` | ||
|
|
||
| :::tip | ||
| The condition is not returned in the response, but you can call `/read` for this tuple to view its condition. | ||
| ::: | ||
|
|
||
| :::warning | ||
| The deletes operation in the Write API does not accept a condition. Attempting to include one will result in an invalid request. | ||
| ::: | ||
|
|
||
| ## Related Sections | ||
|
|
||
| <RelatedSection | ||
| description="Learn more about different types of writes and managing relationship tuples." | ||
| relatedLinks={[ | ||
| { | ||
| title: 'Write API', | ||
| description: 'Details on the write API in the {ProductName} reference guide.', | ||
| link: '/api/service#Relationship%20Tuples/Write', | ||
| }, | ||
| { | ||
| title: 'Transactional Writes', | ||
| description: 'Learn about updating multiple relationship tuples in a single transaction.', | ||
| link: './transactional-writes', | ||
| id: './transactional-writes', | ||
| }, | ||
| { | ||
| title: 'Update relationship tuples in SDK', | ||
| description: 'Learn about how to update relationship tuples in SDK.', | ||
| link: '../getting-started/update-tuples', | ||
| id: '../getting-started/update-tuples', | ||
| } | ||
| ]} | ||
| /> | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We don't need to mention backward compatibility in the docs (someone will read it in a year and not have any context)