Skip to content

Commit 8fc5dcb

Browse files
marcosmarxmmacmvfaunaeeoctavia-squidington-iii
authored
Source Fauna: add seed file information (airbytehq#17355)
* Add fauna source * Update changelog to include the correct PR * Improve docs (#1) * Applied suggestions to improve docs (#2) * Applied suggestions to improve docs * Cleaned up the docs * Apply suggestions from code review Co-authored-by: Ewan Edwards <46354154+faunaee@users.noreply.github.com> * Update airbyte-integrations/connectors/source-fauna/source_fauna/spec.yaml Co-authored-by: Ewan Edwards <46354154+faunaee@users.noreply.github.com> Co-authored-by: Ewan Edwards <46354154+faunaee@users.noreply.github.com> * Flake Checker (#3) * Run ./gradlew :airbyte-integrations:connectors:source-fauna:flakeCheck * Fix all the warnings * Set additionalProperties to true to adhere to acceptance tests * Remove custom fields (airbytehq#4) * Remove custom fields from source.py * Remove custom fields from spec.yaml * Collections that support incremental sync are found correctly * Run formatter * Index values and termins are verified * Stripped additional_columns from collection config and check() * We now search for an index at the start of each sync * Add default for missing data in collection * Add a log message about the index chosen to sync an incremental stream * Add an example for a configured incremental catalog * Check test now validates the simplified check function * Remove collection name from spec.yaml and CollectionConfig * Update test_util.py to ahere to the new config * Update the first discover test to validate that we can find indexes correctly * Remove other discover tests, as they no longer apply * Full refresh test now works with simplified expanded columns * Remove unused imports * Incremental test now adheres to the find_index_for_stream system * Database test passes, so now all unit tests pass again * Remove extra fields from required section * ttl is nullable * Data defaults to an empty object * Update tests to reflect ttl and data select changes * Fix expected records. All unit tests and acceptance tests pass * Cleanup docs for find_index_for_stream * Update setup guide to reflect multiple collections * Add docs to install the fauna shell * Update examples and README to conform to the removal of additional columns * add seed file * auto-bump connector version [ci skip] * add icon in seed Co-authored-by: Neil Macneale <neil.macneale.v@gmail.com> Co-authored-by: Ewan Edwards <46354154+faunaee@users.noreply.github.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
1 parent 65e6168 commit 8fc5dcb

File tree

2 files changed

+105
-0
lines changed

2 files changed

+105
-0
lines changed

airbyte-config/init/src/main/resources/seed/source_definitions.yaml

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -299,6 +299,14 @@
299299
documentationUrl: https://docs.airbyte.com/integrations/sources/faker
300300
sourceType: api
301301
releaseStage: alpha
302+
- name: Fauna
303+
sourceDefinitionId: 3825db3e-c94b-42ac-bd53-b5a9507ace2b
304+
dockerRepository: airbyte/source-fauna
305+
dockerImageTag: dev
306+
documentationUrl: https://docs.airbyte.com/integrations/sources/fauna
307+
icon: fauna.svg
308+
sourceType: database
309+
releaseStage: alpha
302310
- name: File
303311
sourceDefinitionId: 778daa7c-feaf-4db6-96f3-70fd645acc77
304312
dockerRepository: airbyte/source-file

airbyte-config/init/src/main/resources/seed/source_specs.yaml

Lines changed: 97 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2995,6 +2995,103 @@
29952995
supportsNormalization: false
29962996
supportsDBT: false
29972997
supported_destination_sync_modes: []
2998+
- dockerImage: "airbyte/source-fauna:dev"
2999+
spec:
3000+
documentationUrl: "https://github.com/fauna/airbyte/blob/source-fauna/docs/integrations/sources/fauna.md"
3001+
connectionSpecification:
3002+
$schema: "http://json-schema.org/draft-07/schema#"
3003+
title: "Fauna Spec"
3004+
type: "object"
3005+
required:
3006+
- "domain"
3007+
- "port"
3008+
- "scheme"
3009+
- "secret"
3010+
additionalProperties: true
3011+
properties:
3012+
domain:
3013+
order: 0
3014+
type: "string"
3015+
title: "Domain"
3016+
description: "Domain of Fauna to query. Defaults db.fauna.com. See <a href=https://docs.fauna.com/fauna/current/learn/understanding/region_groups#how-to-use-region-groups>the\
3017+
\ docs</a>."
3018+
default: "db.fauna.com"
3019+
port:
3020+
order: 1
3021+
type: "integer"
3022+
title: "Port"
3023+
description: "Endpoint port."
3024+
default: 443
3025+
scheme:
3026+
order: 2
3027+
type: "string"
3028+
title: "Scheme"
3029+
description: "URL scheme."
3030+
default: "https"
3031+
secret:
3032+
order: 3
3033+
type: "string"
3034+
title: "Fauna Secret"
3035+
description: "Fauna secret, used when authenticating with the database."
3036+
airbyte_secret: true
3037+
collection:
3038+
order: 5
3039+
type: "object"
3040+
title: "Collection"
3041+
description: "Settings for the Fauna Collection."
3042+
required:
3043+
- "page_size"
3044+
- "deletions"
3045+
properties:
3046+
page_size:
3047+
order: 4
3048+
type: "integer"
3049+
title: "Page Size"
3050+
default: 64
3051+
description: "The page size used when reading documents from the database.\
3052+
\ The larger the page size, the faster the connector processes documents.\
3053+
\ However, if a page is too large, the connector may fail. <br>\n\
3054+
Choose your page size based on how large the documents are. <br>\n\
3055+
See <a href=\"https://docs.fauna.com/fauna/current/learn/understanding/types#page\"\
3056+
>the docs</a>."
3057+
deletions:
3058+
order: 5
3059+
type: "object"
3060+
title: "Deletion Mode"
3061+
description: "<b>This only applies to incremental syncs.</b> <br>\n\
3062+
Enabling deletion mode informs your destination of deleted documents.<br>\n\
3063+
Disabled - Leave this feature disabled, and ignore deleted documents.<br>\n\
3064+
Enabled - Enables this feature. When a document is deleted, the connector\
3065+
\ exports a record with a \"deleted at\" column containing the time\
3066+
\ that the document was deleted."
3067+
oneOf:
3068+
- title: "Disabled"
3069+
type: "object"
3070+
order: 0
3071+
required:
3072+
- "deletion_mode"
3073+
properties:
3074+
deletion_mode:
3075+
type: "string"
3076+
const: "ignore"
3077+
- title: "Enabled"
3078+
type: "object"
3079+
order: 1
3080+
required:
3081+
- "deletion_mode"
3082+
- "column"
3083+
properties:
3084+
deletion_mode:
3085+
type: "string"
3086+
const: "deleted_field"
3087+
column:
3088+
type: "string"
3089+
title: "Deleted At Column"
3090+
description: "Name of the \"deleted at\" column."
3091+
default: "deleted_at"
3092+
supportsNormalization: false
3093+
supportsDBT: false
3094+
supported_destination_sync_modes: []
29983095
- dockerImage: "airbyte/source-file:0.2.23"
29993096
spec:
30003097
documentationUrl: "https://docs.airbyte.io/integrations/sources/file"

0 commit comments

Comments
 (0)