> ## Documentation Index > Fetch the complete documentation index at: https://docs.prequel.co/llms.txt > Use this file to discover all available pages before exploring further. # Test new destination connection > Test the connection for a new destination before creating it. ## OpenAPI ````yaml /generated/openapi-generated-2023-12-01.json post /test-destination openapi: 3.1.0 info: title: Prequel Data Export description: Data Export API spec version: '2023-12-01' servers: - url: https://api.prequel.co description: Cloud-hosted server (US) - url: https://eu-api.prequel.co description: Cloud-hosted server (EU) - url: https://{custom_host}/ description: Cloud-prem or shared-cloud server security: - ApiKeyAuth: [] paths: /test-destination: post: tags: - Destinations summary: Test new destination connection description: Test the connection for a new destination before creating it. operationId: create-destination-test requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/CreateDestinationRequest' $schema: https://json-schema.org/draft/2020-12/schema $id: >- https://github.com/prequel-co/datafeed/internal/export/api/models/create-destination-request responses: '200': description: OK content: application/json: schema: properties: status: type: string const: success data: $ref: '#/components/schemas/DestinationResponse' $schema: https://json-schema.org/draft/2020-12/schema $id: >- https://github.com/prequel-co/datafeed/internal/export/api/models/destination-response message: type: string type: object required: - status - data - message '400': $ref: '#/components/responses/BadRequest' '401': $ref: '#/components/responses/Unauthorized' '404': $ref: '#/components/responses/NotFound' '500': $ref: '#/components/responses/InternalServerError' security: - ApiKeyAuth: [] components: schemas: CreateDestinationRequest: properties: destination: $ref: '#/components/schemas/DestinationOptions' additionalProperties: false type: object DestinationResponse: properties: destination: $ref: '#/components/schemas/DestinationResource' additionalProperties: false type: object required: - destination DestinationOptions: oneOf: - properties: vendor: const: abs abs: $ref: '#/components/schemas/ABSDestinationOptions' title: Azure Blob Storage required: - vendor - abs title: Azure Blob Storage - properties: vendor: const: athena athena: $ref: '#/components/schemas/AthenaDestinationOptions' title: Amazon Athena required: - vendor - athena title: Amazon Athena - properties: vendor: const: aurora_mysql aurora_mysql: $ref: '#/components/schemas/MySQLDestinationOptions' title: Amazon Aurora MySQL required: - vendor - aurora_mysql title: Amazon Aurora MySQL - properties: vendor: const: aurora_postgres aurora_postgres: $ref: '#/components/schemas/PostgresDestinationOptions' title: Amazon Aurora PostgreSQL required: - vendor - aurora_postgres title: Amazon Aurora PostgreSQL - properties: vendor: const: bigquery bigquery: $ref: '#/components/schemas/BigQueryDestinationOptions' title: Google BigQuery required: - vendor - bigquery title: Google BigQuery - properties: vendor: const: clickhouse clickhouse: $ref: '#/components/schemas/ClickhouseDestinationOptions' title: ClickHouse required: - vendor - clickhouse title: ClickHouse - properties: vendor: const: databricks databricks: $ref: '#/components/schemas/DatabricksDestinationOptions' title: Databricks required: - vendor - databricks title: Databricks - properties: vendor: const: delta_lake delta_lake: $ref: '#/components/schemas/DeltaLakeDestinationOptions' title: Delta Lake required: - vendor - delta_lake title: Delta Lake - properties: vendor: const: gcs gcs: $ref: '#/components/schemas/GCSDestinationOptions' title: Google Cloud Storage required: - vendor - gcs title: Google Cloud Storage - properties: vendor: const: google_sheets google_sheets: $ref: '#/components/schemas/GoogleSheetsDestinationOptions' title: Google Sheets required: - vendor - google_sheets title: Google Sheets - properties: vendor: const: iceberg iceberg: $ref: '#/components/schemas/IcebergDestinationOptions' title: Iceberg required: - vendor - iceberg title: Iceberg - properties: vendor: const: mock mock: $ref: '#/components/schemas/MockDestinationOptions' title: Mock (for testing in staging only) required: - vendor - mock title: Mock (for testing in staging only) - properties: vendor: const: mongodb mongodb: $ref: '#/components/schemas/MongoDBDestinationOptions' title: MongoDB required: - vendor - mongodb title: MongoDB - properties: vendor: const: motherduck motherduck: $ref: '#/components/schemas/MotherDuckDestinationOptions' title: MotherDuck required: - vendor - motherduck title: MotherDuck - properties: vendor: const: mysql mysql: $ref: '#/components/schemas/MySQLDestinationOptions' title: MySQL required: - vendor - mysql title: MySQL - properties: vendor: const: oracle oracle: $ref: '#/components/schemas/OracleDestinationOptions' title: Oracle required: - vendor - oracle title: Oracle - properties: vendor: const: planetscale planetscale: $ref: '#/components/schemas/MySQLDestinationOptions' title: PlanetScale required: - vendor - planetscale title: PlanetScale - properties: vendor: const: postgres postgres: $ref: '#/components/schemas/PostgresDestinationOptions' title: PostgreSQL required: - vendor - postgres title: PostgreSQL - properties: vendor: const: redshift redshift: $ref: '#/components/schemas/RedshiftDestinationOptions' title: Amazon Redshift required: - vendor - redshift title: Amazon Redshift - properties: vendor: const: redshift_serverless redshift_serverless: $ref: '#/components/schemas/RedshiftServerlessDestinationOptions' title: Amazon Redshift Serverless required: - vendor - redshift_serverless title: Amazon Redshift Serverless - properties: vendor: const: sftp sftp: $ref: '#/components/schemas/SFTPDestinationOptions' title: SFTP required: - vendor - sftp title: SFTP - properties: vendor: const: singlestore singlestore: $ref: '#/components/schemas/MySQLDestinationOptions' title: SingleStore required: - vendor - singlestore title: SingleStore - properties: vendor: const: snowflake snowflake: $ref: '#/components/schemas/SnowflakeDestinationOptions' title: Snowflake required: - vendor - snowflake title: Snowflake - properties: vendor: const: sql_server sql_server: $ref: '#/components/schemas/SQLServerDestinationOptions' title: SQL Server required: - vendor - sql_server title: SQL Server - properties: vendor: const: s3 s3: $ref: '#/components/schemas/S3DestinationOptions' title: Amazon S3 required: - vendor - s3 title: Amazon S3 - properties: vendor: const: s3_compatible s3_compatible: $ref: '#/components/schemas/S3CompatibleDestinationOptions' title: S3 Compatible required: - vendor - s3_compatible title: S3 Compatible properties: name: type: string title: Destination name description: >- Descriptive and unique name for this destination. This will only be visible internally and is only used as a reference. frequency_minutes: type: integer enum: - 0 - 10 - 15 - 60 - 360 - 720 - 1440 title: Frequency minutes description: 'Number of minutes between transfers. Allowed values: 15' recipient_id: type: string format: uuid title: Recipient ID description: >- ID of [Recipient](https://docs.prequel.co/export/concepts/recipients) created for this destination. enabled_models: items: type: string type: array description: List of models to enable for this destination. If empty max_concurrent_transfers: type: integer description: >- The maximum number of transfers that can write to this destination at the same time. Transfers with overlapping model assignments cannot be run simultaneously. max_concurrent_queries_per_transfer: type: integer description: >- The maximum number of queries that can be run in parallel for a single transfer. This is used to limit the number of concurrent queries that can be run for a single transfer to avoid overloading the destination. is_enabled: type: boolean description: Whether the destination is enabled (vs paused). default: true machine_type_id: type: string format: uuid type: object required: - name - recipient_id unevaluatedProperties: false DestinationResource: oneOf: - properties: abs: $ref: '#/components/schemas/ABSDestinationResource' required: - abs title: abs - properties: athena: $ref: '#/components/schemas/AthenaDestinationResource' required: - athena title: athena - properties: aurora_mysql: $ref: '#/components/schemas/MySQLDestinationResource' required: - aurora_mysql title: aurora_mysql - properties: aurora_postgres: $ref: '#/components/schemas/PostgresDestinationResource' required: - aurora_postgres title: aurora_postgres - properties: bigquery: $ref: '#/components/schemas/BigQueryDestinationResource' required: - bigquery title: bigquery - properties: clickhouse: $ref: '#/components/schemas/ClickhouseDestinationResource' required: - clickhouse title: clickhouse - properties: databricks: $ref: '#/components/schemas/DatabricksDestinationResource' required: - databricks title: databricks - properties: delta_lake: $ref: '#/components/schemas/DeltaLakeDestinationResource' required: - delta_lake title: delta_lake - properties: gcs: $ref: '#/components/schemas/GCSDestinationResource' required: - gcs title: gcs - properties: google_sheets: $ref: '#/components/schemas/GoogleSheetsDestinationResource' required: - google_sheets title: google_sheets - properties: iceberg: $ref: '#/components/schemas/IcebergDestinationResource' required: - iceberg title: iceberg - properties: mongodb: $ref: '#/components/schemas/MongoDBDestinationResource' required: - mongodb title: mongodb - properties: motherduck: $ref: '#/components/schemas/MotherDuckDestinationResource' required: - motherduck title: motherduck - properties: mysql: $ref: '#/components/schemas/MySQLDestinationResource' required: - mysql title: mysql - properties: oracle: $ref: '#/components/schemas/OracleDestinationResource' required: - oracle title: oracle - properties: planetscale: $ref: '#/components/schemas/MySQLDestinationResource' required: - planetscale title: planetscale - properties: postgres: $ref: '#/components/schemas/PostgresDestinationResource' required: - postgres title: postgres - properties: redshift: $ref: '#/components/schemas/RedshiftDestinationResource' required: - redshift title: redshift - properties: redshift_serverless: $ref: '#/components/schemas/RedshiftServerlessDestinationResource' required: - redshift_serverless title: redshift_serverless - properties: sftp: $ref: '#/components/schemas/SFTPDestinationResource' required: - sftp title: sftp - properties: singlestore: $ref: '#/components/schemas/MySQLDestinationResource' required: - singlestore title: singlestore - properties: snowflake: $ref: '#/components/schemas/SnowflakeDestinationResource' required: - snowflake title: snowflake - properties: sql_server: $ref: '#/components/schemas/SQLServerDestinationResource' required: - sql_server title: sql_server - properties: s3: $ref: '#/components/schemas/S3DestinationResource' required: - s3 title: s3 - properties: s3_compatible: $ref: '#/components/schemas/S3CompatibleDestinationResource' required: - s3_compatible title: s3_compatible properties: id: type: string description: The ID of the destination. vendor: type: string description: The vendor of the destination. name: type: string description: The name of the destination. frequency_minutes: type: integer description: The frequency at which the destination will be updated. id_in_provider_system: type: string description: The ID of the destination in the provider's system. recipient_id: type: string description: The ID of the recipient that this destination is associated with. products: items: type: string type: array description: The list of products that are enabled for this destination. enabled_models: items: type: string type: array description: The list of models that are enabled for this destination. max_concurrent_transfers: type: integer description: >- The maximum number of concurrent transfers allowed for this destination. max_concurrent_queries_per_transfer: type: integer description: >- The maximum number of queries that can be run in parallel for a single transfer. This is used to limit the number of concurrent queries that can be run for a single transfer to avoid overloading the destination. is_enabled: type: boolean description: Whether the destination is enabled (vs paused). created_at: type: string format: date-time description: When this destination was created. updated_at: type: string format: date-time description: When this destination was last updated. last_successful_transfer_ended_at: type: string format: date-time description: The timestamp when the last successful transfer ended. last_completed_transfer: $ref: '#/components/schemas/ScannableTransfer' description: The ID of the last successful transfer. total_rows_transferred_last_billing_period: type: integer total_delta_rows_transferred_last_billing_period: type: integer total_rows_transferred_current_billing_period: type: integer type: object required: - id - vendor - name - recipient_id - created_at - updated_at - last_successful_transfer_ended_at unevaluatedProperties: false ErrorResponse: properties: status: type: string const: error message: type: string data: type: 'null' type: object required: - status - message ABSDestinationOptions: properties: folder: type: string title: Folder Name description: >- Name of folder to create and write data to. It will be created automatically if the folder does not exist. file_format: type: string enum: - parquet - csv - json - jsonl title: File format description: >- File format to write to the bucket. Parquet is the default and recommended format. default: parquet csv: $ref: '#/components/schemas/FileFormatCSV' description: Required details if writing CSV file format. container_name: type: string title: Bucket name description: Name of the Azure Blob Storage container to write data to. storage_account_name: type: string title: Storage account name description: Name of the Azure storage account that owns the container. auth_method: type: string enum: - azure_service_shared_access_signature title: Auth method description: >- Authentication method. Currently must be set to azure_service_shared_access_signature. azure_service_shared_access_signature: $ref: '#/components/schemas/AzureServiceSharedAccessSignature' description: >- Required parameters to authenticate to Azure using a Service shared access signature. disable_manifest: type: boolean title: Disable Manifest Files description: >- Disables JSON manifest files (containing validation metadata) written with each batch of data. default: false additionalProperties: false type: object required: - folder - container_name - storage_account_name - auth_method AthenaDestinationOptions: oneOf: - $ref: '#/components/schemas/AthenaDestinationOptionsAWSIAMRole' - $ref: '#/components/schemas/AthenaDestinationOptionsAWSAccessKeys' discriminator: mapping: aws_access_keys: $ref: '#/components/schemas/AthenaDestinationOptionsAWSAccessKeys' aws_iam_role: $ref: '#/components/schemas/AthenaDestinationOptionsAWSIAMRole' propertyName: auth_method MySQLDestinationOptions: properties: host: type: string title: Host description: Hostname of the destination server. default: example.ab02cdefgxyz.us-east-1.rds.amazonaws.com port: type: integer exclusiveMinimum: 0 title: Port description: Port of the destination server. default: 3306 connection_database: type: string title: Database description: Name of the database to connect to. write_database: type: string pattern: ^[A-Za-z0-9_.]+([-][A-Za-z0-9_.]+)*$ title: Schema description: >- Name of schema to create and write data to. It will be created automatically if the schema does not exist. auth_method: type: string enum: - password_auth title: Auth method description: >- Placeholder value in case authentication options are added. Currently must be set to password_auth. password_auth: $ref: '#/components/schemas/PasswordAuth' description: Required details for username/password authentication. use_ssh_tunnel: type: boolean description: Whether to use an SSH tunnel for the connection. ssh_tunnel: $ref: '#/components/schemas/SSHTunnelParams' description: Required details if using SSH tunneling. disable_ssl: type: boolean title: Disable SSL description: >- Whether to disable SSL for the connection. When false: SSL is preferred. default: false additionalProperties: false type: object required: - host - port - connection_database - write_database - auth_method - password_auth PostgresDestinationOptions: properties: host: type: string title: Host description: Host address of the database. default: example.ab02cdefgxyz.us-east-1.rds.amazonaws.com port: type: integer exclusiveMinimum: 0 title: Port description: Port of the PostgreSQL server. default: 5432 database: type: string title: Database description: Name of database provisioned by the owner of the destination. schema: type: string pattern: ^[A-Za-z0-9_.]+([-][A-Za-z0-9_.]+)*$ title: Schema description: >- Name of schema to create and write data to. It will be created automatically if the schema does not exist. auth_method: type: string enum: - password_auth title: Auth method description: >- Placeholder value in case authentication options are added. Currently must be set to password_auth. password_auth: $ref: '#/components/schemas/PasswordAuth' description: Required details for username/password authentication. use_ssh_tunnel: type: boolean description: Whether to use an SSH tunnel for the connection. ssh_tunnel: $ref: '#/components/schemas/SSHTunnelParams' description: Required details if using SSH tunneling. disable_ssl: type: boolean title: Disable SSL description: >- Whether to disable SSL for the PostgreSQL connection. When false: SSL is required. default: false connection_timeout_in_seconds: type: integer minimum: 0 title: Connection Timeout description: Duration to wait for successful connection to Postgres additionalProperties: false type: object required: - schema - auth_method - password_auth BigQueryDestinationOptions: oneOf: - $ref: '#/components/schemas/BigQueryDestinationOptionsGCPServiceAccountRole' - $ref: >- #/components/schemas/BigQueryDestinationOptionsFederatedGCPServiceAccountRole - $ref: '#/components/schemas/BigQueryDestinationOptionsGCPServiceAccountKey' discriminator: mapping: federated_gcp_service_account_role: $ref: >- #/components/schemas/BigQueryDestinationOptionsFederatedGCPServiceAccountRole gcp_service_account_key: $ref: >- #/components/schemas/BigQueryDestinationOptionsGCPServiceAccountKey gcp_service_account_role: $ref: >- #/components/schemas/BigQueryDestinationOptionsGCPServiceAccountRole propertyName: auth_method ClickhouseDestinationOptions: oneOf: - properties: bucket_vendor: const: bucket_s3 bucket_s3: $ref: '#/components/schemas/ClickhouseS3BucketParams' title: Staging bucket hosted on S3 description: Required details about the S3 bucket used for staging data. required: - bucket_vendor - bucket_s3 title: Staging bucket hosted on S3 - properties: bucket_vendor: const: bucket_gcs bucket_gcs: $ref: '#/components/schemas/ClickhouseGCSBucketParams' title: Staging bucket hosted on GCS description: Required details about the GCS bucket used for staging data. required: - bucket_vendor - bucket_gcs title: Staging bucket hosted on GCS - properties: bucket_vendor: const: bucket_implicit required: - bucket_vendor title: Staging bucket implicit properties: host: type: string title: Host description: Hostname of the ClickHouse server. port: type: integer exclusiveMinimum: 0 title: Port description: Port of the ClickHouse server. cluster: type: string title: Cluster description: >- Optional field to specify the Cluster to use in 'ON CLUSTER' statements. connection_database: type: string title: Connection database description: Name of the ClickHouse database to use for the initial connection. write_database: type: string pattern: ^[A-Za-z0-9_.]+([-][A-Za-z0-9_.]+)*$ title: Write database description: >- Name of the ClickHouse database where tables will be created and data will be written. It will be created automatically if it does not exist. auth_method: type: string enum: - password_auth title: Auth method description: >- Placeholder value in case authentication options are added. Currently must be set to password_auth. password_auth: $ref: '#/components/schemas/PasswordAuth' description: Required details for username/password authentication. disable_ssl: type: boolean title: Disable SSL description: >- Whether to disable SSL for the ClickHouse connection. When false: SSL is enabled. default: false type: object required: - host - port - connection_database - write_database - auth_method - password_auth unevaluatedProperties: false DatabricksDestinationOptions: allOf: - oneOf: - properties: auth_method: const: access_token_auth access_token_auth: $ref: '#/components/schemas/AccessToken' description: Required details if authenticating using an access token. required: - auth_method - access_token_auth - properties: auth_method: const: oauth_client_secret_auth oauth_client_secret_auth: $ref: '#/components/schemas/OauthClientSecretAuth' description: >- Required details if authenticating using a OAuth client secret. required: - auth_method - oauth_client_secret_auth - oneOf: - properties: metastore: const: hive hive: $ref: '#/components/schemas/DatabricksHiveOptions' title: Hive Metastore required: - metastore - hive title: Hive Metastore - properties: metastore: const: unity_catalog required: - metastore title: Unity Catalog Metastore properties: host: type: string title: Server hostname description: >- Server hostname of the SQL endpoint provisioned by the owner of the destination. default: adb-1099123456789876.12.azuredatabricks.net port: type: integer exclusiveMinimum: 0 title: Port description: Port of the Databricks SQL endpoint. default: 443 schema: type: string pattern: ^[A-Za-z0-9_.]+([-][A-Za-z0-9_.]+)*$ title: Schema description: >- Name of schema to create and write data to. It will be created automatically if the schema does not exist. catalog: type: string title: Catalog description: Name of catalog to write data to. http_path: type: string title: HTTP path description: >- HTTP path of the SQL endpoint provisioned by the owner of the destination. default: /sql/1.0/endpoints/a98765ab4c3210a0 use_ssh_tunnel: type: boolean description: Whether to use an SSH tunnel for the connection. ssh_tunnel: $ref: '#/components/schemas/SSHTunnelParams' description: Required details if using SSH tunneling. type: object required: - host - port - schema - catalog - http_path unevaluatedProperties: false DeltaLakeDestinationOptions: oneOf: - properties: storage: const: bucket_abs bucket_abs: $ref: '#/components/schemas/OpenTableFormatABSBucketParams' description: >- Required details about the ABS bucket used for the delta lake table. required: - storage - bucket_abs title: bucket_abs - properties: storage: const: bucket_gcs bucket_gcs: $ref: '#/components/schemas/OpenTableFormatGCSBucketParams' description: >- Required details about the GCS bucket used for the delta lake table. required: - storage - bucket_gcs title: bucket_gcs - properties: storage: const: bucket_s3 bucket_s3: $ref: '#/components/schemas/OpenTableFormatS3BucketParams' description: >- Required details about the S3 bucket used for the delta lake table. required: - storage - bucket_s3 title: bucket_s3 - properties: storage: const: bucket_s3_compatible bucket_s3_compatible: $ref: '#/components/schemas/OpenTableFormatS3CompatibleBucketParams' description: >- Required details about the S3 compatible bucket used for the delta lake table. required: - storage - bucket_s3_compatible title: bucket_s3_compatible properties: retention_window_days: type: integer title: Retention window (days) description: The number of days a file must be deleted before it is vacuumed. default: 7 column_mapping_mode: type: string enum: - none - id - name title: Column mapping mode description: >- Column mapping mode for the Delta Lake table. Use none for no mapping; name to track columns by name (allows rename and drop without rewrite); or id to track columns by numeric ID. default: none deletion_vectors_disabled: type: boolean title: Disable deletion vectors description: >- Whether to disable deletion vectors. When false: deletions are recorded as markers without rewriting table files. default: false change_data_feed_disabled: type: boolean title: Disable change data feed description: >- Whether to disable change data feed. When false: row-level change tracking is maintained for the table. default: false type: object unevaluatedProperties: false GCSDestinationOptions: oneOf: - $ref: '#/components/schemas/GCSDestinationOptionsGCPServiceAccountRole' - $ref: '#/components/schemas/GCSDestinationOptionsGCSHMACKeys' - $ref: >- #/components/schemas/GCSDestinationOptionsFederatedGCPServiceAccountRole discriminator: mapping: federated_gcp_service_account_role: $ref: >- #/components/schemas/GCSDestinationOptionsFederatedGCPServiceAccountRole gcp_service_account_role: $ref: '#/components/schemas/GCSDestinationOptionsGCPServiceAccountRole' gcs_hmac_keys: $ref: '#/components/schemas/GCSDestinationOptionsGCSHMACKeys' propertyName: auth_method GoogleSheetsDestinationOptions: properties: host: type: string pattern: ^(https?:\/\/)?(www.)?docs.google.com\/spreadsheets\/d\/.+(\/)?$ title: Spreadsheet URL default: https://docs.google.com/spreadsheets/d/1234567 additionalProperties: false type: object required: - host IcebergDestinationOptions: oneOf: - properties: catalog: const: catalog_iceberg_rest catalog_iceberg_rest: $ref: '#/components/schemas/IcebergRestCatalogParams' description: >- Required details about the Iceberg Rest Catalog used for the iceberg table. required: - catalog - catalog_iceberg_rest title: catalog_iceberg_rest - properties: catalog: const: catalog_glue catalog_glue: $ref: '#/components/schemas/IcebergAwsGlueCatalogParams' description: >- Required details about the AWS Glue Catalog used for the iceberg table. required: - catalog - catalog_glue title: catalog_glue - properties: catalog: const: catalog_s3_tables catalog_s3_tables: $ref: '#/components/schemas/IcebergAwsS3TablesCatalogParams' description: >- Required details about the AWS S3 Tables bucket used for the iceberg table. required: - catalog - catalog_s3_tables title: catalog_s3_tables - properties: catalog: const: catalog_google_lakehouse catalog_google_lakehouse: $ref: '#/components/schemas/IcebergGoogleLakehouseCatalogParams' description: >- Required details about the Google Lakehouse Catalog used for the iceberg table. required: - catalog - catalog_google_lakehouse title: catalog_google_lakehouse properties: retention_window_days: type: integer title: Retention window (days) description: >- The minimum number of days to retain old snapshots before they are expired. default: 7 type: object unevaluatedProperties: false MockDestinationOptions: properties: schema: type: string description: Mock schema name. database: type: string description: Mock database name. error_code: type: string additionalProperties: false type: object required: - schema - database MongoDBDestinationOptions: oneOf: - $ref: '#/components/schemas/MongoDBDestinationOptionsAWSIAMRole' - $ref: '#/components/schemas/MongoDBDestinationOptionsPasswordAuth' discriminator: mapping: aws_iam_role: $ref: '#/components/schemas/MongoDBDestinationOptionsAWSIAMRole' password_auth: $ref: '#/components/schemas/MongoDBDestinationOptionsPasswordAuth' propertyName: auth_method MotherDuckDestinationOptions: properties: database: type: string title: Database description: MotherDuck database name. schema: type: string title: Schema description: MotherDuck schema name. auth_method: type: string enum: - access_token_auth title: Auth method description: >- Placeholder value in case authentication options are added. Currently must be set to access_token_auth. access_token_auth: $ref: '#/components/schemas/AccessToken' description: The access token authentication object. additionalProperties: false type: object required: - database - schema - auth_method OracleDestinationOptions: oneOf: - properties: disable_ssl: const: false ssl_options: $ref: '#/components/schemas/OracleSSLParams' description: Required details if using SSL connection. required: - ssl_options title: SSL Enabled - properties: disable_ssl: type: boolean title: Disable SSL description: >- Whether to disable SSL for the Oracle connection. When false: SSL is enabled. default: false required: - disable_ssl title: SSL Disabled properties: host: type: string title: Host description: Host address of the database. default: example.ab02cdefgxyz.us-east-1.rds.amazonaws.com port: type: integer exclusiveMinimum: 0 title: Port description: Port of the Oracle database server. default: 1521 database: type: string title: Database description: Name of database provisioned by the owner of the destination. schema: type: string pattern: ^[A-Za-z0-9_.]+([-][A-Za-z0-9_.]+)*$ title: Schema description: >- Name of schema to create and write data to. It will be created automatically if the schema does not exist. auth_method: type: string enum: - password_auth title: Auth method description: >- Placeholder value in case authentication options are added. Currently must be set to password_auth. password_auth: $ref: '#/components/schemas/PasswordAuth' description: Required details for username/password authentication. use_ssh_tunnel: type: boolean description: Whether to use an SSH tunnel for the connection. ssh_tunnel: $ref: '#/components/schemas/SSHTunnelParams' description: Required details if using SSH tunneling. type: object required: - schema - auth_method - password_auth unevaluatedProperties: false RedshiftDestinationOptions: oneOf: - $ref: '#/components/schemas/RedshiftDestinationOptionsAWSAccessKeys' - $ref: '#/components/schemas/RedshiftDestinationOptionsAWSIAMRole' discriminator: mapping: aws_access_keys: $ref: '#/components/schemas/RedshiftDestinationOptionsAWSAccessKeys' aws_iam_role: $ref: '#/components/schemas/RedshiftDestinationOptionsAWSIAMRole' propertyName: auth_method RedshiftServerlessDestinationOptions: oneOf: - $ref: '#/components/schemas/RedshiftServerlessDestinationOptionsAWSIAMRole' - $ref: >- #/components/schemas/RedshiftServerlessDestinationOptionsAWSAccessKeys discriminator: mapping: aws_access_keys: $ref: >- #/components/schemas/RedshiftServerlessDestinationOptionsAWSAccessKeys aws_iam_role: $ref: >- #/components/schemas/RedshiftServerlessDestinationOptionsAWSIAMRole propertyName: auth_method SFTPDestinationOptions: oneOf: - properties: file_encryption: const: pgp pgp: $ref: '#/components/schemas/PGPEncryption' description: Required details for PGP encryption. required: - file_encryption - pgp properties: username: type: string title: Username description: Username to use for the SFTP connection. host: type: string title: Host description: Hostname of the SFTP server. port: type: integer exclusiveMinimum: 0 title: Port default: 22 folder: type: string title: Folder name description: >- Name of folder to create and write data to. It will be created automatically if it does not exist. file_format: type: string enum: - csv - parquet - json - jsonl title: File format description: The format of the files to write to the SFTP server. csv: $ref: '#/components/schemas/FileFormatCSV' description: Required details if writing CSV file format. auth_method: type: string enum: - public_key_auth title: Auth method description: Authentication method. Currently must be set to public_key_auth. public_key_auth: $ref: '#/components/schemas/PublicKeyAuth' title: Public key auth description: Required details for public key authentication. type: object required: - username - host - port - file_format - auth_method - public_key_auth unevaluatedProperties: false SnowflakeDestinationOptions: oneOf: - $ref: '#/components/schemas/SnowflakeDestinationOptionsPasswordAuth' - $ref: '#/components/schemas/SnowflakeDestinationOptionsPublicKeyAuth' discriminator: mapping: password_auth: $ref: '#/components/schemas/SnowflakeDestinationOptionsPasswordAuth' public_key_auth: $ref: '#/components/schemas/SnowflakeDestinationOptionsPublicKeyAuth' propertyName: auth_method SQLServerDestinationOptions: properties: host: type: string title: Host description: Host address of the database. default: example.ab02cdefgxyz.us-east-1.rds.amazonaws.com port: type: integer exclusiveMinimum: 0 title: Port description: Port of the SQL Server instance. default: 1433 database: type: string title: Database description: Name of database provisioned by the owner of the destination. schema: type: string pattern: ^[A-Za-z0-9_.]+([-][A-Za-z0-9_.]+)*$ title: Schema description: >- Name of schema to create and write data to. It will be created automatically if the schema does not exist. auth_method: type: string enum: - password_auth title: Auth method description: >- Placeholder value in case authentication options are added. Currently must be set to password_auth. password_auth: $ref: '#/components/schemas/PasswordAuth' description: Required details for username/password authentication. use_ssh_tunnel: type: boolean description: Whether to use an SSH tunnel for the connection. ssh_tunnel: $ref: '#/components/schemas/SSHTunnelParams' description: Required details if using SSH tunneling. disable_ssl: type: boolean title: Disable SSL description: >- Whether to disable SSL for the SQL Server connection. When false: SSL is enabled. default: false primary_key_collation: type: string title: Primary Key Collation description: >- SQL Server collation to use for primary key column. Leave empty to use database default collation. additionalProperties: false type: object required: - schema - auth_method - password_auth S3DestinationOptions: oneOf: - $ref: '#/components/schemas/S3DestinationOptionsAWSIAMRole' - $ref: '#/components/schemas/S3DestinationOptionsAWSAccessKeys' discriminator: mapping: aws_access_keys: $ref: '#/components/schemas/S3DestinationOptionsAWSAccessKeys' aws_iam_role: $ref: '#/components/schemas/S3DestinationOptionsAWSIAMRole' propertyName: auth_method S3CompatibleDestinationOptions: properties: host: type: string title: Bucket host description: Hostname of the S3-compatible server. default: s3.us-east-1.amazonaws.com port: type: integer exclusiveMinimum: 0 title: Port description: Port of the S3-compatible server. default: 443 folder: type: string title: Folder name description: >- Name of folder to create and write data to. It will be created automatically if it does not exist. file_format: type: string enum: - parquet - csv - json - jsonl title: File format description: >- File format to write to the bucket. Parquet is the default and recommended format. default: parquet csv: $ref: '#/components/schemas/FileFormatCSV' description: Required details if writing CSV file format. bucket_name: type: string title: Bucket name description: Name of the S3-compatible bucket to write data to. disable_ssl: type: boolean title: Disable SSL description: >- Whether to disable SSL for the S3-compatible connection. When false: SSL is enabled. default: false auth_method: type: string enum: - access_keys title: Auth method description: >- Placeholder value in case authentication options are added. Currently must be set to access_keys. access_keys: $ref: '#/components/schemas/AccessKeys' description: Required details for access key authentication. use_ssh_tunnel: type: boolean description: Whether to use an SSH tunnel for the connection. ssh_tunnel: $ref: '#/components/schemas/SSHTunnelParams' description: Required details if using SSH tunneling. disable_manifest: type: boolean title: Disable Manifest Files description: >- Disables JSON manifest files (containing validation metadata) written with each batch of data. default: false additionalProperties: false type: object required: - host - port - folder - bucket_name - auth_method - access_keys ABSDestinationResource: properties: disable_manifest: type: boolean title: Disable Manifest Files description: >- Disables JSON manifest files (containing validation metadata) written with each batch of data. folder: type: string container_name: type: string auth_method: type: string storage_account_name: type: string file_format: type: string enum: - parquet - csv - json - jsonl csv: $ref: '#/components/schemas/FileFormatCSV' additionalProperties: false type: object required: - folder - container_name - auth_method - storage_account_name AthenaDestinationResource: oneOf: - $ref: '#/components/schemas/AthenaDestinationResourceAWSIAMRole' - $ref: '#/components/schemas/AthenaDestinationResourceAWSAccessKeys' discriminator: mapping: aws_access_keys: $ref: '#/components/schemas/AthenaDestinationResourceAWSAccessKeys' aws_iam_role: $ref: '#/components/schemas/AthenaDestinationResourceAWSIAMRole' propertyName: auth_method MySQLDestinationResource: properties: host: type: string port: type: integer connection_database: type: string write_database: type: string auth_method: type: string enum: - password_auth password_auth: $ref: '#/components/schemas/PasswordAuthResource' use_ssh_tunnel: type: boolean ssh_tunnel: $ref: '#/components/schemas/SSHTunnelResource' disable_ssl: type: boolean additionalProperties: false type: object required: - host - port - connection_database - write_database - auth_method - password_auth - use_ssh_tunnel PostgresDestinationResource: properties: host: type: string port: type: integer database: type: string schema: type: string auth_method: type: string enum: - password_auth password_auth: $ref: '#/components/schemas/PasswordAuthResource' use_ssh_tunnel: type: boolean ssh_tunnel: $ref: '#/components/schemas/SSHTunnelResource' disable_ssl: type: boolean connection_timeout_in_seconds: type: integer title: Connection Timeout description: Duration to wait for successful connection to Postgres additionalProperties: false type: object required: - host - port - database - schema - auth_method - password_auth - use_ssh_tunnel BigQueryDestinationResource: oneOf: - $ref: >- #/components/schemas/BigQueryDestinationResourceGCPServiceAccountRole - $ref: >- #/components/schemas/BigQueryDestinationResourceFederatedGCPServiceAccountRole - $ref: '#/components/schemas/BigQueryDestinationResourceGCPServiceAccountKey' discriminator: mapping: federated_gcp_service_account_role: $ref: >- #/components/schemas/BigQueryDestinationResourceFederatedGCPServiceAccountRole gcp_service_account_key: $ref: >- #/components/schemas/BigQueryDestinationResourceGCPServiceAccountKey gcp_service_account_role: $ref: >- #/components/schemas/BigQueryDestinationResourceGCPServiceAccountRole propertyName: auth_method ClickhouseDestinationResource: properties: cluster: type: string host: type: string port: type: integer connection_database: type: string write_database: type: string auth_method: type: string enum: - password_auth password_auth: $ref: '#/components/schemas/PasswordAuthResource' bucket_vendor: type: string bucket_s3: $ref: '#/components/schemas/ClickhouseS3BucketResource' bucket_gcs: $ref: '#/components/schemas/ClickhouseGCSBucketResource' disable_ssl: type: boolean additionalProperties: false type: object required: - host - port - connection_database - write_database - auth_method - password_auth - bucket_vendor DatabricksDestinationResource: oneOf: - properties: metastore: const: hive hive: $ref: '#/components/schemas/DatabricksHiveResource' title: Hive Metastore required: - metastore - hive title: Hive Metastore - properties: metastore: const: unity_catalog required: - metastore title: Unity Catalog Metastore properties: host: type: string port: type: integer schema: type: string catalog: type: string http_path: type: string auth_method: type: string enum: - access_token_auth - oauth_client_secret_auth oauth_client_secret_auth: $ref: '#/components/schemas/OauthClientSecretAuthResource' use_ssh_tunnel: type: boolean ssh_tunnel: $ref: '#/components/schemas/SSHTunnelResource' type: object required: - host - port - schema - catalog - http_path - auth_method - use_ssh_tunnel unevaluatedProperties: false DeltaLakeDestinationResource: properties: storage: type: string bucket_abs: $ref: '#/components/schemas/OpenTableFormatABSBucketResource' bucket_gcs: $ref: '#/components/schemas/OpenTableFormatGCSBucketResource' bucket_s3: $ref: '#/components/schemas/OpenTableFormatS3BucketResource' bucket_s3_compatible: $ref: '#/components/schemas/OpenTableFormatS3CompatibleBucketResource' retention_window_days: type: integer column_mapping_mode: type: string enum: - none - id - name deletion_vectors_disabled: type: boolean change_data_feed_disabled: type: boolean additionalProperties: false type: object required: - storage - retention_window_days - column_mapping_mode GCSDestinationResource: oneOf: - $ref: >- #/components/schemas/GCSDestinationResourceFederatedGCPServiceAccountRole - $ref: '#/components/schemas/GCSDestinationResourceGCPServiceAccountRole' - $ref: '#/components/schemas/GCSDestinationResourceGCSHMACKeys' discriminator: mapping: federated_gcp_service_account_role: $ref: >- #/components/schemas/GCSDestinationResourceFederatedGCPServiceAccountRole gcp_service_account_role: $ref: '#/components/schemas/GCSDestinationResourceGCPServiceAccountRole' gcs_hmac_keys: $ref: '#/components/schemas/GCSDestinationResourceGCSHMACKeys' propertyName: auth_method GoogleSheetsDestinationResource: properties: host: type: string additionalProperties: false type: object required: - host IcebergDestinationResource: properties: catalog: type: string enum: - catalog_glue - catalog_s3_tables - catalog_iceberg_rest - catalog_google_lakehouse title: Catalog catalog_iceberg_rest: $ref: '#/components/schemas/IcebergRestCatalogResource' catalog_s3_tables: $ref: '#/components/schemas/IcebergAwsS3TablesCatalogResource' catalog_glue: $ref: '#/components/schemas/IcebergAwsGlueCatalogResource' catalog_google_lakehouse: $ref: '#/components/schemas/IcebergGoogleLakehouseCatalogResource' retention_window_days: type: integer additionalProperties: false type: object required: - catalog - retention_window_days MongoDBDestinationResource: oneOf: - $ref: '#/components/schemas/MongoDBDestinationResourceAWSIAMRole' - $ref: '#/components/schemas/MongoDBDestinationResourcePasswordAuth' discriminator: mapping: aws_iam_role: $ref: '#/components/schemas/MongoDBDestinationResourceAWSIAMRole' password_auth: $ref: '#/components/schemas/MongoDBDestinationResourcePasswordAuth' propertyName: auth_method MotherDuckDestinationResource: properties: database: type: string schema: type: string auth_method: type: string enum: - access_token_auth additionalProperties: false type: object required: - database - schema - auth_method OracleDestinationResource: properties: host: type: string port: type: integer database: type: string schema: type: string auth_method: type: string enum: - password_auth password_auth: $ref: '#/components/schemas/PasswordAuthResource' use_ssh_tunnel: type: boolean ssh_tunnel: $ref: '#/components/schemas/SSHTunnelResource' disable_ssl: type: boolean additionalProperties: false type: object required: - host - port - database - schema - auth_method - password_auth - use_ssh_tunnel RedshiftDestinationResource: oneOf: - $ref: '#/components/schemas/RedshiftDestinationResourceAWSAccessKeys' - $ref: '#/components/schemas/RedshiftDestinationResourceAWSIAMRole' discriminator: mapping: aws_access_keys: $ref: '#/components/schemas/RedshiftDestinationResourceAWSAccessKeys' aws_iam_role: $ref: '#/components/schemas/RedshiftDestinationResourceAWSIAMRole' propertyName: auth_method RedshiftServerlessDestinationResource: oneOf: - $ref: >- #/components/schemas/RedshiftServerlessDestinationResourceAWSAccessKeys - $ref: '#/components/schemas/RedshiftServerlessDestinationResourceAWSIAMRole' discriminator: mapping: aws_access_keys: $ref: >- #/components/schemas/RedshiftServerlessDestinationResourceAWSAccessKeys aws_iam_role: $ref: >- #/components/schemas/RedshiftServerlessDestinationResourceAWSIAMRole propertyName: auth_method SFTPDestinationResource: oneOf: - properties: file_encryption: const: pgp pgp: $ref: '#/components/schemas/PGPEncryption' description: Required details for PGP encryption. required: - file_encryption - pgp properties: username: type: string host: type: string port: type: integer folder: type: string file_format: type: string enum: - parquet - csv - json - jsonl csv: $ref: '#/components/schemas/FileFormatCSV' auth_method: type: string enum: - public_key_auth public_key_auth: $ref: '#/components/schemas/PublicKeyAuth' type: object required: - username - host - port - folder - file_format - auth_method - public_key_auth unevaluatedProperties: false SnowflakeDestinationResource: properties: username: type: string host: type: string port: type: integer database: type: string schema: type: string auth_method: type: string enum: - password_auth - public_key_auth public_key_auth: $ref: '#/components/schemas/PublicKeyAuth' additionalProperties: false type: object required: - username - host - port - database - schema - auth_method SQLServerDestinationResource: properties: host: type: string port: type: integer database: type: string schema: type: string auth_method: type: string enum: - password_auth password_auth: $ref: '#/components/schemas/PasswordAuthResource' use_ssh_tunnel: type: boolean ssh_tunnel: $ref: '#/components/schemas/SSHTunnelResource' disable_ssl: type: boolean primary_key_collation: type: string additionalProperties: false type: object required: - host - port - database - schema - auth_method - password_auth - use_ssh_tunnel S3DestinationResource: oneOf: - $ref: '#/components/schemas/S3DestinationResourceAWSIAMRole' - $ref: '#/components/schemas/S3DestinationResourceAWSAccessKeys' discriminator: mapping: aws_access_keys: $ref: '#/components/schemas/S3DestinationResourceAWSAccessKeys' aws_iam_role: $ref: '#/components/schemas/S3DestinationResourceAWSIAMRole' propertyName: auth_method S3CompatibleDestinationResource: properties: host: type: string port: type: integer exclusiveMinimum: 0 title: Port default: 443 disable_manifest: type: boolean title: Disable Manifest Files description: >- Disables JSON manifest files (containing validation metadata) written with each batch of data. folder: type: string file_format: type: string enum: - parquet - csv - json - jsonl csv: $ref: '#/components/schemas/FileFormatCSV' bucket_name: type: string auth_method: type: string enum: - access_keys access_keys: $ref: '#/components/schemas/AWSAccessKeysResource' disable_ssl: type: boolean use_ssh_tunnel: type: boolean ssh_tunnel: $ref: '#/components/schemas/SSHTunnelResource' additionalProperties: false type: object required: - host - port - folder - bucket_name - auth_method - use_ssh_tunnel ScannableTransfer: properties: id: type: string source_ids: $ref: '#/components/schemas/StringArray' destination_id: type: string status: type: string is_full_refresh: type: boolean models: $ref: '#/components/schemas/StringArray' model_metrics: $ref: '#/components/schemas/TransferModelMetrics' models_snapshot: $ref: '#/components/schemas/TransferModelsSnapshot' log: type: string submitted_at: type: string format: date-time start_transfer_window_at: type: string format: date-time end_transfer_window_at: type: string format: date-time started_at: type: string format: date-time ended_at: type: string format: date-time rows_transferred: type: integer delta_rows_transferred: type: integer volume_transferred_in_mb: type: number is_debug_mode: type: boolean tags: $ref: '#/components/schemas/TransferTags' actor_id: type: string actor_type: type: string additionalProperties: false type: object required: - id - status - is_full_refresh - models - log - submitted_at - start_transfer_window_at - end_transfer_window_at - rows_transferred - delta_rows_transferred - volume_transferred_in_mb - is_debug_mode - actor_id - actor_type FileFormatCSV: properties: delimiter: type: string title: Delimiter description: The delimiter to use for the CSV file. Must be a single character. additionalProperties: false type: object required: - delimiter AzureServiceSharedAccessSignature: properties: shared_access_signature_token: type: string title: Storage Account SAS Token additionalProperties: false type: object required: - shared_access_signature_token AthenaDestinationOptionsAWSIAMRole: properties: workgroup: type: string title: Workgroup description: >- Optional field to specify the Athena Workgroup to use. Must be configured for Athena engine version 3. If left blank connection_database: type: string title: Query results location description: >- The name of the folder where the query results will be stored. This folder will be created within the S3 bucket. write_database: type: string pattern: ^[A-Za-z0-9_]+$ title: Schema description: >- This is the name for the group of tables loaded into Athena. This is synonymous with database in Athena. bucket_s3: $ref: '#/components/schemas/S3BucketParams' description: Required details about the S3 bucket used for staging data. auth_method: const: aws_iam_role aws_iam_role: $ref: '#/components/schemas/AWSIAMRole' title: Authenticate using an AWS IAM role (recommended) type: object required: - connection_database - write_database - bucket_s3 - auth_method - aws_iam_role title: Authenticate using an AWS IAM role (recommended) unevaluatedProperties: false AthenaDestinationOptionsAWSAccessKeys: properties: workgroup: type: string title: Workgroup description: >- Optional field to specify the Athena Workgroup to use. Must be configured for Athena engine version 3. If left blank connection_database: type: string title: Query results location description: >- The name of the folder where the query results will be stored. This folder will be created within the S3 bucket. write_database: type: string pattern: ^[A-Za-z0-9_]+$ title: Schema description: >- This is the name for the group of tables loaded into Athena. This is synonymous with database in Athena. bucket_s3: $ref: '#/components/schemas/S3BucketParams' description: Required details about the S3 bucket used for staging data. auth_method: const: aws_access_keys aws_access_keys: $ref: '#/components/schemas/AWSAccessKeys' title: Authenticate using AWS access keys description: Required details if authenticating using AWS access keys. type: object required: - connection_database - write_database - bucket_s3 - auth_method - aws_access_keys title: Authenticate using AWS access keys unevaluatedProperties: false PasswordAuth: properties: username: type: string title: Username password: type: string title: Password additionalProperties: false type: object required: - username - password SSHTunnelParams: properties: ssh_tunnel_host: type: string title: SSH tunnel host ssh_tunnel_port: type: integer title: SSH tunnel port default: 22 ssh_tunnel_username: type: string title: SSH tunnel username ssh_public_key: type: string enum: - primary_public_key - secondary_public_key title: SSH Public Key description: >- This is the public SSH key that will be used to establish the SSH tunnel. It should be added to the list of authorized keys on the bastion host. additionalProperties: false type: object required: - ssh_tunnel_host - ssh_tunnel_port - ssh_tunnel_username BigQueryDestinationOptionsGCPServiceAccountRole: properties: project_id: $comment: >- BigQuery Project IDs must consist of 4-30 lowercase letters, numbers, and hyphens, and must begin with a lowercase letter and end with a lowercase letter or number. type: string pattern: ^[a-z][a-z0-9-]{2,28}[a-z0-9]$ title: Project ID schema: $comment: >- BigQuery Dataset IDs must consist of 1-1024 alphanumeric characters or underscores. type: string pattern: ^[a-zA-Z0-9_]+$ title: Dataset description: >- Name of dataset to create and write data to. It will be created automatically if the dataset does not exist. bucket_gcs: $ref: '#/components/schemas/BigQueryGCSBucketParams' description: Required details about the GCS bucket used for staging data. auth_method: const: gcp_service_account_role gcp_service_account_role: $ref: '#/components/schemas/GCPServiceAccountRole' title: Authenticate using a GCP service account (recommended) description: >- Required details if authenticating using Workload Identity Federation. type: object required: - project_id - schema - bucket_gcs - auth_method - gcp_service_account_role title: Authenticate using a GCP service account (recommended) unevaluatedProperties: false BigQueryDestinationOptionsFederatedGCPServiceAccountRole: properties: project_id: $comment: >- BigQuery Project IDs must consist of 4-30 lowercase letters, numbers, and hyphens, and must begin with a lowercase letter and end with a lowercase letter or number. type: string pattern: ^[a-z][a-z0-9-]{2,28}[a-z0-9]$ title: Project ID schema: $comment: >- BigQuery Dataset IDs must consist of 1-1024 alphanumeric characters or underscores. type: string pattern: ^[a-zA-Z0-9_]+$ title: Dataset description: >- Name of dataset to create and write data to. It will be created automatically if the dataset does not exist. bucket_gcs: $ref: '#/components/schemas/BigQueryGCSBucketParams' description: Required details about the GCS bucket used for staging data. auth_method: const: federated_gcp_service_account_role federated_gcp_service_account_role: $ref: '#/components/schemas/FederatedGCPServiceAccountRole' title: Authenticate using federation (only if self-hosting) description: >- Required details if authenticating using Workload Identity Federation. type: object required: - project_id - schema - bucket_gcs - auth_method - federated_gcp_service_account_role title: Authenticate using federation (only if self-hosting) unevaluatedProperties: false BigQueryDestinationOptionsGCPServiceAccountKey: properties: project_id: $comment: >- BigQuery Project IDs must consist of 4-30 lowercase letters, numbers, and hyphens, and must begin with a lowercase letter and end with a lowercase letter or number. type: string pattern: ^[a-z][a-z0-9-]{2,28}[a-z0-9]$ title: Project ID schema: $comment: >- BigQuery Dataset IDs must consist of 1-1024 alphanumeric characters or underscores. type: string pattern: ^[a-zA-Z0-9_]+$ title: Dataset description: >- Name of dataset to create and write data to. It will be created automatically if the dataset does not exist. bucket_gcs: $ref: '#/components/schemas/BigQueryGCSBucketParams' description: Required details about the GCS bucket used for staging data. auth_method: const: gcp_service_account_key gcp_service_account_key: $ref: '#/components/schemas/GCPServiceAccountKey' title: Authenticate using direct access description: Required details if authenticating using a GCP service account key. type: object required: - project_id - schema - bucket_gcs - auth_method - gcp_service_account_key title: Authenticate using direct access unevaluatedProperties: false ClickhouseS3BucketParams: oneOf: - properties: bucket_auth_method: const: aws_access_keys aws_access_keys: $ref: '#/components/schemas/AWSAccessKeys' title: Authenticate using AWS access keys description: Required details if authenticating using AWS access keys. required: - bucket_auth_method - aws_access_keys title: Authenticate using AWS access keys - properties: bucket_auth_method: const: aws_iam_role aws_iam_role: $ref: '#/components/schemas/AWSIAMRole' title: Authenticate using an AWS IAM role (recommended) description: Required details if authenticating using AWS IAM role. required: - bucket_auth_method - aws_iam_role title: Authenticate using an AWS IAM role (recommended) properties: bucket_name: type: string title: Bucket name bucket_region: type: string title: S3 bucket region description: >- The region of the S3 bucket. Must be a valid [AWS region](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.RegionsAndAvailabilityZones.html) type: object required: - bucket_name - bucket_region unevaluatedProperties: false ClickhouseGCSBucketParams: properties: bucket_name: type: string title: Bucket name gcs_hmac_keys: $ref: '#/components/schemas/GCSAccessKeys' description: Required details for GCS HMAC key authentication. additionalProperties: false type: object required: - bucket_name - gcs_hmac_keys AccessToken: properties: access_token: type: string title: Personal access token description: The access token to use for the connection. additionalProperties: false type: object required: - access_token OauthClientSecretAuth: properties: client_id: type: string title: Client ID description: The Databricks-generated OAuth client ID. client_secret: type: string title: Client Secret description: The Databricks-generated OAuth client secret. additionalProperties: false type: object required: - client_id - client_secret DatabricksHiveOptions: oneOf: - properties: bucket_vendor: const: bucket_s3 bucket_s3: $ref: '#/components/schemas/DatabricksS3BucketParams' description: Required details if using an S3 bucket for staging data. required: - bucket_vendor - bucket_s3 title: bucket_s3 properties: {} type: object unevaluatedProperties: false OpenTableFormatABSBucketParams: properties: container_name: type: string title: Bucket name folder: type: string title: Folder Name description: >- Name of folder to create and write data to. It will be created automatically if the folder does not exist. storage_account_name: type: string title: Storage account name bucket_auth_method: type: string enum: - azure_storage_account_access_key title: Auth method azure_storage_account_access_key: $ref: '#/components/schemas/AzureStorageAccountAccessKey' description: Required parameters to authenticate to Azure using an access key. additionalProperties: false type: object required: - container_name - folder - storage_account_name - bucket_auth_method OpenTableFormatGCSBucketParams: oneOf: - properties: bucket_auth_method: const: gcp_service_account_role gcp_service_account_role: $ref: '#/components/schemas/GCPServiceAccountRole' title: Authenticate using a GCP service account (recommended) description: >- Required details if authenticating using Workload Identity Federation. required: - bucket_auth_method - gcp_service_account_role title: Authenticate using a GCP service account (recommended) - properties: bucket_auth_method: const: gcp_service_account_key gcp_service_account_key: $ref: >- #/components/schemas/OpenTableFormatGCSDestinationServiceAccountKey title: Authenticate using direct access description: >- Required details if authenticating using a GCP service account key. required: - bucket_auth_method - gcp_service_account_key title: Authenticate using direct access - properties: bucket_auth_method: const: federated_gcp_service_account_role federated_gcp_service_account_role: $ref: '#/components/schemas/FederatedGCPServiceAccountRole' title: Authenticate using federation (only if self-hosting) description: >- Required details if authenticating using Workload Identity Federation. required: - bucket_auth_method - federated_gcp_service_account_role title: Authenticate using federation (only if self-hosting) properties: bucket_name: type: string title: GCS bucket name folder: type: string title: Folder type: object required: - bucket_name - folder unevaluatedProperties: false OpenTableFormatS3BucketParams: oneOf: - properties: bucket_auth_method: const: aws_iam_role aws_iam_role: $ref: '#/components/schemas/AWSIAMRole' title: Authenticate using an AWS IAM role (recommended) required: - bucket_auth_method - aws_iam_role title: Authenticate using an AWS IAM role (recommended) - properties: bucket_auth_method: const: aws_access_keys aws_access_keys: $ref: '#/components/schemas/AWSAccessKeys' title: Authenticate using AWS access keys description: Required details if authenticating using AWS access keys. required: - bucket_auth_method - aws_access_keys title: Authenticate using AWS access keys properties: bucket_name: type: string title: S3 bucket name bucket_region: type: string title: S3 bucket region description: >- The region of the S3 bucket. Must be a valid [AWS region](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.RegionsAndAvailabilityZones.html) folder: type: string title: Folder name description: >- Name of folder to create and write data to. It will be created automatically if it does not exist. type: object required: - bucket_name - bucket_region - folder unevaluatedProperties: false OpenTableFormatS3CompatibleBucketParams: properties: bucket_host: type: string title: Bucket host default: s3.us-east-1.amazonaws.com bucket_port: type: integer exclusiveMinimum: 0 title: Bucket port default: 443 bucket_name: type: string title: Bucket name folder: type: string title: Folder name description: >- Name of folder to create and write data to. It will be created automatically if it does not exist. bucket_auth_method: type: string enum: - access_keys title: Auth method description: >- Placeholder value in case authentication options are added. Currently must be set to access_keys. access_keys: $ref: '#/components/schemas/AccessKeys' description: Required details for access key authentication. disable_ssl: type: boolean title: Disable SSL additionalProperties: false type: object required: - bucket_host - bucket_port - bucket_name - folder - bucket_auth_method - access_keys GCSDestinationOptionsGCPServiceAccountRole: properties: folder: type: string title: Folder description: >- Name of folder to create and write data to. It will be created automatically if it does not exist. file_format: type: string enum: - parquet - csv - json - jsonl title: File format description: >- File format to write to the bucket. Parquet is the default and recommended format. default: parquet csv: $ref: '#/components/schemas/FileFormatCSV' description: Required details if writing CSV file format. bucket_name: type: string title: GCS bucket name disable_manifest: type: boolean title: Disable Manifest Files description: >- Disables JSON manifest files (containing validation metadata) written with each batch of data. default: false auth_method: const: gcp_service_account_role gcp_service_account_role: $ref: '#/components/schemas/GCPServiceAccountRole' title: Authenticate using a GCP service account (recommended) description: >- Required details if authenticating using Workload Identity Federation. type: object required: - folder - bucket_name - auth_method - gcp_service_account_role title: Authenticate using a GCP service account (recommended) unevaluatedProperties: false GCSDestinationOptionsGCSHMACKeys: properties: folder: type: string title: Folder description: >- Name of folder to create and write data to. It will be created automatically if it does not exist. file_format: type: string enum: - parquet - csv - json - jsonl title: File format description: >- File format to write to the bucket. Parquet is the default and recommended format. default: parquet csv: $ref: '#/components/schemas/FileFormatCSV' description: Required details if writing CSV file format. bucket_name: type: string title: GCS bucket name disable_manifest: type: boolean title: Disable Manifest Files description: >- Disables JSON manifest files (containing validation metadata) written with each batch of data. default: false auth_method: const: gcs_hmac_keys gcs_hmac_keys: $ref: '#/components/schemas/GCSAccessKeys' title: Authenticate using GCS HMAC keys description: Required details if authenticating using GCS HMAC keys. type: object required: - folder - bucket_name - auth_method - gcs_hmac_keys title: Authenticate using GCS HMAC keys unevaluatedProperties: false GCSDestinationOptionsFederatedGCPServiceAccountRole: properties: folder: type: string title: Folder description: >- Name of folder to create and write data to. It will be created automatically if it does not exist. file_format: type: string enum: - parquet - csv - json - jsonl title: File format description: >- File format to write to the bucket. Parquet is the default and recommended format. default: parquet csv: $ref: '#/components/schemas/FileFormatCSV' description: Required details if writing CSV file format. bucket_name: type: string title: GCS bucket name disable_manifest: type: boolean title: Disable Manifest Files description: >- Disables JSON manifest files (containing validation metadata) written with each batch of data. default: false auth_method: const: federated_gcp_service_account_role federated_gcp_service_account_role: $ref: '#/components/schemas/FederatedGCPServiceAccountRole' title: Authenticate using federation (only if self-hosting) description: >- Required details if authenticating using Workload Identity Federation. type: object required: - folder - bucket_name - auth_method - federated_gcp_service_account_role title: Authenticate using federation (only if self-hosting) unevaluatedProperties: false IcebergRestCatalogParams: oneOf: - $ref: '#/components/schemas/IcebergRestCatalogParamsOAuthBearerTokenAuth' - $ref: '#/components/schemas/IcebergRestCatalogParamsOAuthClientSecretAuth' discriminator: mapping: oauth_bearer_token_auth: $ref: '#/components/schemas/IcebergRestCatalogParamsOAuthBearerTokenAuth' oauth_client_secret_auth: $ref: '#/components/schemas/IcebergRestCatalogParamsOAuthClientSecretAuth' propertyName: auth_method IcebergAwsGlueCatalogParams: properties: storage_options: $ref: '#/components/schemas/IcebergAwsStorageOptionsParams' database: type: string pattern: ^[A-Za-z0-9_.]+([-][A-Za-z0-9_.]+)*$ title: Database description: >- Name of database in which to group written iceberg tables. It will be created automatically if the database does not exist. additionalProperties: false type: object required: - storage_options - database IcebergAwsS3TablesCatalogParams: properties: storage_options: $ref: '#/components/schemas/IcebergAwsStorageOptionsParams' namespace: type: string pattern: ^[A-Za-z0-9_.]+([-][A-Za-z0-9_.]+)*$ title: Namespace description: >- Name of collection in which to group written iceberg tables. It will be created automatically if the namespace does not exist. table_bucket_arn: type: string title: Table Bucket ARN description: ARN of S3 Tables bucket where Iceberg data will be written. additionalProperties: false type: object required: - storage_options IcebergGoogleLakehouseCatalogParams: oneOf: - $ref: >- #/components/schemas/IcebergGoogleLakehouseCatalogParamsGCPServiceAccountRole - $ref: >- #/components/schemas/IcebergGoogleLakehouseCatalogParamsFederatedGCPServiceAccountRole - $ref: >- #/components/schemas/IcebergGoogleLakehouseCatalogParamsGCPServiceAccountKey discriminator: mapping: federated_gcp_service_account_role: $ref: >- #/components/schemas/IcebergGoogleLakehouseCatalogParamsFederatedGCPServiceAccountRole gcp_service_account_key: $ref: >- #/components/schemas/IcebergGoogleLakehouseCatalogParamsGCPServiceAccountKey gcp_service_account_role: $ref: >- #/components/schemas/IcebergGoogleLakehouseCatalogParamsGCPServiceAccountRole propertyName: auth_method MongoDBDestinationOptionsAWSIAMRole: properties: host: type: string title: Host description: >- Hostname of the MongoDB server. For Atlas clusters: use the cluster endpoint. default: clusterID.mongodb.net port: type: integer title: Port description: Optional. Leave empty If using Atlas. default: 27017 authentication_database: type: string title: Database description: The MongoDB database to connect to. write_database: type: string pattern: ^[A-Za-z0-9_.]+([-][A-Za-z0-9_.]+)*$ title: Schema description: >- Name of schema to create and write data to. It will be created automatically if the schema does not exist. use_ssh_tunnel: type: boolean description: Whether to use an SSH tunnel for the connection. ssh_tunnel: $ref: '#/components/schemas/SSHTunnelParams' description: Required details if using SSH tunneling. disable_ssl: type: boolean title: Disable SSL description: >- Whether to disable SSL for the MongoDB connection. When false: SSL is enabled. default: false auth_method: const: aws_iam_role aws_iam_role: $ref: '#/components/schemas/AWSIAMRole' title: Authenticate using an AWS IAM role (recommended) type: object required: - host - authentication_database - write_database - auth_method - aws_iam_role title: Authenticate using an AWS IAM role (recommended) unevaluatedProperties: false MongoDBDestinationOptionsPasswordAuth: properties: host: type: string title: Host description: >- Hostname of the MongoDB server. For Atlas clusters: use the cluster endpoint. default: clusterID.mongodb.net port: type: integer title: Port description: Optional. Leave empty If using Atlas. default: 27017 authentication_database: type: string title: Database description: The MongoDB database to connect to. write_database: type: string pattern: ^[A-Za-z0-9_.]+([-][A-Za-z0-9_.]+)*$ title: Schema description: >- Name of schema to create and write data to. It will be created automatically if the schema does not exist. use_ssh_tunnel: type: boolean description: Whether to use an SSH tunnel for the connection. ssh_tunnel: $ref: '#/components/schemas/SSHTunnelParams' description: Required details if using SSH tunneling. disable_ssl: type: boolean title: Disable SSL description: >- Whether to disable SSL for the MongoDB connection. When false: SSL is enabled. default: false auth_method: const: password_auth password_auth: $ref: '#/components/schemas/PasswordAuth' title: Authenticate using a username and password description: Required details if authenticating with username/password. type: object required: - host - authentication_database - write_database - auth_method - password_auth title: Authenticate using a username and password unevaluatedProperties: false OracleSSLParams: properties: wallet: type: string title: Oracle Wallet description: Base64 encoded contents of Oracle Wallet. additionalProperties: false type: object RedshiftDestinationOptionsAWSAccessKeys: properties: host: type: string title: Host description: Host address of the destination. default: cluster-name.ab1cdefgh.us-east-1.redshift.amazonaws.com port: type: integer exclusiveMinimum: 0 title: Port default: 5439 database: type: string title: Database description: Name of database provisioned by the owner of the destination. schema: type: string pattern: ^[A-Za-z0-9_.]+([-][A-Za-z0-9_.]+)*$ title: Schema description: >- Name of schema to create and write data to. It will be created automatically if the schema does not exist. cluster: type: string title: Cluster description: Redshift Cluster identifier username: type: string title: Username description: >- Username for the Redshift connection. Required when using aws_access_keys auth. bucket_s3: $ref: '#/components/schemas/RedshiftS3BucketParams' description: Required details about the S3 bucket used for staging data. use_ssh_tunnel: type: boolean description: Whether to use an SSH tunnel for the connection. ssh_tunnel: $ref: '#/components/schemas/SSHTunnelParams' description: Required details if using SSH tunneling. auth_method: const: aws_access_keys aws_access_keys: $ref: '#/components/schemas/RedshiftAccessKeys' title: Authenticate using AWS access keys description: Required details for username/password authentication. type: object required: - host - port - database - schema - bucket_s3 - auth_method - aws_access_keys title: Authenticate using AWS access keys unevaluatedProperties: false RedshiftDestinationOptionsAWSIAMRole: properties: host: type: string title: Host description: Host address of the destination. default: cluster-name.ab1cdefgh.us-east-1.redshift.amazonaws.com port: type: integer exclusiveMinimum: 0 title: Port default: 5439 database: type: string title: Database description: Name of database provisioned by the owner of the destination. schema: type: string pattern: ^[A-Za-z0-9_.]+([-][A-Za-z0-9_.]+)*$ title: Schema description: >- Name of schema to create and write data to. It will be created automatically if the schema does not exist. cluster: type: string title: Cluster description: Redshift Cluster identifier username: type: string title: Username description: >- Username for the Redshift connection. Required when using aws_access_keys auth. bucket_s3: $ref: '#/components/schemas/RedshiftS3BucketParams' description: Required details about the S3 bucket used for staging data. use_ssh_tunnel: type: boolean description: Whether to use an SSH tunnel for the connection. ssh_tunnel: $ref: '#/components/schemas/SSHTunnelParams' description: Required details if using SSH tunneling. auth_method: const: aws_iam_role aws_iam_role: $ref: '#/components/schemas/AWSIAMRole' title: Authenticate using an AWS IAM role (recommended) description: Required details if authenticating using an AWS IAM role. type: object required: - host - port - database - schema - bucket_s3 - auth_method - aws_iam_role title: Authenticate using an AWS IAM role (recommended) unevaluatedProperties: false RedshiftServerlessDestinationOptionsAWSIAMRole: properties: workgroup: type: string title: Workgroup description: Name of the Redshift Serverless workgroup. host: type: string title: Host description: Workgroup endpoint default: workgroup.123456789.us-east-1.redshift-serverless.amazonaws.com port: type: integer exclusiveMinimum: 0 title: Port default: 5439 database: type: string title: Database description: Name of target database. schema: type: string pattern: ^[A-Za-z0-9_.]+([-][A-Za-z0-9_.]+)*$ title: Schema description: >- Name of schema to create and write data to. It will be created automatically if the schema does not exist. bucket_s3: $ref: '#/components/schemas/RedshiftS3BucketParams' description: Required details about the S3 bucket used for staging data. use_ssh_tunnel: type: boolean description: Whether to use an SSH tunnel for the connection. ssh_tunnel: $ref: '#/components/schemas/SSHTunnelParams' description: Required details if using SSH tunneling. auth_method: const: aws_iam_role aws_iam_role: $ref: '#/components/schemas/AWSIAMRole' title: Authenticate using an AWS IAM role (recommended) description: Required details if authenticating using an AWS IAM role. type: object required: - workgroup - host - port - database - schema - bucket_s3 - auth_method - aws_iam_role title: Authenticate using an AWS IAM role (recommended) unevaluatedProperties: false RedshiftServerlessDestinationOptionsAWSAccessKeys: properties: workgroup: type: string title: Workgroup description: Name of the Redshift Serverless workgroup. host: type: string title: Host description: Workgroup endpoint default: workgroup.123456789.us-east-1.redshift-serverless.amazonaws.com port: type: integer exclusiveMinimum: 0 title: Port default: 5439 database: type: string title: Database description: Name of target database. schema: type: string pattern: ^[A-Za-z0-9_.]+([-][A-Za-z0-9_.]+)*$ title: Schema description: >- Name of schema to create and write data to. It will be created automatically if the schema does not exist. bucket_s3: $ref: '#/components/schemas/RedshiftS3BucketParams' description: Required details about the S3 bucket used for staging data. use_ssh_tunnel: type: boolean description: Whether to use an SSH tunnel for the connection. ssh_tunnel: $ref: '#/components/schemas/SSHTunnelParams' description: Required details if using SSH tunneling. auth_method: const: aws_access_keys aws_access_keys: $ref: '#/components/schemas/AWSAccessKeys' title: Authenticate using AWS access keys description: Required details if authenticating using AWS access keys. type: object required: - workgroup - host - port - database - schema - bucket_s3 - auth_method - aws_access_keys title: Authenticate using AWS access keys unevaluatedProperties: false PGPEncryption: properties: pgp_public_key: type: string title: PGP public key description: PGP public key in ASCII armored format. additionalProperties: false type: object required: - pgp_public_key PublicKeyAuth: properties: public_key: type: string enum: - primary_public_key - secondary_public_key title: Public key description: >- This is the public SSH key that will be used to establish the SSH tunnel. It should be added to the list of authorized keys on the bastion host. additionalProperties: false type: object required: - public_key SnowflakeDestinationOptionsPasswordAuth: properties: host: type: string pattern: .*\.snowflakecomputing.com$ title: Account host description: Account address of the destination. default: db54321.us-central1.gcp.snowflakecomputing.com port: type: integer exclusiveMinimum: 0 title: Port description: Port of the Snowflake server. default: 443 database: type: string title: Database description: Name of database provisioned by the owner of the destination. schema: type: string pattern: ^[A-Za-z0-9_.]+([-][A-Za-z0-9_.]+)*$ title: Schema description: >- Name of schema to create and write data to. It will be created automatically if the schema does not exist. username: type: string title: Username description: The name of the Snowflake user. auth_method: const: password_auth password_auth: $ref: '#/components/schemas/PasswordOnlyAuth' title: Authenticate using a username and password description: Required details for username/password authentication. type: object required: - host - port - database - schema - username - auth_method - password_auth title: Authenticate using a username and password unevaluatedProperties: false SnowflakeDestinationOptionsPublicKeyAuth: properties: host: type: string pattern: .*\.snowflakecomputing.com$ title: Account host description: Account address of the destination. default: db54321.us-central1.gcp.snowflakecomputing.com port: type: integer exclusiveMinimum: 0 title: Port description: Port of the Snowflake server. default: 443 database: type: string title: Database description: Name of database provisioned by the owner of the destination. schema: type: string pattern: ^[A-Za-z0-9_.]+([-][A-Za-z0-9_.]+)*$ title: Schema description: >- Name of schema to create and write data to. It will be created automatically if the schema does not exist. username: type: string title: Username description: The name of the Snowflake user. auth_method: const: public_key_auth public_key_auth: $ref: '#/components/schemas/PublicKeyAuth' description: Required details for public key authentication. type: object required: - host - port - database - schema - username - auth_method - public_key_auth unevaluatedProperties: false S3DestinationOptionsAWSIAMRole: properties: folder: type: string title: Folder name description: >- Name of folder to create and write data to. It will be created automatically if it does not exist. file_format: type: string enum: - parquet - csv - json - jsonl title: File format description: >- File format to write to the bucket. Parquet is the default and recommended format. default: parquet csv: $ref: '#/components/schemas/FileFormatCSV' description: Required details if writing CSV file format. bucket_name: type: string title: S3 bucket name bucket_region: type: string title: S3 bucket region description: >- The region of the S3 bucket. Must be a valid [AWS region](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.RegionsAndAvailabilityZones.html) use_ssh_tunnel: type: boolean description: Whether to use an SSH tunnel for the connection. ssh_tunnel: $ref: '#/components/schemas/SSHTunnelParams' description: Required details if using SSH tunneling. disable_manifest: type: boolean title: Disable Manifest Files description: >- Disables JSON manifest files (containing validation metadata) written with each batch of data. default: false auth_method: const: aws_iam_role aws_iam_role: $ref: '#/components/schemas/AWSIAMRole' title: Authenticate using an AWS IAM role (recommended) type: object required: - folder - bucket_name - bucket_region - auth_method - aws_iam_role title: Authenticate using an AWS IAM role (recommended) unevaluatedProperties: false S3DestinationOptionsAWSAccessKeys: properties: folder: type: string title: Folder name description: >- Name of folder to create and write data to. It will be created automatically if it does not exist. file_format: type: string enum: - parquet - csv - json - jsonl title: File format description: >- File format to write to the bucket. Parquet is the default and recommended format. default: parquet csv: $ref: '#/components/schemas/FileFormatCSV' description: Required details if writing CSV file format. bucket_name: type: string title: S3 bucket name bucket_region: type: string title: S3 bucket region description: >- The region of the S3 bucket. Must be a valid [AWS region](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.RegionsAndAvailabilityZones.html) use_ssh_tunnel: type: boolean description: Whether to use an SSH tunnel for the connection. ssh_tunnel: $ref: '#/components/schemas/SSHTunnelParams' description: Required details if using SSH tunneling. disable_manifest: type: boolean title: Disable Manifest Files description: >- Disables JSON manifest files (containing validation metadata) written with each batch of data. default: false auth_method: const: aws_access_keys aws_access_keys: $ref: '#/components/schemas/AWSAccessKeys' title: Authenticate using AWS access keys description: Required details if authenticating using AWS access keys. type: object required: - folder - bucket_name - bucket_region - auth_method - aws_access_keys title: Authenticate using AWS access keys unevaluatedProperties: false AccessKeys: properties: access_id: type: string title: Bucket access ID secret_key: type: string title: Bucket secret key additionalProperties: false type: object required: - access_id - secret_key AthenaDestinationResourceAWSIAMRole: properties: workgroup: type: string connection_database: type: string write_database: type: string bucket_s3: $ref: '#/components/schemas/S3BucketResource' auth_method: const: aws_iam_role aws_iam_role: $ref: '#/components/schemas/AWSIAMRoleResource' title: Authenticate using an AWS IAM role (recommended) type: object required: - workgroup - connection_database - write_database - bucket_s3 - auth_method - aws_iam_role title: Authenticate using an AWS IAM role (recommended) unevaluatedProperties: false AthenaDestinationResourceAWSAccessKeys: properties: workgroup: type: string connection_database: type: string write_database: type: string bucket_s3: $ref: '#/components/schemas/S3BucketResource' auth_method: const: aws_access_keys aws_access_keys: $ref: '#/components/schemas/AWSAccessKeysResource' title: Authenticate using AWS access keys type: object required: - workgroup - connection_database - write_database - bucket_s3 - auth_method - aws_access_keys title: Authenticate using AWS access keys unevaluatedProperties: false PasswordAuthResource: properties: username: type: string additionalProperties: false type: object required: - username SSHTunnelResource: properties: ssh_tunnel_host: type: string ssh_tunnel_port: type: integer ssh_tunnel_username: type: string ssh_public_key: type: string additionalProperties: false type: object BigQueryDestinationResourceGCPServiceAccountRole: properties: project_id: type: string schema: type: string bucket_gcs: $ref: '#/components/schemas/BigQueryGCSBucketResource' auth_method: const: gcp_service_account_role gcp_service_account_role: $ref: '#/components/schemas/GCPServiceAccountRoleResource' title: Authenticate using a GCP service account (recommended) type: object required: - project_id - schema - bucket_gcs - auth_method - gcp_service_account_role title: Authenticate using a GCP service account (recommended) unevaluatedProperties: false BigQueryDestinationResourceFederatedGCPServiceAccountRole: properties: project_id: type: string schema: type: string bucket_gcs: $ref: '#/components/schemas/BigQueryGCSBucketResource' auth_method: const: federated_gcp_service_account_role federated_gcp_service_account_role: $ref: '#/components/schemas/FederatedGCPServiceAccountRoleResource' title: Authenticate using federation (only if self-hosting) type: object required: - project_id - schema - bucket_gcs - auth_method - federated_gcp_service_account_role title: Authenticate using federation (only if self-hosting) unevaluatedProperties: false BigQueryDestinationResourceGCPServiceAccountKey: properties: project_id: type: string schema: type: string bucket_gcs: $ref: '#/components/schemas/BigQueryGCSBucketResource' auth_method: const: gcp_service_account_key gcp_service_account_key: $ref: '#/components/schemas/GCPServiceAccountKeyResource' title: Authenticate using direct access type: object required: - project_id - schema - bucket_gcs - auth_method - gcp_service_account_key title: Authenticate using direct access unevaluatedProperties: false ClickhouseS3BucketResource: properties: bucket_name: type: string bucket_region: type: string bucket_auth_method: type: string aws_iam_role: $ref: '#/components/schemas/AWSIAMRoleResource' aws_access_keys: $ref: '#/components/schemas/AWSAccessKeysResource' additionalProperties: false type: object required: - bucket_name - bucket_region - bucket_auth_method ClickhouseGCSBucketResource: properties: bucket_name: type: string gcs_hmac_keys: $ref: '#/components/schemas/GCSAccessKeysResource' additionalProperties: false type: object required: - bucket_name - gcs_hmac_keys DatabricksHiveResource: oneOf: - properties: bucket_s3: $ref: '#/components/schemas/DatabricksS3BucketResource' required: - bucket_s3 title: bucket_s3 properties: bucket_vendor: type: string type: object required: - bucket_vendor unevaluatedProperties: false OauthClientSecretAuthResource: properties: client_id: type: string additionalProperties: false type: object required: - client_id OpenTableFormatABSBucketResource: properties: container_name: type: string folder: type: string storage_account_name: type: string bucket_auth_method: type: string additionalProperties: false type: object required: - container_name - folder - storage_account_name - bucket_auth_method OpenTableFormatGCSBucketResource: oneOf: - properties: gcp_service_account_role: $ref: '#/components/schemas/GCPServiceAccountRoleResource' required: - gcp_service_account_role title: gcp_service_account_role - properties: federated_gcp_service_account_role: $ref: '#/components/schemas/FederatedGCPServiceAccountRoleResource' required: - federated_gcp_service_account_role title: federated_gcp_service_account_role - properties: gcp_service_account_key: $ref: >- #/components/schemas/OpenTableFormatGCSDestinationServiceAccountKeyResource required: - gcp_service_account_key title: gcp_service_account_key properties: bucket_name: type: string title: GCS bucket name folder: type: string bucket_auth_method: type: string enum: - gcp_service_account_role - gcp_service_account_key type: object required: - bucket_name - folder - bucket_auth_method unevaluatedProperties: false OpenTableFormatS3BucketResource: oneOf: - properties: bucket_auth_method: const: aws_iam_role aws_iam_role: $ref: '#/components/schemas/AWSIAMRoleResource' title: Authenticate using an AWS IAM role (recommended) required: - bucket_auth_method - aws_iam_role title: Authenticate using an AWS IAM role (recommended) - properties: bucket_auth_method: const: aws_access_keys aws_access_keys: $ref: '#/components/schemas/AWSAccessKeysResource' title: Authenticate using AWS access keys required: - bucket_auth_method - aws_access_keys title: Authenticate using AWS access keys properties: bucket_name: type: string bucket_region: type: string folder: type: string type: object required: - bucket_name - bucket_region - folder unevaluatedProperties: false OpenTableFormatS3CompatibleBucketResource: properties: bucket_host: type: string bucket_port: type: integer exclusiveMinimum: 0 folder: type: string bucket_name: type: string bucket_auth_method: type: string enum: - access_keys access_keys: $ref: '#/components/schemas/AWSAccessKeysResource' disable_ssl: type: boolean title: Disable SSL additionalProperties: false type: object required: - bucket_host - bucket_port - folder - bucket_name - bucket_auth_method GCSDestinationResourceFederatedGCPServiceAccountRole: properties: disable_manifest: type: boolean title: Disable Manifest Files description: >- Disables JSON manifest files (containing validation metadata) written with each batch of data. folder: type: string file_format: type: string enum: - parquet - csv - json - jsonl csv: $ref: '#/components/schemas/FileFormatCSV' bucket_name: type: string auth_method: const: federated_gcp_service_account_role federated_gcp_service_account_role: $ref: '#/components/schemas/FederatedGCPServiceAccountRoleResource' title: Authenticate using federation (only if self-hosting) type: object required: - folder - bucket_name - auth_method - federated_gcp_service_account_role title: Authenticate using federation (only if self-hosting) unevaluatedProperties: false GCSDestinationResourceGCPServiceAccountRole: properties: disable_manifest: type: boolean title: Disable Manifest Files description: >- Disables JSON manifest files (containing validation metadata) written with each batch of data. folder: type: string file_format: type: string enum: - parquet - csv - json - jsonl csv: $ref: '#/components/schemas/FileFormatCSV' bucket_name: type: string auth_method: const: gcp_service_account_role gcp_service_account_role: $ref: '#/components/schemas/GCPServiceAccountRoleResource' title: Authenticate using a GCP service account (recommended) type: object required: - folder - bucket_name - auth_method - gcp_service_account_role title: Authenticate using a GCP service account (recommended) unevaluatedProperties: false GCSDestinationResourceGCSHMACKeys: properties: disable_manifest: type: boolean title: Disable Manifest Files description: >- Disables JSON manifest files (containing validation metadata) written with each batch of data. folder: type: string file_format: type: string enum: - parquet - csv - json - jsonl csv: $ref: '#/components/schemas/FileFormatCSV' bucket_name: type: string auth_method: const: gcs_hmac_keys gcs_hmac_keys: $ref: '#/components/schemas/GCSAccessKeysResource' title: Authenticate using GCS HMAC keys type: object required: - folder - bucket_name - auth_method - gcs_hmac_keys title: Authenticate using GCS HMAC keys unevaluatedProperties: false IcebergRestCatalogResource: properties: host: type: string port: type: integer namespace: type: string warehouse: type: string auth_method: type: string oauth_client_secret_auth: $ref: '#/components/schemas/OauthDelegatedCredentialAuthResource' storage_options: $ref: '#/components/schemas/IcebergStorageOptionsResource' additionalProperties: false type: object required: - host - namespace - warehouse - auth_method - storage_options IcebergAwsS3TablesCatalogResource: properties: storage_options: $ref: '#/components/schemas/IcebergAwsStorageOptionsResource' namespace: type: string table_bucket_arn: type: string additionalProperties: false type: object required: - storage_options - namespace - table_bucket_arn IcebergAwsGlueCatalogResource: properties: storage_options: $ref: '#/components/schemas/IcebergAwsStorageOptionsResource' database: type: string additionalProperties: false type: object required: - storage_options - database IcebergGoogleLakehouseCatalogResource: properties: project_id: type: string catalog_name: type: string schema: type: string bucket_name: type: string auth_method: type: string gcp_service_account_role: $ref: '#/components/schemas/GCPServiceAccountRoleResource' federated_gcp_service_account_role: $ref: '#/components/schemas/FederatedGCPServiceAccountRoleResource' gcp_service_account_key: $ref: >- #/components/schemas/OpenTableFormatGCSDestinationServiceAccountKeyResource additionalProperties: false type: object required: - project_id - catalog_name - schema - bucket_name - auth_method MongoDBDestinationResourceAWSIAMRole: properties: host: type: string port: type: integer authentication_database: type: string write_database: type: string use_ssh_tunnel: type: boolean ssh_tunnel: $ref: '#/components/schemas/SSHTunnelResource' disable_ssl: type: boolean auth_method: const: aws_iam_role aws_iam_role: $ref: '#/components/schemas/AWSIAMRoleResource' title: Authenticate using an AWS IAM role (recommended) type: object required: - host - port - authentication_database - write_database - use_ssh_tunnel - auth_method - aws_iam_role title: Authenticate using an AWS IAM role (recommended) unevaluatedProperties: false MongoDBDestinationResourcePasswordAuth: properties: host: type: string port: type: integer authentication_database: type: string write_database: type: string use_ssh_tunnel: type: boolean ssh_tunnel: $ref: '#/components/schemas/SSHTunnelResource' disable_ssl: type: boolean auth_method: const: password_auth password_auth: $ref: '#/components/schemas/PasswordAuthResource' title: Authenticate using a username and password type: object required: - host - port - authentication_database - write_database - use_ssh_tunnel - auth_method - password_auth title: Authenticate using a username and password unevaluatedProperties: false RedshiftDestinationResourceAWSAccessKeys: properties: host: type: string port: type: integer database: type: string schema: type: string cluster: type: string username: type: string bucket_s3: $ref: '#/components/schemas/RedshiftS3BucketResource' use_ssh_tunnel: type: boolean ssh_tunnel: $ref: '#/components/schemas/SSHTunnelResource' auth_method: const: aws_access_keys aws_access_keys: $ref: '#/components/schemas/RedshiftAccessKeysResource' title: Authenticate using AWS access keys type: object required: - host - port - database - schema - cluster - username - bucket_s3 - use_ssh_tunnel - auth_method - aws_access_keys title: Authenticate using AWS access keys unevaluatedProperties: false RedshiftDestinationResourceAWSIAMRole: properties: host: type: string port: type: integer database: type: string schema: type: string cluster: type: string username: type: string bucket_s3: $ref: '#/components/schemas/RedshiftS3BucketResource' use_ssh_tunnel: type: boolean ssh_tunnel: $ref: '#/components/schemas/SSHTunnelResource' auth_method: const: aws_iam_role aws_iam_role: $ref: '#/components/schemas/AWSIAMRoleResource' title: Authenticate using an AWS IAM role (recommended) type: object required: - host - port - database - schema - cluster - username - bucket_s3 - use_ssh_tunnel - auth_method - aws_iam_role title: Authenticate using an AWS IAM role (recommended) unevaluatedProperties: false RedshiftServerlessDestinationResourceAWSAccessKeys: properties: workgroup: type: string host: type: string port: type: integer database: type: string schema: type: string bucket_s3: $ref: '#/components/schemas/RedshiftS3BucketResource' auth_method: const: aws_access_keys aws_access_keys: $ref: '#/components/schemas/AWSAccessKeysResource' title: Authenticate using AWS access keys type: object required: - workgroup - host - port - database - schema - bucket_s3 - auth_method - aws_access_keys title: Authenticate using AWS access keys unevaluatedProperties: false RedshiftServerlessDestinationResourceAWSIAMRole: properties: workgroup: type: string host: type: string port: type: integer database: type: string schema: type: string bucket_s3: $ref: '#/components/schemas/RedshiftS3BucketResource' auth_method: const: aws_iam_role aws_iam_role: $ref: '#/components/schemas/AWSIAMRoleResource' title: Authenticate using an AWS IAM role (recommended) type: object required: - workgroup - host - port - database - schema - bucket_s3 - auth_method - aws_iam_role title: Authenticate using an AWS IAM role (recommended) unevaluatedProperties: false S3DestinationResourceAWSIAMRole: properties: disable_manifest: type: boolean title: Disable Manifest Files description: >- Disables JSON manifest files (containing validation metadata) written with each batch of data. folder: type: string file_format: type: string enum: - parquet - csv - json - jsonl csv: $ref: '#/components/schemas/FileFormatCSV' bucket_name: type: string bucket_region: type: string use_ssh_tunnel: type: boolean ssh_tunnel: $ref: '#/components/schemas/SSHTunnelResource' auth_method: const: aws_iam_role aws_iam_role: $ref: '#/components/schemas/AWSIAMRoleResource' title: Authenticate using an AWS IAM role (recommended) type: object required: - folder - bucket_name - bucket_region - use_ssh_tunnel - auth_method - aws_iam_role title: Authenticate using an AWS IAM role (recommended) unevaluatedProperties: false S3DestinationResourceAWSAccessKeys: properties: disable_manifest: type: boolean title: Disable Manifest Files description: >- Disables JSON manifest files (containing validation metadata) written with each batch of data. folder: type: string file_format: type: string enum: - parquet - csv - json - jsonl csv: $ref: '#/components/schemas/FileFormatCSV' bucket_name: type: string bucket_region: type: string use_ssh_tunnel: type: boolean ssh_tunnel: $ref: '#/components/schemas/SSHTunnelResource' auth_method: const: aws_access_keys aws_access_keys: $ref: '#/components/schemas/AWSAccessKeysResource' title: Authenticate using AWS access keys type: object required: - folder - bucket_name - bucket_region - use_ssh_tunnel - auth_method - aws_access_keys title: Authenticate using AWS access keys unevaluatedProperties: false AWSAccessKeysResource: properties: access_id: type: string additionalProperties: false type: object required: - access_id StringArray: items: type: string type: array TransferModelMetrics: items: $ref: '#/components/schemas/MetricsForTransferredModel' type: array TransferModelsSnapshot: items: $ref: '#/components/schemas/TransferModelSnapshot' type: array TransferTags: additionalProperties: type: string type: object S3BucketParams: properties: bucket_name: type: string title: S3 bucket name bucket_region: type: string title: S3 bucket region description: >- The region of the S3 bucket. Must be a valid [AWS region](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.RegionsAndAvailabilityZones.html) additionalProperties: false type: object required: - bucket_name - bucket_region AWSIAMRole: properties: aws_iam_role_arn: type: string title: IAM role ARN default: arn:aws:iam::123456789012:role/service-role/role-name additionalProperties: false type: object required: - aws_iam_role_arn AWSAccessKeys: properties: access_id: type: string title: Bucket access ID secret_key: type: string title: Bucket secret key additionalProperties: false type: object required: - access_id - secret_key BigQueryGCSBucketParams: properties: bucket_name: type: string title: GCS bucket name bucket_region: type: string title: GCS bucket region description: >- The region of the staging GCS bucket. Must be a valid GCS [bucket location](https://cloud.google.com/storage/docs/locations) additionalProperties: false type: object required: - bucket_name - bucket_region GCPServiceAccountRole: properties: service_account_email: type: string pattern: ^.*gserviceaccount.com$ format: email title: Service account email default: service-account-name@project-id.iam.gserviceaccount.com additionalProperties: false type: object required: - service_account_email FederatedGCPServiceAccountRole: properties: service_account_email: type: string pattern: ^.*gserviceaccount.com$ format: email title: Service account email default: service-account-name@project-id.iam.gserviceaccount.com workload_identity_federation_metadata: $ref: '#/components/schemas/GcpExternalAccountMetadata' title: Workload identity metadata additionalProperties: false type: object required: - service_account_email GCPServiceAccountKey: properties: service_account_email: type: string pattern: ^.*gserviceaccount.com$ format: email title: Service account email default: service-account-name@project-id.iam.gserviceaccount.com service_account_key: title: JSON Token description: >- The credentials (JSON) that will be used. Create a service account just for this use. additionalProperties: false type: object required: - service_account_email - service_account_key GCSAccessKeys: properties: access_key: type: string title: Bucket access ID description: The access key to use for authentication. secret: type: string title: Bucket secret key description: The secret to use for authentication. additionalProperties: false type: object required: - access_key - secret DatabricksS3BucketParams: oneOf: - properties: aws_access_keys: $ref: '#/components/schemas/AWSAccessKeys' description: Required details if authenticating using AWS access keys. required: - aws_access_keys title: aws_access_keys properties: bucket_name: type: string title: S3 bucket name bucket_region: type: string title: S3 bucket region description: >- The region of the S3 bucket. Must be a valid [AWS region](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.RegionsAndAvailabilityZones.html) type: object required: - bucket_name - bucket_region unevaluatedProperties: false AzureStorageAccountAccessKey: properties: storage_account_access_key: type: string title: Access key additionalProperties: false type: object required: - storage_account_access_key OpenTableFormatGCSDestinationServiceAccountKey: properties: service_account_email: type: string pattern: ^.*gserviceaccount.com$ format: email title: Destination service account email default: service-account-name@project-id.iam.gserviceaccount.com service_account_key: title: JSON Token description: >- The credentials (JSON) that will be used to load data into your BigQuery database. Create a service account just for this use. additionalProperties: false type: object required: - service_account_email - service_account_key IcebergRestCatalogParamsOAuthBearerTokenAuth: properties: host: type: string title: Host default: iceberg.example.net port: type: integer title: Port description: Optional. default: 443 namespace: type: string pattern: ^[A-Za-z0-9_.]+([-][A-Za-z0-9_.]+)*$ title: Namespace description: >- Name of collection in which to group written iceberg tables. It will be created automatically if the namespace does not exist. warehouse: type: string title: Warehouse description: >- The identifier for the warehouse associated with this REST catalog. This might be a storage URI (ex s3://...) or a unique name provided by the catalog service. storage_options: $ref: '#/components/schemas/IcebergStorageOptionsParams' auth_method: const: oauth_bearer_token_auth oauth_bearer_token_auth: $ref: '#/components/schemas/OauthBearerTokenAuth' description: Required details if authenticating with an OAUTH2 bearer token. type: object required: - host - namespace - storage_options - auth_method - oauth_bearer_token_auth unevaluatedProperties: false IcebergRestCatalogParamsOAuthClientSecretAuth: properties: host: type: string title: Host default: iceberg.example.net port: type: integer title: Port description: Optional. default: 443 namespace: type: string pattern: ^[A-Za-z0-9_.]+([-][A-Za-z0-9_.]+)*$ title: Namespace description: >- Name of collection in which to group written iceberg tables. It will be created automatically if the namespace does not exist. warehouse: type: string title: Warehouse description: >- The identifier for the warehouse associated with this REST catalog. This might be a storage URI (ex s3://...) or a unique name provided by the catalog service. storage_options: $ref: '#/components/schemas/IcebergStorageOptionsParams' auth_method: const: oauth_client_secret_auth oauth_client_secret_auth: $ref: '#/components/schemas/OauthDelegatedCredentialAuth' description: >- Required details if authenticating with an OAUTH2 delegated credentials. type: object required: - host - namespace - storage_options - auth_method - oauth_client_secret_auth unevaluatedProperties: false IcebergAwsStorageOptionsParams: properties: storage: type: string enum: - bucket_s3 title: Storage bucket_s3: $ref: '#/components/schemas/OpenTableFormatS3BucketParams' additionalProperties: false type: object required: - storage IcebergGoogleLakehouseCatalogParamsGCPServiceAccountRole: properties: project_id: type: string title: Project ID catalog_name: type: string title: Catalog Name description: Name of BigLake catalog. schema: type: string pattern: ^[A-Za-z0-9_.]+([-][A-Za-z0-9_.]+)*$ title: Schema description: Name of BigLake schema to create and write data to. bucket_name: type: string title: Bucket Name description: Name of GCS bucket to use for the Iceberg table data. auth_method: const: gcp_service_account_role gcp_service_account_role: $ref: '#/components/schemas/GCPServiceAccountRole' title: Authenticate using a GCP service account (recommended) description: Required details if authenticating using a GCP service account role. type: object required: - project_id - catalog_name - auth_method - gcp_service_account_role title: Authenticate using a GCP service account (recommended) unevaluatedProperties: false IcebergGoogleLakehouseCatalogParamsFederatedGCPServiceAccountRole: properties: project_id: type: string title: Project ID catalog_name: type: string title: Catalog Name description: Name of BigLake catalog. schema: type: string pattern: ^[A-Za-z0-9_.]+([-][A-Za-z0-9_.]+)*$ title: Schema description: Name of BigLake schema to create and write data to. bucket_name: type: string title: Bucket Name description: Name of GCS bucket to use for the Iceberg table data. auth_method: const: federated_gcp_service_account_role federated_gcp_service_account_role: $ref: '#/components/schemas/FederatedGCPServiceAccountRole' title: Authenticate using federation (only if self-hosting) description: >- Required details if authenticating using Workload Identity Federation. type: object required: - project_id - catalog_name - auth_method - federated_gcp_service_account_role title: Authenticate using federation (only if self-hosting) unevaluatedProperties: false IcebergGoogleLakehouseCatalogParamsGCPServiceAccountKey: properties: project_id: type: string title: Project ID catalog_name: type: string title: Catalog Name description: Name of BigLake catalog. schema: type: string pattern: ^[A-Za-z0-9_.]+([-][A-Za-z0-9_.]+)*$ title: Schema description: Name of BigLake schema to create and write data to. bucket_name: type: string title: Bucket Name description: Name of GCS bucket to use for the Iceberg table data. auth_method: const: gcp_service_account_key gcp_service_account_key: $ref: '#/components/schemas/OpenTableFormatGCSDestinationServiceAccountKey' title: Authenticate using direct access description: Required details if authenticating using a GCP service account key. type: object required: - project_id - catalog_name - auth_method - gcp_service_account_key title: Authenticate using direct access unevaluatedProperties: false RedshiftS3BucketParams: properties: bucket_name: type: string title: S3 bucket name bucket_region: type: string title: S3 bucket region description: >- The region of the S3 bucket. Must be a valid [AWS region](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.RegionsAndAvailabilityZones.html) additionalProperties: false type: object required: - bucket_name - bucket_region RedshiftAccessKeys: properties: password: type: string title: Password access_id: type: string title: Bucket access ID secret_key: type: string title: Bucket secret key additionalProperties: false type: object required: - password - access_id - secret_key PasswordOnlyAuth: properties: password: type: string title: Password additionalProperties: false type: object required: - password S3BucketResource: properties: bucket_name: type: string bucket_region: type: string additionalProperties: false type: object required: - bucket_name - bucket_region AWSIAMRoleResource: properties: aws_iam_role_arn: type: string additionalProperties: false type: object required: - aws_iam_role_arn BigQueryGCSBucketResource: properties: bucket_name: type: string bucket_region: type: string additionalProperties: false type: object required: - bucket_name - bucket_region GCPServiceAccountRoleResource: properties: service_account_email: type: string additionalProperties: false type: object required: - service_account_email FederatedGCPServiceAccountRoleResource: properties: service_account_email: type: string additionalProperties: false type: object required: - service_account_email GCPServiceAccountKeyResource: properties: service_account_email: type: string additionalProperties: false type: object required: - service_account_email GCSAccessKeysResource: properties: access_key: type: string additionalProperties: false type: object required: - access_key DatabricksS3BucketResource: properties: bucket_name: type: string bucket_region: type: string aws_access_keys: $ref: '#/components/schemas/AWSAccessKeysResource' additionalProperties: false type: object required: - bucket_name - bucket_region - aws_access_keys OpenTableFormatGCSDestinationServiceAccountKeyResource: properties: service_account_email: type: string additionalProperties: false type: object required: - service_account_email OauthDelegatedCredentialAuthResource: properties: client_id: type: string additionalProperties: false type: object IcebergStorageOptionsResource: properties: storage: type: string bucket_abs: $ref: '#/components/schemas/OpenTableFormatABSBucketResource' bucket_gcs: $ref: '#/components/schemas/OpenTableFormatGCSBucketResource' bucket_s3: $ref: '#/components/schemas/OpenTableFormatS3BucketResource' bucket_s3_compatible: $ref: '#/components/schemas/OpenTableFormatS3CompatibleBucketResource' additionalProperties: false type: object required: - storage IcebergAwsStorageOptionsResource: properties: storage: type: string bucket_s3: $ref: '#/components/schemas/OpenTableFormatS3BucketResource' additionalProperties: false type: object required: - storage RedshiftS3BucketResource: properties: bucket_name: type: string bucket_region: type: string additionalProperties: false type: object required: - bucket_name - bucket_region RedshiftAccessKeysResource: properties: access_id: type: string additionalProperties: false type: object required: - access_id MetricsForTransferredModel: properties: model_id: type: string model_name: type: string rows_transferred: type: integer delta_rows_transferred: type: integer volume_transferred_in_mb: type: number start_transfer_window_at: type: string format: date-time end_transfer_window_at: type: string format: date-time most_recent_last_updated_at: type: string format: date-time error: $ref: '#/components/schemas/CodedError' additionalProperties: false type: object required: - model_id - rows_transferred - delta_rows_transferred - volume_transferred_in_mb - start_transfer_window_at - end_transfer_window_at - most_recent_last_updated_at - error TransferModelSnapshot: properties: model_id: type: string model_name: type: string revision_id: type: string additionalProperties: false type: object required: - model_id - model_name - revision_id GcpExternalAccountMetadata: properties: type: type: string audience: type: string subject_token_type: type: string service_account_impersonation_url: type: string token_url: type: string credential_source: $ref: '#/components/schemas/CredentialSource' type: object required: - type - audience - subject_token_type - service_account_impersonation_url - token_url - credential_source IcebergStorageOptionsParams: oneOf: - properties: storage: const: bucket_abs bucket_abs: $ref: '#/components/schemas/OpenTableFormatABSBucketParams' description: >- Required details about the ABS bucket used for the iceberg table. required: - storage - bucket_abs title: bucket_abs - properties: storage: const: bucket_gcs bucket_gcs: $ref: '#/components/schemas/OpenTableFormatGCSBucketParams' description: >- Required details about the GCS bucket used for the iceberg table. required: - storage - bucket_gcs title: bucket_gcs - properties: storage: const: bucket_s3 bucket_s3: $ref: '#/components/schemas/OpenTableFormatS3BucketParams' description: Required details about the S3 bucket used for the iceberg table. required: - storage - bucket_s3 title: bucket_s3 - properties: storage: const: bucket_s3_compatible bucket_s3_compatible: $ref: '#/components/schemas/OpenTableFormatS3CompatibleBucketParams' description: >- Required details about the S3 compatible bucket used for the iceberg table. required: - storage - bucket_s3_compatible title: bucket_s3_compatible properties: {} type: object unevaluatedProperties: false OauthBearerTokenAuth: properties: password: type: string title: Bearer Token additionalProperties: false type: object required: - password OauthDelegatedCredentialAuth: properties: client_id: type: string title: Client ID client_secret: type: string title: Client Secret additionalProperties: false type: object required: - client_id - client_secret CodedError: properties: error: true error_code: type: string description: >- A unique Prequel error code in the format PRQL-XXXX identifying the specific error type. title: type: string description: Short human-readable label for the error. blame: type: string description: 'Who is responsible for the error. One of: first_party' documentation_url: type: string description: Link to Prequel documentation for this specific error code. trace: type: string description: Raw error trace. message: type: string description: User-facing explanation of the error and how to resolve it. additionalProperties: false type: object required: - error - error_code - title - blame - documentation_url - trace - message CredentialSource: properties: environment_id: type: string region_url: type: string url: type: string regional_cred_verification_url: type: string additionalProperties: false type: object required: - environment_id - region_url - url - regional_cred_verification_url responses: BadRequest: description: Bad Request content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' Unauthorized: description: Unauthorized content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' NotFound: description: Not Found content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' InternalServerError: description: Internal Server Error content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' securitySchemes: ApiKeyAuth: type: apiKey in: header name: X-API-KEY ````