airbyte 0.7.0-beta2, Mar 7 25

airbyte 0.7.0-beta2 published on Friday, Mar 7, 2025 by airbytehq

airbyte.DestinationBigquery

Explore with Pulumi AI

airbyte 0.7.0-beta2 published on Friday, Mar 7, 2025 by airbytehq

Example Usage

Coming soon!

Coming soon!

Coming soon!

Coming soon!

package generated_program;

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.core.Output;
import com.pulumi.airbyte.DestinationBigquery;
import com.pulumi.airbyte.DestinationBigqueryArgs;
import com.pulumi.airbyte.inputs.DestinationBigqueryConfigurationArgs;
import java.util.List;
import java.util.ArrayList;
import java.util.Map;
import java.io.File;
import java.nio.file.Files;
import java.nio.file.Paths;

public class App {
    public static void main(String[] args) {
        Pulumi.run(App::stack);
    }

    public static void stack(Context ctx) {
        var myDestinationBigquery = new DestinationBigquery("myDestinationBigquery", DestinationBigqueryArgs.builder()
            .configuration(DestinationBigqueryConfigurationArgs.builder()
                .big_query_client_buffer_size_mb(15)
                .credentials_json("...my_credentials_json...")
                .dataset_id("...my_dataset_id...")
                .dataset_location("US")
                .disable_type_dedupe(true)
                .loading_method(%!v(PANIC=Format method: runtime error: invalid memory address or nil pointer dereference))
                .project_id("...my_project_id...")
                .raw_data_dataset("...my_raw_data_dataset...")
                .transformation_priority("interactive")
                .build())
            .definitionId("92c3eb2b-6d61-4610-adf2-eee065419ed9")
            .workspaceId("acee73dd-54d3-476f-a8ea-d39d218f52cd")
            .build());

    }
}

resources:
  myDestinationBigquery:
    type: airbyte:DestinationBigquery
    properties:
      configuration:
        big_query_client_buffer_size_mb: 15
        credentials_json: '...my_credentials_json...'
        dataset_id: '...my_dataset_id...'
        dataset_location: US
        disable_type_dedupe: true
        loading_method:
          batchedStandardInserts: {}
          gcsStaging:
            credential:
              hmacKey:
                hmacKeyAccessId: 1234567890abcdefghij1234
                hmacKeySecret: 1234567890abcdefghij1234567890ABCDEFGHIJ
            gcsBucketName: airbyte_sync
            gcsBucketPath: data_sync/test
            keepFilesInGcsBucket: Delete all tmp files from GCS
        project_id: '...my_project_id...'
        raw_data_dataset: '...my_raw_data_dataset...'
        transformation_priority: interactive
      definitionId: 92c3eb2b-6d61-4610-adf2-eee065419ed9
      workspaceId: acee73dd-54d3-476f-a8ea-d39d218f52cd

Create DestinationBigquery Resource

Resources are created with functions called constructors. To learn more about declaring and configuring resources, see Resources.

Constructor syntax

new DestinationBigquery(name: string, args: DestinationBigqueryArgs, opts?: CustomResourceOptions);

@overload
def DestinationBigquery(resource_name: str,
                        args: DestinationBigqueryArgs,
                        opts: Optional[ResourceOptions] = None)

@overload
def DestinationBigquery(resource_name: str,
                        opts: Optional[ResourceOptions] = None,
                        configuration: Optional[DestinationBigqueryConfigurationArgs] = None,
                        workspace_id: Optional[str] = None,
                        definition_id: Optional[str] = None,
                        name: Optional[str] = None)

func NewDestinationBigquery(ctx *Context, name string, args DestinationBigqueryArgs, opts ...ResourceOption) (*DestinationBigquery, error)

public DestinationBigquery(string name, DestinationBigqueryArgs args, CustomResourceOptions? opts = null)

public DestinationBigquery(String name, DestinationBigqueryArgs args)
public DestinationBigquery(String name, DestinationBigqueryArgs args, CustomResourceOptions options)

type: airbyte:DestinationBigquery
properties: # The arguments to resource properties.
options: # Bag of options to control resource's behavior.

Parameters

name string: The unique name of the resource.
args DestinationBigqueryArgs: The arguments to resource properties.
opts CustomResourceOptions: Bag of options to control resource's behavior.

resource_name str: The unique name of the resource.
args DestinationBigqueryArgs: The arguments to resource properties.
opts ResourceOptions: Bag of options to control resource's behavior.

ctx Context: Context object for the current deployment.
name string: The unique name of the resource.
args DestinationBigqueryArgs: The arguments to resource properties.
opts ResourceOption: Bag of options to control resource's behavior.

name string: The unique name of the resource.
args DestinationBigqueryArgs: The arguments to resource properties.
opts CustomResourceOptions: Bag of options to control resource's behavior.

name String: The unique name of the resource.
args DestinationBigqueryArgs: The arguments to resource properties.
options CustomResourceOptions: Bag of options to control resource's behavior.

Constructor example

The following reference example uses placeholder values for all input properties.

var destinationBigqueryResource = new Airbyte.DestinationBigquery("destinationBigqueryResource", new()
{
    Configuration = new Airbyte.Inputs.DestinationBigqueryConfigurationArgs
    {
        DatasetId = "string",
        DatasetLocation = "string",
        ProjectId = "string",
        BigQueryClientBufferSizeMb = 0,
        CredentialsJson = "string",
        DisableTypeDedupe = false,
        LoadingMethod = new Airbyte.Inputs.DestinationBigqueryConfigurationLoadingMethodArgs
        {
            BatchedStandardInserts = null,
            GcsStaging = new Airbyte.Inputs.DestinationBigqueryConfigurationLoadingMethodGcsStagingArgs
            {
                Credential = new Airbyte.Inputs.DestinationBigqueryConfigurationLoadingMethodGcsStagingCredentialArgs
                {
                    HmacKey = new Airbyte.Inputs.DestinationBigqueryConfigurationLoadingMethodGcsStagingCredentialHmacKeyArgs
                    {
                        HmacKeyAccessId = "string",
                        HmacKeySecret = "string",
                    },
                },
                GcsBucketName = "string",
                GcsBucketPath = "string",
                KeepFilesInGcsBucket = "string",
            },
        },
        RawDataDataset = "string",
        TransformationPriority = "string",
    },
    WorkspaceId = "string",
    DefinitionId = "string",
    Name = "string",
});

example, err := airbyte.NewDestinationBigquery(ctx, "destinationBigqueryResource", &airbyte.DestinationBigqueryArgs{
Configuration: &.DestinationBigqueryConfigurationArgs{
DatasetId: pulumi.String("string"),
DatasetLocation: pulumi.String("string"),
ProjectId: pulumi.String("string"),
BigQueryClientBufferSizeMb: pulumi.Float64(0),
CredentialsJson: pulumi.String("string"),
DisableTypeDedupe: pulumi.Bool(false),
LoadingMethod: &.DestinationBigqueryConfigurationLoadingMethodArgs{
BatchedStandardInserts: &.DestinationBigqueryConfigurationLoadingMethodBatchedStandardInsertsArgs{
},
GcsStaging: &.DestinationBigqueryConfigurationLoadingMethodGcsStagingArgs{
Credential: &.DestinationBigqueryConfigurationLoadingMethodGcsStagingCredentialArgs{
HmacKey: &.DestinationBigqueryConfigurationLoadingMethodGcsStagingCredentialHmacKeyArgs{
HmacKeyAccessId: pulumi.String("string"),
HmacKeySecret: pulumi.String("string"),
},
},
GcsBucketName: pulumi.String("string"),
GcsBucketPath: pulumi.String("string"),
KeepFilesInGcsBucket: pulumi.String("string"),
},
},
RawDataDataset: pulumi.String("string"),
TransformationPriority: pulumi.String("string"),
},
WorkspaceId: pulumi.String("string"),
DefinitionId: pulumi.String("string"),
Name: pulumi.String("string"),
})

var destinationBigqueryResource = new DestinationBigquery("destinationBigqueryResource", DestinationBigqueryArgs.builder()
    .configuration(DestinationBigqueryConfigurationArgs.builder()
        .datasetId("string")
        .datasetLocation("string")
        .projectId("string")
        .bigQueryClientBufferSizeMb(0)
        .credentialsJson("string")
        .disableTypeDedupe(false)
        .loadingMethod(DestinationBigqueryConfigurationLoadingMethodArgs.builder()
            .batchedStandardInserts()
            .gcsStaging(DestinationBigqueryConfigurationLoadingMethodGcsStagingArgs.builder()
                .credential(DestinationBigqueryConfigurationLoadingMethodGcsStagingCredentialArgs.builder()
                    .hmacKey(DestinationBigqueryConfigurationLoadingMethodGcsStagingCredentialHmacKeyArgs.builder()
                        .hmacKeyAccessId("string")
                        .hmacKeySecret("string")
                        .build())
                    .build())
                .gcsBucketName("string")
                .gcsBucketPath("string")
                .keepFilesInGcsBucket("string")
                .build())
            .build())
        .rawDataDataset("string")
        .transformationPriority("string")
        .build())
    .workspaceId("string")
    .definitionId("string")
    .name("string")
    .build());

destination_bigquery_resource = airbyte.DestinationBigquery("destinationBigqueryResource",
    configuration={
        "dataset_id": "string",
        "dataset_location": "string",
        "project_id": "string",
        "big_query_client_buffer_size_mb": 0,
        "credentials_json": "string",
        "disable_type_dedupe": False,
        "loading_method": {
            "batched_standard_inserts": {},
            "gcs_staging": {
                "credential": {
                    "hmac_key": {
                        "hmac_key_access_id": "string",
                        "hmac_key_secret": "string",
                    },
                },
                "gcs_bucket_name": "string",
                "gcs_bucket_path": "string",
                "keep_files_in_gcs_bucket": "string",
            },
        },
        "raw_data_dataset": "string",
        "transformation_priority": "string",
    },
    workspace_id="string",
    definition_id="string",
    name="string")

const destinationBigqueryResource = new airbyte.DestinationBigquery("destinationBigqueryResource", {
    configuration: {
        datasetId: "string",
        datasetLocation: "string",
        projectId: "string",
        bigQueryClientBufferSizeMb: 0,
        credentialsJson: "string",
        disableTypeDedupe: false,
        loadingMethod: {
            batchedStandardInserts: {},
            gcsStaging: {
                credential: {
                    hmacKey: {
                        hmacKeyAccessId: "string",
                        hmacKeySecret: "string",
                    },
                },
                gcsBucketName: "string",
                gcsBucketPath: "string",
                keepFilesInGcsBucket: "string",
            },
        },
        rawDataDataset: "string",
        transformationPriority: "string",
    },
    workspaceId: "string",
    definitionId: "string",
    name: "string",
});

type: airbyte:DestinationBigquery
properties:
    configuration:
        bigQueryClientBufferSizeMb: 0
        credentialsJson: string
        datasetId: string
        datasetLocation: string
        disableTypeDedupe: false
        loadingMethod:
            batchedStandardInserts: {}
            gcsStaging:
                credential:
                    hmacKey:
                        hmacKeyAccessId: string
                        hmacKeySecret: string
                gcsBucketName: string
                gcsBucketPath: string
                keepFilesInGcsBucket: string
        projectId: string
        rawDataDataset: string
        transformationPriority: string
    definitionId: string
    name: string
    workspaceId: string

DestinationBigquery Resource Properties

To learn more about resource properties and how to use them, see Inputs and Outputs in the Architecture and Concepts docs.

Inputs

In Python, inputs that are objects can be passed either as argument classes or as dictionary literals.

The DestinationBigquery resource accepts the following input properties:

Configuration DestinationBigqueryConfiguration
WorkspaceId string
DefinitionId string: The UUID of the connector definition. One of configuration.destinationType or definitionId must be provided. Requires replacement if changed.
Name string: Name of the destination e.g. dev-mysql-instance.

Configuration DestinationBigqueryConfigurationArgs
WorkspaceId string
DefinitionId string: The UUID of the connector definition. One of configuration.destinationType or definitionId must be provided. Requires replacement if changed.
Name string: Name of the destination e.g. dev-mysql-instance.

configuration DestinationBigqueryConfiguration
workspaceId String
definitionId String: The UUID of the connector definition. One of configuration.destinationType or definitionId must be provided. Requires replacement if changed.
name String: Name of the destination e.g. dev-mysql-instance.

configuration DestinationBigqueryConfiguration
workspaceId string
definitionId string: The UUID of the connector definition. One of configuration.destinationType or definitionId must be provided. Requires replacement if changed.
name string: Name of the destination e.g. dev-mysql-instance.

configuration DestinationBigqueryConfigurationArgs
workspace_id str
definition_id str: The UUID of the connector definition. One of configuration.destinationType or definitionId must be provided. Requires replacement if changed.
name str: Name of the destination e.g. dev-mysql-instance.

configuration Property Map
workspaceId String
definitionId String: The UUID of the connector definition. One of configuration.destinationType or definitionId must be provided. Requires replacement if changed.
name String: Name of the destination e.g. dev-mysql-instance.

Outputs

All input properties are implicitly available as output properties. Additionally, the DestinationBigquery resource produces the following output properties:

CreatedAt double
DestinationId string
DestinationType string
Id string: The provider-assigned unique ID for this managed resource.

CreatedAt float64
DestinationId string
DestinationType string
Id string: The provider-assigned unique ID for this managed resource.

createdAt Double
destinationId String
destinationType String
id String: The provider-assigned unique ID for this managed resource.

createdAt number
destinationId string
destinationType string
id string: The provider-assigned unique ID for this managed resource.

created_at float
destination_id str
destination_type str
id str: The provider-assigned unique ID for this managed resource.

createdAt Number
destinationId String
destinationType String
id String: The provider-assigned unique ID for this managed resource.

Look up Existing DestinationBigquery Resource

Get an existing DestinationBigquery resource’s state with the given name, ID, and optional extra properties used to qualify the lookup.

public static get(name: string, id: Input<ID>, state?: DestinationBigqueryState, opts?: CustomResourceOptions): DestinationBigquery

@staticmethod
def get(resource_name: str,
        id: str,
        opts: Optional[ResourceOptions] = None,
        configuration: Optional[DestinationBigqueryConfigurationArgs] = None,
        created_at: Optional[float] = None,
        definition_id: Optional[str] = None,
        destination_id: Optional[str] = None,
        destination_type: Optional[str] = None,
        name: Optional[str] = None,
        workspace_id: Optional[str] = None) -> DestinationBigquery

func GetDestinationBigquery(ctx *Context, name string, id IDInput, state *DestinationBigqueryState, opts ...ResourceOption) (*DestinationBigquery, error)

public static DestinationBigquery Get(string name, Input<string> id, DestinationBigqueryState? state, CustomResourceOptions? opts = null)

public static DestinationBigquery get(String name, Output<String> id, DestinationBigqueryState state, CustomResourceOptions options)

resources:  _:    type: airbyte:DestinationBigquery    get:      id: ${id}

name: The unique name of the resulting resource.
id: The unique provider ID of the resource to lookup.
state: Any extra arguments used during the lookup.
opts: A bag of options that control this resource's behavior.

resource_name: The unique name of the resulting resource.
id: The unique provider ID of the resource to lookup.

name: The unique name of the resulting resource.
id: The unique provider ID of the resource to lookup.
state: Any extra arguments used during the lookup.
opts: A bag of options that control this resource's behavior.

name: The unique name of the resulting resource.
id: The unique provider ID of the resource to lookup.
state: Any extra arguments used during the lookup.
opts: A bag of options that control this resource's behavior.

name: The unique name of the resulting resource.
id: The unique provider ID of the resource to lookup.
state: Any extra arguments used during the lookup.
opts: A bag of options that control this resource's behavior.

The following state arguments are supported:

Configuration DestinationBigqueryConfiguration
CreatedAt double
DefinitionId string: The UUID of the connector definition. One of configuration.destinationType or definitionId must be provided. Requires replacement if changed.
DestinationId string
DestinationType string
Name string: Name of the destination e.g. dev-mysql-instance.
WorkspaceId string

Configuration DestinationBigqueryConfigurationArgs
CreatedAt float64
DefinitionId string: The UUID of the connector definition. One of configuration.destinationType or definitionId must be provided. Requires replacement if changed.
DestinationId string
DestinationType string
Name string: Name of the destination e.g. dev-mysql-instance.
WorkspaceId string

configuration DestinationBigqueryConfiguration
createdAt Double
definitionId String: The UUID of the connector definition. One of configuration.destinationType or definitionId must be provided. Requires replacement if changed.
destinationId String
destinationType String
name String: Name of the destination e.g. dev-mysql-instance.
workspaceId String

configuration DestinationBigqueryConfiguration
createdAt number
definitionId string: The UUID of the connector definition. One of configuration.destinationType or definitionId must be provided. Requires replacement if changed.
destinationId string
destinationType string
name string: Name of the destination e.g. dev-mysql-instance.
workspaceId string

configuration DestinationBigqueryConfigurationArgs
created_at float
definition_id str: The UUID of the connector definition. One of configuration.destinationType or definitionId must be provided. Requires replacement if changed.
destination_id str
destination_type str
name str: Name of the destination e.g. dev-mysql-instance.
workspace_id str

configuration Property Map
createdAt Number
definitionId String: The UUID of the connector definition. One of configuration.destinationType or definitionId must be provided. Requires replacement if changed.
destinationId String
destinationType String
name String: Name of the destination e.g. dev-mysql-instance.
workspaceId String

Supporting Types

DestinationBigqueryConfiguration, DestinationBigqueryConfigurationArgs

DatasetId string: The default BigQuery Dataset ID that tables are replicated to if the source does not specify a namespace. Read more \n\nhere\n\n.
DatasetLocation string: The location of the dataset. Warning: Changes made after creation will not be applied. Read more \n\nhere\n\n. must be one of ["US", "EU", "asia-east1", "asia-east2", "asia-northeast1", "asia-northeast2", "asia-northeast3", "asia-south1", "asia-south2", "asia-southeast1", "asia-southeast2", "australia-southeast1", "australia-southeast2", "europe-central1", "europe-central2", "europe-north1", "europe-southwest1", "europe-west1", "europe-west2", "europe-west3", "europe-west4", "europe-west6", "europe-west7", "europe-west8", "europe-west9", "europe-west12", "me-central1", "me-central2", "me-west1", "northamerica-northeast1", "northamerica-northeast2", "southamerica-east1", "southamerica-west1", "us-central1", "us-east1", "us-east2", "us-east3", "us-east4", "us-east5", "us-south1", "us-west1", "us-west2", "us-west3", "us-west4"]
ProjectId string: The GCP project ID for the project containing the target BigQuery dataset. Read more \n\nhere\n\n.
BigQueryClientBufferSizeMb double: Google BigQuery client's chunk (buffer) size (MIN=1, MAX = 15) for each table. The size that will be written by a single RPC. Written data will be buffered and only flushed upon reaching this size or closing the channel. The default 15MB value is used if not set explicitly. Read more \n\nhere\n\n. Default: 15
CredentialsJson string: The contents of the JSON service account key. Check out the \n\ndocs\n\n if you need help generating this key. Default credentials will be used if this field is left empty.
DisableTypeDedupe bool: Disable Writing Final Tables. WARNING! The data format in airbytedata is likely stable but there are no guarantees that other metadata columns will remain the same in future versions. Default: false
LoadingMethod DestinationBigqueryConfigurationLoadingMethod: The way data will be uploaded to BigQuery.
RawDataDataset string: The dataset to write raw tables into (default: airbyte_internal)
TransformationPriority string: Interactive run type means that the query is executed as soon as possible, and these queries count towards concurrent rate limit and daily limit. Read more about interactive run type \n\nhere\n\n. Batch queries are queued and started as soon as idle resources are available in the BigQuery shared resource pool, which usually occurs within a few minutes. Batch queries don’t count towards your concurrent rate limit. Read more about batch queries \n\nhere\n\n. The default "interactive" value is used if not set explicitly. Default: "interactive"; must be one of ["interactive", "batch"]

DatasetId string: The default BigQuery Dataset ID that tables are replicated to if the source does not specify a namespace. Read more \n\nhere\n\n.
DatasetLocation string: The location of the dataset. Warning: Changes made after creation will not be applied. Read more \n\nhere\n\n. must be one of ["US", "EU", "asia-east1", "asia-east2", "asia-northeast1", "asia-northeast2", "asia-northeast3", "asia-south1", "asia-south2", "asia-southeast1", "asia-southeast2", "australia-southeast1", "australia-southeast2", "europe-central1", "europe-central2", "europe-north1", "europe-southwest1", "europe-west1", "europe-west2", "europe-west3", "europe-west4", "europe-west6", "europe-west7", "europe-west8", "europe-west9", "europe-west12", "me-central1", "me-central2", "me-west1", "northamerica-northeast1", "northamerica-northeast2", "southamerica-east1", "southamerica-west1", "us-central1", "us-east1", "us-east2", "us-east3", "us-east4", "us-east5", "us-south1", "us-west1", "us-west2", "us-west3", "us-west4"]
ProjectId string: The GCP project ID for the project containing the target BigQuery dataset. Read more \n\nhere\n\n.
BigQueryClientBufferSizeMb float64: Google BigQuery client's chunk (buffer) size (MIN=1, MAX = 15) for each table. The size that will be written by a single RPC. Written data will be buffered and only flushed upon reaching this size or closing the channel. The default 15MB value is used if not set explicitly. Read more \n\nhere\n\n. Default: 15
CredentialsJson string: The contents of the JSON service account key. Check out the \n\ndocs\n\n if you need help generating this key. Default credentials will be used if this field is left empty.
DisableTypeDedupe bool: Disable Writing Final Tables. WARNING! The data format in airbytedata is likely stable but there are no guarantees that other metadata columns will remain the same in future versions. Default: false
LoadingMethod DestinationBigqueryConfigurationLoadingMethod: The way data will be uploaded to BigQuery.
RawDataDataset string: The dataset to write raw tables into (default: airbyte_internal)
TransformationPriority string: Interactive run type means that the query is executed as soon as possible, and these queries count towards concurrent rate limit and daily limit. Read more about interactive run type \n\nhere\n\n. Batch queries are queued and started as soon as idle resources are available in the BigQuery shared resource pool, which usually occurs within a few minutes. Batch queries don’t count towards your concurrent rate limit. Read more about batch queries \n\nhere\n\n. The default "interactive" value is used if not set explicitly. Default: "interactive"; must be one of ["interactive", "batch"]

datasetId String: The default BigQuery Dataset ID that tables are replicated to if the source does not specify a namespace. Read more \n\nhere\n\n.
datasetLocation String: The location of the dataset. Warning: Changes made after creation will not be applied. Read more \n\nhere\n\n. must be one of ["US", "EU", "asia-east1", "asia-east2", "asia-northeast1", "asia-northeast2", "asia-northeast3", "asia-south1", "asia-south2", "asia-southeast1", "asia-southeast2", "australia-southeast1", "australia-southeast2", "europe-central1", "europe-central2", "europe-north1", "europe-southwest1", "europe-west1", "europe-west2", "europe-west3", "europe-west4", "europe-west6", "europe-west7", "europe-west8", "europe-west9", "europe-west12", "me-central1", "me-central2", "me-west1", "northamerica-northeast1", "northamerica-northeast2", "southamerica-east1", "southamerica-west1", "us-central1", "us-east1", "us-east2", "us-east3", "us-east4", "us-east5", "us-south1", "us-west1", "us-west2", "us-west3", "us-west4"]
projectId String: The GCP project ID for the project containing the target BigQuery dataset. Read more \n\nhere\n\n.
bigQueryClientBufferSizeMb Double: Google BigQuery client's chunk (buffer) size (MIN=1, MAX = 15) for each table. The size that will be written by a single RPC. Written data will be buffered and only flushed upon reaching this size or closing the channel. The default 15MB value is used if not set explicitly. Read more \n\nhere\n\n. Default: 15
credentialsJson String: The contents of the JSON service account key. Check out the \n\ndocs\n\n if you need help generating this key. Default credentials will be used if this field is left empty.
disableTypeDedupe Boolean: Disable Writing Final Tables. WARNING! The data format in airbytedata is likely stable but there are no guarantees that other metadata columns will remain the same in future versions. Default: false
loadingMethod DestinationBigqueryConfigurationLoadingMethod: The way data will be uploaded to BigQuery.
rawDataDataset String: The dataset to write raw tables into (default: airbyte_internal)
transformationPriority String: Interactive run type means that the query is executed as soon as possible, and these queries count towards concurrent rate limit and daily limit. Read more about interactive run type \n\nhere\n\n. Batch queries are queued and started as soon as idle resources are available in the BigQuery shared resource pool, which usually occurs within a few minutes. Batch queries don’t count towards your concurrent rate limit. Read more about batch queries \n\nhere\n\n. The default "interactive" value is used if not set explicitly. Default: "interactive"; must be one of ["interactive", "batch"]

datasetId string: The default BigQuery Dataset ID that tables are replicated to if the source does not specify a namespace. Read more \n\nhere\n\n.
datasetLocation string: The location of the dataset. Warning: Changes made after creation will not be applied. Read more \n\nhere\n\n. must be one of ["US", "EU", "asia-east1", "asia-east2", "asia-northeast1", "asia-northeast2", "asia-northeast3", "asia-south1", "asia-south2", "asia-southeast1", "asia-southeast2", "australia-southeast1", "australia-southeast2", "europe-central1", "europe-central2", "europe-north1", "europe-southwest1", "europe-west1", "europe-west2", "europe-west3", "europe-west4", "europe-west6", "europe-west7", "europe-west8", "europe-west9", "europe-west12", "me-central1", "me-central2", "me-west1", "northamerica-northeast1", "northamerica-northeast2", "southamerica-east1", "southamerica-west1", "us-central1", "us-east1", "us-east2", "us-east3", "us-east4", "us-east5", "us-south1", "us-west1", "us-west2", "us-west3", "us-west4"]
projectId string: The GCP project ID for the project containing the target BigQuery dataset. Read more \n\nhere\n\n.
bigQueryClientBufferSizeMb number: Google BigQuery client's chunk (buffer) size (MIN=1, MAX = 15) for each table. The size that will be written by a single RPC. Written data will be buffered and only flushed upon reaching this size or closing the channel. The default 15MB value is used if not set explicitly. Read more \n\nhere\n\n. Default: 15
credentialsJson string: The contents of the JSON service account key. Check out the \n\ndocs\n\n if you need help generating this key. Default credentials will be used if this field is left empty.
disableTypeDedupe boolean: Disable Writing Final Tables. WARNING! The data format in airbytedata is likely stable but there are no guarantees that other metadata columns will remain the same in future versions. Default: false
loadingMethod DestinationBigqueryConfigurationLoadingMethod: The way data will be uploaded to BigQuery.
rawDataDataset string: The dataset to write raw tables into (default: airbyte_internal)
transformationPriority string: Interactive run type means that the query is executed as soon as possible, and these queries count towards concurrent rate limit and daily limit. Read more about interactive run type \n\nhere\n\n. Batch queries are queued and started as soon as idle resources are available in the BigQuery shared resource pool, which usually occurs within a few minutes. Batch queries don’t count towards your concurrent rate limit. Read more about batch queries \n\nhere\n\n. The default "interactive" value is used if not set explicitly. Default: "interactive"; must be one of ["interactive", "batch"]

dataset_id str: The default BigQuery Dataset ID that tables are replicated to if the source does not specify a namespace. Read more \n\nhere\n\n.
dataset_location str: The location of the dataset. Warning: Changes made after creation will not be applied. Read more \n\nhere\n\n. must be one of ["US", "EU", "asia-east1", "asia-east2", "asia-northeast1", "asia-northeast2", "asia-northeast3", "asia-south1", "asia-south2", "asia-southeast1", "asia-southeast2", "australia-southeast1", "australia-southeast2", "europe-central1", "europe-central2", "europe-north1", "europe-southwest1", "europe-west1", "europe-west2", "europe-west3", "europe-west4", "europe-west6", "europe-west7", "europe-west8", "europe-west9", "europe-west12", "me-central1", "me-central2", "me-west1", "northamerica-northeast1", "northamerica-northeast2", "southamerica-east1", "southamerica-west1", "us-central1", "us-east1", "us-east2", "us-east3", "us-east4", "us-east5", "us-south1", "us-west1", "us-west2", "us-west3", "us-west4"]
project_id str: The GCP project ID for the project containing the target BigQuery dataset. Read more \n\nhere\n\n.
big_query_client_buffer_size_mb float: Google BigQuery client's chunk (buffer) size (MIN=1, MAX = 15) for each table. The size that will be written by a single RPC. Written data will be buffered and only flushed upon reaching this size or closing the channel. The default 15MB value is used if not set explicitly. Read more \n\nhere\n\n. Default: 15
credentials_json str: The contents of the JSON service account key. Check out the \n\ndocs\n\n if you need help generating this key. Default credentials will be used if this field is left empty.
disable_type_dedupe bool: Disable Writing Final Tables. WARNING! The data format in airbytedata is likely stable but there are no guarantees that other metadata columns will remain the same in future versions. Default: false
loading_method DestinationBigqueryConfigurationLoadingMethod: The way data will be uploaded to BigQuery.
raw_data_dataset str: The dataset to write raw tables into (default: airbyte_internal)
transformation_priority str: Interactive run type means that the query is executed as soon as possible, and these queries count towards concurrent rate limit and daily limit. Read more about interactive run type \n\nhere\n\n. Batch queries are queued and started as soon as idle resources are available in the BigQuery shared resource pool, which usually occurs within a few minutes. Batch queries don’t count towards your concurrent rate limit. Read more about batch queries \n\nhere\n\n. The default "interactive" value is used if not set explicitly. Default: "interactive"; must be one of ["interactive", "batch"]

datasetId String: The default BigQuery Dataset ID that tables are replicated to if the source does not specify a namespace. Read more \n\nhere\n\n.
datasetLocation String: The location of the dataset. Warning: Changes made after creation will not be applied. Read more \n\nhere\n\n. must be one of ["US", "EU", "asia-east1", "asia-east2", "asia-northeast1", "asia-northeast2", "asia-northeast3", "asia-south1", "asia-south2", "asia-southeast1", "asia-southeast2", "australia-southeast1", "australia-southeast2", "europe-central1", "europe-central2", "europe-north1", "europe-southwest1", "europe-west1", "europe-west2", "europe-west3", "europe-west4", "europe-west6", "europe-west7", "europe-west8", "europe-west9", "europe-west12", "me-central1", "me-central2", "me-west1", "northamerica-northeast1", "northamerica-northeast2", "southamerica-east1", "southamerica-west1", "us-central1", "us-east1", "us-east2", "us-east3", "us-east4", "us-east5", "us-south1", "us-west1", "us-west2", "us-west3", "us-west4"]
projectId String: The GCP project ID for the project containing the target BigQuery dataset. Read more \n\nhere\n\n.
bigQueryClientBufferSizeMb Number: Google BigQuery client's chunk (buffer) size (MIN=1, MAX = 15) for each table. The size that will be written by a single RPC. Written data will be buffered and only flushed upon reaching this size or closing the channel. The default 15MB value is used if not set explicitly. Read more \n\nhere\n\n. Default: 15
credentialsJson String: The contents of the JSON service account key. Check out the \n\ndocs\n\n if you need help generating this key. Default credentials will be used if this field is left empty.
disableTypeDedupe Boolean: Disable Writing Final Tables. WARNING! The data format in airbytedata is likely stable but there are no guarantees that other metadata columns will remain the same in future versions. Default: false
loadingMethod Property Map: The way data will be uploaded to BigQuery.
rawDataDataset String: The dataset to write raw tables into (default: airbyte_internal)
transformationPriority String: Interactive run type means that the query is executed as soon as possible, and these queries count towards concurrent rate limit and daily limit. Read more about interactive run type \n\nhere\n\n. Batch queries are queued and started as soon as idle resources are available in the BigQuery shared resource pool, which usually occurs within a few minutes. Batch queries don’t count towards your concurrent rate limit. Read more about batch queries \n\nhere\n\n. The default "interactive" value is used if not set explicitly. Default: "interactive"; must be one of ["interactive", "batch"]

DestinationBigqueryConfigurationLoadingMethod, DestinationBigqueryConfigurationLoadingMethodArgs

BatchedStandardInserts DestinationBigqueryConfigurationLoadingMethodBatchedStandardInserts: Direct loading using batched SQL INSERT statements. This method uses the BigQuery driver to convert large INSERT statements into file uploads automatically.
GcsStaging DestinationBigqueryConfigurationLoadingMethodGcsStaging: Writes large batches of records to a file, uploads the file to GCS, then uses COPY INTO to load your data into BigQuery.

BatchedStandardInserts DestinationBigqueryConfigurationLoadingMethodBatchedStandardInserts: Direct loading using batched SQL INSERT statements. This method uses the BigQuery driver to convert large INSERT statements into file uploads automatically.
GcsStaging DestinationBigqueryConfigurationLoadingMethodGcsStaging: Writes large batches of records to a file, uploads the file to GCS, then uses COPY INTO to load your data into BigQuery.

batchedStandardInserts DestinationBigqueryConfigurationLoadingMethodBatchedStandardInserts: Direct loading using batched SQL INSERT statements. This method uses the BigQuery driver to convert large INSERT statements into file uploads automatically.
gcsStaging DestinationBigqueryConfigurationLoadingMethodGcsStaging: Writes large batches of records to a file, uploads the file to GCS, then uses COPY INTO to load your data into BigQuery.

batchedStandardInserts DestinationBigqueryConfigurationLoadingMethodBatchedStandardInserts: Direct loading using batched SQL INSERT statements. This method uses the BigQuery driver to convert large INSERT statements into file uploads automatically.
gcsStaging DestinationBigqueryConfigurationLoadingMethodGcsStaging: Writes large batches of records to a file, uploads the file to GCS, then uses COPY INTO to load your data into BigQuery.

batched_standard_inserts DestinationBigqueryConfigurationLoadingMethodBatchedStandardInserts: Direct loading using batched SQL INSERT statements. This method uses the BigQuery driver to convert large INSERT statements into file uploads automatically.
gcs_staging DestinationBigqueryConfigurationLoadingMethodGcsStaging: Writes large batches of records to a file, uploads the file to GCS, then uses COPY INTO to load your data into BigQuery.

batchedStandardInserts Property Map: Direct loading using batched SQL INSERT statements. This method uses the BigQuery driver to convert large INSERT statements into file uploads automatically.
gcsStaging Property Map: Writes large batches of records to a file, uploads the file to GCS, then uses COPY INTO to load your data into BigQuery.

DestinationBigqueryConfigurationLoadingMethodGcsStaging, DestinationBigqueryConfigurationLoadingMethodGcsStagingArgs

Credential DestinationBigqueryConfigurationLoadingMethodGcsStagingCredential: An HMAC key is a type of credential and can be associated with a service account or a user account in Cloud Storage. Read more \n\nhere\n\n.
GcsBucketName string: The name of the GCS bucket. Read more \n\nhere\n\n.
GcsBucketPath string: Directory under the GCS bucket where data will be written.
KeepFilesInGcsBucket string: This upload method is supposed to temporary store records in GCS bucket. By this select you can chose if these records should be removed from GCS when migration has finished. The default "Delete all tmp files from GCS" value is used if not set explicitly. Default: "Delete all tmp files from GCS"; must be one of ["Delete all tmp files from GCS", "Keep all tmp files in GCS"]

Credential DestinationBigqueryConfigurationLoadingMethodGcsStagingCredential: An HMAC key is a type of credential and can be associated with a service account or a user account in Cloud Storage. Read more \n\nhere\n\n.
GcsBucketName string: The name of the GCS bucket. Read more \n\nhere\n\n.
GcsBucketPath string: Directory under the GCS bucket where data will be written.
KeepFilesInGcsBucket string: This upload method is supposed to temporary store records in GCS bucket. By this select you can chose if these records should be removed from GCS when migration has finished. The default "Delete all tmp files from GCS" value is used if not set explicitly. Default: "Delete all tmp files from GCS"; must be one of ["Delete all tmp files from GCS", "Keep all tmp files in GCS"]

credential DestinationBigqueryConfigurationLoadingMethodGcsStagingCredential: An HMAC key is a type of credential and can be associated with a service account or a user account in Cloud Storage. Read more \n\nhere\n\n.
gcsBucketName String: The name of the GCS bucket. Read more \n\nhere\n\n.
gcsBucketPath String: Directory under the GCS bucket where data will be written.
keepFilesInGcsBucket String: This upload method is supposed to temporary store records in GCS bucket. By this select you can chose if these records should be removed from GCS when migration has finished. The default "Delete all tmp files from GCS" value is used if not set explicitly. Default: "Delete all tmp files from GCS"; must be one of ["Delete all tmp files from GCS", "Keep all tmp files in GCS"]

credential DestinationBigqueryConfigurationLoadingMethodGcsStagingCredential: An HMAC key is a type of credential and can be associated with a service account or a user account in Cloud Storage. Read more \n\nhere\n\n.
gcsBucketName string: The name of the GCS bucket. Read more \n\nhere\n\n.
gcsBucketPath string: Directory under the GCS bucket where data will be written.
keepFilesInGcsBucket string: This upload method is supposed to temporary store records in GCS bucket. By this select you can chose if these records should be removed from GCS when migration has finished. The default "Delete all tmp files from GCS" value is used if not set explicitly. Default: "Delete all tmp files from GCS"; must be one of ["Delete all tmp files from GCS", "Keep all tmp files in GCS"]

credential DestinationBigqueryConfigurationLoadingMethodGcsStagingCredential: An HMAC key is a type of credential and can be associated with a service account or a user account in Cloud Storage. Read more \n\nhere\n\n.
gcs_bucket_name str: The name of the GCS bucket. Read more \n\nhere\n\n.
gcs_bucket_path str: Directory under the GCS bucket where data will be written.
keep_files_in_gcs_bucket str: This upload method is supposed to temporary store records in GCS bucket. By this select you can chose if these records should be removed from GCS when migration has finished. The default "Delete all tmp files from GCS" value is used if not set explicitly. Default: "Delete all tmp files from GCS"; must be one of ["Delete all tmp files from GCS", "Keep all tmp files in GCS"]

credential Property Map: An HMAC key is a type of credential and can be associated with a service account or a user account in Cloud Storage. Read more \n\nhere\n\n.
gcsBucketName String: The name of the GCS bucket. Read more \n\nhere\n\n.
gcsBucketPath String: Directory under the GCS bucket where data will be written.
keepFilesInGcsBucket String: This upload method is supposed to temporary store records in GCS bucket. By this select you can chose if these records should be removed from GCS when migration has finished. The default "Delete all tmp files from GCS" value is used if not set explicitly. Default: "Delete all tmp files from GCS"; must be one of ["Delete all tmp files from GCS", "Keep all tmp files in GCS"]

DestinationBigqueryConfigurationLoadingMethodGcsStagingCredential, DestinationBigqueryConfigurationLoadingMethodGcsStagingCredentialArgs

HmacKey DestinationBigqueryConfigurationLoadingMethodGcsStagingCredentialHmacKey

HmacKey DestinationBigqueryConfigurationLoadingMethodGcsStagingCredentialHmacKey

hmacKey DestinationBigqueryConfigurationLoadingMethodGcsStagingCredentialHmacKey

hmacKey DestinationBigqueryConfigurationLoadingMethodGcsStagingCredentialHmacKey

hmac_key DestinationBigqueryConfigurationLoadingMethodGcsStagingCredentialHmacKey

hmacKey Property Map

DestinationBigqueryConfigurationLoadingMethodGcsStagingCredentialHmacKey, DestinationBigqueryConfigurationLoadingMethodGcsStagingCredentialHmacKeyArgs

HmacKeyAccessId string: HMAC key access ID. When linked to a service account, this ID is 61 characters long; when linked to a user account, it is 24 characters long.
HmacKeySecret string: The corresponding secret for the access ID. It is a 40-character base-64 encoded string.

HmacKeyAccessId string: HMAC key access ID. When linked to a service account, this ID is 61 characters long; when linked to a user account, it is 24 characters long.
HmacKeySecret string: The corresponding secret for the access ID. It is a 40-character base-64 encoded string.

hmacKeyAccessId String: HMAC key access ID. When linked to a service account, this ID is 61 characters long; when linked to a user account, it is 24 characters long.
hmacKeySecret String: The corresponding secret for the access ID. It is a 40-character base-64 encoded string.

hmacKeyAccessId string: HMAC key access ID. When linked to a service account, this ID is 61 characters long; when linked to a user account, it is 24 characters long.
hmacKeySecret string: The corresponding secret for the access ID. It is a 40-character base-64 encoded string.

hmac_key_access_id str: HMAC key access ID. When linked to a service account, this ID is 61 characters long; when linked to a user account, it is 24 characters long.
hmac_key_secret str: The corresponding secret for the access ID. It is a 40-character base-64 encoded string.

hmacKeyAccessId String: HMAC key access ID. When linked to a service account, this ID is 61 characters long; when linked to a user account, it is 24 characters long.
hmacKeySecret String: The corresponding secret for the access ID. It is a 40-character base-64 encoded string.

Import

$ pulumi import airbyte:index/destinationBigquery:DestinationBigquery my_airbyte_destination_bigquery ""

To learn more about importing existing cloud resources, see Importing resources.

Package Details

Repository: airbyte airbytehq/terraform-provider-airbyte
License
Notes: This Pulumi package is based on the airbyte Terraform Provider.

airbyte 0.7.0-beta2 published on Friday, Mar 7, 2025 by airbytehq

airbytehq/terraform-provider-airbyte

airbyte.DestinationBigquery

On this page

On this page

Example Usage

Create DestinationBigquery Resource

Constructor syntax

Parameters

Constructor example

DestinationBigquery Resource Properties

Inputs

Outputs

Look up Existing DestinationBigquery Resource

Supporting Types

DestinationBigqueryConfiguration, DestinationBigqueryConfigurationArgs

DestinationBigqueryConfigurationLoadingMethod, DestinationBigqueryConfigurationLoadingMethodArgs

DestinationBigqueryConfigurationLoadingMethodGcsStaging, DestinationBigqueryConfigurationLoadingMethodGcsStagingArgs

DestinationBigqueryConfigurationLoadingMethodGcsStagingCredential, DestinationBigqueryConfigurationLoadingMethodGcsStagingCredentialArgs

DestinationBigqueryConfigurationLoadingMethodGcsStagingCredentialHmacKey, DestinationBigqueryConfigurationLoadingMethodGcsStagingCredentialHmacKeyArgs

Import

Package Details

On this page

On this page