DocumentationAPI Reference

BigQuery

Steps to add a BigQuery data warehouse as a data source.

Suggest Edits

Step 1: Create service account in BigQuery project

  1. In the GCP console, in the same project as your BigQuery instance, navigate to the IAM & Admin menu.

  1. Click into the Service Accounts tab.

  1. Click Create service account at the top of the menu.

  1. In the first step, name the user and click Create and Continue.

  1. In the second step, grant the user the BigQuery User role.

  1. In the third and final step, click Done.
  2. Back in the Service accounts menu, click the Actions dropdown next to the newly created service account and click Manage keys.
  1. Click Add key and then Create new key.
  1. Select the JSON Key type and click Create and make note of the key that is generated.

Step 2: Create a staging bucket

Transferring data from BigQuery requires a temporary staging area within Google Cloud Storage to stage compressed data before copying to a destination.

  1. Log into the Google Cloud Console and navigate to Cloud Storage. Click Create to create a new bucket.

  1. Choose a name for the bucket. Click Continue. Select a location for the staging bucket. Make a note of both the name and the location (region).

🚧

Choosing a location (region)

The location you choose for your staging bucket must match the location of your source dataset in BigQuery.

  1. Click continue and complete the options that follow according to your preferences. Once the options have been filled out, click Create.
  2. On the Bucket details page that appears, click the Permissions tab, and then click Add.

  1. In the New principles dropdown, add the Service Account created in Step 1, select the Storage Admin role, and click Save.

Step 3: Grant permission on BigQuery source dataset

  1. Navigate to your BigQuery project and click on the dataset you want to share. Make a note of the Project ID and Data location.
  2. In the menu on the right, click Sharing and then click Permissions.

  1. Click Add Principal.

  1. Select the Service Account created in Step 1, then select the BigQuery User role and the BigQuery Data Viewer role. Click Save.

Step 4: Connect to Prequel

  1. Use the Project ID, Data Location, Source Dataset Name, Bucket Name, Bucket Region, and Service Account Private Key (.json file) to add the Source using the Prequel API.

Updated 5 months ago