BigQuery
Steps to add a BigQuery data warehouse as a data source.
Step 1: Create service account in BigQuery project
- In the GCP console, in the same project as your BigQuery instance, navigate to the IAM & Admin menu.
- Click into the Service Accounts tab.
- Click Create service account at the top of the menu.
- In the first step, name the user and click Create and Continue.
- In the second step, grant the user the BigQuery User role.
- In the third step ("Grant users access to this service account"), within the Service account users role field, enter the provided Service account (see prerequisite) and click Done.
- Once successfully created, search for the created service account in the service accounts list, click the Service account name to view the details. Select the permissions tab, find the provided principal name (Service account from the prerequisite), click the Edit principal button (pencil icon), click Add another role, select the Service Account Token Creator role & Service Account User role, and click Save.
Step 2: Create a staging bucket
Transferring data from BigQuery requires a temporary staging area within Google Cloud Storage to stage compressed data before copying to a destination.
- Log into the Google Cloud Console and navigate to Cloud Storage. Click Create to create a new bucket.
- Choose a name for the bucket. Click Continue. Select a location for the staging bucket. Make a note of both the name and the location (region).
Choosing a
location
(region)The location you choose for your staging bucket must match the location of your source dataset in BigQuery.
- Click continue and complete the options that follow according to your preferences. Once the options have been filled out, click Create.
- On the Bucket details page that appears, click the Permissions tab, and then click Add.
- In the New principles dropdown, add the Service Account created in Step 1, select the Storage Admin role, and click Save.
Step 3: Grant permission on BigQuery source dataset
- Navigate to your BigQuery project and click on the dataset you want to share. Make a note of the Project ID and Data location.
- In the menu on the right, click Sharing and then click Permissions.
- Click Add Principal.
- Select the Service Account created in Step 1, then select the BigQuery User role and the BigQuery Data Viewer role. Click Save.
Step 4: Connect to Prequel
- Use the Project ID, Data Location, Source Dataset Name, Bucket Name, and Bucket Region in the Admin UI or API to complete the connection.
Updated 9 months ago