> ## Documentation Index > Fetch the complete documentation index at: https://docs.prequel.co/llms.txt > Use this file to discover all available pages before exploring further. # Deployment options > Choosing your preferred Prequel deployment option There are three options for deploying Prequel ## Cloud hosted If you choose Cloud Hosted, you will receive an onboarding email with instructions for signing into Prequel as soon as your cloud hosted deployment is ready. ## Private-cloud hosted In a private-cloud deployment, the Prequel team will be right there with you to ensure a smooth delivery. You will receive an onboarding email with instructions for signing into the shared-cloud instance of Prequel as soon as it's ready. ## Self-hosted Deploy Prequel on cloud infrastructure you control. Select your cloud below for the full guide. Self-hosted deployments rely on a few tools: * **Terraform v1.0.x** for provisioning the services required by Prequel * **Helm 3.9.4+** for installing and upgrading Prequel * **Kubernetes CLI** for managing and inspecting the Kubernetes cluster - Validate that you have access to the Terraform directory and Helm chart we sent over. If not, please email or Slack `support@prequel.co` to request access. - Create the dedicated cloud project where you'd like Prequel to run. We typically recommend creating a new cloud project for this, which allows all resources to be fully sandboxed from any other existing infrastructure and ensures that there will not be contention between VPCs, networks, or other resources. In the project you created for Prequel, navigate to the AWS Certificate Manager and `Request a certificate`. We'll need a certificate here for `*.your-domain.com`, since this is how we'll enable TLS/HTTPS for the Prequel deployment. Grab the ARN of the certificate you created and keep it handy for later. Take a look through `variables.tf` and fill in the required values. We have a `terraform.tfvars.example` that can be your reference. Perform a Terraform dry-run and double check that everything looks good. ```hcl title="Terraform" theme={null} terraform plan ``` Terraform the `main.tf` file. This will create all the necessary infrastructure for Prequel to run. Save the output variables, you'll need them later. ```hcl title="Terraform" theme={null} terraform apply ``` Auth into the cluster we just created. ```shell title="Authenticate to cluster" icon="terminal" theme={null} aws eks update-kubeconfig --name clustername --region clusterregion ``` **K8s cluster access** Once you've setup the Prequel infrastructure and connected to the EKS cluster, we highly recommend adding additional users/groups to the access entries of the cluster. This will ensure that if the original creator of the cluster loses access to the cluster, there will be additional users who can access and manage the cluster. This can be done via UI or via terraform. If you want to do it through terraform, you can leverage the `access_entries` variable, and there is an example in the `terraform.tfvars.example` file. Accessing and managing the cluster is necessary to support the deployment, maintain Prequel and update the Prequel software. Please see AWS documentation below with information how to add users/groups to the access entries. [https://docs.aws.amazon.com/eks/latest/userguide/access-entries.html](https://docs.aws.amazon.com/eks/latest/userguide/access-entries.html) Once the users/groups have been added to the access entries, you can instruct the new users to use the instructions below to create a new kube config to access the cluster. [https://docs.aws.amazon.com/eks/latest/userguide/create-kubeconfig.html](https://docs.aws.amazon.com/eks/latest/userguide/create-kubeconfig.html) Create the following Kubernetes secrets that Prequel requires: ```shell title="Create Kubernetes secrets" icon="terminal" expandable theme={null} # Generate and store secure random values in environment variables export POSTGRES_PASSWORD={your_db_password} export WORKOS_API_KEY={workos_api_key} export SSH_SALT={your_generated_ssh_salt} export ADMIN_API_KEY={your_generated_admin_api_key} export AUTH_TOKEN_KEY={your_generated_auth_token_key} # Store the keys provided to you by Prequel export LICENSE_KEY={license_key_from_prequel} export HONEYCOMB_API_KEY={honeycomb_api_key_from_prequel} # Create secret for Postgres DB credentials kubectl create secret generic datafeed-postgres \ --from-literal=password="${POSTGRES_PASSWORD}" # Create secret for SSH salt (used for hashing public keys) kubectl create secret generic datafeed-ssh-salt \ --from-literal=salt="${SSH_SALT}" # Create secret for Shepherd service kubectl create secret generic datafeed-shepherd \ --from-literal=apiKey="${ADMIN_API_KEY}" \ --from-literal=authToken="${AUTH_TOKEN_KEY}" \ --from-literal=workOSApiKey="${WORKOS_API_KEY}" # Create secret for the Prequel license key kubectl create secret generic datafeed-license \ --from-literal=licenseKey="${LICENSE_KEY}" # Create secret for the Honeycomb API key kubectl create secret generic datafeed-otlp \ --from-literal=api-key="${HONEYCOMB_API_KEY}" ``` Make sure to store these generated values securely for future maintenance and troubleshooting. Each value is: * `datafeed-postgres.password`: The password for your Postgres database. * `datafeed-ssh-salt.salt`: A random 32-char string used for hashing SSH public keys. * `datafeed-shepherd.workOSApiKey`: The WorkOS API key provided to you by Prequel. * `datafeed-shepherd.apiKey`: A random 32-char string used for admin API authentication. * `datafeed-shepherd.authToken`: A random 32-char string used to encrypt/decrypt authentication tokens. * `datafeed-license.licenseKey`: The license key provided to you by Prequel. * `datafeed-otlp.api-key`: The Honeycomb API key provided to you by Prequel. Fill in the `aws_on_prem_values_overrides.yaml` for the Prequel Helm chart. The following values should be set from the secrets created above: * `postgresDb.secretName`: `datafeed-postgres` or the name of the secret created for Postgres DB. * `postgresDb.passwordSecretKey`: `password` or the key in the secret created for Postgres DB that contains the password. * `sshSaltSecretName`: `datafeed-ssh-salt` or the name of the secret created for SSH salt. * `sshSaltSecretKey`: `salt` or the key in the secret created for SSH salt that contains the salt. * `shepherd.secretName`: `datafeed-shepherd` or the name of the secret created for Shepherd service. * `shepherd.workOS.apiKeySecretKey`: `workOSApiKey` or the key in the secret created for Shepherd service that contains the WorkOS API key provided to you by Prequel. * `shepherd.apiKeySecretKey`: `apiKey` or the key in the secret created for Shepherd service that contains the admin API key. * `shepherd.authTokenSecretKey`: `authToken` or the key in the secret created for Shepherd service that contains the authentication token key. * `licenseKeySecretName`: `datafeed-license` or the name of the secret created for the license key. * `licenseKeySecretKey`: `licenseKey` or the key in the secret created for the license key that contains the license key. Install the Prequel Helm chart. ```shell title="Install Helm chart" icon="terminal" theme={null} helm install prequel datafeed-{chart_version}.tgz -f aws_on_prem_values_overrides.yaml ``` The cluster should now be up and running. Nice work, we're almost there! Grab the address of the ingress / LB for the Prequel deployment. ```shell title="Get ingress address" icon="terminal" theme={null} kubectl get ing ``` and look for the `ADDRESS` field. In your domain settings, create DNS records for the three hosts used by Prequel. Specifically, for each, create a `CNAME` record which points to the address from the previous step. ```text title="DNS records" icon="file-lines" expandable theme={null} prequel.your-domain.com # the domain you'll use when hitting the API. prequel-admin.your-domain.com # the UI that admins on your team will use to manage Prequel. data-connect.your-domain.com # the domain your customers will use to connect their data warehouse ``` Notify your Prequel counterpart that the deployment is ready to roll. They'll guide you through next steps: configuring your first source. #### Updating Prequel We'll notify you when a new release is available, and provide you with the release tag. You can then run the following command to update your deployment to the new release. ```shell title="Upgrade Helm release" icon="terminal" theme={null} helm upgrade prequel datafeed-{chart_version}.tgz --reuse-values --set image.tag={provided_release_tag} ``` * Validate that you have access to the Terraform directory and Helm chart we sent over. If not, please email or Slack `support@prequel.co` to request access. * Create the dedicated cloud project where you'd like Prequel to run. We typically recommend creating a new cloud project for this, which allows all resources to be fully sandboxed from any other existing infrastructure. Take a look through `variables.tf` and fill in the required values. We have a `terraform.tfvars.example` that can be your reference. Perform a Terraform dry-run and double check that everything looks good. ```hcl title="Terraform" theme={null} terraform plan ``` Terraform the `main.tf` file. This will create all the necessary infrastructure for Prequel to run. Save the output variables, you'll need them later. ```hcl title="Terraform" theme={null} terraform apply ``` Update your DNS records to point to the `prequel-ingress-ip` returned by the Terraform script. You'll need to create three DNS records. ```text title="DNS records" icon="file-lines" expandable theme={null} prequel.your-domain.com # the domain you'll use when hitting the API. prequel-admin.your-domain.com # the UI that admins on your team will use to manage Prequel. data-connect.your-domain.com # the domain your customers will use to connect their data warehouse ``` **Important**: Workload Identity must be configured to allow Kubernetes service accounts to assume the GCP service account created by Terraform. This is required for Prequel services to access GCP resources. Verify that Workload Identity is enabled on your GKE cluster. If you used the provided Terraform configuration, this should already be enabled. You can check with: ```shell title="Verify Workload Identity" icon="terminal" expandable theme={null} # For zonal clusters gcloud container clusters describe {your_cluster_name} --zone={your_zone} --project={your_cluster_project_id} --format="value(workloadIdentityConfig.workloadPool)" # For regional clusters gcloud container clusters describe {your_cluster_name} --region={your_region} --project={your_cluster_project_id} --format="value(workloadIdentityConfig.workloadPool)" ``` The output should show `{your_cluster_project_id}.svc.id.goog`. If this is empty, Workload Identity is not enabled. **Note**: If your GKE cluster is in a different project than your service account, replace `{your_cluster_project_id}` with the actual project ID where your cluster is deployed. Create the IAM policy binding to allow the Kubernetes service account to impersonate the GCP service account: ```shell title="Bind service accounts" icon="terminal" expandable theme={null} # Replace the placeholder values with your actual values # {your_cluster_project_id} = Project ID where your GKE cluster is deployed # {your_service_account_project_id} = Project ID where your service account is created # {service_account_name} = Name of your service account (from Terraform) gcloud iam service-accounts add-iam-policy-binding \ --role roles/iam.workloadIdentityUser \ --member "serviceAccount:{your_cluster_project_id}.svc.id.goog[default/datafeed]" \ {service_account_name}@{your_service_account_project_id}.iam.gserviceaccount.com # Also bind for the animalcontrol service account gcloud iam service-accounts add-iam-policy-binding \ --role roles/iam.workloadIdentityUser \ --member "serviceAccount:{your_cluster_project_id}.svc.id.goog[default/animalcontrol]" \ {service_account_name}@{your_service_account_project_id}.iam.gserviceaccount.com ``` Verify the workload identity binding: ```shell title="Verify binding" icon="terminal" theme={null} # Test that the binding was created successfully gcloud iam service-accounts get-iam-policy {service_account_name}@{your_service_account_project_id}.iam.gserviceaccount.com ``` You should see the workload identity bindings in the output. Authenticate to the Kubernetes cluster created during infrastructure setup. Create the following Kubernetes secrets required by the Prequel deployment. ```shell title="Create Kubernetes secrets" icon="terminal" expandable theme={null} # Generate and store secure random values in environment variables export POSTGRES_PASSWORD={your_db_password} export WORKOS_API_KEY={workos_api_key} export SSH_SALT={your_generated_ssh_salt} export ADMIN_API_KEY={your_generated_admin_api_key} export AUTH_TOKEN_KEY={your_generated_auth_token_key} # Store the keys provided to you by Prequel export LICENSE_KEY={license_key_from_prequel} export HONEYCOMB_API_KEY={honeycomb_api_key_from_prequel} # Create secret for Postgres DB credentials kubectl create secret generic datafeed-postgres \ --from-literal=password="${POSTGRES_PASSWORD}" # Create secret for SSH salt (used for hashing public keys) kubectl create secret generic datafeed-ssh-salt \ --from-literal=salt="${SSH_SALT}" # Create secret for Shepherd service kubectl create secret generic datafeed-shepherd \ --from-literal=apiKey="${ADMIN_API_KEY}" \ --from-literal=authToken="${AUTH_TOKEN_KEY}" \ --from-literal=workOSApiKey="${WORKOS_API_KEY}" # Create secret for the Prequel license key kubectl create secret generic datafeed-license \ --from-literal=licenseKey="${LICENSE_KEY}" # Create secret for the Honeycomb API key kubectl create secret generic datafeed-otlp \ --from-literal=api-key="${HONEYCOMB_API_KEY}" ``` Make sure to store these generated values securely for future maintenance and troubleshooting. Each value is: * `datafeed-postgres.password`: The password for your Postgres database. * `datafeed-ssh-salt.salt`: A random 32-char string used for hashing SSH public keys. * `datafeed-shepherd.workOSApiKey`: The WorkOS API key provided to you by Prequel. * `datafeed-shepherd.apiKey`: A random 32-char string used for admin API authentication. * `datafeed-shepherd.authToken`: A random 32-char string used to encrypt/decrypt authentication tokens. * `datafeed-license.licenseKey`: The license key provided to you by Prequel. * `datafeed-otlp.api-key`: The Honeycomb API key provided to you by Prequel. Install the `cert-manager` Helm chart. ```shell title="Install cert-manager" icon="terminal" expandable theme={null} helm install cert-manager oci://quay.io/jetstack/charts/cert-manager \ --namespace cert-manager \ --create-namespace \ --version v1.18.2 \ --set crds.enabled=true \ --set ingressShim.defaultIssuerName=letsencrypt-prod \ --set ingressShim.defaultIssuerKind=ClusterIssuer \ --set ingressShim.defaultIssuerGroup=cert-manager.io ``` Fill `gcp_on_prem_values_overrides.yaml` based on your configurations. The following values should be set from the secrets created above: * `postgresDb.secretName`: `datafeed-postgres` or the name of the secret created for Postgres DB. * `postgresDb.passwordSecretKey`: `password` or the key in the secret created for Postgres DB that contains the password. * `sshSaltSecretName`: `datafeed-ssh-salt` or the name of the secret created for SSH salt. * `sshSaltSecretKey`: `salt` or the key in the secret created for SSH salt that contains the salt. * `shepherd.secretName`: `datafeed-shepherd` or the name of the secret created for Shepherd service. * `shepherd.workOS.apiKeySecretKey`: `workOSApiKey` or the key in the secret created for Shepherd service that contains the WorkOS API key provided to you by Prequel. * `shepherd.apiKeySecretKey`: `apiKey` or the key in the secret created for Shepherd service that contains the admin API key. * `shepherd.authTokenSecretKey`: `authToken` or the key in the secret created for Shepherd service that contains the authentication token key. * `licenseKeySecretName`: `datafeed-license` or the name of the secret created for the license key. * `licenseKeySecretKey`: `licenseKey` or the key in the secret created for the license key that contains the license key. Install the Prequel Helm chart. ```shell title="Install Helm chart" icon="terminal" theme={null} helm install prequel datafeed-{chart_version}.tgz -f gcp_on_prem_values_overrides.yaml ``` Notify your Prequel counterpart that the deployment is ready to roll. They'll guide you through next steps: configuring your first source. #### Updating Prequel We'll notify you when a new release is available, and provide you with the release tag. You can then run the following command to update your deployment to the new release. ```shell title="Upgrade Helm release" icon="terminal" theme={null} helm upgrade prequel datafeed-{chart_version}.tgz --reuse-values --set image.tag={provided_release_tag} ```