DocumentationAPI Reference

AWS Aurora

Suggest Edits

Follow these steps to configure your source database and connect to Prequel. Step 1 (creating a read-only endpoint) is optional, but will ensure the Prequel service does not put any unnecessary load on your primary database during data transfer.

Step 1: Create a reader (read-only endpoint) (optional)

  1. In your Amazon RDS Dashboard, click the Aurora PostgreSQL instance to which you want to add a reader.
  2. On the database page, click Actions, then select Add reader from the drop down.
2232
  1. In the Settings section, enter a DB instance identifier. For example, source-transfer-service-reader.
  2. In the DB instance class sections, specify the instance type for the read replica. It can be smaller than the main instance, though AWS may not allow anything smaller.
  3. If you are connecting directly: in the Connectivity section, select the Publicly accessible setting to ensure that the reader is accessible from outside your VPC. Note that it is still only accessible through whitelisted IPs. If you are connecting with an SSH tunnel, this can be set to Not publicly accessible.
1582
  1. Click Add reader.
  2. After a few minutes, the reader status should change to available after it is created.

Step 2: Allow access

Allow read access to a portion of your Aurora PostgreSQL database or the reader you created in Step 1.

Configure the Security Group

  1. In your Amazon RDS > Databases list, click the PostgreSQL instance you want to connect to Prequel.
  2. In the database page, in the Connectivity & security tab, make note of the Endpoint and the Port number.
2220
  1. If you are connecting directly: in the Security section, ensure that set the Publicly accessible setting is set to Yes to ensure that the destination is accessible from outside your VPC. Note that it is still only accessible through whitelisted IPs. If you are connecting with an SSH tunnel, this can be set to No.
2220
  1. Click one of the VPC security groups (usually default). Note: VPC groups are permissive (vs. restrictive) and for instances with multiple VPC security groups, only one needs to be configured with the new inbound rule.
2220

  1. In the Security Groups section, select the Inbound rules tab.

  2. Click Edit inbound rules and then click Add rule.

  3. If you are connecting directly: edit the newly created rule of type Custom TCP with the Port range noted in the first step (usually 5432) and a Custom Source value that includes all of the service IPs. Note: you will need to add /32 to the end of each IP (CIDR notation). If you are connecting through an SSH tunnel, add the security group of the bastion server, as described in Step 3 of AWS SSH Tunneling.

  4. Click Save rules.

2688

Configure network ACLs (access control list)

For database instances in a VCP

  1. In your RDS dashboard, select the PostgreSQL instance.
  2. Click the link to the instance's VPC.
2220
  1. In the VPC menu, click the VPC ID.
2342
  1. In the Details section, click on the link under Main network ACL.
2242
  1. Click on the network ACL ID.
2338

Edit the inbound rules

  1. Click on the Inbound rules tab, and check if there is an existing rule with a Source of 0.0.0.0/0 set to Allow. (This is a default rule created by AWS. If this rule already exists, skip to Edit outbound rules.)
2326
  1. Create the inbound rule (if it doesn't exist). Click Edit inbound rules and either Add new rule or edit an existing rule to allow access to the port number of your database instance (usually 5432) from the Prequel static IP. Click Save changes.

Edit the outbound rules

  1. In the ACL menu, select the Outbound rules tab, and check if there is an existing rule with a Destination of 0.0.0.0/0 set to Allow. (This is a default rule created by AWS. If this rule already exists, skip to the next step.)
2326
  1. Create the outbound rule (if it doesn't exist). Click Edit outbound rules and edit the rules to allow outbound traffic to ports 1024-65535 for Destination 0.0.0.0/0.

Step 3: Create reader user

Create a database user to read the source data.

  1. Open a connection to your Amazon Aurora PostgreSQL database.
  2. Create a user for the data transfer by executing the following SQL command. 
CREATE USER <username> PASSWORD '<some-password>' NOSUPERUSER NOCREATEDB NOCREATEROLE;
  1. Grant user read-only access to all relevant schemas and tables.
GRANT USAGE ON SCHEMA "public" TO <username>;
GRANT SELECT ON ALL TABLES IN SCHEMA "public" TO <username>;
  1. Repeat the step above for all schemas with data that needs to be synced. (Replace "public" with the correct schema name)
    a. Note: if you prefer, you may instead grant usage only the desired tables. 
GRANT USAGE ON SCHEMA "public" TO <username>;
GRANT SELECT ON <table_name_a> TO <username>;
GRANT SELECT ON <table_name_b> TO <username>;
GRANT SELECT ON <table_name_c> TO <username>;

Step 4: Add source to Prequel

Use the cURL request to add the configured source to Prequel.

  1. The name is a descriptive name of the source for your purposes (i.e., a description)
  2. The host [for example, 6.7.8.9 or your-db.sd8jekhrlkhla.us-east-1.rds.amazonaws.com]
  3. The port [most likely 5432]
  4. The vendor: postgres
  5. The database, from Step 2.postgres by default, or whatever database you prefer to use
  6. The username from Step 3
  7. The password from Step 3

Updated 5 months ago