Knowledge Base

Knowledge Base

Bulk Data Delivery via AWS S3

BuildZoom delivers bulk permit, contractor, property, and construction-trend data as compressed CSV files (.csv.gz) through Amazon S3. Each customer receives a dedicated S3 bucket.

This guide explains the available access options and how to retrieve or replicate your files.

Choose an access option

Option Best for How it works
1. BuildZoom-provided IAM user Teams that do not have an AWS account or cannot use their own AWS identities BuildZoom provides credentials for an IAM user that can access your dedicated bucket.
2. Your AWS identity Teams that want to use their own IAM users, roles, or AWS account BuildZoom grants your AWS identity access to your dedicated bucket. This is the preferred option for most AWS customers.
3. Cross-account S3 replication Teams that need files copied automatically into their own S3 bucket Amazon S3 replicates new BuildZoom files to a bucket that you manage.

BuildZoom will confirm the selected option and provide account-specific values during onboarding.

Bucket structure and permissions

Your dedicated bucket may contain these prefixes:

  • from_buildzoom/ — files delivered by BuildZoom
  • from_partner/ — files that you upload for BuildZoom

Standard access includes:

  • read access to from_buildzoom/
  • read, write, and delete access to from_partner/

Values used in this guide

Replace placeholders before running commands or applying policies:

  • {{buildzoom_bucket_name}} — your dedicated BuildZoom bucket
  • {{buildzoom_replication_role_arn}} — the replication role ARN provided by BuildZoom
  • {{your_account_id}} — your 12-digit AWS account ID
  • {{your_role_name}} — the name of your IAM role
  • {{your_destination_bucket}} — the S3 bucket that you manage

BuildZoom will provide the values specific to your account.

Option 1: BuildZoom-provided IAM user

Choose this option if your team cannot use its own AWS account or identity.

BuildZoom will:

  1. Create an IAM user with access limited to your dedicated bucket.
  2. Share the credentials through an approved secure credential-sharing service.
  3. Provide your bucket name and any account-specific instructions.

Configure the AWS CLI

Install the AWS CLI , then create a named profile:

aws configure --profile buildzoom_data

Enter the credentials provided by BuildZoom. Use us-east-1 as the default region and json as the output format unless BuildZoom instructs you otherwise.

AWS Access Key ID [None]: <access key provided by BuildZoom>
AWS Secret Access Key [None]: <secret key provided by BuildZoom>
Default region name [None]: us-east-1
Default output format [None]: json

Do not include AWS credentials in email, support tickets, source code, or shell scripts.

Option 2: Use your own AWS identity

Choose this option to access the BuildZoom bucket through your own AWS environment. BuildZoom can grant access to:

  • a specific IAM user
  • an IAM role
  • your AWS account

Granting access to the AWS account is generally preferred because your administrators can manage access internally without asking BuildZoom to update the bucket policy each time a user or role changes.

Access must be configured on both sides:

  1. BuildZoom allows your AWS identity to access your dedicated bucket.
  2. Your AWS administrator grants the required S3 permissions to the users or roles that need access.

Example IAM policy

Attach a policy like the following to the IAM user or role that will access the data. Replace {{buildzoom_bucket_name}} with the bucket name provided by BuildZoom.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "s3:GetBucketLocation",
      "Resource": "arn:aws:s3:::{{buildzoom_bucket_name}}"
    },
    {
      "Effect": "Allow",
      "Action": "s3:ListBucket",
      "Resource": "arn:aws:s3:::{{buildzoom_bucket_name}}"
    },
    {
      "Effect": "Allow",
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:::{{buildzoom_bucket_name}}/from_buildzoom/*"
    },
    {
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject"],
      "Resource": "arn:aws:s3:::{{buildzoom_bucket_name}}/from_partner/*"
    }
  ]
}

If users must assume a role before accessing the bucket, grant them permission to assume that role:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "sts:AssumeRole",
      "Resource": "arn:aws:iam::{{your_account_id}}:role/{{your_role_name}}"
    }
  ]
}

Use other transfer tools

Because this option uses your own AWS identity, you can retrieve files with the AWS CLI, AWS Management Console, S3-compatible clients, your own applications, or services such as AWS DataSync .

With DataSync or another client, your systems pull data from the BuildZoom bucket. With Option 3, Amazon S3 pushes new files into your destination bucket. Your team is responsible for configuring and operating any third-party or AWS transfer service that it chooses to use.

Option 3: Replicate files to your S3 bucket

Choose this option when files must appear automatically in an S3 bucket that your organization manages. BuildZoom uses native Amazon S3 cross-account replication.

How replication works

  • Only objects under from_buildzoom/ are replicated.
  • New objects are replicated automatically, typically shortly after delivery.
  • Object keys are preserved. For example, s3://{{buildzoom_bucket_name}}/from_buildzoom/file.csv.gz becomes s3://{{your_destination_bucket}}/from_buildzoom/file.csv.gz.
  • Replicated files cannot be placed under an additional prefix such as buildzoom/from_buildzoom/.
  • Your account owns the replicated objects in your destination bucket.
  • Cross-region replication may require approval because it creates additional transfer costs.

Information to send BuildZoom

Provide:

  • your destination bucket ARN, such as arn:aws:s3:::{{your_destination_bucket}}
  • your 12-digit AWS account ID
  • the AWS Region of your destination bucket

BuildZoom will then provide the replication IAM role ARN that must be allowed by your destination bucket policy.

Configure your destination bucket

1. Enable versioning

S3 replication requires versioning on the destination bucket.

aws s3api put-bucket-versioning \
  --bucket {{your_destination_bucket}} \
  --versioning-configuration Status=Enabled

2. Add a destination bucket policy

Add a statement that allows the BuildZoom replication role to write objects. Replace both placeholders with the values provided during setup.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowBuildZoomReplication",
      "Effect": "Allow",
      "Principal": {
        "AWS": "{{buildzoom_replication_role_arn}}"
      },
      "Action": [
        "s3:ReplicateObject",
        "s3:ReplicateDelete",
        "s3:ReplicateTags",
        "s3:ObjectOwnerOverrideToBucketOwner"
      ],
      "Resource": "arn:aws:s3:::{{your_destination_bucket}}/*"
    }
  ]
}

If your destination bucket uses AWS KMS encryption or has additional security controls, your AWS administrator may need to grant extra permissions. Coordinate those requirements with BuildZoom before enabling replication.

3. Notify BuildZoom

After versioning and the bucket policy are in place, notify your BuildZoom contact. BuildZoom will enable replication and confirm the result with a test file.

Work with delivered files

The following examples use the AWS CLI profile from Option 1. If you selected Option 2, replace buildzoom_data with your own profile name or omit --profile when your environment already supplies AWS credentials.

List available files

aws s3 ls s3://{{buildzoom_bucket_name}}/from_buildzoom/ \
  --recursive \
  --profile buildzoom_data

Download one file

aws s3 cp \
  s3://{{buildzoom_bucket_name}}/from_buildzoom/example.csv.gz \
  ./example.csv.gz \
  --profile buildzoom_data

Download all delivered files

aws s3 sync \
  s3://{{buildzoom_bucket_name}}/from_buildzoom/ \
  ./buildzoom-data/ \
  --profile buildzoom_data

Upload a file for BuildZoom

aws s3 cp \
  ./example.csv.gz \
  s3://{{buildzoom_bucket_name}}/from_partner/example.csv.gz \
  --profile buildzoom_data

Troubleshooting

AccessDenied

Confirm that:

  • you are using the expected AWS profile or role
  • your policy references the exact bucket name provided by BuildZoom
  • downloads use from_buildzoom/
  • uploads use from_partner/
  • your BuildZoom contact has completed the corresponding access configuration

Files do not appear in the destination bucket

For replication, confirm that:

  • versioning is enabled on the destination bucket
  • the destination bucket policy uses the exact replication role ARN provided by BuildZoom
  • the policy resource points to arn:aws:s3:::{{your_destination_bucket}}/*
  • no additional prefix is expected before from_buildzoom/
  • any AWS KMS keys allow the required replication access

Support

For help with credentials, bucket access, or replication setup, contact your BuildZoom representative. Include the bucket name, AWS account ID, approximate time of the failed request, and the AWS error message. Never send secret access keys or other credentials.

Use Cases