Knowledge Base

Gryd by BuildZoom Data incremental files help you keep your permit data current without reprocessing the much larger full delivery each time.

Incremental files use the same schemas as full files and contain records added or updated since your previous incremental delivery. A full building_permits file may exceed 40 GB, while a weekly incremental file is typically 50–200 MB.

Incremental and full deliveries

Incremental files do not replace full files. Continue processing periodic full deliveries because some historical changes are not included in incremental deliveries. These include deletions and some updates to Gryd by BuildZoom Data-generated metadata, such as project-type classifiers.

File format and naming

• Incremental files have the same columns and data types as the corresponding full files.
• No incremental-only columns are added.
• A file is delivered only when it has new or updated data.
• Incremental filenames contain .incremental; full-delivery filenames contain .full.
• Full and incremental files may arrive on the same date, depending on your delivery schedules.

Records included

New and updated records

Incremental deliveries may include permits, contractors, properties, and related records added to or updated in the Gryd database during the incremental delivery window.

Incremental contractor files contain contractor records that were added or updated during that window. Contractor inclusion is not limited to contractors referenced by permits in the current incremental building_permits file.

Updated permits

The incremental building_permits file includes supported updates to existing permits, including:

• status changes, such as Applied to Issued or Completed
• changes to descriptions, job values, and other government-supplied fields

Some changes to Gryd by BuildZoom Data-generated metadata do not trigger an incremental permit record. Periodic full deliveries ensure that your copy eventually reflects those changes.

Processing instructions

Process all incremental files in delivery order. Complete one delivery before starting the next.

Permits, properties, contractors, and contractor-related files

Upsert each record using the file's primary key:

Use the same upsert approach for related files according to the primary key defined in your delivery schema.

An upsert inserts a record when its primary key is new and updates the existing record when that key is already present.

Apply upsert logic even to incremental files that currently contain only new records. Incremental deliveries always cover changes since the previous incremental delivery, not since the most recent full delivery. If you process a full resynchronization between incremental deliveries, the next incremental delivery may contain records already loaded from that full delivery.

Permit statuses and classifiers

For every permit included in building_permits, Gryd resends the complete available set of status and classifier records for that permit.

For each affected building_permit_id:

1. Delete the existing status and classifier records associated with that permit.
2. Insert the complete replacement set from the current delivery.

Perform the delete and insert within a transaction when your data platform supports transactions. This prevents consumers from seeing a permit with an incomplete status or classifier history.

Recommended ingestion workflow

For each incremental delivery:

1. Confirm that the complete delivery is available before processing it.
2. Stage all delivered files without changing the source values.
3. Upsert permits, properties, contractors, and related records by primary key.
4. Replace status and classifier records for the permits in the current delivery.
5. Validate record counts and capture any rejected records.
6. Mark the delivery complete only after every file has been processed successfully.

If processing fails, correct the issue and rerun the entire delivery using idempotent upsert and replacement logic.

Full resynchronization

We recommend processing a full delivery monthly or quarterly to ensure synchronization, particularly to capture:

• deletions
• historical corrections
• Gryd-generated metadata changes that do not trigger incremental records

After processing a full delivery, continue with the next incremental delivery in sequence. Use upserts because that incremental delivery may overlap with records already present in the full delivery.

Support

Contact data-support@buildzoomdata.com if you have questions or want to enable incremental deliveries. When reporting an ingestion issue, include the delivery date, affected filename, and relevant record identifiers. Do not send credentials or other sensitive information.