Flat Files

CDL distributes property, assessor, mortgage, deed, AVM, school, and related real estate data to clients and prospects via two channels:

• Flat File Exports - CSV files exported to client S3 locations via Snowflake COPY INTO
• Snowflake Data Shares - Live data shared directly to prospect/partner Snowflake accounts

Flat File Exports

CSV flat file exports are delivered on a configured schedule. The EXPORT_CONFIG table drives all exports - each row defines a dataset, frequency, and delivery method.

Export Frequencies

Folder Structure

Exports are partitioned into folders using different strategies depending on dataset size and delivery needs.

Per State/Date/Month structures use dedicated stored procedures in L40_EXPORT that iterate over each partition value and export files individually. The “None” structure uses direct COPY INTO statements driven by the config table.

Snowflake Data Shares

Data shares provide live, always-current data directly in a Snowflake account - no files, no ETL, no lag. CDL uses Snowflake Secure Data Sharing so recipients query shared objects in their own account without receiving a physical copy of the source data.

How Outbound Sharing Works

Outbound shares deliver CDL datasets to approved Snowflake accounts. Access is granted only to accounts you authorize; recipients see the shared views or tables you expose, not the underlying pipelines or vendor feeds that produce them.

• Public Marketplace listings - Discoverable listings that any Snowflake account can request or subscribe to under the published terms.
• Private / targeted listings - Listings limited to specific Snowflake accounts you invite.
• Direct shares - Account-to-account shares created for a named recipient without a marketplace listing.

Typical setup:

1. Define the share objects (views or tables) that contain only the datasets intended for delivery.
2. Create a share or listing and grant it to the recipient Snowflake account(s).
3. The recipient creates a database from the share and queries it with their own roles and warehouses.
4. Updates in the shared objects appear automatically for the recipient - no re-export step.

How Inbound Sharing Works

Inbound shares are how CDL receives licensed data into Snowflake from upstream providers. Provider identity and feed details are contract-specific and are not listed in this documentation.

• Providers grant a secure share (or listing) to CDL’s Snowflake account.
• CDL mounts the share as a read-only database and loads or transforms only what licensing allows into internal schemas.
• Downstream products (APIs, flat files, outbound shares) are built from CDL-managed objects - not by exposing the original provider share to customers.

This pattern keeps vendor and partner identities, listing IDs, and source-system names out of customer-facing docs while still documenting the delivery model.

Scheduled Tasks

Three Snowflake Tasks orchestrate all exports and notifications.

Daily Export Task (Partitioned Exports)

• Schedule: Every day at 12:00 UTC
• What it does: Reads all Dailies exports with a folder structure (Per State, Per Date, Per Month) from EXPORT_CONFIG. For each enabled row whose execution date matches today, it calls the corresponding stored procedure which iterates over partitions and runs COPY INTO for each.
• On success: Sends a summary email to skyline@cdatalabs.com
• On failure: Sends error email to both the client and skyline@cdatalabs.com

Monthly Export Task (Partitioned Exports)

• Schedule: Every day at 04:00 UTC
• What it does: Same logic as the daily task, but for Monthly exports with a folder structure. Runs daily but only picks up configs whose execution date matches (e.g., day 15 of the month, or * for every day).
• On success/failure: Same email behavior as daily task

Daily Export Email Notification

• Schedule: Every day at 02:00 UTC
• What it does: Sends client-facing daily export summary emails. For each client with alerts enabled in CLIENT_CONFIG, it reads today’s TASK_LOGS entries, maps dataset names to friendly labels (e.g., TOMO_DEED becomes “Deed”), and compiles a summary showing each dataset, row count, and timestamp. Zero-row exports are suppressed.

Task Logs

Every export execution is logged in the TASK_LOGS table. This is the primary audit trail for all export activity.

Each log entry records:

• DATASETNAME - which dataset was exported (e.g., BASEPROPERTIESAGGREGATE, MORTGAGEHISTORYAGGREGATE, L40_EXPORT.ASSESSORLEGACYMAIN)
• CLIENTNAME - which client the export was for
• LOG_MESSAGE - the outcome, typically “Total No of rows exported are {count}” for success, or “ERROR: {message}” for failures
• LOG_TIME - when the export completed