Dataset Webhook Export

xport datasets asynchronously and receive the results via webhook when processing is complete. The dataset export endpoint (/dataset/export, dataset-export-async in the API Reference) uses the same query engine and data model as Query Builder—it returns the same data with the same limitations (e.g. no unit-level data; week is the finest time granularity). This is ideal for large datasets that would timeout with synchronous requests, or when you want to process data in the background without maintaining an active connection.

How It Works

1. Submit Export Request: Send your dataset query along with a webhook URL
2. Receive Confirmation: Get immediate 201 response with export queued
3. Processing: Your export is processed asynchronously in the background
4. Webhook Notification: Your endpoint receives the results when complete

Request Format

The request requires:

• dataset: Your query configuration (columns, filters, limits)
• webhookURL: HTTPS endpoint where results will be sent
• name: Descriptive name for your export
• format: Either ‘csv’ or ‘jsonl’

Webhook Response

When your export completes, you’ll receive a POST request with this payload:

1
{
2
  "queryUUID": "123e4567-e89b-12d3-a456-426614174000",
3
  "name": "Seattle Rental Data",
4
  "format": "csv", 
5
  "payload": { /* your original dataset query */ },
6
  "gcsBlob": {
7
    "url": "https://storage.googleapis.com/...",
8
    "expires": "2024-02-15T10:30:00Z",
9
    "timeTakenMs": 45000,
10
    "numberRows": 12847
11
  },
12
  "requestedOn": "2024-01-15T10:00:00Z",
13
  "completedOn": "2024-01-15T10:00:45Z"
14
}

$
     curl --location 'https://api.hellodata.ai/dataset/export' \
>
--header 'Content-Type: application/json' \
>
--header 'x-api-key: your-api-key' \
>
--data '{
>
    "dataset": {
>
        "scopes": [
>
            {"column": "street_address"},
>
            {"column": "asking_rent", "aggregate": "Avg"}
>
        ],
>
        "filters": [
>
            {"column": "msa", "filter": {"equals": "Seattle, WA"}},
>
            {"column": "bed", "filter": {"in": [1, 2, 3]}}
>
        ],
>
        "limit": 10000
>
    },
>
    "webhookURL": "https://yourapp.com/webhook/dataset-export",
>
    "name": "Seattle Rental Data Export",
>
    "format": "csv"
>
}'

Implementing Your Webhook Handler

Your webhook endpoint should handle the incoming POST request and verify the HMAC signature for security. The signature is included in the SHA256-HMAC-Signature header. You’ll use the body of the request and your API key to create a SHA256 HMAC signature. This signature can be compared to the one in the header in order to verify that the request has come from us.

1
const express = require("express");
2
const crypto = require("crypto");
3
const app = express();
4
5
app.use(express.json());
6
7
app.post("/webhook/dataset-export", (req, res) => {
8
  // Get the HMAC signature from the header
9
  const signature = req.get("SHA256-HMAC-Signature");
10
  if (!signature) {
11
    return res.status(401).send("Invalid signature");
12
  }
13
14
  // Verify webhook authenticity
15
  const hmac = crypto.createHmac("sha256", "your-api-key");
16
  hmac.update(JSON.stringify(req.body));
17
  const expectedSignature = hmac.digest("hex");
18
19
  if (
20
    !crypto.timingSafeEqual(
21
      Buffer.from(expectedSignature, "hex"),
22
      Buffer.from(signature, "hex"),
23
    )
24
  ) {
25
    return res.status(401).send("Invalid signature");
26
  }
27
28
  const { queryUUID, gcsBlob } = req.body;
29
30
  // Download and process your data
31
  // processExportedData(gcsBlob.url);
32
33
  console.log(`Export ${queryUUID} completed:`);
34
  console.log(`- Download URL: ${gcsBlob.url}`);
35
  console.log(`- Rows: ${gcsBlob.numberRows}`);
36
  console.log(`- Processing time: ${gcsBlob.timeTakenMs}ms`);
37
38
  res.status(200).send("OK");
39
});
40
41
app.listen(3000, () => { console.log("Server listening on port 3000"); });

Security & Verification

Always verify the HMAC signature to ensure the webhook request is legitimate:

1. Compute HMAC: Use your API key and the payload
2. Compare Signatures: Match against the SHA256-HMAC-Signature header
3. Reject Invalid Requests: Return 401 for signature mismatches

Important Notes

• Processing Time: Large exports may take several minutes to complete
• File Format: Files are compressed with gzip for efficient transfer
• Webhook Requirements: Your endpoint must be HTTPS and publicly accessible
• Response Expected: Your webhook should respond with 2xx status to confirm receipt
• Tracking: Use the queryUUID to match requests with responses

API vs Query Builder: when to use which

Power BI and similar tools: Use Query Builder / dataset export when you need data for a large number of properties (e.g. entire markets or regions). Use the API when you’re tracking a specific, bounded list of properties (portfolio, deal list, comps).

Pricing: The API is per-request; Query Builder and dataset export require a Query Builder subscription (contact sales for details).

Data granularity and limitations

The dataset export endpoint returns the same data as Query Builder, with the same restrictions:

• No unit-level data. You cannot export individual unit rows (e.g. per-unit rent history). Data is at property, unit-type (bed/bath, floorplan), or aggregated level. Unit-level detail is available via the API (property details and history).
• Time dimension: week at finest. For time breakdowns, week is the highest granularity (as_of_week). Month and quarter are also supported. Finer-than-week (e.g. day-level) breakdowns are not available in Query Builder or dataset export.

Error Handling

If the export fails during processing, your webhook will not be called. Monitor your webhook endpoint for delivery failures.

Use Cases

Data Pipeline Integration

1
// Automatically trigger analysis when new data arrives
2
app.post('/webhook/dataset-export', (req, res) => {
3
    const { gcsBlob, name } = req.body;
4
    
5
    // Download the data
6
    const csvData = await downloadFile(gcsBlob.url);
7
    
8
    // Trigger your data pipeline
9
    await triggerDataPipeline(csvData, name);
10
    
11
    res.status(200).send('OK');
12
});

Scheduled Reports

1
# Process weekly market reports
2
@app.route('/webhook/dataset-export', methods=['POST'])
3
def process_weekly_report():
4
    data = request.get_json()
5
    
6
    if 'weekly' in data['name'].lower():
7
        # Send to reporting system
8
        send_to_reporting_dashboard(data['gcsBlob']['url'])
9
    
10
    return 'OK', 200

Full Documentation

Find complete technical details in our API Reference.