New Feature

Async Dataset Export with Webhook Notifications

  • Introduced /dataset/export endpoint for asynchronous dataset exports with webhook delivery
  • Perfect for large datasets that would timeout with synchronous requests
  • Supports both CSV and JSONL output formats with gzip compression
  • HMAC signature verification ensures webhook authenticity using your API key
  • Download URLs are valid for 30 days from export completion
  • Requires Query Builder subscription for full access

Key Benefits:

  • Process large datasets without timeout constraints
  • Integrate with automated data pipelines and workflows
  • Receive results asynchronously via secure webhook notifications
  • Track exports using unique query UUIDs for request/response matching

Example Request:

1{
2 "dataset": {
3 "scopes": [
4 {"column": "street_address"},
5 {"column": "asking_rent", "aggregate": "Avg"}
6 ],
7 "filters": [
8 {"column": "msa", "filter": {"equals": "Seattle, WA"}},
9 {"column": "bed", "filter": {"in": [1, 2, 3]}}
10 ],
11 "limit": 10000
12 },
13 "webhookURL": "https://yourapp.com/webhook/dataset-export",
14 "name": "Seattle Rental Data Export",
15 "format": "csv"
16}

Webhook Response:

1{
2 "queryUUID": "123e4567-e89b-12d3-a456-426614174000",
3 "name": "Seattle Rental Data Export",
4 "format": "csv",
5 "gcsBlob": {
6 "url": "https://storage.googleapis.com/...",
7 "expires": "2024-07-26T10:30:00Z",
8 "timeTakenMs": 45000,
9 "numberRows": 12847
10 },
11 "hmacSignature": "a1b2c3d4...",
12 "requestedOn": "2024-06-26T10:00:00Z",
13 "completedOn": "2024-06-26T10:00:45Z"
14}

See our Dataset Webhook Export Guide for complete implementation details and webhook handler examples.