What is a Query Pipeline?
A query pipeline enables you to create a chain of dependencies between queries, where:- Each query in the pipeline can depend on the results of previous queries
- The pipeline executes queries in the correct order based on their dependencies
- All queries are executed as a single atomic operation
Use Cases
Query pipelines are ideal for:- Complex Data Transformations: Breaking down complex analysis into smaller, manageable queries
- Data Lineage: Maintaining clear dependencies between related queries
- Coordinated Updates: Ensuring multiple queries are executed together
- Incremental Processing: Building data pipelines where each step depends on the previous one
Response
The endpoint returns apipeline_execution_id which you can use to:
- Check the status of the pipeline execution using the Get Pipeline Execution Status endpoint
- Monitor the progress of individual nodes in the pipeline
- Retrieve results from each query in the pipeline once execution completes
- Credits are consumed based on actual compute resources used for each query in the pipeline
- Use the
performanceparameter to specify execution tier (medium or large) - Query parameters can be passed to parameterize queries in the pipeline
Example Workflow
Headers
API Key for the service
Path Parameters
Unique identifier of the query
Query Parameters
Alternative to using the X-Dune-Api-Key header
Body
application/json
Query pipeline execution request
Response
OK
Unique identifier for the pipeline execution. Use this ID to check the status and retrieve results of the pipeline execution.
Example:
"01HKZJ2683PHF9Q9PHHQ8FW4Q1"