What is a Query Pipeline?
A query pipeline enables you to create a chain of dependencies between queries, where:- Each query in the pipeline can depend on the results of previous queries
- The pipeline executes queries in the correct order based on their dependencies
- All queries are executed as a single atomic operation
Use Cases
Query pipelines are ideal for:- Complex Data Transformations: Breaking down complex analysis into smaller, manageable queries
- Data Lineage: Maintaining clear dependencies between related queries
- Coordinated Updates: Ensuring multiple queries are executed together
- Incremental Processing: Building data pipelines where each step depends on the previous one
Response
The endpoint returns apipeline_execution_id which you can use to:
- Check the status of the pipeline execution using the Get Pipeline Execution Status endpoint
- Monitor the progress of individual nodes in the pipeline
- Retrieve results from each query in the pipeline once execution completes
- Credits are consumed based on actual compute resources used for each query in the pipeline
- Use the
performanceparameter to specify execution tier (medium or large) - Query parameters can be passed to parameterize queries in the pipeline
Example Workflow
Related Endpoints
- Get Query Pipeline - Inspect pipeline structure before execution
- Get Pipeline Status - Check pipeline execution status
- Execute Pipeline - Execute a lineage-based pipeline
Headers
API Key for the service
Path Parameters
Unique identifier of the query
Query Parameters
Alternative to using the X-Dune-Api-Key header
Body
application/json
Query pipeline execution request
Response
OK
Unique identifier for the pipeline execution. Use this ID to check the status and retrieve results of the pipeline execution.
Example:
"01HKZJ2683PHF9Q9PHHQ8FW4Q1"