Dune V2 Intro
Dune Engine V2 is Dune's new query engine that brings a new level of performance, scalability and functionality to Dune's core set of tools that enables wizards to query, extract, and visualize the vast amounts of data on the blockchain.
It leverages Apache Spark to enable increased performance of complex queries, handle data scale, and enable cross-chain queries all within the same UI.
All of the data sources contained in this section are available for querying with the new query engine today. Currently we have the following data available to query:
DuneV2 changes our entire database architecture. We are transitioning away from a PostgresQL database to an Instance of Apache Spark hosted on Databricks. The difference between the two systems can be summarized as follows:
- Instead of PostgresQL, we will now use DatabricksSQL. The change in SQL keywords is minimal but might be relevant for some of your querying habits.
- Spark is a column oriented database in contrast to PostgresQL’s row oriented approach.
- traditional indexes are replaced by column chunk level
You can read more about the changes here:
This will make abstractions more robust, scaleable and easier to work with.