Insert Data

To be able to insert into a table, it must have been created with the /create endpoint.

​

Consistency

A request to this endpoint has two possible outcomes: Either all of the data in the request was inserted, or none of it was. It’s not possible for parts of the request data to be inserted, while the rest is not inserted. In other words, please use and trust the status codes that the endpoint returns. A status code of 200 means that the data in the request was successfully inserted. If you get any other status code, you can safely retry your request after addressing the issue that the error message indicated.

​

Concurrent requests

A limited number of concurrent insertion requests per table is supported. However, there will be a slight performance penalty as we serialize the writes behind the scenes to ensure data integrity. Larger number of concurrent requests per table may result in an increased number of failures. Therefore, we recommend managing your requests within a 5-10 threshold to maintain optimal performance.

​

Supported filetypes

​

CSV files (Content-Type: text/csv)

CSV files must use a comma as delimiter, and the column names in the header must match the column names of the table.

​

JSON files (Content-Type: application/x-ndjson)

These are files where each line is a complete JSON object which creates one table row. Each line must have keys that match the column names of the table.

​

Data types

DuneSQL supports a variety of types which are not natively supported in many data exchange formats. Here we provide guidance on how to work with such types.

​

Varbinary values

When uploading varbinary data using JSON or CSV formats, you need to convert the binary data into a textual representation. Reason being, JSON or CSV don’t natively support binary values. There are many ways to transform binary data to a textual representation. We support hexadecimal and base64 encodings.

​

base64

Base64 is a binary-to-text encoding scheme that transforms binary data into a sequence of characters. All characters are taken from a set of 64 characters.

Example: {"varbinary_column":"SGVsbG8gd29ybGQK"}

​

hexadecimal

In the hexadecimal representation input data should contain an even number of characters in the range [0-9a-fA-F] always prefixed with 0x.

Example: {"varbinary_column":"0x92b7d1031988c7af"}