Insert Data
Insert the data in a file into a table.
To be able to insert into a table, it must have been created with the /create endpoint.
- The data in the files must conform to the schema, and must use the same column names as the schema.
- One successful
/insertrequest consumes 10 credits. - The maximum request size is 1.2GB
Consistency
A request to this endpoint has two possible outcomes: Either all of the data in the request was inserted, or none of it was. It’s not possible for parts of the request data to be inserted, while the rest is not inserted. In other words, please use and trust the status codes that the endpoint returns. A status code of 200 means that the data in the request was successfully inserted. If you get any other status code, you can safely retry your request after addressing the issue that the error message indicated.
Concurrent requests
A limited number of concurrent insertion requests per table is supported. However, there will be a slight performance penalty as we serialize the writes behind the scenes to ensure data integrity. Larger number of concurrent requests per table may result in an increased number of failures. Therefore, we recommend managing your requests within a 5-10 threshold to maintain optimal performance.
Supported filetypes
CSV files (Content-Type: text/csv)
CSV files must use a comma as delimiter, and the column names in the header must match the column names of the table.
JSON files (Content-Type: application/x-ndjson)
These are files where each line is a complete JSON object which creates one table row. Each line must have keys that match the column names of the table.
Data types
DuneSQL supports a variety of types which are not natively supported in many data exchange formats. Here we provide guidance on how to work with such types.
Varbinary values
When uploading varbinary data using JSON or CSV formats, you need to convert the binary data into a textual representation. Reason being, JSON or CSV don’t natively support binary values. There are many ways to transform binary data to a textual representation. We support hexadecimal and base64 encodings.
base64
Base64 is a binary-to-text encoding scheme that transforms binary data into a sequence of characters. All characters are taken from a set of 64 characters.
Example: {"varbinary_column":"SGVsbG8gd29ybGQK"}
hexadecimal
In the hexadecimal representation input data should contain an even number of characters in the range [0-9a-fA-F] always prefixed with 0x.
Example: {"varbinary_column":"0x92b7d1031988c7af"}
Authorizations
The API key of your team or user.
Path Parameters
The namespace of the table to insert into (e.g. my_user).
The name of the table to insert into (e.g. interest_rates).
Body
The body is of type file.
Was this page helpful?