Case · CLI tool · Go
sqlcsv: SQL queries over CSV from the command line
A Go CLI for SQL queries over CSV and TSV. One static binary, standard library only: an own lexer, parser and interpreter. Streaming processing: SELECT, WHERE, GROUP BY, HAVING, ORDER BY, LIMIT, a two-file JOIN (hash join), "did you mean" hints in errors, --explain and output protection from CSV injection (--csv-safe).
- Go
- stdlib only
- SQL
- CSV
- fuzzing
- SQLite (oracle)
- GitHub Actions
01 · How it looks
Screenshots
02 · About
What it is and why
A Go CLI for SQL queries over CSV and TSV. One static binary, standard library only: an own lexer, parser and interpreter. Streaming processing: SELECT, WHERE, GROUP BY, HAVING, ORDER BY, LIMIT, a two-file JOIN (hash join), "did you mean" hints in errors, --explain and output protection from CSV injection (--csv-safe).
Limitations
SELECT only over one or two files; no subqueries, UNION, window functions or CASE.
The right JOIN file is fully in memory; a full sort of 1 million rows takes 1.27 s and 255 MB.
No comparison with other tools was made; it was checked on Windows, CI builds Linux and macOS.
What is inside
- A demo CSV of 1,000,000 rows (39 MB): a GROUP BY aggregate in 0.23 s and 13 MB of memory; an early stop on LIMIT in 0.03 s
- An oracle test: 600 random queries are compared with SQLite (a separate module), numbers to 1e-9
- Fuzzing of the parser and engine: any string as a query does not panic
- Errors with a position and a column-name hint
03 · How to run
Run it locally
git clone <repository on request> sqlcsv
cd sqlcsv
go build -o sqlcsv .
./sqlcsv "SELECT city, COUNT(*) FROM testdata/demo-sales.csv GROUP BY city"04 · Code
The repository is private
The source is in a private repository. Message me on Telegram and I will open access.
Request the code