csvkit: a toolkit for working with CSV files in the terminal

Wed, 07 Oct 2026 00:00:00 +0000

A CSV file arrives. Before loading it into a database, you want to see its columns, check for missing values, and find the records you need. Opening a spreadsheet works, but becomes inconvenient when you have to repeat the same operation across several files. Writing a program for every check is not always worthwhile either.

csvkit brings together command line tools for converting and processing tabular data. It is written in Python and handles many of these tasks with short commands you can save and run again.

CSV has more structure than it seems

Splitting a line on commas might seem sufficient until a value such as "Madrid, downtown" appears. Fields may also contain escaped quotes or line breaks. That is why using cut, grep, or sort directly on the text requires care: those tools do not interpret CSV structure on their own.

csvkit works with rows and columns. You can select a column by name, filter a particular field, and connect operations with |. Transformation commands still output CSV; csvlook, however, produces a table for reading on screen. The official tutorial explains this use of standard input and output.

Installation

The documentation recommends a virtual environment. On Linux or macOS:

python3 -m venv .venv
source .venv/bin/activate
python -m pip install csvkit
csvcut --version

In PowerShell, activate it with .venv\Scripts\Activate.ps1. Installation provides several executables, such as csvcut and csvstat; there is no single command named csvkit to invoke.

A reproducible example

Save this UTF-8 content as sales.csv:

id,customer,category,units,price,status
1,"North Bookshop",books,2,18.50,paid
2,"Cafe, Central",home,1,32.00,pending
3,"North Bookshop",books,3,12.00,paid
4,"South Studio",home,2,25.00,paid

The comma in "Cafe, Central" belongs to the customer name; it does not separate two columns.

Inspect before transforming

csvcut -n sales.csv
csvlook sales.csv
csvstat sales.csv
csvstat --count sales.csv

csvcut -n lists columns and csvlook makes the data easier to read. csvstat summarizes inferred types, null values, and statistics appropriate to each column type. The final command returns 4: it counts records, not physical file lines. This matters when fields contain line breaks.

Select columns and filter rows

csvgrep -c status -r '^paid$' sales.csv \
 | csvcut -c customer,category,units,price \
 > paid_sales.csv

csvsort -c price -r paid_sales.csv | csvlook

csvgrep applies the regular expression to the status column. The ^ and $ anchors match the entire field. We then select four columns and save the result. csvsort sorts by price from highest to lowest; here prices are interpreted as numbers.

A pipe connects one command’s output to the next command’s input. > writes that output to a file, overwriting its contents if it already exists. Use a different name from the input file to avoid truncating it before it is read.

Aggregate with SQL

To calculate paid sales revenue by category:

csvsql --query "
 SELECT category,
 SUM(units * price) AS revenue
 FROM sales
 WHERE status = 'paid'
 GROUP BY category
 ORDER BY revenue DESC
" sales.csv

The result totals 73 for books and 50 for home; decimal formatting may vary. By default, the table name comes from the filename without its extension: sales.csv becomes sales.

csvsql loads the data into an in-memory SQLite database to run this query. It can also generate SQL statements or import data into a database. For monetary calculations requiring exactness, define a precision strategy: SQL execution can introduce floating point effects.

Join files and change formats

Suppose customers.csv contains:

customer,city
"North Bookshop",Bilbao
"Cafe, Central",Madrid
"South Studio",Seville

We can add the city to each sale:

csvjoin --left -c customer sales.csv customers.csv > sales_with_city.csv

csvjoin combines tables using a key. --left keeps every row in the first file, even without a match in the second. If a key appears several times in both files, the join can multiply rows: check that this relationship is the one you need.

For other inputs and outputs:

in2csv sales.xlsx > sales_from_excel.csv
csvjson sales.csv > sales.json
csvformat -D ';' sales.csv > sales_semicolon.csv

in2csv converts an Excel worksheet into tabular data; it does not preserve its presentation. csvjson produces JSON and csvformat changes the output delimiter. Notice the difference: -d specifies the input separator, while -D in csvformat specifies the output separator.

Delimiters, encodings, and types

For a semicolon-delimited file encoded in Windows-1252:

csvlook -d ';' -e cp1252 -y 0 -I export.csv

-y 0 disables automatic dialect detection. -I disables type inference in commands that support it, such as csvlook, csvstat, and csvsql. This is useful for keeping identifiers such as 00123 as text. The common arguments document encoding and format options; also check the individual command’s help.

Think about the next operation before disabling inference: a price treated as text may sort differently from a number. Successfully reading a file does not establish that its data is valid; business rules still need their own checks.

Where it fits

csvkit is convenient for exploring an export, preparing a load, or automating small, repeatable transformations. You can save the commands in a script and review exactly which filters you applied.

As volume or complexity grows, consider an engine such as DuckDB or a library such as Polars. Some csvkit operations need to hold data in memory, and each pipeline command may parse it again. Its main appeal is convenience: turning an unfamiliar CSV into a clear transformation with a handful of instructions.

Csvkit on josiete.com