Skip to content

Instantly share code, notes, and snippets.

View ak2k's full-sized avatar

Adam ak2k

  • New York, NY
View GitHub Profile
@ak2k
ak2k / csvkit-sql-cli-readme.md
Created May 7, 2018 22:20 — forked from dannguyen/csvkit-sql-cli-readme.md
Using bash, csvkit, and SQLite to analyze San Francisco restaurant health inspection data

How to download, import, and analyze San Francisco restaurant inspection data using SQLite3 and csvkit from the command-line.

A quick example of doing data wrangling from the command-line, as well as getting to know one of San Francisco's data sets: the San Francisco restaurant inspections, courtesy of the SF Department of Public Health. I don't normally do database work from the command-line, but importing bulk data into SQLite is pretty frustrating using the available GUIs or just the shell.

So thank goodness for Christopher Groskopf's csvkit, a suite of Unix-like tools that use Python to robustly handle CSV files. There's a lot of great tools in csvkit, but for this gist, I just use csvsql, which can parse a CSV and turn it into properly-flavored SQL to pass directly into your database app of choice.