brennen
/
workings-book
1
0
Fork 0
Code Issues 0 Pull Requests 0 Releases 0 Wiki Activity
a technical notebook
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
175 Commits
1 Branch
1.2 MiB
 
 
 
 
Branch: master
Branches Tags
${ item.name }
Create branch ${ searchTerm }
from 'master'
${ noResults }
workings-book/entries/data_toolchain.md

1.0 KiB
Raw Permalink Blame History

tools & toolchains for data munging & analysis

csvkit

This is super handy. Wish I'd started using it sooner:

csvkit is a suite of utilities for converting to and working with CSV, the king of tabular file formats.

...

csvkit is to tabular data what the standard Unix text processing suite (grep, sed, cut, sort) is to text. As such, csvkit adheres to the Unix philosophy.

  1. Small is beautiful.
  2. Make each program do one thing well.
  3. Build a prototype as soon as possible.
  4. Choose portability over efficiency.
  5. Store data in flat text files.
  6. Use software leverage to your advantage.
  7. Use shell scripts to increase leverage and portability.
  8. Avoid captive user interfaces.
  9. Make every program a filter.

-- csvkit 0.9.0

jq

Also super handy, if a notch less intuitive. Powerful DSL / pretty-printer / filter for working with JSON records at the command line.