Created
December 2, 2016 17:38
-
-
Save looselycoupled/4f23a6ed698b9c62387c596f5613fd6a to your computer and use it in GitHub Desktop.
Bash CSV 'split' Example
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # The following will split a CSV (file.csv) into multiple parts of 1 million lines each | |
| # with each part having its own header. | |
| # | |
| # PREFIX denotes the filename to use for the parts. A number will be added to the end. | |
| tail -n +2 file.csv | | |
| split -d -l 1000000 - --filter='sh -c "{ head -n1 file.csv; cat; } > $FILE"' PREFIX |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment