Skip to content

Instantly share code, notes, and snippets.

@zazazack

zazazack/loadlines.py

Created March 17, 2018 15:57
Show Gist options
  • Save zazazack/2753e07d6a35b9f97d7b55e339d2f253 to your computer and use it in GitHub Desktop.
Save zazazack/2753e07d6a35b9f97d7b55e339d2f253 to your computer and use it in GitHub Desktop.
Download ZIP
Load JSON from multiple `.json` files stored in the JSON Lines format with Python 3.6 builtins
Raw
loadlines.py
from pathlib import Path
import json
p = Path() # assuming .json files are located in the current working directory
data = [json.loads(line) for file in p.glob('*.json') for line in file.read_text().splitlines()]
@zazazack
Copy link
Author

zazazack commented Mar 17, 2018 •
edited
Loading

I developed this method to import JSON Lines formatted data generated by a Scrapy application I built.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment