PGryllos
/ s3_util.py
Created
January 5, 2019 12:04
Using dask's multithreaded scheduler to speedup download of multiple files from s3
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| """ | |
| Using dask's multithreaded scheduler to speedup download of multiple files from | |
| an s3 bucket | |
| """ | |
| import os | |
| from functools import partial | |
| import botocore | |
| import boto3 |