Skip to content

Instantly share code, notes, and snippets.

@hang15

hang15/csv_reader.py

Last active August 8, 2020 05:25
Show Gist options
  • Save hang15/573567db826d4eb7a82a0575c5866264 to your computer and use it in GitHub Desktop.
Save hang15/573567db826d4eb7a82a0575c5866264 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
csv_reader.py
MINIMUN_REMAINING_TIME_MS = 10000
csv_reader = csv.DictReader(bodylines.iter_lines(), fieldnames=fieldnames)
for row in csv_reader:
## process records here
## or alternatively forward records to kinesis or SQS to be processed seperately
if context.get_remaining_time_in_millis() < MINIMUN_REMAINING_TIME_MS:
fieldnames = fieldnames or csv_reader.fieldnames
break
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment