Skip to content

Instantly share code, notes, and snippets.

@lobbin
Created May 30, 2022 15:23
Show Gist options
  • Save lobbin/73b50555beffb362b31fd536f476d67d to your computer and use it in GitHub Desktop.
Save lobbin/73b50555beffb362b31fd536f476d67d to your computer and use it in GitHub Desktop.
Export AWS Timestream table to JSON/CSV
Create S3 Bucket
Create Kinesis Stream
Create Kinesis Firehose DeliveryStream
Connect to Kinesis Stream
Deliver to S3 Bucket
Process with timestream-transformer
s3-ftp-uploader (lambda):
events: using event bridge "Object Created" on above bucket
main:
decode bucket and key params from event
data = s3api.getObject
connect sftp
put file(data)
timestream-transformer (lambda):
main:
recordmap = map event.records:
payload = JSON.parse(record.data)
return payload.toCSV()
return recordmap
timestream-producer (lambda):
schedule: rate(60 minutes)
main:
new KinesisProducer
getLastTimestamp from DynamoDB
SELECT * FROM timestreamTable WHERE time > getLastTimestamp
foreach timestreamEntry:
lastTimestamp = timestreamEntry.time
kinesisProducter.putRecord(timestreamEntry)
setLastTimestamp(lastTimestamp) in DynamoDB
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment