Skip to content

Instantly share code, notes, and snippets.

@jsheedy
Last active February 10, 2026 00:22
Show Gist options
  • Select an option

  • Save jsheedy/ed81cdf18190183b3b7d to your computer and use it in GitHub Desktop.

Select an option

Save jsheedy/ed81cdf18190183b3b7d to your computer and use it in GitHub Desktop.
Sometimes you need a file-like object when all you have is an iterator, for instance when using psycopg2's cursor.copy_from. This class will handle the impedance mismatch.
import io
import sys
class IteratorFile(io.TextIOBase):
""" given an iterator which yields strings,
return a file like object for reading those strings """
def __init__(self, it):
self._it = it
self._f = io.StringIO()
def read(self, length=sys.maxsize):
try:
while self._f.tell() < length:
self._f.write(next(self._it) + "\n")
except StopIteration as e:
# soak up StopIteration. this block is not necessary because
# of finally, but just to be explicit
pass
except Exception as e:
print("uncaught exception: {}".format(e))
finally:
self._f.seek(0)
data = self._f.read(length)
# save the remainder for next read
remainder = self._f.read()
self._f.seek(0)
self._f.truncate(0)
self._f.write(remainder)
return data
def readline(self):
return next(self._it)
@jsheedy

jsheedy commented Mar 7, 2020

Copy link
Copy Markdown
Author

Thanks for asking!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment