mcummins

mcummins / faster_get_multi.py

Last active February 5, 2021 00:06

Fetch a set of keys directly from App Engine memcache, instead of using ndb. Almost 3x faster if everything is in memcache.

	# This Gist is an alternative to ndb.get_multi which is a lot faster in some circumstances.
	# For example, fetching 500 small entities with 90% hit rate is 3x faster than ndb,
	# based on my tests in App Engine production environment (F1).
	# In general it is worthwhile when fetching a large number of small entities, with >50% memcache hit rate.
	# For any keys not in memcache, it falls back on a normal ndb.get_multi

	# The code is based on the original ndb.context code, but avoids the task queue overhead.

	import logging
	from google.appengine.api import memcache