Edward Bordin ed-alertedh

Product Engineer/Data Scientist at Alerte Digital Health. Personal account: @edbordin

ed-alertedh / tf_cuda_subprocess.py

Created November 23, 2018 04:38

Decorator to transparently launch Tensorflow code in a subprocess to ensure GPU memory is freed afterwards

	import os
	import cloudpickle
	from multiprocessing import Pool

	class RunAsCUDASubprocess:
	def __init__(self, num_gpus=0, memory_fraction=0.8):
	self._num_gpus = num_gpus
	self._memory_fraction = memory_fraction

	@staticmethod

ed-alertedh / early_stopping.py

Last active December 21, 2018 12:40

Early stopping bugfixes backported for tensorflow 1.10 and 1.11

	# Copyright 2018 The TensorFlow Authors. All Rights Reserved.
	#
	# Licensed under the Apache License, Version 2.0 (the "License");
	# you may not use this file except in compliance with the License.
	# You may obtain a copy of the License at
	#
	# http://www.apache.org/licenses/LICENSE-2.0
	#
	# Unless required by applicable law or agreed to in writing, software
	# distributed under the License is distributed on an "AS IS" BASIS,

ed-alertedh / grab_gpu.py

Created April 5, 2018 07:56

Restrict CUDA device availability from python - adapted from Yaroslavv Bulatav's code

	import os
	import sys
	import py3nvml


	# GPU picking
	# http://stackoverflow.com/a/41638727/419116

	# credit to Yaroslavv Bulatav for the general idea, the error checking and CUDA_DEVICE_ORDER workaround
	# https://github.com/yaroslavvb/stuff/blob/252668efe602da30a34948bfb8b4f89f1ef9d60a/notebook_util.py

ed-alertedh / validate_tfrecords.py

Last active April 16, 2024 18:57

Utility functions to check for corruption in tfrecord files

	import tensorflow as tf


	def validate_dataset(filenames, reader_opts=None):
	"""
	Attempt to iterate over every record in the supplied iterable of TFRecord filenames
	:param filenames: iterable of filenames to read
	:param reader_opts: (optional) tf.python_io.TFRecordOptions to use when constructing the record iterator
	"""
	i = 0