lopuhin’s gists

lopuhin / pytorch_high_mem.py

Created October 15, 2019 10:06

pytorch high memory usage on CPU inference with variable input shapes

	import os
	os.environ['OMP_NUM_THREADS'] = '1'
	import argparse
	import resource
	import sys
	import time

	import numpy as np
	import torch
	from torchvision.models import resnet34

lopuhin / mkldnn_489.py

Created June 12, 2019 18:35

Repro script for https://github.com/intel/mkl-dnn/issues/489

	import os
	os.environ['OMP_NUM_THREADS'] = '1'
	import argparse
	import resource
	import sys
	import time

	import numpy as np
	import torch
	from torch import nn

lopuhin / resnet_mkldnn.py

Created June 6, 2019 18:27

Faster ResNet CPU inference with MKLDNN + PyTorch 1.1

	$ pip install torch==1.1.0 torchvision==0.3.0
	$ OMP_NUM_THREADS=1 ipython
	Python 3.6.7 (default, Oct 22 2018, 11:32:17)

	In [1]: import torch
	...: from torchvision.models import resnet50

	In [2]: def forward(m, x):
	...: """ resnet without average pooling """
	...: x = m.conv1(x)

lopuhin / RPython traceback and lldb backtrace.txt

Last active March 3, 2019 15:04

cpyext_tp_new_tuple error: pytorch + pypy3.6

	$ python
	Python 3.6.1 (dab365a46514, Feb 05 2019, 16:36:40)
	[PyPy 7.0.0-alpha0 with GCC 6.2.0 20160901] on linux

	$ lldb python -- -c 'import torch'
	(lldb) target create "python"
	Current executable set to 'python' (x86_64).
	(lldb) settings set -- target.run-args "-c" "import torch"
	(lldb) run
	Process 9293 launched: '/home/kostia/ml/pytorch/pypy3.6-v7.0.0/bin/python' (x86_64)

lopuhin / kaggle-trackml-2018-viz3d.py

Last active March 3, 2020 09:55

Simple OpenGL visualization starter for Kaggle TrackML 2018

	#!/usr/bin/env python3
	"""
	Install glumpy dependencies::

	pip install Cython pyopengl
	brew install glfw freetype # on OS X

	Install glumpy::

	pip install glumpy

lopuhin / groups.csv

Created September 12, 2017 10:43

folds for drivendata fish identification

	video_id	group
	00WK7DR6FyPZ5u3A	0
	01wO3HNwawJYADQw	1
	02p3Yn87z0b5grhL	0
	05RagNzyjVdXeUFR	0
	06aFbpjpYHVBLJZa	2
	09GA0YdSz0NvJUJx	3
	0D5zA3WRD5pXjr8j	4
	0EmM5wsVVNqaKNaM	5
	0L5wBDT4CEazgAw6	6

lopuhin / pympler_heap.py

Created January 31, 2017 13:10

Show python heap with pympler

	from pympler.summary import summarize, print_
	from pympler.muppy import get_object
	all_objects = get_objects()
	summary = summarize(all_objects)
	print_(summary)

lopuhin / sklearn_model_serialization.py

Last active November 25, 2016 13:24

We want to serialize a model that might have complex preprocessing, lambdas, etc., so it is not directly pickleable. Here we add custom pickle support and serialize only model parameters.

	from typing import Dict, Any

	from sklearn.feature_extraction.text import TfidfVectorizer
	from sklearn.linear_model import LogisticRegressionCV
	from sklearn.pipeline import make_pipeline, FeatureUnion


	class BaseModel:
	def __init__(self, **kwargs):
	self._kwargs = kwargs

lopuhin / django downloads 2 vs 3.sql

Created May 23, 2016 21:21

	SELECT
	REGEXP_EXTRACT(details.python, r"^([^\.]+)") as python_version,
	COUNT(*) as total_downloads,
	FROM
	TABLE_DATE_RANGE(
	[the-psf:pypi.downloads],
	TIMESTAMP("20160521"),
	TIMESTAMP("20160524")
	)
	WHERE

lopuhin / vmprof data

Created October 23, 2015 14:14

	!! ['Process:', 'pypy', '[30403]']
	!! ['Path:', '/Users/kostia/apps/netdb_demo_2/pypy/bin/pypy']
	!! ['Load', 'Address:', '0x100000000']
	!! ['Identifier:', 'pypy']
	!! ['Version:', '???']
	!! ['Code', 'Type:', 'X86-64']
	!! ['Parent', 'Process:', 'bash', '[26151]']
	!! ['']
	!! ['Date/Time:', '2015-10-23', '17:05:13.105', '+0300']
	!! ['Launch', 'Time:', '2015-10-23', '17:05:12.611', '+0300']

Konstantin Lopuhin lopuhin