ndronen

Here's the output of activate_excess_capacity with my debug statements in it.

The whole log file that follows contains the entire output of the run.

This is after pruning the first task and before training the second task.

Notice that excess capacity goes to 0% even for some layers with only a small %age of pruning.

Notice that after activating excess capacity, the backward grad hook "ERROR" messages start to appear
in the whole log file.

ndronen

"""
This is a straw man proposal to begin discussion of how to change the
PyTorch hooks API to support capture/inspection/modification of
keyword arguments.

    https://github.com/pytorch/pytorch/issues/35643
"""

import unittest

ndronen

/home/ubuntu/conda/miniconda3/envs/distiller-python-3.5/lib/python3.5/importlib/_bootstrap.py:222: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88
  return f(*args, **kwds)
/home/ubuntu/conda/miniconda3/envs/distiller-python-3.5/lib/python3.5/importlib/_bootstrap.py:222: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88
  return f(*args, **kwds)
/home/ubuntu/conda/miniconda3/envs/distiller-python-3.5/lib/python3.5/importlib/_bootstrap.py:222: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88
  return f(*args, **kwds)
/home/ubuntu/conda/miniconda3/envs/distiller-python-3.5/lib/python3.5/importlib/_bootstrap.py:222: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88
  return f(*args, **kwds)
Log file for this run: /home/ubuntu/proj/distiller-python-3.5/examples/pruning_filters_for_efficient_convnets/logs/2018.08.20-10

ndronen

#!/usr/bin/env python
"""
A quick, partial implementation of ENet (https://arxiv.org/abs/1606.02147) using PyTorch.
The original Torch ENet implementation can process a 480x360 image in ~12 ms (on a P2 AWS
instance).  TensorFlow takes ~35 ms.  The PyTorch implementation takes ~25 ms, an improvement
over TensorFlow, but worse than the original Torch.
"""

from __future__ import absolute_import

ndronen

#!/usr/bin/env th

require 'nn';

local cmd = torch.CmdLine()
cmd:text()
cmd:text("What's wrong with this use of nn.Concat?")
cmd:text('Options:')
cmd:option('-noConcat', false, 'do not include concat layer')

ndronen

#!/usr/bin/env th

local cmd = torch.CmdLine()
cmd:text('Demonstration of incompatibility between nn and fbcunn temporal convolutions.')
cmd:text()
cmd:text('Running with -fbconv causes a tensor dimension mismatch error in TemporalConvolutionFB_updateOutput.')
cmd:text()
cmd:text('I think the bug is somewhere in ConvolutionBias.cu.')
cmd:text()
cmd:text('Options:')