PyTorch Build and SpeedUp

IMHO,
building PyTorch from source diminished training in a weird but noticeable way ?!
Approximately 12 seconds per epoch for an AlexNet-like with CUDA, and 30 seconds on CPU-only.
For those who might want to explore different avenues regarding this likely improvement, 
here after a step-by-step for both CUDA and CPU-only installs...

PyTorch is a Python package that provides two high-level features:

Tensor computation (like NumPy) with strong GPU acceleration
Deep neural networks built on a tape-based autograd system

• Prerequisites
  • Instantiate a GPU on GCE Nvidia enabled VM
  • Install Nvidia Driver
  • Install Anaconda
• Build From Source
  • With Cuda
  • CPU Only
• Verify installation

Prerequisites

Instantiate a GPU on GCE Nvidia enabled VM

For the previous step we will need a GCP account. It’s free...

  BASE_URL='https://raw.githubusercontent.com/makramjandar/PyTorchBuildAndSpeedUp/master' \
  && URL="$BASE_URL/instantiateVM.sh" \
  && wget -O - -q "${URL}" | bash instantiateVM.sh

Once the VM has been deployed, in case it's not done automatically, we can login into it from Google Cloud Shell:

  gcloud compute --project $PROJECT_ID ssh --zone $ZONE $MACHINE_NAME

Install Nvidia Driver

  BASE_URL='https://raw.githubusercontent.com/makramjandar/PyTorchBuildAndSpeedUp/master' \
  && URL="$BASE_URL/installNvidia.sh" \
  && wget -O - -q "${URL}" | bash installNvidia.sh

Install Anaconda

When building anything, it’s safer to do it in a conda environment to not pollute your system environment.

  BASE_URL='https://raw.githubusercontent.com/makramjandar/PyTorchBuildAndSpeedUp/master' \
  && URL="$BASE_URL/installConda.sh" \
  && wget -O - -q  "${URL}" \
  && . conda.sh

Build From Source

With Cuda

  BASE_URL='https://raw.githubusercontent.com/makramjandar/PyTorchBuildAndSpeedUp/master' \
  && URL="$BASE_URL/buildPyTorch.sh" \
  && wget -O - -q  "${URL}" | bash buildPyTorch.sh cuda

CPU Only

  BASE_URL='https://raw.githubusercontent.com/makramjandar/PyTorchBuildAndSpeedUp/master' \
  && URL="$BASE_URL/buildPyTorch.sh" \
  && wget -O - -q  "${URL}" | bash buildPyTorch.sh

Verify installation

Still under the pytorch-build environment, let’s run some examples to make sure your installation is correct.

Build the torchvision library from source.

cd ~ && git clone https://github.com/pytorch/vision.git && python ~/vision/setup.py install

Install tqdm (a dependency for downloading torchvision datasets) with pip in order to run the MNIST example.

pip install tqdm

Now download the examples and run MNIST:

cd ~ && git clone https://github.com/pytorch/examples.git && python ~/examples/mnist/python/main.py

Voilà !!

License

PyTorch is BSD-style licensed, as found in the LICENSE file.