Table of Contents

  1. Installation
  2. Tensors: The Building Blocks
  3. Automatic Differentiation with autograd
  4. Building a Simple Neural Network
  5. Training the Neural Network
  6. Common Practices and Best Practices
  7. Conclusion
  8. References

1. Installation

Before we start using PyTorch, we need to install it. You can install PyTorch using pip or conda.

Using pip

pip install torch torchvision

Using conda

conda install pytorch torchvision torchaudio cudatoolkit=11.3 -c pytorch

The torchvision library is often used in conjunction with PyTorch as it provides popular datasets, model architectures, and image transformations.

2. Tensors: The Building Blocks

Tensors are the fundamental data structures in PyTorch, similar to NumPy arrays. They can be used on a CPU or GPU for faster computations.

Creating Tensors

import torch

# Create a tensor from a list
tensor_from_list = torch.tensor([1, 2, 3, 4])
print(tensor_from_list)

# Create a tensor of zeros
zeros_tensor = torch.zeros(3, 3)
print(zeros_tensor)

# Create a tensor of random values
random_tensor = torch.rand(2, 2)
print(random_tensor)

Tensor Operations

# Addition
a = torch.tensor([1, 2])
b = torch.tensor([3, 4])
c = a + b
print(c)

# Matrix multiplication
mat1 = torch.rand(2, 3)
mat2 = torch.rand(3, 2)
result = torch.matmul(mat1, mat2)
print(result)

3. Automatic Differentiation with autograd

One of the key features of PyTorch is its automatic differentiation engine, autograd. It allows us to compute gradients automatically.

import torch

# Create a tensor with requires_grad=True to track operations
x = torch.tensor([2.0], requires_grad=True)

# Define a simple function
y = x**2

# Compute gradients
y.backward()

# Print the gradient
print(x.grad)

In the above code, we first create a tensor x with requires_grad=True. Then we define a function y = x**2. By calling y.backward(), PyTorch computes the gradient of y with respect to x and stores it in x.grad.

4. Building a Simple Neural Network

We can build a simple neural network in PyTorch by subclassing torch.nn.Module.

import torch
import torch.nn as nn

class SimpleNet(nn.Module):
    def __init__(self):
        super(SimpleNet, self).__init__()
        self.fc1 = nn.Linear(10, 20)
        self.fc2 = nn.Linear(20, 1)

    def forward(self, x):
        x = torch.relu(self.fc1(x))
        x = self.fc2(x)
        return x

# Create an instance of the network
net = SimpleNet()
print(net)

In the above code, we define a simple neural network with two fully - connected layers. The __init__ method initializes the layers, and the forward method defines the forward pass of the network.

5. Training the Neural Network

To train the neural network, we need to define a loss function and an optimizer.

import torch
import torch.nn as nn
import torch.optim as optim

# Create some dummy data
inputs = torch.randn(5, 10)
labels = torch.randn(5, 1)

# Create an instance of the network
net = SimpleNet()

# Define a loss function and an optimizer
criterion = nn.MSELoss()
optimizer = optim.SGD(net.parameters(), lr=0.01)

# Training loop
for epoch in range(100):
    # Zero the parameter gradients
    optimizer.zero_grad()

    # Forward pass
    outputs = net(inputs)
    loss = criterion(outputs, labels)

    # Backward pass and optimization
    loss.backward()
    optimizer.step()

    if (epoch + 1) % 10 == 0:
        print(f'Epoch {epoch + 1}, Loss: {loss.item()}')

In the training loop, we first zero the gradients of the optimizer to avoid gradient accumulation. Then we perform a forward pass, compute the loss, perform a backward pass to compute the gradients, and finally update the parameters using the optimizer.

6. Common Practices and Best Practices

Data Loading

Use torch.utils.data.DataLoader to load data in batches. It helps in efficient memory usage and parallel data loading.

from torch.utils.data import DataLoader, TensorDataset

# Create a dataset
data = torch.randn(100, 10)
targets = torch.randn(100, 1)
dataset = TensorDataset(data, targets)

# Create a data loader
dataloader = DataLoader(dataset, batch_size=10, shuffle=True)

Model Saving and Loading

Save the model’s state dictionary for future use.

# Save the model
torch.save(net.state_dict(), 'model.pth')

# Load the model
new_net = SimpleNet()
new_net.load_state_dict(torch.load('model.pth'))

GPU Usage

Move the model and data to the GPU for faster computations.

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
net.to(device)
inputs = inputs.to(device)
labels = labels.to(device)

7. Conclusion

In this blog, we have covered the basics of getting started with PyTorch. We learned about tensors, automatic differentiation, building and training neural networks, and some common and best practices. PyTorch provides a flexible and easy - to - use framework for deep learning. With the knowledge gained from this guide, you can start exploring more advanced topics in deep learning, such as convolutional neural networks, recurrent neural networks, and more.

8. References