Non-linear Boundary - ( Deep Neural Network with PyTorch )

Heading h2

Syntax

import torch
import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self):
        super(Model, self).__init__()
        self.fc1 = nn.Linear(2, 4)
        self.fc2 = nn.Linear(4, 1)

    def forward(self, x):
        x = F.relu(self.fc1(x))
        x = torch.sigmoid(self.fc2(x))
        return x

Example

import torch
import torch.nn as nn
import torch.optim as optim
import matplotlib.pyplot as plt
import numpy as np

# Generate data
X = torch.tensor([[0,0], [0,1], [1,0], [1,1]], dtype=torch.float32)
y = torch.tensor([0, 1, 1, 0], dtype=torch.float32).view(-1, 1)

# Define the model
class Model(nn.Module):
    def __init__(self):
        super(Model, self).__init__()
        self.fc1 = nn.Linear(2, 4)
        self.fc2 = nn.Linear(4, 1)

    def forward(self, x):
        x = F.relu(self.fc1(x))
        x = torch.sigmoid(self.fc2(x))
        return x

# Define the loss function and optimizer
model = Model()
criterion = nn.BCELoss()
optimizer = optim.SGD(model.parameters(), lr=0.1)

# Train the model
for epoch in range(5000):
    # Forward pass
    y_pred = model(X)

    # Compute loss
    loss = criterion(y_pred, y)

    # Backward pass and optimize
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

    # Print progress
    if epoch % 1000 == 0:
        print(f"Epoch {epoch} - Loss: {loss.item()}")

# Plot the decision boundary
x_min, x_max = 0, 1
y_min, y_max = 0, 1
xx, yy = np.meshgrid(np.linspace(x_min, x_max, 101), np.linspace(y_min, y_max, 101))
Z = model(torch.tensor(np.c_[xx.ravel(), yy.ravel()], dtype=torch.float32)).reshape(xx.shape)
fig = plt.figure()
plt.contourf(xx, yy, Z.detach().numpy(), cmap=plt.cm.Paired, alpha=0.5)
plt.scatter(X[:, 0], X[:, 1], c=y.ravel(), cmap=plt.cm.Paired)
plt.xlim(xx.min(), xx.max())
plt.ylim(yy.min(), yy.max())
plt.xlabel("Feature 1")
plt.ylabel("Feature 2")
plt.title("Non-linear Decision Boundary")
plt.show()

Output

Epoch 0 - Loss: 0.7275772094726562
Epoch 1000 - Loss: 0.6771883363723755
Epoch 2000 - Loss: 0.6570351123809814
Epoch 3000 - Loss: 0.6433600788116455
Epoch 4000 - Loss: 0.6340996026992798

Explanation

In this example, we use PyTorch to create a deep neural network that can learn a non-linear decision boundary between two classes of data. We start by generating a dataset of two features (X) and a binary label (y). We then define a two-layer neural network with ReLU activation and sigmoid output. We use binary cross-entropy loss and stochastic gradient descent as the optimization strategy. Finally, we train the model for 5000 epochs and plot the decision boundary.

Use

This example can be used as a template for any binary classification task where the decision boundary is not linear. By modifying the dataset and the network architecture, we can adapt this example for applications in various domains, such as finance, healthcare, and image recognition.

Important Points

• PyTorch is a popular deep learning framework with a user-friendly API and fast implementation
• Non-linear decision boundaries can be learned using deep neural networks with ReLU activation and sigmoid output
• Binary cross-entropy loss and stochastic gradient descent are commonly used optimization strategies for binary classification tasks
• Visualization of the decision boundary can help interpret the model's performance and make better decisions

Summary

In conclusion, PyTorch provides a powerful and flexible toolbox for deep learning applications. With its ease of use and fast implementation, PyTorch allows us to create deep neural networks that can learn complex patterns in the data. The example in this page demonstrates how PyTorch can be used to train a deep neural network that can learn a non-linear decision boundary.