7. Neural Network Ex.1 Binary Classification

Binary classification using neural network is essentially same as a logistic Regression in Machine Learning, That is we will be somehow able to teach our computer to choose between 2 choices. However there is one improvement, Logistic Regression is Made only for Binary Classification, But Neural Network can be made to choose between 3,4 and even more choices (In next chapter - Multi Class Classification)

The Project Structure :

The Sigmoid Function

To convert the raw logit $z$ into a probability (a number between 0 and 1), we pass it through the Sigmoid activation function:

$$\sigma(z) = \frac{1}{1 + e^{-z}}$$

A common decision rule to translate this probability into a final class is:

probability >= 0.5 → Class 1
probability < 0.5  → Class 0

Binary Cross-Entropy (BCE) Loss

To train our model, we need to penalize it when its predicted probability diverges from the true label. Binary Cross-Entropy (BCE) loss does exactly this. It heavily penalizes the model if it is confidently wrong (e.g., predicting 0.99 for a class that is actually 0).

In PyTorch, you could technically apply nn.Sigmoid() and then nn.BCELoss(). However, for numerical stability, it is standard practice to combine them into a single step:

loss_function = torch.nn.BCEWithLogitsLoss()

This function takes the raw logits ($z$) and the target labels directly, applying the sigmoid mathematically under the hood in a way that avoids floating-point precision issues.

Complete Runnable Example

Let's build a simple PyTorch binary classifier. We'll train a small network to classify some dummy 2D data points.

Code Block

import torch
import torch.nn as nn
from torch.utils.data import TensorDataset, DataLoader
import matplotlib.pyplot as plt


# 1. Create dummy data (100 samples, 2 features each)
X_class0 = torch.randn(50, 2) - 2.0
X_class1 = torch.randn(50, 2) + 2.0
inputs = torch.cat([X_class0, X_class1], dim=0)


# Labels: 50 zeros, then 50 ones. Need to be float for BCEWithLogitsLoss
targets = torch.cat([torch.zeros(50, 1), torch.ones(50, 1)], dim=0)


# 2. Setup Dataset and DataLoader
dataset = TensorDataset(inputs, targets)
loader = DataLoader(
    dataset,
    batch_size=10,
    shuffle=True
)


# 3. Define the Binary Classifier using previous architecture
class BinaryClassifier(nn.Module):
    def __init__(self):
        super().__init__()
        self.layer = nn.Linear(2, 1) # 2 inputs, 1 logit output
        
    def forwardpass(self, x):
        return self.layer(x) # Return raw logit!


neuralnetwork = BinaryClassifier()
loss_function = nn.BCEWithLogitsLoss()
optimizer = torch.optim.SGD(
    neuralnetwork.parameters(), 
    lr=0.1
)


# 4. Training Loop
losses = []
for step in range(50):
    for input, target in loader:
        optimizer.zero_grad()
        
        # Forward pass
        score = neuralnetwork.forwardpass(input)
        loss = loss_function(score, target)
        
        # Backward pass and optimize
        loss.backward()
        optimizer.step()
        
        losses.append(loss.item())


# Plot the training loss
plt.plot(losses, color='blue', label='BCE Loss')
plt.xlabel("Step")
plt.ylabel("Loss")
plt.title("Binary Classification Training Loss")
plt.legend()
plt.show()


# Let's test it on a new point!
test_point = torch.tensor([[2.5, 2.5]])
test_logit = neuralnetwork.forwardpass(test_point)
test_prob = torch.sigmoid(test_logit)


print(f"Raw Logit: {test_logit.item():.4f}")
print(f"Probability of Class 1: {test_prob.item():.4f}")
print(f"Prediction: {'Class 1' if test_prob.item() >= 0.5 else 'Class 0'}")

import torch import torch.nn as nn from torch.utils.data import TensorDataset, DataLoader import matplotlib.pyplot as plt # 1. Create dummy data (100 samples, 2 features each) X_class0 = torch.randn(50, 2) - 2.0 X_class1 = torch.randn(50, 2) + 2.0 inputs = torch.cat([X_class0, X_class1], dim=0) # Labels: 50 zeros, then 50 ones. Need to be float for BCEWithLogitsLoss targets = torch.cat([torch.zeros(50, 1), torch.ones(50, 1)], dim=0) # 2. Setup Dataset and DataLoader dataset = TensorDataset(inputs, targets) loader = DataLoader( dataset, batch_size=10, shuffle=True ) # 3. Define the Binary Classifier using previous architecture class BinaryClassifier(nn.Module): def __init__(self): super().__init__() self.layer = nn.Linear(2, 1) # 2 inputs, 1 logit output def forwardpass(self, x): return self.layer(x) # Return raw logit! neuralnetwork = BinaryClassifier() loss_function = nn.BCEWithLogitsLoss() optimizer = torch.optim.SGD( neuralnetwork.parameters(), lr=0.1 ) # 4. Training Loop losses = [] for step in range(50): for input, target in loader: optimizer.zero_grad() # Forward pass score = neuralnetwork.forwardpass(input) loss = loss_function(score, target) # Backward pass and optimize loss.backward() optimizer.step() losses.append(loss.item()) # Plot the training loss plt.plot(losses, color='blue', label='BCE Loss') plt.xlabel("Step") plt.ylabel("Loss") plt.title("Binary Classification Training Loss") plt.legend() plt.show() # Let's test it on a new point! test_point = torch.tensor([[2.5, 2.5]]) test_logit = neuralnetwork.forwardpass(test_point) test_prob = torch.sigmoid(test_logit) print(f"Raw Logit: {test_logit.item():.4f}") print(f"Probability of Class 1: {test_prob.item():.4f}") print(f"Prediction: {'Class 1' if test_prob.item() >= 0.5 else 'Class 0'}")