6. Dataset and DataLoader

We perfected our Neural Network. Now a complex Neural network will need 1000s of training data so we streamline the feeding pipeline here by making a standard plate (dataset) and a standard spoon to eat (loader)

Dataset

A Dataset is a kind of representation of the training data in LLM for easy access. It has 2 parts. input and target.
eg, dataset[0] → (1, 10) # here input is 1 and target is 10

How do We put any data in dataset format

We simply make tensor list of both inputs and targets (saperate) and we initialize dataset object with these both lists

import torch
from torch.utils.data import TensorDataset

inputs = torch.tensor([1, 2, 3, 4, 5])
targets = torch.tensor([10, 20, 30, 40, 50])

dataset = TensorDataset(inputs, targets)

---

Data Loader

when we serve our training data to our neural network we are currently serving it all at once. This means while training the entire Dataset has to be loaded into the GPU Memory. We add batching for it so that only a portion of the entire dataset is loaded into GPU Memory while training. conversely if your GPU VRAM is getting overburdened just make more number of batches while making each batch smaller.

We Also shuffle so that model doesnt learns any order from the order in which data is stored in the dataset.

How is loader serving our data

Loader recieves 2 sets of data from dataset, one is the inputs set, other is target set.
Lets say we made batches =2 now our inputs are of 2 parts , lets say input 1 and input 2
and similarly targets will be broken in same parts. These are stored like this

loader = [[input_batch1,target_batch1] , [input_batch2,target_batch2]]

Notice each element of the loader is 1 batch = [input,target]
now if we iterate through each batch we can do something like this

for a_batch in loader: #a_batch will be = [input,target] 
	input = a_batch[0]
	target = a_batch[1]

or more common representation

for input, target in loader:

Complete Code with Dataset + Dataloader (runnable)

Code Block

import matplotlib.pyplot as plt
import torch
import torch.nn as nn
from torch.utils.data import TensorDataset, DataLoader


losses = []


def relu(x):
    return torch.relu(x)


class NeuralNetwork(nn.Module):
    def __init__(self):
        super().__init__()
        self.linear1 = nn.Linear(2, 10)
        self.lastlayer1 = nn.Linear(10, 1)
    def forwardpass(self, x):
        output1 = self.linear1(x)
        activated_output1 = relu(output1)
        finaloutput = self.lastlayer1(activated_output1)
        return finaloutput


neuralnetwork = NeuralNetwork()


optimizer = torch.optim.SGD(
    neuralnetwork.parameters(),
    lr=0.0001
)


inputs = torch.tensor([
    [10.0, 20.0],
    [11.0, 21.0],
    [12.0, 22.0],
    [13.0, 23.0],
    [14.0, 24.0],
    [15.0, 25.0],
    [16.0, 26.0],
    [17.0, 27.0],
    [18.0, 28.0],
    [19.0, 29.0]
])


targets = torch.tensor([
    [10.0],
    [11.0],
    [12.0],
    [13.0],
    [14.0],
    [15.0],
    [16.0],
    [17.0],
    [18.0],
    [19.0]
])


dataset = TensorDataset(inputs, targets)


loader = DataLoader(
    dataset,
    batch_size=5,
    shuffle=True
)


loss_function = nn.MSELoss()


for step in range(100):
    for input, target in loader:
        optimizer.zero_grad()
        score = neuralnetwork.forwardpass(input)
        loss = loss_function(score, target)
        loss.backward()
        optimizer.step()
        losses.append(loss.item())
    if step % 10 == 0:
        print(loss)


plt.plot(losses)
plt.xlabel("Step")
plt.ylabel("Loss")
plt.title("Training Loss")
plt.show()

import matplotlib.pyplot as plt import torch import torch.nn as nn from torch.utils.data import TensorDataset, DataLoader losses = [] def relu(x): return torch.relu(x) class NeuralNetwork(nn.Module): def __init__(self): super().__init__() self.linear1 = nn.Linear(2, 10) self.lastlayer1 = nn.Linear(10, 1) def forwardpass(self, x): output1 = self.linear1(x) activated_output1 = relu(output1) finaloutput = self.lastlayer1(activated_output1) return finaloutput neuralnetwork = NeuralNetwork() optimizer = torch.optim.SGD( neuralnetwork.parameters(), lr=0.0001 ) inputs = torch.tensor([ [10.0, 20.0], [11.0, 21.0], [12.0, 22.0], [13.0, 23.0], [14.0, 24.0], [15.0, 25.0], [16.0, 26.0], [17.0, 27.0], [18.0, 28.0], [19.0, 29.0] ]) targets = torch.tensor([ [10.0], [11.0], [12.0], [13.0], [14.0], [15.0], [16.0], [17.0], [18.0], [19.0] ]) dataset = TensorDataset(inputs, targets) loader = DataLoader( dataset, batch_size=5, shuffle=True ) loss_function = nn.MSELoss() for step in range(100): for input, target in loader: optimizer.zero_grad() score = neuralnetwork.forwardpass(input) loss = loss_function(score, target) loss.backward() optimizer.step() losses.append(loss.item()) if step % 10 == 0: print(loss) plt.plot(losses) plt.xlabel("Step") plt.ylabel("Loss") plt.title("Training Loss") plt.show()

Now that you have understood how Neural Network can be made youu can check neural network classification systems - 2 Projects