Real data doesn't arrive as one neat tensor. A Dataset says how to fetch one sample; a DataLoader batches and shuffles them for you. Together they're the standard intake pipe that feeds every training loop.
PyTorch splits data handling into two clean jobs. A Dataset answers "how big are you?" (__len__) and "give me sample i" (__getitem__). A DataLoader wraps a dataset and turns it into a batched, shuffled iterable you loop over. Your training loop gains one inner line — for xb, yb in loader: — and otherwise stays exactly the same.
In Lessons 3–4 we shoved all 100 points through at once. That works for 100; it does not work for a million images that don't fit in memory. The fix is mini-batches: process a handful of samples, take a step, repeat.
Only one batch is in memory/on the GPU at a time, so dataset size stops being bounded by RAM.
You take many small steps per pass instead of one big one — usually faster, smoother convergence (this is "mini-batch SGD").
Re-shuffling every epoch stops the model from learning the order of the data instead of the pattern.
optimizer.step()). Batch size is how many samples per step — a key knob you'll see at the top of every script.Dataset — how to get one sampleThe simplest dataset wraps tensors you already have. TensorDataset pairs them up so that indexing returns one (features, target) sample.
from torch.utils.data import TensorDataset, DataLoader, random_split ds = TensorDataset(X, y) # X,y from the earlier lessons print(len(ds)) # 100 ← __len__ xi, yi = ds[0] # ONE sample ← __getitem__ print(xi.shape, yi.shape) # torch.Size([1]) torch.Size([1])
len(ds) and ds[i] can be a dataset. Notice indexing returns a single sample (no batch dimension yet) — adding the batch dimension is the DataLoader's job.DataLoader — batch + shuffle, for freeWrap the dataset in a DataLoader, choose a batch_size, ask it to shuffle, and iterate. It stacks individual samples into a batch tensor with a new leading dimension.
loader = DataLoader(ds, batch_size=16, shuffle=True) print(len(loader)) # 7 ← number of BATCHES (100 / 16, last partial) xb, yb = next(iter(loader)) print(xb.shape, yb.shape) # torch.Size([16, 1]) torch.Size([16, 1])
The only change to Lesson 4's loop is an inner for over the loader. Everything inside is the same five moves — they now run once per batch:
for epoch in range(31): for xb, yb in train_loader: # ← the new line: iterate batches opt.zero_grad() loss = loss_fn(model(xb), yb) loss.backward() opt.step()
epoch 0 | last-batch loss 4.67308 epoch 10 | last-batch loss 0.01068 epoch 30 | last-batch loss 0.00747 learned w=2.002, b=-3.000
Same line recovered, but now via many small batch-steps per epoch. Swap train_loader for one that streams a million images and not a single other line changes.
~/projects/learn/public/courses/pytorch/practice/.venv/bin/python ~/projects/learn/public/courses/pytorch/practice/lesson05_dataloaders.pyYou must measure the model on data it didn't train on, or you can't tell learning from memorizing. random_split carves a dataset into pieces:
train_ds, val_ds = random_split(ds, [80, 20]) # 80 train, 20 validation train_loader = DataLoader(train_ds, batch_size=16, shuffle=True) val_loader = DataLoader(val_ds, batch_size=16) # no shuffle needed for eval
Dataset — the real-data patternFor real data (images on disk, rows of a CSV, tokenized text) you write your own Dataset. It's three methods — and it's the exact shape you'll read in countless repos:
from torch.utils.data import Dataset class SquaresDataset(Dataset): def __init__(self, n): # set up / point at the data source self.x = torch.arange(n, dtype=torch.float32) def __len__(self): # how many samples return len(self.x) def __getitem__(self, i): # fetch + prepare sample i → (input, label) return self.x[i], self.x[i] ** 2 sq = SquaresDataset(5) print(len(sq), sq[3]) # 5 (tensor(3.), tensor(9.))
__getitem__ is where you'd load an image from a path, apply transforms, tokenize a sentence, etc. — returning one ready-to-use sample. Hand that dataset to a DataLoader and batching, shuffling, and (optionally) parallel loading come for free.Source: PyTorch — Datasets & DataLoaders.
Dataset (ds[i]) return?batch_size=16. What is len(loader)?DataLoader(..., batch_size=16)?Dataset implement?__len__, __getitem__). DataLoader = wraps a dataset into a batched, shuffled, iterable stream.optimizer.step().(1,) becomes a batch of shape (batch_size, 1).random_split(ds, [n_train, n_val]), then a DataLoader for each. Shuffle train, not val.Dataset.__getitem__ — load the image/row/text for index i, apply transforms, and return (input, label).Good asks: "what does num_workers do?", "how would a Dataset load images from a folder?", or "why is the last batch smaller and does it matter?" Next we put it all together: a real multi-layer classifier trained on your GPU. Say "go" for Lesson 6.