A tensor is just a typed, n-dimensional array that knows its shape, its dtype, and which device it lives on. Learn to read those three things and you can read the first lines of almost any PyTorch file.
🎯 Mission: read & write real PyTorchEvery PyTorch program — a 5-line script or a 70-billion-parameter language model — is just tensors flowing through operations. The model's inputs, its weights, its outputs, and (next lesson) its gradients are all tensors. Master the tensor and the rest of the course is variations on a theme.
Your mission is to read and write real PyTorch. Open any PyTorch file — a research repo, a tutorial, production model code — and the very first things you meet are tensors being created, reshaped, moved to a GPU, and multiplied together. Nothing else makes sense until this does.
You're new to neural nets, and that's fine: we don't need any of that yet. A tensor is a data-structure idea you already understand from arrays — PyTorch just adds three properties you'll find yourself checking constantly. Get fluent reading those three and the shape-mismatch errors that trip up every beginner stop being mysterious.
x: x.shape (its dimensions), x.dtype (its element type), and x.device (where it lives — CPU or GPU). When PyTorch code confuses you, print those three first. See the glossary for tight definitions.I've already created a practice environment on your Mac so every example runs locally — including on your Apple-Silicon GPU. It's a Python virtual environment (an isolated package sandbox) with torch + numpy installed.
# the practice env (already set up for you): ~/projects/learn/public/courses/pytorch/practice/.venv # run any lesson script with its python directly — no "activate" needed: ~/projects/learn/public/courses/pytorch/practice/.venv/bin/python ~/projects/learn/public/courses/pytorch/practice/lesson01_tensors.py
~/projects/learn/public/courses/pytorch/practice/.venv/bin/python and paste line by line.First, confirm the install and that PyTorch can see your GPU. On Apple Silicon the GPU backend is called MPS (Metal Performance Shaders):
import torch print(torch.__version__) # 2.12.1 print(torch.backends.mps.is_available()) # True ← your GPU is usable
# verified on your machine just now: torch 2.12.1 mps available: True
~/projects/learn/public/courses/pytorch/practice/.venv/bin/python ~/projects/learn/public/courses/pytorch/practice/lesson01_tensors.py. When you see mps:0 in the output, your Apple GPU just did real tensor math. That's lesson 1 done — everything below explains what you saw.Source: Apple — Accelerated PyTorch on Mac · PyTorch MPS backend notes.
The simplest way to make a tensor is from a Python list. A tensor can have any number of dimensions; the number of dimensions is its rank.
x = torch.tensor([[1., 2., 3.], [4., 5., 6.]]) print(x.shape) # torch.Size([2, 3]) → 2 rows, 3 columns print(x.ndim) # 2 → it's a matrix print(x.dtype) # torch.float32 → default for floats print(x.device) # cpu → where it lives right now
1. (with the dot) makes a float32 tensor; writing 1 makes an int64 one. Neural-net math is float math, so models expect float32. A surprising number of beginner errors are really "I accidentally made an int tensor."Source: PyTorch — Tensors tutorial · torch.Tensor reference.
Two operation styles show up constantly, and confusing them is a classic bug.
x + 10 # add 10 to every element x * x # multiply element-by-element (NOT matrix multiply!)
# x * x = tensor([[ 1., 4., 9.], [16., 25., 36.]])
@ operatorThe workhorse of neural nets. Inner dimensions must agree: (2,3) @ (3,1) → (2,1).
w = torch.tensor([[1.], [0.], [-1.]]) # shape (3, 1) x @ w # (2,3) @ (3,1) → (2,1) # tensor([[-2.], # [-2.]]) each row of x dotted with w
sum, mean, max… With no argument they reduce the whole tensor to a scalar. The key skill is dim=: it names the axis you collapse.
x.sum() # tensor(21.) everything → one number x.mean() # tensor(3.5000) x.sum(dim=0) # tensor([5., 7., 9.]) collapse ROWS → one per column x.sum(dim=1) # tensor([6., 15.]) collapse COLS → one per row
dim= as "the axis that disappears"
x has shape (2, 3). dim=0 removes the size-2 axis → result shape (3,). dim=1 removes the size-3 axis → result shape (2,). Once you see dim as "which axis collapses," reductions stop being guesswork. Shapes that don't line up for @ or get broadcast automatically are the #1 source of real PyTorch errors.Source: torch.Tensor reference · Broadcasting semantics.
You constantly need to rearrange a tensor's shape (e.g. flattening an image into a vector before a layer). view reinterprets the same data under a new shape; pass -1 to let PyTorch infer one dimension.
x.view(3, 2) # same 6 numbers, now 3 rows × 2 cols # tensor([[1., 2.], # [3., 4.], # [5., 6.]]) x.view(-1) # flatten to 1-D: tensor([1., 2., 3., 4., 5., 6.])
view vs reshape
They look identical and usually behave the same. view never copies (it requires the data to be contiguous in memory) and errors if it can't; reshape copies only when it has to. When in doubt, reshape is the safe default; you'll see view all over real code.A tensor lives on a device. By default that's the cpu. To use your Apple GPU, move it to mps. The idiomatic, portable pattern — the exact line you'll see at the top of real training scripts — is:
device = "mps" if torch.backends.mps.is_available() else "cpu" xg = x.to(device) print(xg.device) # mps:0 ← now living on the Apple GPU
Writing device this way means the same script runs on your Mac (mps), a CUDA box (swap in "cuda"), or anywhere (cpu) — which is exactly why real repos write it like this.
cpu tensor with an mps one throws a runtime error — you'll hit this the moment you build a model, and now you'll know why. Also: not every op is implemented on MPS yet; set PYTORCH_ENABLE_MPS_FALLBACK=1 to fall back to CPU for the gaps.Source: PyTorch MPS notes · Apple Metal + PyTorch.
You'll constantly see PyTorch code hop to NumPy (the standard Python array library) and back — for loading data, plotting, or metrics. The bridge is two methods:
import numpy as np a = x.numpy() # tensor → ndarray (CPU only) t2 = torch.from_numpy(np.array([10, 20, 30])) # ndarray → tensor print(t2, t2.dtype) # tensor([10, 20, 30]) torch.int64
.numpy() and from_numpy() give you a view of the same memory — mutate one and the other changes. And a GPU (mps) tensor must be moved to CPU first (x.cpu().numpy()), since NumPy can't read GPU memory.Source: PyTorch — Bridge with NumPy.
x has shape (2, 3). What does x.sum(dim=0) return?x * x and x @ w?cpu tensor to an mps tensor. What happens?x has 6 elements. What does x.view(-1) give you?float32), and device (cpu/mps/cuda). Print all three when PyTorch confuses you.* vs @ on tensors?* is elementwise multiply (shapes match or broadcast). @ is matrix multiply — inner dims must agree, (m,k)@(k,n)→(m,n).dim= select?(2,3), dim=0 → shape (3,); dim=1 → shape (2,).device = "mps" if torch.backends.mps.is_available() else "cpu", then x.to(device). Same script runs on Mac GPU, CUDA, or CPU.x.view(-1) do?-1 tells PyTorch to infer that dimension's size from the total element count. view reinterprets without copying (needs contiguous data).x.numpy() and torch.from_numpy(a). On CPU they share memory; a GPU tensor must be moved with x.cpu().numpy() first.torch.tensor([1, 2]) sometimes cause model errors?int64 tensor. Models do float math and expect float32. Write [1., 2.] (or call .float()).This is a conversation, not a lecture. Good things to ask me right now: "walk me through a shape error from a real repo," "show me broadcasting with a concrete example," or "why is matmul the heart of a neural net?" Coming next — Lesson 2: Autograd, the one idea that turns these tensors into something that can learn. When you've run the script and seen mps:0, tell me and we'll go.