Getting Started

tinygrad is a framework that makes it extremely simple to build and train neural networks.

Installation from source

The current recommended way to install tinygrad is from source.

git clone https://github.com/tinygrad/tinygrad.git
cd tinygrad
python3 -m pip install -e .

Don't forget the . at the end!

Installation with pip

pip install tinygrad

NN example

from tinygrad.tensor import Tensor
import tinygrad.nn.optim as optim
 
class TinyBobNet:
  def __init__(self):
    self.l1 = Tensor.uniform(784, 128)
    self.l2 = Tensor.uniform(128, 10)
 
  def forward(self, x):
    return x.dot(self.l1).relu().dot(self.l2).log_softmax()
 
model = TinyBobNet()
optim = optim.SGD([model.l1, model.l2], lr=0.001)
 
# ... complete data loader here
 
out = model.forward(x)
loss = out.mul(y).mean()
optim.zero_grad()
loss.backward()
optim.step()

Imports to get Started

import numpy as np
from tinygrad.helpers import Timing

Tensors

Importing tensor class

from tinygrad.tensor import Tensor

Tensors can be created using an existed data structure

t1 = Tensor([1,2,3,4,5])
na = np.array([1,2,3,4,5])
t2 = Tensor(na)

Tensors can be created with one of the factory methods

full = Tensor.full(shape=(2, 3), fill_value=5) # create a tensor of shape (2, 3) filled with 5
zeros = Tensor.zeros(2, 3) # create a tensor of shape (2, 3) filled with 0
ones = Tensor.ones(2, 3) # create a tensor of shape (2, 3) filled with 1
 
full_like = Tensor.full_like(full, fill_value=2) # create a tensor of the same shape as `full` filled with 2
zeros_like = Tensor.zeros_like(full) # create a tensor of the same shape as `full` filled with 0
ones_like = Tensor.ones_like(full) # create a tensor of the same shape as `full` filled with 1
 
eye = Tensor.eye(3) # create a 3x3 identity matrix
arange = Tensor.arange(start=0, stop=10, step=1) # create a tensor of shape (10,) filled with values from 0 to 9
 
rand = Tensor.rand(2, 3) # create a tensor of shape (2, 3) filled with random values from a uniform distribution
randn = Tensor.randn(2, 3) # create a tensor of shape (2, 3) filled with random values from a normal distribution
uniform = Tensor.uniform(2, 3, low=0, high=10) # create a tensor of shape (2, 3) filled with random values from a uniform distribution between 0 and 10

All the tensors creation methods can take a dtype argument to specify the data type of the tensor.

from tinygrad.helpers import dtypes
 
t3 = Tensor([1, 2, 3, 4, 5], dtype=dtypes.int32)

Tensors allow you to perform operations

t4 = Tensor([1, 2, 3, 4, 5])
t5 = (t4 + 1) * 2
t6 = (t5 * t4).relu().log_softmax()

All these operations are lazy and are only executed when you realize the tensor using .realize() or .numpy().

print(t6.numpy())
# [-56. -48. -36. -20.   0.]

Tinygrad MNIST