Embedding

The Embedding layer is used to create word embeddings.

Usage

from tinygrad.nn import Embedding
 
class TinyNet: 
  def __init__(self, dim, vocab_size=50257):
    self.wte = Embedding(vocab_size, dim)

The size of the vocabulary. This is the maximum integer index + 1.

The dimension of the dense embedding.

;