Qwen3 0.6B From Scratch

A ground-up implementation of Qwen3 0.6B, trained on the fineEDU dataset to explore LLM internals. Built in PyTorch from raw tensors — tokenization, embeddings, rotary positional encodings, grouped-query attention, and the full transformer stack. Trained to completion to validate the architecture and understand every component of a modern decoder-only language model.