Transformers

Building TrOCR from scratch: what wiring a vision encoder to a text decoder actually taught me I spent a weekend answering one question end to end: how does a model look at a picture of text and type out the characters? I already knew the NLP Transformer — Q/K/V, multi-head attention, positional encodings — so I didn’t want a tutorial that treats the Vision Transformer as a magic box. I wanted to build the whole thing from scratch in PyTorch: the ViT encoder, the text decoder, and the cross-attention that joins them, aiming eventually at handwritten Devanagari OCR. ...

Attention: Attention is one of the most talked about thing which became popular after the paper Attention is all you need. To talk about the high overview of attention, it is all about the values that gives importance to some of the part than others in this case. We will try our hands on attempt to code self attention from scratch today. Before starting, lets take a step back and understand how attention comes into place at all. ...