Contents

• TOC {:toc}

구현

Llama

• Multi-head attention
• SwiGLU
• RoPE
• RMSNorm

참조 : Shakespeare 학습 llama 구현 코드 (pytorch)

Head 1

Head 2

Head 2

Head 2