Natural Language Processing
Text preprocessing, embeddings, seq2seq, LLMs.
Technical Articles
01
Subword Tokenization & Statistical Language Modeling Theory
Byte-Pair Encoding (BPE), WordPiece likelihood maximization, Unigram EM tokenization, and Kneser-Ney smoothing.
02
Distributed Representation Theory & Dense Word Embeddings
Word2Vec SGNS optimization, PMI factorization proofs, GloVe log-bilinear models, and FastText subword vectors.
03
Recurrent Neural Networks, LSTMs & Seq2Seq Attention Theory
BPTT gradient stability proofs, LSTM Constant Error Carousel (CEC), GRU equations, and Bahdanau attention.
04
Transformer Architecture & Self-Attention Mechanics for NLP
Scaled dot-product attention proofs, RoPE complex rotation algebra, Pre-LN vs Post-LN gradients, and FlashAttention.
05
Parameter-Efficient Fine-Tuning (PEFT) & LoRA Adaptation Theory
LoRA low-rank factorization, QLoRA NF4 quantization proofs, AdaLoRA SVD pruning, and prompt adaptation.