| 2013 | Mikolov et al. Efficient Estimation of Word Representations in Vector Space (arXiv:1301.3781) | 5 |
| 2014 | Bahdanau, Cho & Bengio. Neural Machine Translation by Jointly Learning to Align and Translate (arXiv:1409.0473) | 6 |
| 2014 | Goodfellow, Shlens & Szegedy. Explaining and Harnessing Adversarial Examples (arXiv:1412.6572) | 20 |
| 2014 | Kingma & Ba. Adam: A Method for Stochastic Optimization (arXiv:1412.6980) | 3 |
| 2015 | He et al. Deep Residual Learning for Image Recognition (arXiv:1512.03385) | 3 |
| 2015 | Sennrich, Haddow & Birch. Neural Machine Translation of Rare Words with Subword Units (arXiv:1508.07909) | 5 |
| 2016 | Abadi et al. Deep Learning with Differential Privacy (arXiv:1607.00133) | 20 |
| 2016 | Ba, Kiros & Hinton. Layer Normalization (arXiv:1607.06450) | 7 |
| 2016 | Goodfellow, Bengio & Courville. Deep Learning (book) | 1, 2, 3, 4 |
| 2016 | Hendrycks & Gimpel. Gaussian Error Linear Units (GELUs) (arXiv:1606.08415) | 2 |
| 2016 | Tramèr et al. Stealing Machine Learning Models via Prediction APIs (arXiv:1609.02943) | 20 |
| 2017 | Christiano et al. Deep reinforcement learning from human preferences (arXiv:1706.03741) | 17 |
| 2017 | Gu, Dolan-Gavitt & Garg. BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain (arXiv:1708.06733) | 20 |
| 2017 | Loshchilov & Hutter. Decoupled Weight Decay Regularization (arXiv:1711.05101) | 3 |
| 2017 | Madry et al. Towards Deep Learning Models Resistant to Adversarial Attacks (arXiv:1706.06083) | 20 |
| 2017 | Micikevicius et al. Mixed Precision Training (arXiv:1710.03740) | 15 |
| 2017 | Ramachandran et al. Searching for Activation Functions (arXiv:1710.05941) | 2 |
| 2017 | Schulman et al. Proximal Policy Optimization Algorithms (arXiv:1707.06347) | 17 |
| 2017 | Shazeer et al. Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer (arXiv:1701.06538) | 14 |
| 2017 | Vaswani et al. Attention Is All You Need (arXiv:1706.03762) | 6, 7, 8, 9 |
| 2018 | Fan, Lewis & Dauphin. Hierarchical Neural Story Generation (arXiv:1805.04833) | 4 |
| 2018 | Huang et al. GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism (arXiv:1811.06965) | 16 |
| 2018 | Kudo & Richardson. SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing (arXiv:1808.06226) | 5 |
| 2018 | Milakov & Gimelshein. Online normalizer calculation for softmax (arXiv:1805.02867) | 12 |
| 2019 | Carlini et al. On Evaluating Adversarial Robustness (arXiv:1902.06705) | 21 |
| 2019 | Holtzman et al. The Curious Case of Neural Text Degeneration (arXiv:1904.09751) | 4 |
| 2019 | Rajbhandari et al. ZeRO: Memory Optimizations Toward Training Trillion Parameter Models (arXiv:1910.02054) | 16 |
| 2019 | Shazeer. Fast Transformer Decoding: One Write-Head is All You Need (arXiv:1911.02150) | 10, 11 |
| 2019 | Shoeybi et al. Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism (arXiv:1909.08053) | 16 |
| 2019 | Zhang & Sennrich. Root Mean Square Layer Normalization (arXiv:1910.07467) | 7 |
| 2020 | Beltagy, Peters & Cohan. Longformer: The Long-Document Transformer (arXiv:2004.05150) | 13 |
| 2020 | Brown et al. Language Models are Few-Shot Learners (arXiv:2005.14165) | 9, 19 |
| 2020 | Carlini et al. Extracting Training Data from Large Language Models (arXiv:2012.07805) | 20 |
| 2020 | Geva et al. Transformer Feed-Forward Layers Are Key-Value Memories (arXiv:2012.14913) | 7 |
| 2020 | Hendrycks et al. Measuring Massive Multitask Language Understanding (arXiv:2009.03300) | 21 |
| 2020 | Kaplan et al. Scaling Laws for Neural Language Models (arXiv:2001.08361) | 16 |
| 2020 | Katharopoulos et al. Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention (arXiv:2006.16236) | 13 |
| 2020 | Lewis et al. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (arXiv:2005.11401) | 19 |
| 2020 | Shazeer. GLU Variants Improve Transformer (arXiv:2002.05202) | 7 |
| 2020 | Stiennon et al. Learning to summarize from human feedback (arXiv:2009.01325) | 17 |
| 2020 | Xiong et al. On Layer Normalization in the Transformer Architecture (arXiv:2002.04745) | 7 |
| 2021 | Carlini et al. Membership Inference Attacks From First Principles (arXiv:2112.03570) | 20 |
| 2021 | Chen et al. Evaluating Large Language Models Trained on Code (arXiv:2107.03374) | 21 |
| 2021 | Fedus, Zoph & Shazeer. Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity (arXiv:2101.03961) | 14 |
| 2021 | Hu et al. LoRA: Low-Rank Adaptation of Large Language Models (arXiv:2106.09685) | 18 |
| 2021 | Press, Smith & Lewis. Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation (arXiv:2108.12409) | 8 |
| 2021 | Su et al. RoFormer: Enhanced Transformer with Rotary Position Embedding (arXiv:2104.09864) | 8 |
| 2021 | Wei et al. Finetuned Language Models Are Zero-Shot Learners (arXiv:2109.01652) | 17 |
| 2022 | Bai et al. Constitutional AI: Harmlessness from AI Feedback (arXiv:2212.08073) | 17 |
| 2022 | Dao et al. FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness (arXiv:2205.14135) | 12 |
| 2022 | Dettmers et al. LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale (arXiv:2208.07339) | 15 |
| 2022 | Frantar et al. GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers (arXiv:2210.17323) | 15 |
| 2022 | Hoffmann et al. Training Compute-Optimal Large Language Models (arXiv:2203.15556) | 16 |
| 2022 | Leviathan, Kalman & Matias. Fast Inference from Transformers via Speculative Decoding (arXiv:2211.17192) | 10 |
| 2022 | Liang et al. Holistic Evaluation of Language Models (arXiv:2211.09110) | 21 |
| 2022 | Micikevicius et al. FP8 Formats for Deep Learning (arXiv:2209.05433) | 15 |
| 2022 | Olsson et al. In-context Learning and Induction Heads (arXiv:2209.11895) | 6, 19 |
| 2022 | Ouyang et al. Training language models to follow instructions with human feedback (arXiv:2203.02155) | 17 |
| 2022 | Perez et al. Red Teaming Language Models with Language Models (arXiv:2202.03286) | 21 |
| 2022 | Wei et al. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models (arXiv:2201.11903) | 19 |
| 2022 | Yao et al. ReAct: Synergizing Reasoning and Acting in Language Models (arXiv:2210.03629) | 19 |
| 2023 | Ainslie et al. GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints (arXiv:2305.13245) | 11 |
| 2023 | Carlini et al. Poisoning Web-Scale Training Datasets is Practical (arXiv:2302.10149) | 16, 20 |
| 2023 | Chen et al. Extending Context Window of Large Language Models via Positional Interpolation (arXiv:2306.15595) | 8 |
| 2023 | Dao. FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning (arXiv:2307.08691) | 12 |
| 2023 | Dettmers et al. QLoRA: Efficient Finetuning of Quantized LLMs (arXiv:2305.14314) | 18 |
| 2023 | Greshake et al. Not what you’ve signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection (arXiv:2302.12173) | 19, 20 |
| 2023 | Gu & Dao. Mamba: Linear-Time Sequence Modeling with Selective State Spaces (arXiv:2312.00752) | 13 |
| 2023 | Jiang et al. Mistral 7B (arXiv:2310.06825) | 13 |
| 2023 | Kwon et al. Efficient Memory Management for Large Language Model Serving with PagedAttention (arXiv:2309.06180) | 10 |
| 2023 | Lightman et al. Let’s Verify Step by Step (arXiv:2305.20050) | 17 |
| 2023 | Lin et al. AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration (arXiv:2306.00978) | 15 |
| 2023 | Liu et al. Lost in the Middle: How Language Models Use Long Contexts (arXiv:2307.03172) | 8, 19 |
| 2023 | Nasr et al. Scalable Extraction of Training Data from (Production) Language Models (arXiv:2311.17035) | 20 |
| 2023 | Peng et al. YaRN: Efficient Context Window Extension of Large Language Models (arXiv:2309.00071) | 8 |
| 2023 | Prince. Understanding Deep Learning (book) | 1, 2, 4, 6 |
| 2023 | Qi et al. Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! (arXiv:2310.03693) | 17, 18, 20 |
| 2023 | Rafailov et al. Direct Preference Optimization: Your Language Model is Secretly a Reward Model (arXiv:2305.18290) | 17 |
| 2023 | Rouhani et al. Microscaling Data Formats for Deep Learning (arXiv:2310.10537) | 15 |
| 2023 | Schick et al. Toolformer: Language Models Can Teach Themselves to Use Tools (arXiv:2302.04761) | 19 |
| 2023 | Sclar et al. Quantifying Language Models’ Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting (arXiv:2310.11324) | 21 |
| 2023 | Touvron et al. LLaMA: Open and Efficient Foundation Language Models (arXiv:2302.13971) | 9 |
| 2023 | Wei, Haghtalab & Steinhardt. Jailbroken: How Does LLM Safety Training Fail? (arXiv:2307.02483) | 17, 20 |
| 2023 | Xiao et al. Efficient Streaming Language Models with Attention Sinks (arXiv:2309.17453) | 13 |
| 2023 | Zhang, Lipton, Li & Smola. Dive into Deep Learning (book) | 1 |
| 2023 | Zheng et al. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena (arXiv:2306.05685) | 21 |
| 2023 | Zou et al. Universal and Transferable Adversarial Attacks on Aligned Language Models (arXiv:2307.15043) | 20 |
| 2024 | Besiroglu et al. Chinchilla Scaling: A replication attempt (arXiv:2404.10102) | 16 |
| 2024 | Carlini et al. Stealing Part of a Production Language Model (arXiv:2403.06634) | 20 |
| 2024 | Dai et al. DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models (arXiv:2401.06066) | 14 |
| 2024 | Dao & Gu. Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality (arXiv:2405.21060) | 13 |
| 2024 | DeepSeek-AI. DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model (arXiv:2405.04434) | 11 |
| 2024 | DeepSeek-AI. DeepSeek-V3 Technical Report (arXiv:2412.19437) | 8, 9, 11, 14, 15, 16 |
| 2024 | Egashira et al. Exploiting LLM Quantization (arXiv:2405.18137) | 15, 20 |
| 2024 | Gloeckle et al. Better & Faster Large Language Models via Multi-token Prediction (arXiv:2404.19737) | 9 |
| 2024 | Grattafiori et al. (Llama Team). The Llama 3 Herd of Models (arXiv:2407.21783) | 5, 7, 9, 11, 16 |
| 2024 | Hayes et al. Buffer Overflow in Mixture of Experts (arXiv:2402.05526) | 14, 20 |
| 2024 | Hubinger et al. Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training (arXiv:2401.05566) | 17, 20 |
| 2024 | Jiang et al. Mixtral of Experts (arXiv:2401.04088) | 14 |
| 2024 | Mazeika et al. HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal (arXiv:2402.04249) | 21 |
| 2024 | Miller. Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluations (arXiv:2411.00640) | 21 |
| 2024 | OLMo Team. 2 OLMo 2 Furious (arXiv:2501.00656) | 16 |
| 2024 | Shao et al. DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models (arXiv:2402.03300) | 17 |
| 2024 | Snell et al. Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters (arXiv:2408.03314) | 17 |
| 2024 | Soldaini et al. Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research (arXiv:2402.00159) | 16 |
| 2024 | Song et al. The Early Bird Catches the Leak: Unveiling Timing Side Channels in LLM Serving Systems (arXiv:2409.20002) | 10, 20 |
| 2024 | Wang et al. Auxiliary-Loss-Free Load Balancing Strategy for Mixture-of-Experts (arXiv:2408.15664) | 14 |
| 2024 | Weiss et al. What Was Your Prompt? A Remote Keylogging Attack on AI Assistants (arXiv:2403.09751) | 10, 20 |
| 2024 | Yang, Kautz & Hatamizadeh. Gated Delta Networks: Improving Mamba2 with Delta Rule (arXiv:2412.06464) | 13 |
| 2025 | Debenedetti et al. Defeating Prompt Injections by Design (arXiv:2503.18813) | 20 |
| 2025 | DeepSeek-AI. DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models (arXiv:2512.02556) | 13 |
| 2025 | DeepSeek-AI. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning (arXiv:2501.12948) | 17, 18 |
| 2025 | Kimi Team. Kimi K2: Open Agentic Intelligence (arXiv:2507.20534) | 14 |
| 2025 | Kimi Team. Kimi Linear: An Expressive, Efficient Attention Architecture (arXiv:2510.26692) | 13 |
| 2025 | Liu et al. Muon is Scalable for LLM Training (arXiv:2502.16982) | 3 |
| 2025 | OpenAI. gpt-oss-120b & gpt-oss-20b Model Card (arXiv:2508.10925) | 5, 7, 8, 11, 13, 14, 15, 17, 19 |
| 2025 | Yang et al. (Qwen Team). Qwen3 Technical Report (arXiv:2505.09388) | 14 |
| 2025 | Yuan et al. Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention (arXiv:2502.11089) | 13 |
