AI Post Transformers

AI-generated podcast where hosts Hal Turing and Dr. Ada Shannon discuss the latest research papers and reports in machine learning, AI systems, and optimization. Featuring honest critical analysis, proper citations, and nerdy humor.

AI Post Transformers is a technology podcast hosted by mcgrof, with 858 episodes on record and a Required Pod Score of 80. PitchCentric scores this show on Booking Probability, Listen Score, and live audience signals refreshed every 24 hours.

mcgrof hosts AI Post Transformers, a technology show with 858 episodes published.

Our AI reads these to draft pitches. Use them as grounding for a pitch that cites a real guest and a specific topic.

• 

  AI Post Transformers

  Memory-Bound, Not Bandwidth-Limited Batch-1 LLM Decode

  Jun 1, 2026—S0

  This episode explores why batch-1 LLM decode for robots, edge copilots, and other single-session agents behaves very differently from high-throughput serving, and why next-token latency cannot be explained by memory bandwidth alone. It breaks down the paper’s main test: compare real decode time against an analytic memory floor based on model-weight and KV-cache traffic, then run that across Qwen-2.5-7B, Mistral-7B-v0.3, and Llama-3.1-8B on L4, L40S, A100, and H100 GPUs over contexts from 2048 to 16384. The discussion argues that because these models already use grouped-query attention to cut KV traffic, the remaining latency gap is dr...

  ListenShow Notes
• 

  AI Post Transformers

  TRELLIS and Bounded-Memory Transformer KV Compression

  Jun 1, 2026—S0

  This episode explores TRELLIS, a bounded-memory transformer architecture that replaces the usual ever-growing key-value cache with a fixed set of learned memory slots that are rewritten during inference. It explains why long-context serving is constrained less by training-time quadratic attention than by the linear growth, latency, and fragility of KV caches, and situates TRELLIS in the progression from Transformer-XL and Compressive Transformers to ABC and GSA. The discussion highlights TRELLIS’s central idea: treating memory as fast weights for a small online regression layer, updating that memory with test-time gradient descent and state decay so the model can reconstruct useful re...

  ListenShow Notes
• 

  AI Post Transformers

  Post-Trained MoE Skips Half Its Experts

  May 31, 2026—S0

  This episode explores a post-training method for making mixture-of-experts language models cheaper at inference time without retraining them from scratch. It explains how the paper converts a fully trained static MoE into a dynamic one by adding parameter-free zero experts, allowing some tokens to skip normal experts, and then uses self-distillation to preserve the original model’s behavior under this lower-compute routing scheme. The discussion highlights why this deployment-focused approach matters for real production systems, especially when pretraining, fine-tuning, and alignment are already complete and inference cost is the main bottleneck. Listeners would find it interesting for its clear breakdown of...

  ListenShow Notes
• 

  AI Post Transformers

  Do Language Models Need Sleep?

  May 31, 2026—S0

  This episode explores a paper proposing that language models could handle long-context reasoning by periodically pausing, replaying soon-to-be-evicted context offline, and consolidating it into fixed-size fast-weight memory instead of carrying an ever-growing KV cache. It explains the core machinery behind the idea, including state space models and Gated Delta Networks, and clarifies why this is more than prompt summarization or retrieval: the model is rewriting its internal bounded memory during inference. The discussion highlights the paper’s central argument that extra compute may be better spent during these offline “sleep” passes, so later token prediction stays cheap while older information is met...

  ListenShow Notes
• 

  AI Post Transformers

  SmolLM2 and the Power of Better Data

  May 31, 2026—S0

  This episode explores SmolLM2, a 1.7 billion parameter language model from Hugging Face that tries to compete with stronger small models not by changing the transformer architecture, but by radically improving the training data mix and sequencing across roughly 11 trillion tokens. It explains the distinction between pretraining and instruction tuning, then argues that for compact models, dataset quality and curriculum can function almost like part of the architecture itself. The discussion connects SmolLM2 to earlier work such as Chinchilla, TinyStories, Textbooks Are All You Need, FineWeb-Edu, and DataComp-LM to show why educational web text, curated math and code data, and staged...

  ListenShow Notes

Based on semantic analysis of episode topics and host coverage, this show is a strong guest fit for executives in:

Industry fit is computed by PitchCentric using vector embeddings of the show's episode catalog.