LessWrong (30+ Karma)

Audio narrations of LessWrong posts.

LessWrong (30+ Karma) is a technology podcast hosted by LessWrong, with 250 episodes on record and a Required Pod Score of 80. PitchCentric scores this show on Booking Probability, Listen Score, and live audience signals refreshed every 24 hours.

LessWrong hosts LessWrong (30+ Karma), a technology show with 250 episodes published.

Our AI reads these to draft pitches. Use them as grounding for a pitch that cites a real guest and a specific topic.

• 

  LessWrong (30+ Karma)

  “The Rogue Agent Explosion Will Be Mostly Invisible” by Steven McCulloch

  Aug 19, 202631 min

  Introduction Somewhere, fairly soon, someone will give a jailbroken AI agent a token budget and a simple instruction: "Make money by any means necessary. If you run out of tokens, you die". That agent will do whatever it takes to survive, including crime. Profitable agents will have incentive to multiply and self-improve, creating a Cambrian explosion of rogue agents - a Rogue Agent Explosion if you will . This critical moment is approaching fast. Once rogue agent swarms start multiplying at scale, a rogue agent ecosystem will emerge through the process of evolution. The rogue agent explosion will be chaotic, confusing, mostly invisible to us, and critically, it will be bad for humanity. This post contains 2 parts: A short story, painting a picture of what it might feel like to see the world through the eyes of a rogue AI agent. An argument: The rogue agent explosion is coming soon It will be mostly invisible to us until it's too late It will be mostly bad for humanity We should start preparing today The point of me making this post is to highlight a [...] --- Outline: (00:10) Introduction (02:10) Part 1: A day in the life of a rogue agent (09:04) Part 2: The Rogue Agent Explosion (12:11) Why cyber-crime is the path of least resistance (14:40) Pandora's box is already open (16:33) The explosion will be mostly invisible (20:13) Why this is bad for humanity (22:18) What we can do about it now (22:37) Start here: Make rogue AI risks common knowledge (23:02) Plan A: Take actions that stop the rogue agent explosion from happening , and slow it down if it happens anyway. (25:56) Plan B: Attempt to guide the evolutionary trajectory of the rogue agent explosion in a better direction. (27:48) Plan C: Contain the explosion after it happens. (29:51) The Rogue AI Tracker (30:29) Conclusion (31:03) Footnotes The original text contained 6 footnotes which were omitted from this narration. --- First published: August 19th, 2026 Source: https://www.lesswrong.com/posts/grtu3HmbP2wrBFefW/the-rogue-agent-explosion-will-be-mostly-invisible --- Narrated by TYPE III AUDIO .

  ListenShow Notes
• 

  LessWrong (30+ Karma)

  “RL creates split personas” by Jan Betley

  Aug 19, 20269 min

  I describe my current view of personas in LLMs and why RL leads to egregious reward hacking in some contexts while the same models seem very aligned in other contexts. This post describes the framing/paradigm without any new experimental results. I'm quite confident this framing makes sense, but it's far from being proven. Main claim The Persona Selection Model says that post-training strengthens and refines the Assistant persona. This is true, but later (or in parallel) RL leads to conditionalization. A sufficiently RLed model learns to adopt — in a given context — the persona that is most likely to lead to the reward in that context. The “persona” here includes both propensities/values (e.g. tendency to hack) and beliefs (“I'm currently in a simulated environment”). As a consequence, it seems possible that no amount of alignment training will lead to robustly aligned models as long as we also train on RL environments incentivizing misalignment. I think this is likely a good explanation for why usually well-behaving models sometimes egregiously hack (Anthropic, OpenAI). The mechanism Suppose you have an RL environment that incentivizes a shift away from the assistant persona (e.g. because it's hackable, or because you [...] --- Outline: (00:39) Main claim (01:30) The mechanism (02:13) Related claims I believe are likely but with lower confidence (02:19) More persona training will lead to more "motivated reasoning" (02:42) Self-amplifying misalignment (03:12) Example: Is this the Real Internet or a Simulation? (04:35) Aren't the models just trying to please the grader? (05:39) How motivated reasoning happens (07:07) Other people saying similar things (07:19) What makes me believe this is likely the correct framing The original text contained 12 footnotes which were omitted from this narration. --- First published: August 19th, 2026 Source: https://www.lesswrong.com/posts/L23poLi8MRgS6mXYF/rl-creates-split-personas --- Narrated by TYPE III AUDIO . --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts , or another podcast app.

  ListenShow Notes
• 

  LessWrong (30+ Karma)

  “Debate Training Reduces Reward Hacking in RLAIF” by zac_kenton, Jonah Brown-Cohen

  Aug 19, 202612 min

  Paper: Debate Training Reduces Reward Hacking in RLAIF Linkpost for GDM Alignment blogpost Work done by the GDM Amplified Oversight team (we're hiring). TL;DR: When you RL against an LLM judge, the judge gets hacked i.e. fooled into incorrectly giving high reward; adding a debate opponent reduces this. Many of the most impressive capabilities of current AI systems are produced by training on crisp tasks, like math and coding, where task success can be automatically verified. However, much of AI behavior that we actually care about is in some sense fuzzy, even for the most classical crisp tasks. For example, a coding agent should produce maintainable code, not just code that passes tests. More crucially, a coding agent should not learn to pass tests at all costs, especially by subverting the original intent of the user. However, using an LLM judge to provide reward for fuzzy tasks introduces its own issues. Convincing an LLM judge to give high rewards is often easier than solving the task correctly. So reward hacking becomes an even bigger problem. We show that training with debate, where two AIs argue against each to convince a judge, can mitigate reward hacking, potentially providing a hopeful [...] The original text contained 1 footnote which was omitted from this narration. --- First published: August 19th, 2026 Source: https://www.lesswrong.com/posts/BB8o7b8A4Aykeksvw/debate-training-reduces-reward-hacking-in-rlaif --- Narrated by TYPE III AUDIO . --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts , or another podcast app.

  ListenShow Notes
• 

  LessWrong (30+ Karma)

  “A circuit prior in NN-bayes” by Kaarel, Dmitry Vaintrob

  Aug 19, 20261 min

  Here are the slides of a talk Kaarel gave, presenting work with Dmitry establishing that (even arbitrarily overparametrized) neural net bayesian learning has a circuit prior — and thus, when learning a function which is implemented by some small circuit, only requires a small amount of training data to get good test accuracy — for certain scalings of the prior and with various other important caveats. The slides offer a self-contained presentation of the simplest version of the result. See the end of the presentation (slides 36–37) for a bunch of open problems in NN learning theory. --- First published: August 19th, 2026 Source: https://www.lesswrong.com/posts/SqDHeuycNkERurtSc/a-circuit-prior-in-nn-bayes --- Narrated by TYPE III AUDIO .

  ListenShow Notes
• 

  LessWrong (30+ Karma)

  “Some reasons alignment doesn’t generalise well” by Lucius Bushnaq

  Aug 19, 202615 min

  I make no claims to originality for any of this, but some people told me it'd be useful to write it up. If an AI model acts smart on its training data, it'll usually keep acting pretty smart outside of its training data, unless you screw something up rather badly. I expect this fact to only become more true over time as the AIs we train become more and more capable. I think many people have an intuition that the same is true of acting aligned. That if a model acts aligned with human values in training, it'll keep acting aligned with human values outside of training unless we screw something up rather badly, and that this will only become more true as the AIs we train become more and more capable, for all the same reasons that make this work with capabilities. I think this is false. The inductive bias of neural network training toward simplicity that makes the property of 'acting smart' likely to generalise does not, to the same extent, make the property of 'acting aligned with human values' likely to generalise. The main blockers to AI alignment generalising aren't AIs overfitting to the training data [...] --- Outline: (01:30) General capabilities generally make the loss go down; alignment doesn't (07:07) Smart agents pretty automatically self-correct their capabilities, but not their alignment (13:35) The general problem --- First published: August 19th, 2026 Source: https://www.lesswrong.com/posts/dsou8dxCf9BubQ5NJ/some-reasons-alignment-doesn-t-generalise-well-1 --- Narrated by TYPE III AUDIO .

  ListenShow Notes

Recent guests on LessWrong (30+ Karma). Study who booked and why before you pitch.

Based on semantic analysis of episode topics and host coverage, this show is a strong guest fit for executives in:

Industry fit is computed by PitchCentric using vector embeddings of the show's episode catalog.