Episode #49
Run the Model Yourself: A Clinician’s Guide to Local AI
This one is a roundtable — no guest, just the three of us. Dr. Junaid Kalia sits down with co-hosts Ed Marx and Dr. Harvey Castro for a hands-on, slightly geeky conversation about something most clinicians have never been told they can do: run capable AI models yourself , on your own hardware, without ever sending a patient’s data to someone else’s cloud. It starts with a provocation — Junaid running a local model on a $2,000 Mac at ~100 tokens/second and asking why he’d keep paying Azure for the privilege. From there the three of them walk through the whole terrain: why Harvey now updates his emergency-medicine book with an AI agent every month, what “edge vs. near-edge vs. far-edge” actually means, whether an Apple laptop can really keep pace with a data center, and the uncomfortable catch — that the best cheap open models right now come out of China, which opens a security and standards question the U.S. hasn’t answered. They close on the part that matters most in medicine: what a model can never show the FDA, and why a rural clinic with no road to it might be the best argument for local AI there is. What You Will Discover: [00:00] Cold open — a $2,000 Mac at ~80–100 tokens/second (why bother going local?) [00:28] Welcome + why write a book when an AI can rewrite it every month [01:37] Who the book is really for: patients (and why the ER is a lot of people’s primary care) [02:34] 13 minutes with your doctor — how AI fills the gap (and visits trending down) [03:08] The home setup: 3 Macs, 128 GB unified memory, one tailnet [03:30] The model onslaught: what you can run for free (routers, Qwen, Kimi K2) [04:09] Edge, near-edge, or cloud — where should the model actually run? [05:00] Can an Apple laptop out-run a data center? (Apple Silicon + MLX + Core ML) [05:30] The catch: the best cheap models are Chinese — and who then owns the standard [06:37] Boeing vs. Airbus: the same fight, now national, now for AI [07:11] What a model can never show the FDA (SBOM + prompt injection) [08:15] The rural clinic more advanced than any academic center [09:19] Three things you can do with AI today Referenced in the show: ️ Harvey Castro’s AI in Emergency Medicine book + the “Pivot or Perish” series — https://www.harveycastromd.com/ ️ Open-weight model families discussed: Qwen (Alibaba), Kimi K2 (Moonshot AI), DeepSeek ️ Apple MLX + Core ML on Apple Silicon; Tailscale for a private multi-Mac “tailnet” ️ Inkling (referenced by Junaid) — a fine-tuning/infrastructure approach for bring-your-own-data models ️ Signal & Symptoms Ep 38 — “Why Will Digital Twins Change the Way We Treat Patients?” Connect with us: Dr. Junaid Kalia, Neurocritical Care Specialist and Founder of SaveLife.AI™ (host) LinkedIn — https://www.linkedin.com/in/junaidkaliamd Website — https://www.junaidkalia.com/ YouTube — https://www.youtube.com/@junaidkaliamd Edward Marx, CEO, Advisor (co-host) LinkedIn — https://www.linkedin.com/in/edwardmarx/ Website — https://www.marxadvisory.com/ YouTube — https://www.youtube.com/@EdwardMarx Dr. Harvey Castro, ER Physician, #DrGPT™ (co-host) LinkedIn — https://www.linkedin.com/in/harveycastromd/ Website — https://www.harveycastromd.com/ Visit our website: https://signalandsymptoms.com/ Get advisory insights about Healthcare AI — at zero cost: https://junaidkaliamd.substack.com/ Medical disclaimer: This conversation is for general informational purposes and is not medical advice. Listening does not create a doctor–patient relationship. The tools and models discussed are general-purpose technology, not cleared medical devices; nothing here is an endorsement of any specific vendor or model for clinical use. Do not enter protected health information into consumer AI tools.
