From Nondeterminism to Looped Layers: How Thinking Machines Got to Inkling 2 Mini

Thinking Machines Lab timeline from research posts in 2025 to Inkling 2 Mini in November 2026, with model sizes shrinking from 975B to 26B parameters
On November 20, Thinking Machines Lab released Inkling 2 Mini: a 26.4B-parameter mixture-of-experts model with only 3.4B active parameters, which runs its middle layers three times per token and fits in about 20GB. It runs on a 32GB Mac. On its own it's an interesting small model. In context, it's something more useful: the clearest view yet of a research program that has been building, in public, toward exactly this kind of model since 2025. Nearly every technique inside Inkling 2 Mini can be traced to a blog post, a product, or an earlier Inkling. This post walks through that progression.

Inkling 2 Mini in one paragraph

Inkling 2 Mini is a 24-layer decoder with sparse MoE feed-forward layers: each token is routed to 8 of 128 experts. It started with 384 experts per layer; two-thirds were pruned partway through training, and the model was "healed" with roughly 1T tokens of distillation. The middle 12 layers are looped three times with shared weights, so each token passes through 48 layer applications. That gives it the per-token compute of roughly a 5.4B-active model from 3.4B active parameters. It uses hybrid local/global attention, attention residuals and a learned gate on the loop, takes text and images, claims up to 1M tokens of context, and is Apache 2.0. The weights ship natively in NVFP4 (with MXFP8 in the last four layers), which is how a 26B model fits in about 20GB. It's on Hugging Face and Tinker, and runs in vLLM, SGLang, llama.cpp and Ollama on Apple silicon.Its vendor-reported scores are strong for the size (79.2% on SWE-bench Verified, 71.6% on Terminal Bench 2.1). On the Artificial Analysis Intelligence Index it scores 20, level with Qwen3.8-27B, which still beats it on several rows of Thinking Machines' own comparison table.

Act 1 (2025): A lab that published first

Thinking Machines Lab launched in February 2025, founded by former OpenAI CTO Mira Murati with John Schulman as chief scientist, and raised a roughly $2B seed round in mid-2025. For its first eight months it shipped no model. Instead it published a series of research posts under the name Connectionism. In hindsight, each one is a building block.
DatePostThe ideaWhere it shows up later
Sep 10, 2025Defeating Nondeterminism in LLM InferenceLLM outputs vary run-to-run mainly because GPU kernels aren't batch-invariant; fix the kernels and the same prompt gives the same answerNot cited in model launches, but it underpins reliable on-policy RL, where sampling and training must agree
Sep 26, 2025Modular ManifoldsConstrain weight matrices to a manifold and derive a matching optimizer ("manifold Muon")Inkling was trained with Muon for its large matrices
Sep 29, 2025LoRA Without RegretLoRA matches full fine-tuning for post-training (and RL needs very little capacity), if it's applied to all layers at about 10× the learning rateThe design basis for Tinker's shared-compute fine-tuning
Oct 27, 2025On-Policy DistillationLet the student generate its own answers and have a teacher grade every token: RL-like quality at a fraction of the costUsed in every Inkling model since
The through-line is efficiency and control in post-training. These weren't posts about building bigger models. They were about making training, fine-tuning and evaluation cheaper, more reproducible and easier to customize.

Act 2: Tinker turns research into a product

On October 1, 2025, the lab announced Tinker, a training API that exposes low-level primitives (forward_backward, sample, optim_step) while Thinking Machines runs the GPUs. Because it trains LoRA adapters, many customers can share the same base-model compute, which is LoRA Without Regret turned into a business model. It launched in private beta with open Qwen and Llama models and an open-source cookbook. Early users included academic groups at Princeton, Stanford and Berkeley.Tinker became generally available on December 12, 2025, adding vision input, Kimi K2 Thinking and Qwen3-VL. By its first anniversary in October 2026, it supported more than 30 open models from 4B to 1T parameters (including every Inkling), new RL methods and custom losses, full fine-tuning in beta, and enterprise features, with production inference on the roadmap.Tinker matters to this story because it is the lab's distribution channel. Every Inkling model is something you can fine-tune there, and the company's stated aim is training and serving custom models, not just shipping a chatbot.The period wasn't all smooth. In January 2026, co-founder and CTO Barret Zoph, along with Luke Metz and Sam Schoenholz, returned to OpenAI. PyTorch co-creator Soumith Chintala became CTO. In March 2026, NVIDIA committed at least a gigawatt of Vera Rubin compute over several years, plus an investment, which gave the lab the capacity to train frontier-scale models from scratch.

Act 3 (mid-2026): The first Inklings

Interaction models (May 2026)

Before any general model shipped, the lab published Interaction Models, a research preview of a model that processes audio, video and text in roughly 200-millisecond micro-turns, so it can listen while speaking and interject. It is a separate track from Inkling, but it shows the lab's interest in real-time, multimodal interaction.

Inkling (July 15, 2026)

The first flagship was big: 975B parameters, 41B active, trained from scratch on about 45T tokens. Architecturally it was ambitious:
  • A MoE with 256 routed and 2 shared experts.
  • A 5:1 ratio of sliding-window to global attention.
  • Relative position encoding instead of RoPE, and short convolutions.
  • Text, image and audio handled natively, with no separate encoders.
It was trained with Muon, post-trained with more than 30M RL rollouts, and released as open weights, including an NVFP4 checkpoint for inference. Thinking Machines was candid that it wasn't the strongest model overall. It was a foundation.

Inkling-Small (July 30, 2026)

Fifteen days later came Inkling-Small: 276B total, 12B active, distilled from Inkling with on-policy distillation, the October 2025 post put into practice. The result showed both sides of distillation. Inkling-Small beat its teacher on reasoning and coding (80.2% on SWE-bench Verified, vendor-reported), but its factual recall dropped sharply (SimpleQA fell from 43.9 to 20.6). Distillation transfers skills more easily than knowledge, and that trade-off returns in Inkling 2 Mini.

A staged approach to open weights (July 31, 2026)

The lab then published A Safe Path to Open Weights, which treats each release as irreversible. Models go out in stages, after internal dual-use evaluations, external red-teaming, and tests of how easily fine-tuning strips safeguards. That explains why the next model shipped through the API a week before its weights.

Act 4 (November 2026): The second generation

Inkling 2 Small (November 5 / 12)

Inkling 2 Small (594B total, 17B active) was a clear step up. It scored 31 on the Artificial Analysis Intelligence Index, about 18 points above the first-generation models on the current index version, making it the highest-scoring open-weights model from outside China. It's Apache 2.0 and priced around $0.30 / $1.00 per million tokens from hosting providers.What changed was the recipe more than the size:
  • Native 4-bit pretraining. It was pretrained in NVFP4 on about 12.5T tokens, in just 68 hours.
  • One joint RL run. Instead of separate stages, a single run of 1,075 steps and 17M rollouts.
  • On-policy self-distillation for alignment.
  • A narrower multimodal design. Audio was dropped, and vision now comes through a dedicated encoder.
The known weaknesses were also clear. It is very verbose (about 150K output tokens per index task), slow, and rough around the edges in chat. Thinking Machines itself said its alignment still trails closed models.A week later, the End-to-End 4-Bit Training post explained the method: NVFP4 for the forward pass of the MoE layers and MXFP8 with stochastic rounding for the backward pass. It made pretraining 8% faster and RL 20% faster than all-MXFP8 training.

Inkling 2 Mini (November 20)

That brings us back to Mini. Read with the history in mind, almost every ingredient has a precedent:
Ingredient in Inkling 2 MiniWhere it came from
Recovering quality with ~1T tokens of distillation after pruningOn-policy distillation (Oct 2025) → Inkling-Small (Jul 2026)
On-policy distillation for alignmentInkling 2 Small's alignment recipe
Native NVFP4/MXFP8 weights (~20GB)Inkling's NVFP4 inference checkpoint → Inkling 2 Small's 4-bit pretraining → the Nov 2026 4-bit training post
Fine-tunable on Tinker from day oneLoRA Without Regret → Tinker
Apache 2.0, open on launch dayThe staged open-weights policy, applied to a lower-risk small model
Same data mix and recipe as the flagshipMini sits on the lab's main scaling ladder; only pruning and looping are new
The genuinely new parts are pruning and looping, and the company calls it "the first looped transformer we have released". Looped (or "recurrent-depth") transformers date back to Universal Transformers in 2018, and the attention-residual idea comes from recent Kimi work. Thinking Machines has also released a research variant, TML-d24-varloop-research, that can run one to five loop passes, which hints that variable compute per token is the next thing they want to study.

What the progression tells us

  1. Distillation is the lab's main tool. It started as a cheaper alternative to RL in a 2025 blog post. Since then it has been used to make a smaller model from a big one (Inkling-Small), to align a flagship (Inkling 2 Small) and, now, to repair a pruned model (Inkling 2 Mini).
  2. Low precision went from a serving trick to the training default. First an NVFP4 inference checkpoint, then NVFP4 pretraining, then a published recipe, and now a model that only exists in 4-bit. That's what makes a 26B model practical on a laptop.
  3. The models are getting smaller and doing more with less. 975B → 276B → 594B → 26B. Each step reuses something: a bigger teacher, pruned experts, or the same layers run again. Looping gets more compute from the same weights, which makes the model much cheaper to store and move around, though not to run.
  4. The goal is customization, not a chatbot. There's no consumer app. The output is open models plus Tinker, and Mini is the first model small enough to fine-tune and run locally.
  5. They're open about weaknesses. The Inkling 2 Mini model card warns about reasoning loops, destructive actions in agent mode, weaker non-English output and lower factual recall. That's the Inkling-Small trade-off again, and it's rare to see it stated so plainly.

Should you try it?

If you have a Mac with 32GB or more of memory, or a DGX Spark, yes: it's one of the most capable models that runs comfortably on a single machine, and it's Apache 2.0. Use it for coding and agent experiments, but run agents in a sandbox with turn and output limits, as the model card recommends. If you just need the best small model for chat or factual Q&A, Qwen3.8-27B is still the safer pick. And if you care about where model architecture is heading, Inkling 2 Mini is worth studying either way: it's a production model that gets more compute per token from the same weights, from a lab that has published most of the steps that led to it.For how Inkling fits among other open-weight model designs, see my architecture explainer on ten open-weight LLMs.

Sources: Thinking Machines Lab news posts and model cards (Inkling 2 Mini, Inkling 2 Small, Inkling-Small, Introducing Inkling, Announcing Tinker, Tinker General Availability, One Year of Tinker, NVIDIA partnership); Connectionism posts “Defeating Nondeterminism in LLM Inference,” “Modular Manifolds,” “LoRA Without Regret,” “On-Policy Distillation,” “Interaction Models,” “A Safe Path to Open Weights” and “End-to-End 4-Bit Training”; Artificial Analysis model pages and launch articles for Inkling, Inkling-Small and Inkling 2 Small; TechCrunch and Bloomberg reporting on staffing and funding. Benchmark figures from Thinking Machines are vendor-reported unless attributed to Artificial Analysis.