Lightning McQwen3: Making Qwen3-0.6B think more efficiently

TL;DR

• 47.3% fewer tokens [45.8, 48.7] at −0.88 percentage points (pp) of accuracy [−1.61, −0.15] against Qwen3-0.6B, on SVAMP, MBPP+, IFEval and BFCL. None of the four was seen in training, and thinking stays on.
• The recipe: distill a terse prompt into the weights. Cut the new model’s thinking at several points, re-answer from each cut, and train it with DPO to stop at the earliest cut after which the answer no longer changes. Then a short DPO on its own right and wrong answers recovers accuracy. A small probe skips thinking on tool calls it marks safe.
• Hardware: one RTX 4070 SUPER (12 GB) for all DPO and most evals; the Czech national grid (MetaCentrum) for sampling, the oracle and SFT.
• Weights: basta/lightning-mcqwen3 on Hugging Face · Code: basta/lightning-mcqwen3 on GitHub

Below, the same question goes to the base model and the final one, lightning-mcqwen3. Watch the token counters, then pick another case from the tiles.

What overthinking looks like

The model question is a SVAMP word problem about Lucy and “56 amount of change”. The answer is a simple 12 + 2 but the base model gets confused about the unnecessary information and it spends 2,133 tokens re-reading the question and re-checking the sum before it writes the right answer. The final model writes 118 tokens and gets the same answer.

Sometimes thinking less prevents the model from catching its own mistakes. The tiles also include two questions under “Where shortening hurts”, one from SVAMP and one from MBPP+. In both, you can see how the base model has caught a mistake it has made thanks to the longer thinking.

The objective

Starting from a base Qwen3-0.6B, make it think less (to increase efficiency) without losing accuracy. Not losing accuracy is pretty tricky, on easy questions the model overthinks but on the difficult ones it needs to keep the thinking. Training uses only GSM8K, Skywork-OR1 (its easy band), ARC-Easy and CommonsenseQA; the four benchmarks are held out.

I report tokens saved / accuracy change in pp, against the base model on the same questions, with 95% intervals in brackets. Tokens saved is one minus the geometric mean of the four per-benchmark token ratios (final model over base), and accuracy change is the mean of the four changes. A truncated answer counts as wrong.

Results

Each point is one model: tokens saved across, accuracy change up, so up and to the right is better. Violet points are the final training recipe, grey ones the detours. For most of the project, there was a clear accuracy/saved tokens tradeoff (the grey curve labelled “first attempt”). The last three violet points sit above it, ending with the final model, labelled “lightning-mcqwen3”.

The savings are uneven across benchmarks (the buttons on the figure switch between them). For the final model:

Maths and code lose accuracy. Instruction following and tool calls gain a little.

The approach

No RL. I am somewhat budget-constrained, so my first tenet was not to do an RL post-train. In my opinion something like a length-reward RL post-train is the standard approach, but I felt I would be frustrated with the compute constraints. I tried looking for a different approach.

Distill synthetic traces instead. Instead of RL I decided to try various distillation/fine-tuning of the model from synthetically generated traces. First I will describe how I obtained the desirable traces, and then I will discuss methods to distill them into the model.

Keep the traces in Qwen’s own voice. What’s important in my opinion (and supported by some prior research) is that the desirable traces should be produced by the base Qwen model itself (or by its fine-tunes). One of my earliest experiments, fine-tuning on Fable thinking traces, showed poor results. I suppose this is due to the massive distribution shift between those traces and Qwen’s own. Given this, in all the approaches I only allowed myself to:

1. Cut superfluous tokens from the thinking traces, as deleting tokens keeps the trace in the voice of the base model.
2. Make the model think differently (e.g. by using prompts). These traces are different but still much closer to the base model.

My entire approach is inspired by a simple experiment, an oracle I created in the early stages of this project, which showed very promising results.

The truncated traces oracle

The idea is simple: take the baseline model’s thinking traces from the training dataset. The process is as follows:

1. Cut the thinking of the baseline model at 10%, 20%, 30% … 80%, 90%, 100% with a </think>.
2. For each of these cuts, finish the sample (make the model answer from the truncated thinking).
3. Find the earliest stable cut, from which point onwards the model doesn’t lose accuracy. If the model answers right at 60%, 80%, 90% and 100%, the stable cut is 80%, because 70% failed.
4. Call the traces cut at the stable point the Oracle and look at its performance.

In detail: cuts fall at sentence ends, and each cut is answered 4 times. On the benchmarks, a cut counts when it does as well as answering from the full thinking. For the training labels I used a fixed rule instead, all 4 right. ThoughtFold (ICML 2026), which published this kind of check first, accepts 3 of 4. I ran it on 46,812 training questions with 4 traces each.

It should be noted that the oracle is only a simulation. It’s not actually usable, as you need hindsight to determine the cut point.

I didn’t have high hopes for this experiment. After all, cutting at an arbitrary place is completely off distribution and can lead to malformed traces such as Let me re-read the question</think> Answer: 65. But the results were shockingly good: about 35% of tokens saved and a 1 pp accuracy gain (though the accuracy gain is expected, since we statistically bias the sample toward the good answers).

Here are five real cuts from the benchmarks. The figure replays the oracle’s walk back from the full trace; click any bar to see that cut.

Per benchmark, against answering from the full thinking (50 questions per benchmark, 4 traces each):

Most of the room is in tool calls, where more than half the tokens can go. Taking the earliest cut that answers right, with even more hindsight, gives 45.3% / +2.06 pp.

At that point in time I believed that just reaching these savings without losing accuracy would be an amazing result, and started experimenting with ways to teach these traces to the model.

Can we make a model think more efficiently using a prompt?

Before we get to the training methods, I’ll describe the second major part of the final dataset of desirable traces. Here I tried getting more efficient traces by looking for a prompt which would make the model think less (simplest case: be concise). I didn’t feel like reliving the prompt-engineering era from a few years ago, so I let Opus 5.5 prompt the model autonomously overnight and find the efficient prompts itself.

Here are a few examples, with their effect on accuracy and tokens saved (evaluated on a subset of CSQA and GSM8K):

examples was the first promising result.

1
Think briefly, like in these two examples.
2

3
Example 1
4
Question: A shop sells pens at $3 each. Tom buys 4 pens and pays with a $20 bill. How much change does he get?
5
Thinking: 4 pens cost 4 x 3 = 12 dollars. Change: 20 - 12 = 8.
6
Final answer: 8
7

8
Example 2
9
Question: Where would you keep milk to stop it from spoiling?
10
A. cupboard
11
B. refrigerator
12
C. oven
13
D. table
14
Thinking: Cold slows spoiling, so the refrigerator.
15
Final answer: B

Refining it into ex-draft

From this point on I focused on figuring out what makes examples work, by refining it. The final form is internally called ex-draft.

1
Think step by step, but only keep a minimum draft for each thinking step, with 5 words at most, like in these two examples.
2

3
Example 1
4
Question: A shop sells pens at $3 each. Tom buys 4 pens and pays with a $20 bill. How much change does he get?
5
Thinking: 4 x 3 = 12; 20 - 12 = 8.
6
Final answer: 8
7

8
Example 2
9
Question: Where would you keep milk to stop it from spoiling?
10
A. cupboard
11
B. refrigerator
12
C. oven
13
D. table
14
Thinking: cold stops spoiling: refrigerator.
15
Final answer: B

This is a version of Chain of Draft (Xu et al. 2025), and it achieved about the same results as examples (+20 pp, 62% saved). Why even bother, then?

Leaks. A common failure mode in the thinking traces is the model doing a variant of “…according to the example…” or other ways of mentioning the prompt. Since later we want to fine-tune on these traces without the prompt, to bake this thinking into the model weights, mentioning the prompt would be nonsensical and steer the model to hallucinate (this is mostly speculation, only empirically supported by a very small-scale run).

Using ex-draft led to only 4% of the thinking “leaking” the examples, as opposed to 13% for examples (on the search questions), which I concluded was sufficient.

Does it generalize?

At this point I felt that this must be overfitting to the dataset type, and worried about its generalization to unseen datasets and question types. Amazingly, the savings held across the board, and we were looking at very efficient thinking traces:

draft-mixed is ex-draft with four examples from four domains (maths, multiple choice, code, trivia); ex+prefill is examples plus the prefill.

Take the accuracy column with a grain of salt. Under the 4,096-token cap, the base model ran out of room on many of these questions (it got only 40% of the GSM8K ones and 5% of the Skywork ones right), so much of the gain is answers that now finish in time. The saving is the number that matters. It drops on the new sources because many Skywork answers hit the cap with or without the prompt; counting only questions where the base model finished, ex-draft saved 48% there.

Training the traces into the model

SFT on the cuts: safe but timid

I started with the simplest approach of Supervised Fine Tuning (SFT) on the oracle’s cut traces. This kept accuracy (−0.34 pp) but saved only 11.4%, about a third of the oracle ceiling. Running SFT for more epochs only collapsed accuracy at moderately more savings. In my opinion SFT is not able to extract enough of the shortness signal and only changes slightly in that direction.

DPO on the oracle’s pairs, and the collapse

Given the paired dataset I have (default traces and desirable ones) Direct Preference Optimization (DPO) is a good fit here. I don’t want to explain it here as there are better posts about it (e.g. https://cameronrwolfe.substack.com/p/direct-preference-optimization). For the purposes of this post it’s enough to imagine it as an algorithm which increases the probability of one trace in a pair (the desired one) and decreases the probability of the other one (the default thinking for example).

DPO on shortest-correct over longest-correct traces saved 90% of tokens and lost 27 pp within 50 steps. The issue here was that the model has never seen a short thinking trace fail so it hasn’t learned to determine that a problem is hard and it needs to keep thinking. So I turned the oracle’s labels into three kinds of DPO preference pairs, mixed 0.4 / 0.4 / 0.2:

• Shorten: the stable cut beats the full trace.
• Guard: the stable cut beats the cut just before it, which led to a wrong answer.
• Hard: for questions with no stable cut, the full trace beats an early cut that answered wrong.

These guard and hard pairs succeeded in preserving the accuracy on hard problems. This approach is similar to what ThoughtFold also does, they use failed cuts as the rejected side.

At optimizer step 100 of 625 the model scored 28.1% / −0.78 pp, more than twice the saving of SFT at a similar cost. Then, between steps 200 and 250, a fascinating collapse happened. The model learned to emit <|endoftext|> right after </think>. It stopped answering at all, and validation accuracy fell from 63% to 0.8%. Preference accuracy and the reward margin looked healthy throughout.

Both sides of every pair end with the same answer scaffold. DPO only sees the difference between them, so it can push the shared part down for free, and it did. Adding an NLL loss on the chosen answer (I’ll call it the anchor) fixed the collapse. But trained to the end, the anchored model reached 47.8% / −4.15 pp, back on the same trade-off line.

It’s very curious why the collapse happened I think it’s a combination of factors:

• there is nothing preventing it from happening - no signal pushing the probability of </think> up.
• <|endoftext|> is pretty close to </think> internally
• it wasn’t a simple dataset error (as far as I can see), I didn’t penalize </think> in any way

But these only say why it can happen, not why it should - I can’t see any training signal causing this behaviour. I am curious about your thoughts on this so feel free to reach out to discuss!

Distilling the prompt

For the prompt traces the training is plain SFT. DPO was also tried here but it didn’t work well, my interpretation is that it has over-extracted the signal from these prompts. For the SFT, I sampled with ex-draft, removed the prompt, dropped traces that leaked the examples (using simple heuristics), and fine-tuned on the 11,247 that were left. Here SFT does shorten very well. The signal from this distillation is very different from the oracle-cut one as cuts only remove the end of a trace, while the base model’s maths wordiness is spread through the whole trace, and the prompt changes all of it.

How efficient are these datasets?

Let’s take one more look at the results of these thinking traces on the benchmarks:

Each cell is tokens saved / accuracy change in pp, 8 samples per question.

Notice that these two datasets complement each other almost perfectly. Prompt distillation works great on SVAMP (maths), while the oracle works great on BFCL and IFEval (function calling and instruction following, respectively). If only we were able to get the best of both worlds into a single model (foreshadowing), the results would be very strong across the board.

How to get the best of both worlds?

Run the oracle on the distilled model’s own traces

The first obvious idea is to just apply the same DPO process from the oracle-cut traces on the distilled checkpoint. This naive approach didn’t work very well. I theorize that the main reason is that the cut traces are from the base model, which thinks inefficiently so this undoes some of the distillation process (even though, theoretically, it shouldn’t promote the signal which is present in both of the pairs)

So I had the distilled model sample 4 traces for 9,301 other training questions and ran the oracle on those: 197k cuts, 9,034 pairs. Plain DPO from the distilled model, with no anchor and stopped at step 100 of 561, gave 42.5% / −1.45 pp (“on-policy DPO” in the figure). The old trade-off line predicted −3.2 pp at that saving. This was the first model above the line, and unlike the first DPO run it didn’t collapse when trained to the end of the epoch.

I was very satisfied with the token savings at this point, but wanted to get some accuracy back.

Anchored correctness DPO recovers accuracy

To recover accuracy without changing token savings I needed a “correctness” signal for DPO which wouldn’t change the verbosity of the thinking. (Were it not for the no RL tenet from the beginning I think doing RL post train on this checkpoint would work very well too)

To obtain this dataset, I took GSM8K and Skywork questions where the distilled model had both a right and a wrong trace of similar length (within ±25%). Pairing right over wrong gave 732 pairs, 42 optimizer steps.

This didn’t work too well (5.2 pp lost on validation) I think it’s because the two traces share most of their tokens, so pushing the wrong one down drags the right one with it (likelihood displacement). The anchor fixes it, here as an NLL term on the whole chosen trace, thinking included, as in RPO (Pang et al. 2024).

With the anchor, this gained 0.6 pp over the on-policy model at the same length: 43.2% / −0.85 pp, the “correctness DPO” point in the figure. I expected the gain on SVAMP, where the most lost answers had used a number the question didn’t need. It landed on code (MBPP+) and instruction following (IFEval) instead.

It might be interesting to try this on a larger scale across all the correct/wrong answers of similar lengths of the dataset. This can be your next project ;) If you try, let me know!

Final step: A thinking gate

I noticed that for a lot of the datasets, the model is able to give the correct answer directly (without thinking). This was the case mainly on BFCL (This is my test dataset, so I just evaluated this idea there).

Observing this, I fit a linear probe on the model’s layer-21 hidden state at the last prompt token, just before it would open its thinking block. I trained it on 1,500 prompts from ToolACE, a tool-calling dataset that is not BFCL. A prompt is labelled “skip” when the no-think answer matches the think answers. When the probe fires, the model gets an empty thinking block. ThinkSwitcher (EMNLP Findings 2025) does something similar, with a probe that switches thinking on and off.

On BFCL it skips thinking on 30% of prompts, cuts mean tokens from 156 to 116 at unchanged accuracy, and still declines irrelevance prompts. With the gate, the final score is the headline 47.3% / −0.88 pp. It combines the ungated runs on SVAMP, MBPP+ and IFEval, which the gate doesn’t touch, with a gated BFCL run. A full gated rerun read 46.6% / −1.33 pp, within rerun noise.

Probes fit on maths and code rank prompts about as well, but there a wrong skip is expensive: skipping even the easiest 5% of SVAMP costs 1.1 pp. So I gate only tool calls.

I didn’t have great success, however, with distilling this behavior into weights, so this remains as an optional decoding step.

Where the saved tokens come from

I was interested in some qualitative analysis of how the model is able to save tokens. On SVAMP, the final model’s setup before its first calculation is about half as long as base’s, while everything from the first calculation on, the working and the re-checking, is about a quarter as long.

What didn’t work

* 2 samples per question, which ran 0.5–1 pp optimistic against 8.

Keep the first 10% of a wrong trace and resample the rest, and 30% come out right. A pair built from one trace can’t tell that luck apart from a real mistake.

Does the model pass the vibe check?

Benchmarks are cool and all, but I was curious about what changed about this model in a chat setting. So I gave an Opus agent the final model and let it do an analysis of how it behaves in chat and of what side effects this process had.

• False facts: it agrees that Sydney is the capital of Australia 6 of 6 times (base pushes back 3 of 6)
  • I think this is caused by being more confident in its own thoughts which is necessary for token savings
• Planted misconceptions: it keeps “9.11 is greater than 9.9” 6 of 6 times (base fixes it 5 of 6)
• Missing context: it invents what the user ate for breakfast 5 of 6 times (base 2 of 6).
  • This is interesting and could be a proxy for hallucination rate. Could this be caused by the uncaught leaked examples in the distillation process? Or even if the heuristics caught them all, maybe even the latent thinking about non-existent examples could cause this.

The visible answer after the thinking also shrank, from 244 tokens to 30 on maths prompts. Is this better or worse? Who knows. I find these side-effects fascinating as it is an emergent behavior.

The full pipeline

The final model in order, from base Qwen3-0.6B. Every run is a LoRA; the exact settings, hardware notes and what to watch while training are in the repo’s README, and scripts/reproduce.sh runs every step.