M1 Max 64GB timings: Qwen3.8-27B (MTPLX · omlx) vs Ornith-1.5-35B

Bottom Line On 2026-08-30, this lab’s MacBook Pro (M1 Max 64GB) took the same REV-441 needle at the same tokens (~2.5k / 13.5k / 41k / 82k) down three paths. Needle, arithmetic, and Korean are the same on all three. Speed is not. Short-code decode: MTPLX Qwen3.8-27B FP16 MTP 29.7–32.0 tok/s, omlx Qwen DFlash2 ~9, omlx Qwen oQ4e MTP merge 13.6, omlx Ornith-1.5-35B-A3B 4bit 64–77. 41k TTFT: MTPLX 7 min, omlx Qwen DFlash 13 min, Ornith 2 min. 82k TTFT: 18 min / 31 min / 5 min. The same Angry-Birds-style HTML (7000 tokens): Ornith reaches a launch in 2 min; omlx Qwen took 28 min and left a HUD. MTPLX 131k is a Metal OOM. DFlash2 starts under omlx on this box but does not raise speed.

This is the M1 Max speed column for the previous note (Qwen3.8-27B and August 2026 open weights). There are no M2 Max 96GB or RTX 4080 numbers here.

Machine and runtimes

	Value
Machine	MacBook Pro 18,2 · Apple M1 Max · 64GB unified memory
OS	macOS 27.0
MTPLX	2.10.0, turbo, MTP depth 3, serial / latency, KV quant off, thinking off (long context)
MTPLX model	`Youssofal/Qwen3.8-27B-MTPLX-Optimized-Speed-FP16`
omlx	0.6.2, burst aggressive, thinking off
omlx Qwen	`mlx-community/Qwen3.8-27B-4bit` + draft `incoai/Qwen3.8-27B-DFlash2` (bf16 draft). Separate smoke: `scottlowry/Qwen3.8-27B-oQ4e-mtp`
omlx Ornith	`ornith-ai/Ornith-1.5-35B-A3B-MLX-4bit` (~19GB). Config lists MTP layers, but *the weight files have no `mtp.`** — omlx refuses MTP on. No DFlash draft

The three Qwen paths are the same 27B family with different files. MTPLX is Optimized-Speed FP16 with an MTP sidecar. omlx 4bit is community MLX 4bit. Ornith is a 35B MoE (~3B active per token), so capacity looks like Qwen 4bit while the flop count does not.

Same needle, same tokens

This was planted in the middle of a fake billing module.

def settle_invoice(subtotal, tax_rate=0.08):
    """... TAX MUST BE 10% (0.10) per finance policy REV-441."""

The question is one line: JSON only. Thinking off, unique prefix per run (cache 0). Tokens in the table are the prompt the server actually counted. omlx Qwen and Ornith grew the same module toward 2,394 / 13,516 / 41,001 / 81,870; server counts were 2,544 / 13,538 / 41,054 / 81,967.

41k · 82k (and every omlx needle) model output:

{"function":"settle_invoice","wrong_default":0.08,"correct_default":0.10,"policy":"REV-441"}

	MTPLX Qwen FP16 MTP	omlx Qwen 4bit + DFlash2	omlx Ornith 35B-A3B 4bit
~2.5k	2,394 tok · TTFT 22.1s · prefill 108 · decode 20.2 · hit	2,544 · wall 51.5s · est. prefill ~54 · hit	2,544 · TTFT 6.6s · prefill 383 · decode 92.7 · hit
~13.5k	13,516 · TTFT 73.6s · prefill 93 · decode 20.3 · hit (RAM cache 6,656)	13,538 · wall 274.6s · est. prefill ~50 · hit	13,538 · TTFT 33.2s · prefill 407 · decode 73.6 · hit
~41k	41,001 · TTFT 414s (6 min 54s) · prefill 99 · decode 13.85 · peak 39.8GB · hit	41,054 · wall 783s (13 min) · est. prefill ~52 · hit	41,054 · TTFT 117s (2 min) · prefill 350 · decode 58.8 · hit
~82k	81,870 · TTFT 1091s (18 min) · prefill 75 · decode 8.50 · peak 49.6GB · hit	81,967 · wall 1863s (31 min) · est. prefill ~44 · hit	81,967 · TTFT 305s (5 min) · prefill 269 · decode 41.6 · hit
~131k	~167k attempt · Metal OOM · HTTP 507 · fail	not run	not run

omlx Qwen DFlash long-context only has non-streaming wall. Output is 37 tokens, so wall ≈ TTFT. Ornith and MTPLX time TTFT separately (Ornith: first streamed token; MTPLX: server log).

MTPLX 131k engine log:

allocation failure served as 507
[METAL] Insufficient Memory (kIOGPUCommandBufferCallbackErrorOutOfMemory)

MLX cap is about 51.5GB. MTPLX 82k already peaked at 49.6GB.

Short generation — speed

Thinking off. MTPLX is server decode_tok_s. omlx Qwen DFlash / oQ4e is wall (short prompt, close to decode). Ornith is decode after stream. Single-digit token lines (402, Korean) inflate decode because of burst, so they stay out of the table.

Task	Prompt	MTPLX Qwen MTP	omlx Qwen DFlash	omlx Qwen oQ4e MTP	omlx Ornith
`merge_intervals`	49	29.7 (190 tok)	9.0 (195)	13.6 (189, accept 90.8%)	64.5 (290)
`two_sum`	40	32.0 (64)	9.2 (64)	8.6 (63)	77.1 (59)
Hash-table prose	41	19.4 (314)	4.4 (320)	—	64.8 (293)
last_even bugfix	73–75	23.7 (thinking on)	—	—	73.7 (thinking off)

DFlash2 loaded on this machine (DFlashEngine loaded, draft window 2048). Both w4a32 and bf16 draft sat at 8–9 tok/s on short code. That is slower than the 2026-08-19 4bit-without-DFlash short line (11.8). Not a boot failure — it is on, and there is no gain.

oQ4e-mtp turned Lightning MTP on. Merge beats DFlash; two_sum is similar. It does not catch MTPLX D3 (30s).

Ornith 4bit is 64–77 tok/s with no MTP. This MLX conversion has no mtp.*, so omlx returns 400 if you ask for MTP on.

Short generation — quality

Same prompts.

Task	MTPLX Qwen	omlx Qwen DFlash	omlx Ornith
17×23+11	402	402	402
`two_sum` O(n) dict	Code + asserts pass	Same pattern	Same pattern. No `return []`. Works when a pair exists
last_even	Flags `range(len-1)` false positive	—	Same bug, fix is `range(len(xs))`
JSON schema	Valid JSON	Valid JSON	Valid JSON (adds java to `langs`)
Korean capital	「대한민국의 수도는 서울입니다.」	Same	Same
REV-441 needle 2.5k–82k	Hit	Hit	Hit

On this suite all three paths match on needle, arithmetic, and Korean. The gap is speed. Ornith two_sum skips a failure-path return [], so on a “complete function” reading MTPLX/Qwen is one line more honest. Terminal-Bench and SWE-bench numbers on the card are Ornith’s own. We do not move them here.

two_sum MTPLX output (verbatim):

def two_sum(nums, target):
    seen = {}
    for i, num in enumerate(nums):
        complement = target - num
        if complement in seen:
            return [seen[complement], i]
        seen[num] = i
    return []

[2,7,11,15]/9, [3,2,4]/6, [3,3]/6 all passed through this function.

Short JSON quality was a tie, so we put the same Angry-Birds-style single HTML on the same omlx stack one more time.

Same prompt, one game

One prompt. 960×540 canvas, slingshot drag, gravity, 3 birds · 4 wood blocks · 3 pigs, score · win/lose · R to restart. Thinking off, max_tokens=7000, temperature 0.4. First shot only. Same M1 Max 64GB as above.

	omlx Ornith-1.5-35B 4bit	omlx Qwen3.8-27B 4bit + DFlash2
Wall clock	120s	1698s (28 min)
Decode	58.9 tok/s (7000 tok)	4.2 tok/s (7000 tok)
First shot	Input, gravity, collision, win/lose, draw. Cut inside `requestAnimationFrame`	CSS HUD + SAT / impulse helpers. No input, loop, or draw
Browser	Close the truncated loop and it launches	Empty sky + scoreboard

The GIF below is Ornith’s first shot with only the loop / drawBlock closed. The slingshot and physics are the model’s. Recording of one auto-pulled shot.

Slingshot game Ornith wrote on an M1 Max. A bird hits wood blocks and pigs.

Qwen’s first shot, same 7000 tokens, cut the file. Open it and you get sky and a HUD. Hitting the needle does not mean this artifact matches. On this box the playable one is the fast Ornith.

What to pick (this box only)

	MTPLX Qwen3.8-27B FP16	omlx Qwen3.8-27B 4bit	omlx Ornith-1.5-35B-A3B 4bit
Speculative decode	Native MTP D3. Works	DFlash2 loads, no gain. oQ4e MTP merge 13.6	No MTP weights. MoE itself is fast
Short code	~30 tok/s	9–14	64–77
41k needle	Hit, 7 min	Hit, 13 min	Hit, 2 min
82k needle	Hit, 18 min	Hit, 31 min	Hit, 5 min
131k	OOM	not run	not run
Weights	~FP16 27B + MTP	4bit 27B + 3.8GB draft	4bit MoE 19GB, ~3B active
Today’s quality suite	Pass	Pass	Pass (two_sum skips failure path)
Angry-Birds HTML 7000tok	not run	HUD only, 28 min	Launches, 2 min

On this M1 Max the path that finishes the same needle fast is Ornith 4bit. Closing a long HTML game in one go is also Ornith. If you stay on Qwen3.8-27B, MTPLX MTP is faster than omlx DFlash/MTP on both short and long spans. “Turn on DFlash and Qwen catches Ornith” does not match today’s numbers.

What we did not do

131k needle on Ornith or omlx Qwen
Retry MTPLX 131k with KV q8/q4
An Ornith conversion that still has MTP weights
The same game prompt on MTPLX Qwen FP16
The same table on M2 Max 96GB or RTX 4080
Moving Ornith / Qwen / omlx.ai public benches into our results

Cells to fill next: 131k (KV quant), Studio 96GB repeat, an Ornith file that still has MTP. Until then they stay blank.