CARD · Card record
Perplexity open-sources Lily, a Metal inference server for exactly one checkpoint
Perplexity open-sourced Lily (Apache-2.0) in the pplx-garden repo: a Rust+Metal Mac inference server whose load-time validation admits exactly one checkpoint — mlx-community/Qwen3.6-35B-A3B-4bit with the HF revision pinned; any other architecture or quantization is rejected. It exposes a minimal OpenAI chat-completions subset, decodes greedily, keeps a two-entry LRU prefix cache, and requires Apple GPU family 10+ with macOS 26+. 704 stars at capture. AlphaSignal reports 1.35x over MLX-LM; the README states no such number but ships dated benchmark reports with reproduction steps.
01
THE STORY · The original
Read the explanation and its limits.
- Originalofficial repositoryhttps://github.com/perplexityai/pplx-garden/tree/main/lily
Why it is worth understanding
An official open-source capability teams can exercise the same day: one checkpoint plus fail-closed load validation is a pointed engineering bet for local inference on Apple Silicon, and the pinned weight revision under Apache-2.0 makes the benchmark reproducible by anyone. The 1.35x figure is aggregator-reported, but the repo ships reproduction steps to check it.
Conditions and limitations
Evidence: Perplexity's official repository README plus AlphaSignal and r/LocalLLaMA aggregation (BitShovel fetched and verified the repo on 2026-09-03). The 1.35x speedup does not appear in the README — it is an aggregator claim awaiting independent reproduction; the support surface is narrow (M5-and-newer Apple silicon, macOS 26+, a single 4-bit checkpoint); stars are attention, not proof of quality; this is not an independent review.
Reading progress is saved in this browser. Save or add a note →
02
THE THREAD · Full timeline
2 events, every row traceable.
A card accumulates records as its story develops: new corroborators, market contrasts, material changes at the original — each dated and linked to its archive day.
- Added to BitShovelEntered the BitShovel radar: 100 entries visible on AlphaSignal daily at collection.
- RecordCorroboration added: AlphaSignal carried the same story this window (title match).
03
SOURCES · Sources and evidence
Check each link's purpose and limits.
Sources and evidence · 3 links
Each link has a purpose and scope. Link counts do not establish independent confirmation; direct support applies only to the statements specified below.
- Perplexity's official repository, surfaced via AlphaSignal and r/LocalLLaMAScope not documented
A per-link statement of support has not been attached.
- r/LocalLLaMA community threadScope not documented
A per-link statement of support has not been attached.
- AlphaSignalScope not documented
A per-link statement of support has not been attached.