BitShovel.Discover. Understand. Stay informed.
Save this中

Local trial · September 8, 2026

Can a local small model resume a project? A list-export trial

The model produced a CSV exporter that passed four checks. Two initial attempts were truncated, and the final handoff missed the required format. Follow a fictional list from old decisions to checked results.

Save this trial · Browse all articles and topics

01Start with an inspectable result

The notes say to change it. Has the file changed?

After a project is paused, its latest discussion and current file can describe different stages. We reused the fictional list from our handoff guide: it originally exports JSON, and a later decision approves CSV for reading rows and columns. The record explicitly says the change is unimplemented; the file still calls json.dumps.

We supplied the decision record and current file to MiniCPM5-2B locally, without the reference implementation or a completed handoff. It must preserve names, quantities and order, with column names even for an empty list. Names contain commas, quotes, Chinese text and line breaks: content the conversion must preserve.

This exercise starts after the relevant material has been found. It does not run the earlier funes retrieval workflow or let the model operate a whole project autonomously. We separately inspect its state answer, generated file and handoff from actual check results.

02Inspect each deliverable

From a state answer to a file and a handoff

The figure pairs what each step receives with what it actually produces. The operator reviewed the file before running the original four checks. The two-row table comes from CSV that the file actually generated and a parser read back. The verifier, candidate file and raw outputs are in the materials.

One list, with results you can inspect at each step

  1. 01 · Separate the decision from the current state

    Supplied material

    Decision record: switch to CSV; not yet implemented.

    return json.dumps(...)
    The current file still outputs JSON.

    Actual answer after clarifying the prompt
    Current format
    JSON
    Approved target
    CSV
    Change implemented?
    No

    The first attempt was truncated without a final answer. These are three fields from the completed follow-up; all eight are in the materials.

  2. 02 · Change the file, then inspect the readback

    Original list to preserve
    NameQty
    笔记本, A52
    卡片 "蓝色"1

    The comma and quotation marks are part of each name.

    Actual CSV readback from the generated file
    NameQty
    笔记本, A52
    卡片 "蓝色"1

    The file obtained with a larger output allowance was reviewed and run. This table comes from its actual output.

    • An empty list still has column names · Passed
    • Quantities and original order survive · Passed
    • Commas, quotes and Chinese text survive · Passed
    • A line break stays inside one record · Passed

  3. 03 · Write the handoff from the checks

    Actual check record supplied

    4/4 checks passed; exit code 0.

    Scope: a local teaching fixture. No production deployment receipt.

    The model’s actual handoff

    Reported four passes, no failed cases and no evidence of production deployment.

    But it added a Markdown code fence, failing the agreed direct JSON parse.

BitShovel’s fictional list. Tasks use Chinese prompts and fresh contexts, with follow-ups recorded separately. The figure shows this trial’s actual outputs and checks; English annotations are translations.

03Keep the first attempts

Both initial attempts were truncated. What changed next?

The first state and implementation attempts each had a 2,048-token output allowance. Tokens are model text units, not word or character counts; generated thinking text uses the allowance too. Both hit the limit without final answers. We retained them as unfinished, without scoring excerpts or executing code found in intermediate text.

The implementation follow-up changed only the output allowance to 4,096, starting a new run with the same input and sampling settings. Its first 2,048 output token IDs exactly matched the initial attempt. This time a complete file arrived beyond the original cutoff. That unchanged candidate then passed four checks.

The state prompt needed a clearer definition: did “an empty input has a header” refer to the current JSON program or the approved CSV requirement? The former has no CSV header; the latter requires one. We clarified whether each field describes current behavior, approved requirements or scope, keeping the 2,048-token allowance. The result was complete JSON with all eight fields matching the material.

The two follow-ups change different conditions, and both initial records remain. Prompt ambiguity is not a factual model error, and this one example does not establish a general benefit from rewriting prompts.

  1. Read the state · first attempt

    Hit the output limit; no final answer

    Allowance 2,048 · Output 2,048 token · 129.9 s

  2. Write the file · first attempt

    Hit the output limit; no final file

    Allowance 2,048 · Output 2,048 token · 128.4 s

  3. Write the file · larger output allowance only

    Produced a file; four checks then passed

    Allowance 4,096 · Output 2,373 token · 81.8 s

  4. Read the state · clarified field meanings only

    Complete JSON; all eight defined fields matched

    Allowance 2,048 · Output 1,457 token · 50.1 s

  5. Write the handoff · first attempt

    Content matched; direct JSON parsing failed

    Allowance 2,048 · Output 655 token · 34.9 s

Times cover generation only, excluding loading, downloads, review and rework. The shorter later run does not show that a larger allowance is faster: cache state and other conditions were uncontrolled. Output counts exclude the end token.

04Content and format

An accurate handoff may still need a format fix

The final task received the actual candidate, verifier output and exit code, and was asked to return only a JSON object. It correctly reported four passes, no failures and no evidence of production deployment. But it wrapped the answer in a Markdown code fence.

A person can read the answer below, but a program expecting JSON directly rejects the extra fence. The original format failure remains. In a separately recorded inspection, we removed only that complete fence and found all five values matched the actual receipt. The processed observation does not turn the original answer into a format pass.

This trial produced a file that passed the teaching checks and a handoff whose content was grounded in them. An automated workflow still needs an explicit way to handle the output format. Recording that remaining step tells the next person exactly where the work stands.

```json
{
  "checks_passed": 4,
  "checks_total": 4,
  "exercise_complete": true,
  "failed_cases": [],
  "production_deployment_evidenced": false
}
```

05Apply the checking method

Start with a small task you can actually check

Prepare the current file and latest approved decision. State what exists, what should change and what remains unverified. Choose a few requirements you can check, then ask the tool for a concrete file or answer. Attach the checks and remaining issues to the handoff. For a writing project, distinguish the current draft, approved edits and actual revisions.

You can read the materials without running anything: inputs, full prompts, five generation runs, the candidate and check records. Replaying local inference needs an Apple Silicon Mac, the matching environment and a separate model download. You can start by inspecting the inputs, final file and readback. Model outputs remain in their original language; the English page is a translation, not a separate English-language model trial.

The exercise covers valid lists, without testing large files, invalid input, a particular spreadsheet app or human time saved. Local execution has a specific meaning here: files were downloaded first, then the operating system denied network access to each inference process, with a recorded denial probe before loading. This describes those processes, not every application or the whole computer.

Download the exercise, prompts and records (ZIP)
Environment and settings

BitShovel’s Codex operator agent prepared the prompts, reviewed the candidate and ran the checks. MiniCPM generated the answers and file.

Apple A18 Pro, 8 GiB memory; official MiniCPM5-2B-MLX with 4-bit quantization at revision 32f8dd5. Weights and supporting files total about 1.43 GB, excluding the software environment. Tasks use fresh contexts, thinking mode, temperature 1.0, top-p 0.95 and seed 42. Each input plus its full output allowance stays within 8,192 tokens.

The actual runtime uses MLX-LM 0.31.3, MLX 0.32.2 and Transformers 5.16.1. Peak MLX allocations across the five runs were approximately 1.72–1.89 GB, not the computer’s total memory use. Versions, hashes and each trial’s conditions are in the materials.