Methods and comparisons · How memory changes the next attempt

What does AI memory retain: facts about you, or lessons for next time?

IBM turns service attempts into operating rules; Tencent trains a specialist to write long-term memory. Remembering, answering correctly, succeeding repeatedly and saving time or money need different evidence.

IBM published its service experiment account on October 20; Hy-Memory-30B-A3B's paper and weights appeared October 15. We compare them on October 21. These are developer approaches requiring an agent or memory framework, rather than a universal chat-history switch. The numbers are author-reported experiments. We checked pinned rules, configuration and result tables without rerunning the models.

Published Oct 21, 2026

An editorial comparison: facts involve addresses, timing and corrections; experience involves rules, conditions and reuse. — Editorial illustration distinguishing two tasks, not a product interface or experiment result. Facts can change; lessons have conditions. Both need checking when used next time. · Open full-size image

In this articleContents4

中文

After a refund, retain a lesson for next time

Explain the outcome, amount and next steps after a refund; itemize proposed order changes before seeking confirmation. These are lessons in IBM's retail rule file. ALTK-Evolve retains operating experience from attempts for later service tasks, rather than copying one customer's entire conversation back into the model.

The pinned code filters rules by support and appends the resulting list after the domain policy. The retail file contains 292 rules, including 31 with support of at least three. Model weights remain unchanged. The added playbook can be opened, filtered and inspected; its advice still needs to fit the current business policy.

Open the actual rule file

Sources and further reading

IBM: learning rules from service attemptsResearch-team article · October 20Experiment aims, airline rule selection and the authors' interpretation, read alongside pinned results.
The actual retail rule setPinned commit · 292 rulesIdentity checks, transaction confirmation and refund explanations; 31 rules have support of at least three.
How rules enter the promptPinned commit · injection and filtering codeRules are filtered by support and appended after the domain policy, without weight updates.

中文

Reliability, fewer turns and lower cost are different results

Across the authors' 40 held-out retail tasks, the share succeeding in all four trials rises from 57.5% to 67.5%. That tests repeatability beyond getting one attempt right, but still uses simulated customers and prescribed tasks. In the GPT-4.1 airline experiment, more guidance is not always better: 14 selected rules reach 45% all-four success, versus 40% with all 101 rules.

In the full retail base table, mean turns fall from 11.02 to 10.67, while total cost rises from $0.0842 to $0.1393 and recorded mean simulation duration goes from 107.2 to 119.8 seconds. Costs include agent and simulated customer; runtime also depends on the environment. These are not production-service estimates, but fewer turns clearly do not guarantee lower cost or latency.

The ranking claim has another boundary: the base split includes training tasks seen during rule learning. Both submission PRs remain unmerged when checked October 21, and the author discloses modified prompts and an expected Unverified listing. We use the controlled comparisons, rather than turning submissions into verified BitShovel model ranks.

Compare the original split, cost and duration tables

Authors' original chart: all-four success rises from 57.5% to 67.5% on 40 held-out retail tasks; on 20 airline tasks, selected rules reach 45% versus 40% for all rules. — IBM's original figure, with axes and conditions intact. The top row is mean accuracy; the bottom requires success in all four attempts. Both use a GPT-4.1 agent and simulated customers. The useful amount of guidance differs by domain. · Open full-size image

Sources and further reading

IBM: learning rules from service attemptsResearch-team article · October 20Experiment aims, airline rule selection and the authors' interpretation, read alongside pinned results.
Original retail split, cost and duration tablesPinned commit · author experiment recordSeparates 40 held-out from 114 base tasks, and agent cost from simulated-user cost and batch duration.
Retail submission and disclosuresLeaderboard-submission PR · September 14114 tasks, four trials, modified prompts and a base split containing training tasks; still unmerged when checked October 21.
Airline submission and verification statusLeaderboard-submission PR · September 21The author expects an Unverified listing; an unmerged submission is not a verified leaderboard win.

中文

Another kind of memory decides what deserves to be stored

Hy-Memory-30B-A3B tackles a different step: writing conversation or work into memory that can help later. It is a specialist with roughly 30 billion total and three billion active parameters, operating inside existing frameworks while leaving the answer model in place. As an editorial example, a move should update place and time rather than blend old and new addresses. This illustrates the task; it is not our product test.

Tencent changes only the writer, holding retrieval, answer model and judging fixed across four harnesses and fixed 20% benchmark subsamples. The equal-weight macro average is 55.49 versus 54.62 for Opus 5, with some individual metrics still lower. This supports studying specialist writing, rather than a claim of superior general chat, coding or every long-term task. These scores cannot be combined with IBM's service all-four success rates.

Read metric breakdowns and the model entry

Tencent's original chart compares memory-writer macro averages across four harnesses: Hy-Memory scores 55.49 and Opus 5 scores 54.62. — Tencent's original figure uses fixed 20% subsamples, averaging six metrics and then four harnesses equally. The axis breaks between 0 and 30, visually enlarging small differences. The 0.02- and 0.01-point margins in Hy-Memory and MemOS are not statistically significant according to the paper. This is not a general model ranking. · Open full-size image

Sources and further reading

Hy-Memory model card and metric breakdownsOfficial model card · pinned revisionReplaces only the memory writer across four fixed harnesses and six metrics; the comparison uses Opus 5, not Opus 5.5.
Hy-Memory: why writing memory needs specialist trainingAuthors' paper v1 · October 15Writer, retriever and reader roles, over-storage controls and limitations of small margins.

中文

Inspect the next answer—and whether memory can be corrected

Both approaches suggest making memory inspectable: does a rule retain its conditions, does a fact preserve its timing and changes, and which record influenced the next answer? Tencent's framework separates work facts from reusable experience and exposes source/project isolation and create, update and delete operations. Interfaces alone do not establish adoption or successful everyday use by a personal assistant.

With an actual assistant, follow one small thread: inspect what was stored, correct an outdated fact, then ask a new-session question that depends on the change. After deletion, check whether it is still used. This is our proposed observation method, not a claim that the research proves correction, forgetting or end-to-end savings for any product. The Muse topic follows visible inspect-and-edit controls.

Continue to Muse's inspect-and-correct controls

Sources and further reading

Hy-Memory: why writing memory needs specialist trainingAuthors' paper v1 · October 15Writer, retriever and reader roles, over-storage controls and limitations of small margins.
Framework sources, scenes and operationsTencent repository README · pinned commitSeparate facts and reusable experiences, project/source isolation and CRUD interfaces; this does not establish adoption by a consumer product.