Repo: https://github.com/milla-jovovich/mempalace
Commit reviewed: 1782628
Date: 2026-04-08
I cloned the repo, installed it in a fresh uv venv, ran the tests, built the package, and then ran a set of smoke tests and one fuller end-to-end example to validate the README and surface rough edges.
cd /root/source/mempalace
uv venv .venv
. .venv/bin/activate
uv pip install -e '.[dev]'
pytest -q
python -m buildOutput:
......... [100%]
9 passed in 0.81s
...
Successfully built mempalace-3.0.0.tar.gz and mempalace-3.0.0-py3-none-any.whl
So: real project, installs fine, tests pass, builds fine. Not fake.
The core tool works, but the repo makes bigger claims than the code really earns.
The pattern that came up repeatedly:
- decent working CLI core
- thin test surface relative to the README swagger
- architecture language that is more elaborate than the implementation
- benchmark story that is not nearly as clean as the README presentation suggests
- some very visible quality-control misses
To show what the tool actually does when given a substantial amount of memory-like material, I created a synthetic transcript with 30 exchanges / 60 facts about a fictional founder and company.
The full transcript is included in this gist as nora_profile.md.
cd /root/source/mempalace
. .venv/bin/activate
PAL=/root/source/mempalace-review-artifacts/e2e-palace
rm -rf "$PAL"
mempalace --palace "$PAL" mine /root/source/mempalace-review-artifacts/e2e --mode convos --wing nora-demo
mempalace --palace "$PAL" status
mempalace --palace "$PAL" wake-up --wing nora-demo=======================================================
MemPalace Mine — Conversations
=======================================================
Wing: nora-demo
Source: /root/source/mempalace-review-artifacts/e2e
Files: 1
Palace: /root/source/mempalace-review-artifacts/e2e-palace
───────────────────────────────────────────────────────
✓ [ 1/1] nora_profile.md +30
=======================================================
Done.
Files processed: 1
Files skipped (already filed): 0
Drawers filed: 30
By room:
technical 1 files
Next: mempalace search "what you're looking for"
=======================================================
=======================================================
MemPalace Status — 30 drawers
=======================================================
WING: nora-demo
ROOM: technical 30 drawers
=======================================================
Two things are true at once:
- It works: the transcript was chunked into 30 drawers and stored.
- The rooming is crude: all 30 facts — personal life, pricing, outages, roadmap, travel habits — got dumped into
technical.
That is a decent demonstration of the repo in miniature. The core filing/search loop is real; the higher-level semantic organization is much flimsier than the README rhetoric.
These are straight CLI runs against the mined 60-fact transcript.
mempalace --palace "$PAL" search "What is Nora's dog's name?" --results 1[1] nora-demo / technical
Source: nora_profile.md
Match: 0.426
> What is Nora's birthday and pet?
Nora was born on 14 February 1991 and her dog is a border collie named Basil.
mempalace --palace "$PAL" search "Where does Nora live?" --results 1[1] nora-demo / technical
Source: nora_profile.md
Match: 0.239
> Who is Nora Levin?
Nora Levin is the founder of Northstar Labs and she lives in Hobart, Tasmania.
mempalace --palace "$PAL" search "Which database replaced MongoDB?" --results 1[1] nora-demo / technical
Source: nora_profile.md
Match: -0.296
> What stack does Lantern use now?
Lantern moved from MongoDB to PostgreSQL and the API is written in Go.
Retrieval is correct. Presentation is still stupid: the result is right, but the printed match score is negative.
mempalace --palace "$PAL" search "Who signs payroll?" --results 1[1] nora-demo / technical
Source: nora_profile.md
Match: 0.253
> Who are the executives and what do they do?
Priya Shah is the COO and she signs payroll every second Tuesday.
mempalace --palace "$PAL" search "Which customer found the March CSV export bug?" --results 1[1] nora-demo / technical
Source: nora_profile.md
Match: 0.434
> Which customer found a bug and what bug was it?
Fern Health found the March CSV export bug and the bug duplicated discount rows.
mempalace --palace "$PAL" search "How long did the May outage last?" --results 1[1] nora-demo / technical
Source: nora_profile.md
Match: 0.334
> What happened during the May outage?
The May outage lasted 47 minutes and it started with a bad RabbitMQ rollout.
mempalace --palace "$PAL" search "What conference talk is Nora giving?" --results 1[1] nora-demo / technical
Source: nora_profile.md
Match: 0.529
> What conference plans does Nora have?
Nora is speaking at SaaS Down Under in July and her talk is titled Boring Systems Win.
mempalace --palace "$PAL" search "When does Nora's passport expire?" --results 1[1] nora-demo / technical
Source: nora_profile.md
Match: 0.315
> What personal reminder matters before that conference?
Before the conference Nora needs to replace her passport because it expires in November.
mempalace --palace "$PAL" search "Which cloud storage service do they use?" --results 1[1] nora-demo / technical
Source: nora_profile.md
Match: 0.222
> What cloud setup does the team use?
Production runs on Fly.io and object storage lives in Cloudflare R2.
mempalace --palace "$PAL" search "What customer support person owns the electronic prescriptions rollout?" --results 1[1] nora-demo / technical
Source: nora_profile.md
Match: 0.032
> What roadmap item is due in June and who owns it?
Electronic prescriptions are due in June and Marta owns the rollout checklist.
mempalace --palace "$PAL" search "What is the average annual contract?" --results 1[1] nora-demo / technical
Source: nora_profile.md
Match: 0.061
> What is the pricing model and average contract size?
Lantern charges per store per month and the average annual contract is 42,000 Australian dollars.
mempalace --palace "$PAL" search "Which airline seat does Nora prefer?" --results 1[1] nora-demo / technical
Source: nora_profile.md
Match: 0.472
> What is her travel habit and favorite airline seat?
Nora always takes handwritten notes on flights and prefers seat 12A whenever she can get it.
This is the fairest read of the project I can give:
- It can ingest a substantial chunk of semi-structured memory.
- It can retrieve specific facts from that memory with useful reliability.
- It returns verbatim source text, which is good.
- It has almost no meaningful higher-order organization in this path beyond chunking + embeddings + metadata.
- The UX polish is rough enough that even good retrieval often gets presented with nonsense scores or nonsense categorization.
The wake-up output for this 60-fact transcript is basically the first 15 stored chunks, because the usual miners do not assign meaningful importance/weight metadata.
## L1 — ESSENTIAL STORY
[technical]
- > Who is Nora Levin? Nora Levin is the founder of Northstar Labs and she lives in Hobart, Tasmania. (nora_profile.md)
- > What is Nora's birthday and pet? Nora was born on 14 February 1991 and her dog is a border collie named Basil. (nora_profile.md)
- > Who is in Nora's family? Nora's partner is Eli Mercer and her younger sister is Tessa Levin. (nora_profile.md)
- > What does Nora drink and avoid? Nora drinks a flat white every morning and she cannot stand fennel. (nora_profile.md)
...
That's not an “essential story” so much as “top of file until char budget runs out.”
mempalace --palace "$PAL" search "ukulele sexual compulsions high school reunion" --results 3[1] nora-demo / technical
Source: nora_profile.md
Match: -0.74
> What does Nora drink and avoid?
Nora drinks a flat white every morning and she cannot stand fennel.
[2] nora-demo / technical
Source: nora_profile.md
Match: -0.806
> What instruments and sports does Nora like?
Nora plays the cello on Sundays and she swims at Sandy Bay twice a week.
Again: the retrieval engine returning something vaguely nearest-neighbor-like is normal. Printing negative match values is still crap.
Command:
mempalace init /root/source/mempalace-review-artifacts/project --yesOutput:
Scanning for entities in: /root/source/mempalace-review-artifacts/project
Reading 2 files...
No entities detected — proceeding with directory-based rooms.
=======================================================
MemPalace Init — Local setup
=======================================================
WING: project
(2 files found, rooms detected from folder structure)
ROOM: documentation
Files from docs/
ROOM: src
Files from src/
ROOM: backend
Files from api/
ROOM: general
Files that don't fit other rooms
───────────────────────────────────────────────────────
Review the proposed rooms above.
Options:
[enter] Accept all rooms
[edit] Remove or rename rooms
[add] Add a room manually
Your choice [enter/edit/add]:
Traceback (most recent call last):
...
EOFError: EOF when reading a line
That is a plain bug. If a flag says --yes, it should actually be non-interactive.
Config says collection name is configurable:
# mempalace/config.py
@property
def collection_name(self):
return self._file_config.get("collection_name", DEFAULT_COLLECTION_NAME)I set:
{
"palace_path": "/root/source/mempalace-review-artifacts/custom-palace",
"collection_name": "custom_drawers"
}Then mined a sample project with HOME pointed at that config.
Check:
configured_collection= custom_drawers
actual_collections= ['mempalace_drawers']
So the app advertises one collection, then writes to another.
Hardcoded call sites include:
mempalace/miner.pymempalace/convo_miner.pymempalace/searcher.pymempalace/cli.pymempalace/layers.py
This is not a cosmetic inconsistency. It means config compatibility is broken.
The graph module says:
Nodes = rooms
Edges = shared rooms across wings (tunnels)
Edge types = halls (the corridors)
But ordinary mined items do not actually carry hall metadata.
Example metadata from normal convo mining:
meta[1] hall= None
{
"added_by": "mempalace",
"chunk_index": 0,
"extract_mode": "exchange",
"filed_at": "2026-04-08T06:25:49.816161",
"ingest_mode": "convos",
"room": "technical",
"source_file": "/root/source/mempalace-review-artifacts/convos/chat1.md",
"wing": "convos"
}
Graph build on the same mined data:
node_count= 1
edge_count= 0
technical halls= []
And the graph code really does depend on hall:
# mempalace/palace_graph.py
hall = meta.get("hall", "")
...
for hall in data["halls"]:
edges.append(...)So the README mythology about halls/tunnels is much stronger than the ordinary data path that actually feeds the graph.
This fails:
mempalace mine /root/source/mempalace-review-artifacts/project --palace /root/source/mempalace-review-artifacts/palaceOutput:
usage: mempalace [-h] [--palace PALACE]
{init,mine,search,compress,wake-up,split,status} ...
mempalace: error: unrecognized arguments: --palace /root/source/mempalace-review-artifacts/palace
Only this works:
mempalace --palace /root/source/mempalace-review-artifacts/palace mine /root/source/mempalace-review-artifacts/projectNot fatal, but absolutely the kind of thing users trip on immediately.
Check:
pyproject= 3.0.0
__init__= 2.0.0
Files:
# pyproject.toml
version = "3.0.0"# mempalace/__init__.py
__version__ = "2.0.0"Small thing, but it signals release discipline is loose.
README:
This gives your local model ~170 tokens of critical facts
Code:
Layer 0: Identity (~100 tokens)
Layer 1: Essential Story (~500-800)
Wake-up cost: ~600-900 tokens (L0+L1)
Those numbers do not sit cleanly together.
This is the biggest credibility problem in the repo.
From benchmarks/longmemeval_bench.py:
Hybrid V4: hybrid_v3 + three targeted fixes for the final 3 misses.
Miss 1 — 'high school reunion' ...
Fix: Add memory/nostalgia patterns ...
Miss 2 — 'Rachel/ukulele' ...
Fix: Extract capitalized proper nouns from question; boost sessions containing them.
Miss 3 — 'sexual compulsions' ...
Fix: Extract quoted phrases from question; boost sessions containing exact quotes.
That is not a clean "general breakthrough" story. That is direct tuning against known misses.
You can call that practical engineering if you like. Fine. But then the README chest-beating needs to come down several notches.
Examples:
# benchmarks/convomem_bench.py
# Bypass SSL for restricted environments
ssl._create_default_https_context = ssl._create_unverified_context
and:
# benchmarks/locomo_bench.py
keys_path = os.path.expanduser("~/.config/lu/keys.json")
That does not look like a clean, polished, publicly reproducible benchmark suite. It looks like a research notebook that escaped into the repo.
To be fair:
- the package installs and builds cleanly
- the basic CLI flow is real
- the core idea of storing verbatim memory and making it searchable is directionally good
- the code is mostly readable
- this is a real OSS tool, not vaporware
If the README were calmer and the benchmark claims were more honest, I would be a lot friendlier to it.
As a usable OSS memory tool: 7/10
As a rigorously presented benchmark/research story: 4/10
The repo is real. The swagger is doing too much work.
I doubled the synthetic dataset and made it nastier: two founders, two companies, similar names, similar executives, similar incidents, similar timelines, similar product codenames.
Dataset file in this gist: nora_norah_confusables.md
It contains paired facts like:
- Nora Levin / Norah Levene
- Northstar Labs / Northlight Health
- Priya Shah / Priya Singh
- Jonas Reed / Jonah Reid
- Hana Kim / Hannah Kim
- Fern Health / Fearn Medical
- Lantern / Atlas Green / Lattice / Atlas Grey
- Fly.io + R2 / Render + S3
- SvelteKit + RabbitMQ / Next.js + NATS
=======================================================
MemPalace Mine — Conversations
=======================================================
Wing: confusable-demo
Source: /root/source/mempalace-review-artifacts/confusable-e2e
Files: 1
Palace: /root/source/mempalace-review-artifacts/confusable-palace
───────────────────────────────────────────────────────
✓ [ 1/1] nora_norah_confusables.md +62
=======================================================
Done.
Files processed: 1
Files skipped (already filed): 0
Drawers filed: 62
By room:
technical 1 files
=======================================================
MemPalace Status — 62 drawers
=======================================================
WING: confusable-demo
ROOM: technical 62 drawers
Again, everything lands in technical. Personal facts, roadmap facts, board facts, outage facts, travel habits — all one bucket.
These were aimed at disambiguation, multi-constraint retrieval, or mild multi-hop reasoning.
Query:
mempalace --palace "$PAL" search "Which Priya signs payroll every second Tuesday?" --results 3Top result:
[1] confusable-demo / technical
Match: 0.119
> Who is the COO at Northstar and what recurring task does she handle?
Priya Shah is the COO at Northstar and she signs payroll every second Tuesday.
Verdict: pass. It distinguishes Priya Shah from Priya Singh.
Query:
mempalace --palace "$PAL" search "Which company uses Next.js and NATS instead of SvelteKit and RabbitMQ?" --results 3Top result:
[1] confusable-demo / technical
Match: 0.101
> What frontend and queue system does Northlight use?
Northlight's dashboard uses Next.js and background jobs run through NATS.
Verdict: pass.
Query:
mempalace --palace "$PAL" search "Who rotated every key within 18 minutes after a GitHub leak?" --results 3Top result:
[1] confusable-demo / technical
Match: 0.332
> What security incident happened at Northstar in January?
An intern committed a test API key to GitHub and Hana rotated every key within 18 minutes.
Verdict: pass. Good discrimination between Hana and Hannah.
Query:
mempalace --palace "$PAL" search "Which founder speaking at SaaS Down Under has a passport expiring in November?" --results 3Top result:
[1] confusable-demo / technical
Match: -0.275
> What personal reminder matters for Nora before her conference?
Before the conference Nora needs to replace her passport because it expires in November.
Verdict: partial. It got the passport fact, but did not retrieve the conference-title chunk that ties this to SaaS Down Under. That's a good example of the system being retrieval-only rather than a true synthesis layer.
Query:
mempalace --palace "$PAL" search "Which company left Heroku because noisy neighbors caused p95 latency spikes?" --results 3Top result:
[1] confusable-demo / technical
Match: 0.35
> What old infrastructure did Northstar leave and why?
Northstar left Heroku in 2024 because noisy neighbors caused p95 latency spikes.
Verdict: pass.
Query:
mempalace --palace "$PAL" search "What is the product that used Atlas Grey and now runs on MySQL with a Rust API?" --results 3Top results:
[1] confusable-demo / technical
Match: 0.205
> What database and API stack does Northlight use now?
Lattice moved from PostgreSQL to MySQL and the API is written in Rust.
[3] confusable-demo / technical
Match: -0.262
> What is Northlight's flagship product and former codename?
Northlight's flagship product is Lattice and its internal codename used to be Atlas Grey.
Verdict: partial. The answer is reconstructable from the top hits, but MemPalace itself is not doing that reconstruction. The user has to synthesize across results.
Query:
mempalace --palace "$PAL" search "Which customer reported the bug that duplicated discount rows?" --results 3Top result:
[1] confusable-demo / technical
Match: 0.302
> Which bug did Fern Health report to Northstar?
Fern Health reported the March CSV export bug and it duplicated discount rows.
Verdict: pass.
Query:
mempalace --palace "$PAL" search "Who runs customer support for the company whose object storage lives in Cloudflare R2?" --results 3Top results:
[1] confusable-demo / technical
Match: 0.134
> What cloud setup does Northstar run on?
Northstar runs production on Fly.io and keeps object storage in Cloudflare R2.
Verdict: fail for end-user purposes. It retrieves the Cloudflare R2 chunk, but not the support-owner chunk. A human can infer the company and go search again; the system itself does not bridge the facts.
Query:
mempalace --palace "$PAL" search "Which company has the noisiest beta customer Riverton Clinics and where is its main office?" --results 3Top results:
[1] confusable-demo / technical
Match: 0.323
> Which customers matter most to Northlight?
Northlight's largest customer is Fearn Medical and its noisiest beta customer is Riverton Clinics.
[3] confusable-demo / technical
Match: -0.178
> What does Northlight Health build and where is its main office?
Northlight Health builds software for radiology clinics and its main office is on King William Street in Adelaide.
Verdict: partial. Again, usable top hits, but no built-in composition of the answer.
Query:
mempalace --palace "$PAL" search "Which founder prefers seat 14F and what pet does she have?" --results 3Top results:
[1] confusable-demo / technical
Match: 0.031
> What travel habit and airline seat does Norah prefer?
Norah color-codes her itineraries and prefers seat 14F whenever possible.
Verdict: partial bordering on fail. It gets the seat fact but not the pet fact. Again, retrieval is local-chunk based.
- Clean pass: 5
- Partial / reconstructable from top hits: 4
- Practical fail: 1
That is honestly not terrible for a tiny local vector-memory tool. It is also nowhere near the mystical “memory palace” framing in the README. The main CLI path is basically embed chunks, store chunks, vector-search chunks, print chunks.
This part is much less magical than the docs make it sound.
At a high level, the normal CLI flow is:
- read files
- chunk them into text snippets (“drawers”)
- attach lightweight metadata (
wing,room,source_file,chunk_index, timestamps) - hand them to ChromaDB
- let Chroma embed and index the text
- query by embedding similarity later
That’s it. No hidden symbolic memory engine in the main path.
The key detail: MemPalace itself does not explicitly create embeddings in the normal CLI path. Chroma does.
Normal ingestion code looks like this:
# mempalace/miner.py
collection.add(
documents=[content],
ids=[drawer_id],
metadatas=[...],
)and in convo mode:
# mempalace/convo_miner.py
collection.add(
documents=[chunk["content"]],
ids=[drawer_id],
metadatas=[...],
)Notice what is not being passed: no embeddings= and no explicit embedding_function= on the collection.
The collection is created like this:
client = chromadb.PersistentClient(path=palace_path)
collection = client.get_collection("mempalace_drawers")
# or create_collection("mempalace_drawers")So Chroma uses its default embedding function.
I checked the installed chromadb package in this environment.
chromadb version: 1.5.6
DefaultEmbeddingFunction -> delegates to ONNXMiniLM_L6_V2
embedding dimension: 384
Specifically, Chroma's DefaultEmbeddingFunction says:
class DefaultEmbeddingFunction(EmbeddingFunction[Documents]):
"""Default embedding function that delegates to ONNXMiniLM_L6_V2."""So in this setup, the normal MemPalace CLI path is relying on:
- ChromaDB for storage + retrieval
- ONNX MiniLM L6 v2 for embeddings
- 384-dimensional vectors
That matters because the repo pitches a grand architecture, but the normal search path is mostly standard Chroma + default sentence embeddings.
Search uses:
# mempalace/searcher.py
results = col.query(
query_texts=[query],
n_results=n_results,
include=["documents", "metadatas", "distances"],
)Passing query_texts= means Chroma embeds the query text using the same collection embedding function and returns nearest neighbors.
So both ingest and query-time embedding are happening inside Chroma, not in MemPalace's own code.
mempalace/miner.py:
- scans readable file extensions
- skips common junk dirs like
.git,node_modules,.venv - chunks by characters with:
CHUNK_SIZE = 800CHUNK_OVERLAP = 100MIN_CHUNK_SIZE = 50
- room assignment uses:
- folder path match
- filename match
- keyword scoring
- fallback
general
mempalace/convo_miner.py:
- prefers a
>-quoted exchange format - makes one chunk from one user turn plus following assistant text
- otherwise falls back to paragraph chunking
- room assignment is crude topic keyword scoring over a few buckets:
technicalarchitectureplanningdecisionsproblems- else
general
That is why the synthetic examples collapse into technical so easily.
mempalace/layers.py describes Layer 1 as selecting high-importance / recent drawers.
But in the normal mined path, drawer metadata is usually just:
wingroomsource_filechunk_indexadded_byfiled_at
The miners do not write meaningful importance, emotional_weight, or weight values.
So Layer 1 usually falls back to a default importance of 3 for every drawer, then takes the first MAX_DRAWERS = 15 after sorting by identical scores.
In practice, that means wake-up is often just a formatted slice of early-ingested chunks, not a genuinely selected “essential story”.
The main search path is not BM25 + vector + rerank.
It is just Chroma similarity search with optional metadata filters:
wingroom
There is no mainline reranker, no explicit lexical fallback, no answer synthesis step, and no graph traversal in the normal CLI search.
That distinction matters because some of the benchmark code contains much fancier custom retrieval logic than the standard user-facing tool.
Search prints:
similarity = round(1 - dist, 3)That assumes Chroma distance behaves like a nice bounded [0,1] similarity gap.
It doesn't necessarily. If distance is greater than 1, this turns into a negative “match” value.
So the bug is not that retrieval is impossible — it's that the repo is presenting raw or semi-raw distance as a user-friendly confidence score without calibrating it.
For ordinary use, MemPalace behaves much more like this:
a thin local RAG memory CLI on top of ChromaDB
than like this:
a deeply structured palace with rooms, halls, tunnels, closets, drawers, and meaningfully layered autobiographical recall
That doesn't make it worthless. It just makes the README theatrical.