estimated visitors
Article · 3 MIN READ
Dev Blog: ProjectMan Intake Infrastructure
A behind-the-scenes look at the notebook migration, medallion lineage, operations console, and Org intake helper work landed for ProjectMan.
Twenty notebooks are a migration problem
The starting point was a real archive: roughly twenty Org notebooks, deep hierarchies, and large flat intake files. Copying notes by hand would not be enough. We needed to preview subtrees, summarize clusters, preserve the source evidence, and let a person decide where uncertain material belonged.
This work turned that problem into a phased intake system with review, rollback, and operating controls. The first architecture slices are in place; the next challenge is making the workflow comfortable against real notebooks.
Give each notebook a proper home
Roadmap 19 established first-class notebook support: records and roots, notebook-scoped browsing and import/export, safe merge and replace imports, source backup manifests, and cross-notebook subtree moves. Capture, workflow filters, and Workbench administration now understand notebooks too.
Intake, Domestic, Journal, Projects, and Events can be separate working domains instead of pretending every heading belongs to one global tree.
Preserve the source as the data changes
Roadmap 20 added three layers with different jobs:
| Layer | What it preserves | Why it matters |
|---|---|---|
| Bronze | Source manifests, batches, file digests, quarantine, and archive metadata | Know what arrived |
| Silver | Current and historical notebook and heading records | Inspect how normalized entities changed |
| Gold | Publication dependencies and freshness | Explain what a derived view was built from |
flowchart TD
Source[Org notebook archive] --> Bronze[Bronze source evidence]
Bronze --> Silver[Silver current state and history]
Silver --> Gold[Gold derived publications]
Silver --> Lineage[Point-in-time and lineage reads]
Unrestricted history and lineage remain behind full-trust gates. Preserving evidence does not mean exposing it to every reader.
Make the architecture operable
Roadmap 21 brought migration, lineage, quarantine, freshness, rollback, privacy, sync, export, and assistant review into the Workbench operations console.
The useful controls include guarded staging rollback previews and applies, privileged-read audit trails, manual notebook file/folder sync with drift detection and backups, and a durable queue for assistant suggestions. Those controls give the architecture a place where someone can actually operate it.
A migration companion in the terminal
The Org intake helper now supports the main stages of the job:
- Inspect: report, scan, review, summarize, and preview hierarchy guidance.
- Decide: validate, mark, and rescan while merging recorded decisions.
- Prepare: export captures, hierarchy, and summaries; verify staging against source evidence.
- Deliver: dry-run or post captures and register summaries.
- Measure: run benchmark fixtures and profile large-file behavior.
Stable hashes and fingerprints connect decisions to their sources. The helper saves decisions atomically, diagnoses partial JSON, restores source truth during export, compares artifacts exactly, and binds staging evidence back to Bronze. Rich keeps the terminal workflow readable.
Review the boundaries before calling it delivered
Code review and acceptance checks exposed places where lineage needed tighter trust boundaries, staging needed source-backed verification, dry runs needed protection against production mutation, and large-file behavior needed measurement. Those findings were addressed before the documentation was marked implemented.
ProjectMan does not automatically know where every note belongs. It can now preserve the archive, show enough context for a person to decide, record the decision, and leave a trail for reprocessing.
The next work is ergonomic: useful subtree previews, visible excerpts, clusters of messy intake, protection for already-organized hierarchies, and an easy way to send uncertain material back through Intake.
