I have been thinking about how to provide my Hermes agent with more context about me since we’ll be working together a lot. I figure the more it knows, the better we’ll be able to collaborate. It occurred to me that the largest somewhat structured sources of insight would be things I have written. I started with this blog since it goes back 16 years and then tried an experiment using archived emails.
Blog Extraction
I gave Hermes a high level goal to read and analyze my blog to propose high-signal memories for openViking to ingest: durable observations like generalities, recurring patterns, significant projects, and long-running interests. I set this up with a main agent doing retrieval, ontology, rubric and final synthesis and subagents doing parallel review of subject-matter clusters and reporting back. Subject to my review and a bit of redaction, the final memories were then submitted to openViking for ingestion.
In all, it only created 16 memories, the first of which is:
Blog-derived public memory approved by Jack: Jack’s public blog “Dhakajack” at blog.templaro.com is a long-running public journal/lab-notebook/reference archive for amateur radio, interactive fiction, electronics/maker projects, software/web experiments, travel/life abroad, and Templaro.com infrastructure history; it spans at least 2010–2026 and should be treated as public source material, not private memory. Sources saved with it: Setting up the blog – Dhakajack; No QSOs in Boca Raton – Dhakajack; Controlling DTR in virtual Windows – Dhakajack; Migration – Dhakajack'
The other memories provide additional detail about my general interests. I had asked for high-level abstraction, and that’s what I got. The blog does not capture all aspects of my life, but it does reasonably reflect my hobbies. In that domain, Hermes did a good job.
I specifically did not want Hermes to pull out details of every blog post (every SOTA peak activated, every IF game reviewed) because that specific information is unlikely to be needed but also because that information will always be available (for the life of the blog at least and perhaps afterwards as a ghost haunting the Internet Archive).
Email Analysis
I have now started a second project along similar lines, but this time ingesting the last year’s worth of outgoing email. The task is to look at content, with whom I correspond with, and the patterns of how I correspond, e.g., my style.
Since I want to keep this information private, I set up my agent to orchestrate processing via local models on the mac – the agent is in charge, but doesn’t ever see email content. I chopped this pipeline into several stages: extraction, summarization, synthesis across all batches, QA (to flag confidential items, weak conclusions, etc.), and a packet for final human review.
The “grunt work” is done by QWEN3.5-9B running with temperature 0, but synthesis and the final summary stage is performed by gemma-4-26B-A4B with a low-but-not-zero temperature value. Both models are MLX, 4bit variants that I have worked with for a while. I did some benchmarking with other models and with various settings and finally locked these two models down for the run.
I learned a few important things. With Hermes serving as orchestrator, it was possible to start these local models with any parameters desired, e.g., context length, temperature, max tokens, etc. On the first run, when I was keeping an eye on system resources as well and noticed memory getting squeezed, I realized that Hermes did not unload one model from memory before switching to the next, so on the next pass, I made that explicit.
Hermes (with GPT 5.5 endpoint at the time) did a good job planning the run, but it needed to be reined in somewhat for my purposes. It always wants to create a transparent, auditable process that would remain on the hard drive for all time, but I’m not in a corporate environment and I don’t need disproportionate overhead. I don’t blame it for trying to impose best practices, but I whittled down the initial proposal significant. I retained the QA step out of guilt and because I wanted to see what kind of issues it surfaced, at least initially.
Running locally has obvious security benefits but it is pretty darn slow. On my hardware there is no option to make it more parallel, the GPU is pegged. I tried playing around with the extraction schema since that it the most time-intensive element, but I did not want to trim much since everything downstream relies on accurate extraction. I did eliminate the number of extracted examples and capped the length of summary descriptions. That helped a little but was balanced by me also modifying the extraction to rely less on fixed classifiers and to be more open-ended. I didn’t want later analysis to be constrained by an assumed ontology.
Going from a few test emails to an entire month shook loose some faults. About 6% of extraction failed, usually due to malformed JSON. This is something that is checked on each processed email and the ones that fail are run through a more minimal and strict prompt, which is always successful. Processing a month of email (around 300-400 outbound emails) takes about 16 hours with this pipeline.
After letting the agent supervise the run month-by-month for two months worth of email, I asked it to write a script to process the remaining months. In full paranoia mode, the script logs to terminal and file, so I can see what is going on, and can resume without repeating anything if it crashes. So far, I had one crash, seemingly due to an oMLX server app issue. The script was then modified to kill the former server process if still running after a crash, to reload the needed model and to resume processing. The script is still running, keeping my mac toasty.
I have some other ideas about extracting other useful information from this sort of workflow, but given the time constraints, the one thing I would alter is sampling density. If I’m looking at something like email over a long period (like years), sampling every email is not worth it. I think I would be fine with some fraction of them, like every fourth email. If I am looking for high level, persistent signals, lower granularity is better because emails within a short period are likely to repeat the same themes.