Incomplete Documentation for 2_jsonl_to_memory.py tool

[Xopher00]

I have noticed that there is a lot of missing documentation for important features in the CAI framework. I see many features in the README.md file with hyperlinks to files that simply do not exist, notably tools/2_jsonl_to_memory.py for converting and preserving memories in between agent instances or runs. This seems like a serious oversight that is either due to missing features, leftover comments from an older version of the framework, or mistakes from using an AI ot write the README. Some clarification on the memory feature would be greatly appreciated.

[SoyGema]

I do agree. Certainly the README.md needs a revision. Which features do you consider more relevant ?
Would you be open to a discussion and a contrib about this?
Could you please give me some context on how are you using memory?
Thanks in advance. I really appreciate this feedback !!

[Xopher00]

I am experimenting with creating automated CAI agents for security scans, running inside docker containers. Basically an instance that runs a scan for an API, returns data, and exits. The idea being that I have one tool for many different types of APIs, instead of different scanners for every type of API or cloud service. I have an implementation that sort of works now. But at this point its sort of a Jack-of-all-Trades, Master of None. I think that it's performance could be improved if memories carried over between different instances. I tried to mount a Docker volume to where logs and memories are being stored, but I am having trouble getting it to save memories automatically.

[SoyGema]

What I understand is that you want to centralize memory across instances, and in each instance you are using an agent. Is this correct? . Have you read src/cai/agents/memory.py and configure the variables ? Can you share them ?
I do agree mem should require some docs

[vmayoral]

Thanks for raising this issue @Xopher00. I reckon there's much to improve in the documentation.

Shortly, the tools/2_jsonl_to_memory.py you refer to is a legacy utility which dissapeared in the transition to 0.4.x. Not to justify it in any way, but unfortunately, we're moving faster than we can document. Project's barely 5 months old since it's disclosure and we have many thousands of users. We need more contributions on the open source side if we're to keep up with the expectations of users like you.

Anyhow, I’ve updated the README to align with our current ICL-first approach and the new workflow for reusing prior runs. So that it's understandable, in a nutshell, the file you were referring to (and which was later moved to memory.py) prototyped RAG within CAI. We initially thought we could unlock new capabilities in Cybersecurity AIs via RAG, but the evidence suggested otherwise. Over time, we shifted to a different strategy—using In-Context Learning (ICL). This wasn’t unique to us; most agentic frameworks came to the same conclusion, since LLMs advanced quickly enough to handle what RAG aimed to solve, simply by leveraging context more effectively.

5376aaf addresses your points.

What changed:

• Replaced the “memory extension” JSONL-to-memory workflow with ICL-based guidance.
• Documented the /load command as the supported way to bring previous logs into context.
• Kept the log-path screenshot and clarified that CAI prints the JSONL path at startup.
• Pointed to legacy notes in src/cai/agents/memory.py; persistent memory (episodic/semantic) is now considered legacy.

New usage (replaces the old JSONL ingestion script)

• Load into current agent:

  CAI>/load logs/cai_20250408_111856.jsonl

Rationale

CAI performs best today by relying on In‑Context Learning, not durable long‑term stores. This reduces complexity, avoids stale state, and keeps the mental model simple while preserving flexibility for advanced users.

Follow-ups wherein I encourage you to contribute @Xopher00:

• Sweep remaining references to tools/2_jsonl_to_memory.py and tag them as legacy (or remove if we choose).
• Experiment yourself with RAG again and argue about it (e.g. add a brief “ICL vs. Legacy RAG Memory” FAQ entry if desired)

As I said at the beginning, lots of attention and demands/requests (many of which aren't always visible/open, given the nature of the project). Our research team is trying to address this by puting down some additional dedicated person-power into it but ultimately, as the project grows, we need additional involvement and community contributions.