In this post, I want to talk about some of the recent changes to the processes of understanding semantic meaning behind Zettelgarden. I am going to d

From Vector Search to Entity Processing: Evolving Zettelgarden's Connection Engine

submited by

Style Pass

2024-12-02 20:30:04

In this post, I want to talk about some of the recent changes to the processes of understanding semantic meaning behind Zettelgarden. I am going to discuss where I want to be going, the first attempts and where we are now.

I have been writing about it for the last few weeks, but Zettelgarden is a digital ‘zettelkasten’ that I have been working on as a personal project and for my own benefit. I have recently been experimenting with integrating LLMs in it to help build connections and for search. Check it out on Github: https://github.com/NickSavage/Zettelgarden

Let's take a step back and remember the core purpose of a Zettelkasten: building meaningful connections between ideas. When I started my Zettelkasten journey on paper, I accumulated around 5,000 physical index cards before realizing I needed a different approach. This collection became one of the driving forces behind developing Zettelgarden - I needed a tool that could help manage and discover connections across this substantial body of knowledge.

While my early experiments with vector search showed promising results, the fundamental challenge that inspired Zettelgarden remains: how to effectively manage connections across a large collection of notes. With paper cards, I noticed a pattern that many Zettelkasten users might find familiar. Making connections between recently added cards comes naturally - our minds easily spot relationships, contradictions, and supporting evidence between ideas we've encountered in the past few weeks or months. It's like working with a moving window of connections, where our recent thoughts are crystal clear, but older notes gradually fade into the background.