In 2016 I decided to have another go at writing my own database because I wanted a database that was blindingly fast for queries but with files that are machine and human-readable. Not to mention that I have a fondness/weakness for plists. Thus cl-naive-store was born.

Like any pet project, it took on a life of its own and has become a bit of a beast (by 2022) with the best of intentions. The reason is that you can customize just about anything (you could say it was designed to be customized) and it has a lot of additional packages that can be loaded to layer on more and more functionality. The “base API“, creating, deleting, querying etc stays the same no matter how many layers of functionality you load. So you can incrementally use more and more of the possible functionality without rewriting code. That means quick proto-type code can be fleshed out later if needed.

In its simplest form cl-naive-store is a collection of plists with or without keys. In its fully loaded form, it’s a lazy-loaded, in memory, persistent, hierarchical, declaratively described, document universe with indexing and sharding thrown in for good measure. You can choose what you want to use and what not its all up to you.

Query speed was always the number one priority, though with lazy loading and sharding the loading and persistence of data runs at a respectable clip. It was also never meant to be a database that would host terabytes of data in one collection. That being said, it can deal with millions of records without too much effort and I would wager that it could deal with tens of millions if sharding is used properly.

One peculiarity of cl-naive-store is that it does not use data definitions (like table definitions found in SQL), a package for such definitions is available (thus the “declaratively described”) but the inner workings of the cl-naive-store don’t need them to load, query or persist data. Collections of data only need a name and maybe keys to function. That means what a document looks like today compared to yesterday has no consequences as far as cl-naive-store is concerned. Such power could come at the cost of your sanity if you abuse it, so a framework for data definitions is supplied but not enforced.

I think cl-naive-store is pretty well documented and there are simple but comprehensive examples.

I have been blessed to have been assisted on this project by some really exceptional people, so it has been a really fulfilling experience in more than one way.

Try it out and let me know what you think.


6 Responses to “cl-naive-store”

  1. Common Lisp: 2022 in review | BOWLING GREEN STAR Says:

    […] Document Store/DB Implemented in Common Lisp […]

  2. Common Lisp: 2022 in review by Tomte - HackTech Says:

    […] Document Store/DB Implemented in Common Lisp […]

  3. Typical Lisp: 2022 in evaluation – Tech Daily Hub News Says:

    […] Document Store/DB Implemented in Common Lisp […]

  4. Common Lisp: 2022 in review Says:

    […] Narrative Store/DB Utilized in Frequent Narrate […]

  5. Józef Says:

    I’ve started using cl-naive-store in my currently internal tool for my team I’m working with as part of my current contract, and so far I love it. Especially the flexibility of it as I do try a lot of things in a lot of scenarios during development. I’ve created also 2 merge request on a repo for some minor things that were not exactly working out of the box and I needed them. Anyway thanx a lot and I will try to make some more contribution along the way, and also make some update on success story when I will use it for production system as I have plans for that!

  6. zaries Says:

    Thank you very much for the feedback! I have accepted the one merge request and will work through the other, see comments on the merge request.

    As far as a production system goes, if you use it in the “classical” sense IE as a database, then I would recommend that you use naive-documents from the start.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: