cl-naive-store

In 2016 I decided to have another go at writing my own database because I wanted a database that was blindingly fast for queries but with files that are machine and human-readable. Not to mention that I have a fondness/weakness for plists. Thus cl-naive-store was born.

Like any pet project, it took on a life of its own and has become a bit of a beast (by 2022) with the best of intentions. The reason is that you can customize just about anything (you could say it was designed to be customized) and it has a lot of additional packages that can be loaded to layer on more and more functionality. The “base API“, creating, deleting, querying etc stays the same no matter how many layers of functionality you load. So you can incrementally use more and more of the possible functionality without rewriting code. That means quick proto-type code can be fleshed out later if needed.

In its simplest form cl-naive-store is a collection of plists with or without keys. In its fully loaded form, it’s a lazy-loaded, in memory, persistent, hierarchical, declaratively described, document universe with indexing and sharding thrown in for good measure. You can choose what you want to use and what not its all up to you.

Query speed was always the number one priority, though with lazy loading and sharding the loading and persistence of data runs at a respectable clip. It was also never meant to be a database that would host terabytes of data in one collection. That being said, it can deal with millions of records without too much effort and I would wager that it could deal with tens of millions if sharding is used properly.

One peculiarity of cl-naive-store is that it does not use data definitions (like table definitions found in SQL), a package for such definitions is available (thus the “declaratively described”) but the inner workings of the cl-naive-store don’t need them to load, query or persist data. Collections of data only need a name and maybe keys to function. That means what a document looks like today compared to yesterday has no consequences as far as cl-naive-store is concerned. Such power could come at the cost of your sanity if you abuse it, so a framework for data definitions is supplied but not enforced.

I think cl-naive-store is pretty well documented and there are simple but comprehensive examples.

I have been blessed to have been assisted on this project by some really exceptional people, so it has been a really fulfilling experience in more than one way.

Try it out and let me know what you think.

Advertisement

Quick Self Signed SSL Certificate with SAN

There are thousands of articles and entries in stackexchange but none of them worked for me out of the box. So after hours of battling with this issue here is my short recipe.

Create the Root Key:

openssl genrsa -out rootCA.key 2048

Self-sign this root certificate:

openssl req -x509 -new -nodes -key rootCA.key -sha256 -days 3560 -out rootCA.pem

Create a certificate request for the domain you want:

You need the following in your san.cnf file to use in creating the certificate request. Just create the file where you are the certificates.

[ req ]
default_bits = 2048
distinguished_name = req_distinguished_name
req_extensions = req_ext
[ req_distinguished_name ]
countryName = Country Name (2 letter code)
stateOrProvinceName = State or Province Name (full name)
localityName = Locality Name (eg, city)
organizationName = Organization Name (eg, company)
commonName = Common Name (e.g. server FQDN or YOUR name)
[ req_ext ]
subjectAltName = @alt_names
[alt_names]
DNS.1 = app.somedomain.co.za

openssl req -out app.somedomain.co.za.csr -newkey rsa:2048 -nodes -keyout app.somedomain.co.za.key -config san.cnf

To verify that the .csr has the SAN in it

openssl req -noout -text -in app.somedomain.co.za.csr | grep DNS

Sign your final domain certificate with your root certificate and makes sure the SAN ends up in the resulting certificate:

openssl x509 -req -in app.somedomain.co.za.csr -CA rootCA.pem -CAkey rootCA.key -CAcreateserial -out app.somedomain.co.za.crt -days 3560 -sha256 -extfile san.cnf -extensions req_ext

Verify the result:

openssl x509 -noout -text -in app.somedomain.co.za.crt | grep DNS:

Verify the result once you have uploaded the certificates to your web server:

echo | openssl s_client -connect app.somedomain.co.za:443 | openssl x509 -noout -text | grep DNS:

What is left is to import your root certificate into your browser, and that is another nightmare since it works differently for different versions of chrome and different operating systems. You will have to google and try until it works, but at least you will have a certificate that you know is not at issue.

Ubuntu 19.10 interferes with Emacs keybindings

So far I had to remap the following key-bindings to not interfere with emacs.

You can use dconf-editor to view all the org.gnome.desktop.wm.keybindings and even edit them.

Here is the list I remapped and what I the remapping using gsettings:

Alt+`

gsettings set org.gnome.desktop.wm.keybindings switch-group "['<Super>Above_Tab']"

Ctrl+Alt+Down

gsettings set org.gnome.desktop.wm.keybindings switch-to-workspace-down "['<Super>Page_Down']"

Ctrl+Alt+Left

gsettings set org.gnome.desktop.wm.keybindings switch-to-workspace-left "['[]']"

Ctrl+Alt+Right

gsettings set org.gnome.desktop.wm.keybindings switch-to-workspace-right "['[]']"

Ctrl+Alt+Up

gsettings set org.gnome.desktop.wm.keybindings switch-to-workspace-up "['<Super>Page_Up']"

Lisp Use

I often find little gems of where lisp is used and where it was used in history but if you google for it later you cant find it. So I am starting my own little list, inspired by this reddit post.

Formal JavaScript Semantics

The current ECMAScript semantics are written informally and contain many amgibuities and errors. In addition, while experimenting with new language features it is difficult to modify the informal semantics and be assured that the result remains self-consistent and still behaves as expected. In an attempt to remedy these problems, waldemar is writing formal semantics for JavaScript together with an engine that can:

• pretty-print the semantics for human readability
• grammar-check the semantics to make sure that the grammar is unambiguous
• type-check the semantics to make sure that the rules are self-consistent
• execute the semantics directly to check their behavior on sample JavaScript programs or verify against a test suite

Check out the actual LISP code here.

Interface Builder

Interface Builder first made its appearance in 1986 written in Lisp (for the ExperLisp product by ExperTelligence). It was invented and developed by Jean-Marie Hullot using the object-oriented features in ExperLisp, and deeply integrated with the Macintosh toolbox. Denison Bollay took Jean-Marie Hullot to NeXT later that year to demonstrate it to Steve Jobs. Jobs immediately recognized its value, and started incorporating it into NeXTSTEP, and by 1988 it was part of NeXTSTEP 0.8. It was the first commercial application that allowed interface objects, such as buttons, menus, and windows, to be placed in an interface using a mouse. One notable early use of Interface Builder was the development of the first WorldWideWeb web browser by Tim Berners-Lee at CERN, made using a NeXT workstation.

…more on Interface Builder by Denis Bollay

ExperTelligence introduced “Interface Builder” in 1986. We took it up to neXt to show Steve Jobs – the rest is history. In 1988, Denison Bollay built a much more dynamic interface tool, in which the interface was fully modifiable AS the program was running. Since it was built in incrementally compiled LISP, all other functions and methods were also modifiable on the fly. Denny took it to Seattle to show Bill Gates, but MicroSoft wanted a version written in basic (no objects, no methods, etc back then). I explained one couldn’t do that without OO. They built Visual Basic.

The Remote Agent Experiment: Debugging Code from 60 Million Miles Away

Debugging a program running on a $100M piece of hardware that is 100 million miles away is an interesting experience. Having a read-eval-print loop running on the spacecraft proved invaluable in finding and fixing the problem.

SBCL on Ubuntu 19.04: Unable to load any of the alternatives – Error – cl-ssl

Since 2007 when I started using Ubuntu I have been doing upgrades to the latest version a month before the release. Yes I am a sucker for pain.

After the upgrade to 19.04 today I had issues loading projects that used cl-ssl getting the following error:

Unable to load any of the alternatives:
   ("libssl.so.1.0.2m" "libssl.so.1.0.2k" "libssl.so.1.0.2"
    "libssl.so.1.0.1l" "libssl.so.1.0.1j" "libssl.so.1.0.1e"
    "libssl.so.1.0.1" "libssl.so.1.0.0q" "libssl.so.1.0.0"
    "libssl.so.0.9.8ze" "libssl.so.0.9.8" "libssl.so.10"
    "libssl.so.4" "libssl.so")

The fix was (after trying installing latest sbcl first) was to update quicklisp libraries.

(ql:update-dist "quicklisp")

I don’t know if there are fixes in cl-ssl, I suspect that maybe it just needed to be recompiled….

react-native suspense no more fetch hell

While reading up about hooks I ran into a blog post by Daishi Kato about a custom useFetch hook. Having a purely hate relationship with fetch in react-native it immediately caught my eye! In the blog post I learned that suspense could solve the fetching of data conundrum. Hooks where forgotten and I went googling suspense.

There where three things I did not like with what I found, first all the code examples where using (or punting) other libraries, the second was that a lot of the code examples where related to code splitting, and third there was just so much over kill. I wanted a simple example (but still useful when applied to real world problems) that did not use external libraries, and after some trial and error here it is.

The gist of the solution is that you need to fetch data, put it some where so you can use it later and a function to display the data where suspense will take care of the delay for you.

react-native hooks = functions not classes

I wanted to try Suspense so spun up an new project with react-native init in 0.59.0-rc.3 (react-native init myprojectname –version react-native@next) .

The examples for Suspense I found also used hooks. So I thought fine might as well investigate that to. So I cut and paste code into the App.js, this got syntax errors and then run time errors etc. What I had not grokked was that hooks is all about eliminating/replacing class components with functional components. And not about doing state with out componentDidMounted and its friends…eish

react-native ubuntu 18.10 quick and easy install

I had to install react-native on a clean laptop so I thought I would pen this recipe to my self and any others who need a quick reference.

Firstly I need to say that react-native is much easier to get up and running than it was 6 months ago! The install of components was never to bad but you had to hack config files to get your first project running, that does not seem to be the case any more. The first time I tried react-native it took me a couple of days to successfully run my first app! This recipe should take you less than 15 min if you have fast internet and can type. And the default app runs first time!

Ubuntu 17.04 – UI Feels more Responsive

I upgraded to 17.04 beta a few days ago and I could swear that the UI perceptively faster than 16.10! Has anybody else experienced this, and do you know why if you have?

Common Lisp Hunchentoot Quicklisp on Ubuntu Server @ Rackspace VM

So after six years I finally get round to revisiting this subject, a lot has changed over the years, but not much either, the basics are still the same.

So you want your own lisp web server to hack away at but where do you start? Well don’t despair, what follows is a recipe for getting such a server running in no time.

SERVER CREATION

So I am using Rackspace servers a bit like you would partition your favourite linux install. We are going to store data on one Rackspace volume, have the live data on another and use a different volume for the server. The basic idea is that you can disconnect your data volumes, spin up a new server and reconnect them if you should need to. This way you can also choose which volumes you need as SSD and which not.

You can find a lot of good “how to” stuff on Rackspace at https://support.rackspace.com/how-to/