- 366 words - 2 min read

I’m happy to be able to share one of the building blocks of the software solution I’m currently working on: fsdb - a Clojure library that provides a reasonably convenient database on top of the file system.

Don’t get me wrong, I’m not talking about a file system database like sqlite here. I’m using the term database loosly. Think of a bunch of config files that make up a database that you want to query. That’s what fsdb can do for you. You point it to a directory and it reads the data from the directory tree and returns a data structure that you can query.

Ok enough talks, here is the example from the README…

% tree example
example
├── people.edn
├── technologies.edn
└── technologies
    └── clojure.yml

$ cat example/people.edn
{:rich {:name "Rich Hickey"}}

$ cat example/technologies.edn
{:clojure {:year "unknown"}}

$ cat example/technologies/clojure.yml
---
year: 2007

Reading this structure with fsdb/read-tree will result in the following data structure:

{:example
 {:people {:rich {:name "Rich Hickey"}}
  :technologies {:clojure {:year 2007}}}}

In the example you can observe multiple aspects of fsdb.

  • The db is spread of multiple files, these will be merged deeply.
  • The files can have different formats.
  • Names of directories & files make up the nesting of the resulting data structure.
  • Later (more specific entries) overwrite former.

So “Having the data your way” means you get to decide when and where to split up your data file into subdirectories and smaller files, which makes it easier to keep track of your data.

In addition to structuring, there are many more reasons you might want to split a database into multiple files.

If your files are under Version Control, splitting them up means, reducing potential merge conflicts and you can even choose to exempt some files from being tracked.

You can also mix and match formats. Some data might be easier to edit in YAML than EDN, or the other way around. (Other formats can easily be supported. Drop me a line or send a Merge Request for the formats you might need.)

Maybe some of your files are generated by surrounding automation, fsdb will help you mix different sources into one queryable data structure.

Please find fsdb on Gitlab and Clojars:

Clojars Project