Clojure in the Database

As a finale, we will persist data in Datomic and attempt some queries on the data-model. For a quick starter on Datomic, check out Datomic from the ground up

Setting up Datomic

Follow the instructions to download Datomic (Select the "Datomic Pro Starter Edition" version) and set up. Use the steps under the section Running the transactor with the dev storage protocol to set up a local transactor. Once set up, the transactor will print a URI like datomic:dev://localhost:4334/<DB-NAME>

Create a dev-transactor.properties using config/samples/dev-transactor-template.properties and filling in the license-key from https://my.datomic.com/account. Then start up a transactor using the cmd: bin/transactor dev-transactor.properties. To verify the starup, open another terminal windown and start a Peer REPL - bin/repl

user=> (require '[datomic.api :as d])
user=> (def uri "datomic:dev://localhost:4334/foo")         
user=> (d/create-database uri)
user=> (def conn (d/connect uri))
user=> (d/q '[:find ?a :where [:db.part/db :db.install/attribute ?a]] (d/db conn))
; #{[39] [40] [41] [10] [42] [11] [43] [12] [44] [13] [45] [14] [46] [15] [47] [16] [17] [18] [50] [19] [51] [52] [62]}

Setting up the Server

Add the following dependencies in project.clj to use Datomic API on the server:

[ch.qos.logback/logback-classic "1.1.2" :exclusions [org.slf4j/slf4j-api]]
[org.slf4j/jul-to-slf4j "1.7.7"]
[org.slf4j/jcl-over-slf4j "1.7.7"]
[org.slf4j/log4j-over-slf4j "1.7.7"]
[org.clojure/tools.namespace "0.2.9"]
[com.datomic/datomic-pro "0.9.5130"]

Let's add some entities for attribute definitions:

; resources/db/schema.edn
[{:db/id #db/id[:db.part/db]
  :db/ident :video/title
  :db/valueType :db.type/string
  :db/cardinality :db.cardinality/one
  :db.install/_attribute :db.part/db}
 {:db/id #db/id[:db.part/db]
  :db/ident :video/url
  :db/valueType :db.type/string
  :db/cardinality :db.cardinality/one
  :db.install/_attribute :db.part/db}]

and some more entities to set up the initial set of videos:

; resources/db/data.edn
[{:db/id #db/id[:db.part/db]
  :db/ident :video/title
  :db/valueType :db.type/string
  :db/cardinality :db.cardinality/one
  :db.install/_attribute :db.part/db}
 {:db/id #db/id[:db.part/db]
  :db/ident :video/url
  :db/valueType :db.type/string
  :db/cardinality :db.cardinality/one
  :db.install/_attribute :db.part/db}]

We also need some code to initialize the DB before the server starts - src/clj/clj_stack/db.clj

(ns clj_stack.db
  (:require [datomic.api :as d]
            [clojure.java.io :as io]))


(def schema-tx
  (read-string (slurp (io/resource "db/schema.edn"))))

(def data-tx
  (read-string (slurp (io/resource "db/data.edn"))))


(def uri "datomic:dev://localhost:4334/clj_stack")

(defn init []
  (d/create-database uri)
  (let [conn (d/connect uri)]
    @(d/transact conn schema-tx)
    @(d/transact conn data-tx)
    (d/q '[:find ?vt :where [?v :video/title ?vt]] (d/db conn))))

We are ready to initialize the DB. Start up the server REPL using lein repl

clj_stack.server=> (require '[clj_stack.db :as db] :reload-all)
clj_stack.server=> (db/init)
; #{["The Functional Final Frontier"] ["Intro to Datomic"]}

Loading a persisted data model

Since we have a persisted data model, we can get rid of the data-model atom defined in service.clj and adapt the videos function to query Datomic for all the videos:

(defn videos [request]
  (let [vs (d/q '[:find (pull ?v [*]) :where [?v :video/title ?t]] (d/db conn))]
    (bootstrap/edn-response {:videos (vec (apply concat vs))})))

• This query seeks any entity that has a :video/title attribute assigned to it
• The pull clause "pulls" the entire entity instead of the id

The client code in core.cljs needs a minor update: The attribute name is now :video/url instead of :url:

          (dom/iframe #js {:className "embed-responsive-item"
                           :src (:video/url video)}))

Similarly, add-video needs to use the new attribute names for the new video

video-data {:video/title title :video/url url}

Note that we also avoided the id attribute which needs to be automatically assigned by Datomic. The automatic id generation is indicated by #db/id[db.part/user] in the transaction below:

; service.clj
(defn add-video [request]
  (let [video (:edn-params (body-params/edn-parser request))]
    @(d/transact conn [(merge {:db/id #db/id[:db.part/user]} video)])
    (ring-resp/response "ok")))

Restart the server and test by adding a new video. It should persist even across database restarts.

Removing videos

While sending out the DELETE request, we should be looking out for :db/id instead of :id

; core.cljs
(edn-xhr {:url (str "/videos/" (:db/id del-video)) :method :delete})

Datomic models data as a set of facts. Even if an attribute changes in value, it is not updated or overwritten. Instead, the new attribute is recorded as a new fact. Therefore, to remove a video, we need to retract all facts about it. By retracting, we are not deleting the facts - in fact, they'll be present in database values previous to the time they are retracted. We are simply marking them invalid at the current point in time.

(defn delete-video [request]
  (let [id (Long/parseLong (get-in request [:path-params :id]))]
    @(d/transact conn [[:db.fn/retractEntity id]])
    (ring-resp/response "ok")))

• retractEntity is a database function that retracts all facts about a given entity
• Note that we use parseLong now that the id is generated by Datomic and fairly larger than an integer