wiki:Clojure Client Tutorial

Version 34 (modified by kedwar10, 13 years ago) (diff)

--

Clojure Client

Getting Started

The first thing you need before using this client is the Leiningen script. This tool makes managing clojure projects relatively easy by providing a REPL, automatically downloading dependencies, compiling your projects into jar files, and many other useful abilities. You can download Leiningen and read its tutorial here ​https://github.com/technomancy/leiningen and here ​https://github.com/technomancy/leiningen/blob/stable/doc/TUTORIAL.md, respectively. If you're having trouble with leiningen, type "lein help" in the shell to see available commands (you can configure leiningen individually for each project, however, so these will differ depending on which directory you are in). Leiningen supports integration with emacs via the swank/slime mechanism, and if you're bent on using this, you can read up on it here ​http://dev.clojure.org/display/doc/Getting+Started (I strongly recommend that you don't attempt this unless you're a emacs expert, as you will likely have to tweak emacs quite a bit to get it working; a simpler solution is to use gvim with the vimclojure script: this provides syntax highlighting and indentation and works out of the box).

Once you have the script in a directory of your choice, add the following to your .cshrc file

"setenv PATH "$PATH":"/path/to/leiningen/sript".

You could skip this step but it will less convenient later on for debugging.

At this point I recommend reading "Joy of Clojure", by Michael Fogus and Chris Houser. This is an excellent introduction to clojure if you are already familiar with lisp or scheme. Much of the inspiration for this client comes from this book so if you're having trouble figuring out what is going on, this will provide most of the background you will need. If you're short on time or funds, just search for online tutorials, there are plenty out there. If you need to quickly look up the documentation on any particular function, just type "(doc f)" in the REPL where f is the function (This will also work for functions that you define provided that you wrote docstrings or added meta-data).

Lisp and Scheme resources:

Some advice for clojure programming:

  • There is no solid way to debug clojure code, so stick to small functions that you can test in the REPL.
  • :pre and :post conditions are excellent features of the language. Here is an example ​http://blog.fogus.me/2009/12/21/clojures-pre-and-post.
  • Leiningen provides "compile" and "run" commands: NEVER USE THESE (unless you want java interop, then you must be careful to recompile the function you wish to export, before running them in the REPL).
  • If you define a record in a namespace and wish to access it in another, you must explicitly import it with the ":import" keyword in your namespace declaration.
  • Giving names to your anonymous functions will dramatically increase the usefulness of clojure error reporting (an example: (fn identity [x] x). Doing this will also allow the function to call itself. Sometimes you can get slightly better error reporting if you compile your project; if you do this, run "lein clean" immediately afterwards.
  • If you want to ensure tail cail optimization, use the "recur" function instead of using your function name. "recur" also works with loops.
  • "Weird" error messages usually result from forgetting to specify the arguments during a function definition or from mismatched parenthesis.
  • The function that you pass as an argument to swap! must be pure (as in no side effects). This is because the update to the atom is retriable so it may get called more then once.
  • Atoms are your go-to way to manage state.
  • If you must define a type, prefer the simplest option. A decision flowchart: ​http://cemerick.com/2011/07/05/flowchart-for-choosing-the-right-clojure-type-definition-form/.

Overview of Code

Once you have lieningen setup, cd to the client directory and type "tree" into the shell.

This organization is the default for the leiningen projects created with the "lein new <myprojectname>" command.

docs contains and html file that provides a sort of annotated presentation of the code. Read this file first to familiarize yourself with the organization of the project. Once you have made your own changes to the client and wish to update the documentation, run "lein marg" in the top level directory. This invokes the marginalia jar in the lib directory and generates a new html file.

In the top level directory there is a file called "project.clj". This file tells leiningen how your project is organized. For details, consult the tutorial, but know that if you wish to add additional clojure libraries to your project, you will need to specify them here, and run "lein deps" to download them. Specific directions for leiningen are provided nearly universally for clojure project, so just look them up as needed.

src contains all of the clj files in your project. The organization of the directory must correspond to the namespaces of your project so edit with care (more on this later, but one thing to note is that if you have a namespace with dash in it, "client.my-ns", for example, the actual filename must be "my_ns.clj" and it must be located in the client directory).

test contains files that leiningen will use for testing your functions via the "lein test" command. This functionality is not critical in clojure since you can debug easily from the REPL, but if you want to batch test functions then you can look into this.

When you create a new project with leiningen, it will automatically provide a core.clj file for you. If your namespaces are organized into a tree, this is the root. When you run the REPL in the top level directory via the "lein repl" command, the core namspace becomes available to you. This is why having leiningen on your path is useful, as you won't always want to load the core namespace. An important detail: namespaces must NOT be cyclic in clojure, so plan accordingly when designing your project.

Client Usage Tutorial

Note: The order of the scheduling and op arguments to client->server has been reversed. Also, the initial data structure passed to the send functions can now be specified by the user.

Now it is time to learn to use the client. cd to the top level directory and typu "lein repl". If everything went smoothly you'll see something about no rlwrap and the prompt. 

$ lein repl
which: no rlwrap in ... (output truncated)
REPL started; server listening on localhost:2192. 
client.core=>

Note that the "core" namespace has loaded. You can launch ioquake with a specific map using the "run-ioquake" function (located in utilities.clj). Type (doc run-ioquake) to see the documentation. 

client.core=> (doc run-ioquake)
-------------------------
client.utilities/run-ioquake
([level] [path level])
  This function launches ioquake with the specified level,
  if the path to ioquake is hardcoded into this function, 
  type '(run-ioquake <level>)' where <level> is a map in 
  the maps folder of ioquake,  else you must provide the path.
nil

For example: 

client.core=> (run-ioquake "/full/path/to/ioquake3.i386" "firstroom")
#<UNIXProcess java.lang.UNIXProcess@2a5ab9>

Since I already have the path predefined, however, this will also work: 

client.core=> (run-ioquake "firstroom")                                                                                                     
#<UNIXProcess java.lang.UNIXProcess@56c3cf>

You should now see ioquake running in a separate window (if you are in fullscreen mode, navigate to setup and switch this off). Now hit backtick (`) to free the mouse from ioquake. Back in the REPL, type "(doc load-quagent)" 

client.core=> (doc load-quagent)
-------------------------
client.commands/load-quagent
([] [moniker])
  This function will load a quagent into the virtual environment.
  Takes a optional 'moniker' (name is a reserved word in clojure)
  argument that specifies the key that can be used to identify the
  quagent and get information about it.  Otherwise a randomly 
  generated name will be made with the prefix 'quagent'.
nil

Let's load a quagent into the map (note that if there is only one spawnpoint on the map, you'll want to move forward a bit in order to avoid getting telefragged). 

client.core=> (load-quagent)                                                                                                                
:quagent240
client.core=> (load-quagent "Bob")
:Bob

You can check which quagents are currently loaded with the 'get-quagents' function. 

client.core=> (get-quagents)
(:Bob :quagent240)

You can use these keywords to make the quagents do things in the environment (Note: this functions are subject to change). 

client.core=> (face-me :Bob)
[]
client.core=> (come-here :Bob)
[]

Note that control of the terminal does not return until the bot has completed his action.

Now say you want to define a function on the fly to send an op to the server, the easiest way to do so is the 'client->server' function. 

client.core=> (doc client->server)
-------------------------
client.protocol-one/client->server
([send-func quagent scheduling op args & fs])
  Provides a usable interface for sending ops to the server.
  Arguments are the type of send function, the quagent to
  execute the op, the scheduling, the op keyword
  (see *codes* in protocol_one.clj) the arguments to the op
  (in vector form) and the dispatch functions to be given to
  the send function.
nil

The send function can be one of the following 

client.core=> (doc send-and-forget)
-------------------------
client.protocol-one/send-and-forget
([quagent id msg])
  Send an op to the server and throw away the results.
nil

client.core=> (doc send-and-get)   
-------------------------
client.protocol-one/send-and-get
([quagent id msg] [quagent id msg f])
  Send a op to the server and block this thread until the
  op completes. Args are a (unique) op id, a msg to send
  to the server and a function f that determines how to 
  combine the current reply from the server with past
  results (defaults to conj).
nil

client.core=> (doc send-and-get-later)
-------------------------
client.protocol-one/send-and-get-later
([quagent id msg] [quagent id msg f])
  This will spawn a separate thread to do the send and get,
  if you try to dereference it before it has completed,
  it will block.  Use future-done? to check the status before
  dereferencing.
nil

client.core=> (doc send-and-watch)    
-------------------------
client.protocol-one/send-and-watch
([quagent id msg wf] [quagent id msg f wf])
  Sends the op to the server and then applies f to update the value.
  When the value changes wf is called (wf must meet the requirements
  of the add-watch function).  The return value is the key for the
  watcher; this is used to delete it if necessary
 [(example)](http://clojure-examples.appspot.com/clojure.core/add-watch?revision=1278516003572). 
nil

Say we want to define a function "get-location-of" that takes one argument (a quagent key) and returns the position as a vector of doubles. From protocol zero, we know that the string to be sent to the server should look like "n lc <id>" where <id> is the op id. One possible implementation would be 

client.core=> (defn get-location-of 
                [quagent]
                (client->server send-and-get 
                                quagent
                                :now
                                :current-location
                                nil))
#'client.core/get-location-of

Note that the protocol zero op codes have been mapped to more user friendly keywords. Here is the full map (subject to change)

  • :move-indefinitely "mi"
  • :move-for "mf"
  • :move-by "mb"
  • :move-to "mt"
  • :jump-once "ju"
  • :rotate "ro"
  • :fire-weapon "fw"
  • :switch-weapon "sw"
  • :set-crouch "sc"
  • :shove "sv"
  • :say "sy"
  • :pick-up "pu"
  • :put-down "pd"
  • :current-health "hc"
  • :max-health "hm"
  • :current-armor "ac"
  • :max-armor "am"
  • :current-location "lc"
  • :current-facing "fc"
  • :can-see "cs"
  • :radar "ra"
  • :what-is "wi"
  • :current-ammo "mc"
  • :range-finder "rf"
  • :check-inventory "ci"
  • :follow "fo"
  • :batch-range-finder "rb"
  • :pop "po"
  • :pause "pa"
  • :forget-all-tasks "fa"
  • :skip "sk"
  • :peek "pk"
  • :now "n"
  • :then "t"
  • :replace "r"

Running this however, didn't yield the expected output. 

client.core=> (get-location-of :Bob)
[{:data ("-128.000000" "136.000000" "40.125000"), :op "lc", :reply-type "rs", :id "1"}]

Instead of getting a vector of doubles, it returned a vector that contains a map. To fix this we need to pass in an dispatch function to control the behaviour of the return value. This function takes two arguments, the previous result, and the most recent reply from the server, and determines how to combine them. For example, say we only wish to return the map. 

client.core=> (defn get-location-of [quagent]
                (client->server send-and-get
                                quagent
                                :now
                                :current-location
                                nil
                                (fn [prev reply]
                                  reply)))
#'client.core/get-location-of

Note the change in return value. 

client.core=> (get-location-of :Bob)                                                                                                  
{:data ("-128.000000" "136.000000" "40.125000"), :op "lc", :reply-type "rs", :id "2"}

Here is the correct version. Note the use of destructuring in the anonymous function 

client.core=> (defn get-location-of [quagent]
                (client->server send-and-get
                                quagent
                                :now
                                :current-location
                                nil
                                (fn [prev {:keys [data]}]
                                  (vec (map #(Double/parseDouble %) data)))))
#'client.core/get-location-of

Now this returns the expected output. 

client.core=> (get-location-of :Bob)
[-128.0 136.0 40.125]

This particular anonymous function is so frequent that that it comes predefined. 

client.core=> (defn get-location-of [quagent]  
                (client->server send-and-get
                                quagent
                                :now
                                :current-location
                                nil
                                data->doubles))                           
#'client.core/get-location-of

Suppose that this particular op took a long time to complete and we didn't want to wait around for it to complete, The way clojure handles this is through the use of "futures". Simply, clojure will do the computation in a separate thread so the current one can continue. send-and-get-later is identitical to send-and-get except that it is passed to the "future" function. 

client.core=> (doc future)
-------------------------
clojure.core/future
([& body])
Macro
  Takes a body of expressions and yields a future object that will
  invoke the body in another thread, and will cache the result and
  return it on all subsequent calls to deref/@. If the computation has
  not yet finished, calls to deref/@ will block.
nil

We need to redefine "get-location-of" with a different send function. 

client.core=> (defn get-location-of [quagent]
                (client->server send-and-get-later ; 
                                quagent
                                :now
                                :current-location
                                nil
                                data->doubles))
#'client.core/get-location-of

Now we need a variable to store the future. 

client.core=> (def f (get-location-of :Bob))
#'client.core/f

In order to check if the computation is done use the "future-done?" function. 

client.core=> (doc future-done?)
-------------------------
clojure.core/future-done?
([f])
  Returns true if future f is done
nil
client.core=> (future-done? f)
true

Now we know f is safe to dereference. 

client.core=> @f
[-128.0 136.0 40.125]

Futures allow us to do more interesting things, however. Say we want to make two bots explore a maze simultaneously, this can't be done if the quagents are all blocking on the same thread. Type control-c in the repl and execute the following. 

client.core=> (run-ioquake "largemaze")   
#<UNIXProcess java.lang.UNIXProcess@5ce40>
client.core=> (load-quagent :Bob) ; remember to stand in the corner         
:Bob
client.core=>(make-gui) ; the quagents use a value iteration to calculate the best policy; brightness denotes higher utility                           
#<JFrame javax.swing.JFrame (output truncated)>
client.core=> (def q1 (future (explore-maze :Bob)))
#'client.core/q1
client.core=> (load-quagent :Joe)                  
:Joe
client.core=> (def q2 (future (explore-maze :Joe)))
#'client.core/q2

Suppose now we need a more sophisticated behaviour whereby the quagent sends out a message to the server and then calls a function every time the server replies before terminating the op. This can be accomplished with the last send function "send-and-watch". Here, an additional watcher function (wf) must be specified that takes four arguments: a (k)ey, (r)efernce, (o)ld value, and (n)ew value. 

client.core=> (defn get-location-of [quagent] 
                (client->server send-and-watch
                                quagent
                                :now
                                :current-location
                                nil
                                (fn [k r o n]
                                 (println "old val " o " new val " n)))
#'client.core/get-location-of
client.core=> (get-location-of :Joe)                                                                                                                               
:watcher245
old val  []  new val  [{:data (64.000000 64.000000 32.125000), :op lc, :reply-type rs, :id 1}]

The key :watcher245 can be used to delete the watch function if so desired. Otherwise it will be removed when the command completes. 

client.core=> (doc remove-watch)
-------------------------
clojure.core/remove-watch
([reference key])
  Alpha - subject to change.
  Removes a watch (set by add-watch) from a reference
nil

Recall that "send-and-watch" can also accept a function to control the combination of previous and current replies as well, so the watcher function argument "o" is not limited to just being a vector of replies.

Sample Domains

Cave explorer

Value iteration for movement planning. Collaborative exploration.

Rovers

Automated planning, STRIPS domain, ICAPS '02.

Current Issues

  • Exploring the maze with multiple bots will occasionally cause one to hang. This is not an issue with server since inserting bots via separate repl's fixes the issue.