wiki:Clojure Client Tutorial

Version 62 (modified by kedwar10, 13 years ago) (diff)

--

Clojure Client

Getting Started

The first thing you need before using this client is the Leiningen script. This tool makes managing clojure projects relatively easy by providing a REPL, automatically downloading dependencies, compiling your projects into jar files, and many other useful abilities. You can download Leiningen and read its tutorial here

If you're having trouble with leiningen, type "lein help" in the shell to see available commands (you can configure leiningen individually for each project, however, so these will differ depending on which directory you are in). Leiningen supports integration with emacs via the swank/slime mechanism, and if you're bent on using this, you can read up on it here ​http://dev.clojure.org/display/doc/Getting+Started (I strongly recommend that you don't attempt this unless you're a emacs expert, as you will likely have to tweak emacs quite a bit to get it working; a simpler solution is to use gvim with the vimclojure script: this provides syntax highlighting and indentation and works out of the box).

Once you have the script in a directory of your choice, add the following to your .cshrc file (assuming you are using the c shell; else lookup specific instructions for your shell) 

setenv PATH "$PATH":"/path/to/leiningen/sript

You could skip this step but it will less convenient later on for debugging.

At this point I recommend reading "Joy of Clojure", by Michael Fogus and Chris Houser. This is an excellent introduction to clojure if you are already familiar with lisp or scheme. Much of the inspiration for this client comes from this book so if you're having trouble figuring out what is going on, this will provide most of the background you will need. If you're short on time or funds, just search for online tutorials, there are plenty out there. If you need to quickly look up the documentation on any particular function, just type "(doc <function name>)" in the REPL (This will also work for functions that you define provided that you wrote docstrings or added meta-data).

Lisp and Scheme resources:

Some advice for clojure programming:

  • For debugging, use ​http://richhickey.github.com/clojure-contrib/trace-api.html.
  • :pre and :post conditions can be used to put constraints on functions. Here is an example ​http://blog.fogus.me/2009/12/21/clojures-pre-and-post.
  • Leiningen provides "compile" and "run" commands: NEVER USE THESE (unless you want java interop, then you must be careful to recompile the function you wish to export, before running them in the REPL).
  • If you define a record in a namespace and wish to access it in another, you must explicitly import it with the ":import" keyword in your namespace declaration.
  • Giving names to your anonymous functions will dramatically increase the usefulness of clojure error reporting (an example: (fn identity [x] x). Doing this will also allow the function to call itself. Sometimes you can get slightly better error reporting if you compile your project; if you do this, run "lein clean" immediately afterwards.
  • If you want to ensure tail cail optimization, use the "recur" function instead of using your function name. "recur" also works with loops.
  • "Weird" error messages usually result from forgetting to specify the arguments during a function definition or from mismatched parenthesis.
  • The function that you pass as an argument to swap! must be pure (as in no side effects). This is because the update to the atom is retriable so it may get called more then once.
  • Atoms are your go-to way to manage state.
  • If you must define a type, prefer the simplest option. A decision flowchart: ​http://cemerick.com/2011/07/05/flowchart-for-choosing-the-right-clojure-type-definition-form/.
  • If a future fails silently, try just running your function in a regular thread instead.

Organization of Code

Once you have lieningen setup, cd to the client directory and type "tree" into the shell.

This organization is the default for the leiningen projects created with the "lein new <myprojectname>" command.

docs contains and html file that provides a sort of annotated presentation of the code. Read this file first to familiarize yourself with the organization of the project. Once you have made your own changes to the client and wish to update the documentation, run "lein marg" in the top level directory. This invokes the marginalia jar in the lib directory and generates a new html file.

In the top level directory there is a file called "project.clj". This file tells leiningen how your project is organized. For details, consult the tutorial, but know that if you wish to add additional clojure libraries to your project, you will need to specify them here, and run "lein deps" to download them. Specific directions for leiningen are provided nearly universally for clojure project, so just look them up as needed.

src contains all of the clj files in your project. The organization of the directory must correspond to the namespaces of your project so edit with care (more on this later, but one thing to note is that if you have a namespace with dash in it, "client.my-ns", for example, the actual filename must be "my_ns.clj" and it must be located in the client directory).

test contains files that leiningen will use for testing your functions via the "lein test" command. This functionality is not critical in clojure since you can debug easily from the REPL, but if you want to batch test functions then you can look into this.

When you create a new project with leiningen, it will automatically provide a core.clj file for you. If your namespaces are organized into a tree, this is the root. When you run the REPL in the top level directory via the "lein repl" command, the core namspace becomes available to you. This is why having leiningen on your path is useful, as you won't always want to load the core namespace. An important detail: namespaces must NOT be cyclic in clojure, so plan accordingly when designing your project.

Client Usage Tutorial

Basic Functionality

Now it is time to learn to use the client. cd to the top level directory and typu "lein repl". If everything went smoothly you'll see something about no rlwrap and the prompt. 

$ lein repl
which: no rlwrap in ... (output truncated)
REPL started; server listening on localhost:2192. 
client.core=>

Note that the "core" namespace has loaded. You can launch ioquake with a specific map using the "run-ioquake" function (located in utilities.clj). Type (doc run-ioquake) to see the documentation. 

client.core=> (doc run-ioquake)
-------------------------
client.utilities/run-ioquake
([level] [path level])
  This function launches ioquake with the specified level,
  if the path to ioquake is hardcoded into this function, 
  type '(run-ioquake <level>)' where <level> is a map in 
  the maps folder of ioquake,  else you must provide the path.
nil

For example: 

client.core=> (run-ioquake "/full/path/to/ioquake3.i386" "sat")
#<UNIXProcess java.lang.UNIXProcess@2a5ab9>

Since I already have the path predefined, however, this will also work: 

client.core=> (run-ioquake "sat")                                                                                                     
#<UNIXProcess java.lang.UNIXProcess@56c3cf>

You should now see ioquake running in a separate window (if you are in fullscreen mode, navigate to setup and switch this off). Now hit backtick (`) to free the mouse from ioquake. Back in the REPL, type "(doc load-quagent)" 

client.core=> (doc load-quagent)
-------------------------
client.commands/load-quagent
([] [moniker])
  This function will load a quagent into the virtual environment.
  Takes a optional 'moniker' (name is a reserved word in clojure)
  argument that specifies the key that can be used to identify the
  quagent and get information about it.  Otherwise a randomly 
  generated name will be made with the prefix 'quagent'.
nil

Let's load a quagent into the map (note that if there is only one spawnpoint on the map, you'll want to move forward a bit in order to avoid getting telefragged). 

client.core=> (load-quagent)
:quagent277
client.core=> (load-quagent "Bob")
:Bob

You can check which quagents are currently loaded with the 'get-quagents' function. 

client.core=> (get-quagents)
#{:Bob :quagent277}

You can use these keywords to make the quagents do things in the environment (Note: this functions are subject to change). 

client.core=> (move :Bob 500 0)
[]

Note that control of the terminal does not return until the bot has completed his action.

Defining New Commands

Now say you want to define a function on the fly to send an op to the server, this can be done with either "send-and-forget", "send-and-get", "send-and-get-later", and "send-and-watch". These functions provide a human-readable interface to protocol zero by mapping all of the op codes to keywords. Here is the full list of keyword arguments subject to change).

  • :move-indefinitely "mi"
  • :move-for "mf"
  • :move-by "mb"
  • :move-to "mt"
  • :jump-once "ju"
  • :rotate "ro"
  • :fire-weapon "fw"
  • :switch-weapon "sw"
  • :set-crouch "sc"
  • :shove "sv"
  • :say "sy"
  • :pick-up "pu"
  • :put-down "pd"
  • :current-health "hc"
  • :max-health "hm"
  • :current-armor "ac"
  • :max-armor "am"
  • :current-location "lc"
  • :current-facing "fc"
  • :can-see "cs"
  • :radar "ra"
  • :what-is "wi"
  • :current-ammo "mc"
  • :range-finder "rf"
  • :check-inventory "ci"
  • :follow "fo"
  • :batch-range-finder "rb"
  • :pop "po"
  • :pause "pa"
  • :forget-all-tasks "fa"
  • :skip "sk"
  • :peek "pk"
  • :now "n"
  • :then "t"
  • :replace "r" 
client.core=> (doc send-and-forget)
-------------------------
client.protocol-one/send-and-forget
([quagent op scheduling args])
  Sends an op to the server and discards the results.
  Use the keywords in *codes* (see above) instead of 
  protocol zero codes. The args to the op should be in 
  a vector.
nil

client.core=> (doc send-and-get)   
-------------------------
client.protocol-one/send-and-get
([quagent op scheduling args init f])
  Sends an op to the server and blocks until it returns.
  Replies are combined using the (init)ial value and 
  (f)unction supplied by the user.   Use the keywords in
  *codes* (see above) instead of protocol zero codes.
  The args to the op should be in a vector.
nil


client.core=> (doc send-and-get-later)
-------------------------
client.protocol-one/send-and-get-later
([quagent op scheduling args init f])
  Sends an op to the server and returns a future 
  that waits for the reply. Replies are combined
  using the (init)ial value and (f)unction supplied
  by the user.  Use the keywords in *codes* (see above)
  instead of protocol zero codes. The args to the op
  should be in a vector.
nil


client.core=> (doc send-and-watch)    
-------------------------
client.protocol-one/send-and-watch
([quagent op scheduling args init f wf])
  Sends an op to the server and processes replies with 
  an (init)ial value and (f)unction supplied by the user;
  when the (acc)ulator value changes, wf is called.
  wf must be a function that accepts 4 arguments,
  as in (fn [reference key old-val new-val] ...).  Use
  the keywords in *codes* (see above) instead of protocol
  zero codes. The args to the op should be in a vector.
nil

Notice that the room has a few items scattered around it, let's define a function "scan-area" that takes two arguments (a quagent key and a radius) and returns a hash map of the positions of the items. Before trying to write the full function it is a good idea just to print out what the server is returning. 

client.core=> (send-and-get :Bob :radar :now [8000] nil (fn [prev data] (println data)))
(0 player 768.875366 90.065201 0.000000)
(2 player 32.000000 -90.000000 0.000000)
(72 info_player_deathmatch 32.000244 -90.000000 0.223811)
(74 quagent_item_treasure 572.168640 20.462269 0.901278)
(75 quagent_item_gold 375.085327 56.309933 1.374918)
(76 quagent_item_gold 1019.278626 42.455196 0.505915)
(77 quagent_item_treasure 697.711304 63.434952 0.739097)
(78 quagent_item_gold 905.141357 8.130102 0.569713)
(79 info_player_deathmatch 0.125000 0.000000 90.000000)
nil

The simplest implementation is to just use the basic radar op from protocol zero with a vector for an initial value and "conj" for the combination function. (Note the use of "pp" to pretty-print the previous result.) 

client.core=> (send-and-get :Bob :radar :now [8000] [] conj)
[("0" "player" "768.875366" "90.065201" "0.000000") ... (output-truncated)
client.core=> (pp)                                          
[("0" "player" "768.875366" "90.065201" "0.000000")
 ("2" "player" "32.000000" "-90.000000" "0.000000")
 ("72" "info_player_deathmatch" "32.000244" "-90.000000" "0.223811")
 ("74" "quagent_item_treasure" "572.168640" "20.462269" "0.901278")
 ("75" "quagent_item_gold" "375.085327" "56.309933" "1.374918")
 ("76" "quagent_item_gold" "1019.278626" "42.455196" "0.505915")
 ("77" "quagent_item_treasure" "697.711304" "63.434952" "0.739097")
 ("78" "quagent_item_gold" "905.141357" "8.130102" "0.569713")
 ("79" "info_player_deathmatch" "0.125000" "0.000000" "90.000000")]
nil

Partitioning these into a map is going to be a little more difficult as multiple positions will need to be stored at each key. However, we know already that the initial data structure needs to be a hash-map and the keys need to be the item type. 

client.core=> (send-and-get :Bob :radar :now [8000] 
                            {}
                            (fn [prev [_ item-type & pos]]
                              (assoc prev item-type pos)))
{"quagent_item_gold" ("905.141357" "8.130102" "0.569713")... (output truncated) 
client.core=> (pp)
{"quagent_item_gold" ("905.141357" "8.130102" "0.569713"),
 "quagent_item_treasure" ("697.711304" "63.434952" "0.739097"),
 "info_player_deathmatch" ("0.125000" "0.000000" "90.000000"),
 "player" ("32.000000" "-90.000000" "0.000000")}
nil

This is progress but the data from the new replies is overriding the previous results. To get the right behaviour, the "merge-with" function must be used to combine the maps. 

client.core=> (send-and-get :Bob :radar :now [8000] 
                            {}
                            (fn [prev [_ item-type & pos]]
                              (merge-with concat prev {item-type (list pos)})))
{"quagent_item_gold" (("375.085327" "56.309933" "1.374918") ... (output truncated)
client.core=> (pp)
{"quagent_item_gold"
 (("375.085327" "56.309933" "1.374918")
  ("1019.278626" "42.455196" "0.505915")
  ("905.141357" "8.130102" "0.569713")),
 "quagent_item_treasure"
 (("572.168640" "20.462269" "0.901278")
  ("697.711304" "63.434952" "0.739097")),
 "info_player_deathmatch"
 (("32.000244" "-90.000000" "0.223811")
  ("0.125000" "0.000000" "90.000000")),
 "player"
 (("768.875366" "90.065201" "0.000000")
  ("32.000000" "-90.000000" "0.000000"))}
nil

These positions can't be used as strings, however, and will need to be converted to doubles. 

client.core=> (send-and-get :Bob :radar :now [8000] 
                            {}
                            (fn [prev [_ item-type & pos]]
                              (merge-with concat 
                                          prev
                                          {item-type (list (map #(Double/parseDouble %) 
                                                                pos))})))
{"quagent_item_gold" ((375.085327 56.309933 1.374918) (1019.278626 42.455196 0.505915) ... (output-truncated)
client.core=> (pp)
{"quagent_item_gold"
 ((375.085327 56.309933 1.374918)
  (1019.278626 42.455196 0.505915)
  (905.141357 8.130102 0.569713)),
 "quagent_item_treasure"
 ((572.16864 20.462269 0.901278) (697.711304 63.434952 0.739097)),
 "info_player_deathmatch"
 ((32.000244 -90.0 0.223811) (0.125 0.0 90.0)),
 "player" ((768.875366 90.065201 0.0) (32.0 -90.0 0.0))}
nil

The process of converting a sequence into doubles is so common that it has been included in the client as "seq->doubles". 

client.core=> (defn scan-area [quagent radius]
                (send-and-get quagent :radar :now [radius] 
                              {} 
                              (fn [prev [_ item-type & pos]]
                                (merge-with concat 
                                            prev
                                            {item-type (list (seq->doubles pos))}))))
#'client.core/scan-area
client.core=> (scan-area :Bob 8000)
{"quagent_item_gold" ([375.085327 56.309933 1.374918] ... (output truncated)
client.core=> (pp)
{"quagent_item_gold"
 ([375.085327 56.309933 1.374918]
  [1019.278626 42.455196 0.505915]
  [905.141357 8.130102 0.569713]),
 "quagent_item_treasure"
 ([572.16864 20.462269 0.901278] [697.711304 63.434952 0.739097]),
 "info_player_deathmatch"
 ([32.000244 -90.0 0.223811] [0.125 0.0 90.0]),
 "player" ([768.875366 90.065201 0.0] [32.0 -90.0 0.0])}
nil

You can now use the predefined "move" command to make the quagent walk to an item. 

client.core=> (apply (partial move :Bob) (take 2 (second (get (scan-area :Bob 8000) "quagent_item_gold")))) 
[]

Now suppose that scanning the area takes an inordinate amount of time and blocking until it completes is no longer practical. By using the "send-and-get-later" function, the results can be computed in a new thread and dereferenced later. 

client.core=> (defn scan-area2 [quagent radius]
                (send-and-get-later quagent :radar :now [radius]
                                    {}
                                    (fn [prev [_ item-type & pos]]
                                      (merge-with concat prev {item-type (list (seq->doubles pos))}))))
#'client.core/scan-area2
client.core=> (def items (scan-area2 :Bob 8000))
#'client.core/items
client.core=> (type items)
clojure.core$future_call$reify__5508
client.core=> (future-done? items)
true
client.core=> @items
{"quagent_item_gold" ([375.085327 56.309933 1.374918] [1019.278626 42.455196 0.505915] ... (output truncated)
client.core=> (pp)
{"quagent_item_gold"
 ([375.085327 56.309933 1.374918]
  [1019.278626 42.455196 0.505915]
  [905.141357 8.130102 0.569713]),
 "quagent_item_treasure"
 ([572.16864 20.462269 0.901278] [697.711304 63.434952 0.739097]),
 "info_player_deathmatch"
 ([32.000244 -90.0 0.223811] [0.125 0.0 90.0]),
 "player" ([611.700012 90.081947 0.0] [32.0 -90.0 0.0])}
nil

(Note that if you try to dereference a future before it completes, it will block the current thread until it does.)

Now let's suppose that every time the quagent reports finding an item, it should print out the distance that item. This can be accomplished with the "send-and-watch" function. 

client.core=> (defn scan-area3 [quagent radius]
                (send-and-watch quagent :radar :now [radius] 
                                nil
                                (fn [prev data]
                                 (rest data))
                                (fn [k r o n]
                                 (println "Item:" (first n) "Distance:" (second n)))))
#'client.core/scan-area3
client.core=> (scan-area3 :Bob 8000)
:watcher311
Item: player Distance: 611.700012
Item: player Distance: 32.000000
Item: info_player_deathmatch Distance: 32.000244
Item: quagent_item_treasure Distance: 572.168640
Item: quagent_item_gold Distance: 375.085327
Item: quagent_item_gold Distance: 1019.278626
Item: quagent_item_treasure Distance: 697.711304
Item: quagent_item_gold Distance: 905.141357
Item: info_player_deathmatch Distance: 0.125000

This function returns a watcher key that can be used to remove the watcher if desired. Note that this example only makes used the (n)ew argument.

Controlling Multiple Quagents

The simplest way to operate multiple quagents on the REPL is to wrap all commands with "future". (This is also the only function that will return directly.) 

client.core=> (future (move :bob 500 0))
#<core$future_call$reify__5508@12d7d02: :pending>
client.core=> (future (move :joe 500 0))
#<core$future_call$reify__5508@45ce17: :pending>

If you want to apply the same command to many quagents, some typing can be saved with the "pmap" function, which is exactly like "map" except the elements are processed in parallel. 

client.core=> (pmap #(move % 1000 0) (get-quagents))  
([] [])

Similarly, "pvalues" builds a lazy sequence of values. 

client.core=> (pvalues (scan-area :bob 8000) (move :joe 1500 180))            
({"quagent_item_gold" ([839.970581 160.375198 0.613917] ... (output truncated)
client.core=> (pp)
({"quagent_item_gold"
  ([839.970581 160.375198 0.613917]
   [703.026123 110.582535 0.733509]
   [142.61908 136.43396 3.618063]),
  "quagent_item_treasure"
  ([493.463074 159.83313 1.045044] [908.39563 139.15329 0.567672]),
  "info_player_deathmatch"
  ([1001.049011 -176.454605 0.007154]
   [999.580505 -178.285645 0.007165]),
  "player" ([1005.275879 -174.131836 0.0] [267.436554 -6.46857 0.0])}
 [])
nil

Scheduling Functions

see ​https://github.com/overtone/at-at

Logic Programming

Sample Domains

Cave explorer

Value iteration for movement planning. Collaborative exploration.

Rovers

Automated planning, STRIPS domain, ICAPS '02.

For the Time domains, in the pddl file, replace the :duration value of navigate with 

:duration (= ?duration (/ (distance ?y ?z) 0.28))

and add the function 

(distance ?x - waypoint ?y - waypoint)

Now add distance data to the pfile. Example: 

 (= (distance waypoint0 waypoint0) 0)
 (= (distance waypoint0 waypoint11) 399)
 (= (distance waypoint0 waypoint10) 410)
 (= (distance waypoint0 waypoint9) 525)
 (= (distance waypoint0 waypoint7) 340)
 (= (distance waypoint0 waypoint8) 256)
 (= (distance waypoint0 waypoint5) 798)
 (= (distance waypoint0 waypoint6) 708)
 (= (distance waypoint0 waypoint2) 925)
 (= (distance waypoint0 waypoint1) 434)
 (= (distance waypoint0 waypoint4) 924)
 (= (distance waypoint0 waypoint3) 697)
 (= (distance waypoint11 waypoint0) 399)
 (= (distance waypoint11 waypoint11) 0)
 (= (distance waypoint11 waypoint10) 604)
 (= (distance waypoint11 waypoint9) 522)
 (= (distance waypoint11 waypoint7) 289)
 (= (distance waypoint11 waypoint8) 592)
 (= (distance waypoint11 waypoint5) 735)
 (= (distance waypoint11 waypoint6) 988)
 (= (distance waypoint11 waypoint2) 979)
 (= (distance waypoint11 waypoint1) 798)
 (= (distance waypoint11 waypoint4) 769)
.
.
.

Current Issues

  • Exploring the maze with multiple bots will occasionally cause one to hang. This is an issue coordinating multiple data structures in the maze explorer code, not in protocol one.