wiki:Clojure Client Tutorial

Version 41 (modified by kedwar10, 13 years ago) (diff)

--

Clojure Client

Getting Started

The first thing you need before using this client is the Leiningen script. This tool makes managing clojure projects relatively easy by providing a REPL, automatically downloading dependencies, compiling your projects into jar files, and many other useful abilities. You can download Leiningen and read its tutorial here ​https://github.com/technomancy/leiningen and here ​https://github.com/technomancy/leiningen/blob/stable/doc/TUTORIAL.md, respectively. If you're having trouble with leiningen, type "lein help" in the shell to see available commands (you can configure leiningen individually for each project, however, so these will differ depending on which directory you are in). Leiningen supports integration with emacs via the swank/slime mechanism, and if you're bent on using this, you can read up on it here ​http://dev.clojure.org/display/doc/Getting+Started (I strongly recommend that you don't attempt this unless you're a emacs expert, as you will likely have to tweak emacs quite a bit to get it working; a simpler solution is to use gvim with the vimclojure script: this provides syntax highlighting and indentation and works out of the box).

Once you have the script in a directory of your choice, add the following to your .cshrc file

"setenv PATH "$PATH":"/path/to/leiningen/sript".

You could skip this step but it will less convenient later on for debugging.

At this point I recommend reading "Joy of Clojure", by Michael Fogus and Chris Houser. This is an excellent introduction to clojure if you are already familiar with lisp or scheme. Much of the inspiration for this client comes from this book so if you're having trouble figuring out what is going on, this will provide most of the background you will need. If you're short on time or funds, just search for online tutorials, there are plenty out there. If you need to quickly look up the documentation on any particular function, just type "(doc f)" in the REPL where f is the function (This will also work for functions that you define provided that you wrote docstrings or added meta-data).

Lisp and Scheme resources:

Some advice for clojure programming:

  • There is no solid way to debug clojure code, so stick to small functions that you can test in the REPL.
  • :pre and :post conditions are excellent features of the language. Here is an example ​http://blog.fogus.me/2009/12/21/clojures-pre-and-post.
  • Leiningen provides "compile" and "run" commands: NEVER USE THESE (unless you want java interop, then you must be careful to recompile the function you wish to export, before running them in the REPL).
  • If you define a record in a namespace and wish to access it in another, you must explicitly import it with the ":import" keyword in your namespace declaration.
  • Giving names to your anonymous functions will dramatically increase the usefulness of clojure error reporting (an example: (fn identity [x] x). Doing this will also allow the function to call itself. Sometimes you can get slightly better error reporting if you compile your project; if you do this, run "lein clean" immediately afterwards.
  • If you want to ensure tail cail optimization, use the "recur" function instead of using your function name. "recur" also works with loops.
  • "Weird" error messages usually result from forgetting to specify the arguments during a function definition or from mismatched parenthesis.
  • The function that you pass as an argument to swap! must be pure (as in no side effects). This is because the update to the atom is retriable so it may get called more then once.
  • Atoms are your go-to way to manage state.
  • If you must define a type, prefer the simplest option. A decision flowchart: ​http://cemerick.com/2011/07/05/flowchart-for-choosing-the-right-clojure-type-definition-form/.
  • If a future fails silently, try just running your function in a regular thread instead.

Overview of Code

Once you have lieningen setup, cd to the client directory and type "tree" into the shell.

This organization is the default for the leiningen projects created with the "lein new <myprojectname>" command.

docs contains and html file that provides a sort of annotated presentation of the code. Read this file first to familiarize yourself with the organization of the project. Once you have made your own changes to the client and wish to update the documentation, run "lein marg" in the top level directory. This invokes the marginalia jar in the lib directory and generates a new html file.

In the top level directory there is a file called "project.clj". This file tells leiningen how your project is organized. For details, consult the tutorial, but know that if you wish to add additional clojure libraries to your project, you will need to specify them here, and run "lein deps" to download them. Specific directions for leiningen are provided nearly universally for clojure project, so just look them up as needed.

src contains all of the clj files in your project. The organization of the directory must correspond to the namespaces of your project so edit with care (more on this later, but one thing to note is that if you have a namespace with dash in it, "client.my-ns", for example, the actual filename must be "my_ns.clj" and it must be located in the client directory).

test contains files that leiningen will use for testing your functions via the "lein test" command. This functionality is not critical in clojure since you can debug easily from the REPL, but if you want to batch test functions then you can look into this.

When you create a new project with leiningen, it will automatically provide a core.clj file for you. If your namespaces are organized into a tree, this is the root. When you run the REPL in the top level directory via the "lein repl" command, the core namspace becomes available to you. This is why having leiningen on your path is useful, as you won't always want to load the core namespace. An important detail: namespaces must NOT be cyclic in clojure, so plan accordingly when designing your project.

Client Usage Tutorial

Note: The order of the scheduling and op arguments to client->server has been reversed. Also, the initial data structure passed to the send functions can now be specified by the user.

New Note: client->server function has been removed; just call send functions with same args.

Now it is time to learn to use the client. cd to the top level directory and typu "lein repl". If everything went smoothly you'll see something about no rlwrap and the prompt. 

$ lein repl
which: no rlwrap in ... (output truncated)
REPL started; server listening on localhost:2192. 
client.core=>

Note that the "core" namespace has loaded. You can launch ioquake with a specific map using the "run-ioquake" function (located in utilities.clj). Type (doc run-ioquake) to see the documentation. 

client.core=> (doc run-ioquake)
-------------------------
client.utilities/run-ioquake
([level] [path level])
  This function launches ioquake with the specified level,
  if the path to ioquake is hardcoded into this function, 
  type '(run-ioquake <level>)' where <level> is a map in 
  the maps folder of ioquake,  else you must provide the path.
nil

For example: 

client.core=> (run-ioquake "/full/path/to/ioquake3.i386" "firstroom")
#<UNIXProcess java.lang.UNIXProcess@2a5ab9>

Since I already have the path predefined, however, this will also work: 

client.core=> (run-ioquake "sat")                                                                                                     
#<UNIXProcess java.lang.UNIXProcess@56c3cf>

You should now see ioquake running in a separate window (if you are in fullscreen mode, navigate to setup and switch this off). Now hit backtick (`) to free the mouse from ioquake. Back in the REPL, type "(doc load-quagent)" 

client.core=> (doc load-quagent)
-------------------------
client.commands/load-quagent
([] [moniker])
  This function will load a quagent into the virtual environment.
  Takes a optional 'moniker' (name is a reserved word in clojure)
  argument that specifies the key that can be used to identify the
  quagent and get information about it.  Otherwise a randomly 
  generated name will be made with the prefix 'quagent'.
nil

Let's load a quagent into the map (note that if there is only one spawnpoint on the map, you'll want to move forward a bit in order to avoid getting telefragged). 

client.core=> (load-quagent)                                                                                                                
:quagent277
client.core=> (load-quagent "Bob")
:Bob

You can check which quagents are currently loaded with the 'get-quagents' function. 

client.core=> (get-quagents)
#{:Bob :quagent277}

You can use these keywords to make the quagents do things in the environment (Note: this functions are subject to change). 

client.core=> (face-me :Bob)
[]
client.core=> (come-here :Bob)
[]

Note that control of the terminal does not return until the bot has completed his action.

Now say you want to define a function on the fly to send an op to the server, this can be done with either "send-and-forget", "send-and-get", "send-and-get-later", and "send-and-watch". These functions provide a human-readable interface to protocol zero by mapping all of the op codes to keywords. Here is the full list of keyword arguments subject to change).

  • :move-indefinitely "mi"
  • :move-for "mf"
  • :move-by "mb"
  • :move-to "mt"
  • :jump-once "ju"
  • :rotate "ro"
  • :fire-weapon "fw"
  • :switch-weapon "sw"
  • :set-crouch "sc"
  • :shove "sv"
  • :say "sy"
  • :pick-up "pu"
  • :put-down "pd"
  • :current-health "hc"
  • :max-health "hm"
  • :current-armor "ac"
  • :max-armor "am"
  • :current-location "lc"
  • :current-facing "fc"
  • :can-see "cs"
  • :radar "ra"
  • :what-is "wi"
  • :current-ammo "mc"
  • :range-finder "rf"
  • :check-inventory "ci"
  • :follow "fo"
  • :batch-range-finder "rb"
  • :pop "po"
  • :pause "pa"
  • :forget-all-tasks "fa"
  • :skip "sk"
  • :peek "pk"
  • :now "n"
  • :then "t"
  • :replace "r" 
client.core=> (doc send-and-forget)
-------------------------
client.protocol-one/send-and-forget
([quagent op scheduling args])
  Sends an op to the server and discards the results.
  Use the keywords in *codes* (see above) instead of 
  protocol zero codes. The args to the op should be in 
  a vector.
nil

client.core=> (doc send-and-get)   
-------------------------
client.protocol-one/send-and-get
([quagent op scheduling args init f])
  Sends an op to the server and blocks until it returns.
  Replies are combined using the (init)ial value and 
  (f)unction supplied by the user.   Use the keywords in
  *codes* (see above) instead of protocol zero codes.
  The args to the op should be in a vector.
nil


client.core=> (doc send-and-get-later)
-------------------------
client.protocol-one/send-and-get-later
([quagent op scheduling args init f])
  Sends an op to the server and returns a future 
  that waits for the reply. Replies are combined
  using the (init)ial value and (f)unction supplied
  by the user.  Use the keywords in *codes* (see above)
  instead of protocol zero codes. The args to the op
  should be in a vector.
nil


client.core=> (doc send-and-watch)    
-------------------------
client.protocol-one/send-and-watch
([quagent op scheduling args init f wf])
  Sends an op to the server and processes replies with 
  an (init)ial value and (f)unction supplied by the user;
  when the (acc)ulator value changes, wf is called.
  wf must be a function that accepts 4 arguments,
  as in (fn [reference key old-val new-val] ...).  Use
  the keywords in *codes* (see above) instead of protocol
  zero codes. The args to the op should be in a vector.
nil

Notice that the room has a few items scattered around it, let's define a function "scan-area" that takes two arguments (a quagent key and a radius) and returns a hash map of the positions of the items. Before trying to write the full function it is a good idea just to print out what the server is returning. 

client.core=> (send-and-get :Bob :radar :now [8000] nil (fn [prev data] (println data)))
(0 player 768.875366 90.065201 0.000000)
(2 player 32.000000 -90.000000 0.000000)
(72 info_player_deathmatch 32.000244 -90.000000 0.223811)
(74 quagent_item_treasure 572.168640 20.462269 0.901278)
(75 quagent_item_gold 375.085327 56.309933 1.374918)
(76 quagent_item_gold 1019.278626 42.455196 0.505915)
(77 quagent_item_treasure 697.711304 63.434952 0.739097)
(78 quagent_item_gold 905.141357 8.130102 0.569713)
(79 info_player_deathmatch 0.125000 0.000000 90.000000)
nil

The simplest implementation is to just use the basic radar op from protocol zero with a vector for an initial value and "conj" for the combination function. (Note the use of "pp" to pretty-print the previous result.) 

client.core=> (send-and-get :Bob :radar :now [8000] [] conj)
[("0" "player" "768.875366" "90.065201" "0.000000") ... (output-truncated)
client.core=> (pp)                                          
[("0" "player" "768.875366" "90.065201" "0.000000")
 ("2" "player" "32.000000" "-90.000000" "0.000000")
 ("72" "info_player_deathmatch" "32.000244" "-90.000000" "0.223811")
 ("74" "quagent_item_treasure" "572.168640" "20.462269" "0.901278")
 ("75" "quagent_item_gold" "375.085327" "56.309933" "1.374918")
 ("76" "quagent_item_gold" "1019.278626" "42.455196" "0.505915")
 ("77" "quagent_item_treasure" "697.711304" "63.434952" "0.739097")
 ("78" "quagent_item_gold" "905.141357" "8.130102" "0.569713")
 ("79" "info_player_deathmatch" "0.125000" "0.000000" "90.000000")]
nil

Partitioning these into a map is going to be a little more difficult as multiple positions will need to be stored at each key. However, we know already that the initial data structure needs to be a hash-map and the keys need to be the item type. 

client.core=> (send-and-get :Bob :radar :now [8000] 
                            {}
                            (fn [prev [_ item-type & pos]]
                              (assoc prev item-type pos)) 
)
{"quagent_item_gold" ("905.141357" "8.130102" "0.569713")... (output truncated) 
client.core=> (pp)
{"quagent_item_gold" ("905.141357" "8.130102" "0.569713"),
 "quagent_item_treasure" ("697.711304" "63.434952" "0.739097"),
 "info_player_deathmatch" ("0.125000" "0.000000" "90.000000"),
 "player" ("32.000000" "-90.000000" "0.000000")}
nil

This is progress but the data from the new replies is overriding the previous results. To get the right behaviour, the "merge-with" function must be used to combine the maps. 

client.core=> (send-and-get :Bob :radar :now [8000] 
                            {}
                            (fn [prev [_ item-type & pos]]
                              (merge-with concat prev {item-type (list pos)})))
{"quagent_item_gold" (("375.085327" "56.309933" "1.374918") ... (output truncated)
client.core=> (pp)
{"quagent_item_gold"
 (("375.085327" "56.309933" "1.374918")
  ("1019.278626" "42.455196" "0.505915")
  ("905.141357" "8.130102" "0.569713")),
 "quagent_item_treasure"
 (("572.168640" "20.462269" "0.901278")
  ("697.711304" "63.434952" "0.739097")),
 "info_player_deathmatch"
 (("32.000244" "-90.000000" "0.223811")
  ("0.125000" "0.000000" "90.000000")),
 "player"
 (("768.875366" "90.065201" "0.000000")
  ("32.000000" "-90.000000" "0.000000"))}
nil

These positions can't be used as strings, however, and will need to be converted to doubles. 

client.core=> (send-and-get :Bob :radar :now [8000] 
                            {}
                            (fn [prev [_ item-type & pos]]
                              (merge-with concat 
                                          prev
                                          {item-type (list (map #(Double/parseDouble %) 
                                                                pos))})))
{"quagent_item_gold" ((375.085327 56.309933 1.374918) (1019.278626 42.455196 0.505915) ... (output-truncated)
client.core=> (pp)
{"quagent_item_gold"
 ((375.085327 56.309933 1.374918)
  (1019.278626 42.455196 0.505915)
  (905.141357 8.130102 0.569713)),
 "quagent_item_treasure"
 ((572.16864 20.462269 0.901278) (697.711304 63.434952 0.739097)),
 "info_player_deathmatch"
 ((32.000244 -90.0 0.223811) (0.125 0.0 90.0)),
 "player" ((768.875366 90.065201 0.0) (32.0 -90.0 0.0))}
nil

Sample Domains

Cave explorer

Value iteration for movement planning. Collaborative exploration.

Rovers

Automated planning, STRIPS domain, ICAPS '02.

Current Issues

  • Exploring the maze with multiple bots will occasionally cause one to hang. This is an issue coordinating multiple data structures in the maze explorer code, not in protocol one.