The Monroe Corpus
Home | People | Projects | Publications | Resources
General
| Participants
| Tasks | Setup
|
Dialogue stats |
Processing |
Transcripts
and annotations
Comments on the dialogues
Table 1:
Comments on dialogues
Dialogue |
Participants |
Task |
Completed |
Extra task |
Completed |
|
|
|
initial task? |
assigned? |
extra task? |
1 |
S1R, S1B |
1 |
yes |
yes |
yes |
2 |
S1B, S1R |
2 |
yes |
yes |
yes |
3 |
S2B, S2R |
3 |
yes |
yes |
no |
4 |
S2R, S2B |
4 |
yes |
yes |
yes |
5 |
S3B, S3R |
5 |
yes |
no |
|
6 |
S3R, S3B |
6 |
yes |
no |
|
7 |
S4R, S4B |
8 |
yes |
no |
|
8 |
S4B, S4R |
7 |
yes |
no |
|
9 |
S5B, S5R |
5 |
yes |
no |
|
10 |
S5R, S5B |
1 |
yes |
no |
|
11 |
S6R, S6B |
6 |
yes |
no |
|
12 |
S6B, S6R |
2 |
yes |
no |
|
13 |
S7B, S7R |
7 |
yes |
no |
|
14 |
S7R, S7B |
3 |
yes |
yes |
yes |
15 |
S8R, S8B |
8 |
yes |
yes |
yes |
16 |
S8B, S8R |
4 |
yes |
yes |
yes |
17 |
S9R, S9B |
8 |
yes |
yes |
no |
18 |
S9B, S9R |
1 |
yes |
yes |
yes |
19 |
S10B, S10R |
4 |
yes |
yes |
yes |
20 |
S10R, S10B |
5 |
yes |
no |
|
Dialogue 1: S1R has trouble hearing S1B, who speaks with
the
microphone a little far away from his face. Tape starts a little into
the dialogue.
Dialogues 7, 8: S4B says ``three'' like
``tree''. In dialogue 7, S4R invents vehicles.
Dialogues 13, 14: Both speakers have colds. Phone rings midway
through dialogue 14.
Dialogues 15, 16: S8R says ``ambulance'' in an
unusual way.
Dialogues 17, 18: S9R either chews his fingers or puts
things in his mouth.
Dialogues 19, 20: S10B has a cold.
|
All dialogues ended with the participants thinking that they had solved
every part of the task. While this was generally true, in one case,
participants failed to solve the extra problem given at the end, and
in another case, participants invented extra police units and
ambulances to help them solve the task (table 1).
Participants did not limit themselves to discussing the task. They
made meta-comments about the domain or task, occasionally commented if
they couldn't hear the other participant, and sometimes made jokes to
each other. There was more of this type of interaction when the
participants knew each other.
Monroe county was known to most of the participants. This was helpful
because fewer participants spent long periods of time trying to find
locations, and so they were able to concentrate on solving the task.
However, it did also mean that the participants made use of locations
(e.g. roads) and names not on the map or in the task description.
Because the dialogue participants were not allowed to make eye contact
during the data collection, we can observe a wide variety of language use
(including location descriptions), but fewer deictics and longer
dialogues.
In order to test our hypothesis that a lack of grounding does
significantly affect dialogue participants, we had two of our
participants return and conduct two dialogues each with one
experimenter, solving tasks they had not previously solved. In the
first dialogue, the experimenter behaved as a human would, processing
speech in real-time and providing feedback. In the second, the
experimenter did not back-channel, perform grounding acts or
interrupt. The people took longer to solve the task in the second
dialogue, paused longer between turns, and said they thought the
experimenter was not paying attention or not understanding. One
exhibited a great deal of frustration, and eventually stopped trying
to solve the task and fell silent, forcing the experimenter to take
over.
These four dialogues do not form part of the Monroe corpus. They
provide merely anecdotal evidence supporting claims that humans rely
on other humans' providing appropriate conversational behaviors.
They neither prove nor disprove that humans would find interactions
with dialogue systems more effective if the system provided human-like
conversational behaviors, but there are some intriguing clues. For
instance, in the dialogues where the experimenter failed to provide
feedback or take appropriate initiative, the inter-speaker pauses
increased to more closely resemble those in typical human-computer
dialogues, and the other participant's contributions became less
free-flowing. It would be interesting to collect dialogues with a
system using a similar experimental setup.
Monroe dialogue statistics
Table 2:
Monroe dialogues: basic statistics
Dialogue |
Length |
Number of |
Number of |
Average |
Average |
|
(seconds) |
turns |
utterances |
turn length |
utt. length |
1 |
458 |
112 |
129 |
3.31 |
4.26 |
2 |
705 |
206 |
352 |
1.68 |
3.37 |
3 |
557 |
77 |
145 |
2.38 |
6.35 |
4 |
1071 |
170 |
337 |
2.30 |
5.59 |
5 |
2341 |
540 |
918 |
2.06 |
4.21 |
6 |
1621 |
384 |
647 |
2.16 |
4.26 |
7 |
1711 |
459 |
735 |
1.66 |
3.25 |
8 |
1434 |
310 |
473 |
2.39 |
4.33 |
9 |
2432 |
490 |
783 |
2.59 |
4.79 |
10 |
584 |
111 |
163 |
3.12 |
5.19 |
11 |
973 |
198 |
295 |
2.74 |
4.64 |
12 |
415 |
99 |
164 |
2.30 |
4.27 |
13 |
2144 |
404 |
605 |
2.77 |
4.88 |
14 |
769 |
188 |
285 |
2.08 |
3.81 |
15 |
579 |
100 |
152 |
3.27 |
5.51 |
16 |
1015 |
201 |
329 |
2.66 |
4.98 |
17 |
890 |
218 |
342 |
1.84 |
3.42 |
18 |
1229 |
288 |
445 |
1.98 |
3.82 |
19 |
1603 |
372 |
564 |
2.28 |
3.96 |
20 |
1371 |
271 |
325 |
3.41 |
5.10 |
Average |
1195.1 |
252 |
399 |
2.45 |
4.50 |
|
Table 2 shows, for each dialogue, the
length of the
dialogue, the number of turns and the number of utterances in that
dialogue, and the average turn and utterance lengths. The total
number of words in the corpus is approximately 52000; the number of
unique words is approximately 1550 including partial words.
The length of the dialogues varies widely, depending partly on task
complexity, but most of the dialogues exhibit a similar structure.
The initial task holder describes the initial situation (including
current tasks, resources, and constraints) to the other discourse
participant. Then the initial task holder describes the plan so far.
In these two phases the initial task holder typically holds the
initiative, with the other discourse participant providing
acknowledgments, backchannels, and clarification questions. The two
discourse participants then collaborate to finish solving the task.
In some dialogues, the initial task holder retains task initiative,
but in others the other discourse participant takes initiative, and in
some there is a good deal of competition for the initiative. Finally,
one or both discourse participants summarize the plan. In those
dialogues where a new task was given, the same process - task description,
task solution, plan summary - is followed.
This document was generated using the
LaTeX2HTML translator Version 98.1 release (February 19th, 1998)
Copyright © 1993, 1994, 1995, 1996, 1997,
Nikos Drakos,
Computer Based Learning Unit, University of Leeds.
The translation was initiated by Amanda Stent on 2001-12-11
Amanda Stent
2001-12-11