comments by annotator C: not sure whether about OK and alright in acceptances (like utt59): should they be assert? "done" seems to be, "fine" not sure. where to draw the line?"
editor's response: OK and alright are not asserts. They don't say anything about the world like "done" does. I'd group "fine" with "OK" and "alright"; they all describe attitudes toward the previous utterance(s).
comments by mcore: First, I plan on changing the dat warnings to remind people to fill in info-level and to not expect directives to have answers if they are not understood.
In my annotations as well as those of others I noticed a problem with utterances such as utt7-utt9. There seems to be a backwards function from utt9 to utt8 Perhaps we need an understanding tag such as "repeat-misheard" (utterance).
utt7 U: CLEAR [CLEAR should be: WHERE ARE] THE PEOPLE utt8 S: I'm afraid I didn't understand you utt9 U: where are the peopleAnother problem is capturing the propose-revise-accept sequence often seen in task-oriented dialogs. Perhaps we need a "revise" tag in the agreement dimension to capture situations such as utt21 below. We also need to make sure to keep a firm distinction between higher-level discourse structure and DAMSL tags so that we are not doubly coding features.
utt18+19 U: How long will that take utt20 S: It will take 6 hours and 54 minutes utt21 U: what if we went along the coast insteadLabeling the dialog composed of utt40-utt43 is tricky. My approach was to label utt40 as just an assert and leave it to the higher-level discourse people to identify a referent for "that" and establish the accept relation themselves. I think the same needs to be done for utt43. Finding the highlighted actions and labeling them as possibly rejected (maybe's) seems beyond the capabilities of DAMSL. I think there are three equally valid approaches to utt42:
utt38 u: and take the people there to delta utt39 s: alright utt40 u: that looks good utt41 s: no problem utt42 u: Delta bridge is out utt43 s: the highlighted actions may need to be revisedIt seems like TRIPS has bags of supposedly equalivalent utterances ("done","your wish is my command", "okay") and ("I didn't understand", "Pardon me?") which I labeled somewhat differently. I labeled "your wish is my command" as a statment but it is idiomatic and I probably shouldn't have. "done" I labeled as statement but not accept or acknowledge. I think it is truly different than "okay" or "alright" which can happen before the action is performed. I also think the acceptance and acknowledgment in "done" is implicit. I labeled "Pardon me?" as an information request since it means "please repeat yourself". One could say the same thing about "I didn't understand" but I didn't feel that way as I could imagine the TRIPS system happily never understanding what you last said. Treating "Pardon me?" as a question sets off one of the dat warnings as it expects an assert as an answer. It would be nice to fix that.
Mystery of Utterances 63 through 69
T54 utt63 U: WHERE ARE THE PEOPLE T55 utt64 S: There are 2 groups-of-people at Delta T56 utt65 U: WHERE ARE ALL THE PEOPLE T57 utt66 S: There are 6 groups-of-people at Delta and 1 at South Delta T58 utt67 U: USE THE HELICOPTER | utt68 U: TO GET THE PEOPLE FROM SOUTH_DELTA TO DELTA T59 utt69 S: There are no people at South Delta T60 utt70 U: USE THE HELICOPTER TO GET THE PEOPLE AT SOUTH_DELTA TO DELTA T61 utt71 S: Your wish is my command
My best guess: definite reference for THE PEOPLE in utt63 is wrong. It gets mapped to the lastest truckload of people which is 2 groups. ALL clarifies things and we get all 6. Utt63 is interpreted as "where are the people we just dropped off?". Utt67 and utt68 are interpreted as referrering to something like the people orginally from south_delta (none). The generator unfortunately chooses to express this by saying "at south delta". PEOPLE AT SOUTH_DELTA is resolved correctly to the people currently there.
1 S: Welcome to trips 97 version 3 point 2. 2 I'm ready to start. M - open option - let's start C - task-man or comm-man? manual gives TM precedence 3 U: HELLO 4 S: Hi M - no backward function for greetings 5 U: SHOW_ME A MAP OF PACIFICA M - showing map is answer C - initially marked as IR, revised later 6 S: Ok 7 U: CLEAR [CLEAR should be: WHERE ARE] THE PEOPLE 8 S: I'm afraid I didn't understand you. D - implicitly asking the speaker to repeat? M - indirect question? 9 U: WHERE ARE THE PEOPLE 10 S: There are 2 groups-of-people at Exodus, 2 at Calypso, 2 at Barnacle and 1 at Abyss A - I don't like coding all answers as acknowledging, but I feel I have no choice --- you say all accepts also acknowledge 11 U: WHERE ARE THE TRANSPORTS 12 S: There are 2 trucks and 1 helicopter at Delta D - not open option because the speaker is not suggesting potential actions. 13 U: USE A TRUCK [missed: TO] GET THE PEOPLE FROM CALYPSO 14 TO DELTA E - In Verbmobil, we perfer functional prosodic segmentation instead of auditory prosodic features to get rid of such boundaries 15 S: Ok B - ok, yes here often got an agreement tag because system seems to confirm not only message understanding (like in maptask or verbmobil) but also performance of the action 16 U: IT [IT mistakenly inserted] HOW_LONG WILL A [A should be THAT] TAKE 17 S: I'm afraid I didn't understand you. D - Not clear if it is a hold. The first question in the agreement decision tree is hard to answer. 18 U: HOW_LONG 19 WILL THAT TAKE 20 S: It will take 6 hours and 54 minutes 21 U: WHAT IF WE WENT ALONG [missed: THE] COAST INSTEAD D - If this were two people planning together, I would say open option. But here it looks like the user is just trying to get more information. C - no influence-on-S even if plural subject, just an idiom 22 S: I'm sorry, but that input didn't make sense to me. A - unclear whether S sees utt19 as open-option, so somewhat strange to code as "hold" --- I suppose coding from human perspective? 23 U: WHAT IF WE WENT ALONG THE COAST INSTEAD M - open-option, unaddressed by 24, dat warning comes up, yuck 24 S: That option would take 8 hours and 52 minutes A - unclear if S sees utt21 as open-option or straight info-request 25 U: I [I mistakenly inserted] FORGET IT C - assert, not influence-on-L, because of "I" 26 S: Ok C - no influence-on-S because of antecedent 27 U: USE THE OTHER TRUCK [missed: TO] GET THE PEOPLE AT EXODUS TO DELTA D - Was the 6 hours 54 minutes implicitly accepted when the 8 hours 52 minutes was rejected? 28 S: Ok 29 U: LET'S USE A HELICOPTER INSTEAD C - no "influence-on-S" because actions are of "s" 30 S: Alright 31 U: NOW SEND TRUCK ONE 32 TO BARNACLE 33 S: I'll get right on it 34 U: PICK UP THE PEOPLE THERE 35 S: Ok 36 U: GO ON TO ABYSS 37 S: Your wish is my command D - This and "I'll get right on it" are really just like "ok" coming from this machine, but I tagged them differently. 38 U: AND TAKE THE PEOPLE THERE TO DELTA 39 S: Alright 40 U: THAT LOOKS GOOD 41 S: No problem A - not sure what system means; take as confirming plan works C - not quite sure what utt41 does D - acknowledgements that are not accepts are usually communication management. 42 U: DELTA_BRIDGE IS OUT F - i need to know more about task to label this correctly 43 S: The highlighted actions may need to be revised. F - i need to know more about task to label this correctly C - could it be TM? could it be act-Dir and/or Offer? would need info-rel as well 44 Ok F - can't label this without knowing more about the task D - This "ok" doesn't seem to be well-formed. I'm assuming that it is a signal understanding that would have sounded better if it came before utterance 44. B - s maintains the turn by uttering ok C - not sure about function of this OK A - don't understand why S says OK here. Ready to listen again? M - accept knowledge of 42 45 U: SEND_A [A mistakenly inserted] TRUCK TWO 46 ALONG THE COAST INSTEAD 47 S: Sorry, I don't know how to do that. C - interpreted as "I can't comply" instead of "I don't understand" 48 U: SEND TRUCK TWO 49 ALONG THE COAST INSTEAD 50 S: Done C - uncertain about commit because action already done 51 U: HAVE THE OTHER TRUCK STOP AT SOUTH_DELTA INSTEAD 52 S: Alright 53 U: UNLOAD A [A should be THE] PEOPLE 54 S: Pardon me? D - responses to directives usually have an agreement feature C - comment=performative because equivalent to "sorry" (???) 55 U: UNLOAD 56 THE PEOPLE 57 S: Fine C - comment=different from "ok, alright" as statement? 58 U: USE THE HELICOPTER [missed TO] GET THE PEOPLE FROM SOUTH_DELTA TO DELTA 59 S: Alright 60 U: WHERE ARE THE PEOPLE 61 LEFT [LEFT mistakenly inserted] 62 S: I don't understand your reference to the people left D - I thought I saw somewhere in the manual that "I don't know" is a reject because it's like saying I can't open the door because my arm is broken. 63 U: WHERE ARE THE PEOPLE 64 S: There are 2 groups-of-people at Delta 65 U: WHERE ARE ALL THE PEOPLE 66 S: There are 6 groups-of-people at Delta and 1 at South Delta 67 U: USE THE HELICOPTER 68 TO GET THE PEOPLE FROM SOUTH_DELTA TO DELTA 69 S: There are no people at South Delta C - don't understand utt69 see 66 A - could be hold, not reject; submissive systems don't ever reject plans but point up flaws. M - misunderstanding, but system doesn't know. can we mark this separately from signal non-understanding 70 U: USE THE HELICOPTER TO GET THE PEOPLE AT SOUTH_DELTA TO DELTA 71 S: Your wish is my command 72 U: [user selects simulate plan from a menu] F - how to label this? D - no verbal action/ no tags C - should it be "commit"? but actions are of the system 73 S: Simulation succeeded in 94 percent of 250 runs. C - comment=should it be accept? no, don't think so (see discussion about "done" in utt50) 74 7 percent of the plan failure points were a move action with truck two resulting in a breakdown.
Click here to send in comments or questions.