CARVIEW |
EMMA: Extensible MultiModal Annotation 1.0:
Implementation Report Plan
23 September 2008
- Editor:
- Michael Johnston, AT&T
- Authors:
- Paolo Baggia, Loquendo
- Patrizio Bergallo, Loquendo
- Daniel C. Burnett, Nuance
- Jerry Carter, Nuance
- Deborah Dahl, Invited Expert
Copyright © 2008 W3C ® ( MIT , ERCIM , Keio), All Rights Reserved. W3C liability, trademark, document use rules apply.
Table of Contents
- 1. Introduction
- 2. Work during the Candidate Recommendation period
- 3. Participating in the implementation report
- 4. Entrance criteria for the Proposed Recommendation phase
- 5. Implementation report requirements
- 6. Systems
- 7. Test assertions
-
Appendices
- Appendix A. Implementation report submission format
- Appendix B. Acknowledgements
1. Introduction
The EMMA Specification entered the Candidate Recommendation period on 14 April 2008.
Preparation of an Implementation Report is a key criterion for moving beyond the Candidate Recommendation phase. This document describes the requirements for the Implementation Report and the process that the Multimodal Working Group will follow in preparing the report.
Note: This Implementation Report Plan was modified as follows:
- 22 January 2008
-
- The Candidate Recommendation period is modified from "?? 2007" to "14 April 2008" as defined in the Candidate Recommendation.
- 23 September 2008
-
- Test assertions 801, 902, 903 and 1501 have been removed, because they are actually not described in the EMMA specification.
1.1 Implementation report objectives
- Must verify that the specification is implementable.
1.2 Implementation report non-objectives
- The IR does not attempt conformance testing of implementations.
2. Work during the Candidate Recommendation period
During the CR period, the Working Group will carry out the following activities:
- Clarification and improvement of the exposition of the specification.
- Disposing of comments that are communicated to the WG during the CR period.
- Preparation of an Implementation Report meeting the criteria outlined in this document.
3. Participating in the implementation report
You are invited to contribute to the assessment of the W3C EMMA 1.0 Specification by participating in the Implementation Report process.
- Deadline for submission of a EMMA Implementation Report is February 29th 2008.
- All the test assertions are reported in the table below.
- Comments on this document or requests for further information should be sent to the Working Group's public mailing list www-multimodal@w3.org (archive). If an updated version of this document or the test assertion set is published, a notification will be sent to this mailing list.
4. Entrance criteria for the Proposed Recommendation phase
The Multimodal Working Group established the following entrance criteria for the Proposed Recommendation phase in the Request for CR:
- Sufficient reports of implementation experience have been gathered to demonstrate that EMMA processors based on the specification are implementable and have compatible behavior.
- Specific Implementation Report Requirements (outlined below) have been met.
- The Working Group has formally addressed and responded to all public comments received by the Working Group.
5. Implementation report requirements
5.1 Detailed requirements for implementation report
- Testimonials from implementers will be included in the IR when provided to document the utility and implementability of the specification.
- IR must cover all specified features in the specification. For
each feature the IR should indicate:
- Feature status: required, optional, other.
- Feature utility/usefulness based on feedback from implementers.
- Implementability of the feature specification.
- Feature status is a factor in test coverage in the report:
- Required specification features must have at least two implementations. Implementations that do not implement a required specification feature must document the reason for not implementing the feature.
- Optional specification features must have at least one implementation. Implementations that do not implement an optional specification feature should document the reason for not implementing the feature.
5.2 Notes on testing
- A implementation report must indicate the outcome of evaluating the implementation with respect to each of the test assertions. Possible
outcomes are "
pass
", "fail
" or "not-impl
". "
5.3 Out of scope
EMMA Implementation Report will not cover:
- Integration with other markup languages (VoiceXML/XHTML/SVG/SRGS)
6. Systems
Note: The testimonials with pink background from "Acme Labs" and "Beta Inc." are just examples and will be replaced with the actually submitted testimonials.
Acme Labs
Exec Summary
The W3C EMMA 1.0 Specification is well-written, easily implementable and extremely useful. Acme Labs used it to describe the recipe for haggis.
Beta Inc.
Exec Summary
The W3C EMMA 1.0 Specification is the best thing since sliced bread, extremely useful, easily implementable and well-written. Beta Inc. used it to build a new generation of interactive services, establish world peace, build a new chianti-spaghetti hybrid vehicle, and mix a perfect martini.
7. Test assertions
The aim of this section is to describe the range of test assertions developed for the EMMA 1.0 Specification. The table lists all the assertions that were derived from the EMMA 1.0 Specification.
The Assert ID column uniquely identifies the assertion. The Feature column indicates the specific elements or attributes which the test assertion applies to. The Spec column identifies the section of the EMMA 1.0 Specification from which the assertion was derived. The REQ column is a Y/N value indicating whether the test assertion is for a feature which is required. The SUB column is a Y/N value indicating whether the test assertion is a subconstraint which is dependent on the implementation of the preceding non subconstraint feature test assertion. The Semantics column specifies the semantics of the feature or the constraint which must be met. The Result column will be annotated with the number of 'pass', 'fail', and 'not implemented' (P/F/NI) in the set of implementation reports.
7.1 Classification of test assertions
Test assertions are classified into two types, basic test assertions which test for the presence of each feature, and sub constraints which only apply if that particular feature is implemented. Generally, sub constraints encode structural constraints that could not be expressed in the EMMA schema. Sub constraints are marked with 'SUB CONSTRAINT:' in the Semantics field.
7.2 EMMA XML Schema conformance
The most fundamental test of a conforming EMMA implementation is that the EMMA documents it utilizes must successfully validate with respect to the EMMA XML Schema.
7.3 EMMA Test assertions
Assert ID | Feature | Spec | Req | Sub | Semantics | Results | ||
---|---|---|---|---|---|---|---|---|
P | F | NI | ||||||
Structural Elements | ||||||||
100 | emma:emma | [3.1] | Y | N | EMMA documents MUST have emma:emma as the root element. | |||
200 | emma:interpretation | [3.2] | Y | N | Application namespace markup representing the interpretation of inputs MUST be contained within the emma:interpretation element. | |||
201 | [3.2] | Y | Y | SUB CONSTRAINT: If the emma:interpretation element is empty, then it MUST be annotated as emma:uninterpreted="true' or emma:no-input="true". | ||||
300 | emma:one-of | [3.3.1] | Y | N | EMMA N-best interpretations MUST be contained within an emma:one-of element. | |||
301 | [3.3.1] | Y | Y | SUB CONSTRAINT: EMMA N-best interpretations contained within an emma:one-of element MUST be ordered best-first in document order where the measure is emma:confidence if present, otherwise the quality metric is platform specific. | ||||
310 | [3.3.1] | Y | Y | SUB CONSTRAINT: If an emma:one-of element contains other emma:one-of elements, (embedded one-of), emma:one-of elements MUST contain a disjunction-type attribute indicating the reason for the multiple interpretations. | ||||
400 | emma:group | [3.3.2] | N | N | Interpretations of distinct inputs MAY be represented in a single EMMA document within the emma:group element. | |||
402 | emma:group-info | [3.3.2.1] | N | N | Information describing the criteria used in determining the grouping of interpretations within an emma:group MAY be indicated within an emma:group-info child element within emma:group. | |||
500 | emma:sequence | [3.3.3] | N | N | A temporally ordered sequence of distinct inputs MAY be contained within an emma:sequence element with document order corresponding to temporal order. | |||
600 | emma:lattice | [3.4] | N | N | Lattices SHOULD be represented as an emma:lattice element containing a series of emma:arc and emma:node elements. | |||
602 | [3.4.1] | Y | Y | SUB CONSTRAINT: The value of the initial attribute on emma:lattice MUST be the value of the from attribute on at least one of the emma:arc elements that it contains. | ||||
603 | [3.4.1] | Y | Y | SUB CONSTRAINT: The value of each space separated number within the final attribute on emma:lattice MUST be the value of the to attribute on at least one of the emma:arc elements that it contains. | ||||
604 | [3.4.1] | Y | Y | SUB CONSTRAINT: The value of a to attribute on emma:arc MUST be the value of a from attribute on at least one of its emma:arc siblings or be one of the final values on the parent emma:lattice element. | ||||
605 | [3.4.1] | Y | Y | SUB CONSTRAINT: The value of a from attribute on emma:arc MUST be the value of a to attribute on at least one of its emma:arc siblings or be the initial value on the parent emma:lattice element. | ||||
606 | [3.4.1] | Y | Y | SUB CONSTRAINT: Any epsilon transitions in the lattice MUST be represented as emma:arc elements with no content other than emma:info. | ||||
607 | [3.4.1] | Y | Y | SUB CONSTRAINT: The content of the emma:arc element MUST be either an application namespace XML element, well-formed character content, an application namespace element and an emma:info element, or well-formed character content and an emma:info element. | ||||
608 | [3.4.1] | Y | Y | SUB CONSTRAINT: There MUST be no more than one emma:node specification for each numbered node in the lattice. | ||||
609 | [3.4.1] | Y | Y | SUB CONSTRAINT: For each emma:node specification, the value of the node-number attribute MUST appear as the value of at some from or to attribute in one of the sibling emma:arc elements. | ||||
700 | emma:literal | [3.5] | Y | N | String literal semantic interpretations MUST be wrapped with the tag emma:literal. | |||
Annotation Elements | ||||||||
800 | emma:model | [4.1.1] | N | N | The data model of the application semantic markup MAY appear within the emma:model element. | |||
810 | [4.2.16] | Y | Y | SUB CONSTRAINT: If emma:model appears as a child of emma:emma, there MUST be emma:one-of or emma:interpretation elements within emma:emma with emma:model-ref attributes that refer to the id of the emma:model child of emma:emma. | ||||
811 | [4.2.16] | Y | Y | SUB CONSTRAINT: If emma:model-ref appears on an emma:interpretation or emma:one-of, there MUST be an emma:model child of the dominating emma:emma whose id value equals the value of the emma:model-ref attribute. | ||||
901 | emma:derived-from | [4.1.2] | N | N | The interpretation of all but the first stage of processing MAY contain an emma:derived-from element with an attribute resource which refers to the interpretation from which that interpretation was derived. | |||
904 | [4.1.2] | Y | Y | SUB CONSTRAINT:If there is more than one emma:derived-from within an emma:interpretation or emma:one-of, then all of the emma:derived-from elements MUST have the attribute composite="true". | ||||
910 | emma:derivation | [4.1.2] | Y | N | The interpretation resulting of the most recent processing step MUST appear as a child of emma:emma. | |||
911 | emma:derivation | [4.1.2] | N | N | If interpretations resulting from earlier processing steps are included they SHOULD appear as children of a single emma:derivation element within emma:emma. | |||
1000 | emma:grammar | [4.1.3] | N | N | The grammar used MAY be referenced using an emma:grammar element within emma:emma. | |||
1001 | [4.2.15] | Y | Y | SUB CONSTRAINT: If an emma:interpretation or emma:one-of is annotated with the emma:grammar-ref attribute, then the value of the emma:grammar-ref MUST reference the id of an emma:grammar element child of emma:emma in the same document. | ||||
1002 | [4.2.15] | Y | Y | SUB CONSTRAINT: If an emma:grammar element appears as a child of emma:emma, then there MUST be an emma:interpretation or emma:one-of element dominated by that emma:emma, whose emma:grammar-ref attribute references the id of the emma:grammar element. | ||||
1100 | emma:info | [4.1.4] | N | N | Application and vendor specific annotations SHOULD appear within emma:info. | |||
Annotation attributes | ||||||||
1201 | emma:tokens | [4.2.1] | N | N | emma:tokens MAY be used to indicate the sequence of tokens recognized in the input by the recognizer. | |||
1300 | emma:process | [4.2.2] | N | N | emma:process MAY be used to identify the process used to assign the interpretation. | |||
1400 | emma:no-input | [4.2.3] | Y | N | emma:no-input="true" MUST be used to designate lack of expected input. | |||
1500 | emma:uninterpreted | [4.2.4] | Y | N | Input for which the processor is unable to assign an interpretation MUST be marked as emma:uninterpreted="true". | |||
1600 | emma:lang | [4.2.5] | N | N | emma:lang MAY be used to designate the human language of input | |||
1700 | emma:signal | [4.2.6] | N | N | emma:signal (with a URI value) MAY be used to designate the location of the signal processed by an EMMA processor. | |||
1701 | emma:signal-size | [4.2.6] | N | N | emma:signal-size MAY be used to indicate the file size in 8-bit octets of the input signal. | |||
1800 | emma:media-type | [4.2.7] | N | N | emma:media-type MAY be used to designate the MIME type of the processed signal. | |||
1900 | emma:confidence | [4.2.8] | N | N | emma:confidence MAY be used to designate the confidence score of an input. | |||
2000 | emma:source | [4.2.9] | N | N | emma:source (with a URI value) MAY be used to designate the source device. | |||
2100 | emma:start | [4.2.10.1] | N | N | emma:start with a millisecond value MAY be used to indicate the absolute start time of an input. | |||
2101 | emma:end | [4.2.10.1] | N | N | emma:end with a millisecond value MAY be used to indicate the absolute end time of an input. | |||
2201 | emma:time-ref-uri | [4.2.10.2] | N | N | The emma:time-ref-uri attribute MAY be used to indicate the URI used as a reference point for relative timestamps. | |||
2202 | emma:time-ref-anchor | [4.2.10.2] | Y | Y | SUB CONSTRAINT: If the emma:time-ref-uri points to an interval, emma:time-ref-anchor MUST appear on the interpretation with a value of start or end, indicating whether to use the start or end of that interval as the reference point. | |||
2203 | emma:offset-to-start | [4.2.10.2] | N | N | The value of emma:offset-to-start MAY be used to indicate the number of milliseconds from the reference point to the start of the current input in a relative timestamp. | |||
2204 | emma:duration | [4.2.10.3] | N | N | The emma:duration attribute value MAY be used to indicate the duration in milliseconds of the current input. | |||
2300 | emma:medium | [4.2.11] | Y | N | emma:medium MUST be included on all EMMA inputs, indicating the medium of input. | |||
2301 | emma:medium | [4.1.2] | Y | Y | SUBCONSTRAINT: When EMMA inputs from different modes are combined to make a composite input, if the combining inputs have different values for emma:medium, then the value on the result MUST be a space separated list of the emma:medium values from the combining modes. | |||
2310 | emma:mode | [4.2.11] | Y | N | emma:mode MUST be included on all EMMA inputs, indicating the mode of input. | |||
2311 | emma:mode | [4.1.2] | Y | Y | SUBCONSTRAINT: When EMMA inputs from different modes are combined to make a composite input, the value of emma:mode on the result MUST be a space separated list of the values on the combining inputs. | |||
2320 | emma:function | [4.2.11] | N | N | emma:function MAY be included on EMMA inputs, providing a classification of the function of the input. | |||
2330 | emma:verbal | [4.2.11] | N | N | emma:verbal MAY be included on EMMA inputs, indicating whether the input is verbal or non-verbal. | |||
2401 | emma:hook | [4.2.12] | N | N | emma:hook MAY be used on application namespace markup to indicate where content from another mode needs to be integrated. | |||
2500 | emma:cost | [4.2.13] | N | N | emma:cost MAY be used to indicate the cost or weight of an interpretation or lattice arc. | |||
2510 | emma:dialog-turn | [4.2.17] | N | N | emma:dialog-turn MAY be used to indicate an identifier for the current dialog turn. | |||
Scope and inheritance of annotations | ||||||||
2600 | emma:one-of | [3.3.1] | Y | N | EMMA annotations appearing on emma:one-of MUST be true of all of the container elements (emma:one-of,emma:interpretation,emma:sequence,emma:group) within the emma:one-of. | |||
2601 | emma:derived-from | [4.3] | Y | N | Each EMMA annotation from the previous stage of a sequential derivation (indicated using emma:derived-from) MUST be true of the interpretations in the current stage of the derivation, unless the annotation is explicitly re-stated. | |||
Endpoint elements and attributes | ||||||||
2700 | emma:endpoint-info | [4.1.5] | N | N | The emma:endpoint-info element MAY be used to contain emma:endpoint elements describing the characteristics of the endpoints. | |||
2701 | emma:endpoint | [4.1.5] | N | N | The emma:endpoint element MAY be used to provide annotations regarding a communication endpoint. | |||
2710 | emma:endpoint-role | [4.2.14] | N | N | The emma:endpoint-role attribute MAY be used to indicate the role of an endpoint. | |||
2711 | emma:endpoint-address | [4.2.14] | N | N | The emma:endpoint-address attribute MAY be used to indicate the address of an endpoint. | |||
2713 | emma:port-type | [4.2.14] | N | N | The emma:port-type attribute MAY be used to indicate the type of port of an endpoint. | |||
2714 | emma:port-num | [4.2.14] | N | N | The emma:port-num attribute MAY be used to indicate the port number of an endpoint. | |||
2715 | emma:message-id | [4.2.14] | N | N | The emma:message-id attribute MAY be used to indicate the message id of an endpoint. | |||
2716 | emma:service-name | [4.2.14] | N | N | The emma:service-name attribute MAY be used to indicate name of service that the system provides. | |||
2717 | emma:endpoint-pair-ref | [4.2.14] | N | N | The emma:endpoint-pair-ref attribute MAY be used to indicate the pairing between sink and source endpoints. | |||
2718 | emma:endpoint-info-ref | [4.2.14] | N | N | The emma:endpoint-info-ref attribute MAY be used to reference the endpoint-info element associated with an interpretation. |
Appendices
Appendix A - Implementation report submission format
The following XML should be used as a template for providing an implementation report.
<system-report name="YOUR-SYSTEM-NAME-HERE"> <testimonial>YOUR-WELL-FORMED-TESTIMOMIAL-CONTENT-HERE</testimonial> <assert id="100" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="200" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="201" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="300" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="301" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="310" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="400" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="402" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="500" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="600" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="602" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="603" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="604" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="605" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="606" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="607" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="608" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="609" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="700" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="800" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="810" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="811" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="901" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="904" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="910" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="911" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="1000" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="1001" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="1002" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="1100" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="1201" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="1300" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="1400" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="1500" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="1600" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="1700" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="1701" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="1800" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="1900" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="2000" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="2100" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="2101" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="2201" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="2202" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="2203" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="2204" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="2300" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="2301" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="2310" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="2311" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="2320" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="2330" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="2401" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="2500" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="2510" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="2600" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="2601" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="2700" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="2701" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="2710" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="2711" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="2713" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="2714" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="2715" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="2716" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="2717" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> <assert id="2718" res="pass|fail|not-impl">OPTIONAL-NOTES-HERE</assert> </system-report>
Appendix B - Acknowledgements
The Multimodal Working Group would like to acknowledge the contributions of several individuals:
- Kazuyuki Ashimura, W3C
- Jin Liu, T-Systems
- Dave Raggett, Volantis, W3C
- Massimo Romanelli, DFKI