Contents	6
	Contributing Authors	8
	Preface	16
	I EVALUATION METHODOLOGY	21
	Chapter 1 THE BLIND MEN AND THE ELEPHANT REVISITED	23
	1. Introduction	24
	2. The Taxonomy	26
	2.1 Believability	32
	2.2 Sociability	34
	2.3 Task and Application Domains	36
	2.4 Agency and Computational Issues	37
	3. Applying the Taxonomy	38
	4. On Production Values	40
	5. Further Classi.cation	41
	6. Conclusions	42
	Notes	43
	References	43
	Chapter 2 EMBODIED CONVERSATIONAL AGENTS ON A COMMON GROUND	47
	1. Introduction	48
	1.1 Motivations and Problems	49
	2. ECAs from a Design Perspective	50
	2.1 The Embodiment	52
	2.2 The Mental Aspects	56
	2.3 Implementation Aspects	59
	2.4 Range of Applicability	60
	3. On Evaluation Methodology	60
	3.1 Why to Evaluate?	61
	3.2 How to De.ne the Evaluation Variables?	66
	3.3 Testing by what Users?	68
	3.4 How to Collect and Evaluate Data?	71
	4. Dimensions of Evaluation	73
	4.1 Usability	74
	4.2 Evaluation of User Perception of ECAs	76
	5. Conclusions	81
	Notes	82
	References	83
	Chapter 3 EMPIRICAL EVALUATION METHODOLOGY FOR EMBODIED CONVERSATIONAL AGENTS	87
	1. Introduction	88
	2. Types of Evaluation Research	88
	2.1 Star Life Cycle	89
	2.2 Formative and Summative Evaluation	91
	3. Research Model, Research Question and Construct	92
	3.1 Research Model	92
	3.2 Research Questions	94
	3.3 Psychological Construct	95
	4. Research Strategy	96
	4.1 Survey	97
	4.2 Experiment	98
	4.3 Case Study	99
	5. Data Collection Methods	100
	5.1 Qualitative Methods	101
	5.2 Quantitative Methods	102
	5.3 More Data Collection Methods	106
	6. Samples, Reliability and Validity	106
	6.1 Samples	106
	6.2 Reliability	108
	6.3 Validity	110
	7. Data Analysis	112
	8. Concluding: Guidelines for Evaluating ECAs	114
	Acknowledgments	116
	Notes	116
	References	116
	Chapter 4 EVALUATING USERS’ REACTIONS TO HUMAN-LIKE INTERFACES	121
	1. Evaluation of Dialogue Systems	122
	1.1 Evaluation of User’s Satisfaction	122
	1.2 Evaluation Criteria for Multimodal Dialogue Systems	123
	2. Prosodic Cues and Non-verbal Behaviour as new Evaluation Measures	124
	2.1 Prosodic and Visual Cues of Emotions	125
	2.2 Prosodic and Visual Cues of Emotions and Evaluation	125
	3. The Investigation	126
	3.1 Material	126
	3.2 Analysis of Prosodic Cues	130
	3.3 Analysis of Non-verbal Communicative Behaviour	135
	4. Conclusions and Further Investigation	140
	Acknowledgments	141
	Notes	141
	References	141
	II THE USER IN FOCUS	145
	Chapter 5 USER-CENTRED DESIGN AND EVALUATION OF AFFECTIVE INTERFACES	147
	1. Introduction	148
	2. Underlying Philosophy and Method	149
	2.1 Our Philosophy	151
	2.2 Our Method	152
	3. Studies of Three Affective Interfaces	157
	4. Agneta & Frida	158
	4.1 Non-correlation of Measurements	159
	4.2 Narrative Experience	161
	4.3 Implications for Design Method	162
	5. The Influencing Machine	163
	5.1 Study Method	164
	5.2 The First In.uencing Machine Study	166
	5.3 The Second Influencing Machine Study	167
	5.4 Implications for Design Method	169
	6. SenToy and FantasyA	170
	6.1 Wizard of Oz	171
	6.2 Second Study of SenToy Used in FantasyA	173
	6.3 Implications for Design Method	174
	7. Discussion	175
	Acknowledgments	177
	Notes	177
	References	177
	Chapter 6 ‘USER AS ASSESSOR’ APPROACH TO EMBODIED CONVERSATIONAL AGENTS	181
	1. Introduction	182
	2. Attention as an Example of the ‘User as Assessor’ Approach	186
	2.1 How Do Humans Attend?	186
	2.2 Conceptualizing Apparent Attention	188
	3. Creating Apparently Attentive ECAs	190
	3.1 Apparent Selectivity in ECAs	190
	3.2 Apparent Breadth in ECAs	195
	4. Conclusion	199
	Notes	201
	References	201
	III EVALUATION OF ECAS	209
	Chapter 7 MORE ABOUT BROWS	211
	1. Introduction	212
	2. About Brows	213
	3. Materials	216
	3.1 Speech	216
	3.2 Animations	218
	4. Experiment 1: Subjective Preference	219
	4.1 Method	219
	4.2 Results (Dutch)	220
	4.3 Results (Italian)	220
	4.4 Discussion	221
	5. Experiment 2: Perceived Prominence	221
	5.1 Method	221
	5.2 Results (Dutch)	222
	5.3 Results (Italian)	224
	5.4 Discussion	224
	6. Experiment 3: Functional Analysis	225
	6.1 Method	225
	6.2 Results (Dutch)	226
	6.3 Results (Italian)	227
	6.4 Discussion	227
	7. General Discussion	228
	7.1 Eyebrows in Dutch and Italian	228
	7.2 About Analysis-by-Synthesis	230
	7.3 Analysis-by-observation	231
	Acknowledgments	233
	Notes	233
	References	234
	Chapter 8 EVALUATION OF MULTIMODAL BEHAVIOUR OF EMBODIED AGENTS	237
	1. Introduction	238
	2. Experimental Setting	240
	2.1 Participants	240
	2.2 Apparatus	241
	2.3 Scenarios	241
	2.4 Independent Variables	242
	2.5 Generation of Multimodal Behaviour	245
	2.6 Dependent Variables	246
	2.7 Data Analysis	247
	3. Results	247
	3.1 Subjective Variables	247
	3.2 Recall Performance	250
	4. Discussion	251
	4.1 Effects of Multimodal Strategies	252
	4.2 Effects of ECAs’ Appearance	253
	4.3 Additional Results	254
	5. Conclusions and Future Directions	254
	Acknowledgments	256
	Notes	256
	References	256
	Chapter 9 ECA AS USER INTERFACE PARADIGM	259
	1. Introduction	260
	1.1 Our Research Objectives	262
	2. Research Framework	264
	2.1 Features of the User	265
	2.2 Features of the ECA	267
	2.3 Features of the Task	270
	2.4 Interaction of Variables	273
	2.5 Approaches to Assessing ECAs	274
	3. An Experimental Study	274
	3.1 Goal and Hypotheses	275
	3.2 Participants, Materials, and Procedure	276
	3.3 Results	279
	3.4 Additional Observations from the Experiment	282
	4. Conclusions	283
	Notes	284
	References	284
	IV EVALUATION OF APPLICATIONS	289
	Chapter 10 TALKING TO DIGITAL FISH	291
	1. Introduction	292
	1.1 Evaluating Animated Characters in Educational Software	293
	1.2 Goals of the Study	294
	2. Methods	294
	2.1 Participants, Task, and Procedure	294
	2.2 Simulation Environment	296
	2.3 Text to Speech Manipulation	297
	2.4 Research Design and Analyses	298
	2.5 Data Coding and Dependent Measures	299
	3. Results	302
	3.1 Engagement in Interface and Ease of Use	302
	3.2 Distribution of Question Types	304
	3.3 Impact of TTS Voice Type on Child Queries	304
	4. Discussion	305
	4.1 Acoustic Characteristics of Animated Character Design	305
	4.2 Conversational Interfaces as Educational Interfaces	308
	4.3 Conclusion	309
	Acknowledgments	310
	Notes	310
	References	310
	Chapter 11 EXPERIMENTAL EVALUATION OF THE USE OF ECAS IN ECOMMERCE APPLICATIONS	313
	1. Introduction	314
	2. Experimental Approach	315
	3. The Phoebe Experiment	317
	3.1 Experimental Procedure	319
	3.2 The Usability Questionnaire	321
	3.3 Results	321
	3.4 Summary	324
	4. The Voice Personae Experiment	324
	4.1 Voice Personae	327
	4.2 Experimental Procedure	328
	4.3 Results	329
	4.4 Summary	332
	5. The Trust Experiment	332
	5.1 Experimental Procedure	335
	5.2 Results	335
	5.3 Summary	338
	6. Conclusion	339
	References	340
	Chapter 12 WHAT WE CAN LEARN FROM AVATAR-DRIVEN INTERNET COMMUNITIES	343
	1. Introduction	344
	2. The sysis NetLife	346
	2.1 The NetLife Platform	346
	2.2 The Flirtboat Application	347
	2.3 The derSpittelberg Application	350
	3. User Data Analysis	351
	3.1 Goals for Data Collection	352
	3.2 Methodology	353
	3.3 Generalized Avatar Pro.le	355
	3.4 Lifestyle Analysis	360
	4. Data Evaluation	361
	4.1 Evaluation Results	362
	5. Conclusion	370
	Acknowledgments	371
	Notes	371
	References	372
	More eBooks at www.ciando.com	0