TY - GEN
T1 - Evorus
T2 - 30th Annual ACM Symposium on User Interface Software and Technology, UIST 2017
AU - Huang, Ting Hao
AU - Chang, Joseph Chee
AU - Swaminathan, Saiganesh
AU - Bigham, Jeffrey P.
N1 - Publisher Copyright:
Copyright © 2017 is held by the owner/author(s).
PY - 2017/10/20
Y1 - 2017/10/20
N2 - Crowd-powered conversational assistants have found to be more robust than automated systems, but do so at the cost of higher response latency and monetary costs. One promising direction is to combined the two approaches for high quality and low cost solutions. However, traditional offline approaches of building automated systems with the crowd requires first collecting training data from the crowd, and then training a model before an online system can be launched. In this paper, we introduce Evorus, a crowd-powered conversational assistant with online-learning capability that automate itself over time. Evorus expands a previous crowd-powered conversation system by reducing its reliance on the crowd over time while maintaining the robustness and reliability of human intelligence, by (i) allowing new chatbots to be added to help contribute possible answers, (ii) learning to reuse past responses to similar queries over time, and (iii) learning to reduce the amount of crowd oversight necessary to retain quality. Our deployment study with 28 users show that automated responses were chosen 12.84% of the time, and voting cost was reduced by 6%. Evorus introduced a new framework for constructing crowd-powered conversation systems that can gradually automate themselves using machine learning, a concept that we believe can be generalize to other types of crowd-powered systems for future research.
AB - Crowd-powered conversational assistants have found to be more robust than automated systems, but do so at the cost of higher response latency and monetary costs. One promising direction is to combined the two approaches for high quality and low cost solutions. However, traditional offline approaches of building automated systems with the crowd requires first collecting training data from the crowd, and then training a model before an online system can be launched. In this paper, we introduce Evorus, a crowd-powered conversational assistant with online-learning capability that automate itself over time. Evorus expands a previous crowd-powered conversation system by reducing its reliance on the crowd over time while maintaining the robustness and reliability of human intelligence, by (i) allowing new chatbots to be added to help contribute possible answers, (ii) learning to reuse past responses to similar queries over time, and (iii) learning to reduce the amount of crowd oversight necessary to retain quality. Our deployment study with 28 users show that automated responses were chosen 12.84% of the time, and voting cost was reduced by 6%. Evorus introduced a new framework for constructing crowd-powered conversation systems that can gradually automate themselves using machine learning, a concept that we believe can be generalize to other types of crowd-powered systems for future research.
UR - http://www.scopus.com/inward/record.url?scp=85041422598&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85041422598&partnerID=8YFLogxK
U2 - 10.1145/3131785.3131823
DO - 10.1145/3131785.3131823
M3 - Conference contribution
AN - SCOPUS:85041422598
T3 - UIST 2017 Adjunct - Adjunct Publication of the 30th Annual ACM Symposium on User Interface Software and Technology
SP - 155
EP - 157
BT - UIST 2017 Adjunct - Adjunct Publication of the 30th Annual ACM Symposium on User Interface Software and Technology
PB - Association for Computing Machinery, Inc
Y2 - 22 October 2017 through 25 October 2017
ER -