Combining Non-Expert and Expert Crowd Work to ConvertWeb APIs to Dialog Systems

Ting Hao K. Huang; Walter S. Lasecki; Alan L. Ritter; Jeffrey P. Bigham

Combining Non-Expert and Expert Crowd Work to ConvertWeb APIs to Dialog Systems

Ting Hao K. Huang
, Walter S. Lasecki
, Alan L. Ritter
, Jeffrey P. Bigham

College of Information Sciences and Technology

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

1 Scopus citations

Abstract

Thousands of web APIs expose data and services that would be useful to access with natural dialog, from weather and sports to Twitter and movies. The process of adapting each API to a robust dialog system is difficult and time-consuming, as it requires not only programming but also anticipating what is mostly likely to be asked and how it is likely to be asked. We present a crowd-powered system able to generate a natural language interface for arbitrary web APIs from scratch without domain-dependent training data or knowledge. Our approach combines two types of crowd workers: non-expert Mechanical Turk workers interpret the functions of the API and elicit information from the user, and expert oDesk workers provide a minimal sufficient scaffolding around the API to allow us to make general queries. We describe our multi-stage process and present results for each stage.

Original language	English (US)
Title of host publication	Proceedings of the 2nd AAAI Conference on Human Computation and Crowdsourcing, HCOMP 2014
Editors	Jeffrey P. Bigham, David Parkes
Publisher	AAAI press
Pages	22-23
Number of pages	2
ISBN (Electronic)	9781577356820
State	Published - Nov 5 2014
Event	2nd AAAI Conference on Human Computation and Crowdsourcing, HCOMP 2014 - Pittsburgh, United States Duration: Nov 2 2014 → Nov 4 2014

Publication series

Name	Proceedings of the 2nd AAAI Conference on Human Computation and Crowdsourcing, HCOMP 2014

Conference

Conference	2nd AAAI Conference on Human Computation and Crowdsourcing, HCOMP 2014
Country/Territory	United States
City	Pittsburgh
Period	11/2/14 → 11/4/14

All Science Journal Classification (ASJC) codes

Computational Theory and Mathematics
Human-Computer Interaction

Cite this

Huang, T. H. K., Lasecki, W. S., Ritter, A. L., & Bigham, J. P. (2014). Combining Non-Expert and Expert Crowd Work to ConvertWeb APIs to Dialog Systems. In J. P. Bigham, & D. Parkes (Eds.), Proceedings of the 2nd AAAI Conference on Human Computation and Crowdsourcing, HCOMP 2014 (pp. 22-23). (Proceedings of the 2nd AAAI Conference on Human Computation and Crowdsourcing, HCOMP 2014). AAAI press.

Huang, Ting Hao K. ; Lasecki, Walter S. ; Ritter, Alan L. et al. / Combining Non-Expert and Expert Crowd Work to ConvertWeb APIs to Dialog Systems. Proceedings of the 2nd AAAI Conference on Human Computation and Crowdsourcing, HCOMP 2014. editor / Jeffrey P. Bigham ; David Parkes. AAAI press, 2014. pp. 22-23 (Proceedings of the 2nd AAAI Conference on Human Computation and Crowdsourcing, HCOMP 2014).

@inproceedings{00c0f54ef86f4df1822812365b8b0432,

title = "Combining Non-Expert and Expert Crowd Work to ConvertWeb APIs to Dialog Systems",

abstract = "Thousands of web APIs expose data and services that would be useful to access with natural dialog, from weather and sports to Twitter and movies. The process of adapting each API to a robust dialog system is difficult and time-consuming, as it requires not only programming but also anticipating what is mostly likely to be asked and how it is likely to be asked. We present a crowd-powered system able to generate a natural language interface for arbitrary web APIs from scratch without domain-dependent training data or knowledge. Our approach combines two types of crowd workers: non-expert Mechanical Turk workers interpret the functions of the API and elicit information from the user, and expert oDesk workers provide a minimal sufficient scaffolding around the API to allow us to make general queries. We describe our multi-stage process and present results for each stage.",

author = "Huang, \{Ting Hao K.\} and Lasecki, \{Walter S.\} and Ritter, \{Alan L.\} and Bigham, \{Jeffrey P.\}",

note = "Publisher Copyright: {\textcopyright} HCOMP 2014. All rights reserved.; 2nd AAAI Conference on Human Computation and Crowdsourcing, HCOMP 2014 ; Conference date: 02-11-2014 Through 04-11-2014",

year = "2014",

month = nov,

day = "5",

language = "English (US)",

series = "Proceedings of the 2nd AAAI Conference on Human Computation and Crowdsourcing, HCOMP 2014",

publisher = "AAAI press",

pages = "22--23",

editor = "Bigham, \{Jeffrey P.\} and David Parkes",

booktitle = "Proceedings of the 2nd AAAI Conference on Human Computation and Crowdsourcing, HCOMP 2014",

}

Huang, THK, Lasecki, WS, Ritter, AL & Bigham, JP 2014, Combining Non-Expert and Expert Crowd Work to ConvertWeb APIs to Dialog Systems. in JP Bigham & D Parkes (eds), Proceedings of the 2nd AAAI Conference on Human Computation and Crowdsourcing, HCOMP 2014. Proceedings of the 2nd AAAI Conference on Human Computation and Crowdsourcing, HCOMP 2014, AAAI press, pp. 22-23, 2nd AAAI Conference on Human Computation and Crowdsourcing, HCOMP 2014, Pittsburgh, United States, 11/2/14.

Combining Non-Expert and Expert Crowd Work to ConvertWeb APIs to Dialog Systems. / Huang, Ting Hao K.; Lasecki, Walter S.; Ritter, Alan L. et al.
Proceedings of the 2nd AAAI Conference on Human Computation and Crowdsourcing, HCOMP 2014. ed. / Jeffrey P. Bigham; David Parkes. AAAI press, 2014. p. 22-23 (Proceedings of the 2nd AAAI Conference on Human Computation and Crowdsourcing, HCOMP 2014).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Combining Non-Expert and Expert Crowd Work to ConvertWeb APIs to Dialog Systems

AU - Huang, Ting Hao K.

AU - Lasecki, Walter S.

AU - Ritter, Alan L.

AU - Bigham, Jeffrey P.

PY - 2014/11/5

Y1 - 2014/11/5

N2 - Thousands of web APIs expose data and services that would be useful to access with natural dialog, from weather and sports to Twitter and movies. The process of adapting each API to a robust dialog system is difficult and time-consuming, as it requires not only programming but also anticipating what is mostly likely to be asked and how it is likely to be asked. We present a crowd-powered system able to generate a natural language interface for arbitrary web APIs from scratch without domain-dependent training data or knowledge. Our approach combines two types of crowd workers: non-expert Mechanical Turk workers interpret the functions of the API and elicit information from the user, and expert oDesk workers provide a minimal sufficient scaffolding around the API to allow us to make general queries. We describe our multi-stage process and present results for each stage.

AB - Thousands of web APIs expose data and services that would be useful to access with natural dialog, from weather and sports to Twitter and movies. The process of adapting each API to a robust dialog system is difficult and time-consuming, as it requires not only programming but also anticipating what is mostly likely to be asked and how it is likely to be asked. We present a crowd-powered system able to generate a natural language interface for arbitrary web APIs from scratch without domain-dependent training data or knowledge. Our approach combines two types of crowd workers: non-expert Mechanical Turk workers interpret the functions of the API and elicit information from the user, and expert oDesk workers provide a minimal sufficient scaffolding around the API to allow us to make general queries. We describe our multi-stage process and present results for each stage.

UR - https://www.scopus.com/pages/publications/84988368596

UR - https://www.scopus.com/pages/publications/84988368596#tab=citedBy

M3 - Conference contribution

AN - SCOPUS:84988368596

T3 - Proceedings of the 2nd AAAI Conference on Human Computation and Crowdsourcing, HCOMP 2014

SP - 22

EP - 23

BT - Proceedings of the 2nd AAAI Conference on Human Computation and Crowdsourcing, HCOMP 2014

A2 - Bigham, Jeffrey P.

A2 - Parkes, David

PB - AAAI press

T2 - 2nd AAAI Conference on Human Computation and Crowdsourcing, HCOMP 2014

Y2 - 2 November 2014 through 4 November 2014

ER -

Huang THK, Lasecki WS, Ritter AL, Bigham JP. Combining Non-Expert and Expert Crowd Work to ConvertWeb APIs to Dialog Systems. In Bigham JP, Parkes D, editors, Proceedings of the 2nd AAAI Conference on Human Computation and Crowdsourcing, HCOMP 2014. AAAI press. 2014. p. 22-23. (Proceedings of the 2nd AAAI Conference on Human Computation and Crowdsourcing, HCOMP 2014).

Combining Non-Expert and Expert Crowd Work to ConvertWeb APIs to Dialog Systems

Abstract

Publication series

Conference

All Science Journal Classification (ASJC) codes

Other files and links

Fingerprint

Cite this