TY - GEN
T1 - Towards building a scholarly big data platform
T2 - 2014 14th IEEE/ACM Joint Conference on Digital Libraries, JCDL 2014
AU - Wu, Zhaohui
AU - Wu, Jian
AU - Khabsa, Madian
AU - Williams, Kyle
AU - Chen, Hung Hsuan
AU - Huang, Wenyi
AU - Tuarob, Suppawong
AU - Choudhury, Sagnik Ray
AU - Ororbia, Alexander
AU - Mitra, Prasenjit
AU - Giles, C. Lee
N1 - Publisher Copyright:
© 2014 IEEE.
PY - 2014/12/1
Y1 - 2014/12/1
N2 - We introduce a big data platform that provides various services for harvesting scholarly information and enabling efficient scholarly applications. The core architecture of the platform is built on a secured private cloud, crawls data using a scholarly focused crawler that leverages a dynamic scheduler, processes by utilizing a map reduce based crawl-extraction-ingestion (CEI) workflow, and is stored in distributed repositories and databases. Services such as scholarly data harvesting, information extraction, and user information and log data analytics are integrated into the platform and provided by an OAI and RESTful API. We also introduce a set of scholarly applications built on top of this platform including citation recommendation and collaborator discovery.
AB - We introduce a big data platform that provides various services for harvesting scholarly information and enabling efficient scholarly applications. The core architecture of the platform is built on a secured private cloud, crawls data using a scholarly focused crawler that leverages a dynamic scheduler, processes by utilizing a map reduce based crawl-extraction-ingestion (CEI) workflow, and is stored in distributed repositories and databases. Services such as scholarly data harvesting, information extraction, and user information and log data analytics are integrated into the platform and provided by an OAI and RESTful API. We also introduce a set of scholarly applications built on top of this platform including citation recommendation and collaborator discovery.
UR - http://www.scopus.com/inward/record.url?scp=84919397810&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84919397810&partnerID=8YFLogxK
U2 - 10.1109/JCDL.2014.6970157
DO - 10.1109/JCDL.2014.6970157
M3 - Conference contribution
AN - SCOPUS:84919397810
T3 - Proceedings of the ACM/IEEE Joint Conference on Digital Libraries
SP - 117
EP - 126
BT - 2014 IEEE/ACM Joint Conference on Digital Libraries, JCDL 2014
PB - Institute of Electrical and Electronics Engineers Inc.
Y2 - 8 September 2014 through 12 September 2014
ER -