Complex network theory based web services composition benchmark toolkit

In recent years, while many research proposals have been made toward novel algorithmic solutions of a myriad of web services composition problems, their validation has been less than satisfactory. One of the reasons for this problem is the lack of real benchmark web services data with which researchers can test and verify their proposals. In this chapter, to remedy this challenge, we present a novel benchmark toolkit, WSBen, which is capable of generating synthetic web services data with diverse scenarios and configurations using complex network theory. Web services researchers therefore can evaluate their web services discovery and composition algorithms in a more systematic fashion. The development of WSBen is inspired by our preliminary study on real-world web services crawled from the Web. The proposed WSBen can: (1) generate a collection of synthetic web services files in the WSDL format conforming to diverse complex network characteristics; (2) generate queries and ground truth sets for testing discovery and composition algorithms; (3) prepare auxiliary files to help further statistical analysis; (4) convert WSDL test sets to the formats that conventional AI planners can read; and (5) provide a graphical interface to control all these functions. To illustrate the application of the WSBen, in addition, we present case studies selected from three domains: (1) web services composition; (2) AI planning; and (3) the laws of networks in Physics community. The WSBen toolkit is available at: http://pike.psu.edu/sw/wsben/. This chapter is an invited extension of authors' previous publication (Oh & Lee, 2009).