Over the last decade, a great amount of effort and resources have been invested in the development of Semantic Web Service (SWS) frameworks. Numerous description languages, frameworks, tools, and matchmaking and composition algorithms have been proposed. Nevertheless, when faced with a real-world problem, it is still very hard to decide which of these different approaches to use.

In this book, the editors present an overall overview and comparison of the main current evaluation initiatives for SWS. The presentation is divided into four parts, each referring to one of the evaluation initiatives. Part I covers the long-established first two tracks of the Semantic Service Selection (S3) Contest - the OWL-S matchmaker evaluation and the SAWSDL matchmaker evaluation. Part II introduces the new S3 Jena Geography Dataset (JGD) cross evaluation contest. Part III presents the Semantic Web Service Challenge. Lastly, Part IV reports on the semantic aspects of the Web Service Challenge. The introduction to each part provides an overview of the evaluation initiative and overall results for its latest evaluation workshops. The following chapters in each part, written by the participants, detail their approaches, solutions and lessons learned.