Critical assessment of Shape Retrieval Tools (SRTs)

In today’s design — manufacturing context, designers often modify existing 3D shapes (or design models) instead of creating a new design from scratch. This requires the ability to search an existing database of designs/3D models to identify and extract similar designs. Shape Retrieval Tools (SRTs) have been developed to provide an essential role in saving time and effort to retrieve and generate new designs. The capabilities of commercially available SRTs vary based on the form of the input design model, the search technique or algorithm used, the search/retrieval time, ease of use, and the quality of results. The focus of this paper is to study of their capabilities, performances, and differences and develop criteria to compare the effectiveness and performance of such Shape Retrieval Tools. Current search evaluation methods, such as precision and recall, are based on human interpretation of the results. This paper presents a holistic set of metrics for comparing the performance and effectiveness of SRTs, including data input options (to search), effectiveness of the search process, the associated retrieval time, overall ease of use, and additional data retrieval details. An algorithm is proposed to objectively analyze the search results based on the proposed Model Match Ratio (MMR), computed by the variance between the input and retrieved geometries. The search results are usually presented in a rank order list. A Precision Sequence Metric (PSM) is developed to evaluate the retrieved list by ranking the retrieved results based on the MMR for evaluating the quality of the search. The proposed evaluation algorithm was tested on several design models (and their subsequent retrieval results) involving three SRTs (Vizseek, Geolus, and CADENAS); the results of the comparison of the performance of these SRTs are discussed in this paper.