Expertise evaluation using PathFinder networks scaling in ranking of satellite images

In this article we propose a methodology to evaluate the level of expertise of image analysts when searching domain-specific images by semantics. We apply our methodology to ranking high-resolution satellite images by semantics. Our methodology applies PathFinder Network Scaling methods to create concept maps for representing associations of semantics to regions of a feature space for each image analyst. The relevance of each node in a concept map is evaluated using a hits authority algorithm. The expertise of each image analyst is then evaluated by comparing to ground truth models using the Kendall tau rank correlation coefficient. Our system allows us to identify areas of expert disagreement by evaluating the relative difference individual models place on features as well as recommend areas of that needs to be stressed by novice image analysts.