Unsupervised Evaluation of Multiple Node Ranks by Reconstructing Local Structures

A problem that frequently occurs when mining complex networks is selecting algorithms with which to rank the relevance of nodes to metadata groups characterized by a small number of examples. The best algorithms are often found through experiments on labeled networks or unsupervised structural community quality measures. However, new networks could exhibit characteristics different from the labeled ones, whereas structural community quality measures favor dense congregations of nodes but not metadata groups spanning a wide breadth of the network. To avoid these shortcomings, in this work we propose using unsupervised measures that assess node rank quality across multiple metadata groups through their ability to reconstruct the local structures of network nodes; these are retrieved from the network and not assumed. Three types of local structures are explored: linked nodes, nodes up to two hops away and nodes forming triangles. We compare the resulting measures alongside unsupervised structural community quality ones to the AUC and NDCG of supervised evaluation in one synthetic and four real-world labelled networks. Our experiments suggest that our proposed local structure measures are often more accurate for unsupervised pairwise comparison of ranking algorithms, especially when few example nodes are provided. Furthermore, the ability to reconstruct the extended neighborhood, which we call HopAUC, manages to select a near-best among many ranking algorithms in most networks.