iCGPN: Interaction-Centric Graph Parsing Network for Human-Object Interaction Detection

Abstract

Human-Object Interaction (HOI) detection aims to infer different interactions, which occur between humans and related objects of images. HOI is usually represented by a triplet and can be modeled as a graph. Thus, with global structural information of images, graph-based methods can detect interactions. However, in existing graph networks, although different fully-connected graphs are built, all detected bounding boxes are regarded as graph nodes equally or different types of nodes according to the category, thereby the dominant role of humans in HOI is ignored. In addition, object node representations mainly focus on appearance features, contributing little to HOI inference. To address these issues, a novel graph-based HOI detection model, named interaction-centric graph parsing network (iCGPN), models one human node as a central node, and other nodes as semantic nodes. Firstly, for each detected human instance, a human-centric fully-connected graph is constructed to learn related HOIs. Secondly, in order to reflect the difference between central nodes and semantic nodes, we design different feature representations and model different edge relationships. Through introducing the attention mechanism, global information related to human-object interaction is explored to enrich the semantic node representation, in which spatial layout, relative locations and object categories information are also combined. Finally, a multi-relation graph convolutional network is applied to update the node feature and infer the HOI. Furthermore, a multi-IOU random shift scheme is proposed to augment the data of the training set to fit the object detection deviation and enhance the generalization ability of our network. Extensive experimental results show that iCGPN achieves very competitive results in comparison with state-of-the-arraph-based methods on the V-COCO and HICO-DET datasets, which demonstrate the effectiveness of the proposed method.