Semantic Cross-Pose Correspondence from a Single Example

Abstract

This article focuses on predicting how an object can be transformed to a semantically meaningful pose relative to another object, given only one or few examples. Current pose correspondence methods rely on vast 3D object datasets and do not actively consider semantic information, which limits the objects to which they can be applied. We present a novel method for learning cross-object pose correspondence. The proposed method detects interacting object parts, performs one-shot part correspondence, and uses geometric and visual-semantic features. Given one example of two objects posed relative to each other, the model can learn how to transfer the demonstrated relations to unseen object instances.

Sicelukwanda Zwane

PhD (10/2020-09/2025)

Marc Deisenroth

Google DeepMind Chair of Machine Learning and Artificial Intelligence