Robo-ABC: Affordance Generalization Beyond Categories via Semantic Correspondence for Robot Manipulation
DenseMatcher is a method for generalization the “understanding” of one object to another. The best way to understand this, is to view the following images from the paper.
In other words, DenseMatcher computes 3D correspondences between objects.
It works by using SD-DINO (a combination of Stable Diffusion and DINO) to extract 2D features from different angles of the 3D asset. The features from different views are averaged, providing the feature for each vertex.
However, as seen in the image above, the features are noisy. Therefore, the features are then refined with DiffusionNet. This is an architecture meant for meshes.
After the features have been refined, a functional map is solved to compute correspondences.
Link to paper. All images in this post are from the paper.
Back to top