| CARVIEW |
Select Language
HTTP/2 200
server: GitHub.com
content-type: text/html; charset=utf-8
last-modified: Fri, 27 Dec 2024 09:00:50 GMT
access-control-allow-origin: *
strict-transport-security: max-age=31556952
etag: W/"676e6cc2-515a"
expires: Mon, 29 Dec 2025 21:03:33 GMT
cache-control: max-age=600
content-encoding: gzip
x-proxy-cache: MISS
x-github-request-id: C616:2B0FD4:953CE0:A76107:6952EA4D
accept-ranges: bytes
age: 0
date: Mon, 29 Dec 2025 20:53:33 GMT
via: 1.1 varnish
x-served-by: cache-bom-vanm7210065-BOM
x-cache: MISS
x-cache-hits: 0
x-timer: S1767041614.510738,VS0,VE214
vary: Accept-Encoding
x-fastly-request-id: eb55b722da3dc9871a58edf4cca8adda4c09cfb7
content-length: 5183
MeTTA: Single-View to 3D Textured Mesh Reconstruction with Test-Time Adaptation
We propose a test-time adaptation pipeline to reconstruct a
3D mesh with PBR texture from a single-view image. “Ref. Image” refers to the reference
input image. “Seg. Image” refers to the object-segmented image from “Ref. Image”.
-->
MeTTA: Single-View to 3D Textured Mesh Reconstruction with Test-Time Adaptation
1Grad. School of AI, POSTECH,
2Dept. of Electrical Engineering, POSTECH,
3Bucketplace,
4Institute for Convergence Research and Education in Advanced Technology, Yonsei University
†denotes corresponding authors
†denotes corresponding authors
British Machine Vision Conference (BMVC) 2024
Best Poster Award
Best Poster Award
Abstract
Reconstructing 3D from a single view image is a long-standing challenge.
One of the popular approaches to tackle this problem is learning-based methods,
but dealing with the test cases unfamiliar with training data (Out-of-distribution; OoD)
introduces an additional challenge. To adapt for unseen samples in test time,
we propose MeTTA, a test-time adaptation (TTA) exploiting generative prior.
We design joint optimization of 3D geometry, appearance, and pose to handle OoD cases.
However, the alignment between the reference image and the 3D shape via the estimated viewpoint could be erroneous,
which leads to ambiguity. To address this ambiguity, we carefully design learnable virtual cameras and their self-calibration.
In our experiments, we demonstrate that MeTTA effectively deals with OoD scenarios
at failure cases of existing learning-based 3D reconstruction models and enables obtaining a realistic appearance
with physically based rendering (PBR) textures.
MeTTA
We propose a test-time adaptation pipeline to reconstruct a
3D mesh with PBR texture from a single-view image. “Ref. Image” refers to the reference
input image. “Seg. Image” refers to the object-segmented image from “Ref. Image”.
Cross-domain Results
In-domain Results (Pix3D)
PBR Editing
BibTeX
@inproceedings{yu2024metta,
title={MeTTA: Single-View to 3D Textured Mesh Reconstruction with Test-Time Adaptation},
author={Yu-Ji, Kim and Ha, Hyunwoo and Youwang, Kim and Surh, Jaeheung and Ha, Hyowon and Oh, Tae-Hyun},
booktitle={The British Machine Vision Conference (BMVC)},
year={2024}
}