Aligning Text, Images, and 3D Structure Token-by-Token
We present a unified LLM that aligns language, images, and structured 3D scenes, demonstrating applications in rendering, recognition, instruction following, and 3D QA.
| CARVIEW |
Select Language
HTTP/2 200
server: GitHub.com
content-type: text/html; charset=utf-8
last-modified: Tue, 07 Oct 2025 06:35:04 GMT
access-control-allow-origin: *
strict-transport-security: max-age=31556952
etag: W/"68e4b498-bd7d"
expires: Mon, 29 Dec 2025 10:41:12 GMT
cache-control: max-age=600
content-encoding: gzip
x-proxy-cache: MISS
x-github-request-id: 5FEA:2916CC:8B822B:9C8EF9:69525870
accept-ranges: bytes
age: 0
date: Mon, 29 Dec 2025 10:31:12 GMT
via: 1.1 varnish
x-served-by: cache-bom-vanm7210027-BOM
x-cache: MISS
x-cache-hits: 0
x-timer: S1767004273.634047,VS0,VE210
vary: Accept-Encoding
x-fastly-request-id: c905533292c9a702cb7a2439d4e04b22188265ca
content-length: 12757
Aadarsh Sahoo — Caltech Ph.D. Student
Skip to content
Aadarsh Sahoo
Aadarsh Sahoo
Ph.D. student at Caltech, advised by Prof. Georgia Gkioxari and Prof. Pietro Perona.
About
I am a Ph.D. student at Caltech advised by Prof. Georgia Gkioxari and Prof. Pietro Perona.
Previously, I was a student researcher at the MIT–IBM Watson AI Lab, pretraining large language models and doing research on vision–language models with Dr. Rameswar Panda, Dr. Rogerio Feris, and Prof. Yoon Kim. I completed a dual degree (B.Tech + M.Tech) at IIT Kharagpur, where I worked in the Computer Vision and Intelligence Research Lab under Prof. Abir Das.
In Summer 2021, I worked with Prof. Kate Saenko (Boston University) and Prof. Trevor Darrell (UC Berkeley) as a research intern for the DARPA LwLL project.
Research
My research focuses on building multimodal foundation models that perceive, reason, and interact in 3D environments. I am particularly interested in developing unified representations that bridge geometry, vision, and language-guided systems that can understand spatial relationships, engage in grounded conversations, and perform complex reasoning tasks. My work spans 3D tokenization for sequential modeling, conversational visual understanding, and parameter-efficient adaptation of large models. Ultimately, I aim to advance embodied AI systems that can robustly interpret and act within real-world environments to benefit society.
News
-
Preprint “Aligning Text, Images, and 3D Structure Token-by-Token (Kyvo)” released on arXiv (project page).
-
“XPL: A Cross-Model framework for Semi-Supervised Prompt Learning in VLMs” accepted by TMLR.
-
Attended ICVSS 2024 in Sicily.
-
Recognized as an Outstanding Reviewer for ECCV 2024.
-
Named an Outstanding Reviewer for CVPR 2024.
-
Started my PhD at Caltech.
-
I have been selected as a Kortschak Scholar at Caltech!
-
Paper on Anytime Domain Adaptation accepted at ICLR 2023.
-
Select, Label, and Mix (SLM) received the Best Paper Honorable Mention at WACV 2023.
-
Paper on Partial Domain Adaptation accepted at WACV 2023.
-
Started as a student researcher at the MIT–IBM Watson AI Lab in Cambridge, MA.
-
Extended Abstract on Partial Domain Adaptation accepted at the NeurIPS DistShift Workshop 2021.
-
Paper on Domain Adaptation in Action Recognition accepted at NeurIPS 2021.
-
Volunteer at the CVPR 2021 workshop Dynamic Neural Networks Meets Computer Vision (DNetCV).
-
Started internship at UC Berkeley and Boston University.
-
Paper accepted at the ECCV 2020 Workshop on Imbalance Problems in Computer Vision (IPCV).
-
Joined CVIR, IIT Kharagpur as an undergraduate researcher in Computer Vision.
-
Selected for Computer Science and Engineering major (<1% acceptance).
-
Admitted to IIT Kharagpur via JEE Advanced 2017.
Publications
XPL: A Cross-Model framework for Semi-Supervised Prompt Learning in Vision–Language Models
Omprakash Chakraborty, Aadarsh Sahoo, Rameswar Panda, Abir Das — TMLR, 2024
A semi-supervised prompt learning framework leveraging unlabeled data to improve VLM adaptation via cross-model consistency.
AnyDA: Anytime Domain Adaptation
Omprakash Chakraborty, Aadarsh Sahoo, Rameswar Panda, Abir Das — ICLR, 2023
We propose a domain-alignment approach with switchable depth, width, and input resolution to realize accuracy–efficiency trade-offs under different constraints.
Contrast and Mix: Temporal Contrastive Video Domain Adaptation with Background Mixing
Aadarsh Sahoo, Rutav Shah, Rameswar Panda, Kate Saenko, Abir Das — NeurIPS, 2021
We introduce a temporal contrastive learning approach for unsupervised video domain adaptation by leveraging video speed, background mixing, and target pseudo-labels.
Select, Label, and Mix: Learning Discriminative Invariant Feature Representations for Partial Domain Adaptation
Aadarsh Sahoo, Rameswar Panda, Rogerio Feris, Kate Saenko, Abir Das — NeurIPS-W 2021; WACV 2023 (Best Paper Honorable Mention)
We develop a ‘Select, Label, and Mix’ (SLM) framework that aims to learn discriminative invariant feature representations for partial domain adaptation.
Mitigating Dataset Imbalance via Joint Generation and Classification
Aadarsh Sahoo, Ankit Singh, Rameswar Panda, Rogerio Feris, Abir Das — ECCV-W, 2020 (Oral)
We introduce a dataset repair strategy combining a classifier with a GAN to augment minority-class examples.
Services
Contact
Email: aadarsh.sahoo.99@gmail.com
Profiles: Google Scholar · LinkedIn · GitHub · X/Twitter