| CARVIEW |
Select Language
HTTP/2 301
server: GitHub.com
content-type: text/html
location: https://arijitray.com/
x-github-request-id: 2EBE:21D6A4:87852E:982869:695221E5
accept-ranges: bytes
age: 0
date: Mon, 29 Dec 2025 06:38:30 GMT
via: 1.1 varnish
x-served-by: cache-bom-vanm7210084-BOM
x-cache: MISS
x-cache-hits: 0
x-timer: S1766990310.803099,VS0,VE205
vary: Accept-Encoding
x-fastly-request-id: 19c7558bb90efc8e8bf8e5f500b6a900055d5d0a
content-length: 162
HTTP/2 200
server: GitHub.com
content-type: text/html; charset=utf-8
last-modified: Fri, 12 Dec 2025 16:05:00 GMT
access-control-allow-origin: *
etag: W/"693c3d2c-ea45"
expires: Mon, 29 Dec 2025 06:48:30 GMT
cache-control: max-age=600
content-encoding: gzip
x-proxy-cache: MISS
x-github-request-id: DA98:36A0B4:8682AC:9726E7:695221E2
accept-ranges: bytes
age: 0
date: Mon, 29 Dec 2025 06:38:30 GMT
via: 1.1 varnish
x-served-by: cache-bom-vanm7210098-BOM
x-cache: MISS
x-cache-hits: 0
x-timer: S1766990310.174694,VS0,VE211
vary: Accept-Encoding
x-fastly-request-id: 420b4c3172db2c1e8d7c1dd30f66d741ab3c731c
content-length: 12462
Arijit Ray's Webpage
Google ScholarI wish to teach machines to help humans achieve more.
About me:
I am a Computer Vision Ph.D. Student, advised by Kate Saenko and Bryan Plummer at Boston University, and by Ranjay Krishna at the University of Washington. I received my M.S from Virginia Tech, advised by Devi Parikh.
I am interested in advancing visual and textual reasoning in AI models. To address the difficulty in obtaining detailed reasoning annotations, my work uses simulations to teach causal interactions and reinforcement learning to discover reasoning processes that generalize to the real-world.
If you are interested to collaborate or just chat about research on multimodal models and world models, say hi!
Selected Highlights:
- 2025: Excited to spend my summer at Google, working with the Pixel team and Leonidas Guibas from Deepmind.
- 2024: Our paper on feedback-guided autonomous driving got accepted to CVPR 2024 as a highlight.
- 2023: One of my student's paper on diverse reactions to multimodal content got accepted as an oral at an ICCV Workshop 2023.
- 2022: Excited to be spending my summer as a Research Scientist Intern at Meta (Facebook) AI (FAIR), working on the compositionality of large vision-language models.
- 2022: I started a PhD; We started [AI+X] of BU and Harvard, where we brainstorm research/venture ideas on how AI can impact different verticals.
- 2019: I won runners-up at the SRI CVT Shark Tank Competition that supported my mini-project on understanding image-text content to reduce radicalization of opinions on social media.
- 2017: The weed vs plant detection system I helped develop for precision fertilizing played a key part in the acquisition of Blue River Technologies by John Deere for 305 million USD.
- 2014: Our UAV to help locate natural disaster victims was featured in national news: Deccan Chronicle, Indian Express
- 2012: I won an Academic Merit Scholarship from SRM University that waives a part of my undergraduate tuition.
Selected publications (all):
2025
Arijit Ray,
Ahmed Abdelkader,
Chengzhi Mao,
Bryan A. Plummer,
Kate Saenko,
Ranjay Krishna,
Leonidas Guibas,
Wen-Sheng Chu
Mull-Tokens: Modality-Agnostic Latent Thinking
arXiv 2025
TL;DR: Instead of text reasoning or explicit image thoughts, using modality-agnostic thinking tokens pretrained using multimodal thoughts is more effective for visual reasoning tasks.
SIMS-V: Simulated Instruction-Tuning for Spatial Video Understanding
arXiv 2025
TL;DR: Using simulators to create spatially-rich video training data for multimodal language models.
Our 7B-parameter model fine-tuned on just 25K simulated examples outperforms the larger 72B baseline and achieves competitive performance with proprietary models on real-world spatial reasoning benchmarks.
Abhay Deshpande,
Yuquan Deng,
Arijit Ray,
Jordi Salvador,
Winson Han,
Jiafei Duan,
Kuo-Hao Zeng,
Yuke Zhu,
Ranjay Krishna,
Rose Hendrix
GraspMolmo: Generalizable Task-Oriented Grasping via Large-Scale Synthetic Data Generation
TL;DR: Use textual reasoning (e.g., one should grasp "handle" of cup to drink tea) and visual reasoning
(e.g., in this image, the cup is grasped by the "handle")
along with an object grasping dataset (data of 6 DOF arm positions to grasp objects) to
generate high-level task description (e.g., how should I grasp cup to drink tea?) to precise 6DOF grasp data.
Arijit Ray,
Ellis Brown,
Jiafei Duan,
Reuben Tan,
Dina Bashkirova,
Rose Hendrix,
Kiana Ehsani,
Aniruddha Kembhavi,
Bryan A. Plummer,
Ranjay Krishna*,
Kuo-Hao Zeng*,
Kate Saenko*
SAT: Dynamic Spatial Aptitude Training for Multimodal Language Models
TL;DR: Simulated spatial aptitude data (SAT) using 3D physics engines involving object and camera movements can improve spatial reasoning in real images and videos for MLMs
while maintaining pretraining commonsense. When instruction-tuned on SAT, LLaVA-13B outperforms some larger MLMs like GPT4-V and Gemini-1.5-pro in spatial reasoning.
2024
Arijit Ray,
Dina Bashkirova,
Reuben Tan,
Kuo-Hao Zeng,
Bryan A. Plummer,
Ranjay Krishna,
Kate Saenko
R2D3: Imparting Spatial Reasoning by Reconstructing 3D Scenes from 2D Images
Tech report 2024
TL;DR: A visual anchor with the corresponding 3D location in text helps multimodal language models perform more accurate 3D estimation.
FED: Feedback-Guided Autonomous Driving
TL;DR: MLMs can benefit autonomous driving by understanding natural language feedback and refining the next waypoint prediction.
2023
Cola: A Benchmark for Compositional Text-to-image Retrieval
TL;DR: Multimodal layers are the most responsible for compositional reasoning rather than the visual or textual encoders. Tuning multimodal layers over frozen representations
are more effective than tuning similar amount of parameters in the individual encoders or even the entire model.
Dina Bashkirova,
Arijit Ray,
Rupayan Mallick,
Sarah Adel Bargal,
Jianming Zhang,
Ranjay Krishna,
Kate Saenko
Lasagna: Layered Score Distillation for Disentangled Object Relighting
TL;DR: Synthetically generated examples using 3D graphics engines are effective in teaching physics-aware edits like relighting if we use score-distillation to avoid overfitting.
Socratis: Are Large Multimodal Models Emotionally Aware?
ICCV Workshops 2023 (oral), Workshop on Emotionally and Culturally Intelligent AI
TL;DR: A preliminary benchmark to test MLMs on why different people may feel different emotions for diverse reasons while viewing the same image-text content.
2021
User-targeted content generation using multimodal embeddings
Knowing What VQA Does Not: Pointing to Error-Inducing Regions to Improve Explanation Helpfulness
2019
Sunny and Dark Outside?! Improving Answer Consistency in VQA through Entailed Question Generation
EMNLP 2019, also at CVPR-W 2019 VQA and Visual Dialog Workshop
TL;DR: Use a learned teacher module to reward VQA models for predicting consistent answers to entailed questions.
2016
Question Relevance in VQA: Identifying Non-Visual And False-Premise Questions
TL;DR: Image captions are a surprisingly strong baseline to identify whether a question is relevant to an image or not.
Press Coverage
- 2023: Generative AI Podcast: I was interviewed on AI and analyzing social media responses using language models.
- 2019: TechXplore, Phys.org: An image-guessing game to evaluate the helpfulness of machine explanations
- 2014: Deccan Chronicle, Indian Express, Engineering.Careers360: UAV with Facial Recognition Capabilities for helping locating natural disaster victims, Click here
Hobbies
When I am not training LLMs, I love going to techno (a subgenre of electronic music) fests, making latte art, and engineering simple gadgets. In middle school, I opened an informal research society to encourage fellow students to take an interest in science by constructing simple gadgets. We won multiple accolades in school and city-level exhibitions.
Miscellanea
- Paper writing tips and tricks: Writing a good paper (by Jitendra Mailk), shortening papers (by Devi Parikh), Writing Introductions (by Kate Saenko)
- Paul Graham's essays: Some of my favorites: Crazy New Ideas, The Bus Ticket Theory of Genius, How to Think for Yourself, Cities and Ambition
- On social media: Is it bad or beneficial to society, and why running a social media is hard: Twitter thread by Yishan (former CEO of Reddit).
- Quotes
Contact
Best way to reach me would be to drop an email to array at bu dot edu.