CARVIEW

MOTORHOMES

Select Language

HTTP/2 200 server: GitHub.com content-type: text/html; charset=utf-8 last-modified: Tue, 28 Oct 2025 22:19:18 GMT access-control-allow-origin: * etag: W/"69014166-b3ef" expires: Sun, 28 Dec 2025 23:28:32 GMT cache-control: max-age=600 content-encoding: gzip x-proxy-cache: MISS x-github-request-id: F769:1387E:805339:9030B6:6951BAC8 accept-ranges: bytes age: 0 date: Sun, 28 Dec 2025 23:18:32 GMT via: 1.1 varnish x-served-by: cache-bom-vanm7210037-BOM x-cache: MISS x-cache-hits: 0 x-timer: S1766963912.458576,VS0,VE222 vary: Accept-Encoding x-fastly-request-id: 1d47ade6e2f424beedb3a422e274b97709eee264 content-length: 9954 Aljosa's Web Corner

Aljosa Osep, Ph.D.

Hi, I'm Aljosa! I am a Senior Research Scientist at NVIDIA, working on learning to understand the dynamic world from raw, unlabeled streams of sensory data.

I come from the Alpine side of Slovenia. I earned my Ph.D. from RWTH Aachen University under the supervision of Prof. Bastian Leibe. I was a postdoctoral fellow at the Technical University of Munich and the Robotics Institute, Carnegie Mellon University.

Research Statement (2019) Twitter Scholar

Research

My research focuses on enabling AI systems to robustly understand the dynamic, 3D world from raw sensor streams, such as video and LiDAR. Key areas include learning directly from raw data, tracking and segmenting objects, understanding complex spatiotemporal scenes, and predicting future events in open-world environments. Hover over each topic below to explore related publications.

Learning From Raw Sensory Data

4D Scene Understanding

Tracking and Panoptic Perception

Forecasting & Behavioral AI

News

October 2025: Having a talk about recent research at the legendary GRASP lab at UPenn on segmenting what we cannot (directly) see 👻: link.
June 2024: Having a talk at the CVPR Area Chair meeting, titled Learning To Understand The World From Video.
June 2024: I joined NVIDIA as a Senior Research Scientist!
March 2024: Our paper on Learning to segment anything in Lidar (SAL) was featured at GTC2024! Check out the NVIDIA AI Tools for Autonomous Vehicle Developers.
September 2022: Two papers accepted to NeurIPS 2022! Excited to be back to NOLA soon!
June 2022: I was featured in the TWIMLAI podcast! Listen to the episode.
August 2021: I am one of the three persons listed as outstanding reviewers for all top-tier computer vision conferences in 2020/21. See the informal analysis by Simon Niklaus! Thanks to ACs for the recognition and Simon for pointing this out.
August 2021: I was awarded Borchers Plaquette at RWTH Aachen University for outstanding doctoral dissertation!
June 2021: I am spending a year at the Carnegie Mellon University (The Robotics Institute, CMU Argo AI Center for Autonomous Vehicle Research) in Pittsburgh! Thanks to Deva Ramanan for hosting me!
April 2020: Learned how to make pancakes! Check out the evidence.

Students Supervised

Ayça Takmaz (NVIDIA, intern) ETH Zurich
Yushan Zhang (NVIDIA, intern) Linköping University
Neehar Peri (NVIDIA, intern) Carnegie Mellon University
Xindi Wu (CMU, 2022) → Princeton
Vladimir Fomenko (TUM, 2022) → Microsoft, OpenAI
Anirudh S Chakravarthy (CMU, 2022) → Cruise AI
Meghana Reddy Ganesina (CMU, 2022) → Zoox
Abhinav Agarwalla (CMU, 2022) → Prior Argo AI, now Neural Magic
Vladimir Yugay (TUM, 2022) → University of Amsterdam
Alexandr Kim (TUM, 2019-2021) → Meta
Yang Liu (TUM, 2020) → Huawei
Manuel Kolmet (TUM, 2021) → GLASS Imaging
Anselm Coogan (TUM, 2021) → Scandit
Maximilian Kempa (TUM, 2020) → Bosch
Mehmet Aygün (TUM, 2020) → University of Edinburgh
Johannes Gross (RWTH Aachen, 2019)
Deyvid Kochanov (RWTH Aachen, 2016)
Dirk Klostermann (RWTH Aachen, 2015) → BMW

Talks

June 2024: CVPR 2024 Area Chair Panel, invited talk: Learning To Understand The World From Video, Slides
June 2023: CVPR 2023, Visual Perception via Learning in an Open World, invited talk: Learning To Understand The World From Video, Slides
June 2023: University of Ljubljana, invited talk: Learning To Understand The World From Video, Slides
February 2023: University of Ljubljana, invited talk: Learning To Understand The World From Video, Slides
October 2022: ECCV’22 Workshop on 3D Perception in Autonomous Driving Details, Slides
October 2022: ECCV’22 Workshop on Cross-Modal Human-Robot Interaction Details, Slides
June 2022: Slovenian data-science meetup talk (in Slovenian): Slides
April 2022: UT Austin AI colloquium, Unifying Segmentation, Tracking, and Forecasting, Slides
September 2021: ICCV’21 Workshop on 3D Object Detection from Images, 4D Panoptic LiDAR Segmentation, Slides
July 2021: RSS 2021 on Workshop on Behavioral Inference of Remotely Sensed Multi-agent Systems, invited talk, Tracking Every Object and Pixel, Slides
July 2021: RSS 2021 Workshop on Perception and Control for Autonomous Navigation in Crowded, Dynamic Environments, invited talk, Tracking Every Object and Pixel, Slides, Talk
June 2021: CVPR'21 JackRobbot dataset and benchmark (JRDB) workshop talk, Tracking Every Object and Pixel, Slides
April 2021: Cornell Robotics Seminar, Slides
September 2020: University of Bonn - Research talk, Slides
June 2019: RWTH Aachen University - Thesis Defense, Slides
June 2019: Georgia Tech - Research Talk, Slides
March 2019: Carnegie Mellon University VASC Seminar, Slides

Teaching

Summer 2022: Lecturer for IN2375: Computer Vision III: Detection, Segmentation and Tracking (CV3DST) (TU Munich)
Summer 2022: Lecturer for IN2346: Introduction to Deep Learning (I2DL) (TU Munich)
Winter 2020/21: IN2157: Fundamental Algorithms (TU Munich, lecturer)
Summer 2019/20: IN2375: Computer Vision 3: Detection, Segmentation and Tracking (TU Munich, guest lecturer), Lectures available on YouTube
Winter 2019/20: IN2157: Fundamental Algorithms (TU Munich, lecturer), TUM Moodle, IN2375: Computer Vision 3: Detection, Segmentation and Tracking (TU Munich, guest lecturer)
Summer 2016/17: Machine Learning (RWTH Aachen, exercise class)
Winter 2014/15: Computer Vision (RWTH Aachen, exercise class)
Winter 2013/14: Machine Learning (RWTH Aachen, exercise class)

Service

Area Chair (AC) for ICLR, CVPR, ECCV, ICCV, WACV, ACCV.
I am the main organizer of 6th BMTT MOTChallenge Workshop: Segmenting and Tracking Every Point and Pixel at ICCV'21 workshop, and co-organizer of: 7th Workshop on Benchmarking Multi-Target Tracking: How Far Can Synthetic Data Take us? at CVPR'22, Tracking and its many guises Workshop at ECCV'2020, Multi-Object Tracking and Segmentation Workshop at CVPR'2020.
Reviewer for (machine learning, vision conferences) CVPR, ECCV, ICCV, BMVC, NeurIPS, ICML, ICLR; (robotics conferences) ICRA, IROS, RSS; (journals) IJCV, RAL, TPAMI.
I am in RSS Pioneers 2021 program committee and on the ECCV'24 organization team!

Publications

G. Brasó, A. Ošep, L. Leal-Leal-Taixé: Native Segmentation Vision Transformers, Neural Information Processing Systems (NeurIPS), 2025.
paper page

A. Takmaz, C. Saltori, N. Peri, T. Meinhardt, R. de Lutio, L. Leal-Taixé, A. Ošep: Towards Learning to Complete Anything in Lidar, International Conference on Machine Learning (ICML), 2025.
paper page

Y. Zhang, A. Ošep, L. Leal-Taixé, T. Meinhardt: Zero-Shot 4D Lidar Panoptic Segmentation, Conference on Computer Vision and Pattern Recognition (CVPR), 2025.
paper page

A. Ošep, T. Meinhardt, F. Ferroni, N. Peri, D. Ramanan, L. Leal-Taixe: Better Call SAL: Towards Learning to Segment Anything in Lidar, European Conference on Computer Vision (ECCV), 2024.
paper video page

A. Chakravarthy, M. Ganesina, P. Hu, L. Leal-Taixe, S. Kong, D. Ramanan, A. Ošep: Lidar Panoptic Segmentation in an Open World, International Journal of Computer Vision (IJCV), 2024.
paper code

J. Seidenschwarz, A. Ošep, F. Ferroni, S. Lucey, L. Leal-Taixe: What Moves Together Belongs Together, Conference on Computer Vision and Pattern Recognition (CVPR), 2024.
paper page code

C. Saltori, A. Ošep, E. Ricci, L. Leal-Taixé: Walking Your LiDOG: A Journey Through Multiple Domains for LiDAR Semantic Segmentation, International Conference on Computer Vision (ICCV), 2023.
paper video code

A. Agarwalla, X. Huang, J. Ziglar, F. Ferroni, L. Leal-Taixé, J. Hays, A. Ošep, D. Ramanan: Lidar Panoptic Segmentation and Tracking without Bells and Whistles, International Conference on Intelligent Robots and Systems (IROS), 2023.
paper page

X. Wu, K. Lau, F. Ferroni, A. Ošep, D. Ramanan: Pix2Map: Cross-modal Retrieval for Inferring Street Maps from Images, Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
paper video poster page

V. Fomenko, I. Elezi, D. Ramanan, L. Leal-Taixé, A. Ošep: Learning to Discover and Detect Objects, Neural Information Processing Systems (NeurIPS), 2022.
paper video poster page code

P. Dendorfer, V. Yugay, A. Ošep, L. Leal-Taixé: Quo Vadis: Is Trajectory Forecasting the Key Towards Long-Term Multi-Object Tracking?, Neural Information Processing Systems (NeurIPS), 2022.
paper video code

A. Kim, G. Brasó, A. Ošep, L. Leal-Taixé: PolarMOT: How far can geometric relations take us in 3D multi-object tracking?, European Conference on Computer Vision (ECCV), 2022.
paper video poster code

Q. Zhou, S. Agostinho, A. Ošep, L. Leal-Taixé: Is Geometry Enough for Matching in Visual Localization?, European Conference on Computer Vision (ECCV), 2022.
paper code

L. Nunes, X. Chen, R. Marcuzzi, A. Ošep, L. Leal-Taixé, C. Stachniss, J. Behley: Unsupervised Class-Agnostic Instance Segmentation of 3D LiDAR Data for Autonomous Vehicles, IEEE Robotics and Automation Letters (RA-L), 2022.
paper code

M. Gladkova, N. Korobov, N. Demmel, A. Ošep, L. Leal-Taixé, D. Cremers: DirectTracker: 3D Multi-Object Tracking Using Direct Image Alignment and Photometric Bundle Adjustment, International Conference on Intelligent Robots and Systems (IROS), 2022.
paper video page

N. Peri, J. Luiten, M. Li, A. Ošep, L. Leal-Taixé, D. Ramanan: Forecasting from LiDAR via Future Object Detection, Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
paper code

M. Kolmet, Q. Zhou, A. Ošep, L. Leal-Taixé: Text2Pos: Text-to-point-cloud cross-modal localization, Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
paper code

Y. Liu, I. Zulfikar, J. Luiten, A. Dave, D. Ramanan, B. Leibe, A. Ošep, L. Leal-Taixé: Opening up Open-World Tracking, Conference on Computer Vision and Pattern Recognition (CVPR), oral, 2022.
paper code

S. Agostinho, A. Ošep, A. Del Bue, L. Leal-Taixé: (Just) A Spoonful of Refinements Helps the Registration Error Go Down, International Conference on Computer Vision (ICCV) (oral), 2021.
paper code

M. Fabbri, G. Brasó, G. Maugeri, A. Ošep, R. Gasparini, O. Cetintas, S. Calderara, L. Leal-Taixé, R. Cucchiara: MOTSynth: How Can Synthetic Data Help Pedestrian Detection and Tracking?, International Conference on Computer Vision (ICCV), 2021.
paper video

M. Aygün, A. Ošep, M. Weber, M. Maximov, C. Stachniss, J. Behley, L. Leal-Taixé: 4D Panoptic LiDAR Segmentation, Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
paper video poster code

A. Kim, A. Ošep, L. Leal-Taixé: EagerMOT: 3D Multi-Object Tracking via Sensor Fusion, IEEE International Conference on Robotics and Automation (ICRA), 2021.
paper video code

M. Weber, J. Xie, M. Collins, Y. Zhu, P. Voigtlaender, H. Adam, B. Green, A. Geiger, B. Leibe, D. Cremers, A. Os̆ep, L. Leal-Taixé, L. Chen: STEP: Segmenting and Tracking Every Pixel, NeurIPS Benchmarks and Datasets, 2022.
paper code

P. Dendorfer, A. Ošep, L. Leal-Taixé: Goal-GAN: Multimodal Trajectory Prediction Based on Goal Position Estimation, Asian Conference on Computer Vision (ACCV), 2020.
paper video page code

P. Dendorfer, A. Ošep, A. Milan, K. Schindler, D. Cremers, I. Reid, S. Leal-Taixé: MOTChallenge: A Benchmark for Single-camera Multiple Target Tracking, International Journal of Computer Vision (IJCV), 2020.
paper

J. Luiten, A. Ošep, P. Dendorfer, P. Torr, A. Geiger, L. Leal-Taixé, B. Leibe: HOTA: A Higher Order Metric for Evaluating Multi-Object Tracking, International Journal of Computer Vision (IJCV), 2020.
paper code blog

S. Mahadevan*, A. Athar*, A. Ošep, S. Hennen, L. Leal-Taixé, B. Leibe: Making a Case for 3D Convolutions for Object Segmentation in Videos, British Machine Vision Conference (BMVC), 2020.
paper video code

A. Athar*, S. Mahadevan*, A. Ošep, L. Leal-Taixé, B. Leibe: STEm-Seg: Spatio-temporal Embeddings for Instance Segmentation in Videos, European Conference on Computer Vision (ECCV), 2020.
paper video code

Y. Xu, A. Ošep, Y. Ban, R. Horaud, L. Leal-Taixé, X. Alameda-Pineda: How To Train Your Deep Multi-Object Tracker, Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
paper video code

J. Gross, A. Ošep, B. Leibe: AlignNet-3D for Fast Point Cloud Registration of Partially Observed Objects, International Conference on 3D Vision (3DV), 2019.
paper video poster code

P. Voigtlaender, M. Krause, A. Ošep, J. Luiten, B. Sekar, A. Geiger, B. Leibe: {MOTS}: Multi-Object Tracking and Segmentation, Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
paper video code

A. Ošep, P. Voigtlaender, M. Weber, J. Luiten, B. Leibe: 4D Generic Video Object Proposals, International Conference on Robotics and Automation (ICRA), 2020.
paper video code teaser

A. Ošep, P. Voigtlaender, J. Luiten, S. Breuers, B. Leibe: Large-Scale Object Mining for Object Discovery from Unlabeled Video, International Conference on Robotics and Automation (ICRA), 2019.
paper video

A. Ošep, W. Mehner, P. Voigtlaender, B. Leibe: Track, then Decide: Category-Agnostic Vision-based Multi-Object Tracking, International Conference on Robotics and Automation (ICRA), 2018.
paper video code

A. Ošep, P. Voigtlaender, J. Luiten, S. Breuers, B. Leibe: Towards Large-Scale Video Object Mining, ECCV 2018 Workshop on Interactive and Adaptive Learning in an Open World, 2018.
paper

A. Ošep, W. Mehner, M. Mathias, B. Leibe: Combined Image- and World-Space Tracking in Traffic Scenes, International Conference on Robotics and Automation (ICRA), 2017.
paper video code teaser

D. Klostermann, A. Ošep, J. Stueckler, B. Leibe: Unsupervised Learning of Shape-Motion Patterns for Objects in Urban Street Scenes, British Machine Vision Conference (BMVC), 2016.
paper video

D. Kochanov, A. Ošep, J. Stueckler, B. Leibe: Scene Flow Propagation for Semantic Mapping and Object Discovery in Dynamic Street Scenes, International Conference on Intelligent Robots and Systems (IROS), 2016.
paper video

A. Ošep, A. Hermans, F. Engelmann, D. Klostermann, M. Mathias, B. Leibe: Multi-Scale Object Candidates for Generic Object Tracking in Street Scenes, International Conference on Robotics and Automation (ICRA), 2016.
paper

D. Mitzel, J. Diesel, A. Ošep, U. Rafi, B. Leibe: A Fixed-Dimensional 3D Shape Representation for Matching Partially Observed Objects in Street Scenes, International Conference on Robotics and Automation (ICRA), 2015.
paper

M. Weinmann, A. Ošep, R. Ruiters, R. Klein: Multi-View Normal Field Integration for 3D Reconstruction of Mirroring Objects, International Conference on Computer Vision (ICCV), 2013.
paper

M. Weinmann, R. Ruiters, A. Ošep, C. Schwartz, R. Klein: Fusing Structured Light Consistency and Helmholtz Normals for 3D Reconstruction, British Machine Vision Conference (BMVC), 2012.
paper

Original Source | Taken Source

Aljosa Osep, Ph.D.

Research

Relevant Papers: Learning From Raw Sensory Data

Relevant Papers: 4D Scene Understanding

Relevant Papers: Tracking and Panoptic Perception

Relevant Papers: Forecasting & Behavioral AI