HTTP/2 301 server: GitHub.com content-type: text/html location: https://kianaehsani.com/forcecvpr2020 x-github-request-id: B05F:2685F2:83A823:9426A5:69520EFE accept-ranges: bytes age: 0 date: Mon, 29 Dec 2025 05:17:50 GMT via: 1.1 varnish x-served-by: cache-bom-vanm7210064-BOM x-cache: MISS x-cache-hits: 0 x-timer: S1766985470.390268,VS0,VE199 vary: Accept-Encoding x-fastly-request-id: 8ae272a0d51655b4fa22797afae074d5991d1176 content-length: 162 HTTP/2 301 server: GitHub.com content-type: text/html location: https://kianaehsani.com/forcecvpr2020/ x-github-request-id: D524:3A7A40:846AFB:94EA8D:69520EFE accept-ranges: bytes date: Mon, 29 Dec 2025 05:17:50 GMT via: 1.1 varnish age: 0 x-served-by: cache-bom-vanm7210071-BOM x-cache: MISS x-cache-hits: 0 x-timer: S1766985471.739473,VS0,VE196 vary: Accept-Encoding x-fastly-request-id: 3084be7901653dead019fd063d264d83ab0fdf1a content-length: 162 HTTP/2 200 server: GitHub.com content-type: text/html; charset=utf-8 last-modified: Mon, 15 Jun 2020 23:16:41 GMT access-control-allow-origin: * etag: W/"5ee80159-127d" expires: Mon, 29 Dec 2025 05:27:51 GMT cache-control: max-age=600 content-encoding: gzip x-proxy-cache: MISS x-github-request-id: B267:2F7ECD:8517A7:9595AB:69520EFE accept-ranges: bytes age: 0 date: Mon, 29 Dec 2025 05:17:51 GMT via: 1.1 varnish x-served-by: cache-bom-vanm7210071-BOM x-cache: MISS x-cache-hits: 0 x-timer: S1766985471.954485,VS0,VE224 vary: Accept-Encoding x-fastly-request-id: 603c4baadeb9a1db9907d9078044240e51246694 content-length: 1742

Use the Force Luke!
Learning to Predict Physical Forces by Simulating Effects

K Ehsani, S Tulsiani, S Gupta, A Farhadi, A Gupta (CVPR'20)

[Arxiv PDF] [Code and Data] [Video] [Slides]

---

When we humans look at a video of human-object interaction, we can not only infer what is happening but we can even extract actionable information and imitate those interactions. On the other hand, current recognition or geometric approaches lack the physicality of action representation. In this paper, we take a step towards more physical understanding of actions. We address the problem of inferring contact points and the physical forces from videos of humans interacting with objects. One of the main challenges in tackling this problem is obtaining ground-truth labels for forces. We sidestep this problem by instead using a physics simulator for supervision. Specifically, we use a simulator to predict effects, and enforce that estimated forces must lead to same effect as depicted in the video.

Our quantitative and qualitative results show that:

1. We can predict meaningful forces from videos whose effects lead to accurate imitation of the motions observe.
2. By jointly optimizing for contact point and force prediction, we can improve the performance on both tasks in comparison to independent training.
3. We can learn a representation from this model that generalizes to novel objects using few shot examples.

Oral Presentation

Qualitative Video