| CARVIEW |
Events
- A Featured Session Talk at SIGGRAPH ASIA [Dec. 2024]
- Keynote at WACV2024 FRCSyn Workshop [Jan. 2024]
- Presenting AI-Mediated 3D Video Conferencing at SIGGRAPH2023 [Aug. 2023]
- Tutorial Keynote at MIRU 2023 in Japan [Jul. 2023]
- Keynote at CVPR Workshop on Media Forensics [Jun. 2023]
- Our digital avatar synthesis demo won Best-in-show award at SIGGRAPH RTL 2021 [Aug. 2021]
- Invited to demo “DEEPFAKED” at World Economic Forum in Davos [Jan. 2020]
- Gave a TEDx talk at TEDxCharlottesville [Nov. 2019]
Hey, welcome to my homepage! My name is Koki Nagano (in Kanji, my first name means “Lighting”, and “Hope”). I work at the intersection of Computer Graphics, Vision and AI. Currently, I am a Principal Research Scientist at NVIDIA. My research focuses on human understanding and synthesis, human interaction including agents and robots, and trustworthy visual computing including detection and prevention of visual misinformation in collaboration with DARPA. My PhD thesis focused on the topic of high-fidelity 3D human capture (USC, 2017), and I was supervised by Prof. Paul Debevec at USC ICT. Previously I was a Principal Scientist at Pinscreen Inc. During my PhD, I worked for Weta Digital on Hollywood blockbuster movies and for the Meta Reality Labs in Pittsburgh. I got my Bachelor of Engineering from Tokyo Institute of Technology with focus on Environmental Design.
Internship at NVIDIA
We hire research interns. If you are interested, please
send me an email.
Highlights
TED Talk
Face-ing Our Realistic Digital Self
TED /
TEDxCharlottesville
What You See Is What You GAN
Rendering Every Pixel for High-Fidelity Geometry in 3D GANs
Project
AI-Mediated 3D Video Conferencing
AI-Mediated 3D Video Conferencing
SIGGRAPH Demo,
Computex Keynote
Live 3D Portrait
Real-Time Radiance Fields for Single-Image Portrait View Synthesis
Project
Authenticating Avatars
Avatar Fingerprinting for Authorized Use of Synthetic Talking-Head Videos
Project
Live Neural Talking Avatar (SIGGRAPH Best-in-show)
I am AI: AI-driven Digital Avatar Made Easy
featured on
SIGGRAPH Blog,
fxguide
AI Virtual Assistant
AI-Synthesized Avatars: From Real-Time Deepfakes To Photoreal AI Virtual Assistant
featured on
Wired,
befores and afters
PaGAN Neural Face Renderer
New Real-time Hybrid AI + CGI Face Replacement
featured on
fxguide
AI Face Replacement
Advanced Face AI & Neural Rendering
featured on
fxguide
SIGGRAPH ASIA 2019
Deep Face Normalization
video
Live Virtual Try-on
Personalized Avatars for Realtime Virtual Try-on
featured on
fxguide.
Detecting Deepfakes
Protecting World Leaders Against Deep Fakes
Paper featured on
MIT Tech Review,
Al Jazeera,
The Verge.
Interactive 3D Hair Modeling
VR Hair Salon for Avatars
featured on
befores and afters
World Economic Forum AMNC 2019
The Human Element: Digital Mimicry
watch
demo
Netflix Follow This
watch episode on
Netflix
SIGGRAPH ASIA RTL 2018
Pinscreen Avatars in your Pocket: Mobile paGAN engine and Personalized Gaming
featured on
fxguide.
CBC The Fifth State
Deepfake video: The weaponization of fake news - The Fifth Estate
watch
episode
SIGGRAPH RTL 2018
Deep Learning-Based Photoreal Avatars for Online Virtual Worlds in iOS
featured on
MIT Tech Review,
CBS,
PBS,
fxguide.
SIGGRAPH 2015
Skin Microstructure Deformation With Displacement Map Convolution
featured on
CNET,
Fast Company,
CG Channel,
GIZMODO.
SIGGRAPH 2015
Computer Animation Festival
Skin Stretch: Simulating Dynamic Skin Microgeometry
featured on
GIZMODO,
The Verge,
MOTHERBOARD,
fxguide.
Life-size Holographics 3D Display
An Autostereoscopic Projector Array Optimized for 3D Facial Display
featured on
Wall Street Journal,
4gamer.
In the News
-
Deepfakes: A threat to democracy or just a bit of fun?, BBC, Jan 2020
-
Deepfake videos: Can you really believe what you see?, BBC, Jan 2020
-
Deepfakes, Bloomberg, Jan 2020
-
Fake videos and AI rattle nerves in Davos, Reuters, Jan 2020
-
GO THERE in Davos, CNN, Jan 2020
-
Deepfakes : la menace devient réelle, LCI, Jan 2020
-
DNN framework that removes perspective distortion and predicts frontal faces, Auror, Jan 2020
-
Pinscreen’s Real-Time DeepFake Demo, 80lv, Jan 2020
-
Virtual Manipulation ‘Deepfake’ videos can be hard to detect, Fox Business News, Jan 2020
-
Fake Trump video? How to spot deepfakes on Facebook and YouTube ahead of the presidential election, USA TODAY, Jan 2020
-
‘Perfectly real’ deepfakes will arrive in 6 months to a year, technology pioneer Hao Li says, CNBC, Jan 2020
-
How the pros make deepfakes I Deepfakes explained, DW, Dec 2019
-
Comment reconnaître un hypertrucage?, Radio Canada, Dec 2019
-
Fake Busters, NHK Closeup Gendai, Dec 2019
-
Prepare for the Deepfake Era of Web Video, WIRED, Oct 2019
-
Can you tell? Fake video “deep fake” threat and countermeasures Concerns spread to “real life”, FNN Prime, Aug 2019
-
The world’s top deepfake artist is wrestling with the monster he created, MIT Technology Review, Aug 2019
-
The rise of realistic fake videos and how it will affect our lives, i24NEWS, Aug 2019
-
Deepfake detection algorithms will never be enough, The Verge+, June 2019
-
How to spot manipulated video | The Fact Checker , The Washington Post, June 2019
-
Why it’s getting harder to spot a deepfake video, CNN, June 2019
-
Top AI researchers race to detect ‘deepfake’ videos: ‘We are outgunned’, The Washington Post, June 2019
-
Fake videos pose a challenge for lawmakers, Aljazeera, June 2019
-
A new deepfake detection tool should keep world leaders safe—for now, MIT Technology Review, June 2019
-
A New Method Of Spotting Deepfake Videos Looks For the Subtle Movements We Don’t Realize We Make, GIZMODO, June 2019
-
Deepfake debunking tool may protect presidential candidates. For now. Sometimes, CNET, June 2019
-
Why ‘deepfake’ videos are becoming more difficult to detect, PBS, June 2019
-
Manipulierte Videos-Wie Deep Fakes die Welt gefährden, ZDF, May 2019
-
Deep fake: peut-on encore faire confiance aux images ?, CANAL+, Mar 2019
-
Enquête | Deepfake : le vrai du faux, Radio-Canada, Jan 2019
-
The Deepfake | The War Over Truth | The Lie Detectors, The Fifth Estate, CBC, Nov 2018
-
US midterm election and the danger of fake news, NTV (Nippon TV), Nov 2018
-
All it takes is a selfie to manipulate someone’s face, and literally put words in their mouth, CBS Evening News, April 2018
-
Get Your Geek On: Image Manipulation Technology, Channel One News, Mar 2018
-
The CG Actors in ‘Logan’ You Never Knew Were There, Cartoon Brew, Mar 2017
-
Lighthouse magazine special interview, Lighthouse magazine, Nov 2016
-
Hot Stuff! Guncy’s Eye, CGWORLD Magazine (Japan), December 2015
-
SIGGRAPH 2015 – Incredibly Detailed & Next-Generation Skin Microstructure Deformation Showcased, DSOG, Sep 2015
-
Engineers adopt a flesh approach to making gaming characters more lifelike, Imperial College London (image), Aug 2015
-
A Graphics Breakthrough Makes Perfect CGI Skin, GIZMODO, Aug 2015
-
See the groundbreaking graphics, pressfire, Aug 2015
-
A Graphical Breakthrough Will Permit Lifelike Skin Texture In Future Games, News Everyday, Aug 2015
-
CGI-Breaktrough Renders Skin On A Whole New Level, StuxGeek, Aug 2015
-
Research on 3D skin illustration: realistic facial expressions in real time, PC Games Hardware, Aug 2015
-
Digital Faces Are Looking More Human Than Ever, The Verge, Aug 2015
-
Will This New Development in CGI Skin Overcome the Uncanny Valley?, Mental floss, Aug 2015
-
CGI Skin Just Got a Whole Lot More Realistic, MOTHERBOARD, Aug 2015
-
Breakthrough In Motion Capture Creates Perfect CGI Skin, Tech Times, Aug 2015
-
This Graphical Breakthrough Will Allow Lifelike Skin Texture In Future Games, UNILAD, Aug 2015
-
Tech Breakthrough Will Make Skin Look Real In Future Video Games, ZOCKME, Aug 2015
-
New performance capture tech makes photorealistic skin a reality, GAMASUTRA, Aug 2015
-
Skin rendering is getting a facelift, rugged gamers, Aug2015
-
Convincing computer-generated people take one step closer, BT, Aug 2015
-
Check Out This Almost Perfect CGI Skin, SOFTPEDIA, Aug2015
-
Researchers Develop Ultra-Realistic CGI Skin, TryEngineering, Aug 2015
-
SIGGRAPH technical papers highlight, fxguide, Aug 2015
-
Videos: the best of Siggraph 2015’s technical papers, CG Channel.com, Jun 2015
-
USC dazzles with new technology that enhances CG skin dramatically, CNET Tomorrow Daily, Jun 2015
-
Advances In Skin Rendering Technology Are A Little Gross But Mostly Fascinating, digg, Jun 2015
-
It Feels Like These CGI Skin Designers Are Just Trying To Scare Us Now, GIZMODO, Jun 2015
-
Skin Microstructure Deformation, CG Society, Jun 2015
-
This Is The Most Realistic CGI Skin We’ve Ever Seen, Fast Company, Jun 2015
-
Simulating human skin gets closer to reality, 3DTotal, Jun 2015
-
Can You Tell This CGI Skin Apart From Real Human Skin?, Neatorama, Jun 2015
-
Researchers create creepy, hyperrealistic CGI skin, Slash Gear, Jun 2015
-
Skin Rendering Technology: How To Make Animated Skin Look Incredibly Real, Medical Daily, Jun 2015
-
This CGI Skin Looks Unbelievably Real!, Geeks are Sexy, Jun 2015
-
Skin Microstructure Deformation With Displacement Map Convolution, CG Record, Jun 2015
-
Skin Stretch, Prosthetic Knowledge, Jun 2015
-
Technology of the demo video to express the shade by the expansion and contraction and wrinkling of the skin of the face in real time, DNA, Jun 2015
-
New CGI Skin is the most realistic yet, Hopes&Fears, Jun 2015
-
Too real! Reproduction of skin with CG had evolved up to here!, tozanabo, Jun 2015
-
As real as CGI skin gets (so far…), Red Shark, Jun 2015
-
To express the exquisite wrinkles of the skin surface in the CG!, 3D people, Jun 2015
-
Skin Microstructure Deformation with Displacement Map Convolution, CG Everything, Jun 2015
-
Simulating Skin Microstructure Deformation, CG Press, Jun 2015
-
SIGGRAPH Asia Preview, fxguide, Nov 2019
-
Real-Time Live! Stuns at SIGGRAPH 2019, ACM SIGGRAPH, Aug 2019
-
IT’S LIKE YOU’VE JUST STEPPED OUT OF A…VR HAIR SALON, befores and afters, Jul 2019
-
THE NEW ARTIFICIAL INTELLIGENCE FRONTIER OF VFX, VFX Voice, March 2019
-
SIGGRAPH Asia 2018 Technical Papers Trailer, ACM SIGGRAPH, Nov 2018
-
Pinscreen’s AI-based real-time facial mapping technology, Weekly Famitsu, Nov 2018
-
“Follow this”, Netflix Original and Buzzfeed, Aug 2018
-
12 Cool New Pieces Of Animation Tech We Saw At SIGGRAPH 2018, Cartoonbrew, Aug 2018
-
A.I. at SIGGRAPH: Part 2. Pinscreen at Real Time Live, fxguide, Aug 2018
-
WHAT THE HECK IS SIGGRAPH’S REAL-TIME LIVE! (AND WHY YOU SHOULD DEFINITELY SEE IT), vfxblog, Jul 2018
-
Manipulation von Videos: Was ist noch echt?, ARD, Feb 2018
-
What used to take a sophisticated Hollywood production company week could soon be accomplished in seconds by anyone with a smartphone, LA Times, Feb 2018
-
All the Face-Tracking Tech Behind Apple’s Animoji, WIRED, Oct 2017
-
The 7 best highlights from Siggraph 2017, Digital Arts, Aug 2017
-
Face me part1: photorealistic facial texture from a single still, fxguide, Dec 2016
-
Neural Networks Can Now Turn a Single Photo into a Creepy 3D Face Render, GIZMODE, Dec 2016
-
Exhibit allows virtual ‘interviews’ with Holocaust survivors, Associated Press, Sep 2017
-
How USC’s Automultiscopic 3D Display Works, Tested, Sep 2016
-
A picture drawn in the air that “you can touch”, and a realistic 3 dimensional image, Emerging Technologies report sequel of SIGGRAPH 2015, 4gamer, Sep 2015
-
Finally, A Convincing 3D Display That Doesn’t Require Glasses, GIZMODO, Aug 2015 (Also in GIZMODO JAPAN!)
-
An Auto-Multiscopic Projector Array for Interactive Digital Humans, Creative Applications, Aug 2015
-
Auto-Multiscopic Projector Array for Interactive Digital Humans, Prosthetic Knowledge, Jun 2015
-
Emerging Technologies Submissions: Ones To Watch, SIGGRAPHITTI Issue 3, Jun 2015
-
Holograms add new dimension to Holocaust survivor’s story, NBC Today, May 2015
-
How to live forever, Morgan Spurlock’s CNN Inside Man Episode 2, Apr 2014
-
3-D Projector Blurs Lines Between Fantasy, Reality, Wall Street Journal, Sep 2013
-
An Autostereoscopic Projector Array Optimized for 3D Facial Display, 4gamer, Aug 2013
-
Highlights of SIGGRAPH 2013 Emerging Technologies, SIGGRAPHITTI, May 2013