Avatar Digitization From a Single Image For Real-Time Rendering (SIGGRAPH Asia 2017)

  Рет қаралды 62,125

Hao Li

Hao Li

6 жыл бұрын

SIGGRAPH Asia 2017 Paper Video: We present a fully automatic framework that digitizes a complete 3D head with hair from a single unconstrained image. Our system offers a practical and consumer-friendly end-to-end solution for avatar personalization in gaming and social VR applications. The reconstructed models include secondary components (eyes, teeth, tongue, and gums) and provide animation- friendly blendshapes and joint-based rigs. While the generated face is a high-quality textured mesh, we propose a versatile and efficient polygonal strips (polystrips) representation for the hair. Polystrips are suitable for an extremely wide range of hairstyles and textures and are compatible with existing game engines for real-time rendering. In addition to integrating state-of-the-art advances in facial shape modeling and appearance inference, we propose a novel single-view hair generation pipeline, based on 3D-model and texture retrieval, shape re nement, and polystrip patching optimization. The performance of our hairstyle retrieval is enhanced using a deep convolutional neural network for semantic hair attribute classi - cation. Our generated models are visually comparable to state-of-the-art game characters designed by professional artists. For real-time settings, we demonstrate the exibility of polystrips in handling hairstyle variations, as opposed to conventional strand-based representations. We further show the e ectiveness of our approach on a large number of images taken in the wild, and how compelling avatars can be easily created by anyone.

Пікірлер: 61
@1989worms1
@1989worms1 4 жыл бұрын
This is amazing! This is perfect for everything including gaming. Keep up the good work, Mr. Hao Li!
@reloadoriginal9709
@reloadoriginal9709 6 жыл бұрын
the real-time targeting and rt illumination is NEXT LEVEL.
@VRchitecture
@VRchitecture 6 жыл бұрын
Impressive! And I believe this is just the beginning…
@RyRyQTPi
@RyRyQTPi 6 жыл бұрын
This is amazing! Insanely cool.
@boydzhang
@boydzhang 6 жыл бұрын
Hi, Hao. I observe in many similar demos, subtle trembling of expression does exist. Could you name the cause and how can it be improved?
@English_Lessons_Pre-Int_Interm
@English_Lessons_Pre-Int_Interm 5 жыл бұрын
I have so many ideas after watching this, but all I can write is that I am stunned. The Korean child is so brutal, and the lady at the end is so ... moving.
@yzjacob
@yzjacob 6 жыл бұрын
Amaaaaaaaazing!
@jingzheyuan3488
@jingzheyuan3488 6 жыл бұрын
This is really cool! :D
@JorgeGamaliel
@JorgeGamaliel 6 жыл бұрын
Amazing!!. Iwould like to use your program in order to make a lot of avatars of my family and me with my photos
@TiagoTiagoT
@TiagoTiagoT 6 жыл бұрын
How do the models compare to actual 3D scans of the real faces?
@ShrubRustle
@ShrubRustle 6 жыл бұрын
amazing! i assume there will be an ability to add options to edit neck size/weight, and probably some head depth correction? some of the heads seem a bit shallow, and the necks, particularly on the world leaders featured, are much thinner than those of their real life counterparts. also, are textures for braids/dreadlocks in the works? I noticed the celebrity's braids at 5:40 were translated to straight hair, which could get you in trouble in some cultural contexts.
@ibrremote
@ibrremote 6 жыл бұрын
The head-mounted camera-based performance: eyeball left-right movements not captured as much.
@English_Lessons_Pre-Int_Interm
@English_Lessons_Pre-Int_Interm 5 жыл бұрын
If I have a PC, how do I use this avatar software? Only with an Android emulator?
@visentengbusilak7607
@visentengbusilak7607 6 жыл бұрын
Hi Sir I just want to know if we can use the file that Avatar processed???
@English_Lessons_Pre-Int_Interm
@English_Lessons_Pre-Int_Interm 5 жыл бұрын
Is there a similar solution for a webcam+PC?
@qiaohui220
@qiaohui220 6 жыл бұрын
Super cool!
@raportagenoffiziell3277
@raportagenoffiziell3277 6 жыл бұрын
Download for PC? Please ?
@TiagoTiagoT
@TiagoTiagoT 6 жыл бұрын
Why not first generate a low res texture mixed with some parametric texture for fine details, and let the user already start using the avatar while the high res texture is generated in the background?
@I.R.Donsen
@I.R.Donsen 6 жыл бұрын
Any plans for a Windows or Android version?
@gloriousday4338
@gloriousday4338 6 жыл бұрын
How do I buy a copy? ...I looked all over the web site.
@visentengbusilak7607
@visentengbusilak7607 6 жыл бұрын
can we use that in 3d print???
@onee
@onee 6 жыл бұрын
This would be actually great for a game like the Sims.
@kebman
@kebman 6 жыл бұрын
Yes yes, or Pornhub!
@StuartWilson4d
@StuartWilson4d 6 жыл бұрын
Is this available anywhere?
@JasonMayes
@JasonMayes 6 жыл бұрын
This is truly amazing. Is there any way to export the 3D model to WebGL / Three.js for use on the web? I would love to do this with my face and use the animated model :-)
@HaoLi
@HaoLi 6 жыл бұрын
www.pinscreen.com
@derindarbecoma156
@derindarbecoma156 6 жыл бұрын
amazing. where is for windows
@handet.6235
@handet.6235 6 жыл бұрын
any code tutorial?
@andrydanielfanomezantsoaph570
@andrydanielfanomezantsoaph570 6 жыл бұрын
Soft name?
@CybeRin
@CybeRin 6 жыл бұрын
Неплохо!
@danielwhatling3686
@danielwhatling3686 6 жыл бұрын
what software is that
@3ghighspeed599
@3ghighspeed599 3 жыл бұрын
Hi which software is it? please let me know the software name? or give me the website link please
@Molkalda
@Molkalda 6 жыл бұрын
impressive and you even used trumps idiotic face making. oh and just a quick question (I obviously don't understand how any of this works) but would it be possible to write something that scans the internet/KZfaq for face close up shots and automatically create a database of faces and animation that can be used for things like character creation/ animation?
@lightcorrection
@lightcorrection 6 жыл бұрын
*_Site link plz_*
@frederikpedersen9633
@frederikpedersen9633 4 жыл бұрын
name on the softsare you use?? :)
@HaoLi
@HaoLi 4 жыл бұрын
www.pinscreen.com
@CariagaXIII
@CariagaXIII 5 жыл бұрын
honestly this tech are never released for public download
@000jimbojones000
@000jimbojones000 6 жыл бұрын
Sell it to Bethesda. They could build it into the next Elder scrolls or Fallout. So you can play as yourself... How cool is that?
@mageenderman
@mageenderman 4 жыл бұрын
If they can I'd rather them make it something anyone could use not just Bethesda
@ProgressiveForest
@ProgressiveForest 6 жыл бұрын
Is the hair also generated? Amazing work.
@freeshavaacadooo1095
@freeshavaacadooo1095 6 жыл бұрын
how does it figure out the back of the head and shit?!?!?!
@kevinwitteveen4997
@kevinwitteveen4997 6 жыл бұрын
It doesn't. It's most likely fake.
@yechen5490
@yechen5490 6 жыл бұрын
fucking amazing
@NSPlayer
@NSPlayer 6 жыл бұрын
When are you gonna release it? Why hide this?
@HaoLi
@HaoLi 6 жыл бұрын
yes on Apple App Store: itunes.apple.com/us/app/pinscreen-instant-3d-avatars/id1322184255?ls=1&mt=8
@Dane411
@Dane411 6 жыл бұрын
Why not for Android?
@HaoLi
@HaoLi 6 жыл бұрын
we are working on it :-)
@Dane411
@Dane411 6 жыл бұрын
I actually can't believe this is real. I've been researching for quite some time, how to get in a little time a working 3d model with enough accuracy regarding the input subject (if alive, 3d scan, if dead, 2D RGB photo/video), and there are several interesting papers, ie. Total Moving Face Scanning ECCV 2014, but most of them showcase an extremely promising technology but little to none hints on how to have a working software to mess with :P Congratulations for the hard work, and the amazing technology you've right there! :)
@sealhuntersclan1775
@sealhuntersclan1775 6 жыл бұрын
6:45 imagine going on omegle and you see this
@chitrikart2328
@chitrikart2328 3 жыл бұрын
We wouldn't have a nice day ahead of me and my family is in the morning and I don't know what's wrong with being a feminist too by saying stuff like that I can do it too late to office today as I am not talking about your ass bitch get ur hands off her to be a policeman then are you going to tell him to not become policeman because they support women in the morning and I don't know what's wrong with being a feminist.
@Mr_i_o
@Mr_i_o 6 жыл бұрын
Awesome stuff, kinda funny it gives these asians some white people noses - something for version 2.0 ? :D
@HaoLi
@HaoLi 6 жыл бұрын
Yeah, well observed, we are working on that, should be updated in not too long :-)
@Mr_i_o
@Mr_i_o 6 жыл бұрын
Cool great to hear, any plans for release or licencing? Our team uses Unreal Engine 4, and your work would be amazing - may even qualify you for Unreal Dev Grant if you adapted it to the engine (c++).
@coole8
@coole8 6 жыл бұрын
How the fuck does he know the back of the headddd
@HaoLi
@HaoLi 6 жыл бұрын
it doesn't but it predicts a plausible solution
@Avetho
@Avetho 6 жыл бұрын
Jerry Grauert If the program can't see the mullet, it'll assume it isn't even there. Unless it has an option for multiple viewpoints for higher accuracy. Then I'd make the mullet.
@selenefrost6267
@selenefrost6267 6 жыл бұрын
MY guess would be that it has a database of hairstyles then it selects the one with the closest match to the front and puts the back of that hairstyle on maybe mixes and matches. if the creator sees this could you tell me how close i got it?
@Kazanchev
@Kazanchev 6 жыл бұрын
Hairstyle on the back of the head can't be predicted, pure BS.
@toomuchtruth
@toomuchtruth 6 жыл бұрын
They probably just trained the algorithm with a bunch of images showing the front and back of heads. Thus the algorithm makes an educated guess what the back looks like from the front based on images it's studied before.
ClothCap (SIGGRAPH 2017)
5:41
Michael Black
Рет қаралды 59 М.
We Need to Rethink Exercise - The Workout Paradox
12:00
Kurzgesagt – In a Nutshell
Рет қаралды 5 МЛН
Iron Chin ✅ Isaih made this look too easy
00:13
Power Slap
Рет қаралды 36 МЛН
Пранк пошел не по плану…🥲
00:59
Саша Квашеная
Рет қаралды 6 МЛН
Stay on your way 🛤️✨
00:34
A4
Рет қаралды 22 МЛН
Самый Молодой Актёр Без Оскара 😂
00:13
Глеб Рандалайнен
Рет қаралды 12 МЛН
MrBeast Just Destroyed His Career..
22:02
MorePegasus
Рет қаралды 5 МЛН
Did We Just Change Animation Forever?
23:02
Corridor Crew
Рет қаралды 3,3 МЛН
3 Hours vs. 3 Years of Blender
17:44
Isto Inc.
Рет қаралды 4,4 МЛН
Driving High-Resolution Facial Scans with Video Performance Capture
5:33
ICT Vision and Graphics Lab
Рет қаралды 80 М.
WARNING ARTISTS: AI speedpaints are here!!
11:12
Lola Holliday
Рет қаралды 120 М.
S24 Ultra and IPhone 14 Pro Max telephoto shooting comparison #shorts
0:15
Photographer Army
Рет қаралды 10 МЛН
ВАЖНО! Не проверяйте на своем iPhone после установки на экран!
0:19
ГЛАЗУРЬ СТЕКЛО для iPhone и аксессуары OTU
Рет қаралды 6 МЛН
8 Товаров с Алиэкспресс, о которых ты мог и не знать!
49:47
РасПаковка ДваПаковка
Рет қаралды 168 М.
iPhone socket cleaning #Fixit
0:30
Tamar DB (mt)
Рет қаралды 17 МЛН