During the state of Unreal stream, my colleague Matt Stoneham presented some cool example of how we used the ML Deformer system in UE5 to approximate a complex muscle deformation setup that normally takes about 1 minute per frame. This flesh deformation in UE 5.2 takes around 0.02 milliseconds on the CPU and 0.29 milliseconds on the GPU and uses around 18.7 mb of GPU memory. This is on PS5. We still think we can optimize this even further though! We also applied another ML Deformer model for the cloth. This produces better results on clothing than the model we used for the flesh. The cloth model is a bit more heavy, but even with clothing it still runs at around 1 to 2 milliseconds when fully clothed. This is a high fidelity character. Simpler models run much faster. It has also been used in Fortnite for example. Below is a little recap video which shows some parts of the presentation that show the deformer in action. I hope we can share a higher quality demo later on. I think the ML Deformer can be very cool for hero characters and specific monsters or animals that really show some complex muscle deformation. Keep in mind that you can train it on almost any deformation though. It doesn't have to be cloth or muscles. I'm proud of the talented team that pulled it off to create this demo and asset! The full source assets of this demo will be shared later on, so you can experiment with it yourself as well! Vojislav Pavkov did an amazing job setting up the muscles in Houdini! And of course this is a multi-team effort, between research, animation, rendering and tech art etc! There is a very talented team behind all this. #mldeformer #ue5
Is there a way to utilize this with Daz3D? I ustiluze Daz3D for generating ergonomic audit illustrations for product design projects. Having more realistic body deformation as the models are posed would further the realism of the images. Even if they are all just rendered matte gray to emphasize the product being illustrated.
Very cool! What is the GPU/CPU impact per bone? and is there any different going on here than this paper aside from the PCA? https://graphics.berkeley.edu/papers/Bailey-FDD-2018-08/Bailey-FDD-2018-08.pdf The results look really natural, curious if there is a different method you guys are using to manage the crossover between different models per bone. Found this a little problematic in my own implementations of this, you can extra multiple bones as training inputs per model, but the amount of training data required increases in tandem as well with each input??
Really amazing. Thanks for the detailed overview. Do you plan to provide pretrained models for 18x metahuman bodies? Or will we be able to use the sample that mentioned to be available later to use it on metahumans? We are experienced Houdini users.
Hello John, superb work! This is very interesting to me as I'm developing a muscle model generation toolset in Houdini and wondering how hard it is to plug in the resulting simulation from Houdini? I watched the full presentation and am very curious of that workflow.
Virtual fitness influencers coming soon
John van der Burg This is a live 3D rendering I recorded of my arm, instantly rendered the instant it is captured. Feel free to reach out. https://1drv.ms/v/s!AsWv7-JbSv0NhYQMFbL6UmkDxUljoQ
Small feedback: I may be in the minority here, but seeing a video start with a close-up of a guy's naked back, especially before I knew what I was watching, made me... uncomfortable. This could easily be fixed by reordering the video to lead with a clothed example (or at least a face in frame) and then move to the bare back once it has been established what I'm looking at.
Principal Animation Programmer at Epic Games/Unreal Engine
1 年Full presentation with tech details at: https://www.youtube.com/live/teTroOAGZjM?feature=share&t=19000