Tips and settings for sharpening and denoising 360 footage without compromises – Before/after comparisons!

Scroll down for settings you can copy/paste if you just want the tips ??

When working with 360 video, every pixel counts. Even with 11k footage from the best cameras in the industry, you still have noise and unsharp details. Noise in VR can be especially annoying to the eye. Being able to sharpen and denoise intelligently will get you so much more out of your material.

Background

6 years ago, MANND filmed a bunch of 360 footage in India and other locations around the world for their legendary VR documentary, X-Ray Fashion, which we helped build the experience for. The footage is great, but I remember even then we wished we for example could lower some of the noise in the darker areas of the footage. We tried a bunch of things to improve it using classical means, but it always seemed to come with too many drawbacks, like mudding of the image.

Now, as announced previously by MANND, the X-Ray Fashion just received a grant to get upgraded into a lighter more eco-friendly version that would make it easier to distribute to educational institutions. Though Manyone helped make the original version, we will not be involved in this upgrade. Nonetheless I couldn't help myself to take a second look at those original 360 videos ??

In this article I will show you the settings I found to improve the original material to 60fps, sharper details and no noise in dark areas... without adding to the file size.

Settings for sharpening and denoising 360 footage

Since 2018 a lot has happened on the subject of sharpening and noise reduction. Especially at my team we've fallen in love with the swizz-army-knife Topaz Labs and in particular Topaz Video AI. Time and time again we've come back to this tool. You can literally use it for anything and quickly you realise that no material is so bad it can't be saved.

One thing to note: We were quite early adopters of the Topaz Labs suite at Manyone so we got it for a pretty good price. But that also mean that our license is a bit outdated. If you buy the software now, the algorithms might have improved further and the UI might look a bit different.

The footage was filmed in 8k with the Insta360. One of the top of the line cameras of its time. It's a pretty good all round camera, but dark areas has quite some noise and it does feel like all details could do with some sharpening. It's also only 30fps.

Settings

Resolution: 8k. For all of the footage I went with keeping the original resolution at 7680x3840. While upscaling to 11k is possible it's also much more demanding for standalone devices to play back and takes up a lot more space. The performance could be used better elsewhere in the experience. Upscaling is also a bit of a speculative practice with 360 footage imo.

FPS: 60fps. One awesome feature of Topaz Video AI is the ability to add synthetic frames to footage. In 360 video, 60fps makes a world of difference, so enabling this setting was a given. Is it as good as native 60fps recorded by camera? of course not. But it's pretty good, and this allows way worse and way smaller cameras to compete with the large ones. Slower frame rates allow for more light to be captured as well so that's a pro if you're deciding on what settings to use in your next recording.

AI enhancement model: Proteus / Manual. This is the core place where I improve the footage. This is where you sharpen and denoise. What I wanted was naturally sharper edges for a more crisp feel and especially I wanted the noise gone (which can be really visible in a VR headset.

Depending on the footage I used two kinds of settings:

1. For general footage in normal light: Revert compression (50) – ****Recover details (0) – Sharpen (0) – Reduce noise (70) – Dehalo (0) – Antialias / Deblur (100).
2. For extremely dark footage (The catwalk scene): Revert compression (100) – Recover details (0) – Sharpen (50) – Reduce noise (100) – Dehalo (50) – Antialias / Deblur (100).

Export format: h265 Main, mp4 container, bitrate 60 Mb/s. This should allow direct playback on standalone VR android devices. You could potentially lower the bitrate to save space.

Results

That's it! With these settings the material looks like it's suddenly in HD, but resolution wise nothing has changed. Scroll on to see the results of my tests. The Before is on the left side and the AI enhanced result is on the right ??

When comparing don't just look at the obvious details. Try to look inside the darkest spots too to see how the splats of noise are gone.

If you're reading this text, thanks ??