Sponsored Post: From RAW in Real-time: The Photographic Power of OPPO's New MariSilicon Xby Sponsored Post on December 15, 2021 2:30 PM EST
- Posted in
- Sponsored Post
With smartphones becoming a centerpiece in our lives, their ability to do more and more with incredible power and staggering efficiency is only going to grow more important. Through computational photography, smartphone cameras are able to achieve impressive results, but there are drawbacks. This method of creating images demands serious performance, and it can take time to render the final images. This can mean image previews don’t line up with the final result, and it can create a large gap in image quality between still photography and video, which can’t get the benefit of stacking multiple frames together for a wider dynamic range and cleaner imagery.
The demands of AI image processing are steep, and even phone SoC’s with ISPs and Neural Processing Units (NPUs) on board simply lack the power to perform this AI processing. This lack of power ultimately means shortcomings in image quality that wouldn’t be necessary with a sufficiently powerful NPU. In order to better handle the sheer amount of data coming from cameras, OPPO has built a dedicated and powerful NPU that can go well beyond the capabilities of current smartphones.
To demonstrate the huge leap a powerful NPU can offer, OPPO is introducing its first imaging NPU — MariSilicon X — which will launch in the Find X Series in early 2022. The company has spent 10 years working on its imaging technology, from lenses and sensors to in-house algorithms and now its own NPU. With the vertical integration of these features, OPPO’s MariSilicon X is set to offer new levels of photographic performance for smartphones.
With a cutting-edge Imaging NPU built from the ground up, the MariSilicon X is able to offer serious performance. The built-in NPU can hit 18 TOPS@INT8 of processing and achieves an efficiency of 11.6 TOPS/watt. Its processing speeds need memory that can keep up, so OPPO has included a Terabit memory subsystem and an extra 8.5GB/s of DDR memory bandwidth on MariSilicon X. The memory plays a role in this efficiency as do many AI operations being hardwired into the NPU’s design. With this high performance, the MariSilicon X can perform noise reduction on 4K video at 40fps, fast enough to provide this in real-time.
Currently the smartphone image processing pipeline involves capturing an image and compressing that image down from RAW into RGB and YUV — a process that sees much of the original image data lost. Phones then run their computational photography work on these compressed images, which severely limits the potential quality of the final product. The MariSilicon X’s NPU steps right into the middle of this process, as it has the speeds necessary to handle an image processing pipeline in the RAW domain. In other words, it can run its computational photography work on the original image while all of its data is still intact.
The MariSilicon X enables video capture at 4K with Ultra HDR in 20-bit and a 120dB dynamic range by combining multiple HDR frames in the RAW domain while recording. The effect is individual video frames that are on a level footing with still photography. The power of this processing and the speed it can achieve even allows for 4K AI Night Video — a first-time capability for Android — using RAW data with a final output that’s as vivid as the live preview.
MariSilicon X supports 20bit Ultra-HDR Video
When combined with specialized hardware, the MariSilicon X NPU can take things even further. The MariSilicon X can support OPPO’s RGBW Pro mode using an RGBW sensor, which then lets it process red, green, blue (RGB) and W signals separately and combine them into a single image with an 8.6dB improvement to dynamic range and 70% more texture.
With the limitations of the current imaging pipeline and the breakthrough potential of a dedicated NPU with enough power to process images in the RAW domain, the benefit of adding imaging NPUs to smartphones is evident. They’ll allow users to capture photos at a higher quality and turn night video into a reality.
MariSilicon X brings 4K AI Night Video on Android smartphones for the first time where light intensities lower than 5 lux
You can learn more about the MariSilicon X and all of OPPO’s forthcoming innovations and technological offerings at OPPO Inno Day 2021.
Post Your CommentPlease log in or sign up to comment.
View All Comments
Duncan Macdonald - Wednesday, December 15, 2021 - linkInteresting - however there is some critical missing data - what is the power consumption in its different modes ? Phones have a very limited total energy and a limited power consumption (due to heat) - how much would the MariSilicon X impact those limits ?
There is also the inevitable question of how much a phone's price would be inflated to include this chip.
dxu - Thursday, December 16, 2021 - linkMariSilicon X would consume 797mW when performing noise reduction on 4K video at 40fps.
Source: https://mp.weixin.qq.com/s/_wWZ8kYUTSy350itGQuStA (In Chinese, but has much more detail
brucethemoose - Wednesday, December 15, 2021 - linkDoes anyone from OPPO read the comment section?
This is a great idea for a sponsored post. TBH it's my favorite from Anandtech so far, and it actually increases my interest in buying an OPPO phone.
While performance specs are nice, but ya'll should post more technical details, like the layout of the various silicon blocks, memory specs, die sizes, fab process and so on. Those nitty gritty details are big draw here at Anandtech, even if they aren't particularly relevant to what us end users do with a phone.
brucethemoose - Wednesday, December 15, 2021 - linkAlso, Anandtech *still* needs an edit button, even if it only has a 3 minute timer.
Duwelon - Wednesday, December 15, 2021 - linkEdit burtons are so 1999, long live the unedible web!
Slash3 - Thursday, December 16, 2021 - linkPlease do not eat the web.
Fulljack - Thursday, December 16, 2021 - linknow anandtech needs upvote button
zodiacfml - Wednesday, December 15, 2021 - linkThis is a better post vs from other sites. Amazing if this does video better than the iphone. the iphone pretty much up there in terms of processing with crazy wide dynamic range that the look of the video is much like HDR look of photographers around 10 years ago.
I also did not know smartphone computational photography works on compressed data until now
Byte - Sunday, December 19, 2021 - linkseriously, the iphone 13 pro vids look twice as vibrant as real life when im shooting them, and looks like twice the detail of what i can see. we are past futuramas real life HD.
linuxgeex - Wednesday, December 22, 2021 - linkIt's a shame though that with all that dynamic range available, iPhone still tends to clip highlights horrendously.