Massive ISP Upgrades

I’ve been hearing for some time now that 2022 flagships will have massive camera upgrades, and the new features of the next-gen SoCs being described by MediaTek and now also Qualcomm explain why that is.

The new ISP of the Snapdragon 8 Gen 1 falls under a new marketing name – “Snapdragon Sight”, and includes large improvements of the capabilities of the IP blocks within the image processing chain.

The big flagship feature being advertised is the fact that the new ISP is now capable of 18-bits of color depth per channel, up from the previous generation 14-bit ISP. While mobile image sensors nowadays still are barely 12-bit native in terms of their ADCs, the ushering of new HDR techniques such as staggered HDR capture, where exposures are immediately subsequent to each other on the sensor’s readout, means that new phones now are able to capture images a lot faster, recombining them into higher bit-depth results. Particularly here, the new 18-bit ISP pipeline now allows for three exposure HDR stacking off these new sensors.

The increased bit-depth should allow for an increase of 4 stops in dynamic range (or 2^4 = 16x the range), which greatly helps with very contrasting environments and challenging lighting situations. This is quite beyond any other camera solution right now, and being able to have this implemented in such a hardware fashion sort of blurs the line between traditional image capture techniques and the more software-defined computational photography methods of the last few years.

Indeed, the new ISP architecture seems to be very much a way to implement many of the existing computational photography techniques into fixed-function blocks: there’s a new neural-net controlled 3AA (auto-exposure, auto-focus, auto-white-balance) and face detection block, which sounds eerily similar to Google’s HDRnet implementations.

Night mode is also said to be vastly improved through a new multi-frame noise reduction and image stacking block, being able to now stack and align up to 30 images, and achieve also much finer detail this generation. Qualcomm here claims up to 5x better night mode shots.

Further improvements include a new distortion correction block that’s able to now also correct for chromatic aberrations, and a hardware video Bokeh engine, being able to operate at up to 4K video recording. Think of it as the same as the new Cinematic mode on the new A15 iPhones, but not only limited to 1080p.

Qualcomm notes that all the AI/ML/neural network features on the ISP are actually run and accelerated on the ISP itself, meaning that it is not offloaded onto the Hexagon dedicated ML processing blocks or the GPU.

Just as a note- Qualcomm’s 3.2Gigapixel/s throughput metric here seems low compared to the Dimensity 9000’s 9Gpixel/s, it’s possible that the companies are advertising very different metrics, with MediaTek advertising the throughput of lower-bit depth pixels coming from the image sensors per frame, while Qualcomm quoting the full bit depth pixel processing within the ISP itself.

In terms of video encoders and decoders, the new chip allows for 8K HDR recording now, but otherwise is seemingly on par with the Snapdragon 888 media blocks. Unfortunately, this also means no AV1 decoding this year yet again. Qualcomm isn’t part of the Alliance for Open Media consortium and instead is backing VVC/H.266 and EVC, however with AV1 being actively pushed by Google and YouTube, and seeing large adoptions such as by Netflix, it’s becoming questionable for Qualcomm to still not support the format in 2022 devices.

AI Performance - Iterative, but solid

Last year’s Hexagon IP block was a very large change for the Snapdragon 888. At the time, Qualcomm moved on from a more segregated DSP/AI architecture to a single more fused-together block being able to operate on scalar, vector, and tensor operations at the same time. This year’s iteration is an improvement of that larger change. Qualcomm notes that amongst many changes, they’ve doubled up on the shared memory of the block, allowing for greater performance for larger ML models (which are growing at a very fast pace).

Qualcomm didn’t note any TOPS figures this time around, instead stating we’re seeing 2x the tensor throughput performance, and smaller increases for scalar and vector processing. They do quote a day-1 to day-1 performance increase of 4x when compared to the Snapdragon 888, via a combination of both hardware and software improvements, but of course that figure is smaller when comparing both platforms on an equal software footing.

Power efficiency for AI workloads is said to be 70% better this generation, which is actually more significant, and should help with more demanding sustained ML workloads.

X65 Modem Integrated

In terms of connectivity, the Snapdragon 8 Gen 1 is quite straightforward, as it integrates the X65 modem IP that Qualcomm had already announced as a discrete model earlier this year.

The improvements here are the fact that it’s a 3GPP Release 16 compatible modem, including new features such as uplink carrier aggregation. Other improvements are 300MHz of Sub-6 bandwidth on 3 100MHz carriers, and an increase of the mmWave bandwidth from 800 MHz to 1000MHz, allowing a new peak theoretical downlink speeds of 10Gbps.

Introduction & Specifications - CPU & GPU Conclusion & First Impressions
POST A COMMENT

219 Comments

View All Comments

  • mode_13h - Friday, December 3, 2021 - link

    > HEVC is the winner in terms of adoption as hardware support beats claims made companies.

    Talk about an unfair comparison! HEVC was standardized long before AV1 was a thing!
    Reply
  • name99 - Thursday, December 2, 2021 - link

    That's an exceptionally naive analysis.
    Most of the member of AOM have zero interest in ACTUALLY supporting yet another codec. AV1 (and AV2) exist purely to ensure that the REAL codecs (VVC, and EVC as the royalty-free version) come with reasonable terms so no-one (*cough* MPEG LA *cough*) tries to pull some shit, like kinda-sorta happened with h.264 in the early days of streaming.
    Reply
  • Zoolook - Wednesday, December 8, 2021 - link

    Qualcomm is one of the companies behind HEVC, they get license money for every hevc enabled part that sells, why would they support the competition?
    They won't unless they really have to, and so far they don't.
    Reply
  • name99 - Thursday, December 2, 2021 - link

    Of course VVC use is non-existent. It was finalized in 2020 (as opposed to AV1 in 2018).

    Apple had their first h.265 decoder in iPhones a year after 265 was finalized BUT this was not really publicized until a year later, so that when Apple talked about h.265 as their new preferred codec, they could point to a large pre-existing HW base.
    Chances are they will follow the same strategy for 266, suggesting that A15 may already have a decoder as part of the media block, but h.266 will not be discussed as part of Apple's preferred codecs until at least next year.

    I *think* Mediatek have a decoder, so it may be present on their chipsets. But all these things take time...
    Reply
  • name99 - Thursday, December 2, 2021 - link

    AOM and AV1 have done their job.
    - licensing around future codecs will be more sane, without any backsliding
    - EVC is essentially equivalent to AV1 in performance and cpu cost, but is an ITU/ISO standard AND is royalty-free

    There's just no job left for AV1 to do except persist as a *possibility* if someone owning a patent that's part of VVC behaves stupidly.
    Reply
  • mode_13h - Friday, December 3, 2021 - link

    > AOM and AV1 have done their job.

    That seems premature. For it to be a credible threat, AV1 needs hardware support. Otherwise, if there's a situation with lots of VVC hardware and no AV1 hardware, then the VVC patent holders might feel they have the leverage to force stricter licensing terms.
    Reply
  • vladx - Wednesday, December 1, 2021 - link

    HEVC us used by Netflix, Amazon Prime and pretty much everything outside Google's ecosystem. Just because they recently adopted AV1, doesn't mean Netflix has dropped HEVC which has much wider hardware support. Reply
  • GeoffreyA - Wednesday, December 1, 2021 - link

    vladx, HEVC may be more widespread, but was a lacklustre codec in many ways. Arguably, it never reached the excellence of its predecessor. AV1 is noticeably better than HEVC, and is in the same class, but slightly behind, VVC. Reply
  • vladx - Wednesday, December 1, 2021 - link

    What do you mean? It has 50% better compression than H.264 and it's even more efficient than AV1 in terms of bandwidth usage as comparable quality settings, AV1 is only better space-wise which is less important since more space is cheaper to buy than more bandwidth. Only Google decided to cheapen out on paying royalties because of their NIH syndrome but that doesn't make HEVC lackluster as a codec in any way whatsoever. Reply
  • vladx - Wednesday, December 1, 2021 - link

    And when I ssay that HEVC is more efficient bandwidth-wise than AV1, I'm referring of course to resolutions of 4K and above.. AV1 is better at 1080p and below but those resolutions are not really a concern in terms of bandwidth consumption. Reply

Log in

Don't have an account? Sign up now