DxOMark Pixel 3 score based on 'default settings'
Let's talk for a moment about what it means to test the quality of a camera on a smartphone as we enter the year 2019. The folks at DxOMark suggest that their smartphone camera tests are done "using only their default settings." When it comes to a modern smartphone camera, is only looking at default settings for a review really fair? When it comes to a smartphone camera with a significant amount of post-processing, is it really fair to consider the device's default results VS those of a less processing-intense smartphone camera setup?
Before we get too deep here, if you just came here for the score, first of all how dare you? But also, it's a score of 101 for the Google Pixel 3, which means its currently in 8th place in the mobile phone camera chart overall. It's beaten by, in order, Huawei P20 Pro, iPhone XS Max, HTC U12+, Galaxy Note 9, Xiaomi Mi MIX 3, Huawei P20, and iPhone XR. And iPhone XR has the same score as the Google Pixel 3, but appears in the chart ahead of the Pixel 3 for... whatever reason.
Hardware
Sony provides the vast majority of camera sensors to smartphone makers around the world. They just do – they've done so for the vast majority of the time smartphones have existed in mass numbers. At the same time, Qualcomm provides the bulk of the smartphone processors in the world – and these processors and their many connected components make the phone work, and in some cases provide a significant amount of the image processing, too.
Regardless of this, the end-product images any smartphone camera captures can indeed be significantly different in quality from images captured by a very similar smartphone. Each individual manufacturer is responsible for fitting the camera hardware to the rest of their hardware – and their software. The difference between a good manufacturer and a fly-by-night manufacturer can be giant.
Testing Defaults
Software combined with quality hardware engineering make all the difference in a smartphone's ability to capture high quality photos. When it comes to a smartphone like the Google Pixel 3, the big difference comes in the post-processing software. Google has a system with which the end-user is able to snap extremely high-quality photos with very little technical skill.
SEE TOO: Google Pixel 3 Frankenphone!
The Google Pixel smartphone series continues to get updates years after release – Google continues to improve their camera via software year after year. This is sort of true of other cameras – especially when it comes to companies that sell their phones with importance placed on software updates. But for Google, it seems as if the camera hardware couldn't matter less – not when there's magic post-processing to be done!
Is testing the default settings in a smartphone really the most fair way to test a smartphone's camera? What is DxOMark really testing, if they're willing to use whatever settings the smartphone manufacturer's set as default? It's not the hardware anymore, surely. It would seem, instead, that DxOMark is now testing the photos captured by the average smartphone user – the user that never, ever adjusts settings or attempts to capture photos with any app other than the camera app included by the manufacturer. Is there a better way?