- Google is building on its new Insulin Resistance Trends health feature with PhotoScan
- PhotoScan is ‘an investigational deep learning framework’ that can estimate body fat using a phone’s camera with ‘near DXA accuracy’
- It’s not yet available to the public, but the research ‘demonstrate[s] the feasibility’ of such features
As the Google Pixel Watch 5 launched last week alongside a slew of new Pixel phones, one of the most interesting new developments was the introduction of the Pixel Watch’s new Health Guardian suite of tools, including Insulin Resistance Trends.
Insulin Resistance Trends showcases a month-by-month breakdown containing estimations of how your body responded to blood sugar spikes — a crucial component in weight management and the development of Type-2 diabetes.
Body composition — your ratio of different kinds of fat, skeletal muscle, and bone — is also an excellent marker of insulin resistance and metabolic health, so body scans are often used to help determine insulin resistance. Body comp is often measured with medical-grade ‘DXA’ scans, which use X-Rays, or commercially available devices at home.
These home devices can be the best smart scales or even some smartwatches, such as the Samsung Galaxy Watch Ultra 2. Without medical DXA scanners, they usually use a process called ‘bioelectrical impedence analysis’, or BIA. A low-level electric current is sent through the user’s body, moving through fat, muscle, and bone at different speeds. The speed of the electrical current helps the device estimate the percentages of each material in the body: a useful thing to have on a health-monitoring device like a smartwatch.
However, Google is busy researching an even more accurate version of a body scan without using any specialist equipment or BIA kit — in fact, it’s just a smartphone camera and an onboard AI model. This in-development feature is called PhotoScan, appropriately.
According to its research blog, Google has been developing PhotoScan by matching smartphone photographs of subjects with medical-grade DXA scans, along with other subject information, then running it all through an AI model. The AI model eventually achieved ‘strong DXA agreement’ when estimating body fat percentage and other metrics, leading to ‘near-DXA accuracy for predicting insulin resistance’.
The researchers conclude that ‘clinical DXA imaging delivers the most accurate body composition but lacks scalability, whereas wearable BIA sensors offer convenience but are limited to basic body fat percentage.
‘Our PhotoScan approach offers a promising middle ground, estimating granular body composition from standard smartphone imagery with near-DXA accuracy.’ Sorry, Samsung — it looks like we’ll be pivoting away from BIA sensors and towards smartphone cameras for body fat percentage calculation in future.
While the technology doesn’t seem to be commercially available yet, it does ‘demonstrate the feasibility of smartphone-based body composition estimation,’ and I’d be surprised not to see it make its way to the best Pixel phones eventually.
