SDSC-Connect 25th May 2023

Towards Biomedical Data Science & Precision Medicine | Conference

HMP

The Human Measurement Project

Abstract

In certain developing countries, child malnutrition is a common problem. To prevent it, it is important to track the nutrition status of children. This is conventionally done by visiting children at their home and taking basic measurements of height, weight, and other relevant attributes. This process is done manually and is both tedious as well as error prone. The bmAi project aims to develop a mobile phone based solution to make the process of measuring height and weight both easier as well as faster. It is a joint project between EssentialTech (ET), Signal Processing Lab (LTS5), and SDSC, in collaboration with Lausanne University Hospital (CHUV) and Terre Des Hommes (TDH). The goal of the project is to estimate the height and weight of an individual from a frontal and lateral picture of an individual taken in real-life setting using an uncalibrated mobile phone camera. This is a very challenging problem from the point of view of computer vision.

Starting Date / Status

April 2018

COMPLETED

PI / Partners

EssentialTech EPFL

Data Science Laboratory (EPFL)

Signal Processing Laboratory 5 (EPFL)

Neonatology, Faculty of Biology and Medicine (UNIL)

In-Zone Inter Faculty Center (UNIGE)

Description

Problem:

  • Malnutrition remains widespread worldwide.

  • 1/3 of African children are stunted.

  • Anthropometric measurements are essential to diagnose malnutrition & other health conditions.

  • Measurements taken on site with manual means are often imprecise/unreliable, especially with children. The idea is to use advances in machine learning and computer vision to obtain these measurements purely from images taken using mobile phones.

Solution:

  • A mobile application based on Ai that can estimate body measures from a cellphone image.

  • A sustainable business model for deployment in both developed and developing countries.

Impact:

  • Fight malnutrition in low income contexts

  • Innovative medical diagnoses based on Ai-enabled anthropometry.

  • Dual business models: premium in industrial context vs freemium in low-income contexts.

Progress

The goal of the project is to estimate the height and weight of an individual from a frontal and lateral picture of an individual taken in real-life setting using an uncalibrated mobile phone camera. This is a very challenging problem from the point of view of computer vision. Two approaches were taken to work towards a solution.

The first approach involved the estimation of height followed by the estimation of weight in two steps. In order to estimate the height, the joint locations of limbs were detected using a method available in existing literature. The distances between joints were used to estimate the full height of the individual based on similar annotated data. In order to detect the weight, first the silhouette of the individual is detected using an existing method. Assuming a near elliptical shape and given the height, the volume of an individual is estimated using the frontal and lateral silhouettes as the cross-sectional measurements. Density values are regressed upon for each pixel-level slice of the estimated volume, using which the weight can be estimated.

The main drawback of the two-step process is that an error in the computation of the height accumulates with the error error in the estimation of the weight. To avoid this, a second, single step approach is taken. A deep network is trained to simultaneously compute the silhouette, the skeletal joints, as well as the attributes of height and weight in a single step, taking only a frontal image as input (see figure below). A paper presenting this was a to accepted in ICASSP 2020 [1].

Since the images and data related to the child images were small in number and privacy sensitive, all training was performed using publicly available data for adults. The idea is to perform the training on these and then fine-tune the models for the child images. The result was that the model attained acceptable error (less than 10%) in predicting height, but more that that for weight estimation.

Publications

 

  • [1] Height and Weight Estimation from Unconstrained Images, Can Yilmaz Altinigne, Dorina Thanou, Radhakrishna Achanta, IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2020

  • [2] Human biases in body measurement estimation, Kirill Martynov, Kiran Garimella, Robert West, EPJ Data Science 2020