Task 4.1. Development of mobile apps to collect dietary information (PIs Jean-Noel Patillon (CEA); Marc Schoenauer(INRIA); G. Fagherazzi (CESP))
Mobile devices provide a unique vehicle for collecting dietary information that proposes a playful environment to respondents to increase the reliability of their involvement compared to classical approaches. The planned applications include capture of images before and after food eaten and if the meal is based on processed food, the bare code of the food to estimate the amount and type of food consumed.

Aims. In order to start the experiment as soon as possible, two different applications will be developed. First application will be based on transformation of an on-line dietary questionnaire, initially designed by the “Health across generations” team (Inserm U1018), into a user-friendly mobile (iOs and Android) smartphone app adapted to French modern meal pattern and that can be filled-up in 20mn. Additional questions will be added to capture important variables identified in WP1. End of the first year, a second app will be available, integrating image-assisted methods for more precised dietary assessment. Indeed, recent reviews indicate that images enhance self-report by revealing unreported foods and identify misreporting errors not captured by traditional methods alone (Gemming et al., 2015).
Methods and data. The use of image-based dietary assessment methods shows promise for improving dietary self-report. This is a playful self-administered food record designed to address the burden and human error associated with conventional methods of dietary assessment. Users will take images of foods and beverages at all eating occasions using a mobile telephone or mobile device with an integrated camera, (e.g., Android or iOS based smartphones). Once the images are taken, the images are transferred to a back-end server for automated analysis. The first step in this process is image analysis, i.e., segmentation, feature extraction, and classification, allows for automated food identification (Krizhevsky et al., 2012). Portion size estimation is also automated via segmentation and geometric shape template modeling. When the food provides from processed food, the bare code will be also inputted to establish correlation with the analysis of the WP3. The results will be indexed to provide a detailed diet analysis for use in epidemiologic or intervention studies.
Expected results: M6 we will deliver a first version of the app allowing rapid diet evaluation; M18 We will deliver a V2 version of the app, which will include images-assisted methods and bare code reading to identify and index the food.
Risks. The main difficulty is the regularity and the completeness of the food record.
Collaboration: This task will be based on a close collaboration between epidemiologists, nutritionists and specialists of multimedia information retrieval. We will allocate a budget to subcontract the development of the apps. This app will be used in the work package 5 within the intervention study and the experiments, but more broadly will be available for the entire scientific community willing to measure diet in an efficient way.

Task 4.2. Development of mobile devices to provide tailored dietary recommendations (PIs Jean-Noel Patillon (CEA); Michele Sebag (CNRS-INRIA); Marc Schoenauer(INRIA); O. Allais (Aliss))
The key and challenging feature of the application is to provide recommendations on healthier items that minimize the wellbeing costs of behavior change. Based on the individual segmentation done in WP1 and WP2 as well as the classification of food quality in WP3 as well as the diet of specific individuals, recorded with the app developed in the task 4.1.

Aims. To provide automatically individualized nutritional tips to the app users, according to their dietary patterns and socioeconomic environment.
Methods and data. The application will use machine learning methods to match dietary recommendations with individuals’ preferences determined in a first step. The individuals, using the app developed in task 4.1, records their current food consumption in order to define their food repertory (preferences). On the basis of the clustering approach and the analysis of consumers cost to comply with nutritional guidelines developed in WP2, we will propose a metric of the efforts required to make healthier food choices. These metrics will be used to measure the distance between the current dietary pattern and the nutritional target. Different strategies will be investigated (global effort based on the distance between the gravity center of both the individual dietary pattern and the ideal one, distance between each foodstuff of both clusters, etc.) in order to identify the diet change that minimizes the compliance effort while providing the highest health benefit. From this analysis a set of recommendations will be generated from only change the brand of the food to more restrictive recommendations. Based on the experimentation conducted in WP5, adjustment will be made in the way to define the effort and consequently the recommendations.
Risks. The major difficulty of this task is to find the balance in the recommendations which allows to modify the food habit without transforming the meal in a painful event.
Collaboration. This task will be based on a close collaboration between epidemiologists, economists and specialists of semantic analysis and machine learning. We will allocate a budget to subcontract the development of the apps. This app will be used in the work package 5 within the experiments.