Our faces provide us with a lot of early warning signs. In many cases, it can be one of the first clues to our health. It is important to be aware of seemingly “normal” conditions on your face! 

What readily available tool do we have daily access to that can provide us with a simple reading on our physical state? We need a quick and efficient assessment of our faces, so that we can get a deeper insight into our bodies and our health.

To resolve this issue, we can create and program an application that can do this efficiently for us! This app will be able to take a photo of the user’s face, and then detect the potential illnesses that the user may have, based on their facial features. In this project, I will be designing this application. I will create my own machine learning models and implement them into this app!

This app will be extremely simple and will take up less than a few minutes in someone’s day.

 

Also, there's so many people in this world that have diabetes!

Source: Figure prepared by the Library of Parliament using data obtained from Government of Canada, “Canadian Chronic Disease Surveillance System (CCDSS),” Database, accessed 20 December 2021.

This is a chart of numbers in diabetes in 2017. This chart shows us the rates of diabetes per 100,000 people in Canada. The numbers are absolutely terrifying! The highest numbers are in the age range of 65-79. Approximately 1,800 males (for every 100,000 males) between the ages of 65-79 have diabetes, as of 2017. Of course, these numbers have increased the past few years.

 

Before I started to actually develop my project, I conducted initial research about different medical conditions:

Part 1 - Research:

Diabetes:

I decided to start with 1 medical condition. From there, I will continue moving forward with different illnesses and research on the varying symptoms for each illness. 

Diabetes is an extremely common chronic illness. As of late 2023, more than half a billion people (of all ages) have this condition. Diabetes occurs when one’s body cannot properly regulate blood sugar levels. When the food we eat gets digested in our bodies, a large portion of the food gets converted into glucose (sugars) and enters our bloodstreams. When the quantity of glucose in our blood (also called our blood sugar) increases, the pancreas starts generating insulin. Insulin is a vital hormone that enters our bloodstreams and signals our body cells. This signal tells the body cells to make their membranes selectively permeable for the glucose to pass through. 

When someone has diabetes, the pancreas either cannot generate enough insulin for the body cells, or the cells in the body gradually became resistant to the insulin.

Facial Symptoms of Diabetes:

1. Xanthelasma palpebrarum - Xanthelasma palpebrarum is a relatively harmless yellow growth that appears between the nose and the edge of the eyes, and can spread onto the eyelid as well. These patches normally are cholesterol deposits that build up underneath the skin. Xanthelasma is mainly a symptom of diabetes and high blood sugar, but it can also be a symptom of hyperlipidemia, which is when the body contains high cholesterol amounts. Generally, cholesterol is a waxy substance that our body mainly creates, but we also consume it in the foods we eat, like eggs. The yellow buildup between the eyelid and nose is the cholesterol buildup (below). Scientists are unaware of why cholesterol in the body builds up underneath the skin to form xanthelasma palpebrarum.

Taken from Medical library - MedPix 

 

2.  Granuloma annulare - Granuloma annulare is another possible symptom one may experience if they have diabetes. It is a skin condition where multiple reddish bumps cluster around a darker and sunken red center. Although the cause for the formation of granuloma annulare is completely unknown, researchers and scientists have recognized that granuloma annulare often appears when one has diabetes. Many also say it can be considered a symptom of thyroid disease. Below, you can see the reddish clusters on the cheeks of the subject. It is believed that granuloma annulare is caused because of skin inflammation.

In the picture, you can see how the inner center of the cheeks is darker, while the outer portion is a lighter red.

Taken from Medical Library - DermNetNS

 

Now, using this information, I can begin developing my application and move on to the first stage of development.

 

Part 2 - App Development:

First, I would like to show the resources I have used for the development portion of this project. All of these resources have helped me create my application.

• Intel Core i7 iMac
• iPhone 11 (iOS version 17.2.1)
• Xcode - downloaded
• Supervisely (ML model annotations) - https://supervisely.com
• Roboflow (ML model annotations) - https://roboflow.com
• RectLabel (ML model annotations) - https://rectlabel.com
• Create ML - downloaded
• Autodesk Sketchbook (design application logo)
• FreeFormatter (XML file to JSON file converter)
• Medical copyright-free media libraries for ML model:
  • Open-i
  • MedPix
  • Wikimedia Commons
• LucidChart (I built a flow diagram of my app using this).

I initially used Supervisely and Roboflow for my machine learning (ML) model annotations. Unfortunately, both tools did not provide me with proper JSON file annotation format. I ended up using RectLabel, an Apple-based application. Instead of providing me with JSON formatted files, it provided me with XML files. I then used FreeFormatter to convert the XML files to JSON files. My full development process is attached in my logbook.

Camera App:

For my project, I decided to use Xcode to build my interface, so it could run smoothly on a variety of Apple devices. Xcode is Apple’s integrated application development environment. Xcode primarily uses the programming language of Swift. I installed it on my iMac and created a new project, which I named “Face Doctor.” My model (which I discuss in depth later) is integrated into my camera app!

First, I had to build the base for the app, which mainly consists of the camera. To start, I decided to import features that would be essential to develop the camera. 

I import 3 modules:

• SwiftUI - SwiftUI is a framework for all Apple devices.
• CoreML - To be used later on for my ML model.
• Vision - Vision is mainly used for facial detection scenarios; it goes hand-in-hand with CoreML. A few examples of scenarios used with the Vision framework are image registration, barcode recognition, facial detection, etc.

Here is a photo of some of my code in my CameraView file in my navigation tab in my Xcode project.

 

 

Model Training:

I named my ML model: "DiabetesClassifier". I also had to collect lots of images before I started to train my model. During training, it will use these images to figure out how to classify symptoms such as "granuloma annulare" and "xanthelasma palpebrarum."

Above, you can see the count of my labels/identifiers. There is a count of 37 for the label of "xanthelasma palpebarum" and there is a count of 33 for my identifier of "granuloma annulare."

From here, I started to train my machine learning model to detect both of these classes: "granuloma annulare" and "xanthelasma palpebrarum." The training results and performance is elaborated on in the "Analysis" section of this platform.

 

App - How it Works:

Below is a step-by-step process of how FaceDoctor works.

 

                     

 

From the above images, you can see the structure of my application. It goes through a series of steps:

1. Opening the app - click
2. Once the app is opened, there is a button in the center, the "Open Camera" button. To start the diabetes detection, you have to open the camera.
3. Once you click the button, you go to a new page. This screen has a camera, which you can take pictures with.
4. For this purpose, I took a picture of a photo of a man with the diabetes symptom "granuloma annulare." It is discussed above.
5. Once I chose the button "Use photo," I go to a new screen that displays the photo I took. Right underneath, there is another button which says "Analyze."
6. Once clicked, the "Analyze" button leads us to our object detection results, generated by our machine learning model. The result is displayed. In this case, the model has detected granuloma annulare with a confidence of 76%. Right below, there is feedback given to the user. 

 

I have also represented FaceDoctor in a flow chart, as shown below. This is similar to pseudocodes (used in programming). I created this flow chart with the help of Lucid Chart (website). I have referenced this website above.

Training My App:

This is the graph that was being shown to me while my machine learning model was being trained. This was showing me how well the model was performing. it started with a training accuracy of approximately 65%, but over time it increased to 100% training accuracy. I had 11 iterations in my ML model training. In machine learning, the "iterations" are the adjustments that the model makes to become more and more accurate at detecting the object. 

In the graph, you also see a label by the name of "Validation Accuracy." When I have a high validation accuracy, the machine learning model is telling me that my image classifier is accurately classifying data that it has not been trained with. In this case, my training ended with a validation accuracy of 78.6%. That's quite good!

Performance of App:

After my training was done (shown by the graph above), I had to test the performance of the app. This is a crucial part of the development process, because this helps you understand how well your app works and if there are any majors issues that need to be debugged and solved.

 

I actually looked at the performance of my app directly in Create ML. Generally, the accuracy of my app was 96%. From the picture above, CreateML told me what images were the hardest to classify, and what my ML model incorrectly predicted. 

 

 

 

After looking in more depth at some of the incorrect classifications, I was able to understand in more detail, how my app was performing. Above, you can see how my machine learning model incorrectly detected granuloma annulare with a confidence percentage of 51%. It also had a confidence of 49% that it was xanthelasma palpebrarum, which was the actual identifer/label. 

My app is currently performing quite well! I am impressed by the accuracy it has.

To conclude this project, applications like “FaceDoctor” are extremely vital for our society. It is important to have daily access to tools that can help you understand the condition of your health! It’s crucial to know what is occurring inside your body! FaceDoctor will benefit many people. I hope to continue working on this project and developing it more, to detect more illnesses!

This application has finally finished its first level of development. This model detects a few symptoms of diabetes, specifically granuloma annulare, and xanthelasma palpebrarum. Although there is still more progress to make, I hope to continue working on this model. 

This app actually works quite well! It can accurately detect whichever illness is shown to the camera. Although I aimed to create an object detection model, I ended up creating an image classification ML model. 

My project is currently in a beta version of development. I will be expanding my idea very soon. I hope to be able to detect more illnesses, using similar machine learning models. 

To conclude this project, applications like “FaceDoctor” are extremely vital for our society. It is important to have daily access to tools that can help you understand the condition of your health! It’s crucial to know what is occurring inside your body! FaceDoctor will benefit many people. I hope to continue working on this project and developing it more, to detect more illnesses!

CDC. “What is diabetes? | CDC.” Centers for Disease Control and Prevention, https://www.cdc.gov/diabetes/basics/diabetes.html. Accessed 25 January 2024.

Cleveland Clinic. “Insulin: What It Is, What It Does, How To Take It & Side Effects.” Cleveland Clinic, 17 January 2024, https://my.clevelandclinic.org/health/articles/22601-insulin. Accessed 25 January 2024.

Cleveland Clinic. “Xanthelasma: What It Is, Causes and Treatment.” Cleveland Clinic, 25 June 2022, https://my.clevelandclinic.org/health/diseases/23385-xanthelasma. Accessed 25 January 2024.

Corliss, Julie, and Scott Leighton. “In your eyes: Clues to heart disease risk?” Harvard Health, 1 September 2021, https://www.health.harvard.edu/heart-health/in-your-eyes-clues-to-heart-disease-risk. Accessed 25 January 2024.

Diabetes Canada. “What is diabetes?” Diabetes Canada, https://www.diabetes.ca/about-diabetes/what-is-diabetes. Accessed 25 January 2024.

IHME. “Global diabetes cases to soar from 529 million to 1.3 billion by 2050.” Institute for Health Metrics and Evaluation, 22 June 2023, https://www.healthdata.org/news-events/newsroom/news-releases/global-diabetes-cases-soar-529-million-13-billion-2050. Accessed 25 January 2024.

John Hopkins Medicine. “Granuloma Annulare.” Johns Hopkins Medicine, https://www.hopkinsmedicine.org/health/conditions-and-diseases/granuloma-annulare. Accessed 25 January 2024.

Ludmann, Paula. “Diabetes: 10 warning signs that can appear on your skin.” American Academy of Dermatology, 4 January 2024, https://www.aad.org/public/diseases/a-z/diabetes-warning-signs. Accessed 25 January 2024.

MD Anderson Cancer Center. “9 Top Skin Cancer Symptoms & Signs.” MD Anderson Cancer Center, https://www.mdanderson.org/cancer-types/skin-cancer/skin-cancer-symptoms.html. Accessed 25 January 2024.

MedLine Plus. “Blood Sugar | Blood Glucose | Diabetes.” MedlinePlus, 10 April 2023, https://medlineplus.gov/bloodglucose.html. Accessed 25 January 2024.

MedPix. MedPix, http://medpix.nlm.nih.gov. Accessed 25 January 2024.

Mekala, Srinivas, et al. “Face to BMI Prediction Using Machine Learning Algorithms.” IJRASET, 2023, https://www.ijraset.com/research-paper/face-to-bmi-prediction-using-machine-learning-algorithms. Accessed 25 January 2024.

Open-i. “Open-i.” NIH Open Access Biomedical Image Search Engine, http://openi.nlm.nih.gov. Accessed 25 January 2024.

Pearce, Lara. “Clues to your Health Could be Written all Over your Face.” YouTube: Home, 9 November 2017, https://www.huffpost.com/archive/au/entry/clues-to-your-health-could-be-written-all-over-your-face_au_5cd38382e4b0acea9501fd0d. Accessed 25 January 2024.

The Physicians of The Doctors, thehealthy.com. “9 signs of disease that are written all over your face.” Reader's Digest Asia, https://www.rdasia.com/uncategorized/9-signs-of-disease-that-are-written-all-over-your-face. Accessed 25 January 2024.

return2health. “Mirror, Mirror on the Wall: Facial Clues about Your Health.” return2.health, 22 March 2017, https://return2.health/articles/mirror-mirror-wall-facial-clues-health. Accessed 25 January 2024.

Sheikh, Zilpah. “What Does Your Face Say About Your Health?” WebMD, 24 August 2023, https://www.webmd.com/skin-problems-and-treatments/ss/slideshow-face-your-health. Accessed 25 January 2024.

Spratt, Susan E. “9 Signs of Disease That Are Written All Over Your Face.” The Healthy, 1 September 2019, https://www.thehealthy.com/beauty/face-body-care/signs-disease-face/. Accessed 25 January 2024.

Watson, Bronwen. “What clues is your face giving away about your health?” MyMed.com, https://www.mymed.com/health-wellness/interesting-health-info/what-does-your-face-say-about-your-health/what-clues-is-your-face-giving-away-about-your-health. Accessed 25 January 2024.

World Health Organization. “Diabetes.” YouTube: Home, 9 November 2017, https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=&ved=2ahUKEwiz6a3etMSDAxWAMjQIHfNmDhcQFnoECCIQAQ&url=https%3A%2F%2Fwww.who.int%2Fhealth-topics%2Fdiabetes&usg=AOvVaw00PoveLnLuSdWFkOSSfb04&opi=89978449. Accessed 25 January 2024.

 

 

I would also like to acknowledge all of the resources I have used for the development portion of this project (machine learning and programming in Xcode). All of these resources have helped me create my application. Most of these are also acknowledged in the "Method" section.

• Intel Core i7 iMac
• iPhone 11 (iOS version 17.2.1)
• Xcode - downloaded
• YouTube
• CamScanner (Logbook pictures)
• Supervisely (ML model annotations) - https://supervisely.com
• Roboflow (ML model annotations) - https://roboflow.com
• RectLabel (ML model annotations) - https://rectlabel.com
• Create ML - downloaded
• Autodesk Sketchbook (design application logo)
• FreeFormatter (XML file to JSON file converter)
• Medical copyright-free media libraries for ML model:
  • Open-i
  • MedPix
  • Wikimedia Commons

Thank you to the 2024 CYSF Committee! The Calgary Youth Science Fair is a brilliant way for young students to showcase their work, and I truly appreciate the opportunity I have received to participate in this event.

I would also like to thank all of the teachers at Fairview Junior High that have guided me throughout my project and have contributed to its success! All the teachers at Fairview school have contributed to my growth.

Finally, I would like to express my heartfelt gratitude to my family and friends, who have all strongly supported me throughout the development of my application. 

Once again, thank you to all!