During its Google I/O virtual event, the search giant shared a preview of an AI-powered dermatology assist tool that helps users understand what’s going on with issues related to the body’s largest organ: their skin, hair and nails. Using many of the same techniques that detect diabetic eye disease or lung cancer in CT scans, this tool gets you closer to identifying dermatologic issues — like a rash on your arm that’s bugging you — using your phone’s camera.

How does it work?

Each year, Google sees almost ten billion searches related to skin, nail and hair issues. Two billion people worldwide suffer from dermatologic issues, but there’s a global shortage of specialists. While many people’s first step involves going to a Google Search bar, it can be difficult to describe what you’re seeing on your skin through words alone.

Google’s AI-powered dermatology assist tool is a web-based application that the company hopes to launch as a pilot later this year, to make it easier to figure out what might be going on with your skin. Once you launch the tool, simply use your phone’s camera to take three images of the skin, hair or nail concern from different angles. You’ll then be asked questions about your skin type, how long you’ve had the issue and other symptoms that help the tool narrow down the possibilities. The AI model analyzes this information and draws from its knowledge of 288 conditions to give you a list of possible matching conditions that you can then research further.

For each matching condition, the tool will show dermatologist-reviewed information and answers to commonly asked questions, along with similar matching images from the web. The tool is not intended to provide a diagnosis nor be a substitute for medical advice as many conditions require clinician review, in-person examination, or additional testing like a biopsy. Rather, Google hopes it provides users with access to authoritative information so you can make a more informed decision about your next step.

The development process

Google’s dermatology tool is the culmination of over three years of machine learning research and product development. To date, the company has published several peer-reviewed papers that validate the AI model and more are in the works.

The landmark study, featured in Nature Medicine, debuted Google’s deep learning approach to assessing skin diseases and showed that the AI system can achieve accuracy that is on par with U.S. board-certified dermatologists. The most recent paper in JAMA Network Open demonstrated how non-specialist doctors can use AI-based tools to improve their ability to interpret skin conditions.

The model also accounts for factors like age, sex, race and skin types — from pale skin that does not tan to brown skin that rarely burns. Google developed and fine-tuned its model with de-identified data encompassing around 65,000 images and case data of diagnosed skin conditions, millions of curated skin concern images and thousands of examples of healthy skin — all across different demographics.

Availability

Recently, the AI model that powers the tool successfully passed clinical validation, and the tool has been CE marked as a Class I medical device in the EU. In the coming months, Google plans to build on this work so more people can use this tool to answer questions about common skin issues. If you’re interested in this tool, sign up here to be notified.