University of Illinois Extension
How accurate are photo-based plant identification apps?

[February 18, 2022]    Plant identification can be incredibly challenging for beginning botanists or even experienced plant people. There are tons of terms to learn and understand just to start identifying plants by leaves and other plant parts. Throw in a leafless winter, and it makes for a difficult skill to develop with confidence.

In recent years, photo-based smart phone applications (apps) have been developed to aid with plant identification in the field. These apps boast their accuracy and many claim to provide identification for a wide array of species simply based on an image of the foliage. Do these apps work? And, if so, how accurate can they be?

Research on Phone App Accuracy

A research paper published in the most recent edition of the scientific journal, “Arboriculture and Urban Forestry”, explored the accuracy of photo-based plant identification apps. Researchers with the Rutgers Urban Forestry Program designed and implement this study which assessed the apps’ abilities to identify photos of 55 common street trees and native forest trees in New Jersey.

For their study, experienced arborists photographed both bark and leaves of trees and submitted their photos to the 6 most commonly downloaded plant identification apps based on downloads from the Apple App Store® in June of 2020. The specific apps assessed include: iNaturalist™, PlantNet™, Leafsnap™, PlantSnap™, PictureThis™, and Plant Identification™.

At least 4 images of both bark and leaves for each tree species were submitted to each app and researchers observed and recorded the results. The apps use photo recognition software programmed to identify leaves and bark based on the common characteristics of these plant parts.

Do Phone Apps Really Work?

To me, it is extraordinary that these apps are even able to process visual data and generate near accurate results. As a trained botanist, I know that 2 leaves for the same tree can often vary widely. While there is typically a recognizable pattern and some distinctive characteristics common across all leaves on the same plant, certain species (typically within the same genus) can have very similar leaf characteristics, making it possible for a single plant to have individual leaves in its canopy which are distinctive of its true species and other leaves that would lead you to believe it’s another, similar-looking species.

A good example of confusing similarity would be white oak (Quercus alba), bur oak (Quercus macrocarpa) and swamp white oak (Quercus bicolor), who’s leaves can have a very similar or very different pattern of lobes based on where you look in the canopy of an individual plant. Shade leaves, or leaves on the interior of the canopy, are often larger with more distinct lobes than leaves in full sun.

Knowledgeable botanists are aware of these nuances and take care to assess several leaves at differing locations within the canopy. However, beginners often make the common mistake of looking at one low-hanging leaf, which may or may not be the best example to observe. Due to instances like this, I have always discounted the use of plant identification apps and have not recommended them to beginners. However, some promising results were uncovered by Rutgers.

What does the research say?

Each phone app makes suggestions, often several, for species-level identification of the plant picture in question. The Rutgers study found that while species-level identification by leaf pictures was not always the most accurate (83.9% to 40.9% accurate), across all the apps observed, genus-level identification by leaves was pretty good, reporting accuracies from 97.3% to 71.8%. Across all apps and all species, identification by bark pictures alone was not nearly as accurate as identification by leaves.

For identification by leaves, the most accurate two apps were PictureThis™ (97.3% accurate to genus, 83.9% to species) and iNaturalist™ (92.3% accurate to genus, 69.6% to species). These results suggest that phone apps can really help beginners rapidly arrive at a to genus-level identification. With the aid of a good guidebook, beginners can quickly reach species-level determinations since the possibilities are narrowed down by the phone app.

App User Communities Boosts Accuracy with Crowd-Sourced Identification

While the Rutgers study presents some fascinating data, this research did not assess the community aspect that some of these apps provide. Several of the apps in this study also offer an option where users can ask the community of other app users to identify their plant photo. I have found that community responses on these apps are typically highly accurate to species-level and often come from experts. So, when beginners can combine phone apps with other tools, such as community responses and the use of guidebooks or other reference materials, these applications have promising potential.

References:
Schmidt el al. 2022. An analysis of the accuracy of photo-based plant identification applications on fifty-five tree species. Arboriculture & Urban Forestry. 48(1): 27-43.

[SOURCE: Ryan Pankau, Horticulture Educator, University of Illinois Extension]