Swipe Right: a Comparison of Accuracy of Plant Identification Apps for Toxic Plants (2024)

Abstract

Keywords: Toxic plants, Smartphone applications, Plant identification

Introduction

Plant exposures remain in the top 25 reported exposures to US poison centers, occurring most commonly in children < 5years old, and the identity of the plant is often unknown [1, 2]. Plant ingestions rarely cause serious poisoning, as children account for the majority of these exposures often with small exploratory ingestions that do not result in toxicity. More serious outcomes can occur, however, when the toxins of poisonous plants are concentrated by being brewed into a tea, made into a slurry, or are ingested intentionally in suicide attempts [1–4].

Toxic plant exposures contribute to public anxiety by posing several challenges for healthcare providers and those exposed. Emergency departments often lack staff able to identify toxic plants involved in exposure, even if the specimen is brought in to the hospital. When reported to poison control centers, specialists are often limited by over-the-phone verbal descriptions of plants ingested and symptoms described. If a photo is provided, the provider can use descriptive plant morphology information to perform an internet search (which can be inefficient or yield inaccurate results), or emergently contact a botanist, which may not be feasible in real-time. In the home, parents may feel helpless and worried when their child ingests an unknown plant, uncertain how to proceed with limited tools to use for plant identification. Various methods to aid in identification, including faxing of images of exposed plants and internet searches describing the characteristics of the plant, have been employed with varying success [5]. There is ample room for improvement in the arena of toxic plant identification for use when quickly evaluating acutely exposed patients.

The near-ubiquity of smartphones and ongoing advances in technology have enabled the development of plant identification apps. The apps currently available have been developed by entrepreneurs and botanists for research purposes; the crowd-based formatting provides a large sample size not constrained to geographic areas. Plant identification applications (ID apps) on smartphones have been improving in accuracy and increasing in availability for home identification of plant species [6–9]. These apps have been proposed for assisting with identification of toxic plants involved in exposures; however, none of these have been validated for this use.

We compared the three highest rated smartphone plant ID apps for their ability to identify toxic plants by photographing a sample of commonly encountered toxic plants and recording each app’s accuracy. We compared results of each of three different plant ID apps to determine which was the most accurate across different species.

Methods

We used three popular iPhone and Android plant ID apps, PictureThis (PT), PlantSnap (PS), and Pl@ntNet (PN), to identify commonly encountered toxic plants in gardens and public spaces. The most popular apps were determined using the crowd-based rating scores in the Apple App Store. The top 3 highest rated apps were downloaded onto personal iPhone 8’s, owned by the researchers. Androids were not used in this study. Each app was purchased independently by the researchers; there was no funding for this study, and it was exempt from IRB review. All apps use plant morphology to compare the photograph taken of a plant with an encyclopedic plant database for identification. PictureThis was created by Glority Global Group Ltd., a company that creates several picture-based smartphone identification apps; it claims 95% accuracy in plant identification (source: picturethisai.com). PlantSnap was created by an American entrepreneur and utilizes an online encyclopedic database of plants for identification (source: plantsnap.com). Pl@ntNet is a self-described “citizen-science project on plant biodiversity” whose origin is with a French group interested in open-access botany education and biodiversity research (source: plantnet.org/en). All apps use artificial intelligence to continuously improve based on new entries by subscribers. At the time of our data collection, the cost of each app for unlimited use without advertisem*nts was $1.99 per month, $0.99 per month, and free, respectively.

A prioriwe calculated we needed at least 10 samples per group to detect a 40% difference between the accuracy of two apps. Apps were used to photograph the entire plant, leaves, and flowers (if applicable) of ≥ 10 different plants for each species. In the case of plants with no flowers, only leaves were photographed, or if the plant had another defining feature such as a seed pod or berry, we photographed that organ and recorded under the flower category for simplicity (for example, for R. communis, we used the spiked seed pod instead of flower, and for P. americana, we used berry sprig instead of flower). Pl@ntNet did not allow for a whole plant identification option (this app has the user specify what portion of the plant to identify, i.e., leaf, flower, bark, berry). Plants were photographed in their natural environment, and backgrounds and composition were not standardized in order to replicate a real-world use scenario. When many plants of one species were present, more than 10 individual plants were photographed due to availability.

We gathered a convenience sample of toxic plants to include in our study from commonly encountered toxic plants in Southern California from September 2019 to June 2020, and several other common species found during travel within the USA. Plants were identified in community spaces, encountered daily by most of the inhabitants of the areas. We ultimately identified 17 different toxic species for inclusion in the study. Two toxicologists performed plant identification, and inter-researcher reliability was confirmed with interrater reliability of 98% on a random sample of 41 plants (40/41 agreement; 95% CI 87.4 to 100). Correct plant identification by the toxicologists was confirmed by visual inspection of photographs of plants performed by a PhD professor of Botany (author SM). This study focused on plant morphology identification and geographic location, as that is the most likely route of identification in the case of plant poisonings.

For each plant species, a composite score of correct identification was compared among apps. Composite score was calculated by adding up all 3 components of identification (for Pl@ntNet, composite score was flower plus leaf scores). We calculated proportions of correct plant identifications with 95% confidence intervals (denoted in square brackets []). Plants identified in community parks, gardens, and open spaces are listed in Table ​Table11.

Results

Composite results for each app and plant species are listed in Table ​Table11 and depicted in Fig.1. PT and PN correctly identified more toxic plants among the plants photographed than PS. For the composite score, PT had the best performance with 10/17 (59% [36 to 78]) plant species identified 100% correctly, regardless of leaves, flowers, or both photographed, as opposed to 8/17 (47% [26 to 69]) for PN and 1/17 (5.9% [1.1 to 27]) for PS. PT performed significantly better than the other two apps for identifying Podocarpus macrophyllus (100% [72 to 100] vs PS 40% [17 to 69] and PN 20% [6 to 51]) and much better (although both not statistically significant) for Toxicodendron diversilobum (100% [72 to 100] vs PS 30% [11 to 60] and PN 50% [24 to 76]). PT and PN both performed better than PS for Nerium oleander (statistically significant (S)), Cascabela thevetia, Brugmansia suaveolens (S), Datura wrightii (S), Ricinus communis (S), Nicotiana glauca (S), Phytolacca americana (S), Lantana camara (S), Digitalis purpurea (S), and Conium maculatum (S). All apps did not significantly differ in accuracy identifying Euphorbia tirucalli, Toxicodendron radicans, Rhododendron indicum, Aglaonema commutatum, and Dieffenbachia seguine although PT most often performed best, followed by PN (not statistically significant). Of note, many of the dumbcane were identified by all apps as Chinese or Philippine evergreen (genus Aglaonema), which has comparable toxicity with dumbcane, and therefore was counted as an affirmation.

At worst, PT performed at 84% accuracy for L. camara (74 to 94), but for all other plants, this app identified species with over 90% accuracy. PN was the second best performing app with widely variable 20 to 100% accuracy for plants identified. PS performed the worst with 100% accuracy on only one plant species and all others ranging from 23 to 85% accuracy.

When comparing total observations regardless of species or plant organ specificity, PictureThis with 440/457 (96% [94 to 98]) observations correctly identified outperformed Pl@ntNet with 307/336 (91% [88 to 94]) correct identifications and PlantSnap with 255/457 (56% [51 to 60]) correct.

Discussion

We found that the three apps evaluated in our study were able to identify toxic plants, with varying degrees of accuracy. PictureThis was the most accurate and most consistent across all plant species tested, although at best, it 100% accurately identified only 10/17 toxic plant species. In a field where incorrect identification of a toxic plant could have dire consequences, these apps are not yet ready for use as a stand-alone tool for use in acute unknown plant poisonings. Our study demonstrates that these apps may be used with caution by the public, by medical personnel in consultation with poison control centers, or by poison control center staff or medical toxicologists themselves to assist in toxic plant identification when an encounter or ingestion occurs. This study was not powered to validate the apps for overall toxic plant identification accuracy, but they were tested in their current state as available for purchase by the public. Further studies need to be conducted to determine overall accuracy of these apps for toxic plant identification and to determine if their performance is better than that of a medical toxicologist or poison control center for toxic plant identification. The focus of our study is for use of these apps in the clinical setting, but home identification of toxic plants would be useful for the lay public as well. It is important for early adopters of plant identification apps for the identification of toxic plants to understand that there are limitations to this use of the apps, and that further improvements in the apps need to occur to place full trust on their results.

Compared with prior published data on toxic plant identification, these apps in their current state may already perform better than currently utilized methods of toxic plant identification in the clinical setting. The most accurate and precise method of plant species identification is via genetic identification using plant material, but this method is not feasible in situations in which an acutely poisoned patient arrives to the emergency department. Contact with a botanist could increase likelihood of correct plant identification, but this, too, is often not feasible in a clinical setting. Remote plant identification via sharing of home cell phone photos has been done successfully in the past [10], but this relies on manual comparison with internet image search engine results for matching, a time consuming task that is fraught with difficulty and inconsistency. Over the last two decades, computer-based applications have been increasingly put to use for plant identification. A study published in 2010 demonstrated probabilistic neural networks based on leaf morphology used for computerized identification of over 30 different plant species that had a 91% average recognition accuracy [11]. Comparatively, at the time of this study, PictureThis reports 95% accuracy for general plant identification on its website (not referenced). Improving smart phone technology could eventually bridge this gap of plant identification, but as our study demonstrates, these apps need to be validated with a large sample of toxic plants prior to wide adoption of their use for toxic plant exposures.

Medical toxicologists and poison control centers are often consulted to assist with identification of plants ingested by patients, but their performance at accurate plant identification in the absence of a toxidrome is possibly highly variable. Poison centers and medical toxicologists may be able to identify commonly occurring toxic plants in their geographical region, but to the authors’ knowledge, the accuracy of their plant identification skills has not been published. Once a toxidrome has manifested in the clinical setting, this may improve a toxicologist’s accuracy at culprit plant identification, but at that point, when significant toxicity has manifested, treatment may be started too late and any attempt at gastrointestinal decontamination may have reduced efficacy. In their current state, with the results interpreted with caution, these apps may aid toxicologists’ toxic plant identification accuracy. These may have more applicability in the future as smart phone technology and the crowd-sourced plant libraries continue to improve.

There are some limitations to morphology-based plant identification, which these apps rely on for identification. DNA analysis of plant matter is the most accurate method of plant identification, but this is not feasible in most clinical settings. In lieu of analyzing plant species’ genetic material, computer-based plant identification aids have focused on using plant morphology, such as leaf and petal shape, to identify genera and species of plants. Relying on morphology alone, some plants may not be correctly identified. In the case of T. diversilobum and T. radicans, misidentification of the two species for the other does not result in significant variation in treatment: both have the same clinical effects in susceptible individuals. In the case of other plants with grouped leaves-of-three, non-toxic plants may be mistaken for Toxicodendron sp. This mistake would not be likely to have significant consequences, unless leading to unnecessary pre- or overtreatment. Conversely, in the case of differentiating C. maculatum from Daucus carota, more severe consequences could manifest as a result of plant morphology misidentification, including death. With these examples in mind, although not ready for use as stand-alone plant identification, the use of these apps may assist in real-time clinical plant identification, particularly when used with caution for ruling-in possible toxic plant ingestion.

To our knowledge, no published study has examined the accuracy of medical toxicologist or poison control center staff at identifying toxic plants. Several studies have attempted to bridge the gap between available technology and human plant identification. In one study evaluating the efficacy of training courses utilizing intensive visual learning computer-based applications, the average class of veterinary student learners could identify correctly 93% of a collection of 40 poisonous species [12]. Although this percentage is better than the best app in our study, such training took months of in-person use of the visual learning applications.

All three apps in our study were easy to use without training and could correctly identify plants photographed with variable accuracy. Adequate lighting and simple backgrounds improved plant identification accuracy. Foreground obstruction made correct plant identification less likely. User variability likely contributes to each app’s variable accuracy in this study and in use by individuals in the community.

Limitations

There are several limitations to our study. This study used only iPhone 8 due to convenience. We expect comparable results with Android phones of a similar generation. Older model smartphones may not perform as well, and newer models may perform better. This study was not designed for evaluation of different technology models.

We did not standardize composition or background for each photograph. We attempted to replicate real-world use of the app, in which a photograph may be taken in variable compositions. The lack of standardization likely contributed to the performance of each app in our small study, although the use of only two photographers likely limited the extent of variability encountered.

Greater sampling of a larger number of toxic plants including common and obscure plants should be tested prior to recommending widespread adoption of plant identification apps for identifying toxic plants. As plant identification apps are ever-improving due to their utilization of machine-learning technology, these apps should continue to become better at identifying toxic plants in the future. Lastly, of note, we did not verify our plant identification through the gold standard of genetic identification as described earlier [13–16]. We did only identify commonly encountered toxic plants that we feel confident identifying and that we encounter often. Although the medical toxicologists in consultation with a PhD Botanist feel confident in their correct identification of the plants involved in this study, there is a chance of misidentification within the sample.

Although one of the apps, Pl@ntNet, was available for a free download, cost and availability of a smartphone at home and of the apps themselves may be a limitation to widespread use of this technology for toxic plant identification. However, the apps are able to identify plants in a photo regardless if it is taken by that particular phone, or remotely shared with that phone at a later time. The ability of these apps to identify plants in photos taken at any time increases the utility, especially if the above limitations of cost or smartphone availability are faced.

Conclusion

Identification of unknown toxic plants can cause challenges for medical personnel, medical toxicologists, and parents of young children, and until recently, the public and healthcare providers have had limited tools to assist in plant identification. We demonstrated that publicly available plant identification apps can identify toxic plants with varying accuracy. Of the apps tested, PictureThis most often correctly identified toxic plants among the plants photographed, followed by Pl@ntNet. A plant identification app may be a useful tool for the lay public, medical clinicians, and poison centers to assist in identifying toxic plants, although more thorough validation of these apps for this purpose and technological advances are required before widespread use can be advocated. Results of plant identification apps for identification of toxic plants can be accurate but should be interpreted with caution and the data used in consultation with experts in the field.

Authors’ Contribution

Authors JO and CT contributed to the study conception and design. Material preparation, data collection, and analysis were performed by JO and CT. SM performed plant identity verification. The manuscript was written by JO and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Funding Information

None.

Compliance with Ethical Standards

Footnotes

References