Res. Plant Dis > Volume 30(1); 2024 > Article
Hue, Kim, Lee, Choi, Sim, Jeon, Ahn, Han, and Kim: Artificial Intelligence Plant Doctor: Plant Disease Diagnosis Using GPT4-vision


Integrated pest management is essential for controlling plant diseases that reduce crop yields. Rapid diagnosis is crucial for effective management in the event of an outbreak to identify the cause and minimize damage. Diagnosis methods range from indirect visual observation, which can be subjective and inaccurate, to machine learning and deep learning predictions that may suffer from biased data. Direct molecular-based methods, while accurate, are complex and time-consuming. However, the development of large multimodal models, like GPT-4, combines image recognition with natural language processing for more accurate diagnostic information. This study introduces GPT-4-based system for diagnosing plant diseases utilizing a detailed knowledge base with 1,420 host plants, 2,462 pathogens, and 37,467 pesticide instances from the official plant disease and pesticide registries of Korea. The AI plant doctor offers interactive advice on diagnosis, control methods, and pesticide use for diseases in Korea and is accessible at

Plant diseases are health problems in plants primarily caused by various pathogens such as fungi, bacteria, viruses, and nematodes. While it is difficult to pinpoint the exact time when humans began to recognize that plants can get sick, the awareness of plant diseases has existed in human history for a long time. The scientific understanding that plant diseases are caused by biological pathogens began to develop in 1863 when Anton de Bary proved that potato blight is caused by a biological pathogen (Matta, 2010). Traditionally, plant diseases were diagnosed visually, but this method heavily relied on experience and had limitations in differentiating complex and diverse pathogens. Serological tests are also widely used for diagnosing plant diseases, but they are primarily limited to viruses. With the advancement of molecular biology in the 20th century, methods for diagnosing by analyzing DNA/RNA were developed (Martin et al., 2000). This method is very accurate and can distinguish different pathogens that show the same symptoms, but it is time-consuming, costly, and requires specialized skills and equipment. Recently, to overcome these limitations, biochemical diagnostic methods using volatile compounds or the introduction of spectroscopic techniques have been adopted (Farber et al., 2019; Riu et al., 2022).
Meanwhile, with the advancement of computer vision technology, various machine learning/deep learning algorithms have been developed to analyze plant disease images and diagnose them quickly (Jackulin and Murugavalli, 2022). These methods, trained with large volumes of plant disease photo data, provide consistent and accurate diagnoses and can process vast amounts of data quickly, making them useful for detecting plant diseases in large-scale farms. However, they require high-quality training data and can suffer in diagnostic performance if the data is insufficient or biased. Moreover, diagnosing based solely on images has limitations in detecting characteristics of host plants or complex environmental factors that may not be fully apparent in the image data. Nevertheless, the introduction of the latest artificial intelligence (AI) technology, especially large multimodal models (LMMs), has helped overcome these limitations in the fields of life sciences and dentistry (Huang et al., 2023; Wang et al., 2023). In plant disease diagnosis, models like GPT-4 can analyze text and image data simultaneously, enabling deeper contextual understanding and comprehensive diagnostics (Qing et al., 2023). Through this, AI can go beyond simple image classification to comprehensively analyze various data, such as environmental factors, climate change, and the physiological state of plants, thereby enhancing the accuracy and reliability of plant disease diagnosis.
This study has developed an AI plant doctor system that recognizes plants and lesions and diagnoses plant diseases using GPT-4-vision, an LMM capable of text and image recognition (Fig. 1). The system was developed using OpenAI's application programming interface (API), runs on the Streamlit package, and is programmed in Python 3. One method to utilize GPT as an AI plant doctor involves employing prompt engineering to limit the tasks the GPT model can perform, thereby recognizing GPT as an expert in plant diseases. These prompts, coded as instructions, are used to create user-defined GPT models and endow GPT with unique characteristics. To prevent the extraction of prompts, they are set to be private. However, it is disclosed that the prompts are designed to discourage excessive use of chemical pesticides and emphasize sustainable practices. Furthermore, in the case of queries unrelated to plant diseases, users are prompted to ask questions related to plant diseases.
Fig. 1.
The concept of Artificial Intelligence Plant Doctor incorporated with a knowledge base. The GPT4-vision-based AI plant doctor receives input as an image and text. The AI is incorporated with a knowledge base, including lists of plant diseases and pesticides. For the output, the GPT offers specific diagnoses, causal explanations, intervention methods, prevention tips, and further resources for the provided information.
To differentiate from the standard GPT-4 model, it utilizes a vast knowledge base including 1,420 host plant taxa and 2,462 pathogen taxa listed in the 6.1 edition of the Korean Plant Disease Index (Kim et al., 2023) and 37,467 cases of pesticides available for each plant recorded in the Rural Development Administration's Pesticide Safety Information System. This data is stored in the knowledge base in the form of comma-separated values (CSV) files in a table format. The plant disease list includes the Korean, English, and scientific names of the host plants, along with the Korean and English disease names and scientific names of the pathogens. The pesticide list includes the plant name, disease name, pesticide name, generic name, trade name, mammalian toxicity, fish toxicity, mode of action, dilution rate, application rate, optimal application timing, application method, safety application timing, safe application frequency, formulation, and company name. In conjunction with prompts, the knowledge base serves as a method to endow the user-defined GPT with unique characteristics, enabling GPT to respond to user queries within a limited scope of knowledge. When a user inputs a host plant, limiting the pathogens that can infect the host through a list of plant diseases acts as a safeguard to enhance GPT's accuracy and reduce the probability of incorrect answers.
To enhance the accuracy of diagnosis, the system allows text input of the host plant's name, description of symptoms, and environmental factors. For instance, in the case of rice blast, which recently became a major problem in South Korea, it typically shows acute, watery dark brown to chronic fusiform symptoms, and occurs frequently in low temperatures and high humidity during summer (Song et al., 2022). Thus, if a user provides text about the shape or color of lesions, and the crop's growing environment, along with the host plant, the system analyzes both image and text data to comprehensively assess the type of pathogen. Based on this information, the system can analyze images and related text data to recognize complex patterns and accurately identify various types of plant diseases (Fig. 2). As a result, the AI plant doctor informs the user about the description of the lesions and the pathogens that could potentially infect the plant based on the photo queried by the user in a conversational manner, and provides advice on pesticide treatment in response to further inquiries from the user. Additionally, for user convenience, it provides links to plant disease-related websites such as the National Crop Pest Management System, Pesticide Safety Information System, and Korean Plant Disease Index.
Fig. 2.
The interface of AI Plant Doctor website translated into English (from Korean). The interface is designed to assist users in diagnosing and managing plant health issues by combining user-submitted photos and text information with artifical intelligence analysis.
Despite the significant potential of the AI plant doctor system developed in this study, there are several important issues to consider. Firstly, unlike traditional machine learning/deep learning models, the pre-trained GPT model lacks standardized methods for measuring accuracy and reliability. This poses considerable challenges in objectively evaluating and benchmarking the model's performance. To mitigate this drawback, this research attempted to enhance the model's accuracy and reliability by adding a knowledge base. However, this approach also necessitates a rich and detailed text database describing lesions and the environmental conditions of plant disease occurrence. Collecting and managing such data is a time-consuming and resource-intensive task, but it will improve the system's performance. Furthermore, there are questions about the system's ability to distinguish between plant diseases and non-biological stresses, such as nutrient deficiency, cold damage, or heat stress. Since various stress responses in plants can be similar, accurately differentiating them is a complex issue. This requires a high degree of accuracy and situational awareness, and with the current level of technology, such distinctions may not be perfect. These issues represent challenges faced by this research and will be a primary focus in future research and development efforts.
In conclusion, the fusion of AI technology and plant disease diagnosis represents a significant advancement in modern agriculture and plant pathology research. This innovation enables farmers and researchers to diagnose plant diseases quickly, providing insights necessary to respond to the spread of plant diseases. The progress in AI is impacting not only plant disease research but also sustainable agricultural practices, crop improvement, and the enhancement of global food security. Researchers can now use large-scale data for more precise diagnostics and understand the complex patterns and correlations necessary to comprehend the emergence and spread of plant diseases. This opens a new era in the field of agriculture, holding the potential to offer a better future for humanity and the environment through continuous technological innovation and research.


Conflicts of Interest

No potential conflict of interest relevant to this article was reported.


This research was supported by the MSIT (Ministry of Science and ICT), Korea, under the National Program for Excellence in SW) supervised by the IITP (Institute of Information & Communications Technology Planning & Evaluation) in 2024 (2023-0-00028).


Farber, C., Mahnke, M., Sanchez, L. and Kurouski, D. 2019. Advanced spectroscopic techniques for plant disease diagnostics. A review. TrAC, Trends Anal. Chem. 118: 43-49.
Huang, H., Zheng, O., Wang, D., Yin, J., Wang, Z., Ding, S. et al. 2023. ChatGPT for shaping the future of dentistry: the potential of multimodal large language model. Int. J. Oral Sci. 15: 29.
crossref pmid pmc pdf
Jackulin, C. and Murugavalli, S. 2022. A comprehensive review on detection of plant disease using machine learning and deep learning approaches. Meas.: Sens. 24: 100441.
Kim, S. H., Choi, J., Choi, Y. J., Park, B. Y., Lee, S. H., Kim, G. H. et al. 2023. Introduction of list of plant diseases in Korea 6.1st edition (2023 revised version). Res. Plant Dis. 29: 331-344. (In Korean)
crossref pdf
Martin, R. R., James, D. and Lévesque, C. A. 2000. Impacts of molecular diagnostic technologies on plant disease management. Annu. Rev. Phytopathol. 38: 207-239.
crossref pmid
Matta, C. 2010. Spontaneous generation and disease causation: Anton de Bary's experiments with Phytophthora infestans and late blight of potato. J. Hist. Biol. 43: 459-491.
crossref pmid pdf
Qing, J., Deng, X., Lan, Y. and Li, Z. 2023. GPT-aided diagnosis on agricultural image based on a new light YOLOPC. Comput. Elec-tron. Agric. 213: 108168.
Riu, M., Son, J. S., Oh, S. K. and Ryu, C. M. 2022. Aromatic agriculture: volatile compound-based plant disease diagnosis and crop protection. Res. Plant Dis. 28: 1-18. (In Korean)
crossref pdf
Song, S., Chung, H., Kim, K. H. and Kim, K. T. 2022. Analysis of rice blast outbreaks in Korea through text mining. Res. Plant Dis. 28: 113-121. (In Korean)
crossref pdf
Wang, D. Q., Feng, L. Y., Ye, J. G., Zou, J. G. and Zheng, Y. F. 2023. Ac-celerating the integration of ChatGPT and other large‐scale AI models into biomedical research and healthcare. MedComm Future Med. 2: e43.
METRICS Graph View
  • 0 Crossref
  •  0 Scopus 
  • 489 View
  • 41 Download

Ki-Tae Kim

Related articles

Editorial Office
Rm,904 (New Bldg.) The Korean Science & Technology Center 22, Teheran-ro 7-gil, Gangnam-gu, Seoul 06130, Korea
Tel: +82-2-557-9360    Fax: +82-2-557-9361    E-mail:                

Copyright © 2024 by The Korean Society of Plant Pathology.

Developed in M2PI

Close layer
prev next