Purpose: To evaluate the performance of Automated Machine Learning (AutoML) models in diagnosing ocular toxoplasmosis (OT) and classifying its inflammatory activity from fundus photographs. Design: Cross-sectional study. Methods: Fundus photographs from OT patients in two Colombian referral centers and an open-source OT database were classified into active OT, inactive OT, and no OT. Image quality assessment excluded images with artifacts but included blurry images due to vitritis. Photos were uploaded to Amazon Web Services S3 and Google Cloud Bucket. Two models were developed on each platform: a binary model (active/inactive OT vs. no OT) and a multiclass model (active OT, inactive OT, and no OT). Datasets were split into 70percent-flag-change for training, 20percent-flag-change for testing, and 10percent-flag-change for validation. Sensitivity, specificity, precision, accuracy, F1-score, the area under the precision-recall curve (AUPRC), and Cohen's Kappa were calculated for each platform and model. An external validation using an open-source image bank was performed. Results: The binary model on AWS showed a sensitivity of 0.97, specificity of 0.98, and AUPRC of 1.00, while the Google Cloud binary model had a sensitivity of 0.82, specificity of 0.91, and AUPRC of 0.91. The multiclass model on AWS achieved an F1 score of 0.88, with Cohen's Kappa of 0.81, while the Google Cloud model reached an F1 score of 0.88, with Cohen's Kappa of 0.82. External validation for Google Cloud achieved an accuracy of 87.5percent-flag-change and 80.3percent-flag-change in the binary and multiclass models, respectively. Conclusions: AutoML is a powerful tool for diagnosing OT and classifying inflammatory activity, potentially guiding diagnosis and treatment decisions.
