Tag: llm

We present a systematic evaluation of modern multimodal large language models (LLMs) for the classification of mean-motion and secular resonances from images of resonant arguments. Four benchmark datasets (RB-TEST, RB-PILOT, RB-SMALL, RB-FULL) were constructed to cover clear, ambiguous, and transient cases, with both binary and three-class outputs. Using standardized prompts (a full prompt for large models and a simplified variant for small models that cannot process complex instructions), we tested flagship commercial models, large open-source models, and small locally runnable models. Commercial LLMs reach F1=100% on simple cases and up to 94% on the three-class RB-SMALL dataset, while the best open-source models also reach 100% on unambiguous cases and 76% on the complex ones. On the full binary benchmark, open-source models approach commercial performance (F1≈90–96%). Most errors occur in transient and resonance-sticking regimes. The results show that LLMs can perform resonance classification at levels comparable to those of classical or machine-learning methods without training or fine-tuning, and that even small open-source models achieve practically useful accuracy. The released benchmarks establish a reproducible standard for evaluating LLMs on dynamical astronomy tasks.

This study explores how well various machine learning classifiers can identify mean-motion resonances in the main belt using supervised learning. The most popular classifiers are assessed: k-Nearest Neighbours, Decision Tree, Gradient Boosting, AdaBoost, Random Forest, and Naïve Bayes. In contrast to previous studies that often relied on default ML configurations, this research conducts a detailed investigation, fine-tuning, and testing of each classifier across various parameters. The results show that simpler models, especially k-Nearest Neighbours and Decision Tree, perform better than more complex ones, particularly in terms of F1 scores. The paper provides guides on selecting features, parameters, and training set sizes for optimal classifier performance and outlines a method for developing effective machine-learning models for asteroid classification.

Classical machine learning has been actively utilized in astronomy to address various challenges, including predicting orbital stability, classifying asteroids, galaxies, and other objects, and analyzing images. However, the emerging trend in artificial intelligence involves the use of large language models such as GPT-4 and ChatGPT. These models are trained on a large corpus of text and can perform a wide range of natural language processing tasks, including text generation, translation, summarization, and classification. Surprisingly, these capabilities present significant potential for application in astronomy. This paper demonstrates how the new model gpt-4-vision-preview can analyze visual patterns and accurately classify asteroids as resonant or nonresonant with high accuracy. This process requires no training, fine-tuning, or coding beyond writing the appropriate prompt in natural language. Moreover, this approach can be extended to other common problems within astronomy.