Paper Information | SBC BOOKS

Fill in your paper information

Language (*)

Title (*)

Keywords (*)

Abstract (*)

Solving Ordinary Differential Equations (ODEs) requires mathematical modeling and reasoning skills that remain challenging for large language models (LLMs). This paper evaluates five LLMs, DeepSeek-R1, Claude-Haiku-4.5, Llama-3.3-70B-Instruct, Gemini-2.5-Pro, and GPT-5.2, using ten first-order ODE application problems and a task involving analogous problem generation. We propose a multi-dimensional framework assessing mathematical modeling, Python code generation, solution accuracy, and the generation of
analogous problems. Results show strong modeling performance across all models (96%–98%), demonstrating their ability to translate natural language descriptions into mathematical formulations; when generating analogous problems, scores ranged from 85% to 100%, enabling direct performance comparisons against each model’s own baseline. However, performance declines in code generation tasks, exposing limitations in multi-step reasoning and execution. Claude-Haiku-4.5 achieves the highest overall score (84%), while Llama-3.3-70B-Instruct performs best in code generation. GPT-5.2 records the lowest overall performance. These findings reveal a gap between conceptual understanding and reliable end-to-end problem solving, highlighting the importance of human oversight and hybrid symbolic-numerical approaches.

Pages (*)

File Link

English Information

Title

Keywords

Abstract

(*) To change the order drag the item to the new position.

Authors

#	Name
1	Domingos Samanjata(kasams@usp.br)
2	René Santin(renevs@usp.br)
3	Ricardo Marcacini(ricardo.marcacini@usp.br)
4	Solange Rezende(solange@icmc.usp.br)

(*) To change the order drag the item to the new position.

Reference

#	Reference
1	Domingos Samanjata Ph.D. Candidate, Institute of Mathematical and Computer Sciences (ICMC), University of São Paulo (USP), Brazil Email: kasams@usp.br
2	René V. Santin Ph.D. Candidate, Institute of Mathematical and Computer Sciences (ICMC), University of São Paulo (USP), Brazil Email: renevs@usp.br
3	Ricardo R. Marcacini Associate Professor, Institute of Mathematical and Computer Sciences (ICMC), University of São Paulo (USP), Brazil Ph.D. Co Supervisor Email: ricardo.marcacini@usp.br
4	Solange O. Rezende Full Professor, Institute of Mathematical and Computer Sciences (ICMC), University of São Paulo (USP), Brazil Ph.D. Supervisor Email: solange@icmc.usp.br

Paper Registration

Fill in your paper information

English Information

Authors

Reference