The article \u201cProof or Bluff? Evaluating Large Language Models at the 2025 American Mathematical Olympiad\u201d assesses the ability of current state-of-the-art large language models (LLMs) to solve six challenging proof-based problems from the prestigious 2025 American Mathematical Olympiad (USAMO). Unlike previous mathematical benchmarks that focused on numerical answers, this study assesses the models\u2019 ability to generate detailed, rigorous mathematical proofs similar to those required in real-world advanced mathematics competitions.<\/p>\n\n\n\n
This research tested how well the latest AI language models can solve some of the most difficult mathematical problems given to top high school students in the United States. It found that these AI systems are not yet good at writing detailed and correct mathematical proofs, often making errors in logic or calculations. They typically do not realize when they are wrong and cannot be fully trusted without human verification. This shows that while AI has made great progress, it still needs significant improvement before it can handle the most challenging types of mathematical reasoning that humans do.<\/p>\n\n\n\n
Overall poor performance: All LLMs tested struggled significantly on the USAMO problems, scoring an average of below 5 out of 42 points, reflecting a large gap in their mathematical reasoning and proof generation skills despite their success in previous competitions focused on numerical answers.<\/p>\n\n\n\n
Common failure modes: The models frequently made incorrect logical steps, unjustified assumptions, lacked creativity in exploring alternative solution strategies, and occasionally made algebraic or arithmetic errors. They tended to generalize incorrectly from small examples and sometimes assumed the required final answers even when this was not justified.<\/p>\n\n\n\n
Models consistently confident but wrong: Unlike humans who know when they are wrong, LLMs confidently claimed solutions even when they are incorrect, highlighting the challenges in trusting AI for rigorous mathematical problems without human validation.<\/p>\n\n\n\n
Mixed quality of solutions: Some models (such as O3-MINI and O1-PRO) generally generated clearer and more structured evidence, while others produced chaotic, disorganized answers.<\/p>\n\n\n\n
Difficulties with automated grading: Attempts to use other LLMs to automatically grade solutions have failed, as raters often overestimated the correctness of incorrect answers.<\/p>","protected":false},"excerpt":{"rendered":"
\u010clanak “Dokaz ili blef? Evaluacija modela velikih jezika na Ameri\u010dkoj matemati\u010dkoj olimpijadi 2025.” procjenjuje sposobnost trenutnih najsuvremenijih modela velikih jezika (LLM) da rije\u0161e \u0161est izazovnih problema temeljenih na dokazima s presti\u017ene Ameri\u010dke matemati\u010dke olimpijade (USAMO) 2025. Za razliku od prethodnih matemati\u010dkih mjerila usmjerenih na numeri\u010dke odgovore, ova studija procjenjuje sposobnost modela da generiraju detaljne, rigorozne […]<\/p>\n","protected":false},"author":3,"featured_media":550,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_et_pb_use_builder":"off","_et_pb_old_content":"\n
\u010clanak \"Dokaz ili blef? Evaluacija modela velikih jezika na Ameri\u010dkoj matemati\u010dkoj olimpijadi 2025.\" procjenjuje sposobnost trenutnih najsuvremenijih modela velikih jezika (LLM) da rije\u0161e \u0161est izazovnih problema temeljenih na dokazima s presti\u017ene Ameri\u010dke matemati\u010dke olimpijade (USAMO) 2025. Za razliku od prethodnih matemati\u010dkih mjerila usmjerenih na numeri\u010dke odgovore, ova studija procjenjuje sposobnost modela da generiraju detaljne, rigorozne matemati\u010dke dokaze sli\u010dne onima koji se zahtijevaju u stvarnim naprednim matemati\u010dkim natjecanjima.<\/p>\n\n\n\n
Ovo istra\u017eivanje testiralo je koliko dobro najnoviji AI jezi\u010dni modeli mogu rije\u0161iti neke od najte\u017eih matemati\u010dkih problema zadanih najboljim srednjo\u0161kolcima u SAD-u. Otkrilo je da ovi AI sustavi jo\u0161 nisu dobri u pisanju detaljnih i ispravnih matemati\u010dkih dokaza, \u010desto grije\u0161e\u0107i u logici ili izra\u010dunima. Obi\u010dno ne shva\u0107aju kada su u krivu i ne mogu im se u potpunosti vjerovati bez ljudske provjere. To pokazuje da iako je AI postigao veliki napredak, jo\u0161 uvijek treba zna\u010dajno pobolj\u0161anje prije nego \u0161to se mo\u017ee nositi s najizazovnijim vrstama matemati\u010dkog zaklju\u010divanja koje ljudi rade.<\/p>\n\n\n\n
Ukupno lo\u0161i rezultati: Svi testirani LLM-ovi zna\u010dajno su se mu\u010dili s USAMO problemima, posti\u017eu\u0107i prosje\u010dne rezultate ispod 5 od 42 boda, \u0161to odra\u017eava veliki jaz u njihovim vje\u0161tinama matemati\u010dkog zaklju\u010divanja i generiranja dokaza unato\u010d uspjehu na ranijim natjecanjima usmjerenima na numeri\u010dke odgovore.<\/p>\n\n\n\n
Uobi\u010dajeni na\u010dini neuspjeha: Modeli su \u010desto radili pogre\u0161ne logi\u010dke korake, neopravdane pretpostavke, nedostajala im je kreativnost u istra\u017eivanju alternativnih strategija rje\u0161enja i povremeno su radili algebarske ili aritmeti\u010dke pogre\u0161ke. Skloni su pogre\u0161nom generaliziranju iz malih primjera i ponekad su pretpostavljali potrebne kona\u010dne odgovore \u010dak i kada to nije bilo opravdano.<\/p>\n\n\n\n
Modeli dosljedno samouvjereni, ali u krivu: Za razliku od ljudi koji znaju kada ne uspiju, LLM-ovi su samouvjereno tvrdili rje\u0161enja \u010dak i kada su neto\u010dna, isti\u010du\u0107i izazove u povjerenju u AI za rigorozne matemati\u010dke probleme bez ljudske validacije.<\/p>\n\n\n\n
Mje\u0161ovita kvaliteta rje\u0161enja: Neki modeli (poput O3-MINI i O1-PRO) op\u0107enito su generirali jasnije i strukturiranije dokaze, dok su drugi proizvodili kaoti\u010dne, neorganizirane odgovore.<\/p>\n\n\n\n
Pote\u0161ko\u0107e s automatskim ocjenjivanjem: Poku\u0161aji kori\u0161tenja drugih LLM-ova za automatsko ocjenjivanje rje\u0161enja nisu uspjeli, jer su ocjenjiva\u010di \u010desto precjenjivali ispravnost pogre\u0161nih odgovora.<\/p>\n","_et_gb_content_width":"","footnotes":""},"categories":[14],"tags":[],"class_list":["post-549","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-radovi"],"acf":{"radovi_source_url":"https:\/\/arxiv.org\/pdf\/2503.21934v1","radovi_button_label":"Pro\u010ditajte izvorni rad"},"_links":{"self":[{"href":"https:\/\/brain.hr\/en\/wp-json\/wp\/v2\/posts\/549","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/brain.hr\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/brain.hr\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/brain.hr\/en\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/brain.hr\/en\/wp-json\/wp\/v2\/comments?post=549"}],"version-history":[{"count":5,"href":"https:\/\/brain.hr\/en\/wp-json\/wp\/v2\/posts\/549\/revisions"}],"predecessor-version":[{"id":788,"href":"https:\/\/brain.hr\/en\/wp-json\/wp\/v2\/posts\/549\/revisions\/788"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/brain.hr\/en\/wp-json\/wp\/v2\/media\/550"}],"wp:attachment":[{"href":"https:\/\/brain.hr\/en\/wp-json\/wp\/v2\/media?parent=549"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/brain.hr\/en\/wp-json\/wp\/v2\/categories?post=549"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/brain.hr\/en\/wp-json\/wp\/v2\/tags?post=549"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}