{"id":1439,"date":"2026-01-23T10:16:00","date_gmt":"2026-01-23T10:16:00","guid":{"rendered":"https:\/\/brain.hr\/?p=1439"},"modified":"2026-02-02T13:36:24","modified_gmt":"2026-02-02T13:36:24","slug":"zasto-najpametniji-modeli-grijese-na-najjednostavnijim-zadacima","status":"publish","type":"post","link":"https:\/\/brain.hr\/en\/zasto-najpametniji-modeli-grijese-na-najjednostavnijim-zadacima\/","title":{"rendered":"Why do the \u201csmartest\u201d models make mistakes on the simplest tasks? (2\/6)"},"content":{"rendered":"

In the first part of our series, we explored how the refined knowledge base used by language models is created (you can read that article at this link<\/a>). However, even with such impressive knowledge, users often notice unusual errors in simple operations, such as the model's inability to accurately count the letters in a word or write a term backwards.<\/p>\n\n\n\n

How is it possible that a system that solves complex logical problems makes mistakes at the elementary school level? In his analysis, Andrej Karpathy reveals tokenization as the fundamental text processing mechanism directly responsible for these deviations.<\/p>\n\n\n\n

AI does not perceive letters, but basic language units<\/strong><\/h3>\n\n\n\n

When we read the word \"classroom\", our cognitive system recognizes it as a series of individual letters: U-\u010c-I-O-N-I-C-A. We intuitively assume that artificial intelligence (AI) visualizes text in the same way. However, language models have a completely different \"vision\".<\/p>\n\n\n\n

Models do not interpret individual letters, but see text as a series of numerical codes that we call tokens. Tokens are the basic building blocks of language for AI. They can be whole words, smaller parts of words, or even spaces associated with text.<\/p>\n\n\n\n

A simpler word like \u201cHello\u201d represents a single token (a single numeric code) to the model, while a more complex word like \u201cubiquitous\u201d represents a sequence of exactly three tokens instead of a string of ten letters.<\/p>\n\n\n\n

Imagine trying to count the letters in a word, but someone blindfolded you and simply said, \u201cThese are three dice.\u201d You don\u2019t see the symbols on the dice, only their entirety. That\u2019s exactly what happens to the model. Information about individual letters is often condensed and hidden within the token. When you ask the AI \u200b\u200babout the number of letters, it doesn\u2019t count visually, but tries to statistically guess the answer based on the learned patterns associated with that token.<\/p>\n\n\n\n

The \u201cmental arithmetic\u201d challenge and fixed computational budget<\/strong><\/h3>\n\n\n\n

This problem goes beyond linguistics and goes deep into the realm of mathematics. Karpathy warns that models have a limited amount of \u201ccomputational effort\u201d per token generated.<\/p>\n\n\n\n

When you give a model a task like calculating the product of 324 and 56 and demand an immediate answer, you put it at a disadvantage. The model must perform the entire operation in a single step to generate the next token. This is equivalent to asking a student to solve a complex task purely by heart, in a split second, without the possibility of using paper and pencil.<\/p>\n\n\n\n

Models have a fixed computational \u201cbudget\u201d per word. For complex operations, this budget is often insufficient, which leads to calculation errors if the system is only required to provide the final result without showing the procedure.<\/p>\n\n\n\n

Methodological recommendations: how to optimize working with models?<\/strong><\/h3>\n\n\n\n

Understanding token architecture offers us two key tools for improving accuracy in the educational process:<\/p>\n\n\n\n

    \n
  1. Encouraging \u201cChain of Thought\u201d:<\/strong> Never require a model to only provide the final solution to a complex problem. The instruction \u201cthink step by step\u201d or \u201cshow the procedure\u201d allows the model to use multiple tokens for a single task. This gives it the space to spread its effort over multiple steps, which dramatically increases accuracy.<\/li>\n\n\n\n
  2. Using external tools (Code Interpreter):<\/strong> For tasks that require precise character counting or complex mathematics, it is advisable to use the code writing option (Python). When the model writes code, it stops guessing the solution \u201cby heart\u201d and uses a precise digital calculator that sees numbers and letters exactly as they are written. For example, instead of the query \u201cHow many letters are in the word otolaryngologist?\u201d where the AI \u200b\u200btries to answer \u201cfrom memory\u201d and often makes mistakes due to tokens, try typing the prompt:<\/em>\u201cUse Python (or write code) to count exactly how many letters are in the word otolaryngologist\u201d <\/li>\n<\/ol>\n\n\n\n

    In conclusion, the model\u2019s errors on elementary tasks do not indicate a lack of information, but are a direct consequence of an architecture that is primarily adapted to processing broader concepts, rather than individual characters. <\/p>\n\n\n\n

    In an educational context, it is therefore important to assess whether a particular query requires a structured representation of the procedure (a chain of thought) or the application of specialized tools such as programming code to achieve maximum precision.<\/p>\n\n\n\n

    Source:<\/strong> This article is based on an analysis of Andrej Karpathy's technical lecture: Deep Dive into LLMs like ChatGPT<\/a> i drugi je u nizu \u010dlanaka o dubinskoj arhitekturi jezi\u010dnih modela. U sljede\u0107em nastavku istra\u017eujemo “osnovni model” i poja\u0161njavamo za\u0161to AI u po\u010detnoj fazi ne funkcionira kao asistent, ve\u0107 kao sustav koji nastoji predvidjeti i nastaviti zapo\u010deti tekst imitiraju\u0107i stilove dokumenata na kojima je u\u010dio.<\/p>","protected":false},"excerpt":{"rendered":"

    U prvom nastavku na\u0161eg serijala istra\u017eili smo kako nastaje pro\u010di\u0161\u0107ena baza znanja kojom se slu\u017ee jezi\u010dni modeli (taj \u010dlanak mo\u017eete pro\u010ditati na ovoj poveznici). No, \u010dak i uz takvo impresivno znanje, korisnici \u010desto primje\u0107uju neobi\u010dne pogre\u0161ke u jednostavnim operacijama poput nemogu\u0107nosti modela da to\u010dno prebroji slova u nekoj rije\u010di ili da ispi\u0161e pojam unazad. Kako […]<\/p>\n","protected":false},"author":8,"featured_media":1440,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_et_pb_use_builder":"off","_et_pb_old_content":"","_et_gb_content_width":"","footnotes":""},"categories":[14],"tags":[],"class_list":["post-1439","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-radovi"],"acf":{"radovi_source_url":"","radovi_button_label":"Pro\u010ditajte izvorni rad"},"_links":{"self":[{"href":"https:\/\/brain.hr\/en\/wp-json\/wp\/v2\/posts\/1439","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/brain.hr\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/brain.hr\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/brain.hr\/en\/wp-json\/wp\/v2\/users\/8"}],"replies":[{"embeddable":true,"href":"https:\/\/brain.hr\/en\/wp-json\/wp\/v2\/comments?post=1439"}],"version-history":[{"count":5,"href":"https:\/\/brain.hr\/en\/wp-json\/wp\/v2\/posts\/1439\/revisions"}],"predecessor-version":[{"id":1566,"href":"https:\/\/brain.hr\/en\/wp-json\/wp\/v2\/posts\/1439\/revisions\/1566"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/brain.hr\/en\/wp-json\/wp\/v2\/media\/1440"}],"wp:attachment":[{"href":"https:\/\/brain.hr\/en\/wp-json\/wp\/v2\/media?parent=1439"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/brain.hr\/en\/wp-json\/wp\/v2\/categories?post=1439"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/brain.hr\/en\/wp-json\/wp\/v2\/tags?post=1439"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}