(To mitigate the pedantry: Given it’s a rather dispassionate response in the context of a provocation, it is probably a very weak “care” though. Just because it’s nonzero doesn’t mean it’s significant.)
This might be happening because of the ‘elegant’ (incredibly hacky) way openai encodes multiple languages into their models. Instead of using all character sets, they use a modulo operator on each character, to make all Unicode characters represented by a small range of values. On the back end, it somehow detects which language is being spoken, and uses that character set for the response. Seeing as the last line seems to be the same mathematical expression as what you asked, my guess is that your equation just happened to perfectly match some sentence that would make sense in the weird language.
Can’t find the exact source–I’m on mobile right now–but the code for the gpt-2 encoder uses a utf-8 to unicode look up table to shrink the vocab size. github.com/openai/gpt-2/blob/master/…/encoder.py
There are bindings in java and c++, but python is the industry standard for AI. The libraries for machine learning are actually written in c++, but use python language bindings. Python doesn’t tend to slow things down since machine learning is gpu-bound anyway. There are also library specific programming languages which urges the user to make pythonic code that can be compiled into c++.
I suppose it’s conceivable that there’s a bug in converting between different representations of Unicode, but I’m not buying and of this “detected which language is being spoken” nonsense or the use of character sets. It would just use Unicode.
The modulo idea makes absolutely no sense, as LLMs use tokens, not characters, and there’s soooooo many tokens. It would make no sense to make those tokens ambiguous.
I completely agree that it’s a stupid way of doing things, but it is how openai reduced the vocab size of gpt-2 & gpt-3. As far as I know–I have only read the comments in the source code– the conversion is done as a preprocessing step. Here’s the code to gpt-2: github.com/openai/gpt-2/blob/master/…/encoder.py I did apparently make a mistake, as the vocab reduction is done through a lut instead of a simple mod.
Gotta remember they were trained off of the internet. Which is to say the largest body of people loadly professing the opinions are fact and refusing to say otherwise.
There is no single person responsible for Cyrillic script. It is mostly believed to be created by mixing and changing Greek and Glagolic scripts by the scholars of Preslav Literary School, which was indeed in Bulgaria. After a while, Peter the Great changed it a lot. And then Stalin stomped out almost all the deviations in the usage of the script.
The last part is mostly why it is considered Russian. A lot of languages suffered because of Moscow just forcing them to use the version of Cyrillic that Russians were using.
Cyrillic is literally greek+glagolitic and it was partly a diplomatic creation of the Eastern Roman Empire(aka Byzantine Empire), in order to bring the slavs culturally closer to them.
Russians have nothing to do with it, other than them claiming they are the continuation of Eastern Roman Empire, something which is kinda laughable but whatever dont let your dreams be dreams.