Warning GPT-4o: DON'T translate to Chinese (MIT)
Рет қаралды 1,749
2 ай бұрынMIT (Massachusetts Institute of Technology) Tech review reports on massive problems with GPT-4o regarding the Chinese language, discovering heavy tokenizer "pollution".
Warning if you use this AI to translate business correspondence into Chinese, since MIT reports on a heavy data pollution with Chinese tokens.
Currently double check the translation results of GPT-4o with an independent source, especially your business communication to your Chinese partners. Otherwise you might find your company and yourself in a strange business situation ....
All rights w/ authors:
GPT-4o’s Chinese token-training data is polluted by spam and porn websites
www.technologyreview.com/2024...
#airesearch
#gpt4o
@Quaintcy 2 ай бұрын
I suspect teaming up with microsoft and using their search data is a mistake
@mshonle Ай бұрын
It’s time we move to something more advanced and curated than BPE for tokenizers. The “just add more data, scale!” crowd seems to have a major blind spot here. The “bitter lesson” applies to neural architectures and generalizing it to other big data tasks is, to put it nicely, a failed experiment.
@justindressler5992 Ай бұрын
So the new model is meant to be better at multilingual it was well of the bigger gains. But I fails to translate for 1.2 billion people, classic.
@propeacemindfortress 2 ай бұрын
crap in crap out...
@dragonbone1020 Ай бұрын
I just asked it to write a business letter in Chinese and it was all fine.
@code4AI Ай бұрын
Thanks for the update. Running multiple generations of models in parallel on multiple clusters helps with internal switching.
Infinity Circus
Рет қаралды 7 МЛН
Garena Free Fire Global
Рет қаралды 42 МЛН
Special Effects Funny 44
Рет қаралды 3,1 МЛН
notebook-31
Рет қаралды 115 М.