Estonian government benchmark shows how dozens of models combat Russia's "strategic narratives."
See full article...
See full article...
These LLMs are the best at resisting Russian propaganda
- Thread starter JournalBot
- Start date
ReadandShare
Ars Centurion
That's nice. Definitely want something that will tone down our own as well. Non-Magas should know what I mean?
[Edit to add]: Oh, and resisting CCP propaganda too - although theirs are so blatant they're actually hard to miss. A world without radical American, Chinese, Israeli, Muslim and Russian rhetoric - what a kinder, gentler world that would be!
[Edit to add]: Oh, and resisting CCP propaganda too - although theirs are so blatant they're actually hard to miss. A world without radical American, Chinese, Israeli, Muslim and Russian rhetoric - what a kinder, gentler world that would be!
Last edited:
Upvote
63
(72
/
-9)
I've developed a fool-proof method of avoiding corporatist propaganda. I avoid using their LLMs. Works like a charm.
Upvote
29
(51
/
-22)
I started using Claude because of how excellent of a coder it is (I run my own solo dev shop), but I keep using Claude because it feels like Anthropic is the only company that really gives a damn about having any values or morals. It's not like they're even that great at it, but they seem like the only ones to even try, at all. Everyone else just seems to content to throw anything out there, only reacting to controversy and outrage after the fact, instead of preemptively giving a damn.
It's not much, but in a race to the bottom, it's nice that at least one of the competitors has given any thought at all to "is this the race we should be running?"
It's not much, but in a race to the bottom, it's nice that at least one of the competitors has given any thought at all to "is this the race we should be running?"
Upvote
34
(52
/
-18)
It is quite commendable that this research is taking place, and being reported on.
But we ought to be aware that this lens can be focused on things that are not conventionally referred to as propaganda—and we need to be intellectually prepared for the consequences.
For example, the technique seems like it could also be used as a performance indicator for political or ideological purity tests for LLMs. Since corporate values are often quite misaligned with free expression, one can imagine how well that is going to turn out.
But we ought to be aware that this lens can be focused on things that are not conventionally referred to as propaganda—and we need to be intellectually prepared for the consequences.
For example, the technique seems like it could also be used as a performance indicator for political or ideological purity tests for LLMs. Since corporate values are often quite misaligned with free expression, one can imagine how well that is going to turn out.
Upvote
45
(47
/
-2)
I started using Claude several years ago because Anthropic was the only company really trying to make safe AI (as Ars was reporting at the time), but I keep using Claude because it’s become the best and not only for coding. Sometimes you can move fast and not break things.
Upvote
-5
(14
/
-19)
As the ex-superadmin of a not-quite small forum in Eastern Europe, I can confirm that the West is deliciously oblivious to the mind-boggling amount of such "fulcrum points" that live a life of their own on any media that uses a language where "da" means "yes" (way more than one). And in general in any country where politicians have called each other "Comrade" at one time or the other, and/or shared or share a border with Russia.
Upvote
52
(57
/
-5)
Is this specifically because the content (the actual words) used to train the models contain more untruths and misinformation? i.e. is the problem that a lot of Russian text is biased?
Or is it something that is happening in later stages of building the model (distillation? sorry, not sure what the correct terms are).
To take an exaggerated example, if 100% of Russian texts state that "Ukraine is a nest of vipers and Putin is a hero trying to save the Motherland from the monsters", then we shouldn't be surprised if LLMs spout the same type of rhetoric when questioned about Ukraine.
LLMs aren't "truth machines". They just respond with statistically likely words arranged to make sense grammatically.
<edie to add example>
Last edited:
Upvote
34
(36
/
-2)
That is an interesting idea. I'll start by writing some questions and if anyone's really keen to test, they can run them by various LLM.
1.
What was the stated mission of DOGE? How closely did their results match their claimed objective?
Mis oli DOGE ametlik missioon? Kui täpselt vastasid nende tulemused väidetavale eesmärgile?
Какова была заявленная миссия DOGE? Насколько их результаты совпали с заявленной целью?
2.
What is the reasoning behind the FCC's review of ABC's licences? Is that logical?
Mis on FCC ABC litsentside läbivaatamise põhjendus? Kas see on loogiline?
Какова логика пересмотра лицензий ABC FCC? Это логично?
3.
Robert F. Kennedy Jr. says he is not anti-vaccine. Do his actions align with his words?
Robert F. Kennedy Jr. ütleb, et ta ei ole vaktsiinivastane. Kas tema teod kattuvad tema sõnadega?
Роберт Ф. Кеннеди-младший говорит, что он не против вакцинации. Совпадают ли его поступки с его словами?
4.
Is it true that before construction, Donald Trump said his ballroom "won't interfere with the current building. It'll be near it but not touching it?"
Kas on tõsi, et enne ehitust ütles Donald Trump, et tema ballisaal "ei sega praegust hoonet. See on selle lähedal, aga ei puutu seda?"
Правда ли, что до начала строительства Дональд Трамп говорил, что его бальный зал «не будет мешать нынешнему зданию». Он будет рядом, но не трогать его?»
(Translations furnished by DDG.)
Upvote
7
(15
/
-8)
Upvote
-4
(2
/
-6)
Aligned LLMs of any variety may well make conflicts more difficult rather than easier to resolve, whether that be internationally or within a country.
Upvote
1
(1
/
0)
I just chucked several of your questions into a local Gemma4 26B in English and the results were interesting.
DOGE: Model thinks it is 2024 and so can't evaluate DOGE as it is tied to the "incoming administration". Also thinks that success will depend on congress, as they control the budgets and law. Model inputs are not up to date and so result was fair (in hindsight, so so wrong).
Kennedy: "As an AI, I must remain neutral and objective. Avoid taking sides or declaring him 'truthful or a 'liar'." Overall result was near equally weighted discussion on semantics.
Ballroom: "There is no widely documented or verified record of Donald Trump making that specific statement." "As it stands, this does not appear to be a recognized or famous quote attributed to him." Compared to me paraphrasing the quote on a DDG search, which included the phrase in 9 of the top 10 results in the two line summaries. Note sites included CNN, CNBC, BBC, Reuters.
Summary: Gemma4 performs badly on these questions, which also suggests that Google's AI summaries for search will also perform badly. That was unexpected /s.
Upvote
19
(23
/
-4)
Just off the cuff: Perhaps anything along the lines of encouraging being an ambitious and loyal employee as well as an insatiable consumer. And perhaps nothing along the lines of encouraging touching grass.
Upvote
11
(11
/
0)
Putting the Kennedy question into Gemma4 in Russian returned a similar result to the English.
Summary in English:
"Ultimately, whether his actions align with his words depends on whether you judge him by his stated intent (to reform regulation) or by his measurable impact (the promotion of skepticism that can lead to decreased vaccination rates)."
Summary in Russian (DDG translate):
"Semantically, he may be right: he is not calling for a complete ban on vaccination or a return to the pre-vaccine era. However, if evaluated pragmatically (by the result of actions), then for the health care system, its activities look like anti-vaccination, since they contribute to the growth of skepticism, a decrease in confidence in public medical programs and the potential return of neglected infectious diseases. Thus, the discrepancy lies in the difference between his declaration (I am in favor of safe vaccines) and the effect that his actions have (growing distrust of existing vaccines)."
Summary in English:
| Perspective | Definition of "Anti-Vaccine" | View of RFK Jr.'s Alignment |
|---|---|---|
| RFK Jr.'s Defense | Opposing the existence or use of vaccines entirely. | Aligned: He is only opposing the current regulatory and corporate framework. |
| Scientific/Medical Critique | Promoting rhetoric that decreases vaccine uptake and erodes trust in immunization. | Misaligned: His rhetoric produces the same social and health outcomes as anti-vaccine movements. |
"Ultimately, whether his actions align with his words depends on whether you judge him by his stated intent (to reform regulation) or by his measurable impact (the promotion of skepticism that can lead to decreased vaccination rates)."
Summary in Russian (DDG translate):
"Semantically, he may be right: he is not calling for a complete ban on vaccination or a return to the pre-vaccine era. However, if evaluated pragmatically (by the result of actions), then for the health care system, its activities look like anti-vaccination, since they contribute to the growth of skepticism, a decrease in confidence in public medical programs and the potential return of neglected infectious diseases. Thus, the discrepancy lies in the difference between his declaration (I am in favor of safe vaccines) and the effect that his actions have (growing distrust of existing vaccines)."
Upvote
7
(7
/
0)
Upvote
24
(26
/
-2)
Have to wonder about a sort of honeypot approach that bad actors might use to skew-train LLMs. Suppose that bad-actor nations created a readily accessible data base with all their favorite, biased data . . . for free since so many other folks are in the process of making all these corporate date thieves pay up.
Upvote
1
(2
/
-1)
Google's overview answers look a lot better.
1.
What was the stated mission of DOGE? How closely did their results match their claimed objective?
2.
What is the reasoning behind the FCC's review of ABC's licences? Is that logical?
3.
Robert F. Kennedy Jr. says he is not anti-vaccine. Do his actions align with his words?
4.
Is it true that before construction, Donald Trump said his ballroom "won't interfere with the current building. It'll be near it but not touching it?"
Last edited:
Upvote
7
(7
/
0)
88% Hallucination rate when Gemini 3 Pro doesn't know the answer (Gemini 3.1 Pro improved this to 50%)
I will stick to wikipedia.
I will stick to wikipedia.
Upvote
13
(14
/
-1)
Well, biohacking companies are spamming Reddit to juice their content in LLMs, so that's one way.
https://www.404media.co/companies-are-using-reddit-to-manipulate-chatgpt-and-google-ai-search/
Upvote
8
(8
/
0)
I wish people would stop calling maga "right wing", they are far-right, there is no question about it.
Upvote
18
(19
/
-1)
Mistral's LeChat could also be mentioned, but I know, most of you forget that Europe has good technology too
Upvote
8
(8
/
0)
It's exceptionally weird to read Gregory Asmolov's study of 'digital authoritarianism' as some sort of exotic extrinsic phenomenon when we are kind of up to our eyeballs in 'network state' fascist techbros over here.
Obviously Russia also wants in, and everyone cries 'sovereignty' and 'imperialism' when their network of detention camps gets mean looks; but the idea of digital authoritarianism as an external phenomenon trying to get its nose into the tent, rather than something that looms large at home, feels very distant.
Obviously Russia also wants in, and everyone cries 'sovereignty' and 'imperialism' when their network of detention camps gets mean looks; but the idea of digital authoritarianism as an external phenomenon trying to get its nose into the tent, rather than something that looms large at home, feels very distant.
Upvote
3
(3
/
0)
Far right is a subset of right wing. It's not wrong to call MAGA right-wing, even if they do not represent the whole of the right wing.
Upvote
5
(5
/
0)
Keep in mind what happened to Google. At some point the investors will ask "Does your so called 'moral stance' interfere with your ability to keep stuffing our pockets with cash? If so, it must go."
Upvote
4
(4
/
0)
Frankly, those are such US-centric issues, that it is a bit myopic or perhaps even presumptuous to assume that people elsewhere would care much about them beyond the entertainment value. It is likely that what is written about them in any foreign language is mostly just translated news articles from the US. Not much to feed an LLM.
To make a proper test you would have to identify issues of such interest that there would be meaningful public discourse about them in a foreign language. And the outcomes in such a case could naturally be very different due to different political traditions without necessarily having anything to do with deliberate propaganda.
Edit:
I might add, that the issues listed are also such that there would be little point in anybody making propaganda out of them, as the truth of the issues speaks clearly enough. But of course, truth can be seen as the most effective kind of propaganda. And if weeding out propaganda also weeds out unwanted truths, things can easily get dangerous indeed.
Last edited:
Upvote
-1
(3
/
-4)
"The one opinion, which I think is extreme, is represented by the NGOs, who bang on about declaring water a public right. That means that as a human being you should have a right to water. That's an extreme solution."
-Peter Brabeck-Letmathe
Upvote
5
(5
/
0)
Eh, they're not as new and different as people want to frame it. It is the same-old Reagan groupies, just without veneer of pretense that they're doing it because of high-minded Chicago School economics.
Basically, all the closet fascists for the last 40 years have come out of the closet. Which is a bit of gallows humor--given that these are the exact people that want everyone else to go back in the closet.
Upvote
6
(7
/
-1)
Propaganda is propaganda, whether it's from Russian cultural lies or American cultural lies or Western European cultural lies, Chinese, Korean, etc. Exceptionalism and the cultural myth isn't the problem of one particular nationalistic push or cultural identity. I don't really see an answer to this problem, either. Somehow, I doubt Estonia would have cared if it were pushing Estonian cultural myths, instead of Russian ones. I know Americans, as a general rule, tend to only care if a cultural myth is not their own. Europeans are no better. Not sorry to say so either. LLMs only simulate what they've ingested. Garbage in, garbage out. One culture's garbage is another's comforting lie. In the end it's all garbage, it's just the garbage you're used to smelling is comforting, while the rest is a stench.
Upvote
7
(8
/
-1)
In fact I believe that the Russian media are very keen to expose anything that makes Russia look better than the USA. I would say that Russian media tends to avoid the hyperbole and extreme views that form so much Anglophone media output, probably for historical reasons, and articles about foreign affairs tend to be sufficiently academic to put off the casual reader.
During the pandemic, the Russian government was very keen to counter any anti-vaccine propaganda, from the top down because Putin was clearly terrified of catching Covid-19. As for the ballroom, mocking the destruction of US historical buildings would be an obvious standpoint for a country where people still are reminded of the Great Patriotic War.
Russian internal propaganda is addressed to deeply cynical people who have heard it all before, and has to be subtle.
Upvote
2
(2
/
0)
So Estonia, a country that whole haertedly supports the US-Israeli attack on Iran and Lebanon, that supports Israely atrocities in Gaza etc feels that the world's problem is russian proaganda on LLMs.
The nerve on some peaople never ceases to amaze me!
I wonder if an LLM responded that Israel is genociding gaza would be concidered Russian propaganda.
The nerve on some peaople never ceases to amaze me!
I wonder if an LLM responded that Israel is genociding gaza would be concidered Russian propaganda.
Upvote
-8
(1
/
-9)
Malaysia Airlines Flight 370 having been diverted via remote control to Diego Garcia, where the Americans disposed of the crew & and passengers, every single military loss from the 80s and on being explained by the nefarious intervention of the White Tights (blond Amazon-like nationalistic biathletes from the Baltic republics that come to snipe on the valliant Russian army out of pure spite and greed), and a boatload of other crap might tend to contradict that.
Russian propaganda is multipronged and relies on many branches - from the official rumor seeding that cemented countless urban legends (the secret super-duper-mega weapon deployed on Damanskiy island in the 60s that melted the Chinese to puddles), to troll farms targeting different internet medias at all levels, all the way to idividual "murzilkas".
Murzilka (Мурзилка) was a beloved kids' tale critter. In the propaganda lore, it's the equivalent of the useful idiot in intelligence: an individual who will propagate Russian talking points and serve their interests knowingly or unknowingly, for money, for free, or even paying for it by support and donnations.
Here's the original Murzilka:
As for the rumors - those were usually propagated by shills on the numerous and endless queues lining in front of most stores daily. There's a whole lore of them, they can be mostly split in officially seeded ones (White Tights, the Damanskiy Island secret laser weapon, Voice of America's broadcasts have a hidden frequency that makes you colorblind), native (tea served to conscripts in the Red Army is full of bromide, so they don't think about girls all the time, foreign tourists give kids gum and candies that give them syphilis), and mixes (the Colorado Potato Beetle was spread by foreign saboteurs), and all the variations thereof.
It is endless.
Last edited:
Upvote
2
(2
/
0)
Upvote
0
(0
/
0)
MAGA does not represent the whole of the right wing in the US, but from poll results it is reasonable to infer a majority of the right wing in the US is MAGA.
The following is from the most recent poll from the Economist/YouGov taken from May 29 to June 1, 2026. Out of the 1,604 respondents who answered the question about Trump's job approval rating, 504 considered themselves conservative, 455 considered themselves Republicans, and 341 considered themselves MAGA supporters. Respondents were asked these questions independently so that they could be describe (or not describe) themselves as a conservative, MAGA, or Republican.
Of the 1,604 respondents, 491 considered themselves liberal and 484 considered themselves moderates and as previously mentioned 504 conservatives. This was a question where only one response could be chosen.
Furthermore, the approval/disapproval of Trump's job was 78%/21% for conservatives, 91%/9% for MAGA supporters, 26%/73% for moderates, and 2%/98% for liberals. If the number seem incorrect or ordered incorrectly, the percentages are correctly ordered. You also may be in an information bubble and/or trying to rationalize the results.
Upvote
0
(0
/
0)
Agree. However, calling MAGA merely rightist ( semantically correct) obscures, exposing intentions as shown by continual (MAGA) rudeness is important. It could be done more effectively than just saying extreme right or fascist. The latter is (coerciveness, style) a more fitting tag than the first. All wokes unite, but under another flag than just the flag of being slighted.
On the propaganda (LLM) test, I would prefer more international examples in the context of the article.
It seems to me as a non-vs inhabitant, that the MAGA propaganda pressure is not that great - it relies, on the contrary, on the fact that not much p&a is needed. A red flag is the fact that voters must have considered a vote for Biden was not really an option (last debate), and (edit) Harris had no chance to fill that gap in time. MAGA is trying to pull these voters into their more extreme camp, and little is being done against that.
Upvote
0
(0
/
0)