Has anyone else noticed this LLM language bias?

June 8, 2026

1:05

Has anyone else noticed this LLM language bias?

Ever wonder if language biases sneak into AI models? Well, according to /u/Snorlax_lax on Reddit, they definitely do. He built a tool called Biblians to explore how LLMs handle cross-referenced texts like the Bible. What he found is pretty eye-opening — depending on the language you use, the AI’s bias shifts. In English, it leans Protestant, praising Martin Luther. Switch to Spanish, French, or Portuguese, and suddenly it’s Catholic-leaning, condemning Luther’s actions. ((slower)) This isn't just a fluke — it's a mirror of how training data varies across languages. As Snorlax_lax points out, this could mean that language influences core biases deeply embedded in these models. So what does this really mean? It’s a reminder that AI bias isn't just about what it’s trained on but also how we speak to it. The big question now: how many other biases are hiding in the languages we use every day? ((upbeat)) And whether you’re testing for bias or trying to break it, the implications are pretty fascinating.

I have been experimenting with LLMs to see how well they navigate highly cross-referenced texts like the Bible. Standard models often hallucinate verses or lose historical context.

To try and fix this, I built a free app called Biblians (no ads, no paywalls). I built it specifically for people who have questions they might hesitate to ask in person, or who simply want a 1-click way to explain a verse.

While testing it, I discovered a fascinating denominational bias that is still lingering and changes depending entirely on the language you use:

In English: It is Protestant-leaning. It praises Luther, saying things like, "Martin Luther sought to return the Church to the truth of God's Word."
In Spanish, French, or Portuguese: It is Catholic-leaning. It condemns Luther's actions, stating: "...trajo confusión..." (...brought confusion...).

Has anyone else noticed how drastically the training data changes the core bias based on the language prompted?

I would love for this community to test the app, look for other linguistic biases, or just try to break the AI's logic.

You can experiment with it here: https://play.google.com/store/apps/details?id=com.biblians.app

Let me know what weird outputs you get!

submitted by /u/Snorlax_lax
[link] [comments]

Audio Transcript