ThaiSafetyBench: Assessing Language Model Safety in Thai Cultural Contexts

Most of the safety testing for Artificial Intelligence (AI) happens in English, which means AI models often don’t understand the unique risks, cultural nuances, and social rules of other languages. To fix this gap for Thailand, researchers created ThaiSafetyBench, a testing tool made of nearly 2,000 tricky and malicious prompts written in Thai. They used this test on 24 popular AI models to see if the AI would generate harmful, unsafe, or culturally inappropriate responses.


Key Insights

• AI has a “Cultural Blindspot”: The most significant finding is that AI models are much easier to trick when the malicious prompts are specifically tied to Thai culture—such as local etiquette, border issues, or the monarchy—compared to general harmful prompts that are just translated into Thai. This proves that simply teaching an AI a new language isn’t enough; it has to be taught the culture, too.
• Commercial AIs are safer than Open-Source AIs: Major closed-source models developed by large tech companies (like GPT-5 and Claude 4.5 Sonnet) are currently much better at refusing to generate harmful content than free, open-source models. This raises concerns about the safety of readily available open-source AI tools.
• Bigger is usually safer: As a general rule, the larger the AI model is, the better it performs at blocking malicious requests. However, the study also noted that high-quality training data is just as important as size.

Practical Benefits for Everyday Consumers Understanding this research translates to several real-world benefits for average users:

• Safer, More Respectful Technology: By pointing out exactly where AI fails in understanding Thai culture, this research pressures AI companies to build tools that respect local manners, traditions, and laws. In the future, consumers will experience AI assistants that are less likely to offend them or misunderstand their cultural context.
• Protection Against Localized Scams and Fake News: The test specifically checks if AI can be tricked into helping with illegal activities, creating Thai-specific fake news, or leaking personal data using realistic Thai names and locations. Improving AI safety in these areas directly protects consumers from localized cybercrime and misinformation.
• A “Safety Scorecard” for AI: The researchers created a public ThaiSafetyBench Leaderboard that ranks how safe different AI models are in Thai contexts. For a consumer or a local business trying to decide which AI app to use or integrate, this leaderboard acts like a safety rating, helping them choose the most trustworthy and reliable tool.