Machine Translation Hallucination Detection for Low and High Resource Languages using Large Language Models
Paper
• 2407.16470 • Published
AI & ML interests
None defined yet.
Recent Activity
View all activity
kenza-ily
authored 2 papers 4 days ago
kenza-ily
authored a paper 5 days ago
hannayukhymenko
submitted a paper to Daily Papers 12 days ago
hannayukhymenko
posted an
update 13 days ago
Post
1916
Do you translate your benchmarks from English correctly? 🤔
Turns out, for many languages it is much harder than you can imagine!
Introducing Recovered in Translation 🌍 together with @aalexandrov
https://ritranslation.insait.ai
Translating benchmarks is a painful process, requiring a lot of manual inspection and adjustments. You start from setting up the whole pipeline and adapting to every format type, including task specifics. There already exist some massive benchmarks, but they still have some simple (and sometimes silly) bugs, which can hurt the evaluations :( We present a novel automated translation framework to help with that!
Eastern and Southern European languages introduce richer linguistic structures compared to English and for benchmarks which heavily rely on grammatical coherence machine translation presents a risk of harming evaluations. We discover potential answer leakage or misleading through grammatical structure of the questions. Some benchmarks are also just outdated and need to be retranslated with newer and better models.
We present a framework with novel test-time scaling methods which allow to control time and cost investments, while at the same time mitigate the need for human-in-the-loop verification. While working on Ukrainian-focused MamayLM models, we had to translate 10+ benchmarks in a short span of time. Finding human evaluators is costly and time-consuming, same goes for using professional translators. With our pipeline we were able to do it in 3 days🏎️
We hope our findings will help enable stronger multilingual evaluations and developments. We release all produced benchmarks on Hugging Face together with the source code and Arxiv paper 🤗
Paper: Recovered in Translation: Efficient Pipeline for Automated Translation of Benchmarks and Datasets (2602.22207)
Code: https://github.com/insait-institute/ritranslation
Benchmarks: https://huggingface.co/collections/INSAIT-Institute/multilingual-benchmarks
Turns out, for many languages it is much harder than you can imagine!
Introducing Recovered in Translation 🌍 together with @aalexandrov
https://ritranslation.insait.ai
Translating benchmarks is a painful process, requiring a lot of manual inspection and adjustments. You start from setting up the whole pipeline and adapting to every format type, including task specifics. There already exist some massive benchmarks, but they still have some simple (and sometimes silly) bugs, which can hurt the evaluations :( We present a novel automated translation framework to help with that!
Eastern and Southern European languages introduce richer linguistic structures compared to English and for benchmarks which heavily rely on grammatical coherence machine translation presents a risk of harming evaluations. We discover potential answer leakage or misleading through grammatical structure of the questions. Some benchmarks are also just outdated and need to be retranslated with newer and better models.
We present a framework with novel test-time scaling methods which allow to control time and cost investments, while at the same time mitigate the need for human-in-the-loop verification. While working on Ukrainian-focused MamayLM models, we had to translate 10+ benchmarks in a short span of time. Finding human evaluators is costly and time-consuming, same goes for using professional translators. With our pipeline we were able to do it in 3 days🏎️
We hope our findings will help enable stronger multilingual evaluations and developments. We release all produced benchmarks on Hugging Face together with the source code and Arxiv paper 🤗
Paper: Recovered in Translation: Efficient Pipeline for Automated Translation of Benchmarks and Datasets (2602.22207)
Code: https://github.com/insait-institute/ritranslation
Benchmarks: https://huggingface.co/collections/INSAIT-Institute/multilingual-benchmarks
- 1 reply
hannayukhymenko
authored a paper 16 days ago
tellarin
authored a paper 17 days ago
Post
3223
🤔 Who would win ?
- a fully subsidized ai lab
- 3 random students named
kurakurai ?
demo : Tonic/fr-on-device
if you like it give the demo a little star and send a shoutout to : @MaxLSB @jddqd and @GAD-cell for absolutely obliterating the pareto frontier of the french language understanding .
- a fully subsidized ai lab
OR - 3 random students named
demo : Tonic/fr-on-device
if you like it give the demo a little star and send a shoutout to : @MaxLSB @jddqd and @GAD-cell for absolutely obliterating the pareto frontier of the french language understanding .
- 4 replies
Post
3247
🙋🏻♂️hello my lovelies ,
it is with great pleasure i present to you my working one-click deploy 16GB ram completely free huggingface spaces deployment.
repo : Tonic/hugging-claw (use git clone to inspect)
literally the one-click link : Tonic/hugging-claw
you can also run it locally and see for yourself :
docker run -it -p 7860:7860 --platform=linux/amd64 \
-e HF_TOKEN="YOUR_VALUE_HERE" \
-e OPENCLAW_GATEWAY_TRUSTED_PROXIES="YOUR_VALUE_HERE" \
-e OPENCLAW_GATEWAY_PASSWORD="YOUR_VALUE_HERE" \
-e OPENCLAW_CONTROL_UI_ALLOWED_ORIGINS="YOUR_VALUE_HERE" \
registry.hf.space/tonic-hugging-claw:latest
just a few quite minor details i'll take care of but i wanted to share here first
it is with great pleasure i present to you my working one-click deploy 16GB ram completely free huggingface spaces deployment.
repo : Tonic/hugging-claw (use git clone to inspect)
literally the one-click link : Tonic/hugging-claw
you can also run it locally and see for yourself :
docker run -it -p 7860:7860 --platform=linux/amd64 \
-e HF_TOKEN="YOUR_VALUE_HERE" \
-e OPENCLAW_GATEWAY_TRUSTED_PROXIES="YOUR_VALUE_HERE" \
-e OPENCLAW_GATEWAY_PASSWORD="YOUR_VALUE_HERE" \
-e OPENCLAW_CONTROL_UI_ALLOWED_ORIGINS="YOUR_VALUE_HERE" \
registry.hf.space/tonic-hugging-claw:latest
just a few quite minor details i'll take care of but i wanted to share here first
-
- 2 replies
Cartinoe5930
authored a paper about 1 month ago
tellarin
authored 2 papers about 1 month ago
azminetoushikwasi
submitted a paper to Daily Papers about 1 month ago
tellarin
submitted a paper to Daily Papers about 1 month ago
Sri-Vigneshwar-DJ
posted an
update about 1 month ago
Post
1408
Just released a new dataset designed for training reasoning models on Meta (Facebook/Instagram) advertising fatigue detection!
What is it? A GRPO (Group Relative Policy Optimization) training dataset with 200+ carefully crafted scenarios covering:
🔍 Fatigue Signal Detection: CTR drops, CPM spikes, frequency analysis
🩺 Performance Diagnosis: Root cause analysis frameworks
📋 Strategy: Creative refresh cadence, testing frameworks
📊 Analysis: ROI calculations, metric interpretation
Why GRPO? GRPO training helps models learn structured reasoning. Each response follows the <thinking> and <answer> format.
Check it out here: Sri-Vigneshwar-DJ/meta-fatigue-grpo-dataset
What is it? A GRPO (Group Relative Policy Optimization) training dataset with 200+ carefully crafted scenarios covering:
🔍 Fatigue Signal Detection: CTR drops, CPM spikes, frequency analysis
🩺 Performance Diagnosis: Root cause analysis frameworks
📋 Strategy: Creative refresh cadence, testing frameworks
📊 Analysis: ROI calculations, metric interpretation
Why GRPO? GRPO training helps models learn structured reasoning. Each response follows the <thinking> and <answer> format.
Check it out here: Sri-Vigneshwar-DJ/meta-fatigue-grpo-dataset
Sri-Vigneshwar-DJ
posted an
update about 2 months ago
Post
220
🏙️ Hugging Face Community Post
Title: 🧬 Experimenting with "Dynamic Chaos" in Tamil SLMs
Hi everyone! I just published a new experimental study on Small Language Model (SLM) resilience.
I took the Qwen2.5-0.5B model and put it through a "Chaos Phase" to see how much weight data a tiny model can lose before its understanding of classical Tamil grammar breaks.
Key highlights of the study:
Target Data: Fine-tuned on the Thirukkural (1,330 couplets + modern explanations).
The Chaos Step: Applied 20% random weight pruning but implemented "Layer Protection" for the Token Embeddings and LM Head to keep the characters readable.
Compression: 4-bit (Q4_K_M) quantization for extreme efficiency.
Result: A surrealist classical Tamil model that is ultra-light (~300MB) and ultra-fast!
Check out the model and the experiment logic here: Sri-Vigneshwar-DJ/qwen-tamil-chaos-v1
Title: 🧬 Experimenting with "Dynamic Chaos" in Tamil SLMs
Hi everyone! I just published a new experimental study on Small Language Model (SLM) resilience.
I took the Qwen2.5-0.5B model and put it through a "Chaos Phase" to see how much weight data a tiny model can lose before its understanding of classical Tamil grammar breaks.
Key highlights of the study:
Target Data: Fine-tuned on the Thirukkural (1,330 couplets + modern explanations).
The Chaos Step: Applied 20% random weight pruning but implemented "Layer Protection" for the Token Embeddings and LM Head to keep the characters readable.
Compression: 4-bit (Q4_K_M) quantization for extreme efficiency.
Result: A surrealist classical Tamil model that is ultra-light (~300MB) and ultra-fast!
Check out the model and the experiment logic here: Sri-Vigneshwar-DJ/qwen-tamil-chaos-v1
Sri-Vigneshwar-DJ
posted an
update about 2 months ago
Post
318
Performance Marketing meets "Thinking Mode" 🧠
I’m excited to release hawky-ai-Qwen3-0.6B-Marketing-MoT, a specialized SLM designed for deep strategic reasoning in performance marketing.
While small at 0.6B parameters, this model punches way above its weight class by utilizing a Mixture of Thoughts (MoT) framework. It doesn't just give you an answer; it thinks through the logic of Meta Ads scaling, GA4 attribution, and unit economics before providing a strategic recommendation.
Key Features:
Thinking-First: Trained on 1,500+ critical thinking scenarios.
MoT Framework: 5 distinct reasoning styles (Linear, Exploratory, Critical, Deconstructive, Analogical).
SLM Speed: Perfect for low-latency, high-precision marketing audits.
Check it out on Hugging Face: 🔗 Sri-Vigneshwar-DJ/hawky-ai-Qwen3-0.6B-Marketing-MoT
I’m excited to release hawky-ai-Qwen3-0.6B-Marketing-MoT, a specialized SLM designed for deep strategic reasoning in performance marketing.
While small at 0.6B parameters, this model punches way above its weight class by utilizing a Mixture of Thoughts (MoT) framework. It doesn't just give you an answer; it thinks through the logic of Meta Ads scaling, GA4 attribution, and unit economics before providing a strategic recommendation.
Key Features:
Thinking-First: Trained on 1,500+ critical thinking scenarios.
MoT Framework: 5 distinct reasoning styles (Linear, Exploratory, Critical, Deconstructive, Analogical).
SLM Speed: Perfect for low-latency, high-precision marketing audits.
Check it out on Hugging Face: 🔗 Sri-Vigneshwar-DJ/hawky-ai-Qwen3-0.6B-Marketing-MoT
takarajordan
posted an
update about 2 months ago
Post
194
At takara I'm constantly reading papers, I wonder if anyone can train a model to predict popular papers on our dataset?
takara-ai/daily-papers-popularity
takara-ai/daily-papers-popularity
- 1 reply
Sri-Vigneshwar-DJ
posted an
update 2 months ago
Post
2190
Introducing Hawky-AI H1 4B PM: The First Open-Source LLM for Performance Marketing 🎯
Hey HF Community! 👋
Just released the first LLM fine-tuned specifically for Performance Marketing.
What is it?
Gemma 3 4B distilled from Claude Opus 4.5 with expert-level marketing knowledge.
Covers:
📱 Meta Ads (campaign structure, bidding, scaling, creative fatigue)
🔍 Google Ads (Quality Score, Performance Max, lead gen)
📊 Measurement (ROAS vs MER, incrementality, LTV:CAC)
🎨 Creative Strategy (hook rates, A/B testing, funnel creative)
Why we built it:
Generic LLMs say "optimize your targeting" — not helpful. This model gives specific frameworks like "frequency at 4.5 + CTR drop = creative fatigue, here's the fix..."
Technical:
Base: Gemma 3 4B
Method: QLoRA (r=64)
Teacher: Claude Opus 4.5
🔗 Model: Sri-Vigneshwar-DJ/hawky-ai-H1-4b-PM
Built by Hawky.ai
Try it and let us know what you think! 🚀
Hey HF Community! 👋
Just released the first LLM fine-tuned specifically for Performance Marketing.
What is it?
Gemma 3 4B distilled from Claude Opus 4.5 with expert-level marketing knowledge.
Covers:
📱 Meta Ads (campaign structure, bidding, scaling, creative fatigue)
🔍 Google Ads (Quality Score, Performance Max, lead gen)
📊 Measurement (ROAS vs MER, incrementality, LTV:CAC)
🎨 Creative Strategy (hook rates, A/B testing, funnel creative)
Why we built it:
Generic LLMs say "optimize your targeting" — not helpful. This model gives specific frameworks like "frequency at 4.5 + CTR drop = creative fatigue, here's the fix..."
Technical:
Base: Gemma 3 4B
Method: QLoRA (r=64)
Teacher: Claude Opus 4.5
🔗 Model: Sri-Vigneshwar-DJ/hawky-ai-H1-4b-PM
Built by Hawky.ai
Try it and let us know what you think! 🚀
Sri-Vigneshwar-DJ
posted an
update 2 months ago
Post
1385
🦅 Introducing Hawky AI H1 Mini 4B: A Domain-Specific Model for Performance Marketing
Hey HuggingFace community! 👋
We're excited to share our first open-source release: **Hawky AI H1 Mini 4B Experimental** - a Gemma 3 4B model fine-tuned specifically for Meta advertising and performance marketing strategy.
🎯 Why We Built This
At [Hawky.ai](https://hawky.ai), we build AI-powered creative intelligence tools for performance marketers. We work with major agencies (WPP, Madison, GroupM) and brands (TVS Motors, Tanishq, Bajaj Finserv) on campaign optimization.
We wanted to explore: Can a small, domain-specific model provide expert-level guidance on performance marketing?
Specifically, we focused on Meta's Andromeda algorithm - the AI system that now powers ad delivery across Facebook and Instagram. Understanding Andromeda is crucial for modern media buying, but the knowledge is scattered and constantly evolving.
🧠 What Makes This Different
Chain-of-Thought Reasoning
The model doesn't just answer - it **thinks through problems** step-by-step:
Sri-Vigneshwar-DJ/hawky-ai-h1-mini-4b-experimental
Hey HuggingFace community! 👋
We're excited to share our first open-source release: **Hawky AI H1 Mini 4B Experimental** - a Gemma 3 4B model fine-tuned specifically for Meta advertising and performance marketing strategy.
🎯 Why We Built This
At [Hawky.ai](https://hawky.ai), we build AI-powered creative intelligence tools for performance marketers. We work with major agencies (WPP, Madison, GroupM) and brands (TVS Motors, Tanishq, Bajaj Finserv) on campaign optimization.
We wanted to explore: Can a small, domain-specific model provide expert-level guidance on performance marketing?
Specifically, we focused on Meta's Andromeda algorithm - the AI system that now powers ad delivery across Facebook and Instagram. Understanding Andromeda is crucial for modern media buying, but the knowledge is scattered and constantly evolving.
🧠 What Makes This Different
Chain-of-Thought Reasoning
The model doesn't just answer - it **thinks through problems** step-by-step:
Sri-Vigneshwar-DJ/hawky-ai-h1-mini-4b-experimental