VaishalBusiness/opus

🌍 VGT ONNX Model Hub

---

Welcome to the VGT (Vaishal's Global Translator) ONNX Model Hub — a large-scale open-source collection of ~1,000+ pretrained models converted to ONNX for blazing fast inference, research, and production deployment.

This hub brings together models from leading open-source projects (like Helsinki-NLP MarianMT) and makes them universally accessible via ONNX.

✨ Key highlights:

• ✅ 1,000+ ONNX models for translation, NLP, and beyond
• ✅ Plug-and-play with Hugging Face tokenizers
• ✅ Optimized for fast inference with ONNX Runtime
• ✅ Easy to fetch programmatically (no manual downloads!)
• ✅ Fully open-source, respecting original licenses

🔗 Browse the complete catalog here: yet to add

---

📂 Repository Structure

Each model lives in its own folder, for example:

Helsinki-NLP-opus-mt-en-de/
├── config.json
├── decoder_model.onnx
├── decoder_model_merged.onnx
├── decoder_with_past_model.onnx
├── encoder_model.onnx
├── generation_config.json
├── source.spm
├── target.spm
├── special_tokens_map.json
├── tokenizer_config.json
└── vocab.json

File breakdown:

• encoder_model.onnx → encoder graph
• decoder_model.onnx → base decoder
• decoder_model_merged.onnx → optimized decoder (recommended for speed)
• decoder_with_past_model.onnx → decoder with caching (past key/values)
• Tokenizer files → vocab.json, source.spm, target.spm, etc.
• Configs → model + tokenizer configs

---

🚀 Usage

1. Install dependencies

pip install huggingface_hub onnxruntime transformers sentencepiece

2. Fetch and use a model programmatically

Example with Helsinki-NLP-opus-mt-en-de (replace with any model name from the catalog):

"""
ONNX Runtime MarianMT Translation Demo
======================================

This demo performs text translation using:
- HuggingFace MarianTokenizer
- ONNX Runtime Encoder
- ONNX Runtime Merged Decoder with KV Cache

The decoder uses past key/value attention caching to avoid
recomputing previous tokens, making autoregressive generation
significantly faster.

Model:
    Helsinki-NLP/opus-mt-en-de

Task:
    English → German translation
"""


import json
import numpy as np
import onnxruntime as ort

from huggingface_hub import snapshot_download
from transformers import MarianTokenizer


# ============================================================
# 1. Download Model Files
# ============================================================

print("\n[1/6] Downloading ONNX model...")

model_root = snapshot_download(
    repo_id="VaishalBusiness/opus",
    allow_patterns="Helsinki-NLP-opus-mt-en-de/*",
)

model_dir = f"{model_root}/Helsinki-NLP-opus-mt-en-de"

print(f"Model loaded from:\n{model_dir}")


# ============================================================
# 2. Load Tokenizer and Model Configuration
# ============================================================

print("\n[2/6] Loading tokenizer and configuration...")

tokenizer = MarianTokenizer.from_pretrained(model_dir)


with open(f"{model_dir}/config.json") as file:
    config = json.load(file)


with open(f"{model_dir}/generation_config.json") as file:
    generation_config = json.load(file)


# Generation parameters
eos_token_id = generation_config.get(
    "eos_token_id",
    config.get("eos_token_id")
)

pad_token_id = generation_config.get(
    "pad_token_id",
    config.get("pad_token_id")
)

decoder_start_token_id = generation_config.get(
    "decoder_start_token_id",
    config.get(
        "decoder_start_token_id",
        pad_token_id
    )
)


# Transformer architecture parameters
num_layers = config["decoder_layers"]
num_heads = config["decoder_attention_heads"]

hidden_size = config["d_model"]
head_dim = hidden_size // num_heads


print(
    f"""
Transformer Configuration:
--------------------------
Layers       : {num_layers}
Attention heads : {num_heads}
Head dimension : {head_dim}
Hidden size    : {hidden_size}
"""
)


# ============================================================
# 3. Tokenize Input Text
# ============================================================

print("\n[3/6] Encoding input text...")


source_text = "Hello, how are you?"


encoded = tokenizer(
    source_text,
    return_tensors="np"
)


input_ids = encoded["input_ids"].astype(np.int64)
attention_mask = encoded["attention_mask"].astype(np.int64)


print("Input tokens:")
print(input_ids)


# ============================================================
# 4. Run Encoder
# ============================================================

print("\n[4/6] Running encoder...")


encoder = ort.InferenceSession(
    f"{model_dir}/encoder_model.onnx"
)


encoder_outputs = encoder.run(
    None,
    {
        "input_ids": input_ids,
        "attention_mask": attention_mask
    }
)


encoder_hidden_states = encoder_outputs[0]


print(
    "Encoder output shape:",
    encoder_hidden_states.shape
)



# ============================================================
# 5. Initialize Decoder With KV Cache
# ============================================================

print("\n[5/6] Initializing decoder KV cache...")


decoder = ort.InferenceSession(
    f"{model_dir}/decoder_model_merged.onnx"
)


decoder_inputs = {
    item.name
    for item in decoder.get_inputs()
}


decoder_outputs = [
    item.name
    for item in decoder.get_outputs()
]


batch_size = encoder_hidden_states.shape[0]


# Empty cache for first decoding step.
# Shape:
# [batch, heads, sequence_length, head_dimension]
#
# sequence_length = 0 because no tokens have been generated yet.

empty_cache = np.zeros(
    (
        batch_size,
        num_heads,
        0,
        head_dim
    ),
    dtype=np.float32
)


past_key_values = {}


for layer in range(num_layers):

    past_key_values[
        f"past_key_values.{layer}.decoder.key"
    ] = empty_cache

    past_key_values[
        f"past_key_values.{layer}.decoder.value"
    ] = empty_cache


    # Encoder KV cache is generated during the first decoder call
    past_key_values[
        f"past_key_values.{layer}.encoder.key"
    ] = empty_cache

    past_key_values[
        f"past_key_values.{layer}.encoder.value"
    ] = empty_cache



# ============================================================
# 6. Autoregressive Generation Loop
# ============================================================

print("\n[6/6] Generating translation...\n")


generated_tokens = [
    decoder_start_token_id
]


decoder_input_ids = np.array(
    [[decoder_start_token_id]],
    dtype=np.int64
)


# False = first decoder pass
# True  = reuse KV cache
use_cache_branch = np.array(
    [False],
    dtype=bool
)



MAX_LENGTH = 128


for step in range(MAX_LENGTH):


    # Prepare decoder inputs
    feed = {

        "input_ids":
            decoder_input_ids,

        "encoder_hidden_states":
            encoder_hidden_states,

        "encoder_attention_mask":
            attention_mask,

        "use_cache_branch":
            use_cache_branch,
    }


    # Add cached attention states
    feed.update(
        {
            key: value
            for key, value in past_key_values.items()
            if key in decoder_inputs
        }
    )


    # Run decoder
    output = decoder.run(
        decoder_outputs,
        feed
    )


    output = dict(
        zip(
            decoder_outputs,
            output
        )
    )


    # Select highest probability token
    logits = output["logits"]

    next_token = int(
        np.argmax(
            logits[0, -1]
        )
    )


    generated_tokens.append(
        next_token
    )


    # Stop at EOS token
    if next_token == eos_token_id:
        break



    # Update KV cache
    updated_cache = dict(
        past_key_values
    )


    for layer in range(num_layers):

        updated_cache[
            f"past_key_values.{layer}.decoder.key"
        ] = output[
            f"present.{layer}.decoder.key"
        ]

        updated_cache[
            f"past_key_values.{layer}.decoder.value"
        ] = output[
            f"present.{layer}.decoder.value"
        ]


        # Encoder cache only needs to be stored once
        if step == 0:

            updated_cache[
                f"past_key_values.{layer}.encoder.key"
            ] = output[
                f"present.{layer}.encoder.key"
            ]

            updated_cache[
                f"past_key_values.{layer}.encoder.value"
            ] = output[
                f"present.{layer}.encoder.value"
            ]


    past_key_values = updated_cache


    # Next step only feeds the newly generated token
    decoder_input_ids = np.array(
        [[next_token]],
        dtype=np.int64
    )


    use_cache_branch = np.array(
        [True],
        dtype=bool
    )



# ============================================================
# Decode Output Tokens
# ============================================================

translation = tokenizer.decode(
    [
        token
        for token in generated_tokens[1:]
        if token != eos_token_id
    ],
    skip_special_tokens=True
)


print("=" * 60)
print("Translation Result")
print("=" * 60)
print(translation)
print("=" * 60)

✅ This code works for any model in the hub. Just replace Helsinki-NLP-opus-mt-en-de with your desired model folder name.

---

📋 Model Catalog

• The full catalog of ~the models is available at yet to add.

• Each entry includes:

  • 📌 Model identifier (e.g. Helsinki-NLP-opus-mt-tc-base-bat-zle)
  • 🔗 Direct download links to ONNX artifacts
  • 🗂 Tokenizer + config files
  • 📖 Original Hugging Face model card

We recommend filtering by language pairs or model family when browsing.

---

📝 Attribution

This project would not exist without the incredible open-source community.

• All models come from Helsinki-NLP, a cornerstone of multilingual machine translation.
• The Hugging Face ecosystem provides model hosting, tokenizers, and configs.
• The ONNX community enables fast inference across platforms.

🙏 Huge thanks to all original authors and contributors.

---

📜 Licensing

• Each model retains the license of its original version.
• This hub provides ONNX conversions only; licenses are not overridden.
• Users are responsible for complying with the license terms of each model.
• When using a model, please cite the original authors and respect attribution requirements.

---

🤝 Contributing

We welcome contributions!

• Found a model that needs fixing? Submit a PR.
• Want to add a new ONNX conversion? We’d love to include it.
• Issues and improvements are always appreciated.

---

⚡ Final Notes

This hub is designed to make state-of-the-art NLP models available at scale, ready to drop into production or research pipelines. With 1,440+ ONNX models, you can cover a vast range of languages and tasks, all with the speed of ONNX Runtime.

🔗 Explore → yet to add

🚀 Deploy → Hugging Face Hub + ONNX Runtime

---