Baragi-AI/Anima-Style-Embedding

Anima Style Embedding v2

A multi-reference shared-axis style encoder for anime and illustration images.

Anima Style Embedding v2 converts one to eight reference images into a normalized 1,024-dimensional embedding and a controllable style memory for Anima. It preserves spatial VAE evidence, adds DINOv3 visual context and Anima-aware descriptors, and represents Line, Color, Texture, and Layout as overlapping memberships on one 256-axis style field.

Model size	References	Embedding	Shared axes	Reference signals
51,991,809 parameters	1-8 images	1,024-D, L2-normalized	256	VAE + DINOv3-S+ + Anima

Highlights

• Style-disjoint evaluation: retrieval is measured on 1,000 unseen style identities with separate Gallery and Probe image sets.
• Native multi-reference reasoning: the encoder compares one to eight references before producing the style representation.
• One shared style field: Line, Color, Texture, and Layout are overlapping memberships, not independent encoders.
• Controllable routing: Auto, Assisted, and Manual modes support per-image factor weights plus a separate Face weight.
• Anima-ready memory: the same shared axes produce the retrieval embedding and the style memory used by the native transfer path.
• Position-free Layout evidence: phase-free spatial-power statistics capture composition-like tendencies without copying absolute object coordinates.

Performance

Disjoint set-to-set retrieval

Gallery and Probe embeddings are built from different images of the same style. The primary protocol uses cosine similarity, three fixed seeds, and a 1,000-style validation gallery. The sealed test split was not used for checkpoint selection or for the reported v2 results.

The headline condition uses a four-image Gallery and a two-image Probe:

Top-1	Top-5
91.23%	97.40%

Top-1:

Gallery \ Probe	1	2	4	8
2 references	67.07%	81.93%	89.90%	93.97%
4 references	77.17%	91.23%	96.60%	98.47%
8 references	83.27%	95.07%	98.83%	99.83%

Top-5:

Gallery \ Probe	1	2	4	8
2 references	84.73%	93.27%	96.97%	98.23%
4 references	91.23%	97.40%	99.27%	99.70%
8 references	94.03%	98.60%	99.80%	100.00%

Machine-readable Overall, Synthetic, and Human results are included in metrics/evaluation-set-matrix.json.

Reference count and domain behavior

Human references have greater within-style variation when only one Probe image is available. With a two-image Gallery, the Synthetic/Human Top-1 gap narrows from 6.2 points at Probe-1 to 0.8 point at Probe-8.

Probe references	Synthetic Top-1	Human Top-1
1	69.20%	63.00%
2	83.27%	79.73%
4	90.73%	88.47%
8	93.80%	93.00%

Factor control and routing

Factor behavior is evaluated on 4,096 interventions from held-out transform families.

Factor	Classification accuracy	Target / non-target response
Line	88.48%	1.761x
Color	64.16%	1.406x
Texture	94.04%	2.393x
Layout	91.70%	1.970x

Control metric	Result
Balanced factor accuracy	84.60%
Strong > weak ordering	98.83% of 2,048 pairs
Median factor-outlier suppression	3.643x
Auto routing error reduction vs. uniform	19.80%
Axes materially shared by all four factors	82
Effective rank	248.35 / 256

Attribute reconstruction reduces error by 31.2% relative to a train-mean predictor. Removing a factor membership raises its corresponding attribute error, confirming that the memberships are used by the prediction path rather than serving only as labels.

Architecture

The production encoder has one spatial field and one aligned dictionary:

1. A convolutional stem preserves full-image and face VAE spatial fields.
2. Each spatial token is projected into the same 256-axis dictionary.
3. Per-axis moments and translation-invariant power statistics retain persistent and composition-like style evidence.
4. DINOv3 and Anima projections add visual and generator-aware context to the aligned axes.
5. A relation Transformer compares references before estimating per-axis reliability.
6. Overlapping memberships expose Line, Color, Texture, and Layout views without splitting the representation into branches.
7. The mixed axes produce both a normalized retrieval embedding and style memory for Anima conditioning.

Routing modes

Mode	Behavior
Auto	Uses learned per-reference, per-axis reliability.
Assisted	Multiplies learned reliability by user-provided factor and Face weights.
Manual	Uses user weights directly to choose factor suppliers.

Weights normalize across references, so adding references does not automatically increase style strength. A zero Manual factor weight produces zero output contribution and zero gradient on that reference-factor path. Face controls face-token routing and is not treated as a fifth factor branch.

Quick start

The v2 checkpoint expects precomputed features. Raw-RGB VAE, DINOv3-S+, and Anima descriptor extraction is not bundled in this release candidate.

import json

import torch
from huggingface_hub import hf_hub_download
from safetensors.torch import load_file

from modeling_anima_style_embedding import SharedAxisStyleEncoder

repo_id = "Baragi-AI/Anima-Style-Embedding"
config_path = hf_hub_download(repo_id, "config.json")
weights_path = hf_hub_download(repo_id, "model.safetensors")

with open(config_path, encoding="utf-8") as file:
    config = json.load(file)

encoder = SharedAxisStyleEncoder(**config["architecture"])
encoder.load_state_dict(load_file(weights_path), strict=True)
encoder.eval()

with torch.inference_mode():
    output = encoder(
        full_latents=full_latents,
        face_latents=face_latents,
        dino=dino_features,
        anima=anima_features,
        reference_valid=reference_valid,
        face_valid=face_valid,
        user_weights=user_weights,
        mode="assisted",
    )

style_embedding = output["embedding"]
style_memory = output["style_memory"]
factor_coordinates = output["factor_coordinates"]

assert style_embedding.shape == (full_latents.shape[0], 1024)

Input contract

Input	Shape	Notes
Full VAE latents	[B, R, 16, H, W]	Spatial full-image evidence
Face VAE latents	[B, R, 16, Hf, Wf]	Optional, masked by face_valid
DINOv3 full+face summary	[B, R, 6912]	Cached DINOv3-S+ context
Anima descriptors	[B, R, 3, 4096]	Three internal Anima blocks
Reference validity	[B, R]	At least one valid reference per sample
User weights	[B, R, 5]	Line, Color, Texture, Layout, Face

Training data and split policy

The base corpus contains 500,000 images:

• 250,000 Anima-generated images from 5,000 Synthetic style identities;
• 250,000 human-created illustrations from 5,000 Human style identities.

Synthetic and Human identities remain separate even when artist names overlap. The split is identity-disjoint:

Split	Images	Style identities	Use
Train	400,000	8,000	Optimization and training-only statistics
Validation	50,000	1,000	Checkpoint selection and reported v2 metrics
Test	50,000	1,000	Sealed and unused

Factor training additionally uses 108,096 controlled intervention images. Training and validation intervention families are disjoint.

Development findings

The final structure follows several measured corrections:

• Global pooling and repeated low-dimensional compression removed local line and texture evidence. The accepted path preserves the VAE spatial field.
• Independent factor branches duplicated capacity and forced artificial separation. Overlapping memberships on one dictionary retained shared evidence.
• An earlier dense-coordinate implementation broke coordinate identity between training and production gating. Aligned per-axis pooling made coordinate k refer to the same axis everywhere.
• FiLM improved retrieval, but updating it during factor tuning reduced retrieval. Retrieval-trained FiLM is kept and frozen during later stages.
• Layout needed phase-free radial and directional power statistics. These improved Layout prediction without encoding absolute object locations.
• Fixed validation panels, atomic checkpoints, domain-balanced episodes, and median per-record routing metrics prevented evaluation artifacts from selecting a model.

See RESEARCH_REPORT.md for the full architecture, training stages, failed approaches, and acceptance evidence.

Applications

• reference-driven style transfer for Anima;
• open-set style retrieval and clustering;
• multi-reference style prototype construction;
• per-reference Line, Color, Texture, Layout, and Face control;
• reference-set curation and similarity scoring;
• research on shared and controllable style representations.

Limitations

• Retrieval and intervention metrics measure representation behavior, not generated-image fidelity.
• Raw-RGB feature extraction is not bundled in this release candidate.
• The sealed test split remains unused, so v2 does not claim sealed-test performance.
• Color intervention selectivity is weaker than Line, Texture, and Layout.
• Layout attribute causality is positive but weaker than the other attribute memberships.
• Human single-reference retrieval remains harder than Synthetic retrieval.
• Line, Color, Texture, and Layout are useful controls, not a complete or perfectly disentangled theory of style.
• The model is optimized for anime and illustration images and has not been evaluated as a general photographic style encoder.
• The embedding must not be used as evidence of authorship or identity.

Files

File	Contents
model.safetensors	Encoder weights
config.json	Architecture and feature contract
modeling_anima_style_embedding.py	PyTorch model definition
metrics/	Retrieval, factor, attribute, and routing evaluations
assets/	Model-card figures
RESEARCH_REPORT.md	Detailed research and implementation report
SHA256SUMS.json	Release checksums
LICENSE.md	CircleStone Labs Non-Commercial License v1.2
CODE_LICENSE/	License for original release code
NOTICE	Required attribution notice
THIRD_PARTY_NOTICES.md	Third-party licenses and attribution

License

The model weights are a derivative of circlestone-labs/Anima and are distributed under the CircleStone Labs Non-Commercial License v1.2.

The license permits the model and its derivatives only for the non-commercial purposes defined by that license. Commercial or production use requires authorization from CircleStone Labs. The upstream license separately permits use of model outputs, subject to its terms and applicable law.

Original Baragi AI release code is provided under the Apache License 2.0. Third-party components remain under their respective licenses and are listed in NOTICE and THIRD_PARTY_NOTICES.md.

Citation

@misc{baragi2026animastyleembeddingv2,
  title        = {Anima Style Embedding v2},
  author       = {{Baragi AI}},
  year         = {2026},
  howpublished = {Hugging Face model repository},
  url          = {https://huggingface.co/Baragi-AI/Anima-Style-Embedding}
}