AMD Releases Instella-MoE-16B-A3B: A Fully Open Mixture-of-Experts LLM With 2.8B Active Parameters Trained On Instinct GPUs

Sicherheitsmeldung mit Schweregrad noch nicht bewertet. Technische Details im Tab „Originaltext“; empfohlene Schritte in der Checkliste.

Originaltext der Quelle (nicht übersetzt) – die deutsche Zusammenfassung findest du im Tab „Kurzfassung".

AMD released Instella-MoE-16B-A3B, a fully open Mixture-of-Experts language model trained from scratch on Instinct MI300X and MI325X GPUs. The model holds 16B total parameters but activates only 2.8B per token. AMD is publishing weights from every training stage, along with data mixtures, training configs, and inference code. Two systems-level choices carry the release: Gated Multi-head Latent Attention and FarSkip-Collective connectivity.

Is it deployable?

Partly. The weights ship under a ResearchRAIL license for academic and research purposes only, so this is not a drop-in commercial model. The training codebase is MIT licensed, and that is the more reusable asset here.

Company level: AI research labs, university groups, and enterprise R&D teams with data-center GPU capacity. Not a fit for lean startups wanting a hosted commercial endpoint.

Industries: semiconductor and cloud infrastructure, AI tooling vendors, and academic research.

Applications: reproducing an end-to-end MoE recipe, studying expert-parallel serving, evaluating 64K long-context behavior, and running RL post-training experiments.

Serving cost: 16B parameters in BF16 need roughly 32 GB of weight memory, so one high-memory accelerator suffices. AMD ships SGLang inference code.

https://rocm.blogs.amd.com/artificial-intelligence/instella-moe/README.html

Architecture

Instella-MoE is a decoder-only MoE with 27 layers, hidden size 2048, 16 attention heads, and a 128,896-token vocabulary. Each MoE layer uses 2 shared experts plus 6 routed experts selected from 64. That yields 2.8B active parameters …

Vollständiger Originalartikel ↗

• Betroffenheit im eigenen Stack prüfen: Versionen/Komponenten abgleichen.
• Originalquelle / Hersteller-Advisory lesen: https://www.marktechpost.com/2026/08/01/amd-instella-moe-16b-a3b-fully-open-mixture-of-experts-llm/
• Verfügbaren Patch oder Workaround einspielen und dokumentieren.