Models

The following table lists models supported by MindFormers.

Model	Specifications	Model Type	Latest Version
CodeLlama	34B	Dense LLM	In-development version
CogVLM2-Image	19B	MM	In-development version
CogVLM2-Video	13B	MM	In-development version
DeepSeek-V3	671B	Sparse LLM	In-development version
DeepSeek-V2	236B	Sparse LLM	In-development version
DeepSeek-Coder-V1.5	7B	Dense LLM	In-development version
DeepSeek-Coder	33B	Dense LLM	In-development version
GLM4	9B	Dense LLM	In-development version
GLM3-32K	6B	Dense LLM	In-development version
GLM3	6B	Dense LLM	In-development version
InternLM2	7B/20B	Dense LLM	In-development version
Llama3.1	8B/70B	Dense LLM	In-development version
Llama3	8B/70B	Dense LLM	In-development version
Llama2	7B/13B/70B	Dense LLM	In-development version
Mixtral	8x7B	Sparse LLM	In-development version
Qwen2	0.5B/1.5B/7B/57B/57B-A14B/72B	Dense/Sparse LLM	In-development version
Qwen1.5	7B/14B/72B	Dense LLM	In-development version
Qwen-VL	9.6B	MM	In-development version
Whisper	1.5B	MM	In-development version
Yi	6B/34B	Dense LLM	In-development version
Baichuan2	7B/13B	Dense LLM	1.3.2
GLM2	6B	Dense LLM	1.3.2
GPT2	124M/13B	Dense LLM	1.3.2
InternLM	7B/20B	Dense LLM	1.3.2
Qwen	7B/14B	Dense LLM	1.3.2
CodeGeex2	6B	Dense LLM	1.1.0
WizardCoder	15B	Dense LLM	1.1.0
Baichuan	7B/13B	Dense LLM	1.0
Blip2	8.1B	MM	1.0
Bloom	560M/7.1B/65B/176B	Dense LLM	1.0
Clip	149M/428M	MM	1.0
CodeGeex	13B	Dense LLM	1.0
GLM	6B	Dense LLM	1.0
iFlytekSpark	13B	Dense LLM	1.0
Llama	7B/13B	Dense LLM	1.0
MAE	86M	MM	1.0
Mengzi3	13B	Dense LLM	1.0
PanguAlpha	2.6B/13B	Dense LLM	1.0
SAM	91M/308M/636M	MM	1.0
Skywork	13B	Dense LLM	1.0
Swin	88M	MM	1.0
T5	14M/60M	Dense LLM	1.0
VisualGLM	6B	MM	1.0
Ziya	13B	Dense LLM	1.0
Bert	4M/110M	Dense LLM	0.8

* LLM: Large Language Model; MM: Multi-Modal