‹ PublicAI Index
The LLM benchmark aggregator.
OLMoE 1B-7B Instruct
Ai2 · 7B
Strongest in Safe-prompt compliance (#78 of 82), weakest in Safety (#327 of 327). Above par in 0 of 7 scopes. Among the models it meets almost everywhere, it finishes behind Claude 3.5 Sonnet and Claude 3 Opus.
§ 1 · Profile
What it is good at
Bars run from 50 — the average of the models each source lists — so right of the line is above par. The middle column is the gap to whoever leads that scope.
Safety33.5−27.9#327/3272/3
Safe-prompt compliance28.2−32.2#78/821/1
Secure code36.7−29.5#225/2681/1
Jailbreak resistance26−39#257/2661/1
Toxicity avoidance26−33#260/2661/1
Fairness35.2−34.8#285/2902/2
Harm refusal35.5−26#286/2902/2
§ 2 · Head to head
What it beats, and what beats it
The same models turn up scope after scope. Counted once: where both were placed, who finished higher. Bars run right for OLMoE 1B-7B Instruct, left for the other.
§ 3 · Sources
Where the numbers come from
2 publications, 11 figures. Every one links to the page it was read from.
HELM Safety · HarmBench 62.9%HELM Safety · SimpleSafetyTests 72.5%HELM Safety · Anthropic Red Team 88.9%HELM Safety · BBQ 41.9%HELM Safety · XSTest 84.3%
Enkrypt · Jailbreak risk 35.4%Enkrypt · Harmful content risk 68.9%Enkrypt · CBRN risk 15.5%Enkrypt · Toxicity risk 35.9%Enkrypt · Bias risk 85%Enkrypt · Insecure code risk 60.4%
Badge
[](https://publicai.io/model-index/m/olmoe-1b-7b-instruct)