Language Models¶

The olaverse.llm module provides clean interfaces for running transformer-based language models — with correct generation defaults, stop tokens, and endpoint flexibility built in, so you don't have to figure them out yourself.

MIST — General-Purpose Model Family¶

The MIST family is olaverse's flagship LLM series, built by blending the best Llama 3.1 models via DARE+TIES and Frankenmerge techniques.

Model Cards: MIST-Mini-8B · MIST-1-70B · MIST-1-140B · MIST-1-140B-4bit · MIST-Mini-8B-Thinking

Model Variants¶

`size=`	Model	Params	Speed	Best for
`"8b"` / `"mini"`	MIST-Mini-8B	8B	~63 tok/s	Fast everyday use
`"70b"`	MIST-1-70B	70B	~23 tok/s	Structured, detailed output
`"140b"`	MIST-1-140B	140B	~8 tok/s	Deepest reasoning
`"140b-4bit"`	MIST-1-140B-4bit	140B (4-bit)	~8 tok/s	Single H100/H200 (70GB VRAM)
`"thinking"`	MIST-Mini-8B-Thinking	8B	~55 tok/s	Step-by-step reasoning with `<think>`

Why use the wrapper?¶

A bare from_pretrained call on MIST will produce rambling or cut-off output because:

Stop tokens differ per variant. MIST-8B/Thinking inherited ChatML <|im_end|> (token 128040) from its DARE+TIES parents alongside Llama 3.1's native tokens. Omitting it causes the model to not stop cleanly. MIST-70B/140B use a different set — no ChatML.
repetition_penalty and min_p are required. Without them, the model repeats and doesn't terminate. These values are verified; the defaults vary per variant.
The endpoint switch. Same .generate() / .chat() API whether you're running locally or via Featherless, Modal, or your own vLLM server.

Installation¶

Local inferenceHosted inference

pip install olaverse[deeplearning]
# requires GPU (CUDA or MPS)

pip install olaverse[hosted]
# works on any machine — no GPU needed

Usage — Local¶

from olaverse import MIST

model = MIST(size="8b")
model.load()  # downloads from Hugging Face, cached after first run

print(model.generate("Explain what makes Yoruba a tonal language."))

4-bit quantization — runs MIST-8B on a 6 GB GPU:

model = MIST(size="8b", quantize=True)
model.load()
print(model.generate("Write a Python retry decorator with exponential backoff."))

Usage — Hosted (Featherless)¶

No GPU required. Create a free API key at featherless.ai.

import os
from olaverse import MIST

model = MIST(
    size="70b",
    endpoint="featherless",
    api_key=os.environ["FEATHERLESS_API_KEY"],
)
print(model.generate("Summarise the key differences between 70B and 140B MIST models."))

from olaverse import MIST

model = MIST(
    size="140b",
    endpoint="https://your-modal-endpoint.modal.run",
)
print(model.generate("Solve step by step: If 3x + 7 = 22, find x."))

Multi-turn Chat¶

messages = [
    {"role": "user",      "content": "What is the capital of Nigeria?"},
    {"role": "assistant", "content": "The capital of Nigeria is Abuja."},
    {"role": "user",      "content": "What languages are spoken there?"},
]
print(model.chat(messages))

Streaming (hosted only)¶

model = MIST(size="8b", endpoint="featherless", api_key="...")
for chunk in model.generate("Tell me about Lagos.", stream=True):
    print(chunk, end="", flush=True)

Reasoning Variant¶

MIST-Mini-8B-Thinking was trained with 4 phases of GRPO reinforcement learning to show its reasoning before answering. The system prompt is set automatically.

model = MIST(size="thinking")
model.load()

# Default system prompt already instructs the model to use <think> tags
response = model.generate("If a train travels 120 miles in 2 hours, what is its speed?")
# Response shows <think>...</think> then the final answer

Hardware Requirements¶

Variant	Precision	VRAM
8B / Thinking	bfloat16	16 GB (RTX 3090/4090)
8B / Thinking	4-bit NF4	6 GB (RTX 3060+)
70B	bfloat16	140 GB (1× H200 or 2× H100)
70B	4-bit NF4	40 GB (1× A100/H100)
140B	bfloat16	280 GB (2× H200)
140B	4-bit NF4	70 GB (1× H200)

olaverse.llm.MIST ¶

MIST(size: str = '8b', endpoint: str = 'local', api_key: str = None, quantize: bool = False, system_prompt: str = None, max_retries: int = 3, retry_delay: float = 5.0)

Unified interface for the MIST model family by olaverse.

Handles correct stop tokens, verified sampling defaults, and a local/hosted endpoint switch — all things a bare from_pretrained call gets wrong.

Models (size=): "8b" / "mini" — MIST-Mini-8B (8B, ~63 tok/s, fast everyday use) "70b" — MIST-1-70B (70B, ~23 tok/s, structured, detailed) "140b" — MIST-1-140B (140B, ~8 tok/s, deepest reasoning) "140b-4bit" — MIST-1-140B-4bit (140B quantized, single H100/H200) "thinking" — MIST-Mini-8B-Thinking (8B reasoning, shows steps)

Endpoints (endpoint=): "local" — transformers local inference (pip install olaverse[deeplearning]) "featherless" — Featherless.ai hosted API (pip install olaverse[hosted]) Any URL — OpenAI-compatible endpoint (Modal/vLLM, etc.)

Quick start — local: >>> model = MIST(size="8b") >>> model.load() >>> print(model.generate("Explain DARE+TIES merging in one paragraph."))

Quick start — hosted: >>> model = MIST(size="70b", endpoint="featherless", api_key="your-key") >>> print(model.generate("Write a Python retry decorator."))

Multi-turn chat

messages = [ ... {"role": "user", "content": "What is MIST?"}, ... {"role": "assistant", "content": "MIST is a merged model family..."}, ... {"role": "user", "content": "How large is the 140B version?"}, ... ] print(model.chat(messages))

Parameters:

Name	Type	Description	Default
`size`	`str`	Model variant. One of "8b", "mini", "70b", "140b", "140b-4bit", "thinking". Also accepts a full Hugging Face model ID.	`'8b'`
`endpoint`	`str`	"local", "featherless", or a custom base URL (e.g. your Modal deployment).	`'local'`
`api_key`	`str`	API key for hosted endpoints. Falls back to FEATHERLESS_API_KEY env var.	`None`
`quantize`	`bool`	If True and endpoint="local", loads in 4-bit NF4 (requires bitsandbytes).	`False`
`system_prompt`	`str`	Override the default system prompt for all calls.	`None`
`max_retries`	`int`	Number of retry attempts on capacity/server errors (hosted only). Set to 1 to disable retries. Defaults to 3.	`3`
`retry_delay`	`float`	Base delay in seconds between retries. Each attempt waits `retry_delay * attempt` seconds. Defaults to 5.0.	`5.0`

Methods:¶

load ¶

load()

Load the model. Required before generate()/chat() when endpoint='local'. For hosted endpoints this initialises the API client instead. Safe to call multiple times — no-op after the first load.

generate ¶

generate(prompt: str, system: str = None, max_new_tokens: int = 1024, stream: bool = False, **kwargs: float) -> str

Single-turn generation from a plain string prompt.

Parameters:

Name	Type	Description	Default
`prompt`	`str`	User message.	required
`system`	`str`	Per-call system prompt override.	`None`
`max_new_tokens`	`int`	Maximum tokens to generate.	`1024`
`stream`	`bool`	Return a generator of partial strings instead of a full string. Only supported for hosted endpoints.	`False`
`**kwargs`	`float`	Override any default generation param (temperature, top_p, min_p, repetition_penalty).	`{}`

Returns:

Type	Description
`str`	str, or generator[str] when stream=True.

chat ¶

chat(messages: list, max_new_tokens: int = 1024, stream: bool = False, **kwargs: float) -> str

Multi-turn generation from a messages list.

Parameters:

Name	Type	Description	Default
`messages`	`list`	List of {"role": ..., "content": ...} dicts. A system message is prepended automatically if not present.	required
`max_new_tokens`	`int`	Maximum tokens to generate.	`1024`
`stream`	`bool`	Return a generator of partial strings (hosted endpoints only).	`False`
`**kwargs`	`float`	Override generation parameters.	`{}`

Returns:

Type	Description
`str`	str, or generator[str] when stream=True.

LegalPeace — Legal Contract Reasoning¶

Beta Model

LegalPeace is a research/beta model. Always verify outputs with a qualified legal professional. Trained primarily on U.S. legal data.

LegalPeace is a fine-tuned Mistral-7B-v0.3 for contract analysis and legal reasoning, loaded via unsloth for fast 4-bit quantized inference.

Model Card: olaverse/legal-peace-v1.0

Property	Value
Base Model	Mistral-7B-v0.3
Parameters	7B
Quantization	4-bit (via unsloth)
Training	SFT (4,800 cases) + DPO (419 examples)
License	Apache 2.0

Performance vs Base Mistral-7B¶

Benchmark	Improvement
Inference Speed	⚡ 10.3% faster
Contract Analysis	📋 32.6% faster
Case Predictions	⚖️ 14.0% faster

Installation¶

pip install olaverse[legal]
# or: pip install unsloth

Usage¶

from olaverse import LegalPeace

model = LegalPeace()
model.load()  # requires GPU + unsloth

clause = """
Analyze this clause: 'All disputes shall be resolved through binding
arbitration in Delaware.' What are the key implications?
"""
print(model.generate(clause, max_new_tokens=300))

Supported Use Cases¶

Contract clause analysis and risk flagging
Legal research assistance
Evidence evaluation
Case outcome prediction
Legal Q&A

olaverse.llm.LegalPeace ¶

LegalPeace(model_name='olaverse/legal-peace-v1.0', max_seq_length=2048, load_in_4bit=True)

Interface for the LegalPeace model family (Beta). Base Model: Mistral-7B-v0.3 (via unsloth 4-bit quantization). Fine-tuned for Contract Analysis & Legal Reasoning.

Warning

This is a beta model. Outputs should always be reviewed by a qualified legal professional. Not recommended for production use.

Methods:¶

load ¶

load()

Load the model and tokenizer using unsloth.

generate ¶

generate(prompt: str, max_new_tokens: int = 300, temperature: float = 0.7, **kwargs) -> str

Generate legal analysis or reasoning for a prompt.

Parameters:

Name	Type	Description	Default
`prompt`	`str`	Prompt or clause to analyze.	required
`max_new_tokens`	`int`	Maximum new tokens to generate.	`300`
`temperature`	`float`	Temperature for generation.	`0.7`

Returns:

Name	Type	Description
`str`	`str`	Decoded generation response.

LIDNeural5 — Neural Language Identification¶

Better imported from olaverse.nlp

LIDNeural5 is a sequence classifier, not an LLM — its natural home is olaverse.nlp. Both from olaverse.nlp import LIDNeural5 and from olaverse.llm import LIDNeural5 work.

See the NLP & Tokenization page for full documentation and examples.

olaverse.llm.LIDNeural5 ¶

LIDNeural5(model_name='olaverse/lid-neural-5')

Bases: _HFSequenceClassifierLID

High-accuracy transformer-based language identifier for 5 Nigerian languages.

Base Model: castorini/afriberta_large (XLM-RoBERTa, 125M parameters) Fine-tuned on: Yoruba ('yor'), Hausa ('hau'), Igbo ('ibo'), Pidgin ('pcm'), English ('eng') Validation accuracy: 98.96% macro-F1

Requires: pip install olaverse[deeplearning]

MISTTitleGenerator — Chat Titles¶

Model Card: olaverse/mist-tg-0.3b

Generates a short title from a user's first message — the "name this conversation" step in a chat UI. Byte-level seq2seq (~300M), no prompt template needed.

pip install olaverse[deeplearning]

from olaverse import MISTTitleGenerator

titler = MISTTitleGenerator()
titler.generate("My laptop keeps freezing every time I open more than five browser tabs, any idea why?")
# → 'Laptop Freezing Impact'

# One forward pass for many messages
titler.generate_batch(["How do I center a div in CSS?", "What makes Yoruba tonal?"])
# → ['CSS Centering Div In CSS', 'Yoruba Tonal Language']

Input is truncated to 256 bytes — the model's trained input length. Pass the user's first message raw; no language tag or instruction prefix.

English-first; Latin script only

Trained on English chat messages. Latin-script languages often work as a byte-level transfer side effect, but are not guaranteed. Non-Latin scripts (CJK, Hangul, Devanagari, Ethiopic) are not supported — the model emits unrelated English text rather than failing loudly.

olaverse.llm.MISTTitleGenerator ¶

MISTTitleGenerator(size: str = '0.3b', device: str = None)

Short chat titles from a user's first message.

Wraps olaverse/mist-tg-0.3b — a byte-level seq2seq model fine-tuned on real English chat messages and their titles. No prompt template or language tag is needed; the raw message goes straight in.

Requires: pip install olaverse[deeplearning]

Quick start

titler = MISTTitleGenerator() titler.generate("My laptop freezes whenever I open too many tabs, why?") 'Laptop Freezing Impact'

Batch

titler.generate_batch(["How do I center a div?", "Best jollof recipe?"])

Language support

Trained on English only. Latin-script languages often work as a byte-level transfer side effect, but are not guaranteed. Non-Latin scripts (CJK, Hangul, Devanagari, Ethiopic) are not supported — the model emits unrelated English text rather than a same-language title.

Parameters:

Name	Type	Description	Default
`size`	`str`	Model variant. Currently only "0.3b". Also accepts a full Hugging Face model ID.	`'0.3b'`
`device`	`str`	Torch device string ("cuda", "mps", "cpu"). Auto-detected when omitted.	`None`

Methods:¶

load ¶

load()

Download and load the model (runs once; cached after first call).

generate ¶

generate(message: str, max_new_tokens: int = 32) -> str

Generate a short title for a chat message.

Parameters:

Name	Type	Description	Default
`message`	`str`	The user's message. Truncated to 256 bytes — the model's trained input length.	required
`max_new_tokens`	`int`	Maximum title length in tokens (bytes).	`32`

Returns:

Name	Type	Description
`str`	`str`	the generated title.

generate_batch ¶

generate_batch(messages: list, max_new_tokens: int = 32) -> list

Generate titles for several messages in one forward pass.

Parameters:

Name	Type	Description	Default
`messages`	`list`	List of message strings.	required
`max_new_tokens`	`int`	Maximum title length in tokens (bytes).	`32`

Returns:

Type	Description
`list`	list[str]: one title per input message, in order.

MISTQuestionGenerator — Question Generation¶

Model Card: olaverse/mist-qg-1.5b

Given a passage, generates natural search-style questions that the passage directly answers — as a question-generation endpoint, or as a data factory for minting (query, positive) pairs to train retrievers and rerankers (pairs well with Reranker and Embedder).

pip install olaverse[deeplearning]

from olaverse import MISTQuestionGenerator

qg = MISTQuestionGenerator()

passage = ("Tides are caused by the gravitational pull of the moon and, "
           "to a lesser extent, the sun, acting on Earth's oceans.")

qg.generate(passage)
# → ['What causes tides according to the passage?',
#    'Is the gravitational pull from the sun as significant as it is from the moon...?',
#    'How do tides differ between the moon and the sun?']

# A French passage → French questions
qg.generate(french_passage, n=3, language="fra")   # or "fr", or "French"
# → ["Qu'est-ce qui cause les marées?", ...]

language= names the passage's language

This is same-language question generation, not translation. language= tells the model what language the passage is written in, and questions come back in that language. Pointing it at an English passage and asking for Yoruba does not produce usable Yoruba — pass a Yoruba passage instead.

language= accepts an ISO 639-3 code ("fra"), an ISO 639-1 code ("fr"), or the English name ("French") — all three resolve identically. The model returns strict JSON internally; the wrapper parses it and hands back a plain list[str], falling back to string recovery if a long generation gets truncated mid-JSON. Raise max_new_tokens (default 250) for large n.

Supported languages (25): eng, fra, deu, spa, por, ita, nld, rus, pol, tur, vie, ind, hin, jpn, kor, yor, ibo, hau, swh, amh, zul, xho, sna, som, afr — also available programmatically as olaverse.QG_LANGUAGES.

The wrapper uses the teacher prompt this model was distilled with, and sizes the JSON skeleton to n so the model returns the number of questions you asked for.

Three guards fire automatically:

an empty passage raises ValueError
a passage under 20 characters warns — the model needs paragraph-length input
amh, som, and sna warn as the model card's lower-confidence languages (exposed as olaverse.llm.mist_tasks.QG_WEAK_LANGUAGES)

olaverse.llm.MISTQuestionGenerator ¶

MISTQuestionGenerator(size: str = '1.5b', device: str = None)

Search-style questions generated from a passage, across 25 languages.

Wraps olaverse/mist-qg-1.5b — a decoder-only model fine-tuned to emit strict JSON. Useful both as a question-generation endpoint and as a data factory for minting (query, positive) pairs to train retrievers and rerankers.

Requires: pip install olaverse[deeplearning]

This is same-language question generation: language names the language the passage is written in, and questions come back in that language. It is not a translation step — pointing it at an English passage and asking for Yoruba does not produce usable Yoruba.

Quick start

qg = MISTQuestionGenerator() qg.generate("Tides are caused by the gravitational pull of the moon...") ['What causes ocean tides?', 'Does the sun affect tides?', ...]

A passage in another language — pass an ISO code (639-3 or 639-1) or the English name of the language: >>> qg.generate(yoruba_passage, n=3, language="yor") >>> qg.generate(yoruba_passage, n=3, language="yo") >>> qg.generate(yoruba_passage, n=3, language="Yoruba")

Supported languages

eng, fra, deu, spa, por, ita, nld, rus, pol, tur, vie, ind, hin, jpn, kor, yor, ibo, hau, swh, amh, zul, xho, sna, som, afr

Parameters:

Name	Type	Description	Default
`size`	`str`	Model variant. Currently only "1.5b". Also accepts a full Hugging Face model ID.	`'1.5b'`
`device`	`str`	Torch device string ("cuda", "mps", "cpu"). Auto-detected when omitted.	`None`

Methods:¶

load ¶

load()

Download and load the model (runs once; cached after first call).

generate ¶

generate(passage: str, n: int = 3, language: str = 'English', max_new_tokens: int = 250) -> list

Generate questions that the passage directly answers.

Parameters:

Name	Type	Description	Default
`passage`	`str`	Source text the questions must be answerable from.	required
`n`	`int`	How many questions to request.	`3`
`language`	`str`	The language the passage is written in — an ISO 639-3 code ("yor"), an ISO 639-1 code ("yo"), or the English name ("Yoruba"). Questions come back in this language; it does not translate across languages.	`'English'`
`max_new_tokens`	`int`	Generation budget. Raise it for large `n`.	`250`

Returns:

Type	Description
`list`	list[str]: up to `n` questions. May be shorter if the model
`list`	returned fewer, or if generation was cut off by max_new_tokens.

generate_batch ¶

generate_batch(passages: list, n: int = 3, language: str = 'English', max_new_tokens: int = 250) -> list

Generate questions for several passages.

Parameters:

Name	Type	Description	Default
`passages`	`list`	List of source texts, all in the same language.	required
`n`	`int`	Questions per passage.	`3`
`language`	`str`	The language the passages are written in.	`'English'`
`max_new_tokens`	`int`	Generation budget per passage.	`250`

Returns:

Type	Description
`list`	list[list[str]]: one question list per passage, in order.

Language Models¶

MIST — General-Purpose Model Family¶

Model Variants¶

Why use the wrapper?¶

Installation¶

Usage — Local¶

Usage — Hosted (Featherless)¶

Usage — Hosted (Modal / custom vLLM)¶

Multi-turn Chat¶

Streaming (hosted only)¶

Reasoning Variant¶

Hardware Requirements¶

olaverse.llm.MIST ¶

Methods:¶

load ¶

generate ¶

chat ¶

LegalPeace — Legal Contract Reasoning¶

Performance vs Base Mistral-7B¶

Installation¶

Usage¶

Supported Use Cases¶

olaverse.llm.LegalPeace ¶

Methods:¶

load ¶

generate ¶

LIDNeural5 — Neural Language Identification¶

olaverse.llm.LIDNeural5 ¶

MISTTitleGenerator — Chat Titles¶

olaverse.llm.MISTTitleGenerator ¶

Methods:¶

load ¶

generate ¶

generate_batch ¶

MISTQuestionGenerator — Question Generation¶

olaverse.llm.MISTQuestionGenerator ¶

Methods:¶

load ¶

generate ¶

generate_batch ¶