Skip to content

Instantly share code, notes, and snippets.

@jaggzh

jaggzh/01-prompt.txt

Created September 28, 2025 19:36
Show Gist options
  • Save jaggzh/8e2a3892d835bece4f3c218661c6ca85 to your computer and use it in GitHub Desktop.
Save jaggzh/8e2a3892d835bece4f3c218661c6ca85 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
01-prompt.txt
User (NON-deep):
comparison of characteristics of texts captured by different common small vector embeddings. What types of things are captured/represented in each of them? We should include the most common 20 to 40 depending on variations/information spread, and include model memory consumption and speed along with the details on what's captured or what they excel in and what they miss. An estimated popularity rank should be included but it doesn't need to be reliable. Only models runnable locally/foss. Before we begin, help me resume this request for a deep research prompt; I'm not actually sure if I'm asking the right questions. I want a reference I can turn to for making decisions on model use when working on different projects. We should include less popular models as well if they have special characteristics making them more suitable for some tasks.
cgpt:
Great target. Here’s a tight research brief + “ready-to-run” master prompt that locks scope, outputs, and criteria so we can build a dependable, decision-making reference for local, open-source small embedding models.
# Research brief (what we’ll do)
## Objective
Create a practitioner-oriented comparison of ~20–40 **local, FOSS** text embedding models (“small” encoders) that focuses on:
1. what kinds of textual information each tends to capture well vs poorly,
2. practical resource needs (memory/VRAM) and speed, and
3. when to pick which model for common project types.
## Inclusion criteria
* **Open weights**, permissive license suitable for local/commercial use (Apache-2.0, MIT, BSD, etc.).
* **Runnable locally** on CPU and/or consumer GPU without proprietary runtimes.
* **“Small” encoders**: ≤ ~1.5B parameters (most are <<1B); typical output dims 256–1536.
* **Primary use**: general text embeddings (bi-encoder). (Cross-encoders/rerankers excluded except when a model is marketed as “dual-use”.)
* **Variety**: include less-popular models if they offer distinctive traits (e.g., instruction-tuned embeddings, multilingual coverage, domain specialization, long-context, very small CPU-friendly).
## What we’ll report per model (schema)
* **Identity**: model name, org, version/date, license, link.
* **Footprint**: params (M), default precision, **RAM/VRAM** at load (FP16/FP32 + common int8/int4 quant options if officially supported), **embedding dim**, **max sequence length**.
* **Tokenizer**: type (WordPiece/BPE/Unigram), vocab size, Unicode/emoji handling notes.
* **Speed** (single-pass encode throughput):
* CPU (AVX2) texts/sec @ batch 1 and batch 32.
* GPU (consumer, e.g., 12–24GB) texts/sec @ batch 32 and 128.
* Notes: measured on short (avg 128 tokens) and long (avg 512) texts.
* **Captured signals** (1–5 rating + one-line evidence per item):
* Topical similarity (bag-of-concepts)
* **Paraphrase robustness** (lexical variation)
* **Word order / syntax sensitivity**
* **Negation/contrast** handling
* **Antonymy** separation (vs mere topical overlap)
* **Named entity** fidelity (spelling variance, rare NE handling)
* **Entity typing & relations** (light relational knowledge)
* **Numeric expressions** (quantities, ranges, dates)
* **Temporal expressions** (before/after, recency words)
* **Style/tone** (formality, writing style)
* **Sentiment/subjectivity** leakage
* **Instruction following** (query vs passage modes, if applicable)
* **Domain transfer**: code, legal, biomedical, finance (brief)
* **Multilingual coverage** (languages, cross-lingual alignment)
* **Length robustness** (128→2k tokens), truncation sensitivity
* **Stopword/punctuation** influence
* **OOV/rare-word** handling
* **Chunk-boundary stability** (small chunk shifts)
* **Training objective & pooling**: (e.g., contrastive, STS, multi-task; mean vs CLS; norm) + implications.
* **Benchmarks** (headline only): MTEB overall + category highlights (retrieval/STS/clustering/rerank) and any noted strengths/weaknesses (e.g., strong on multilingual STS, weak on long-doc retrieval).
* **Known pitfalls**: e.g., over-reliance on lexical overlap, brittle negation, sentence-length bias.
* **Best-fit use cases**: short FAQ search, long-doc RAG, multilingual Q/A, code search, domain X, CPU-only, tiny-memory pipelines, etc.
* **Estimated popularity rank** (rough): composite of HF downloads, GitHub stars, papers/mentions (method below).
## Popularity estimate (rough & optional)
* Normalize **HF monthly downloads**, **GitHub stars of repo**, and **recent citations/mentions**; average the z-scores → **Popularity Rank (lower=more popular)**. Flag low-confidence when signals are sparse.
## Measurement approach (speed & memory)
* **Hardware baselines**:
* CPU: 8C/16T AVX2 (or AVX-512 if available), 32GB RAM.
* GPU: RTX 4090-class (24GB) or 4070-class (12GB).
* **Batch & length**: measure short (128 tok) and long (512 tok), batches {1, 32, 128} where feasible.
* **Precision**: FP16/FP32; int8/int4 if officially supported or via common toolchains (bitsandbytes/ggml/gguf).
* **Memory**: peak resident set at load + steady-state; output vector size overhead (dim × dtype × batch).
* Note: report **representative** numbers; emphasize relative ordering rather than absolute micro-optimizations.
## Benchmarks & probes (what “characteristics” means in practice)
* **MTEB** (overall + task slices: retrieval, STS, clustering, reranking).
* **BEIR** / MIRACL for out-of-domain + multilingual retrieval.
* **Targeted probes** (small synthetic/curated): negation, antonymy, word-order swaps, number perturbations (“3 → 30”), date shifts, named-entity misspellings, punctuation/stopword toggles, chunk boundary shifts.
* **Domain mini-panels**: short legal/biomed/finance/code retrieval sets (open datasets).
## Outputs
* **Master table** (CSV + Markdown) with all fields above.
* **One-page “cheat sheet”**: “If you need X, prefer A/B; avoid C because Y.”
* **Short notes per model** (≤10 lines) summarizing captures/excels/misses.
* **Appendix**: measurement details, hardware, dataset list, and the popularity ranking method.
## Candidate roster (to be verified during execution)
Families we’ll sample from to reach 20–40 models (final list will filter for license & recency):
* **Sentence-Transformers** classics: all-MiniLM-L6/L12, all-mpnet-base, paraphrase-MiniLM (mono & multilingual), LaBSE-based variants, Sentence-T5 variants.
* **E5** (intfloat): e5-small/base/large-v2; multilingual-e5.
* **BGE** (BAAI): bge-small/base/large-en-v1.5; **bge-m3**; multilingual bge.
* **GTE / Qwen-derived**: gte-small/base/large-en-v1.5; multilingual gte; newer Qwen-GTE variants.
* **UAE** (WhereIsAI): UAE-Large/Small.
* **Nomic**: nomic-embed-text-v1.x (CPU-friendly).
* **Jina**: jina-embeddings v2 (en / multilingual).
* **MXBAI / mixedbread**: mxbai-embed-large-v1 (and small if available).
* **Arctic-Embed** (Snowflake) small/multilingual flavors.
* **ModernBERT-Embed** small/base (Neural Magic).
* **Chinese-centric**: m3e (moka-ai) small/base/large; text2vec-chinese.
* **Code-aware** (if intended for text too): code-search-derived encoders that work reasonably for general text.
(Exact versions + licenses will be confirmed; anything non-FOSS or not runnable locally will be dropped.)
---
# “Master prompt” you can give me to execute the study
> **Goal:** Build a decision reference comparing local, FOSS small embedding models (20–40 models).
> **Inclusion:** Open weights, permissive license; ≤~1.5B params; general text bi-encoders; include distinctive/less-popular models where useful.
> **Report per model:** name, version/date, license, link; params; precision; RAM/VRAM at load (FP16/FP32 & int8/int4 if supported); embedding dim; max seq len; tokenizer; CPU & GPU throughput (texts/sec) at batch {1,32,128} for 128- and 512-token inputs; training objective & pooling; normalization; multilingual coverage; domain notes; MTEB/BEIR/MIRACL headlines; targeted probe results (negation, antonymy, word order, numbers, dates, named-entity misspellings, punctuation/stopwords, chunk-boundary sensitivity, length robustness, instruction following); known pitfalls; best-fit use cases.
> **Popularity:** compute a rough rank from HF downloads, GitHub stars, and recent mentions (z-score average).
> **Benchmarks:** MTEB (overall + category), BEIR/MIRACL slices; small domain panels (legal/biomed/finance/code) with public datasets; targeted probes listed above.
> **Measurement setup:** CPU (8C/16T AVX2, 32GB RAM) and GPU (RTX 4090-class 24GB); FP16/FP32 and int8/int4 if supported; record peak/steady memory; use batch 1/32/128; inputs of 128 and 512 tokens.
> **Deliverables:**
>
> 1. A CSV + Markdown table with all fields;
> 2. A cheat-sheet mapping common needs to top 3 model picks (“If you need X → pick A/B; avoid C because Y”);
> 3. Short per-model notes (≤10 lines);
> 4. Appendix with methods, datasets, and popularity method.
> **Candidate pool to consider:** Sentence-Transformers (MiniLM/mpnet/T5/LaBSE), E5 (mono/multi), BGE (incl. M3), GTE/Qwen-GTE, UAE (WhereIsAI), Nomic-embed-text, Jina v2, MXBAI (mixedbread), Arctic-Embed, ModernBERT-Embed, m3e/text2vec (Chinese), and any recent small multilinguals.
> **Constraints:** FOSS only; runnable locally; drop closed or API-only.
> **Output style:** concise, practitioner-friendly; emphasize relative trade-offs; call out “what they capture/excel/miss” clearly; include caveats when evidence is weak.
---
# Optional refinements (use if this matches your projects)
* **Define “small” by output dim too**: include a sub-view for very small vectors (≤384 dims) to guide index size decisions.
* **Quantization-first view**: add a table showing which models retain accuracy best at int8/int4.
* **Long-context subset**: for models with ≥4k token windows, add a mini-ranking on long-doc retrieval.
* **Reranking pairing**: recommend a lightweight cross-encoder per embedding for high-precision top-k rerank (still local/FOSS).
* **Multilingual matrix**: mini heatmap of cross-lingual retrieval across 8–12 languages.
---
# Sample row (structure illustration)
* **Model**: Example-Embed-base-vX (Apache-2.0)
* **Params / Dim / MaxLen**: 110M / 768 / 512
* **Memory (FP16 / int8)**: ~220MB / ~110MB
* **Speed (CPU texts/s, 128tok)**: b1: 55, b32: 850 • **GPU** b32: 7,500; b128: 19,000
* **Training / Pooling**: Contrastive (STS+retrieval); mean-pool + L2-norm
* **Captured signals (1–5)**: Paraphrase 4.5; Word order 2.5; Negation 2.0; Antonymy 3.0; Named entities 3.5; Numbers 2.5; Temporal 2.0; Style 2.5; Sentiment 3.0; Instruction 1.0; Multilingual 1.0; Length robustness 3.0; Stopwords 2.0; OOV 3.0; Chunk stability 3.5
* **Benchmarks (headlines)**: MTEB overall: mid-tier; strong STS; weak long-doc retrieval
* **Excels at**: short FAQ search; semantic dedup on English web text
* **Misses**: brittle on negation and numeric ranges; not multilingual
* **Best-fit**: CPU-only services needing small index + fast build
* **Popularity rank (lower is better)**: 9 / 35 (low-confidence)
---
If this brief looks right, say “run it” and I’ll produce the full comparison in that format. If you want adjustments (different hardware baselines, model cap, or extra probes), specify them and I’ll fold them in.
User (deep):
I should not restrict it to 40. If the arena has many more they will likely be found during the research and may be included. 40 is not a hard limit. A section of the report should explain terminology used; a mini glossary near each area where those sets of terms are used. For easier manual filtering, some columns may be appropriately each duplicated into a "quantized" version of the column. Produce as an html page web UI with mouse-over for on-hover glossary result. Sorting ability *and* elaborate filtering UI with sliders and check boxes where appropriate. Should be usable in all major browsers. If html/js can handle it in-page, exporting/downloading the data would be a good but optional feature.
Raw
embedding_models_comparison-v5.html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1"/>
<title>Local FOSS Embedding Models — Deep Comparison (30 models)</title>
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/[email protected]/media/css/jquery.dataTables.min.css">
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/[email protected]/dist/nouislider.min.css">
<style>
body { font-family: system-ui, -apple-system, Segoe UI, Roboto, 'Helvetica Neue', Arial, 'Noto Sans', 'Liberation Sans', sans-serif; margin: 16px; }
h1 { margin-bottom: 8px; }
.controls { display: grid; grid-template-columns: repeat(3, minmax(280px, 1fr)); gap: 12px; margin: 12px 0 20px; align-items: start; }
.control-card { border: 1px solid #ddd; border-radius: 10px; padding: 12px; background: #fafafa; }
.control-card-wide { grid-column: span 3; }
.term { border-bottom: .15em dashed #9a9; cursor: help; position: relative; }
.tooltip-box { position: absolute; max-width: 420px; background: #111; color: #fff; padding: 10px 12px; border-radius: 8px; font-size: 13px; line-height: 1.35; z-index: 9999; display: none; pointer-events: none; }
.legend small { color: #666; }
table.dataTable thead th { position: sticky; top: 0; background: #fff; cursor: help; }
.pill { display:inline-block; padding:2px 8px; border-radius:999px; background:#eef; margin-right:4px; font-size:12px; }
.hint { color:#666; font-size:12px; }
.toggles { display:flex; flex-wrap: wrap; gap:8px 14px; max-height: 220px; overflow: auto; }
.toggle-item { white-space: nowrap; }
.range-label { display:flex; justify-content:space-between; font-size:12px; color:#444; margin-top: 4px; }
.range-box { margin-top:8px; }
.checkbox-group { display: flex; flex-direction: row; gap: 4px; }
.checkbox-group label { display: flex; align-items: center; gap: 6px; }
#glossary-list { columns: 2; column-gap: 24px; }
#glossary-list dt { font-weight:600; margin-top:8px; }
#glossary-list dd { margin: 0 0 8px 0; color:#333; }
</style>
</head>
<body>
<h1>Local FOSS Text Embedding Models — Comparison & Reference (30 models)</h1>
<p class="hint">All 30 rows are on one page. Hover underlined terms for definitions. Hover column headers for explanations. Use field toggles and sliders to filter, then export as CSV.</p>
<div class="controls">
<div class="control-card control-card-wide">
<strong>Field toggles</strong>
<div id="fieldToggles" class="toggles"></div>
<div class="hint">Show/hide columns to focus on what matters for your use case.</div>
</div>
<div class="control-card">
<strong>Parameters (M)</strong>
<div id="paramsSlider" class="range-box"></div>
<div class="range-label"><span id="paramsMinLbl"></span><span id="paramsMaxLbl"></span></div>
<div class="hint">Model size in millions of parameters.</div>
</div>
<div class="control-card">
<strong>Embedding Dimensions</strong>
<div id="dimSlider" class="range-box"></div>
<div class="range-label"><span id="dimMinLbl"></span><span id="dimMaxLbl"></span></div>
<div class="hint">Size of output vectors; affects storage and compute.</div>
</div>
<div class="control-card">
<strong>Max Sequence Length</strong>
<div id="maxlenSlider" class="range-box"></div>
<div class="range-label"><span id="maxlenMinLbl"></span><span id="maxlenMaxLbl"></span></div>
<div class="hint">Maximum input length in tokens before truncation.</div>
</div>
<div class="control-card">
<strong>Normalized FP16 Memory</strong>
<div id="memSlider" class="range-box"></div>
<div class="range-label"><span id="memMinLbl"></span><span id="memMaxLbl"></span></div>
<div class="hint">0 = smallest model, 1 = largest model (based on FP16 memory).</div>
</div>
<div class="control-card">
<strong>Speed Category</strong>
<div class="checkbox-group">
<label><input type="checkbox" class="speedBox" value="Fast" checked> Fast</label>
<label><input type="checkbox" class="speedBox" value="Medium" checked> Medium</label>
<label><input type="checkbox" class="speedBox" value="Slow" checked> Slow</label>
</div>
<div class="hint">Relative inference speed (heuristic based on model size).</div>
</div>
<div class="control-card">
<strong>Language Support</strong>
<div class="checkbox-group">
<label><input type="checkbox" class="langBox" value="English" checked> English</label>
<label><input type="checkbox" class="langBox" value="Multilingual" checked> Multilingual</label>
<label><input type="checkbox" class="langBox" value="Chinese" checked> Chinese</label>
</div>
<div class="hint">Primary language capabilities of the model.</div>
</div>
<div class="control-card">
<strong>Linguistic Capabilities (min scores)</strong>
<label>Negation ≥ <input type="number" id="negMin" min="1" max="5" step="0.1" value="0" style="width:60px"> </label>
<label>Word order ≥ <input type="number" id="woMin" min="1" max="5" step="0.1" value="0" style="width:60px"> </label>
<label>Numbers/Dates ≥ <input type="number" id="numMin" min="1" max="5" step="0.1" value="0" style="width:60px"> </label>
<label>Long context ≥ <input type="number" id="lcMin" min="1" max="5" step="0.1" value="0" style="width:60px"> </label>
<div class="hint">Relative scores (1=weakest, 5=strongest). Use with reranking for precision tasks.</div>
</div>
<div class="control-card">
<strong>Export & Search</strong>
<button id="exportCsv">Download Filtered CSV</button><br/>
<span class="hint">Use the search box in the table to filter by domain keywords like "legal", "biomed", "code", "reviews".</span>
<div style="margin-top:8px;"><span class="pill">FAQ</span><span class="pill">Blogs</span><span class="pill">Reviews</span><span class="pill">Legal</span><span class="pill">Biomed</span><span class="pill">Finance</span><span class="pill">Code</span><span class="pill">JSON</span><span class="pill">Books</span></div>
</div>
</div>
<table id="tbl" class="display" style="width:100%"></table>
<h2>Glossary</h2>
<p class="hint">The page auto-highlights technical terms in the table. Hover any underlined term for a detailed definition.</p>
<dl id="glossary-list"></dl>
<div id="tooltip" class="tooltip-box"></div>
<script src="https://code.jquery.com/jquery-3.6.0.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/[email protected]/media/js/jquery.dataTables.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/[email protected]/dist/nouislider.min.js"></script>
<script>
const DATA =
[
{
"code_score" : 1.6,
"dates_score" : 2.6,
"description" : "Small, fast English retriever tuned for semantic search (use 'query:'/'passage:' prefixes for best results).",
"dimension_category" : "Medium (385-768)",
"domains_caution" : "Long narrative books; numeric/date queries; heavy multilingual",
"domains_good" : "FAQ search, KB lookup, deduplication, clustering",
"embedding_dim" : 512,
"family" : "BGE",
"json_score" : 1.8,
"languages" : "English",
"license" : "Apache-2.0",
"long_context_flag" : "No",
"long_context_score" : 2.7,
"max_seq_len" : 512,
"memory_category" : "Small (<0.5GB)",
"memory_fp16_gb" : 0.066,
"memory_fp32_gb" : 0.132,
"memory_norm_fp16_0_1" : 0.023,
"model_name" : "BAAI/bge-small-en-v1.5",
"multilingual_score" : 1,
"named_entity_score" : 3,
"negation_score" : 2.6,
"notes" : "Best on short passages; can degrade on long/negated queries without reranking.",
"numbers_score" : 2.6,
"organization" : "BAAI",
"parameters_m" : 33,
"pooling" : "CLS pooling + L2 normalization",
"sentiment_score" : 3,
"speed_category" : "Fast",
"style_tone_score" : 2.8,
"tense_gender_score" : 2.8,
"train_objective" : "Contrastive learning for dense retrieval; supports asymmetric/symmetric retrieval; hard negatives",
"word_order_score" : 2.8
},
{
"code_score" : 1.6,
"dates_score" : 2.9,
"description" : "Strong English baseline; widely used for RAG and search; instruction prefixes adjust query/doc behavior.",
"dimension_category" : "Medium (385-768)",
"domains_caution" : "Fine-grained negation or numeric ranges without rerank",
"domains_good" : "General RAG, enterprise search, blog/news retrieval",
"embedding_dim" : 768,
"family" : "BGE",
"json_score" : 1.8,
"languages" : "English",
"license" : "Apache-2.0",
"long_context_flag" : "No",
"long_context_score" : 2.7,
"max_seq_len" : 512,
"memory_category" : "Small (<0.5GB)",
"memory_fp16_gb" : 0.218,
"memory_fp32_gb" : 0.436,
"memory_norm_fp16_0_1" : 0.182,
"model_name" : "BAAI/bge-base-en-v1.5",
"multilingual_score" : 1.5,
"named_entity_score" : 3,
"negation_score" : 2.9,
"notes" : "Sensitive to prompt format; benefits from reranking on tricky queries (negation, antonyms).",
"numbers_score" : 2.9,
"organization" : "BAAI",
"parameters_m" : 109,
"pooling" : "CLS pooling + L2 normalization",
"sentiment_score" : 3,
"speed_category" : "Medium",
"style_tone_score" : 2.8,
"tense_gender_score" : 2.8,
"train_objective" : "Contrastive retrieval with instruction prefixes; hard-negative mining",
"word_order_score" : 3
},
{
"code_score" : 1.6,
"dates_score" : 3.1,
"description" : "Higher-accuracy English dense retriever; heavier VRAM/latency.",
"dimension_category" : "High (769-1024)",
"domains_caution" : "Mobile/edge, ultra-low latency",
"domains_good" : "High-precision retrieval, legal/finance knowledge bases",
"embedding_dim" : 1024,
"family" : "BGE",
"json_score" : 1.8,
"languages" : "English",
"license" : "Apache-2.0",
"long_context_flag" : "No",
"long_context_score" : 2.7,
"max_seq_len" : 512,
"memory_category" : "Medium (0.5-1.5GB)",
"memory_fp16_gb" : 0.67,
"memory_fp32_gb" : 1.34,
"memory_norm_fp16_0_1" : 0.6548,
"model_name" : "BAAI/bge-large-en-v1.5",
"multilingual_score" : 1.5,
"named_entity_score" : 3,
"negation_score" : 3.2,
"notes" : "Not ideal for edge/CPU constrained deployments or small devices.",
"numbers_score" : 3.1,
"organization" : "BAAI",
"parameters_m" : 335,
"pooling" : "CLS pooling + L2 normalization",
"sentiment_score" : 3,
"speed_category" : "Slow",
"style_tone_score" : 2.8,
"tense_gender_score" : 2.8,
"train_objective" : "Large contrastive retriever; instruction-tuned; hard negatives",
"word_order_score" : 3.3
},
{
"code_score" : 1.6,
"dates_score" : 2.6,
"description" : "Versatile: dense retrieval, sparse hybrid scoring, multilingual alignment.",
"dimension_category" : "High (769-1024)",
"domains_caution" : "Tiny-memory apps; strict latency requirements",
"domains_good" : "Multilingual search, hybrid dense+sparse, heterogeneous corpora",
"embedding_dim" : 1024,
"family" : "BGE",
"json_score" : 1.8,
"languages" : "English+Multilingual",
"license" : "Apache-2.0",
"long_context_flag" : "No",
"long_context_score" : 2.7,
"max_seq_len" : 512,
"memory_category" : "Small (<0.5GB)",
"memory_fp16_gb" : 0.218,
"memory_fp32_gb" : 0.436,
"memory_norm_fp16_0_1" : 0.182,
"model_name" : "BAAI/bge-m3",
"multilingual_score" : 3.3,
"named_entity_score" : 3,
"negation_score" : 2.6,
"notes" : "Heavier vectors (1024-D) and extra complexity; not the smallest option.",
"numbers_score" : 2.6,
"organization" : "BAAI",
"parameters_m" : 109,
"pooling" : "CLS pooling + L2 normalization",
"sentiment_score" : 3,
"speed_category" : "Medium",
"style_tone_score" : 2.8,
"tense_gender_score" : 2.8,
"train_objective" : "Multi-function (dense/lexical), multilingual, multi-granularity; contrastive + hybrid signals",
"word_order_score" : 2.8
},
{
"code_score" : 1.6,
"dates_score" : 2.6,
"description" : "Higher scores than bge-m3 but significantly heavier resource requirements.",
"dimension_category" : "High (769-1024)",
"domains_caution" : "On-device use, resource-constrained environments",
"domains_good" : "Cross-lingual enterprise search, curated corpora with multiple languages",
"embedding_dim" : 1024,
"family" : "BGE",
"json_score" : 1.8,
"languages" : "English+Multilingual",
"license" : "Apache-2.0",
"long_context_flag" : "No",
"long_context_score" : 2.7,
"max_seq_len" : 512,
"memory_category" : "Medium (0.5-1.5GB)",
"memory_fp16_gb" : 0.67,
"memory_fp32_gb" : 1.34,
"memory_norm_fp16_0_1" : 0.6548,
"model_name" : "BAAI/bge-m3-large",
"multilingual_score" : 3.7,
"named_entity_score" : 3,
"negation_score" : 2.6,
"notes" : "Not suitable for edge/CPU only deployments.",
"numbers_score" : 2.6,
"organization" : "BAAI",
"parameters_m" : 335,
"pooling" : "CLS pooling + L2 normalization",
"sentiment_score" : 3,
"speed_category" : "Slow",
"style_tone_score" : 2.8,
"tense_gender_score" : 2.8,
"train_objective" : "As above (larger backbone); improved multilingual & hybrid performance",
"word_order_score" : 2.8
},
{
"code_score" : 1.6,
"dates_score" : 2.6,
"description" : "Tiny, fast, robust general-purpose encoder.",
"dimension_category" : "Small (≤384)",
"domains_caution" : "Long-form retrieval; complex negation",
"domains_good" : "Lightweight Q&A, FAQ, tagging, clustering",
"embedding_dim" : 384,
"family" : "E5",
"json_score" : 1.8,
"languages" : "English",
"license" : "Apache-2.0",
"long_context_flag" : "No",
"long_context_score" : 2.7,
"max_seq_len" : 512,
"memory_category" : "Small (<0.5GB)",
"memory_fp16_gb" : 0.066,
"memory_fp32_gb" : 0.132,
"memory_norm_fp16_0_1" : 0.023,
"model_name" : "intfloat/e5-small-v2",
"multilingual_score" : 1.5,
"named_entity_score" : 3,
"negation_score" : 2.6,
"notes" : "Lower ceiling on nuanced reasoning and long context.",
"numbers_score" : 2.6,
"organization" : "intfloat",
"parameters_m" : 33,
"pooling" : "Mean pooling + L2 normalization",
"sentiment_score" : 3,
"speed_category" : "Fast",
"style_tone_score" : 2.8,
"tense_gender_score" : 2.8,
"train_objective" : "E5 contrastive pretraining (CCPairs) with query/passage role tokens; no special prompts required",
"word_order_score" : 2.8
},
{
"code_score" : 1.6,
"dates_score" : 2.6,
"description" : "Balanced speed/quality; easy integration (no prefixes).",
"dimension_category" : "Medium (385-768)",
"domains_caution" : "Lengthy books/code bases",
"domains_good" : "General search/RAG, product search, support portals",
"embedding_dim" : 768,
"family" : "E5",
"json_score" : 1.8,
"languages" : "English",
"license" : "Apache-2.0",
"long_context_flag" : "No",
"long_context_score" : 2.7,
"max_seq_len" : 512,
"memory_category" : "Small (<0.5GB)",
"memory_fp16_gb" : 0.218,
"memory_fp32_gb" : 0.436,
"memory_norm_fp16_0_1" : 0.182,
"model_name" : "intfloat/e5-base-v2",
"multilingual_score" : 1.5,
"named_entity_score" : 3,
"negation_score" : 3,
"notes" : "Less strong on long documents than large models.",
"numbers_score" : 3,
"organization" : "intfloat",
"parameters_m" : 109,
"pooling" : "Mean pooling + L2 normalization",
"sentiment_score" : 3,
"speed_category" : "Medium",
"style_tone_score" : 2.8,
"tense_gender_score" : 2.8,
"train_objective" : "E5 contrastive pretraining (CCPairs) + supervised fine-tuning",
"word_order_score" : 3.1
},
{
"code_score" : 1.6,
"dates_score" : 2.6,
"description" : "High-accuracy English encoder; slower & heavier.",
"dimension_category" : "High (769-1024)",
"domains_caution" : "Ultra-low latency APIs",
"domains_good" : "Mission-critical retrieval, legal/biomed, high-recall search",
"embedding_dim" : 1024,
"family" : "E5",
"json_score" : 1.8,
"languages" : "English",
"license" : "Apache-2.0",
"long_context_flag" : "No",
"long_context_score" : 2.7,
"max_seq_len" : 512,
"memory_category" : "Medium (0.5-1.5GB)",
"memory_fp16_gb" : 0.66,
"memory_fp32_gb" : 1.32,
"memory_norm_fp16_0_1" : 0.6444,
"model_name" : "intfloat/e5-large-v2",
"multilingual_score" : 1.5,
"named_entity_score" : 3,
"negation_score" : 3.3,
"notes" : "Not suitable for edge deployments or small GPUs.",
"numbers_score" : 3.2,
"organization" : "intfloat",
"parameters_m" : 330,
"pooling" : "Mean pooling + L2 normalization",
"sentiment_score" : 3,
"speed_category" : "Slow",
"style_tone_score" : 2.8,
"tense_gender_score" : 2.8,
"train_objective" : "E5 large; improved retrieval & STS; supervised fine-tuning",
"word_order_score" : 3.4
},
{
"code_score" : 1.6,
"dates_score" : 2.6,
"description" : "Small footprint GTE variant with competitive quality for its size.",
"dimension_category" : "Small (≤384)",
"domains_caution" : "Long narratives; math-heavy docs",
"domains_good" : "Short-text clustering, dedup, Q&A",
"embedding_dim" : 384,
"family" : "GTE",
"json_score" : 1.8,
"languages" : "English",
"license" : "Apache-2.0",
"long_context_flag" : "No",
"long_context_score" : 2.7,
"max_seq_len" : 512,
"memory_category" : "Small (<0.5GB)",
"memory_fp16_gb" : 0.12,
"memory_fp32_gb" : 0.24,
"memory_norm_fp16_0_1" : 0.0795,
"model_name" : "thenlper/gte-small",
"multilingual_score" : 1.5,
"named_entity_score" : 3,
"negation_score" : 2.6,
"notes" : "Less robust on long context and numeric/temporal reasoning.",
"numbers_score" : 2.6,
"organization" : "THENLPER",
"parameters_m" : 60,
"pooling" : "Mean pooling + L2 normalization",
"sentiment_score" : 3,
"speed_category" : "Medium",
"style_tone_score" : 2.8,
"tense_gender_score" : 2.8,
"train_objective" : "Multi-stage contrastive; instruction signals",
"word_order_score" : 2.8
},
{
"code_score" : 1.6,
"dates_score" : 2.6,
"description" : "Solid baseline; general-purpose RAG/search.",
"dimension_category" : "Medium (385-768)",
"domains_caution" : "Dense code or table-heavy corpora",
"domains_good" : "Docs/wiki/blogs, support tickets, emails",
"embedding_dim" : 768,
"family" : "GTE",
"json_score" : 1.8,
"languages" : "English",
"license" : "Apache-2.0",
"long_context_flag" : "No",
"long_context_score" : 2.7,
"max_seq_len" : 512,
"memory_category" : "Small (<0.5GB)",
"memory_fp16_gb" : 0.22,
"memory_fp32_gb" : 0.44,
"memory_norm_fp16_0_1" : 0.1841,
"model_name" : "thenlper/gte-base",
"multilingual_score" : 1.5,
"named_entity_score" : 3,
"negation_score" : 3,
"notes" : "Requires reranking for tricky negations/antonyms.",
"numbers_score" : 2.6,
"organization" : "THENLPER",
"parameters_m" : 110,
"pooling" : "Mean pooling + L2 normalization",
"sentiment_score" : 3,
"speed_category" : "Medium",
"style_tone_score" : 2.8,
"tense_gender_score" : 2.8,
"train_objective" : "Multi-stage contrastive; instruction signals",
"word_order_score" : 2.8
},
{
"code_score" : 1.6,
"dates_score" : 2.6,
"description" : "Higher accuracy than gte-base but heavier.",
"dimension_category" : "High (769-1024)",
"domains_caution" : "On-device/edge applications",
"domains_good" : "High-recall search, analysis on curated datasets",
"embedding_dim" : 1024,
"family" : "GTE",
"json_score" : 1.8,
"languages" : "English",
"license" : "Apache-2.0",
"long_context_flag" : "No",
"long_context_score" : 2.7,
"max_seq_len" : 512,
"memory_category" : "Medium (0.5-1.5GB)",
"memory_fp16_gb" : 0.67,
"memory_fp32_gb" : 1.34,
"memory_norm_fp16_0_1" : 0.6548,
"model_name" : "thenlper/gte-large",
"multilingual_score" : 1.5,
"named_entity_score" : 3,
"negation_score" : 3.2,
"notes" : "Not suitable for edge/CPU or strict latency constraints.",
"numbers_score" : 2.6,
"organization" : "THENLPER",
"parameters_m" : 335,
"pooling" : "Mean pooling + L2 normalization",
"sentiment_score" : 3,
"speed_category" : "Slow",
"style_tone_score" : 2.8,
"tense_gender_score" : 2.8,
"train_objective" : "Multi-stage contrastive; instruction signals (large)",
"word_order_score" : 3.3
},
{
"code_score" : 1.6,
"dates_score" : 2.6,
"description" : "Generalist encoder; balanced retrieval/STS/clustering.",
"dimension_category" : "Medium (385-768)",
"domains_caution" : "Cross-lingual retrieval",
"domains_good" : "General English semantic search, clustering",
"embedding_dim" : 768,
"family" : "UAE",
"json_score" : 1.8,
"languages" : "English",
"license" : "Apache-2.0",
"long_context_flag" : "No",
"long_context_score" : 2.7,
"max_seq_len" : 512,
"memory_category" : "Small (<0.5GB)",
"memory_fp16_gb" : 0.218,
"memory_fp32_gb" : 0.436,
"memory_norm_fp16_0_1" : 0.182,
"model_name" : "WhereIsAI/UAE-Small-V1",
"multilingual_score" : 1.5,
"named_entity_score" : 3,
"negation_score" : 2.6,
"notes" : "Not the strongest for long-doc retrieval.",
"numbers_score" : 2.6,
"organization" : "WhereIsAI",
"parameters_m" : 109,
"pooling" : "Mean pooling + L2 normalization",
"sentiment_score" : 3,
"speed_category" : "Medium",
"style_tone_score" : 2.8,
"tense_gender_score" : 2.8,
"train_objective" : "Universal AnglE Embedding (instruction-style); robust across tasks",
"word_order_score" : 2.8
},
{
"code_score" : 1.6,
"dates_score" : 2.6,
"description" : "High accuracy; heavier latency/VRAM.",
"dimension_category" : "High (769-1024)",
"domains_caution" : "Low-power devices",
"domains_good" : "Enterprise RAG, curated knowledge bases",
"embedding_dim" : 1024,
"family" : "UAE",
"json_score" : 1.8,
"languages" : "English",
"license" : "Apache-2.0",
"long_context_flag" : "No",
"long_context_score" : 2.7,
"max_seq_len" : 512,
"memory_category" : "Medium (0.5-1.5GB)",
"memory_fp16_gb" : 0.67,
"memory_fp32_gb" : 1.34,
"memory_norm_fp16_0_1" : 0.6548,
"model_name" : "WhereIsAI/UAE-Large-V1",
"multilingual_score" : 1.5,
"named_entity_score" : 3,
"negation_score" : 3.2,
"notes" : "Not suitable for edge or CPU-only devices.",
"numbers_score" : 2.6,
"organization" : "WhereIsAI",
"parameters_m" : 335,
"pooling" : "Mean pooling + L2 normalization",
"sentiment_score" : 3,
"speed_category" : "Slow",
"style_tone_score" : 2.8,
"tense_gender_score" : 2.8,
"train_objective" : "UAE large; improved retrieval & clustering",
"word_order_score" : 2.8
},
{
"code_score" : 1.6,
"dates_score" : 2.6,
"description" : "Competitive to BGE-base with improvements from better negative mining.",
"dimension_category" : "Medium (385-768)",
"domains_caution" : "Multilingual, long noisy logs",
"domains_good" : "Search/RAG on clean English corpora",
"embedding_dim" : 768,
"family" : "GIST",
"json_score" : 1.8,
"languages" : "English",
"license" : "Apache-2.0",
"long_context_flag" : "No",
"long_context_score" : 2.7,
"max_seq_len" : 512,
"memory_category" : "Small (<0.5GB)",
"memory_fp16_gb" : 0.218,
"memory_fp32_gb" : 0.436,
"memory_norm_fp16_0_1" : 0.182,
"model_name" : "avsolatorio/GIST-Embedding-v0",
"multilingual_score" : 1.5,
"named_entity_score" : 3,
"negation_score" : 3,
"notes" : "Heavier than MiniLM-class; English-centric.",
"numbers_score" : 2.6,
"organization" : "avsolatorio",
"parameters_m" : 109,
"pooling" : "Mean pooling + L2 normalization",
"sentiment_score" : 3,
"speed_category" : "Medium",
"style_tone_score" : 2.8,
"tense_gender_score" : 2.8,
"train_objective" : "Contrastive with in-sample hard negatives (GIST)",
"word_order_score" : 2.8
},
{
"code_score" : 1.6,
"dates_score" : 2.6,
"description" : "Tiny GIST variant; great speed with reasonable quality.",
"dimension_category" : "Small (≤384)",
"domains_caution" : "Deep domain retrieval",
"domains_good" : "Edge devices, chat memory, tag/cluster",
"embedding_dim" : 384,
"family" : "GIST",
"json_score" : 1.8,
"languages" : "English",
"license" : "Apache-2.0",
"long_context_flag" : "No",
"long_context_score" : 2.7,
"max_seq_len" : 512,
"memory_category" : "Small (<0.5GB)",
"memory_fp16_gb" : 0.066,
"memory_fp32_gb" : 0.132,
"memory_norm_fp16_0_1" : 0.023,
"model_name" : "avsolatorio/GIST-small-Embedding-v0",
"multilingual_score" : 1.5,
"named_entity_score" : 3,
"negation_score" : 2.6,
"notes" : "Lower ceiling for nuanced reasoning.",
"numbers_score" : 2.6,
"organization" : "avsolatorio",
"parameters_m" : 33,
"pooling" : "Mean pooling + L2 normalization",
"sentiment_score" : 3,
"speed_category" : "Fast",
"style_tone_score" : 2.8,
"tense_gender_score" : 2.8,
"train_objective" : "As above (small backbone) with hard negatives",
"word_order_score" : 2.8
},
{
"code_score" : 1.6,
"dates_score" : 2.6,
"description" : "Very small & fast; performs close to bigger models on some tasks.",
"dimension_category" : "Small (≤384)",
"domains_caution" : "Complex, under-specified queries",
"domains_good" : "Fast semantic dedup/search",
"embedding_dim" : 384,
"family" : "NoInstruct",
"json_score" : 1.8,
"languages" : "English",
"license" : "Apache-2.0",
"long_context_flag" : "No",
"long_context_score" : 2.7,
"max_seq_len" : 512,
"memory_category" : "Small (<0.5GB)",
"memory_fp16_gb" : 0.066,
"memory_fp32_gb" : 0.132,
"memory_norm_fp16_0_1" : 0.023,
"model_name" : "avsolatorio/NoInstruct-small-Embedding-v0",
"multilingual_score" : 1.5,
"named_entity_score" : 3,
"negation_score" : 2.6,
"notes" : "Less robust on complex queries without instructions.",
"numbers_score" : 2.6,
"organization" : "avsolatorio",
"parameters_m" : 33,
"pooling" : "Last-token pooling + L2 normalization",
"sentiment_score" : 3,
"speed_category" : "Fast",
"style_tone_score" : 2.8,
"tense_gender_score" : 2.8,
"train_objective" : "Non-instruction small contrastive embedding",
"word_order_score" : 2.8
},
{
"code_score" : 1.6,
"dates_score" : 2.6,
"description" : "Long context (8k), multilingual coverage, strong retrieval accuracy.",
"dimension_category" : "Medium (385-768)",
"domains_caution" : "Low-latency mobile/edge",
"domains_good" : "Large-scale RAG, multilingual knowledge bases, long documents",
"embedding_dim" : 768,
"family" : "Nomic",
"json_score" : 1.8,
"languages" : "Multilingual",
"license" : "Apache-2.0",
"long_context_flag" : "Yes",
"long_context_score" : 4,
"max_seq_len" : 8192,
"memory_category" : "Large (≥1.5GB)",
"memory_fp16_gb" : 1,
"memory_fp32_gb" : 2,
"memory_norm_fp16_0_1" : 1,
"model_name" : "nomic-ai/nomic-embed-text-v1",
"multilingual_score" : 3.8,
"named_entity_score" : 3,
"negation_score" : 3.3,
"notes" : "Heavier latency/VRAM; not ideal for edge.",
"numbers_score" : 2.6,
"organization" : "Nomic AI",
"parameters_m" : 500,
"pooling" : "Mean pooling + L2 normalization",
"sentiment_score" : 3,
"speed_category" : "Slow",
"style_tone_score" : 2.8,
"tense_gender_score" : 2.8,
"train_objective" : "Long-context contrastive training; instruction prefixes for roles",
"word_order_score" : 2.8
},
{
"code_score" : 1.6,
"dates_score" : 2.6,
"description" : "High max length; good English retrieval; easy to run.",
"dimension_category" : "Medium (385-768)",
"domains_caution" : "Cross-lingual retrieval",
"domains_good" : "General English RAG/search with long pages",
"embedding_dim" : 768,
"family" : "Jina",
"json_score" : 1.8,
"languages" : "English",
"license" : "Apache-2.0",
"long_context_flag" : "No",
"long_context_score" : 3.4,
"max_seq_len" : 8192,
"memory_category" : "Small (<0.5GB)",
"memory_fp16_gb" : 0.218,
"memory_fp32_gb" : 0.436,
"memory_norm_fp16_0_1" : 0.182,
"model_name" : "jinaai/jina-embeddings-v2-base-en",
"multilingual_score" : 1.5,
"named_entity_score" : 3,
"negation_score" : 2.6,
"notes" : "Not multilingual; numbers/dates mediocre without rerank.",
"numbers_score" : 2.6,
"organization" : "Jina AI",
"parameters_m" : 109,
"pooling" : "Mean pooling + L2 normalization",
"sentiment_score" : 3,
"speed_category" : "Medium",
"style_tone_score" : 2.8,
"tense_gender_score" : 2.8,
"train_objective" : "Instruction-tuned contrastive embedding; long max length",
"word_order_score" : 2.8
},
{
"code_score" : 1.6,
"dates_score" : 2.6,
"description" : "Good speed/length balance for English web pages.",
"dimension_category" : "Medium (385-768)",
"domains_caution" : "Complex reasoning over numbers/dates",
"domains_good" : "Edge-ish setups with long text ingestion",
"embedding_dim" : 512,
"family" : "Jina",
"json_score" : 1.8,
"languages" : "English",
"license" : "Apache-2.0",
"long_context_flag" : "No",
"long_context_score" : 3.2,
"max_seq_len" : 8192,
"memory_category" : "Small (<0.5GB)",
"memory_fp16_gb" : 0.066,
"memory_fp32_gb" : 0.132,
"memory_norm_fp16_0_1" : 0.023,
"model_name" : "jinaai/jina-embeddings-v2-small-en",
"multilingual_score" : 1.5,
"named_entity_score" : 3,
"negation_score" : 2.6,
"notes" : "Less nuanced reasoning; English only.",
"numbers_score" : 2.6,
"organization" : "Jina AI",
"parameters_m" : 33,
"pooling" : "Mean pooling + L2 normalization",
"sentiment_score" : 3,
"speed_category" : "Fast",
"style_tone_score" : 2.8,
"tense_gender_score" : 2.8,
"train_objective" : "Small instruction-tuned embedding; long max length",
"word_order_score" : 2.8
},
{
"code_score" : 1.6,
"dates_score" : 2.6,
"description" : "Top-tier accuracy among compact LLM-free encoders.",
"dimension_category" : "High (769-1024)",
"domains_caution" : "Edge/CPU constraints",
"domains_good" : "Precision-oriented RAG, legal/finance support search",
"embedding_dim" : 1024,
"family" : "MXBAI",
"json_score" : 1.8,
"languages" : "English",
"license" : "Apache-2.0",
"long_context_flag" : "No",
"long_context_score" : 2.7,
"max_seq_len" : 512,
"memory_category" : "Medium (0.5-1.5GB)",
"memory_fp16_gb" : 0.67,
"memory_fp32_gb" : 1.34,
"memory_norm_fp16_0_1" : 0.6548,
"model_name" : "mixedbread-ai/mxbai-embed-large-v1",
"multilingual_score" : 1.5,
"named_entity_score" : 3,
"negation_score" : 3.2,
"notes" : "Heavier; 1024-D vectors.",
"numbers_score" : 3.1,
"organization" : "mixedbread-ai",
"parameters_m" : 335,
"pooling" : "Mean pooling + L2 normalization",
"sentiment_score" : 3,
"speed_category" : "Slow",
"style_tone_score" : 2.8,
"tense_gender_score" : 2.8,
"train_objective" : "Contrastive with loss improvements; instruction signals",
"word_order_score" : 2.8
},
{
"code_score" : 1.6,
"dates_score" : 2.6,
"description" : "Classic strong baseline; widely adopted.",
"dimension_category" : "Medium (385-768)",
"domains_caution" : "Very long documents",
"domains_good" : "General semantic search, clustering",
"embedding_dim" : 768,
"family" : "SBERT",
"json_score" : 1.8,
"languages" : "English",
"license" : "Apache-2.0",
"long_context_flag" : "No",
"long_context_score" : 2.7,
"max_seq_len" : 512,
"memory_category" : "Small (<0.5GB)",
"memory_fp16_gb" : 0.218,
"memory_fp32_gb" : 0.436,
"memory_norm_fp16_0_1" : 0.182,
"model_name" : "sentence-transformers/all-mpnet-base-v2",
"multilingual_score" : 1.5,
"named_entity_score" : 3,
"negation_score" : 2.6,
"notes" : "Moderate latency vs MiniLM; not long-context.",
"numbers_score" : 2.6,
"organization" : "SBERT",
"parameters_m" : 109,
"pooling" : "Mean pooling + L2 normalization",
"sentiment_score" : 3,
"speed_category" : "Medium",
"style_tone_score" : 2.8,
"tense_gender_score" : 2.8,
"train_objective" : "Siamese/bi-encoder fine-tuned for STS & retrieval (mpnet backbone)",
"word_order_score" : 3.1
},
{
"code_score" : 1.6,
"dates_score" : 2.6,
"description" : "Extremely fast & tiny vectors; great for scale.",
"dimension_category" : "Small (≤384)",
"domains_caution" : "Nuanced QA, long context",
"domains_good" : "High-QPS search, dedup, autocomplete, tag/cluster",
"embedding_dim" : 384,
"family" : "SBERT",
"json_score" : 1.8,
"languages" : "English",
"license" : "Apache-2.0",
"long_context_flag" : "No",
"long_context_score" : 2,
"max_seq_len" : 256,
"memory_category" : "Small (<0.5GB)",
"memory_fp16_gb" : 0.044,
"memory_fp32_gb" : 0.088,
"memory_norm_fp16_0_1" : 0,
"model_name" : "sentence-transformers/all-MiniLM-L6-v2",
"multilingual_score" : 1.5,
"named_entity_score" : 3,
"negation_score" : 2.2,
"notes" : "Lower accuracy on complex reasoning & long text.",
"numbers_score" : 2.6,
"organization" : "SBERT",
"parameters_m" : 22,
"pooling" : "Mean pooling + L2 normalization",
"sentiment_score" : 3,
"speed_category" : "Fast",
"style_tone_score" : 2.8,
"tense_gender_score" : 2.8,
"train_objective" : "Distilled MiniLM; contrastive fine-tuning for STS & retrieval",
"word_order_score" : 2.5
},
{
"code_score" : 1.6,
"dates_score" : 2.6,
"description" : "Tiny & fast; robust to paraphrase.",
"dimension_category" : "Small (≤384)",
"domains_caution" : "Numeric/date heavy queries",
"domains_good" : "Fast paraphrase search/dedup",
"embedding_dim" : 384,
"family" : "SBERT",
"json_score" : 1.8,
"languages" : "English",
"license" : "Apache-2.0",
"long_context_flag" : "No",
"long_context_score" : 2.7,
"max_seq_len" : 256,
"memory_category" : "Small (<0.5GB)",
"memory_fp16_gb" : 0.066,
"memory_fp32_gb" : 0.132,
"memory_norm_fp16_0_1" : 0.023,
"model_name" : "sentence-transformers/paraphrase-MiniLM-L12-v2",
"multilingual_score" : 1.5,
"named_entity_score" : 3,
"negation_score" : 2.6,
"notes" : "Limited on numerical/date reasoning.",
"numbers_score" : 2.6,
"organization" : "SBERT",
"parameters_m" : 33,
"pooling" : "Mean pooling + L2 normalization",
"sentiment_score" : 3,
"speed_category" : "Fast",
"style_tone_score" : 2.8,
"tense_gender_score" : 2.8,
"train_objective" : "Paraphrase-tuned MiniLM (contrastive)",
"word_order_score" : 2.8
},
{
"code_score" : 1.6,
"dates_score" : 2.6,
"description" : "Light multilingual encoder; good for clustering & search.",
"dimension_category" : "Medium (385-768)",
"domains_caution" : "High-precision multilingual RAG",
"domains_good" : "Cross-lingual clustering/search (small corpora)",
"embedding_dim" : 512,
"family" : "SBERT",
"json_score" : 1.8,
"languages" : "Multilingual (15+)",
"license" : "Apache-2.0",
"long_context_flag" : "No",
"long_context_score" : 2.7,
"max_seq_len" : 512,
"memory_category" : "Small (<0.5GB)",
"memory_fp16_gb" : 0.132,
"memory_fp32_gb" : 0.264,
"memory_norm_fp16_0_1" : 0.0921,
"model_name" : "sentence-transformers/distiluse-base-multilingual-cased-v1",
"multilingual_score" : 3,
"named_entity_score" : 3,
"negation_score" : 2.6,
"notes" : "Lower accuracy than larger multilingual models.",
"numbers_score" : 2.6,
"organization" : "SBERT",
"parameters_m" : 66,
"pooling" : "Mean pooling + L2 normalization",
"sentiment_score" : 3,
"speed_category" : "Medium",
"style_tone_score" : 2.8,
"tense_gender_score" : 2.8,
"train_objective" : "Multilingual knowledge distillation; paraphrase-tuned",
"word_order_score" : 2.8
},
{
"code_score" : 1.6,
"dates_score" : 2.6,
"description" : "Very lightweight multilingual semantic search.",
"dimension_category" : "Small (≤384)",
"domains_caution" : "Complex multilingual QA",
"domains_good" : "Cross-lingual search on short text",
"embedding_dim" : 384,
"family" : "SBERT",
"json_score" : 1.8,
"languages" : "Multilingual (50+)",
"license" : "Apache-2.0",
"long_context_flag" : "No",
"long_context_score" : 2.7,
"max_seq_len" : 512,
"memory_category" : "Small (<0.5GB)",
"memory_fp16_gb" : 0.066,
"memory_fp32_gb" : 0.132,
"memory_norm_fp16_0_1" : 0.023,
"model_name" : "sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2",
"multilingual_score" : 3.2,
"named_entity_score" : 3,
"negation_score" : 2.6,
"notes" : "Weaker on harder queries; long context; numeric reasoning.",
"numbers_score" : 2.6,
"organization" : "SBERT",
"parameters_m" : 33,
"pooling" : "Mean pooling + L2 normalization",
"sentiment_score" : 3,
"speed_category" : "Fast",
"style_tone_score" : 2.8,
"tense_gender_score" : 2.8,
"train_objective" : "Paraphrase-tuned multilingual MiniLM",
"word_order_score" : 2.8
},
{
"code_score" : 1.6,
"dates_score" : 2.6,
"description" : "Good Chinese semantic search at small footprint.",
"dimension_category" : "Medium (385-768)",
"domains_caution" : "Long Chinese books; numeric/date QA",
"domains_good" : "Chinese FAQ/search, social comments",
"embedding_dim" : 512,
"family" : "m3e",
"json_score" : 1.8,
"languages" : "Chinese(+EN)",
"license" : "Apache-2.0",
"long_context_flag" : "No",
"long_context_score" : 2.7,
"max_seq_len" : 512,
"memory_category" : "Small (<0.5GB)",
"memory_fp16_gb" : 0.066,
"memory_fp32_gb" : 0.132,
"memory_norm_fp16_0_1" : 0.023,
"model_name" : "moka-ai/m3e-small",
"multilingual_score" : 2.5,
"named_entity_score" : 3,
"negation_score" : 2.6,
"notes" : "English performance lower; long docs weaker.",
"numbers_score" : 2.6,
"organization" : "moka-ai",
"parameters_m" : 33,
"pooling" : "Mean pooling + L2 normalization",
"sentiment_score" : 3,
"speed_category" : "Fast",
"style_tone_score" : 2.8,
"tense_gender_score" : 2.8,
"train_objective" : "Chinese-centric contrastive retriever (m3e)",
"word_order_score" : 2.8
},
{
"code_score" : 1.6,
"dates_score" : 2.6,
"description" : "Stronger Chinese retrieval; moderate resources.",
"dimension_category" : "Medium (385-768)",
"domains_caution" : "Cross-lingual alignment",
"domains_good" : "Chinese RAG/search",
"embedding_dim" : 768,
"family" : "m3e",
"json_score" : 1.8,
"languages" : "Chinese(+EN)",
"license" : "Apache-2.0",
"long_context_flag" : "No",
"long_context_score" : 2.7,
"max_seq_len" : 512,
"memory_category" : "Small (<0.5GB)",
"memory_fp16_gb" : 0.218,
"memory_fp32_gb" : 0.436,
"memory_norm_fp16_0_1" : 0.182,
"model_name" : "moka-ai/m3e-base",
"multilingual_score" : 2.8,
"named_entity_score" : 3,
"negation_score" : 2.6,
"notes" : "Weaker English nuance; limited long doc length.",
"numbers_score" : 2.6,
"organization" : "moka-ai",
"parameters_m" : 109,
"pooling" : "Mean pooling + L2 normalization",
"sentiment_score" : 3,
"speed_category" : "Medium",
"style_tone_score" : 2.8,
"tense_gender_score" : 2.8,
"train_objective" : "Chinese-centric contrastive retriever (base)",
"word_order_score" : 2.8
},
{
"code_score" : 1.6,
"dates_score" : 2.6,
"description" : "Popular Chinese SBERT-style model for general semantic similarity.",
"dimension_category" : "Medium (385-768)",
"domains_caution" : "Cross-lingual retrieval",
"domains_good" : "Chinese review sentiment, zh semantic search",
"embedding_dim" : 768,
"family" : "text2vec",
"json_score" : 1.8,
"languages" : "Chinese",
"license" : "MIT",
"long_context_flag" : "No",
"long_context_score" : 2.7,
"max_seq_len" : 512,
"memory_category" : "Small (<0.5GB)",
"memory_fp16_gb" : 0.218,
"memory_fp32_gb" : 0.436,
"memory_norm_fp16_0_1" : 0.182,
"model_name" : "shibing624/text2vec-base-chinese",
"multilingual_score" : 2,
"named_entity_score" : 3,
"negation_score" : 2.6,
"notes" : "Less robust outside zh; numbers/dates limited.",
"numbers_score" : 2.6,
"organization" : "shibing624",
"parameters_m" : 109,
"pooling" : "Mean pooling + L2 normalization",
"sentiment_score" : 3.5,
"speed_category" : "Medium",
"style_tone_score" : 2.8,
"tense_gender_score" : 2.8,
"train_objective" : "Chinese paraphrase/contrastive training",
"word_order_score" : 2.8
},
{
"code_score" : 1.6,
"dates_score" : 2.6,
"description" : "Strong base with high max length; competitive accuracy.",
"dimension_category" : "Medium (385-768)",
"domains_caution" : "Multilingual corpora",
"domains_good" : "Long-page English RAG/search",
"embedding_dim" : 768,
"family" : "Stella",
"json_score" : 1.8,
"languages" : "English",
"license" : "Apache-2.0",
"long_context_flag" : "No",
"long_context_score" : 3.5,
"max_seq_len" : 8192,
"memory_category" : "Small (<0.5GB)",
"memory_fp16_gb" : 0.22,
"memory_fp32_gb" : 0.44,
"memory_norm_fp16_0_1" : 0.1841,
"model_name" : "nfgrad/stella-base-en-v2",
"multilingual_score" : 1.5,
"named_entity_score" : 3,
"negation_score" : 2.6,
"notes" : "English-only; numeric/date still moderate.",
"numbers_score" : 2.6,
"organization" : "nfgrad",
"parameters_m" : 110,
"pooling" : "Mean pooling + L2 normalization",
"sentiment_score" : 3,
"speed_category" : "Medium",
"style_tone_score" : 2.8,
"tense_gender_score" : 2.8,
"train_objective" : "Stella base; instruction-style embedding optimized for retrieval/STS",
"word_order_score" : 2.8
},
{
"code_score" : 1.6,
"dates_score" : 2.6,
"description" : "Modern GTE flavor with strong retrieval; long max length.",
"dimension_category" : "Medium (385-768)",
"domains_caution" : "Cross-lingual applications",
"domains_good" : "Docs/wiki/blogs; support portals",
"embedding_dim" : 768,
"family" : "GTE",
"json_score" : 1.8,
"languages" : "English",
"license" : "Apache-2.0",
"long_context_flag" : "No",
"long_context_score" : 3.4,
"max_seq_len" : 8192,
"memory_category" : "Small (<0.5GB)",
"memory_fp16_gb" : 0.218,
"memory_fp32_gb" : 0.436,
"memory_norm_fp16_0_1" : 0.182,
"model_name" : "Alibaba-NLP/gte-base-en-v1.5",
"multilingual_score" : 1.5,
"named_entity_score" : 3,
"negation_score" : 2.6,
"notes" : "English-only; careful truncation still needed.",
"numbers_score" : 2.6,
"organization" : "Alibaba",
"parameters_m" : 109,
"pooling" : "Mean pooling + L2 normalization",
"sentiment_score" : 3,
"speed_category" : "Medium",
"style_tone_score" : 2.8,
"tense_gender_score" : 2.8,
"train_objective" : "Qwen-GTE variant; multi-stage contrastive",
"word_order_score" : 2.8
}
]
;
// Comprehensive glossary with aliases
const GLOSSARY = {
"semantic search": "Retrieval based on meaning rather than exact keyword overlap; queries and documents are mapped into the same vector space and nearest neighbors are returned.",
"dense retrieval": "Embeds queries and documents as dense vectors; relevance computed via vector similarity (cosine or dot product).",
"symmetric retrieval": "Same embedding behavior for query and document (good for STS/clustering).",
"asymmetric retrieval": "Encodes queries and documents differently (via instruction/prefixes) so short queries match long passages better.",
"hybrid retrieval": "Combines dense vectors with sparse lexical signals (e.g., BM25) to improve recall and robustness.",
"BM25": "A classic sparse term-weighting retrieval function used in lexical search; often combined with dense retrieval (hybrid).",
"contrastive learning": "Training that pulls together related text pairs and pushes apart unrelated pairs using a contrastive loss.",
"hard negatives": "Challenging non-matching examples (topic-similar but wrong) to make the model more discriminative.",
"instruction tuning": "Fine-tuning with instructions or role prefixes (e.g., 'query:', 'passage:') so the model adapts to task intent.",
"bi-encoder": "Two identical encoders (shared weights) independently embed query and document; enables efficient ANN search.",
"dual encoder": "Same as bi-encoder; two encoders may or may not share parameters.",
"cross-encoder": "A reranker that scores query+document jointly; slower but very accurate for top-k reranking.",
"pooling": "How token embeddings are reduced to a single vector: mean pooling averages tokens; CLS uses [CLS]; last-token uses the final token.",
"CLS": "Special classification token used by BERT-like models; often taken as the sentence representation (CLS pooling).",
"mean pooling": "Averaging token embeddings (with attention mask) to produce the sentence vector.",
"last-token pooling": "Uses the representation of the final (non-padding) token as the sentence vector.",
"L2 normalization": "Scales the final vector to unit length; makes cosine similarity equal dot product and stabilizes retrieval.",
"cosine similarity": "Similarity measure based on the angle between vectors; invariant to vector magnitude.",
"dot product": "Similarity via inner product; equals cosine similarity when vectors are L2-normalized.",
"negation": "Linguistic operator that inverts meaning (e.g., 'not good'); many embeddings blur negated vs. affirmative.",
"antonymy": "Opposite meanings (e.g., hot vs. cold); models can confuse antonyms if topical overlap dominates.",
"word order": "Sensitivity to token order; many encoders capture limited order and lean toward bag-of-concepts.",
"noisy": "Text with typos, OCR artifacts, boilerplate, markup, duplicated headers, tables/emoji or mixed languages; degrades embeddings.",
"numbers": "Numeric expressions, ranges and magnitudes; often weakly encoded without normalization or reranking.",
"dates": "Absolute or relative temporal expressions; many models encode them shallowly.",
"tense": "Verb tense (past/present/future); helps resolve temporal ordering of events.",
"gender": "Gendered references (he/she/they; gendered roles); some models conflate entities.",
"named entity": "People, places, organizations, products; fidelity matters for disambiguation and exact matches.",
"chunk boundary": "Where a long document is split into chunks; small shifts can change vectors and neighbors.",
"quantization": "Reduces numeric precision (e.g., int8/int4) to shrink memory and often speed up inference; small accuracy loss typical.",
"fp16": "Half precision floating point (16-bit); halves memory vs. fp32; widely available on recent GPUs.",
"fp32": "Single precision (32-bit) floating point; higher memory/compute cost but ubiquitous.",
"int8": "8-bit integer quantization; reduces memory and may speed CPU inference; minor accuracy loss typical.",
"int4": "4-bit quantization; very small memory but can reduce accuracy; support varies by runtime.",
"vector dimension": "Size of the output embedding (e.g., 384, 768, 1024); affects index size and search speed.",
"throughput": "Embeddings per second; depends on batch size, model size and hardware.",
"batch size": "Number of texts processed in parallel; larger batches increase GPU efficiency but need more memory.",
"context length": "Maximum tokens an input can contain; longer helps long documents but increases compute.",
"truncation": "Dropping tokens beyond the max length; can discard salient info if chunking is poor.",
"RAG": "Retrieval-Augmented Generation: fetch relevant context via embeddings and feed it to a generator (LLM).",
"STS": "Semantic Textual Similarity: benchmark/task measuring how similar two texts are in meaning.",
"clustering": "Grouping similar items (vectors) together; embeddings enable unsupervised grouping of related texts.",
"reranking": "Re-ordering retrieved candidates using a slower but more accurate model (e.g., cross-encoder).",
"ANN": "Approximate nearest neighbor search for fast vector retrieval (FAISS, HNSW, etc.).",
"FAISS": "Facebook AI Similarity Search; popular ANN library for vectors.",
"HNSW": "Hierarchical Navigable Small World graph; an ANN index structure.",
"VRAM": "GPU memory required to load and run a model; impacts batch size and latency.",
"tokenizer": "Module that splits text into tokens before encoding; common types include BPE, WordPiece and Unigram.",
"BPE": "Byte Pair Encoding tokenizer; merges frequent character pairs to form tokens.",
"WordPiece": "Subword tokenizer used in BERT; similar to BPE with different training objective.",
"Unigram": "Probabilistic subword tokenizer used in SentencePiece.",
"OOV": "Out-of-vocabulary: words not directly in the tokenizer's vocabulary; handled via subwords.",
"RoBERTa": "A robustly optimized BERT pretraining approach; backbone for many encoders.",
"BERT": "Bidirectional Encoder Representations from Transformers; common backbone for sentence embeddings.",
"MiniLM": "A lightweight Transformer distilled from larger models; fast encoders (e.g., MiniLM-L6).",
"MPNet": "Masked and Permuted Pretraining for Language Understanding; backbone used by all-mpnet-base-v2.",
"Qwen": "Backbone family used in some GTE models.",
"RoPE": "Rotary Positional Embeddings; helps extend context length in some models.",
"SwiGLU": "Gated activation function used by some long-context encoders.",
"parameters": "Trainable weights in the model; more parameters generally improve accuracy but increase memory and compute.",
"embedding dimension": "The size of the output vector; higher dimensions capture more nuance but increase storage and compute.",
"max sequence length": "Maximum number of tokens that can be processed without truncation.",
"train objective": "The loss function and training methodology used to learn the embeddings.",
"long context": "Ability to process and understand longer input sequences effectively.",
"multilingual": "Support for multiple languages beyond English.",
"sentiment": "Ability to capture emotional tone and opinion (positive, negative, neutral).",
"style": "Ability to capture writing style, formality, and tone differences.",
"CCPairs": "Colossal Clean Crawled Corpus pairs; a large dataset used for training E5 models.",
"MTEB": "Massive Text Embedding Benchmark; standard evaluation suite for embedding models."
};
const ALIASES = {
"semantic search": ["semantic-search"],
"dense retrieval": ["dense-retrieval", "dense vector retrieval"],
"asymmetric retrieval": ["asymmetric"],
"symmetric retrieval": ["symmetric"],
"hybrid retrieval": ["hybrid", "dense+sparse"],
"contrastive learning": ["contrastive"],
"hard negatives": ["hard-negative", "hard negs"],
"instruction tuning": ["instruction-tuned", "instruction", "prefix prompting", "prefixes"],
"bi-encoder": ["bi encoder", "biencoder"],
"cross-encoder": ["cross encoder", "crossencoder"],
"mean pooling": ["mean-pooling"],
"last-token pooling": ["last token pooling"],
"L2 normalization": ["L2-normalization", "L2 norm", "normalize"],
"named entity": ["NER", "entities"],
"quantization": ["quantized", "quantise", "quantize"],
"fp16": ["half precision"],
"fp32": ["single precision"],
"ANN": ["approximate nearest neighbour", "approximate nearest neighbor"],
"FAISS": ["faiss"],
"HNSW": ["hnsw"],
"recall@k": ["recall", "R@k"],
"precision": ["prec"],
"throughput": ["tput"],
"batch size": ["batchsize", "bs"],
"context length": ["max length", "max tokens", "seq length"],
"truncation": ["truncate", "trunc"]
};
// Column header tooltips
const COLUMN_TOOLTIPS = {
"Model": "The name of the embedding model.",
"Org": "The organization or team that created the model.",
"Family": "The model family or series (e.g., BGE, E5, GTE).",
"License": "The open source license under which the model is released.",
"Params (M)": "Number of trainable parameters in millions. More parameters generally improve accuracy but increase memory and compute.",
"Dim": "The size of the output vector (dimensions). Higher dimensions capture more nuance but increase storage and compute.",
"MaxLen": "Maximum sequence length (in tokens) that can be embedded without truncation.",
"Languages": "Languages supported by the model.",
"Memory FP32 (GB)": "Approximate memory required to load the full model in 32-bit floating point (GB).",
"Memory FP16 (GB)": "Approximate memory required in 16-bit floating point (GB).",
"Mem (norm 0–1)": "Normalized memory usage from 0 (smallest) to 1 (largest) based on FP16 memory.",
"Speed": "Relative encoding speed. Fast models embed text quickly; slow models are heavier.",
"Train Objective": "The primary training objective used to learn the embeddings, such as contrastive learning.",
"Pooling": "Pooling method used to aggregate token-level representations into a single embedding.",
"Description": "A brief description summarizing the model's design and intended use.",
"Notes": "Additional notes about strengths, weaknesses, or special requirements.",
"Domains (Good)": "Application domains where this model performs well.",
"Domains (Caution)": "Application domains where this model may struggle or perform poorly.",
"Long Ctx": "Whether the model has explicit long context support (8k+ tokens).",
"Negation (1–5)": "How well the model handles negation and logical inversions (1=weak, 5=strong).",
"Word order (1–5)": "Sensitivity to word order and syntax (1=weak, 5=strong).",
"Numbers (1–5)": "Ability to handle numeric expressions and ranges (1=weak, 5=strong).",
"Dates (1–5)": "Ability to handle temporal expressions and dates (1=weak, 5=strong).",
"Tense/Gender (1–5)": "Handling of verb tense and gendered references (1=weak, 5=strong).",
"Named entity (1–5)": "Fidelity for people, places, organizations, products (1=weak, 5=strong).",
"Long context (1–5)": "Performance on long documents and passages (1=weak, 5=strong).",
"Multilingual (1–5)": "Cross-lingual capability and performance (1=weak, 5=strong).",
"Code (1–5)": "Ability to handle programming code and technical syntax (1=weak, 5=strong).",
"JSON/Structured (1–5)": "Performance on structured data formats like JSON, XML (1=weak, 5=strong).",
"Sentiment (1–5)": "Ability to capture emotional tone and opinion (1=weak, 5=strong).",
"Style/Tone (1–5)": "Sensitivity to writing style, formality, and tone (1=weak, 5=strong)."
};
const TERM_INDEX = (() => {
const map = new Map();
for (const k of Object.keys(GLOSSARY)) map.set(k.toLowerCase(), k);
for (const [canon, arr] of Object.entries(ALIASES)) {
for (const a of arr) map.set(a.toLowerCase(), canon);
}
return map;
})();
function escapeReg(s) { return s.replace(/[-\/\\^$*+?.()|[\]{}]/g, '\\$&'); }
function annotateText(text) {
if (!text) return text;
let html = text;
const terms = Array.from(TERM_INDEX.keys()).sort((a,b) => b.length - a.length);
for (const key of terms) {
const canon = TERM_INDEX.get(key);
// Negative lookbehind to avoid matching inside existing span tags
const re = new RegExp('(?!.*<span[^>]*>.*?)\\b(' + escapeReg(key) + ')\\b(?![^<]*</span>)', 'gi');
html = html.replace(re, (m) => '<span class="term" data-term="' + canon.replace(/"/g, '&quot;') + '">' + m + '</span>');
}
return html;
}
function buildTable() {
const columns = [
{ title: "Model", data: "model_name" },
{ title: "Org", data: "organization" },
{ title: "Family", data: "family" },
{ title: "License", data: "license" },
{ title: "Params (M)", data: "parameters_m" },
{ title: "Dim", data: "embedding_dim" },
{ title: "MaxLen", data: "max_seq_len" },
{ title: "Languages", data: "languages" },
{ title: "Memory FP32 (GB)", data: "memory_fp32_gb" },
{ title: "Memory FP16 (GB)", data: "memory_fp16_gb" },
{ title: "Mem (norm 0–1)", data: "memory_norm_fp16_0_1" },
{ title: "Speed", data: "speed_category" },
{ title: "Train Objective", data: "train_objective", render: (d) => annotateText(d) },
{ title: "Pooling", data: "pooling", render: (d) => annotateText(d) },
{ title: "Description", data: "description", render: (d) => annotateText(d) },
{ title: "Notes", data: "notes", render: (d) => annotateText(d) },
{ title: "Domains (Good)", data: "domains_good", render: (d) => annotateText(d) },
{ title: "Domains (Caution)", data: "domains_caution", render: (d) => annotateText(d) },
{ title: "Long Ctx", data: "long_context_flag" },
{ title: "Negation (1–5)", data: "negation_score" },
{ title: "Word order (1–5)", data: "word_order_score" },
{ title: "Numbers (1–5)", data: "numbers_score" },
{ title: "Dates (1–5)", data: "dates_score" },
{ title: "Tense/Gender (1–5)", data: "tense_gender_score" },
{ title: "Named entity (1–5)", data: "named_entity_score" },
{ title: "Long context (1–5)", data: "long_context_score" },
{ title: "Multilingual (1–5)", data: "multilingual_score" },
{ title: "Code (1–5)", data: "code_score" },
{ title: "JSON/Structured (1–5)", data: "json_score" },
{ title: "Sentiment (1–5)", data: "sentiment_score" },
{ title: "Style/Tone (1–5)", data: "style_tone_score" }
];
const table = $('#tbl').DataTable({
data: DATA,
columns,
paging: false,
searching: true,
info: true,
order: [[0, 'asc']],
autoWidth: false,
drawCallback: function() { attachTooltips(); }
});
// Field toggles
const toggles = document.getElementById('fieldToggles');
table.columns().every(function(idx) {
const name = columns[idx].title;
const id = 'col_' + idx;
const wrap = document.createElement('label');
wrap.className = 'toggle-item';
wrap.innerHTML = `<input type="checkbox" id="${id}" checked> ${name}`;
wrap.querySelector('input').addEventListener('change', (e) => {
table.column(idx).visible(e.target.checked);
});
toggles.appendChild(wrap);
});
// Range sliders using noUiSlider - create without callbacks first
function makeRangeSlider(elId, min, max, start) {
const el = document.getElementById(elId);
noUiSlider.create(el, {
start: start || [min, max],
connect: true,
range: { min: min, max: max },
step: 1,
behaviour: 'tap-drag'
});
return el;
}
const paramsMin = Math.min(...DATA.map(d => d.parameters_m)),
paramsMax = Math.max(...DATA.map(d => d.parameters_m));
const dimMin = Math.min(...DATA.map(d => d.embedding_dim)),
dimMax = Math.max(...DATA.map(d => d.embedding_dim));
const maxlenMin = Math.min(...DATA.map(d => d.max_seq_len)),
maxlenMax = Math.max(...DATA.map(d => d.max_seq_len));
const memMin = 0, memMax = 1;
// Create all sliders first
window.pSl = makeRangeSlider('paramsSlider', paramsMin, paramsMax, [paramsMin, paramsMax]);
window.dSl = makeRangeSlider('dimSlider', dimMin, dimMax, [dimMin, dimMax]);
window.mlSl = makeRangeSlider('maxlenSlider', maxlenMin, maxlenMax, [maxlenMin, maxlenMax]);
window.mSl = makeRangeSlider('memSlider', 0, 1, [0, 1]);
// Then add callbacks after all are created
window.pSl.noUiSlider.on('update', function(values) {
document.getElementById('paramsMinLbl').textContent = 'Min: ' + Math.round(values[0]);
document.getElementById('paramsMaxLbl').textContent = 'Max: ' + Math.round(values[1]);
applyFilters();
});
window.dSl.noUiSlider.on('update', function(values) {
document.getElementById('dimMinLbl').textContent = 'Min: ' + Math.round(values[0]);
document.getElementById('dimMaxLbl').textContent = 'Max: ' + Math.round(values[1]);
applyFilters();
});
window.mlSl.noUiSlider.on('update', function(values) {
document.getElementById('maxlenMinLbl').textContent = 'Min: ' + Math.round(values[0]);
document.getElementById('maxlenMaxLbl').textContent = 'Max: ' + Math.round(values[1]);
applyFilters();
});
window.mSl.noUiSlider.on('update', function(values) {
document.getElementById('memMinLbl').textContent = 'Min: ' + Number(values[0]).toFixed(2);
document.getElementById('memMaxLbl').textContent = 'Max: ' + Number(values[1]).toFixed(2);
applyFilters();
});
function applyFilters() {
// Check if all sliders are initialized before proceeding
if (!window.pSl || !window.dSl || !window.mlSl || !window.mSl) return;
const [pMin, pMax] = window.pSl.noUiSlider.get().map(Number);
const [dMin, dMax] = window.dSl.noUiSlider.get().map(Number);
const [lMin, lMax] = window.mlSl.noUiSlider.get().map(Number);
const [mMin, mMax] = window.mSl.noUiSlider.get().map(Number);
const okSpeed = Array.from(document.querySelectorAll('.speedBox:checked')).map(x => x.value);
const picked = Array.from(document.querySelectorAll('.langBox:checked')).map(x => x.value);
// Clear existing custom filters
$.fn.dataTable.ext.search = $.fn.dataTable.ext.search.filter(f => !f.__custom);
const customFilter = function(settings, data) {
const params = parseFloat(data[4]), dim = parseFloat(data[5]), maxlen = parseFloat(data[6]), mem = parseFloat(data[10]);
const speed = data[11], langs = data[7];
let langOk = false;
for (const p of picked) {
if ((p === 'English' && /English/.test(langs)) ||
(p === 'Multilingual' && /Multilingual/.test(langs)) ||
(p === 'Chinese' && /Chinese/.test(langs))) {
langOk = true;
}
}
const negMin = parseFloat(document.getElementById('negMin').value) || 0;
const woMin = parseFloat(document.getElementById('woMin').value) || 0;
const numMin = parseFloat(document.getElementById('numMin').value) || 0;
const lcMin = parseFloat(document.getElementById('lcMin').value) || 0;
const neg = parseFloat(data[19]), wo = parseFloat(data[20]),
num = Math.min(parseFloat(data[21]), parseFloat(data[22])), lc = parseFloat(data[25]);
return (params >= pMin && params <= pMax) &&
(dim >= dMin && dim <= dMax) &&
(maxlen >= lMin && maxlen <= lMax) &&
(mem >= mMin && mem <= mMax) &&
okSpeed.includes(speed) && langOk &&
(neg >= negMin) && (wo >= woMin) && (num >= numMin) && (lc >= lcMin);
};
customFilter.__custom = true;
$.fn.dataTable.ext.search.push(customFilter);
table.draw();
}
// Initialize labels
document.getElementById('paramsMinLbl').textContent = 'Min: ' + paramsMin;
document.getElementById('paramsMaxLbl').textContent = 'Max: ' + paramsMax;
document.getElementById('dimMinLbl').textContent = 'Min: ' + dimMin;
document.getElementById('dimMaxLbl').textContent = 'Max: ' + dimMax;
document.getElementById('maxlenMinLbl').textContent = 'Min: ' + maxlenMin;
document.getElementById('maxlenMaxLbl').textContent = 'Max: ' + maxlenMax;
document.getElementById('memMinLbl').textContent = 'Min: 0.00';
document.getElementById('memMaxLbl').textContent = 'Max: 1.00';
// Attach filter listeners
document.querySelectorAll('.speedBox,.langBox,#negMin,#woMin,#numMin,#lcMin').forEach(el =>
el.addEventListener('input', applyFilters));
applyFilters();
// Export functionality
document.getElementById('exportCsv').addEventListener('click', function() {
const headers = table.columns().header().toArray().map(h => h.textContent);
const rows = table.rows({ filter: 'applied' }).data().toArray();
const csv = [headers.join(',')].concat(rows.map(r =>
headers.map((_, i) => ('' + r[i]).replace(/,/g, ';')).join(','))).join('\n');
const blob = new Blob([csv], { type: 'text/csv;charset=utf-8;' });
const url = URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = 'embedding_models_filtered.csv';
a.click();
URL.revokeObjectURL(url);
});
// Populate glossary
const gl = document.getElementById('glossary-list');
const entries = Object.entries(GLOSSARY).sort((a, b) => a[0].localeCompare(b[0]));
for (const [term, def] of entries) {
const dt = document.createElement('dt');
dt.textContent = term;
gl.appendChild(dt);
const dd = document.createElement('dd');
dd.textContent = def;
gl.appendChild(dd);
}
// Add header tooltips
$('#tbl thead th').each(function(index) {
const columnTitle = columns[index].title;
if (COLUMN_TOOLTIPS[columnTitle]) {
$(this).attr('title', COLUMN_TOOLTIPS[columnTitle]);
}
});
}
// Tooltip functionality
function attachTooltips() {
const tip = document.getElementById('tooltip');
// Term tooltips
document.querySelectorAll('.term').forEach(el => {
el.addEventListener('mouseenter', () => {
const k = (el.dataset.term || el.textContent).toLowerCase();
const canon = TERM_INDEX.get(k) || k;
const def = GLOSSARY[canon] || GLOSSARY[canon.toLowerCase()] || 'Definition not found';
tip.innerHTML = '<strong>' + (el.dataset.term || el.textContent) + '</strong><br/>' + def;
tip.style.display = 'block';
});
el.addEventListener('mousemove', (e) => {
tip.style.left = (e.pageX + 12) + 'px';
tip.style.top = (e.pageY + 12) + 'px';
});
el.addEventListener('mouseleave', () => tip.style.display = 'none');
});
// Header tooltips
document.querySelectorAll('#tbl thead th').forEach(el => {
el.addEventListener('mouseenter', () => {
const title = el.textContent;
const def = COLUMN_TOOLTIPS[title];
if (def) {
tip.innerHTML = '<strong>' + title + '</strong><br/>' + def;
tip.style.display = 'block';
}
});
el.addEventListener('mousemove', (e) => {
tip.style.left = (e.pageX + 12) + 'px';
tip.style.top = (e.pageY + 12) + 'px';
});
el.addEventListener('mouseleave', () => tip.style.display = 'none');
});
}
$(function() { buildTable(); });
</script>
</body>
</html>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment