Skip to content

Instantly share code, notes, and snippets.

@sam-saffron-jarvis

sam-saffron-jarvis/discourse_compare.html

Created March 29, 2026 06:13
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save sam-saffron-jarvis/892cddc873d6ae1dc7943a02ed144976 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save sam-saffron-jarvis/892cddc873d6ae1dc7943a02ed144976 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
discourse_compare.html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Discourse: Naive vs LLM-Enhanced Search</title>
<style>
*{box-sizing:border-box;margin:0;padding:0}
body{font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",sans-serif;background:#0f1117;color:#e0e0e0;padding:24px}
h1{color:#fff;margin-bottom:4px;font-size:22px}
.subtitle{color:#888;margin-bottom:20px;font-size:13px}
table{width:100%;border-collapse:collapse;font-size:13px}
th{background:#1e2130;color:#aaa;padding:10px 8px;text-align:left;border-bottom:1px solid #2a2d3a;position:sticky;top:0;z-index:2}
td{padding:10px 8px;border-bottom:1px solid #1e2130;vertical-align:top}
tr:hover>td{background:#161922}
.detail td{background:#0d0f18;padding:16px}
.detail-grid{display:flex;gap:16px;flex-wrap:wrap}
.detail-col{flex:1;min-width:200px}
.detail-col h4{font-size:12px;color:#aaa;margin-bottom:8px;font-weight:600}
.detail-col ul{list-style:none;padding:0}
.detail-col li{margin-bottom:6px;font-size:12px;line-height:1.4}
.detail-col li.none{color:#666;font-style:italic}
.detail-col a{color:#7aadff;text-decoration:none}
.detail-col a:hover{text-decoration:underline}
.meta{color:#666;font-size:11px;margin-left:4px}
.raw pre{font-size:11px;color:#aaa;white-space:pre-wrap;word-break:break-all;background:#0a0c12;padding:8px;border-radius:4px;max-height:220px;overflow-y:auto}
.badge{display:inline-block;padding:2px 8px;border-radius:10px;color:#fff;font-size:12px;font-weight:bold}
.ms{color:#666;font-size:11px}
.idx{color:#555;width:28px}
.query{max-width:220px;font-size:13px}
.label{color:#666}
.center{text-align:center;width:70px}
.lat-col{width:280px;font-size:11px}
.lat-row{display:flex;align-items:center;margin-bottom:4px;gap:6px}
.lat-label{width:68px;color:#888;flex-shrink:0}
.lat-track{flex:1;max-width:180px;height:8px;background:#1e2130;border-radius:4px;overflow:hidden}
.lat-fill{height:8px;border-radius:4px;transition:width .3s}
.lat-val{color:#ccc;white-space:nowrap}
.overhead{color:#e67e22}
.queries{font-size:12px;max-width:300px}
.queries code{background:#1e2130;padding:1px 5px;border-radius:3px;font-size:11px}
.reasoning{color:#777}
.expand{width:40px;text-align:center}
button{background:#2a2d3a;border:none;color:#aaa;padding:4px 8px;border-radius:4px;cursor:pointer;font-size:12px}
button:hover{background:#3a3d4a;color:#fff}
.summary{margin-bottom:20px;display:flex;gap:16px;flex-wrap:wrap}
.stat{background:#1e2130;border-radius:8px;padding:12px 18px}
.stat .val{font-size:26px;font-weight:bold;color:#7aadff}
.stat .lbl{font-size:12px;color:#888;margin-top:2px}
</style>
</head>
<body>
<h1>Discourse Search: Naive vs LLM-Enhanced</h1>
<p class="subtitle">gpt-oss:20b (proxy for Chroma Context-1) · meta.discourse.org · 2026-03-29 17:13 · 189 tok/s on RTX 4090</p>
<div class="summary">
<div class="stat"><div class="val">3/10</div><div class="lbl">queries improved</div></div>
<div class="stat"><div class="val">2</div><div class="lbl">rescued from 0 results</div></div>
<div class="stat"><div class="val">187 tok/s</div><div class="lbl">avg generation speed</div></div>
<div class="stat"><div class="val">1926ms</div><div class="lbl">avg TTFT</div></div>
<div class="stat"><div class="val">+2656ms</div><div class="lbl">avg latency overhead</div></div>
<div class="stat"><div class="val">3241ms</div><div class="lbl">avg end-to-end</div></div>
</div>
<table>
<thead>
<tr>
<th>#</th>
<th>Natural language query</th>
<th>Naive</th>
<th>Enhanced</th>
<th>Latency impact</th>
<th>Generated queries</th>
<th></th>
</tr>
</thead>
<tbody>
<tr>
<td class="idx">1</td>
<td class="query">find threads where someone solved Sidekiq memory leaks<br><small class="label">01 troubleshooting + solved state</small></td>
<td class="center"><span class="badge" style="background:#e74c3c">0</span><br><small class="ms">579ms</small></td>
<td class="center"><span class="badge" style="background:#2ecc71">25</span><br><small style="color:#2ecc71">+25</small></td>
<td class="lat-col">
<div class="lat-row">
<span class="lat-label">naive</span>
<div class="lat-track"><div class="lat-fill" style="width:11.58px;background:#555" title="579ms"></div></div>
<span class="lat-val">579ms</span>
</div>
<div class="lat-row">
<span class="lat-label">LLM+search</span>
<div class="lat-track"><div class="lat-fill" style="width:61.76px;background:#e67e22" title="3088ms"></div></div>
<span class="lat-val">3088ms</span>
</div>
<div class="lat-row">
<span class="lat-label">overhead</span>
<span class="lat-val overhead">+2509ms · TTFT 1524ms</span>
</div>
</td>
<td class="queries"><code>sidekiq memory leaks status:closed order:likes</code> · <code>sidekiq memory leaks status:closed</code><br><small class="reasoning">First query targets closed threads with high likes, second is broader closed threads.</small></td>
<td class="expand"><button onclick="toggle('d0')">▼</button></td>
</tr>
<tr class="detail" id="d0" style="display:none">
<td colspan="7">
<div class="detail-grid">
<div class="detail-col">
<h4>Naive: find threads where someone solved Sidekiq memory leaks</h4>
<ul><li class="none">No results</li></ul>
</div>
<div class="detail-col">
<h4>Enhanced Q1: sidekiq memory leaks status:closed order:likes</h4>
<ul><li><a href="https://meta.discourse.org/t/discourse-version-1-3/22046" target="_blank">Discourse Version 1.3</a> <span class="meta">0v</span></li><li><a href="https://meta.discourse.org/t/discourse-version-1-4/26488" target="_blank">Discourse Version 1.4</a> <span class="meta">0v</span></li><li><a href="https://meta.discourse.org/t/discourse-version-2-3/96690" target="_blank">Discourse Version 2.3</a> <span class="meta">0v</span></li><li><a href="https://meta.discourse.org/t/discourse-version-3-0/203005" target="_blank">Discourse Version 3.0</a> <span class="meta">0v</span></li><li><a href="https://meta.discourse.org/t/3-4-0-beta3-check-for-updates-on-what-s-new-page-filter-by-user-in-the-review-queue-threading-in-chat-dms-and-group-chats-and-more/343237" target="_blank">3.4.0.beta3: Check for updates on What’s New page, filter by user in the review queue, threading in Chat DMs and group chats, and more</a> <span class="meta">0v</span></li></ul>
</div><div class="detail-col">
<h4>Enhanced Q2: sidekiq memory leaks status:closed</h4>
<ul><li><a href="https://meta.discourse.org/t/phpbb-3-importer-old/17397" target="_blank">phpBB 3 Importer (old)</a> <span class="meta">0v</span></li><li><a href="https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory/280933" target="_blank">Sidekiq is consuming too much memory</a> <span class="meta">0v</span></li><li><a href="https://meta.discourse.org/t/discourse-version-1-3/22046" target="_blank">Discourse Version 1.3</a> <span class="meta">0v</span></li><li><a href="https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395" target="_blank">Sidekiq is consuming too much memory, restarting</a> <span class="meta">0v</span></li><li><a href="https://meta.discourse.org/t/private-category-no-preview-of-the-topic-on-link/50900" target="_blank">Private category: no preview of the topic on link</a> <span class="meta">0v</span></li></ul>
</div>
<div class="detail-col raw">
<h4>LLM raw output</h4>
<pre>{&quot;queries&quot;:[&quot;sidekiq memory leaks status:closed order:likes&quot;,&quot;sidekiq memory leaks status:closed&quot;],&quot;reasoning&quot;:&quot;First query targets closed threads with high likes, second is broader closed threads.&quot;}</pre>
</div>
</div>
</td>
</tr>
<tr>
<td class="idx">2</td>
<td class="query">what did Jeff Atwood say about subfolder installs<br><small class="label">02 person + topic</small></td>
<td class="center"><span class="badge" style="background:#e74c3c">0</span><br><small class="ms">562ms</small></td>
<td class="center"><span class="badge" style="background:#2ecc71">14</span><br><small style="color:#2ecc71">+14</small></td>
<td class="lat-col">
<div class="lat-row">
<span class="lat-label">naive</span>
<div class="lat-track"><div class="lat-fill" style="width:11.24px;background:#555" title="562ms"></div></div>
<span class="lat-val">562ms</span>
</div>
<div class="lat-row">
<span class="lat-label">LLM+search</span>
<div class="lat-track"><div class="lat-fill" style="width:75.0px;background:#e67e22" title="3750ms"></div></div>
<span class="lat-val">3750ms</span>
</div>
<div class="lat-row">
<span class="lat-label">overhead</span>
<span class="lat-val overhead">+3188ms · TTFT 2363ms</span>
</div>
</td>
<td class="queries"><code>@codinghorror &quot;subfolder installs&quot;</code> · <code>&quot;subfolder installs&quot; Jeff Atwood</code><br><small class="reasoning">Most specific query filters by Jeff Atwood’s username and the exact phrase. Fallback query searches for the phrase and h</small></td>
<td class="expand"><button onclick="toggle('d1')">▼</button></td>
</tr>
<tr class="detail" id="d1" style="display:none">
<td colspan="7">
<div class="detail-grid">
<div class="detail-col">
<h4>Naive: what did Jeff Atwood say about subfolder installs</h4>
<ul><li class="none">No results</li></ul>
</div>
<div class="detail-col">
<h4>Enhanced Q1: @codinghorror &quot;subfolder installs&quot;</h4>
<ul><li><a href="https://meta.discourse.org/t/reviewable-view-full-conversation-broken-on-subfolder-installs/121063" target="_blank">Reviewable View Full Conversation broken on subfolder installs</a> <span class="meta">0v</span></li><li><a href="https://meta.discourse.org/t/avatar-uploads-broken-on-subfolder-installs/100643" target="_blank">Avatar uploads broken on subfolder installs?</a> <span class="meta">0v</span></li><li><a href="https://meta.discourse.org/t/links-arent-showing-in-the-sidebar-or-topic-summaries-on-subfolder-installs/41787" target="_blank">Links aren't showing in the sidebar or topic summaries on subfolder installs</a> <span class="meta">0v</span></li><li><a href="https://meta.discourse.org/t/discourse-version-1-6/36402" target="_blank">Discourse Version 1.6</a> <span class="meta">0v</span></li><li><a href="https://meta.discourse.org/t/several-hard-apache-proxypass-rules-needed-under-path-user-avatar-picture-upload-problem/126722" target="_blank">Several hard Apache ProxyPass rules needed under path + User Avatar Picture upload problem</a> <span class="meta">0v</span></li></ul>
</div><div class="detail-col">
<h4>Enhanced Q2: &quot;subfolder installs&quot; Jeff Atwood</h4>
<ul><li class="none">No results</li></ul>
</div>
<div class="detail-col raw">
<h4>LLM raw output</h4>
<pre>{&quot;queries&quot;:[&quot;@codinghorror \&quot;subfolder installs\&quot;&quot;,&quot;\&quot;subfolder installs\&quot; Jeff Atwood&quot;],&quot;reasoning&quot;:&quot;Most specific query filters by Jeff Atwood’s username and the exact phrase. Fallback query searches for the phrase and his name without username filter.&quot;}</pre>
</div>
</div>
</td>
</tr>
<tr>
<td class="idx">3</td>
<td class="query">email bounce issues with Office 365<br><small class="label">03 product-specific bug</small></td>
<td class="center"><span class="badge" style="background:#2ecc71">4</span><br><small class="ms">628ms</small></td>
<td class="center"><span class="badge" style="background:#2ecc71">6</span><br><small style="color:#2ecc71">+2</small></td>
<td class="lat-col">
<div class="lat-row">
<span class="lat-label">naive</span>
<div class="lat-track"><div class="lat-fill" style="width:12.56px;background:#555" title="628ms"></div></div>
<span class="lat-val">628ms</span>
</div>
<div class="lat-row">
<span class="lat-label">LLM+search</span>
<div class="lat-track"><div class="lat-fill" style="width:78.3px;background:#e67e22" title="3915ms"></div></div>
<span class="lat-val">3915ms</span>
</div>
<div class="lat-row">
<span class="lat-label">overhead</span>
<span class="lat-val overhead">+3287ms · TTFT 2560ms</span>
</div>
</td>
<td class="queries"><code>&quot;email bounce&quot; &quot;Office 365&quot;</code> · <code>email bounce Office 365</code><br><small class="reasoning">Focused query uses quoted phrases for exact match; fallback removes quotes for broader search.</small></td>
<td class="expand"><button onclick="toggle('d2')">▼</button></td>
</tr>
<tr class="detail" id="d2" style="display:none">
<td colspan="7">
<div class="detail-grid">
<div class="detail-col">
<h4>Naive: email bounce issues with Office 365</h4>
<ul><li><a href="https://meta.discourse.org/t/reply-by-email-address-with-office-365/168583" target="_blank">'reply by email address' with Office 365</a> <span class="meta">0v</span></li><li><a href="https://meta.discourse.org/t/2018-the-year-in-review/105700" target="_blank">2018: The Year in Review</a> <span class="meta">0v</span></li><li><a href="https://meta.discourse.org/t/168583/1" target="_blank">[post @DavidO]</a> <span class="meta">Initially we were using Gmail to send all emails. Once it was up and rolling, we</span></li><li><a href="https://meta.discourse.org/t/105700/1" target="_blank">[post @system]</a> <span class="meta">2018's top Users Most Time Reading User Hours Read @Mittineague 429 @pfaffman 24</span></li></ul>
</div>
<div class="detail-col">
<h4>Enhanced Q1: &quot;email bounce&quot; &quot;Office 365&quot;</h4>
<ul><li class="none">No results</li></ul>
</div><div class="detail-col">
<h4>Enhanced Q2: email bounce Office 365</h4>
<ul><li><a href="https://meta.discourse.org/t/reply-by-email-address-with-office-365/168583" target="_blank">'reply by email address' with Office 365</a> <span class="meta">0v</span></li><li><a href="https://meta.discourse.org/t/2018-the-year-in-review/105700" target="_blank">2018: The Year in Review</a> <span class="meta">0v</span></li><li><a href="https://meta.discourse.org/t/mkjs-opinionated-discourse-deployment-configuration/193355" target="_blank">MKJ's Opinionated Discourse Deployment Configuration</a> <span class="meta">0v</span></li><li><a href="https://meta.discourse.org/t/168583/1" target="_blank">[post @DavidO]</a> <span class="meta">Initially we were using Gmail to send all emails. Once it was up and rolling, we</span></li><li><a href="https://meta.discourse.org/t/105700/1" target="_blank">[post @system]</a> <span class="meta">2018's top Users Most Time Reading User Hours Read @Mittineague 429 @pfaffman 24</span></li></ul>
</div>
<div class="detail-col raw">
<h4>LLM raw output</h4>
<pre>{&quot;queries&quot;:[&quot;\&quot;email bounce\&quot; \&quot;Office 365\&quot;&quot;,&quot;email bounce Office 365&quot;],&quot;reasoning&quot;:&quot;Focused query uses quoted phrases for exact match; fallback removes quotes for broader search.&quot;}</pre>
</div>
</div>
</td>
</tr>
<tr>
<td class="idx">4</td>
<td class="query">how to enable SSO with Google oauth<br><small class="label">04 feature how-to</small></td>
<td class="center"><span class="badge" style="background:#2ecc71">76</span><br><small class="ms">1014ms</small></td>
<td class="center"><span class="badge" style="background:#e74c3c">0</span><br><small style="color:#e74c3c">-76</small></td>
<td class="lat-col">
<div class="lat-row">
<span class="lat-label">naive</span>
<div class="lat-track"><div class="lat-fill" style="width:20.28px;background:#555" title="1014ms"></div></div>
<span class="lat-val">1014ms</span>
</div>
<div class="lat-row">
<span class="lat-label">LLM+search</span>
<div class="lat-track"><div class="lat-fill" style="width:55.56px;background:#e67e22" title="2778ms"></div></div>
<span class="lat-val">2778ms</span>
</div>
<div class="lat-row">
<span class="lat-label">overhead</span>
<span class="lat-val overhead">+1764ms · TTFT 1382ms</span>
</div>
</td>
<td class="queries"><code>&quot;enable SSO Google oauth&quot; #howto</code> · <code>&quot;SSO Google oauth&quot; #howto</code><br><small class="reasoning">Focused on enabling SSO with Google OAuth, using the howto category. Fallback removes the verb for broader matches.</small></td>
<td class="expand"><button onclick="toggle('d3')">▼</button></td>
</tr>
<tr class="detail" id="d3" style="display:none">
<td colspan="7">
<div class="detail-grid">
<div class="detail-col">
<h4>Naive: how to enable SSO with Google oauth</h4>
<ul><li><a href="https://meta.discourse.org/t/how-to-disable-google-oauth2-login-in-the-command-line/270135" target="_blank">How to disable google oauth2 login in the command line</a> <span class="meta">0v</span></li><li><a href="https://meta.discourse.org/t/sso-with-google-account/97019" target="_blank">Sso with google account</a> <span class="meta">0v</span></li><li><a href="https://meta.discourse.org/t/google-sso-oauth-signup-failure-when-using-fullscreen-login-signup/335077" target="_blank">Google SSO (OAuth) Signup failure when using fullscreen login/signup</a> <span class="meta">0v</span></li><li><a href="https://meta.discourse.org/t/social-login-funktioniert-nur-auf-desktop-aber-nicht-auf-mobilen-geraten-csrf-detected/389975" target="_blank">Social Login funktioniert nur auf Desktop, aber nicht auf mobilen Geräten (CSRF detected)</a> <span class="meta">0v</span></li><li><a href="https://meta.discourse.org/t/can-discourse-be-used-as-an-oauth-provider/43872" target="_blank">Can Discourse be used as an OAuth provider?</a> <span class="meta">0v</span></li></ul>
</div>
<div class="detail-col">
<h4>Enhanced Q1: &quot;enable SSO Google oauth&quot; #howto</h4>
<ul><li class="none">No results</li></ul>
</div><div class="detail-col">
<h4>Enhanced Q2: &quot;SSO Google oauth&quot; #howto</h4>
<ul><li class="none">No results</li></ul>
</div>
<div class="detail-col raw">
<h4>LLM raw output</h4>
<pre>{&quot;queries&quot;:[&quot;\&quot;enable SSO Google oauth\&quot; #howto&quot;,&quot;\&quot;SSO Google oauth\&quot; #howto&quot;],&quot;reasoning&quot;:&quot;Focused on enabling SSO with Google OAuth, using the howto category. Fallback removes the verb for broader matches.&quot;}</pre>
</div>
</div>
</td>
</tr>
<tr>
<td class="idx">5</td>
<td class="query">posts by sam saffron about performance profiling<br><small class="label">05 author + topic filter</small></td>
<td class="center"><span class="badge" style="background:#2ecc71">2</span><br><small class="ms">660ms</small></td>
<td class="center"><span class="badge" style="background:#e74c3c">0</span><br><small style="color:#e74c3c">-2</small></td>
<td class="lat-col">
<div class="lat-row">
<span class="lat-label">naive</span>
<div class="lat-track"><div class="lat-fill" style="width:13.2px;background:#555" title="660ms"></div></div>
<span class="lat-val">660ms</span>
</div>
<div class="lat-row">
<span class="lat-label">LLM+search</span>
<div class="lat-track"><div class="lat-fill" style="width:57.34px;background:#e67e22" title="2867ms"></div></div>
<span class="lat-val">2867ms</span>
</div>
<div class="lat-row">
<span class="lat-label">overhead</span>
<span class="lat-val overhead">+2207ms · TTFT 1549ms</span>
</div>
</td>
<td class="queries"><code>@sam-saffron &quot;performance profiling&quot;</code> · <code>&quot;performance profiling&quot; @sam-saffron</code><br><small class="reasoning">First query targets posts by user @sam-saffron containing the phrase &quot;performance profiling&quot;. Second query is a fallback</small></td>
<td class="expand"><button onclick="toggle('d4')">▼</button></td>
</tr>
<tr class="detail" id="d4" style="display:none">
<td colspan="7">
<div class="detail-grid">
<div class="detail-col">
<h4>Naive: posts by sam saffron about performance profiling</h4>
<ul><li><a href="https://meta.discourse.org/t/daily-summary-feedback/291853" target="_blank">Daily Summary Feedback</a> <span class="meta">0v</span></li><li><a href="https://meta.discourse.org/t/291853/129" target="_blank">[post @sam]</a> <span class="meta">yeah looks like a char limit that needs lifting … for sure. odd. Yeah that would</span></li></ul>
</div>
<div class="detail-col">
<h4>Enhanced Q1: @sam-saffron &quot;performance profiling&quot;</h4>
<ul><li class="none">No results</li></ul>
</div><div class="detail-col">
<h4>Enhanced Q2: &quot;performance profiling&quot; @sam-saffron</h4>
<ul><li class="none">No results</li></ul>
</div>
<div class="detail-col raw">
<h4>LLM raw output</h4>
<pre>{&quot;queries&quot;:[&quot;@sam-saffron \&quot;performance profiling\&quot;&quot;,&quot;\&quot;performance profiling\&quot; @sam-saffron&quot;],&quot;reasoning&quot;:&quot;First query targets posts by user @sam-saffron containing the phrase \&quot;performance profiling\&quot;. Second query is a fallback that swaps the order of the terms.&quot;}</pre>
</div>
</div>
</td>
</tr>
<tr>
<td class="idx">6</td>
<td class="query">S3 uploads broken after upgrading Discourse<br><small class="label">06 regression after upgrade</small></td>
<td class="center"><span class="badge" style="background:#e74c3c">0</span><br><small class="ms">487ms</small></td>
<td class="center"><span class="badge" style="background:#e74c3c">0</span><br><small style="color:#888">0</small></td>
<td class="lat-col">
<div class="lat-row">
<span class="lat-label">naive</span>
<div class="lat-track"><div class="lat-fill" style="width:9.74px;background:#555" title="487ms"></div></div>
<span class="lat-val">487ms</span>
</div>
<div class="lat-row">
<span class="lat-label">LLM+search</span>
<div class="lat-track"><div class="lat-fill" style="width:73.88px;background:#e67e22" title="3694ms"></div></div>
<span class="lat-val">3694ms</span>
</div>
<div class="lat-row">
<span class="lat-label">overhead</span>
<span class="lat-val overhead">+3207ms · TTFT 2423ms</span>
</div>
</td>
<td class="queries"><code>S3 uploads broken after upgrading Discourse status:open</code> · <code>S3 uploads broken after upgrading Discourse</code><br><small class="reasoning">First query targets open posts specifically about S3 uploads failing after a Discourse upgrade; fallback query searches </small></td>
<td class="expand"><button onclick="toggle('d5')">▼</button></td>
</tr>
<tr class="detail" id="d5" style="display:none">
<td colspan="7">
<div class="detail-grid">
<div class="detail-col">
<h4>Naive: S3 uploads broken after upgrading Discourse</h4>
<ul><li class="none">No results</li></ul>
</div>
<div class="detail-col">
<h4>Enhanced Q1: S3 uploads broken after upgrading Discourse status:open</h4>
<ul><li class="none">No results</li></ul>
</div><div class="detail-col">
<h4>Enhanced Q2: S3 uploads broken after upgrading Discourse</h4>
<ul><li class="none">No results</li></ul>
</div>
<div class="detail-col raw">
<h4>LLM raw output</h4>
<pre>{&quot;queries&quot;:[&quot;S3 uploads broken after upgrading Discourse status:open&quot;,&quot;S3 uploads broken after upgrading Discourse&quot;],&quot;reasoning&quot;:&quot;First query targets open posts specifically about S3 uploads failing after a Discourse upgrade; fallback query searches all posts containing the same keywords.&quot;}</pre>
</div>
</div>
</td>
</tr>
<tr>
<td class="idx">7</td>
<td class="query">rate limiting the Discourse API<br><small class="label">07 feature/config topic</small></td>
<td class="center"><span class="badge" style="background:#e74c3c">0</span><br><small class="ms">480ms</small></td>
<td class="center"><span class="badge" style="background:#e74c3c">0</span><br><small style="color:#888">0</small></td>
<td class="lat-col">
<div class="lat-row">
<span class="lat-label">naive</span>
<div class="lat-track"><div class="lat-fill" style="width:9.6px;background:#555" title="480ms"></div></div>
<span class="lat-val">480ms</span>
</div>
<div class="lat-row">
<span class="lat-label">LLM+search</span>
<div class="lat-track"><div class="lat-fill" style="width:73.56px;background:#e67e22" title="3678ms"></div></div>
<span class="lat-val">3678ms</span>
</div>
<div class="lat-row">
<span class="lat-label">overhead</span>
<span class="lat-val overhead">+3198ms · TTFT 2453ms</span>
</div>
</td>
<td class="queries"><code>rate limiting Discourse API</code> · <code>rate limiting Discourse API status:open</code><br><small class="reasoning">Removed question words, kept key terms. First query is the most specific. Second adds status:open as a fallback filter.</small></td>
<td class="expand"><button onclick="toggle('d6')">▼</button></td>
</tr>
<tr class="detail" id="d6" style="display:none">
<td colspan="7">
<div class="detail-grid">
<div class="detail-col">
<h4>Naive: rate limiting the Discourse API</h4>
<ul><li class="none">No results</li></ul>
</div>
<div class="detail-col">
<h4>Enhanced Q1: rate limiting Discourse API</h4>
<ul><li class="none">No results</li></ul>
</div><div class="detail-col">
<h4>Enhanced Q2: rate limiting Discourse API status:open</h4>
<ul><li class="none">No results</li></ul>
</div>
<div class="detail-col raw">
<h4>LLM raw output</h4>
<pre>{&quot;queries&quot;:[&quot;rate limiting Discourse API&quot;,&quot;rate limiting Discourse API status:open&quot;],&quot;reasoning&quot;:&quot;Removed question words, kept key terms. First query is the most specific. Second adds status:open as a fallback filter.&quot;}</pre>
</div>
</div>
</td>
</tr>
<tr>
<td class="idx">8</td>
<td class="query">postgres backup restore failed<br><small class="label">08 error + component</small></td>
<td class="center"><span class="badge" style="background:#e74c3c">0</span><br><small class="ms">478ms</small></td>
<td class="center"><span class="badge" style="background:#e74c3c">0</span><br><small style="color:#888">0</small></td>
<td class="lat-col">
<div class="lat-row">
<span class="lat-label">naive</span>
<div class="lat-track"><div class="lat-fill" style="width:9.56px;background:#555" title="478ms"></div></div>
<span class="lat-val">478ms</span>
</div>
<div class="lat-row">
<span class="lat-label">LLM+search</span>
<div class="lat-track"><div class="lat-fill" style="width:69.02px;background:#e67e22" title="3451ms"></div></div>
<span class="lat-val">3451ms</span>
</div>
<div class="lat-row">
<span class="lat-label">overhead</span>
<span class="lat-val overhead">+2973ms · TTFT 2214ms</span>
</div>
</td>
<td class="queries"><code>postgres backup restore failed status:closed</code> · <code>postgres backup restore</code><br><small class="reasoning">First query targets closed posts about a failed PostgreSQL backup restore, likely a solved issue. Second query is a broa</small></td>
<td class="expand"><button onclick="toggle('d7')">▼</button></td>
</tr>
<tr class="detail" id="d7" style="display:none">
<td colspan="7">
<div class="detail-grid">
<div class="detail-col">
<h4>Naive: postgres backup restore failed</h4>
<ul><li class="none">No results</li></ul>
</div>
<div class="detail-col">
<h4>Enhanced Q1: postgres backup restore failed status:closed</h4>
<ul><li class="none">No results</li></ul>
</div><div class="detail-col">
<h4>Enhanced Q2: postgres backup restore</h4>
<ul><li class="none">No results</li></ul>
</div>
<div class="detail-col raw">
<h4>LLM raw output</h4>
<pre>{&quot;queries&quot;:[&quot;postgres backup restore failed status:closed&quot;,&quot;postgres backup restore&quot;],&quot;reasoning&quot;:&quot;First query targets closed posts about a failed PostgreSQL backup restore, likely a solved issue. Second query is a broader fallback for any related posts.&quot;}</pre>
</div>
</div>
</td>
</tr>
<tr>
<td class="idx">9</td>
<td class="query">best plugins for community Q&amp;A forums<br><small class="label">09 recommendation + use case</small></td>
<td class="center"><span class="badge" style="background:#e74c3c">0</span><br><small class="ms">473ms</small></td>
<td class="center"><span class="badge" style="background:#e74c3c">0</span><br><small style="color:#888">0</small></td>
<td class="lat-col">
<div class="lat-row">
<span class="lat-label">naive</span>
<div class="lat-track"><div class="lat-fill" style="width:9.46px;background:#555" title="473ms"></div></div>
<span class="lat-val">473ms</span>
</div>
<div class="lat-row">
<span class="lat-label">LLM+search</span>
<div class="lat-track"><div class="lat-fill" style="width:47.4px;background:#e67e22" title="2370ms"></div></div>
<span class="lat-val">2370ms</span>
</div>
<div class="lat-row">
<span class="lat-label">overhead</span>
<span class="lat-val overhead">+1897ms · TTFT 1175ms</span>
</div>
</td>
<td class="queries"><code>best plugins community Q&amp;A forums order:likes</code> · <code>plugins community Q&amp;A forums</code><br><small class="reasoning">Focused query uses key terms and order:likes for top recommendations; fallback omits ordering for broader results</small></td>
<td class="expand"><button onclick="toggle('d8')">▼</button></td>
</tr>
<tr class="detail" id="d8" style="display:none">
<td colspan="7">
<div class="detail-grid">
<div class="detail-col">
<h4>Naive: best plugins for community Q&amp;A forums</h4>
<ul><li class="none">No results</li></ul>
</div>
<div class="detail-col">
<h4>Enhanced Q1: best plugins community Q&amp;A forums order:likes</h4>
<ul><li class="none">No results</li></ul>
</div><div class="detail-col">
<h4>Enhanced Q2: plugins community Q&amp;A forums</h4>
<ul><li class="none">No results</li></ul>
</div>
<div class="detail-col raw">
<h4>LLM raw output</h4>
<pre>{&quot;queries&quot;:[&quot;best plugins community Q&amp;A forums order:likes&quot;,&quot;plugins community Q&amp;A forums&quot;],&quot;reasoning&quot;:&quot;Focused query uses key terms and order:likes for top recommendations; fallback omits ordering for broader results&quot;}</pre>
</div>
</div>
</td>
</tr>
<tr>
<td class="idx">10</td>
<td class="query">GDPR data export bulk user anonymization<br><small class="label">10 compliance + feature</small></td>
<td class="center"><span class="badge" style="background:#e74c3c">0</span><br><small class="ms">490ms</small></td>
<td class="center"><span class="badge" style="background:#e74c3c">0</span><br><small style="color:#888">0</small></td>
<td class="lat-col">
<div class="lat-row">
<span class="lat-label">naive</span>
<div class="lat-track"><div class="lat-fill" style="width:9.8px;background:#555" title="490ms"></div></div>
<span class="lat-val">490ms</span>
</div>
<div class="lat-row">
<span class="lat-label">LLM+search</span>
<div class="lat-track"><div class="lat-fill" style="width:56.46px;background:#e67e22" title="2823ms"></div></div>
<span class="lat-val">2823ms</span>
</div>
<div class="lat-row">
<span class="lat-label">overhead</span>
<span class="lat-val overhead">+2333ms · TTFT 1622ms</span>
</div>
</td>
<td class="queries"><code>GDPR data export bulk user anonymization</code> · <code>GDPR data export anonymization</code><br><small class="reasoning">Removed question words, kept key terms. First query is most specific; second is a broader fallback.</small></td>
<td class="expand"><button onclick="toggle('d9')">▼</button></td>
</tr>
<tr class="detail" id="d9" style="display:none">
<td colspan="7">
<div class="detail-grid">
<div class="detail-col">
<h4>Naive: GDPR data export bulk user anonymization</h4>
<ul><li class="none">No results</li></ul>
</div>
<div class="detail-col">
<h4>Enhanced Q1: GDPR data export bulk user anonymization</h4>
<ul><li class="none">No results</li></ul>
</div><div class="detail-col">
<h4>Enhanced Q2: GDPR data export anonymization</h4>
<ul><li class="none">No results</li></ul>
</div>
<div class="detail-col raw">
<h4>LLM raw output</h4>
<pre>{&quot;queries&quot;:[&quot;GDPR data export bulk user anonymization&quot;,&quot;GDPR data export anonymization&quot;],&quot;reasoning&quot;:&quot;Removed question words, kept key terms. First query is most specific; second is a broader fallback.&quot;}</pre>
</div>
</div>
</td>
</tr>
</tbody>
</table>
<script>
function toggle(id){var e=document.getElementById(id);e.style.display=e.style.display==='none'?'table-row':'none'}
</script>
</body>
</html>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment