333 lines
8.4 KiB
Markdown
333 lines
8.4 KiB
Markdown
# Critical Protocol Issue: Grok Encrypted Reasoning Causing UI Freeze
|
|
|
|
**Discovered**: 2025-11-11 (Second occurrence)
|
|
**Severity**: HIGH - Causes UI to appear "done" when still processing
|
|
**Model Affected**: x-ai/grok-code-fast-1
|
|
|
|
---
|
|
|
|
## 🔴 The Problem
|
|
|
|
### What User Experienced
|
|
|
|
1. **Normal streaming**: Text and reasoning flowing, UI updating
|
|
2. **Sudden stop**: All UI updates stop, appears "done"
|
|
3. **3-second freeze**: No blinking, no progress indication
|
|
4. **Sudden result**: ExitPlanMode tool call appears all at once
|
|
|
|
### Root Cause: Grok's Encrypted Reasoning
|
|
|
|
**Grok has TWO types of reasoning:**
|
|
|
|
#### Type 1: Visible Reasoning (FIXED ✅)
|
|
```json
|
|
{
|
|
"delta": {
|
|
"content": "",
|
|
"reasoning": "\n- The focus is on analyzing...", // ✅ We handle this
|
|
"reasoning_details": [...]
|
|
}
|
|
}
|
|
```
|
|
**Our fix:** Check `delta?.content || delta?.reasoning` ✅
|
|
|
|
#### Type 2: Encrypted Reasoning (NOT FIXED ❌)
|
|
```json
|
|
{
|
|
"delta": {
|
|
"content": "", // EMPTY
|
|
"reasoning": null, // NULL!
|
|
"reasoning_details": [{
|
|
"type": "reasoning.encrypted",
|
|
"data": "3i1VWVQdDqjts4+HVDHkk0B...", // Encrypted blob
|
|
"id": "rs_625a4689-e9e3-de62-2ac2-68eab172552c"
|
|
}]
|
|
}
|
|
}
|
|
```
|
|
|
|
**Problem:** Our current fix checks `delta?.content || delta?.reasoning`:
|
|
- `content` = `""` (empty) ❌
|
|
- `reasoning` = `null` ❌
|
|
- Result: **NO text_delta sent!** ❌
|
|
|
|
---
|
|
|
|
## 📊 Event Sequence from Logs
|
|
|
|
### From logs/claudish_2025-11-11_04-09-24.log
|
|
|
|
```
|
|
04:16:20.376Z - Last visible reasoning: "The focus is on analyzing..."
|
|
04:16:20.377Z - [Proxy] Sending content delta: "\n- The focus is..."
|
|
|
|
... 2.574 SECOND GAP - NO EVENTS SENT ...
|
|
|
|
04:16:22.951Z - Encrypted reasoning chunk received (reasoning: null)
|
|
04:16:22.952Z - Tool call starts: ExitPlanMode
|
|
04:16:22.957Z - finish_reason: "tool_calls"
|
|
04:16:23.029Z - Usage stats
|
|
04:16:23.030Z - Stream closed
|
|
```
|
|
|
|
**What our proxy sent to Claude Code:**
|
|
```
|
|
1. Text deltas (visible reasoning) ✅
|
|
2. ... NOTHING for 2.5+ seconds ... ❌❌❌
|
|
3. Tool call suddenly appears ✅
|
|
4. Message complete ✅
|
|
```
|
|
|
|
**Claude Code UI interpretation:**
|
|
- Last text_delta at 20.377
|
|
- No more deltas for 2.5 seconds → "Must be done"
|
|
- Hides progress indicators
|
|
- Tool call appears → "Show result"
|
|
|
|
User sees: **UI says "done" → 3 second freeze → sudden result**
|
|
|
|
---
|
|
|
|
## 🎯 The Fix
|
|
|
|
### Option 1: Detect Encrypted Reasoning (Quick Fix)
|
|
|
|
Check for `reasoning_details` array with encrypted data:
|
|
|
|
```typescript
|
|
// In streaming handler (around line 783)
|
|
const textContent = delta?.content || delta?.reasoning || "";
|
|
|
|
// NEW: Check for encrypted reasoning
|
|
const hasEncryptedReasoning = delta?.reasoning_details?.some(
|
|
(detail: any) => detail.type === "reasoning.encrypted"
|
|
);
|
|
|
|
if (textContent) {
|
|
// Send visible content
|
|
sendSSE("content_block_delta", {
|
|
index: textBlockIndex,
|
|
delta: { type: "text_delta", text: textContent }
|
|
});
|
|
} else if (hasEncryptedReasoning) {
|
|
// ✅ NEW: Send placeholder during encrypted reasoning
|
|
log(`[Proxy] Encrypted reasoning detected, sending placeholder`);
|
|
sendSSE("content_block_delta", {
|
|
index: textBlockIndex,
|
|
delta: { type: "text_delta", text: "." } // Keep UI alive
|
|
});
|
|
}
|
|
```
|
|
|
|
**Pros:**
|
|
- Simple, targeted fix
|
|
- Shows progress during encrypted reasoning
|
|
- Minimal code change
|
|
|
|
**Cons:**
|
|
- Adds visible dots to output (minor cosmetic issue)
|
|
- Grok-specific
|
|
|
|
### Option 2: Adaptive Ping Frequency (Better Solution)
|
|
|
|
Send pings more frequently when no content deltas are flowing:
|
|
|
|
```typescript
|
|
// Track last content delta time
|
|
let lastContentDeltaTime = Date.now();
|
|
let pingInterval: NodeJS.Timeout | null = null;
|
|
|
|
// Start adaptive ping
|
|
function startAdaptivePing() {
|
|
if (pingInterval) clearInterval(pingInterval);
|
|
|
|
pingInterval = setInterval(() => {
|
|
const timeSinceLastContent = Date.now() - lastContentDeltaTime;
|
|
|
|
// If no content for >1 second, ping more frequently
|
|
if (timeSinceLastContent > 1000) {
|
|
sendSSE("ping", { type: "ping" });
|
|
log(`[Proxy] Adaptive ping (${timeSinceLastContent}ms since last content)`);
|
|
}
|
|
}, 1000); // Check every 1 second
|
|
}
|
|
|
|
// In content delta handler
|
|
if (textContent) {
|
|
lastContentDeltaTime = Date.now(); // Update timestamp
|
|
sendSSE("content_block_delta", ...);
|
|
}
|
|
```
|
|
|
|
**Pros:**
|
|
- Universal solution (works for all models)
|
|
- No visible artifacts in output
|
|
- Keeps UI responsive during any quiet period
|
|
- Proper use of ping events
|
|
|
|
**Cons:**
|
|
- More complex implementation
|
|
- Additional ping overhead (minimal)
|
|
|
|
### Option 3: Hybrid Approach (Best)
|
|
|
|
Combine both: detect encrypted reasoning AND use adaptive pings:
|
|
|
|
```typescript
|
|
const textContent = delta?.content || delta?.reasoning || "";
|
|
const hasEncryptedReasoning = delta?.reasoning_details?.some(
|
|
(detail: any) => detail.type === "reasoning.encrypted"
|
|
);
|
|
|
|
if (textContent || hasEncryptedReasoning) {
|
|
lastContentDeltaTime = Date.now(); // Update activity timestamp
|
|
|
|
if (textContent) {
|
|
// Send visible content
|
|
sendSSE("content_block_delta", {
|
|
index: textBlockIndex,
|
|
delta: { type: "text_delta", text: textContent }
|
|
});
|
|
} else {
|
|
// Encrypted reasoning detected, log but don't send visible text
|
|
log(`[Proxy] Encrypted reasoning detected (keeping connection alive)`);
|
|
}
|
|
}
|
|
|
|
// Adaptive ping handles keep-alive during quiet periods
|
|
```
|
|
|
|
**Pros:**
|
|
- Best of both worlds
|
|
- No visible artifacts
|
|
- Universal solution
|
|
- Properly detects model-specific behavior
|
|
|
|
---
|
|
|
|
## 🧪 Test Case
|
|
|
|
### Reproduce the Issue
|
|
|
|
```bash
|
|
# Use Grok model with complex query
|
|
./dist/index.js "Analyze the Claudish codebase" --model x-ai/grok-code-fast-1
|
|
|
|
# Watch for:
|
|
1. Normal streaming starts ✅
|
|
2. Progress indicators active ✅
|
|
3. Sudden stop - appears "done" ❌
|
|
4. 2-3 second freeze ❌
|
|
5. Result suddenly appears ❌
|
|
```
|
|
|
|
### Expected After Fix
|
|
|
|
```bash
|
|
# Same command after fix
|
|
./dist/index.js "Analyze the Claudish codebase" --model x-ai/grok-code-fast-1
|
|
|
|
# Should see:
|
|
1. Normal streaming starts ✅
|
|
2. Progress indicators stay active ✅
|
|
3. Continuous pings during encrypted reasoning ✅
|
|
4. Smooth transition to result ✅
|
|
```
|
|
|
|
---
|
|
|
|
## 📝 Implementation Checklist
|
|
|
|
- [ ] Detect encrypted reasoning in `reasoning_details` array
|
|
- [ ] Implement adaptive ping frequency (1-second check interval)
|
|
- [ ] Track last content delta timestamp
|
|
- [ ] Send pings when >1 second since last content
|
|
- [ ] Test with Grok models
|
|
- [ ] Test with other models (ensure no regression)
|
|
- [ ] Update snapshot tests to handle ping patterns
|
|
- [ ] Document in README
|
|
|
|
---
|
|
|
|
## 🔍 Code Locations
|
|
|
|
### File: `src/proxy-server.ts`
|
|
|
|
**Line 783** - Content delta handler (needs update):
|
|
```typescript
|
|
// Current (partially fixed for visible reasoning)
|
|
const textContent = delta?.content || delta?.reasoning || "";
|
|
if (textContent) {
|
|
sendSSE("content_block_delta", ...);
|
|
}
|
|
|
|
// Needed: Add encrypted reasoning detection + adaptive ping
|
|
```
|
|
|
|
**Line 644-651** - Ping interval (needs enhancement):
|
|
```typescript
|
|
// Current: Fixed 15-second interval
|
|
const pingInterval = setInterval(() => {
|
|
sendSSE("ping", { type: "ping" });
|
|
}, 15000);
|
|
|
|
// Needed: Adaptive interval based on content flow
|
|
```
|
|
|
|
---
|
|
|
|
## 💡 Why This Happens
|
|
|
|
**Grok's Reasoning Model:**
|
|
1. **Visible reasoning**: Shows thinking process to user
|
|
2. **Encrypted reasoning**: Private reasoning, only for model
|
|
|
|
When doing complex analysis:
|
|
- Starts with visible reasoning ✅
|
|
- Switches to encrypted reasoning (for sensitive/internal logic)
|
|
- Encrypted reasoning can take 2-5 seconds ❌
|
|
- Then emits tool call
|
|
|
|
**Our proxy issue:**
|
|
- We handle visible reasoning ✅
|
|
- We ignore encrypted reasoning ❌
|
|
- Claude Code sees silence → assumes done ❌
|
|
|
|
---
|
|
|
|
## 📈 Impact
|
|
|
|
**Before Fix:**
|
|
- 2-5 second UI freeze during encrypted reasoning
|
|
- User confusion ("Is it stuck?")
|
|
- Appears broken/unresponsive
|
|
|
|
**After Fix:**
|
|
- Continuous progress indication
|
|
- Smooth streaming experience
|
|
- Professional UX
|
|
|
|
**Protocol Compliance:**
|
|
- Before: 95% (ignores encrypted reasoning periods)
|
|
- After: 98% (handles all reasoning types + adaptive keep-alive)
|
|
|
|
---
|
|
|
|
## 🔗 Related Issues
|
|
|
|
- **GROK_REASONING_PROTOCOL_ISSUE.md** - First discovery of visible reasoning
|
|
- This is the **second variant** of the same root cause
|
|
|
|
**Timeline:**
|
|
1. Nov 11, 03:59 - Found visible reasoning issue (186 chunks)
|
|
2. Nov 11, 04:16 - Found encrypted reasoning issue (2.5s freeze)
|
|
|
|
Both caused by Grok's non-standard reasoning fields!
|
|
|
|
---
|
|
|
|
**Status**: Ready to implement
|
|
**Priority**: HIGH (affects user experience significantly)
|
|
**Effort**: 15-30 minutes for Option 3 (hybrid approach)
|
|
**Recommended**: Option 3 (detect encrypted reasoning + adaptive ping)
|