Java Development with AI Assistants: A 5-Way Comparison

Comparing GitHub Copilot CLI (Grep vs. LSP) with VS Code, IntelliJ Copilot, and IntelliJ ACP

Executive Summary

This report compares five approaches for AI-assisted Java development, evaluating code intelligence depth, accuracy, token/cost efficiency, and developer experience:

Copilot CLI — Default (grep/glob): Text-based pattern matching. Zero setup, but no understanding of Java semantics. Requires the most LLM round-trips and consumes the most tokens per navigation task.
Copilot CLI — Java LSP (Eclipse JDT LS): Adds semantic code intelligence to the CLI. The best balance of power and minimalism — 50-70% fewer premium requests and ~90% fewer tokens than the grep approach, with zero false positives on code navigation.
GitHub Copilot in VS Code (Agent Mode): Full IDE experience with access to the Java Language Server (Eclipse JDT LS running as a VS Code extension), workspace indexing, terminal, and extension tools. Deep context gathering via embeddings. Excellent Java support, but context is partially opaque to the user.
GitHub Copilot plugin in IntelliJ IDEA (Agent Mode): Leverages IntelliJ's PSI (Program Structure Interface) — the most advanced Java code intelligence engine available. Agent mode can use 60+ MCP tools. However, PSI access for the Copilot agent is mediated and not as deep as what JetBrains' own AI uses natively.
IntelliJ IDEA AI Assistant via ACP: The newest approach — uses the Agent Client Protocol to connect any ACP-compliant agent (Copilot, Claude, Gemini, Cursor, etc.) to IntelliJ's full PSI and MCP infrastructure. Deepest possible Java intelligence through native IDE APIs. Open standard, vendor-neutral.

Bottom line: For pure Java code intelligence quality, IntelliJ-based approaches (4 and 5) provide the deepest understanding thanks to PSI. For terminal-first developers who want efficiency without an IDE, Copilot CLI + LSP (approach 2) is the optimal choice — it brings 80-90% of the IDE benefit to the command line. The grep-only approach (1) should be considered a fallback, not a primary strategy for Java.

1. Architecture Comparison

┌──────────────────────────────────────────────────────────────────────────┐
│                         AI Model (Cloud LLM)                             │
└───────┬──────────┬──────────┬───────────────┬───────────────┬────────────┘
        │          │          │               │               │
   ┌────▼────┐ ┌───▼────┐ ┌──▼──────────┐ ┌──▼──────────┐ ┌──▼──────────┐
   │ CLI     │ │ CLI    │ │ VS Code     │ │ IntelliJ    │ │ IntelliJ   │
   │ Grep/   │ │ LSP    │ │ Extension   │ │ Copilot     │ │ ACP        │
   │ Glob    │ │ (JDT)  │ │ Host        │ │ Plugin      │ │ Framework  │
   └────┬────┘ └───┬────┘ └──┬──────────┘ └──┬──────────┘ └──┬──────────┘
        │          │          │               │               │
   ┌────▼────┐ ┌───▼─────┐ ┌─▼───────────┐ ┌─▼───────────┐ ┌─▼───────────┐
   │ripgrep  │ │Eclipse  │ │Eclipse JDT  │ │ IntelliJ    │ │ IntelliJ    │
   │(text)   │ │JDT LS   │ │LS + VS Code │ │ PSI Engine  │ │ PSI + MCP   │
   │         │ │(semantic)│ │Java Ext.    │ │ + MCP tools │ │ + ACP       │
   └─────────┘ └─────────┘ └─────────────┘ └─────────────┘ └─────────────┘

How Each Approach Accesses Code Intelligence

Approach	Code Intelligence Engine	Protocol	Depth
CLI Grep/Glob	None (text matching via ripgrep)	N/A	Surface-level
CLI + LSP	Eclipse JDT Language Server	LSP (stdio)	Compilation-level semantic
VS Code	Eclipse JDT LS via Java Extension Pack	LSP (in-process) + Embeddings	Compilation-level + workspace indexing
IntelliJ Copilot	IntelliJ PSI + MCP tools	Proprietary IDE APIs	Full AST + type system + build model
IntelliJ ACP	IntelliJ PSI + MCP + ACP	ACP (JSON-RPC) + MCP	Full AST + type system + build model

2. Code Intelligence Depth

2.1 CLI Grep/Glob — No Code Intelligence

The grep/glob tools use ripgrep for fast text pattern matching. They have zero understanding of Java:

Cannot resolve types, generics, or inheritance
Cannot distinguish method overloads
Matches in comments, strings, and documentation create false positives
No understanding of Maven/Gradle module boundaries

Source: copilot-agent-runtime/src/tools/grep.ts:63-227 — ripgrep-based implementation with regex pattern matching.

2.2 CLI + Eclipse JDT LS — LSP-Level Semantics

Eclipse JDT LS performs actual Java compilation and provides 9 semantic operations:

Go-to-definition, find references, implementations
Call hierarchy (incoming/outgoing calls)
Workspace symbol search, document outline
Hover with Javadoc and type signatures
Semantic rename across files

Key strength: Understands generics, annotations, Spring framework patterns, Maven/Gradle modules. Key limitation: Only helps with .java files. No understanding of XML configs, properties files, or non-Java code.

Source: copilot-agent-runtime/src/tools/lsp.ts:36-835 — 9 LSP operations with structured result formatting.

2.3 VS Code with Java Extension — LSP + Workspace Intelligence

VS Code uses the same Eclipse JDT LS engine as the CLI LSP approach, but with important enhancements:

Persistent indexing: The Java Language Server runs continuously in the Extension Host, maintaining a warm index of the entire workspace. No cold-start penalty on each query.
Embedding-based context selection: Copilot in VS Code uses an embedding model to find the most relevant code snippets for each prompt, even from files you haven't opened.
Extension tools: Agent mode can access any VS Code extension's contributed tools — including Java-specific extensions for build management, testing, and debugging.
Live editor buffers: Context comes from your actual editing state, including unsaved changes.
Built-in tools: File search, workspace search, terminal execution, and the full Java Language Server are all available as agent tools.

Key advantage over CLI LSP: The always-warm language server + embedding-based retrieval means VS Code can proactively include relevant context the LLM didn't explicitly ask for.

Sources: VS Code Copilot Agent Mode, VS Code Agent Tools, Copilot Embedding Model.

2.4 IntelliJ IDEA Copilot Plugin — PSI Engine

IntelliJ's Program Structure Interface (PSI) is fundamentally more powerful than LSP for Java:

Full AST with semantic resolution: PSI maintains a complete, live Abstract Syntax Tree of your entire project with all type references resolved. This is deeper than LSP, which only resolves on demand.
Build system awareness: Native Maven/Gradle integration means PSI understands your dependency tree, module boundaries, and effective classpath.
Annotation processing: PSI understands Spring annotations (@Service, @Autowired, @Transactional), Lombok, MapStruct, and other annotation processors at a deeper level.
Refactoring engine: IntelliJ's refactoring engine (accessible via MCP tools) can perform safe transformations that account for the full project model.
60+ MCP tools: Agent mode in IntelliJ exposes file operations, code intelligence (find usages, navigate to definition), testing, git, build actions, and terminal commands.

Key advantage over VS Code: PSI was built specifically for Java (and JVM languages) and provides deeper semantic understanding than the more generic LSP protocol. IntelliJ's "find usages" is more accurate than LSP's "find references" because it understands Java-specific patterns.

Key limitation: The Copilot plugin mediates PSI access — the agent may not get the full depth of what IntelliJ knows. Context window and token limits still apply.

Sources: GitHub Copilot IntelliJ Plugin, Agent Mode in JetBrains, Vibe Coding with Copilot in JetBrains.

2.5 IntelliJ ACP (AI Assistant) — Deepest Integration

The Agent Client Protocol (ACP) is the newest approach, co-developed by JetBrains and Zed. It standardizes how any AI agent communicates with any IDE — similar to how LSP standardized language support.

Full PSI exposure: ACP agents get access to IntelliJ's PSI tree, refactoring engine, code inspections, and navigation via MCP servers.
Agent-agnostic: You can use Copilot, Claude Code, Gemini CLI, Cursor, or any ACP-compliant agent. The code intelligence comes from the IDE, not the agent.
Bidirectional communication: The IDE can push context to the agent (open files, diagnostics, build results), and the agent can request specific information.
Native UX: Inline diffs, refactoring previews, and undo support work exactly as with IntelliJ's built-in AI Assistant.
No vendor lock-in: Switch agents without losing code intelligence capabilities.

Key advantage: The deepest possible Java intelligence because the agent uses IntelliJ's own native APIs rather than a translation layer. JetBrains AI Assistant built on this architecture reportedly achieves better Java context quality than the Copilot plugin.

Key limitation: Still maturing (ACP registry launched January 2026). Not all agents fully exploit the available PSI tools yet.

Sources: JetBrains ACP, ACP Agent Registry, Bring Your Own Agent, ACP Documentation.

3. Performance Comparison

Metric	CLI Grep	CLI + LSP	VS Code	IntelliJ Copilot	IntelliJ ACP
First query latency	~50ms	30-120s (indexing)	~5s (extension startup)	~3s (PSI ready)	~3s (PSI ready)
Subsequent queries	~50ms	100-500ms	50-200ms	50-200ms	50-200ms
Multi-step navigation	3-8 tool calls	1-2 tool calls	1 (IDE resolves internally)	1 (IDE resolves internally)	1 (IDE resolves internally)
Memory overhead	~0 MB	~1 GB (JDT LS)	~1.5 GB (VS Code + JDT LS)	~2 GB (IntelliJ)	~2 GB (IntelliJ)
Startup time	Instant	5-15s (JDT LS process)	10-30s (extension host)	Already running	Already running
Works offline	❌ (needs LLM)	❌ (needs LLM)	❌ (needs LLM)	❌ (needs LLM)	❌ (needs LLM)
Headless/SSH	✅	✅	⚠️ (Remote SSH ext.)	❌	❌

Notes:

IDE-based approaches (VS Code, IntelliJ) benefit from the language server/PSI being always warm — no cold start per query.
CLI LSP has a one-time indexing cost when JDT LS first starts for a project, but subsequent queries are fast.
The CLI approaches work natively over SSH/headless — a critical advantage for server-side development.

4. Token Usage & Premium Request Efficiency

This is where the approaches diverge significantly.

How Context Is Gathered

Approach	Context Strategy	Token Overhead
CLI Grep	LLM must search iteratively → reads full file contents → high token volume	Very High
CLI + LSP	LLM gets precise, structured results (locations, symbols, type info) → minimal tokens	Low
VS Code	IDE pre-selects relevant snippets via embeddings + LSP → medium-sized context window	Medium
IntelliJ Copilot	IDE gathers PSI-derived context → focused, relevant snippets	Medium-Low
IntelliJ ACP	IDE pushes rich PSI context + agent can query MCP tools → most efficient context	Low

Estimated Cost Per Navigation Task

Scenario: "Find all implementations of ChatModel and understand their constructors"

Approach	Tool Calls	Tokens Consumed	Premium Requests	LLM Round-trips
CLI Grep	6-10	12,000-15,000	3-5	3-5 turns
CLI + LSP	3-5	1,000-1,500	1-2	1-2 turns
VS Code	1-3	2,000-4,000	1-2	1-2 turns
IntelliJ Copilot	1-3	1,500-3,000	1-2	1-2 turns
IntelliJ ACP	1-3	1,500-3,000	1-2	1-2 turns

Key insight: The CLI + LSP approach is competitive with IDE approaches on token efficiency because LSP returns the same structured, minimal results. The main IDE advantage is that context pre-selection (via embeddings or PSI) means the model gets better background context without explicitly asking for it.

5. Accuracy of Code Navigation

Scenario	CLI Grep	CLI + LSP	VS Code	IntelliJ Copilot	IntelliJ ACP
Find class definition	⚠️ May match comments	✅ Exact	✅ Exact	✅ Exact	✅ Exact
Find all references	⚠️ False positives	✅ Accurate	✅ Accurate	✅ Most accurate	✅ Most accurate
Understand inheritance	❌ Manual grep	✅ goToImplementation	✅ Type hierarchy	✅ Full hierarchy	✅ Full hierarchy
Method overloading	❌ Cannot distinguish	✅ Signature-aware	✅ Signature-aware	✅ Full type resolution	✅ Full type resolution
Spring annotations	⚠️ Text match only	⚠️ Basic (depends on JDT LS Spring support)	⚠️ Basic	✅ Deep (IntelliJ Spring plugin)	✅ Deep (IntelliJ Spring plugin)
Cross-module refs	⚠️ Must search all dirs	✅ Maven/Gradle aware	✅ Maven/Gradle aware	✅ Native module support	✅ Native module support
Semantic rename	❌	✅ LSP rename	✅ LSP rename	✅ IntelliJ refactoring	✅ IntelliJ refactoring
Call hierarchy	❌	✅ incomingCalls/outgoingCalls	✅ Call hierarchy	✅ Call hierarchy	✅ Call hierarchy
Annotation processing (Lombok, etc.)	❌	⚠️ Limited	⚠️ Limited	✅ Full support	✅ Full support

IntelliJ's PSI advantage is most visible with:

Spring framework annotations and dependency injection
Lombok-generated code
Complex generics and type inference
Multi-module Maven/Gradle projects with complex dependency trees

6. Setup Complexity & Requirements

Approach	Setup	Requirements	Cost
CLI Grep	Zero	Copilot subscription	Free (included)
CLI + LSP	2 min (plugin install)	Java 21+, ~1 GB RAM	Free (included)
VS Code	5 min (install extensions)	VS Code + Java Extension Pack + ~1.5 GB RAM	Free (included)
IntelliJ Copilot	5 min (install plugin)	IntelliJ IDEA + Copilot subscription + ~2 GB RAM	IntelliJ license + Copilot sub
IntelliJ ACP	10 min (configure agent)	IntelliJ 2025.3+ + AI Assistant + ACP agent	IntelliJ license + optional AI sub

7. Pros & Cons Summary

CLI Grep/Glob

✅ Pros	❌ Cons
Zero setup, works anywhere	No Java semantics
Fast one-off text searches	Most LLM round-trips
Works over SSH/headless	Highest token consumption
No memory overhead	False positives in results

CLI + Eclipse JDT LS (LSP)

✅ Pros	❌ Cons
Semantic Java navigation	Requires Java 21+ & 1 GB RAM
50-70% fewer premium requests vs grep	30-120s initial indexing
Works over SSH/headless	Experimental feature
Structured, minimal token output	Only helps with .java files
Same engine as VS Code Java support

VS Code (Agent Mode)

✅ Pros	❌ Cons
Full IDE experience	Heavier than CLI (~1.5 GB)
Embedding-based context selection	Less Java-specialized than IntelliJ
Rich extension ecosystem	Cannot match IntelliJ's Spring/Lombok depth
Built-in terminal + preview	Requires desktop or Remote SSH
Always-warm language server

IntelliJ Copilot Plugin (Agent Mode)

✅ Pros	❌ Cons
Deepest Java understanding (PSI)	Requires IntelliJ license
Best Spring framework support	~2 GB memory overhead
60+ MCP tools available	Desktop only
Native refactoring engine	Copilot's PSI access is mediated
Build system-aware context

IntelliJ ACP (AI Assistant)

✅ Pros	❌ Cons
Full PSI exposure to agents	Newest approach (still maturing)
Agent-agnostic (use any LLM provider)	Requires IntelliJ 2025.3+
Open standard (no vendor lock-in)	Not all agents exploit full PSI depth yet
Native UX (diffs, undo, previews)	Requires IntelliJ license
Bidirectional IDE↔Agent communication

8. When to Use Which

Scenario	Best Approach
Quick one-off text search	CLI Grep
Terminal-first Java development (SSH, servers)	CLI + LSP
Full-stack development (Java + JS + CSS)	VS Code
Enterprise Spring Boot application	IntelliJ Copilot or ACP
Want to switch between AI providers freely	IntelliJ ACP
CI/CD or constrained environments	CLI Grep
Budget-conscious (minimize premium requests)	CLI + LSP
Maximum Java code intelligence quality	IntelliJ ACP
Remote development over SSH	CLI + LSP

9. Updated Executive Summary

Five approaches exist for AI-assisted Java development, each representing a different trade-off between code intelligence depth, cost efficiency, and setup complexity:

For terminal-first developers: The Copilot CLI + Java LSP approach is the clear winner. It brings compilation-level Java understanding to the command line with 50-70% fewer premium requests than the grep-only approach. Setup takes 2 minutes (copilot plugin install jdubois/java-lsp-eclipse-jdt-ls), and it works natively over SSH. The grep/glob fallback remains available for non-Java files.

For VS Code users: The Java Extension Pack gives you the same JDT LS engine as the CLI LSP approach, plus workspace embeddings and extension tools. It's the best all-round option for mixed-language projects but slightly less deep than IntelliJ for complex Spring/Java Enterprise work.

For IntelliJ IDEA users: You get the deepest Java code intelligence available anywhere, thanks to PSI. The Copilot plugin with Agent Mode is excellent. For maximum flexibility, the ACP approach lets you use any AI agent while retaining IntelliJ's full code intelligence — this is the most future-proof architecture.

The grep-only CLI approach should be treated as a fallback, not a primary strategy for Java development. It requires 3-5× more premium requests and produces inferior results compared to any approach with code intelligence.

Approach	Code Intelligence	Cost Efficiency	Setup	Best For
CLI Grep	⭐	⭐	⭐⭐⭐⭐⭐	Quick text searches
CLI + LSP	⭐⭐⭐⭐	⭐⭐⭐⭐⭐	⭐⭐⭐⭐	Terminal-first Java dev
VS Code	⭐⭐⭐⭐	⭐⭐⭐⭐	⭐⭐⭐⭐	Full-stack development
IntelliJ Copilot	⭐⭐⭐⭐⭐	⭐⭐⭐⭐	⭐⭐⭐	Enterprise Java
IntelliJ ACP	⭐⭐⭐⭐⭐	⭐⭐⭐⭐	⭐⭐⭐	Max intelligence + flexibility

Confidence Assessment

Claim	Confidence	Basis
CLI tool priority order and LSP integration	High	Direct source code analysis of copilot-agent-runtime
CLI LSP token savings (50-90%)	Medium-High	Based on output format analysis and tool call count comparison
VS Code uses same JDT LS engine	High	Well-documented; Eclipse JDT LS is the standard Java LS for VS Code
IntelliJ PSI is deeper than LSP for Java	High	Architectural fact; PSI maintains full live AST vs LSP's on-demand resolution
IntelliJ Copilot agent has 60+ MCP tools	Medium	Reported in multiple sources; exact count may vary by version
ACP provides full PSI access to agents	Medium	Documented by JetBrains; actual depth depends on agent implementation
IDE approaches consume fewer premium requests than CLI grep	High	Structural advantage: IDEs pre-select context, reducing LLM round-trips
ACP is still maturing	High	Registry launched January 2026; rapidly evolving ecosystem

References

Source Code (Local Analysis)

copilot-agent-runtime/src/tools/lsp.ts:36-835 — LSP tool implementation
copilot-agent-runtime/src/tools/grep.ts:63-227 — Grep tool implementation
copilot-agent-runtime/src/prompts/cli/system.ts:215 — Tool preference order
copilot-agent-runtime/src/prompts/shared/tools.ts:92-129 — Dynamic LSP detection
copilot-agent-runtime/src/lsp/formatters.ts:1-150 — LSP result formatters
copilot-agent-runtime/src/lsp/LSPClient.ts:75-146 — LSP client factory
java-lsp-eclipse-jdt-ls/bin/launch-jdtls:63-76 — JDT LS launcher
java-lsp-eclipse-jdt-ls/lsp.json — LSP server registration

jdubois/java-ai-assistants-5-way-comparison.md

Select an option

No results found