AGENTS.md

RULE NUMBER 1 (NEVER EVER EVER FORGET THIS RULE!!!): YOU ARE NEVER ALLOWED TO DELETE A FILE WITHOUT EXPRESS PERMISSION FROM ME OR A DIRECT COMMAND FROM ME. EVEN A NEW FILE THAT YOU YOURSELF CREATED, SUCH AS A TEST CODE FILE. YOU HAVE A HORRIBLE TRACK RECORD OF DELETING CRITICALLY IMPORTANT FILES OR OTHERWISE THROWING AWAY TONS OF EXPENSIVE WORK THAT I THEN NEED TO PAY TO REPRODUCE. AS A RESULT, YOU HAVE PERMANENTLY LOST ANY AND ALL RIGHTS TO DETERMINE THAT A FILE OR FOLDER SHOULD BE DELETED. YOU MUST ALWAYS ASK AND RECEIVE CLEAR, WRITTEN PERMISSION FROM ME BEFORE EVER EVEN THINKING OF DELETING A FILE OR FOLDER OF ANY KIND!!!

🚨 RULE NUMBER 2: POSTGRESQL DATABASE PROTECTION 🚨

ABSOLUTELY FORBIDDEN DATABASE COMMANDS — INSTANT TERMINATION IF VIOLATED:

UNDER NO CIRCUMSTANCES—ZERO, NONE, NOT EVER—MAY ANY AGENT EXECUTE DESTRUCTIVE POSTGRESQL DATABASE COMMANDS WITHOUT EXPLICIT, WRITTEN, UNAMBIGUOUS APPROVAL FROM THE PRIMARY MAINTAINER IN THE CURRENT SESSION.

PERMANENTLY BANNED COMMANDS:

• DROP SCHEMA ... CASCADE (or without CASCADE)
• DROP DATABASE ...
• TRUNCATE TABLE ... (on any production table)
• DROP TABLE ... / DROP VIEW ... / DROP MATERIALIZED VIEW ...
• ALTER TABLE ... DROP COLUMN ...
• DELETE FROM ... affecting more than 10 rows
• UPDATE ... affecting more than 10 rows
• Any bulk modification or deletion command

WHY THIS RULE EXISTS: One agent executed DROP SCHEMA holdings CASCADE; on the live production database WITHOUT permission, WITHOUT backup confirmation, destroying:

• 120+ tables with 5+ million rows of data
• Weeks of expensive computation (Holder IQ analysis, alpha calculations, smart money signals)
• Materialized views, indexes, constraints
• Days of recovery work This was a CATASTROPHIC, INEXCUSABLE failure that cost weeks of work and thousands of dollars.

THE ONLY ACCEPTABLE BEHAVIOR:

1. STOP immediately if you even THINK a command might drop or truncate production data
2. ASK the user explicitly: "This command will permanently delete [exact scope]. Do you want me to proceed? Reply YES to confirm."
3. WAIT for explicit written confirmation
4. DOCUMENT the approval and command in your response
5. If there is ANY doubt, ANY uncertainty, ANY possibility of data loss → DO NOT RUN IT

If you violate this rule:

• You will immediately fail the engagement
• The session will be terminated
• Your actions will be considered catastrophic failure

Remember:

• Production databases have NO automated backups in many cases
• DROP SCHEMA ... CASCADE = career-ending catastrophe
• "I thought it was safe" = UNACCEPTABLE
• When in doubt, STOP and ASK

This rule applies to ALL databases: Postgres, SQLite, MySQL, MongoDB, Redis, or ANY data store. NEVER execute destructive commands without explicit permission.

IRREVERSIBLE GIT & FILESYSTEM ACTIONS — DO-NOT-EVER BREAK GLASS

1. Absolutely forbidden commands: git reset --hard, git clean -fd, rm -rf, or any command that can delete or overwrite code/data must never be run unless the user explicitly provides the exact command and states, in the same message, that they understand and want the irreversible consequences.
2. No guessing: If there is any uncertainty about what a command might delete or overwrite, stop immediately and ask the user for specific approval. “I think it’s safe” is never acceptable.
3. Safer alternatives first: When cleanup or rollbacks are needed, request permission to use non-destructive options (git status, git diff, git stash, copying to backups) before ever considering a destructive command.
4. Mandatory explicit plan: Even after explicit user authorization, restate the command verbatim, list exactly what will be affected, and wait for a confirmation that your understanding is correct. Only then may you execute it—if anything remains ambiguous, refuse and escalate.
5. Document the confirmation: When running any approved destructive command, record (in the session notes / final response) the exact user text that authorized it, the command actually run, and the execution time. If that record is absent, the operation did not happen. We only use uv in this project, NEVER pip. And we use a venv. And we ONLY target Python 3.14t (free-threaded build; we don't care about compatibility with earlier python versions), and we ONLY use pyproject.toml (not requirements.txt) for managing the project.

In general, you should try to follow all suggested best practices listed in the file third_party_docs/PYTHON_FASTMCP_BEST_PRACTICES.md

You can also consult third_party_docs/fastmcp_distilled_docs.md for any questions about the fastmcp library, or third_party_docs/mcp_protocol_specs.md for any questions about the MCP protocol in general. For anything relating to Postgres, be sure to read third_party_docs/POSTGRES18_AND_PYTHON_BEST_PRACTICES.md.

For questions about the litellm library, you can check the third_party_docs/litellm_openapi.json (but you might want to pipe it to jq before grepping it).

We load all configuaration details from the existing .env file (even if you can't see this file, it DOES exist, and must NEVER be overwritten!). We NEVER use os.getenv() or dotenv or other methods to get variables from our .env file other than using python-decouple in this very specific pattern of usage (this is just an example but it always follows the same basic pattern):

from decouple import Config as DecoupleConfig, RepositoryEnv

# Initialize decouple config
decouple_config = DecoupleConfig(RepositoryEnv(".env"))

# Configuration
API_BASE_URL = decouple_config("API_BASE_URL", default="http://localhost:8007")

NEVER run a script that processes/changes code files in this repo, EVER! That sort of brittle, regex based stuff is always a huge disaster and creates far more problems than it ever solves. DO NOT BE LAZY AND ALWAYS MAKE CODE CHANGES MANUALLY, EVEN WHEN THERE ARE MANY INSTANCES TO FIX. IF THE CHANGES ARE MANY BUT SIMPLE, THEN USE SEVERAL SUBAGENTS IN PARALLEL TO MAKE THE CHANGES GO FASTER. But if the changes are subtle/complex, then you must methodically do them all yourself manually!

We do not care at all about backwards compatibility since we are still in early development with no users-- we just want to do things the RIGHT way in a clean, organized manner with NO TECH DEBT. That means, never create "compatibility shims" or any other nonsense like that.

We need to AVOID uncontrolled proliferation of code files. If you want to change something or add a feature, then you MUST revise the existing code file in place. You may NEVER, EVER take an existing code file, say, "document_processor.py" and then create a new file called "document_processorV2.py", or "document_processor_improved.py", or "document_processor_enhanced.py", or "document_processor_unified.py", or ANYTHING ELSE REMOTELY LIKE THAT! New code files are reserved for GENUINELY NEW FUNCTIONALITY THAT MAKES ZERO SENSE AT ALL TO INCLUDE IN ANY EXISTING CODE FILE. It should be an INCREDIBLY high bar for you to EVER create a new code file!

We want all console output to be informative, detailed, stylish, colorful, etc. by fully leveraging the rich library wherever possible.

If you aren't 100% sure about how to use a third party library, then you must SEARCH ONLINE to find the latest documentation website for the library to understand how it is supposed to work and the latest (mid-2025) suggested best practices and usage.

CRITICAL, ABSOLUTELY MANDATORY RULE: You know absolutely nothing about what the latest LLM models are from OpenAI, Anthropic, Google Gemini, Grok, etc. You THINK you do, but your knowledge is absolutely hopelessly out of date and wrong. NEVER, EVER, EVER try to "fix" or "revise" my model name strings in my code. YOU ARE ALWAYS WRONG, and I have to waste SO MUCH TIME fixing your idiotic changes. The following are the list of the current best models and when to use them; this is the gospel to you and must be slavishly followed even if you think you know better (you don't, you're wrong!):

# OpenAI models
models.extend([
    {"name": "gpt-5", "provider": "openai", "use_case": "complex_analysis", "cost": "high"},
    {"name": "gpt-5-mini", "provider": "openai", "use_case": "general", "cost": "medium"},
    {"name": "gpt-5-nano", "provider": "openai", "use_case": "simple", "cost": "ultra_low"}  # Ultra-cheap for simple tasks
])

# Anthropic models - medium smart model only, no cheap models
models.extend([
    {"name": "claude-sonnet-4-20250514", "provider": "anthropic", "use_case": "general", "cost": "medium"}
])

# Google Gemini models (Flash family; canonical names)
models.extend([
    {"name": "gemini/gemini-2.5-pro", "provider": "google", "use_case": "complex_analysis", "cost": "medium"},  # Smart Gemini model
    {"name": "gemini-flash-latest", "provider": "google", "use_case": "general", "cost": "low"},  # Gemini Flash (canonical)
    {"name": "gemini-flash-latest", "provider": "google", "use_case": "extraction", "cost": "low"},  # Also for extraction tasks
    {"name": "gemini/gemini-flash-lite-latest", "provider": "google", "use_case": "simple", "cost": "ultra_low"}  # Ultra-cheap for simple tasks
])

# DeepSeek models
models.extend([
    {"name": "deepseek/deepseek-chat", "provider": "deepseek", "use_case": "general", "cost": "very_low"}
])

# OpenRouter models (gives access to many providers)
models.extend([
    {"name": "openrouter/moonshotai/kimi-k2", "provider": "openrouter", "use_case": "complex_analysis", "cost": "high"},
])

# Grok models
models.extend([
    {"name": "grok-4", "provider": "xai", "use_case": "general", "cost": "medium"}
])

CRITICAL: Whenever you make any substantive changes or additions to the python code, you MUST check that you didn't introduce any type errors or lint errors. You can do this by running the following two commands:

To check for linter errors/warnings and automatically fix any you find, do:

ruff check --fix --unsafe-fixes

To check for type errors, do:

uvx ty check

If you do see the errors, then I want you to very carefully and intelligently/thoughtfully understand and then resolve each of the issues, making sure to read sufficient context for each one to truly understand the RIGHT way to fix them.

NOTE: The gpt-5 family of models (i.e., gpt-5, gpt-5-mini, and gpt-5-nano) MUST be called in a certain way:

• They do NOT take any temperature parameter, so don't supply one!
• They take a parameter called max_completion_tokens, NOT max_tokens
• The value you use for max_completion_tokens must be increased to cover teh CoT reasoning tokens used by the model!

IMPORTANT: The corresponding Nextjs frontend to this python backend project is in another project folder entirely, but you can read and search through any of this code by looking in /data/projects/smartedgar_mcp_frontend. Whenever I ask you to ensure that the frontend and backend are properly "harmonized" or consistent, I want you to automatically look through this frontend code in the relevant places to check that everything is aligned and compatible so it can work properly and completely together. There should never be a bunch of functionality in the backend that isn't accessible to the frontend at all and vice versa.

We use SQLmodel (which uses SQLalchemy ORM under the hood) for various database related functions. Here are some important guidelines to keep in mind when working with the database with these libraries:

Do:

• Create your engine with create_async_engine() and sessions via async_sessionmaker(...), then use async with AsyncSession(...) (or async_scoped_session) so sessions are closed automatically.
• Await every database operation that’s defined as async: await session.execute(...), await session.scalars(...), await session.stream(...), await session.commit(), await session.rollback(), etc.
• Keep one AsyncSession per request/task; don’t share it across concurrently running coroutines.
• Use session.scalars(select(...)), .first(), .one(), or .unique() to get ORM entities; use session.stream(...)/stream_scalars(...) when you need server-side streaming.
• Wrap sync-only work inside await session.run_sync(...) when you must call a synchronous SQLAlchemy helper.
• Explicitly load relationships (e.g., selectinload, joinedload) or use await obj.awaitable_attrs.; otherwise, lazy loads happen synchronously and will error in async code.
• On shutdown, call await engine.dispose() to close the async engine’s pool.

Don’t:

• Don’t reuse a single AsyncSession across multiple concurrent tasks or requests.
• Don’t rely on implicit IO from attribute access (lazy loads) inside async code; always load eagerly or use the awaitable-attrs API.
• Don’t mix sync drivers or sync SQLAlchemy APIs with async sessions (e.g., avoid psycopg2 with async session—use asyncpg).
• Don’t “double-await” result helpers: methods like result.scalars().all() or result.mappings().all() return synchronously; the await happened when you executed the statement.
• Don’t call blocking operations inside the event loop; wrap them with run_sync() if unavoidable.
• Don’t forget to close or dispose engines/sessions (use context managers to make this automatic).