06-setting-up-rag-with-postgre-sql-and-pgvector.json

{
  "title": "Supercharging Your AI: Setting Up RAG with PostgreSQL and pgvector",
  "content": "## Introduction\n\nImagine having a conversation with an AI that not only understands your questions but also draws upon a vast repository of knowledge to provide accurate, contextually relevant answers. This isn't science fiction—it's the power of Retrieval-Augmented Generation (RAG) systems. In this blog post, we'll dive into the exciting world of RAG and show you how to harness this technology using PostgreSQL and pgvector.\n\n## What is RAG, and Why Should You Care?\n\nRAG is like giving your AI a photographic memory and the ability to instantly recall relevant information. It combines the power of large language models with a knowledge base, allowing the AI to provide more accurate, up-to-date, and contextually relevant responses. Whether you're building a chatbot, a question-answering system, or any AI application that requires access to specific knowledge, RAG is your secret weapon.\n\n## Prerequisites\n\nBefore we embark on this RAG adventure, make sure you have:\n\n- PostgreSQL 13 or later\n- Python 3.7 or later\n- pip (Python package manager)\n- Docker (for building our custom PostgreSQL image)\n\n## Step 1: Building a Custom PostgreSQL Image with pgvector\n\nLet's start by creating a supercharged PostgreSQL image with the pgvector extension pre-installed. This will save us time and ensure consistency across different environments.\n\n1. Create a new directory for your Dockerfile:\n\n```bash\nmkdir postgres-pgvector && cd postgres-pgvector\n```\n\n1. Create a file named `Dockerfile` with the following content:\n\n```dockerfile\nFROM postgres:14\n\nENV POSTGRES_DB=vectordb\nENV POSTGRES_USER=vectoruser\nENV POSTGRES_PASSWORD=vectorpass\n\nRUN apt-get update && apt-get install -y \\\n    build-essential \\\n    git \\\n    postgresql-server-dev-14\n\nRUN git clone https://github.com/pgvector/pgvector.git\n\nRUN cd pgvector && \\\n    make && \\\n    make install\n\nRUN apt-get remove -y build-essential git postgresql-server-dev-14 && \\\n    apt-get autoremove -y && \\\n    apt-get clean && \\\n    rm -rf /var/lib/apt/lists/* /pgvector\n\nRUN echo \"CREATE EXTENSION vector;\" > /docker-entrypoint-initdb.d/init.sql\n```\n\n1. Build the Docker image:\n\n```bash\ndocker build -t postgres-pgvector .\n```\n\n1. Run the container:\n\n```bash\ndocker run -d --name postgres-vector -p 5432:5432 postgres-pgvector\n```\n\nYou now have a PostgreSQL instance running with pgvector ready to go.\n\n## Step 2: Install Required Python Packages\n\nLet's set up our Python environment:\n\n```bash\npip install psycopg2-binary pgvector sentence-transformers torch transformers\n```\n\n## Step 3: Create a Table for Vector Storage\n\nConnect to your PostgreSQL database and run:\n\n```sql\nCREATE TABLE documents (\n    id SERIAL PRIMARY KEY,\n    content TEXT,\n    embedding vector(384)\n);\n```\n\n## Step 4: Inserting Documents and Embeddings\n\nNow, let's populate our database with some interesting facts:\n\n```python\nimport psycopg2\nfrom sentence_transformers import SentenceTransformer\nimport numpy as np\n\nmodel = SentenceTransformer('all-MiniLM-L6-v2')\n\n# Connect to the PostgreSQL database\nconn = psycopg2.connect(\"dbname=vectordb user=vectoruser password=vectorpass host=localhost\")\ncur = conn.cursor()\n\n# Sample documents\ndocuments = [\n    \"The Great Wall of China is visible from space, but it's a common myth that it can be seen from the moon.\",\n    \"Honey never spoils. Archaeologists have found pots of honey in ancient Egyptian tombs that are over 3,000 years old and still perfectly edible.\",\n    \"The world's oldest known living tree is a Great Basin Bristlecone Pine that is over 4,800 years old.\",\n    \"Octopuses have three hearts: two pump blood through each of the two gills, while the third pumps blood through the body.\",\n    \"The shortest war in history lasted only 38 minutes. It was between Britain and Zanzibar on August 27, 1896.\"\n]\n\n# Insert documents and their embeddings\nfor doc in documents:\n    embedding = model.encode(doc)\n    cur.execute(\"INSERT INTO documents (content, embedding) VALUES (%s, %s)\",\n                (doc, embedding.tolist()))\n\nconn.commit()\ncur.close()\nconn.close()\n\nprint(\"Documents inserted successfully!\")\n```\n\n## Step 5: Implementing RAG Query\n\nNow for the exciting part—let's implement our RAG system:\n\n```python\nimport psycopg2\nfrom sentence_transformers import SentenceTransformer\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\nimport torch\n\n# Initialize models\nembedding_model = SentenceTransformer('all-MiniLM-L6-v2')\nllm_model = AutoModelForCausalLM.from_pretrained(\"gpt2\")\ntokenizer = AutoTokenizer.from_pretrained(\"gpt2\")\n\n# Connect to the PostgreSQL database\nconn = psycopg2.connect(\"dbname=vectordb user=vectoruser password=vectorpass host=localhost\")\ncur = conn.cursor()\n\ndef query_similar_documents(query, top_k=1):\n    query_embedding = embedding_model.encode(query)\n    \n    cur.execute(\"\"\"\n        SELECT content, 1 - (embedding <=> %s) AS similarity\n        FROM documents\n        ORDER BY similarity DESC\n        LIMIT %s\n    \"\"\", (query_embedding.tolist(), top_k))\n    \n    return cur.fetchall()\n\ndef generate_response(query, context):\n    prompt = f\"Context: {context}\\n\\nQuestion: {query}\\n\\nAnswer:\"\n    input_ids = tokenizer.encode(prompt, return_tensors=\"pt\")\n    \n    with torch.no_grad():\n        output = llm_model.generate(input_ids, max_length=100, num_return_sequences=1, no_repeat_ngram_size=2)\n    \n    return tokenizer.decode(output[0], skip_special_tokens=True)\n\n# Example usage\nuser_query = \"Tell me an interesting fact about nature.\"\nsimilar_docs = query_similar_documents(user_query)\ncontext = similar_docs[0][0]  # Use the most relevant document as context\n\nresponse = generate_response(user_query, context)\n\nprint(f\"User Query: {user_query}\")\nprint(f\"Context: {context}\")\nprint(f\"AI Response: {response}\")\n\ncur.close()\nconn.close()\n```\n\n## Bringing It All Together\n\nCongratulations! You've just built a RAG system that can:\n\n1. Store and retrieve vector embeddings of documents efficiently using PostgreSQL and pgvector.\n2. Find the most relevant context for a given query.\n3. Generate human-like responses using a language model, augmented with retrieved information.\n\nThis system forms the foundation for creating intelligent chatbots, question-answering systems, and other AI applications that can tap into specific knowledge bases.\n\nHappy coding!",
  "tags": [
    "PostgreSql",
    "PGVector",
    "RAG",
    "Vector Database",
    "AI",
    "NLP",
    "Docker",
    "GPT",
    "LLM"
  ],
  "canonicalUrl": "",
  "publishStatus": "draft",
  "publishedAt": "2024-09-30T21:28:28+05:30"
}