title: "Supercharging Your AI: Setting Up RAG with PostgreSQL and pgvector" date: 2024-09-30T21:28:28+05:30 draft: false ShowToc: true TocOpen: false summary: "Learn how to implement a powerful Retrieval-Augmented Generation (RAG) system using PostgreSQL and pgvector. This comprehensive guide covers everything from setting up a custom PostgreSQL Docker image to creating a fully functional RAG query system with vector embeddings and language model inference." tags: ["PostgreSql", "PGVector", "RAG", "Vector Database", "AI", "NLP", "Docker", "GPT", "LLM"] categories: ["Database", "Artificial Intelligence", "Tutorial"] cover:
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
def image_to_ascii(img, width=100, height=50): | |
# Convert to numpy array if it's not already | |
if not isinstance(img, np.ndarray): | |
img = np.array(img) | |
# Ensure the image is in the correct format (H, W, C) | |
if img.ndim == 3 and img.shape[0] == 3: | |
img = np.transpose(img, (1, 2, 0)) | |
# Normalize the image if it's not in [0, 1] range |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
{ | |
"title": "Supercharging Your AI: Setting Up RAG with PostgreSQL and pgvector", | |
"content": "## Introduction\n\nImagine having a conversation with an AI that not only understands your questions but also draws upon a vast repository of knowledge to provide accurate, contextually relevant answers. This isn't science fiction—it's the power of Retrieval-Augmented Generation (RAG) systems. In this blog post, we'll dive into the exciting world of RAG and show you how to harness this technology using PostgreSQL and pgvector.\n\n## What is RAG, and Why Should You Care?\n\nRAG is like giving your AI a photographic memory and the ability to instantly recall relevant information. It combines the power of large language models with a knowledge base, allowing the AI to provide more accurate, up-to-date, and contextually relevant responses. Whether you're building a chatbot, a question-answering system, or any AI application that requires access to specific knowledge, RAG is your secret weapon.\n\n## Prerequisites\n\nB |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import argparse | |
import json | |
import re | |
import sys | |
from datetime import datetime | |
import frontmatter | |
def convert_markdown_to_medium(input_file, output_file): |
title: "Git Aliases to Supercharge Your Workflow" date: 2024-03-21 draft: false ShowToc: true TocOpen: false math: true summary: "Git aliases are a powerful workflow tool that create shortcuts to frequently used Git commands." tags: [ "git", "version-control", "workflow", "development", "productivity", "command-line", "aliases", "git-commands", "git-aliases", "coding",
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
""" | |
- based no splits 1000 splits | |
- based on time eg 10s | |
- based on time interval 2:10 -> 2:50 | |
- 10 people upload 1 gb 2 file daily | |
- we have design a system to split it into 1000 partitions | |
Solution: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
def fibonacci(n): | |
if n <= 2: | |
return 1 | |
return fibonacci(n - 1) + fibonacci(n - 2) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from django.urls import path, include | |
from rest_framework import routers | |
from .views import UploadViewSet | |
router = routers.DefaultRouter() | |
router.register(r'upload', UploadViewSet, basename="upload") | |
# Wire up our API using automatic URL routing. | |
urlpatterns = [ | |
path('', include(router.urls)), |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from rest_framework.serializers import Serializer, FileField | |
# Serializers define the API representation. | |
class UploadSerializer(Serializer): | |
file_uploaded = FileField() | |
class Meta: | |
fields = ['file_uploaded'] |
NewerOlder