VenkataSakethDakuri

from typing import Any, Dict, List, Tuple

import torch
from diffusers import (
    AutoencoderKLCogVideoX,
    CogVideoXDPMScheduler,
    CogVideoXImageToVideoPipeline,
    CogVideoXTransformer3DModel,
)
from diffusers.models.embeddings import get_3d_rotary_pos_embed

VenkataSakethDakuri

Eval engineering is the discipline of designing and version-controlling automated test suites that rigorously quantify whether each model or prompt change is an improvement, acting like unit tests for AI systems.

Conditional Evals can save costs especially in workflows.
LLM as a judge, heuristics based evals, golden datasets.

In Lora finetuning can create a pipeline where we try out test suites for a certain set of hyperparameters and store the adapter along with the eval results. We can iterate over different values of rank, alpha, target modules based on eval results. 

Benchmarking refers to standardised comparison bw predefined tasks. Evaluation refers to the overall model performance and suitability for intended task.

RAG

VenkataSakethDakuri

import requests

import os

from dotenv import load_dotenv


#import the VAPI_API_KEY from the .env file
load_dotenv()
VAPI_API_KEY = os.getenv("VAPI_API_KEY")

VenkataSakethDakuri

Follow this notebook for gain insights over different techniques to make small language models along with some results of my experiments:
https://notebooklm.google.com/notebook/eee2c93a-12a8-4dba-9311-a76b464c58ac

VenkataSakethDakuri

Follow this notebook for insights on key features of Weaviate, using weaviate with k8s and many more:
https://notebooklm.google.com/notebook/bec57f15-3091-43dd-b93a-70a09de31274

VenkataSakethDakuri

Thinking in first principles. Think of 10X better solution. Combine solutions from different industries.
MOSCOW, RICE, JTBD, Kano.

Remove AI from the business and check if it still works.
The only to win in hypercompetitive AI space is to be overly obsessed with the problem we are trying to solve. Even if the odds are extremely low I'll still work on it.
Hiring generalists might be better because they understand context of different tasks.

Masterpieces come when we try a lot.
Creativity is combinatorial, combining different existing solutions in novel ways.

VenkataSakethDakuri

Suppose we make requests in comet regarding our banking mails (after connecting our mail to comet), all AI processing happens locally itself (this involves using heuristics like sender, subject keywords), if we want to use advanced features which are not possible locally then minimal data needed is sent to cloud and data is processed. This "minimal" data is determined algorithmically by AI + heuristics. We can control different settings to manage the data that is transmitted. We can also view the data collected by comet.   


Pre-tokenization masking of restricted content using named entity recognition (NER) can be used to mask sensitive information.

VenkataSakethDakuri

Table Stakes
Better Parsers
Chunk Sizes
Hybrid Search
Metadata Filters

Advanced Retrieval
Reranking
Recursive Retrieval
Embedded Tables

VenkataSakethDakuri

Time Series Forecasting Techniques
Smoothing Based Techniques:
Simple Moving Average
Simple Exponential Smoothing
Holt’s Linear Trend
Holt Winter’s Exponential Smoothing

ARIMA Based Techniques:
AR
MA

VenkataSakethDakuri

Tools
Model-controlled
Functions invoked by the model
Retrieve / search
Send a message
Update DB records

Resources
Application-controlled
Data exposed to the application