csiebler

policyRule: {
    if: {
            field: 'Microsoft.CognitiveServices/accounts/deployments/sku.name'
            equals: 'ProvisionedManaged'
    }
    then: {
        effect: 'deny'
    }
}

csiebler

<policies>
    <inbound>
        <base />
        <!-- Re-add the api-version query parameter, so the backends know which version we want -->
        <set-query-parameter name="api-version" exists-action="override">
            <value>@(context.Request.OriginalUrl.Query.GetValueOrDefault("api-version"))</value>
        </set-query-parameter>
      
        <!-- Getting the main variable where we keep the list of backends -->
        <cache-lookup-value key="listBackends" variable-name="listBackends" />

csiebler

Load balancing multiple AOAI resources and tracking usage for one or more use cases

Decision points

• Which PTU deployments (models, regions)
• Which PAYGO deployments (models, regions)
• Token tracking level (use case, teams, etc.)

Load Balancing Strategy

csiebler

import os
import requests
from datetime import datetime
from azure.identity import DefaultAzureCredential, ChainedTokenCredential, ManagedIdentityCredential

# see https://learn.microsoft.com/en-us/azure/developer/python/sdk/authentication-on-premises-apps?tabs=azure-portal
# this Service Principal has the Cognitive Services OpenAI User role on the AOAI resources
os.environ["AZURE_CLIENT_ID"] = "xxxxx"
os.environ["AZURE_TENANT_ID"] = "xxxxx"
os.environ["AZURE_CLIENT_SECRET"] = "xxxxx"

csiebler

{"title": "JPMorgan Chase’s Jamie Dimon and his family to sell $141 million of stock in 2024", "content": "JPMorgan Chase chief executive Jamie Dimon and his family plan to sell one million of their shares in the bank starting next year, according to a new securities filing.\n\nDimon — who will use his stock trading plans to offload his shares — and his family currently own roughly 8.6 million shares in the company. The move marks Dimon’s first stock sale during his 17 years at the company’s helm.\n\nJPMorgan Chase shares closed at $140.76 on Thursday, putting the transaction’s worth at roughly $141 million.\n\n“Mr. Dimon continues to believe the company’s prospects are very strong and his stake in the company will remain very significant,” JPMorgan Chase said in the filing.\n\nCNN has reached out to the company for comment.\n\nShares of JPMorgan Chase have climbed roughly 5% this year during what’s been a tough environment for banks. The Federal Reserve’s aggressive pace of interest rate hikes, which began i

csiebler

csiebler

import os
import openai
from dotenv import load_dotenv
from llama_index import GPTVectorStoreIndex, SimpleDirectoryReader, PromptHelper, LangchainEmbedding, ServiceContext
from llama_index.llms import AzureOpenAI
from langchain.embeddings import OpenAIEmbeddings

# Load environment variables (set OPENAI_API_KEY and OPENAI_API_BASE in .env)
load_dotenv()

csiebler

# Make sure you have the latest openai and langchain versions running:
# pip install openai --upgrade
# pip install langchain --upgrade
# This code was tested with openai==0.27.8 and langchain==0.0.240

import os
import openai
from dotenv import load_dotenv
from langchain.chat_models import AzureChatOpenAI
from langchain import LLMChain

csiebler

Model: gpt-35-turbo

System message:

You are a bot that generates SSML for Text-to-Speech for Azure Speech. The text is in German, but contains English words. You should create a proper SSML, but for all English words, use the English voice. Here is an example:

<?xml version="1.0" encoding="UTF-8"?>
<speak xmlns="http://www.w3.org/2001/10/synthesis" xmlns:emo="http://www.w3.org/2009/10/emotionml" xmlns:mstts="http://www.w3.org/2001/mstts" version="1.0" xml:lang="en-US">
   <voice name="en-US-JennyMultilingualNeural">
 Hallo, ich arbeite bei

csiebler

import os
import openai
from dotenv import load_dotenv
from langchain.llms import AzureOpenAI
from langchain.embeddings import OpenAIEmbeddings

# Load environment variables (set OPENAI_API_KEY and OPENAI_API_BASE in .env)
load_dotenv()

# Configure OpenAI API