lmolkova

Here're some items that we've considered in the past to be useful in azure-core or clientcore related to observability, but never got to it.

• [Azure Core only] Make slf4j dependency optional - backport Slf4JLoggerShim. Related to Azure/azure-sdk-for-java#38421

• [Azure Core only] Merge tracing and metrics plugins into one - Azure/azure-sdk-for-java#41436 or backport plugin-free OTel support from clientcore

• Allow adding arbitrary key-value-pairs on instrumentation context and stamping them on all nested logs and spans (similar to MDC):

lmolkova

TODOs:

• AI weaver - weaver generate should generate new schema behind feature flag - Liudmila will create an issue

Phases:

1. ingest new format
2. resolve new format - need to advertise and socialize
3. modify resolution to be on the new format

| signal/object | identity property in semsonv | in proto |

lmolkova

package live_check_advice

import rego.v1

required_attributes := [
    "http.request.method",
    "server.address",
    "server.port",
    "url.full",
]

lmolkova

from enum import Enum
import json
from typing import Annotated, Any, List, Literal, Optional, Union
from pydantic import BaseModel, Field, RootModel

class TextPart(BaseModel):
    """
    Describes text content sent to or received from the model.
    """
    type: Literal['text']

lmolkova

RequestOptions scenarios

These are the scenarios we support today, not necessarily the ones we have to provide in the final design.

1. Users can provide static (for a group of operations) configuration. E.g.: configure custom logger for all requests peformed in scope of this API call

2. Provide dynamic (per-request) configuration. E.g. set request body or operation-specific context

So RequestOptions is a mix of (usually) per-request and (usually) static config.

lmolkova

Observability in clientcore

• Key changes
• Champion scenarios: user app
  • Default experience
  • Default experience: I want to use custom correlation-id
  • Default experience: add HTTP logs
  • Experience with OTel + logback + HTTP logs
• Champion scenarios: client lib
• Next steps

lmolkova

private string GetFinishReason(ChatFinishReason finishReason) => finishReason switch
{
    ChatFinishReason.Stop => "stop",
    ChatFinishReason.Length => "length",
    ChatFinishReason.ToolCalls => "tool_calls",
    // ...
    _ => finishReason.ToString(),
};

private void LogChoice(ChatCompletion message)

lmolkova

import json
import os
import time
from typing import override
from opentelemetry.instrumentation.httpx import HTTPXClientInstrumentor
#HTTPXClientInstrumentor().instrument()

from openai import AssistantEventHandler
from openai.types.beta.threads.runs import RunStep

lmolkova

public class Main {
  
    public static void main(String[] args) throws IOException {
        var logProvider = SdkLoggerProvider.builder()
          .addLogRecordProcessor(SimpleLogRecordProcessor.create(OtlpGrpcLogRecordExporter.getDefault())).build();
        EventLoggerProvider eventLoggerProvider = SdkEventLoggerProvider.create(logProvider);

        var logger = eventLoggerProvider.get("test-logger");
        var event = new GenAIChoice()
                .setId(1)

lmolkova

let get_role = (event_name:string) { 
  iff(event_name == "gen_ai.choice", "assistant", split(event_name, ".")[1])
};
let gen_ai_spans = dependencies
| where isnotnull(customDimensions["gen_ai.system"])
| project 
    operation_ParentId,
    operation_Id,
    id,
    duration,