Custom LLM Tracing Setup

How to setup custom LLM tracing in your project

Prerequisite Please see here to get and set your API Key environment variable.

Install the lytix SDK

Python

pip3 install lytix-py

(Reccomended) Set you API key via the environment variable

export LX_API_KEY=<your-api-key-here>

Set your API key in the code

Python

from optimodel import LytixCreds

LytixCreds.setAPIKey('<your-api-key-here>')

Capture Model Trace

Capture custom model IO events outside of any SDK or client

Python

import asyncio
from lytix_py import lytix


userInput = "Whats the capital of France?"

"""
The function to be traced
"""


@lytix.trace(modelName="testModelName")
async def getResponse(logger):
    modelOutput = "Paris is the capital of france"

    # Set our defined params however we want
    lytix.setMessage(userInput, "user")

    # Set a system message
    lytix.setMessage("You are a helpful assistant", "system")

    # Set the output
    lytix.setMessage("Ouput of model", "assistant")

    # Optional values
    lytix.setUserIdentifier("testUser")
    lytix.setSessionId("testSession")

    return modelOutput


async def main():
    response = await getResponse("What is the capital of France?")
    print(response)


if __name__ == "__main__":
    asyncio.run(main())

You’ll now see the model IO trace in the Lytix platform here.