Track Application Logic ↗

wandb tutorial beginner
no
Summary: Learn how to track data flow and metadata in your LLM applications

Original Documentation

Documentation Index#

Fetch the complete documentation index at: https://docs.wandb.ai/llms.txt Use this file to discover all available pages before exploring further.

Learn how to track data flow and metadata in your LLM applications

In the Track LLM inputs & outputs tutorial, the basics of tracking the inputs and outputs of your LLMs was covered.

In this tutorial you will learn how to:

  • Track data as it flows through your application
  • Track metadata at call time

Tracking nested function calls#

LLM-powered applications can contain multiple LLMs calls and additional data processing and validation logic that is important to monitor. Even deep nested call structures common in many apps, Weave will keep track of the parent-child relationships in nested functions as long as weave.op() is added to every function you’d like to track.

Building on the quickstart example, the following code adds additional logic to count the returned items from the LLM and wrap them all in a higher level function. Additionally, the example uses weave.op() to trace every function, its call order, and its parent-child relationship:

    import weave
    import json
    from openai import OpenAI

    client = OpenAI()

    @weave.op()
    def extract_dinos(sentence: str) -> dict:
        response = client.chat.completions.create(
            model="gpt-4o",
            messages=[
                {
                    "role": "system",
                    "content": """Extract any dinosaur `name`, their `common_name`, \
    names and whether its `diet` is a herbivore or carnivore, in JSON format."""
                },
                {
                    "role": "user",
                    "content": sentence
                }
                ],
                response_format={ "type": "json_object" }
            )
        return response.choices[0].message.content

    @weave.op()
    def count_dinos(dino_data: dict) -> int:
        # count the number of items in the returned list
        k = list(dino_data.keys())[0]
        return len(dino_data[k])

    @weave.op()
    def dino_tracker(sentence: str) -> dict:
        # extract dinosaurs using a LLM
        dino_data = extract_dinos(sentence)

        # count the number of dinosaurs returned
        dino_data = json.loads(dino_data)
        n_dinos = count_dinos(dino_data)
        return {"n_dinosaurs": n_dinos, "dinosaurs": dino_data}

    weave.init('jurassic-park')

    sentence = """I watched as a Tyrannosaurus rex (T. rex) chased after a Triceratops (Trike), \
    both carnivore and herbivore locked in an ancient dance. Meanwhile, a gentle giant \
    Brachiosaurus (Brachi) calmly munched on treetops, blissfully unaware of the chaos below."""

    result = dino_tracker(sentence)
    print(result)
    ```

**Nested functions**

When you run the above code, you see the the inputs and outputs from the two nested functions (`extract_dinos` and `count_dinos`), as well as the automatically-logged OpenAI trace.

    <img src="https://mintcdn.com/wb-21fd5541/aRvhhwVWqlxBzke5/images/tutorial_tracing_2_nested_dinos.png?fit=max&auto=format&n=aRvhhwVWqlxBzke5&q=85&s=738409adb3ed286689728052142a50a5" alt="Nested Weave Trace" data-og-width="1354" width="1354" data-og-height="1334" height="1334" data-path="images/tutorial_tracing_2_nested_dinos.png" data-optimize="true" data-opv="3" srcset="https://mintcdn.com/wb-21fd5541/aRvhhwVWqlxBzke5/images/tutorial_tracing_2_nested_dinos.png?w=280&fit=max&auto=format&n=aRvhhwVWqlxBzke5&q=85&s=0f941a7bd42d46f1f0da20fd1feb1957 280w, https://mintcdn.com/wb-21fd5541/aRvhhwVWqlxBzke5/images/tutorial_tracing_2_nested_dinos.png?w=560&fit=max&auto=format&n=aRvhhwVWqlxBzke5&q=85&s=f1f089586c8aec79a33e20d2e387db51 560w, https://mintcdn.com/wb-21fd5541/aRvhhwVWqlxBzke5/images/tutorial_tracing_2_nested_dinos.png?w=840&fit=max&auto=format&n=aRvhhwVWqlxBzke5&q=85&s=02fec50e7b2e83e5182d06b5c23763bb 840w, https://mintcdn.com/wb-21fd5541/aRvhhwVWqlxBzke5/images/tutorial_tracing_2_nested_dinos.png?w=1100&fit=max&auto=format&n=aRvhhwVWqlxBzke5&q=85&s=50bbeb0e61fbd2c8699258686330d499 1100w, https://mintcdn.com/wb-21fd5541/aRvhhwVWqlxBzke5/images/tutorial_tracing_2_nested_dinos.png?w=1650&fit=max&auto=format&n=aRvhhwVWqlxBzke5&q=85&s=3054d1a352386af19d87dd9c61f60d90 1650w, https://mintcdn.com/wb-21fd5541/aRvhhwVWqlxBzke5/images/tutorial_tracing_2_nested_dinos.png?w=2500&fit=max&auto=format&n=aRvhhwVWqlxBzke5&q=85&s=980ae93931c6dce6b5a5d6452b731f24 2500w" />
  <span class="tab-end"></span>

  <span class="tab-start" data-tab-title="TypeScript"></span>
```typescript
    import OpenAI from 'openai';
    import * as weave from 'weave';

    const openai = new OpenAI();

    const extractDinos = weave.op(async (sentence: string) => {
      const response = await openai.chat.completions.create({
        model: 'gpt-4o',
        messages: [
          {
            role: 'system',
            content:
              'Extract any dinosaur `name`, their `common_name`, names and whether its `diet` is a herbivore or carnivore, in JSON format.',
          },
          {role: 'user', content: sentence},
        ],
        response_format: {type: 'json_object'},
      });
      return response.choices[0].message.content;
    });

    const countDinos = weave.op(async (dinoData: string) => {
      const parsed = JSON.parse(dinoData);
      return Object.keys(parsed).length;
    });

    const dinoTracker = weave.op(async (sentence: string) => {
      const dinoData = await extractDinos(sentence);
      const nDinos = await countDinos(dinoData);
      return {nDinos, dinoData};
    });

    async function main() {
      await weave.init('jurassic-park');

      const sentence = `I watched as a Tyrannosaurus rex (T. rex) chased after a Triceratops (Trike),
            both carnivore and herbivore locked in an ancient dance. Meanwhile, a gentle giant
            Brachiosaurus (Brachi) calmly munched on treetops, blissfully unaware of the chaos below.`;

      const result = await dinoTracker(sentence);
      console.log(result);
    }

    main();

    ```

**Nested functions**

When you run the above code, you see the the inputs and outputs from the two nested functions (`extractDinos` and `countDinos`), as well as the automatically-logged OpenAI trace.

    <img src="https://mintcdn.com/wb-21fd5541/aRvhhwVWqlxBzke5/images/tutorial_tracing_2_nested_dinos.png?fit=max&auto=format&n=aRvhhwVWqlxBzke5&q=85&s=738409adb3ed286689728052142a50a5" alt="Nested Weave Trace" data-og-width="1354" width="1354" data-og-height="1334" height="1334" data-path="images/tutorial_tracing_2_nested_dinos.png" data-optimize="true" data-opv="3" srcset="https://mintcdn.com/wb-21fd5541/aRvhhwVWqlxBzke5/images/tutorial_tracing_2_nested_dinos.png?w=280&fit=max&auto=format&n=aRvhhwVWqlxBzke5&q=85&s=0f941a7bd42d46f1f0da20fd1feb1957 280w, https://mintcdn.com/wb-21fd5541/aRvhhwVWqlxBzke5/images/tutorial_tracing_2_nested_dinos.png?w=560&fit=max&auto=format&n=aRvhhwVWqlxBzke5&q=85&s=f1f089586c8aec79a33e20d2e387db51 560w, https://mintcdn.com/wb-21fd5541/aRvhhwVWqlxBzke5/images/tutorial_tracing_2_nested_dinos.png?w=840&fit=max&auto=format&n=aRvhhwVWqlxBzke5&q=85&s=02fec50e7b2e83e5182d06b5c23763bb 840w, https://mintcdn.com/wb-21fd5541/aRvhhwVWqlxBzke5/images/tutorial_tracing_2_nested_dinos.png?w=1100&fit=max&auto=format&n=aRvhhwVWqlxBzke5&q=85&s=50bbeb0e61fbd2c8699258686330d499 1100w, https://mintcdn.com/wb-21fd5541/aRvhhwVWqlxBzke5/images/tutorial_tracing_2_nested_dinos.png?w=1650&fit=max&auto=format&n=aRvhhwVWqlxBzke5&q=85&s=3054d1a352386af19d87dd9c61f60d90 1650w, https://mintcdn.com/wb-21fd5541/aRvhhwVWqlxBzke5/images/tutorial_tracing_2_nested_dinos.png?w=2500&fit=max&auto=format&n=aRvhhwVWqlxBzke5&q=85&s=980ae93931c6dce6b5a5d6452b731f24 2500w" />
  <span class="tab-end"></span>
<span class="tab-group-end"></span>

## Tracking metadata

You can track metadata by using the `weave.attributes` context manager and passing it a dictionary of the metadata to track at call time.

Continuing our example from above:

<span class="tab-group-start"></span>
  <span class="tab-start" data-tab-title="Python"></span>
```python
    import weave

    weave.init('jurassic-park')

    sentence = """I watched as a Tyrannosaurus rex (T. rex) chased after a Triceratops (Trike), \
    both carnivore and herbivore locked in an ancient dance. Meanwhile, a gentle giant \
    Brachiosaurus (Brachi) calmly munched on treetops, blissfully unaware of the chaos below."""

    # track metadata alongside our previously defined function
    with weave.attributes({'user_id': 'lukas', 'env': 'production'}):
        result = dino_tracker(sentence)
    ```
  <span class="tab-end"></span>

  <span class="tab-start" data-tab-title="TypeScript"></span>
```plaintext
    This feature is not available in TypeScript yet.  Stay tuned!
    ```
  <span class="tab-end"></span>
<span class="tab-group-end"></span>

<span class="callout-start" data-callout-type="note"></span>
  We recommend tracking metadata at run time, such as your user IDs and your code's environment status (development, staging, or production).

  To track system settings, such as a system prompt, we recommend using [Weave Models](/weave/guides/core-types/models)
<span class="callout-end"></span>

## What's next?

* Follow the [App Versioning tutorial](/weave/tutorial-weave_models) to capture, version, and organize ad-hoc prompt, model, and application changes.
Link last verified June 7, 2026. View original ↗
Source: Weights & Biases Docs
Link last verified: 2026-03-04