Fireworks ↗

instructor guide intermediate
no

Original Documentation

Fireworks provides efficient and cost-effective AI models with enterprise-grade reliability. This guide shows you how to use Instructor with Fireworks’s models for type-safe, validated responses.

Quick Start#

Install Instructor with Fireworks support:

pip install "instructor[fireworks-ai]"

Simple User Example (Sync)#

from fireworks.client import Fireworks
import instructor
from pydantic import BaseModel

# Initialize the client
client = Fireworks()

# Enable instructor patches
client = instructor.from_provider("fireworks/llama-v3-70b-instruct")


class User(BaseModel):
    name: str
    age: int


# Create structured output
user = client.create(
    messages=[
        {
            "role": "user",
            "content": "Extract: Jason is 25 years old",
        }
    ],
    response_model=User,
)

print(user)
# > User(name='Jason', age=25)

Simple User Example (Async)#

import instructor
from pydantic import BaseModel
import asyncio

client = instructor.from_provider(
    "fireworks/llama-v3-70b-instruct",
    async_client=True,
)


class User(BaseModel):
    name: str
    age: int


async def extract_user():
    user = await client.create(
        messages=[
            {
                "role": "user",
                "content": "Extract: Jason is 25 years old",
            }
        ],
        response_model=User,
    )
    return user


# Run async function
user = asyncio.run(extract_user())
print(user)  # User(name='Jason', age=25)

Nested Example#

from fireworks.client import Fireworks
import instructor
from pydantic import BaseModel


# Enable instructor patches
client = instructor.from_provider("fireworks/llama-v3-70b-instruct")


class Address(BaseModel):
    street: str
    city: str
    country: str


class User(BaseModel):
    name: str
    age: int
    addresses: list[Address]


# Create structured output with nested objects
user = client.create(
    messages=[
        {
            "role": "user",
            "content": """
                Extract: Jason is 25 years old.
                He lives at 123 Main St, New York, USA
                and has a summer house at 456 Beach Rd, Miami, USA
            """,
        }
    ],
    response_model=User,
)

print(user)
#> {
#>     'name': 'Jason',
#>     'age': 25,
#>     'addresses': [
#>         {
#>             'street': '123 Main St',
#>             'city': 'New York',
#>             'country': 'USA'
#>         },
#>         {
#>             'street': '456 Beach Rd',
#>             'city': 'Miami',
#>             'country': 'USA'
#>         }
#>     ]
#> }

Streaming Support#

Instructor has two main ways that you can use to stream responses out

  1. Iterables: These are useful when you’d like to stream a list of objects of the same type (Eg. use structured outputs to extract multiple users)
  2. Partial Streaming: This is useful when you’d like to stream a single object and you’d like to immediately start processing the response as it comes in.

Partial Streaming Example#

from fireworks.client import Fireworks
import instructor
from pydantic import BaseModel


# Enable instructor patches
client = instructor.from_provider("fireworks/llama-v3-70b-instruct")


class User(BaseModel):
    name: str
    age: int
    bio: str


user = client.create_partial(
    messages=[
        {
            "role": "user",
            "content": "Create a user profile for Jason + 1 sentence bio, age 25",
        },
    ],
    response_model=User,
)

for user_partial in user:
    print(user_partial)
    # name=None age=None bio=None
    # name='Jason' age=None bio=None
    # name='Jason' age=25 bio="When he's"
    # name='Jason' age=25 bio="When he's not working as a graphic designer, Jason can usually be found trying out new craft beers or attempting to cook something other than ramen noodles."

Iterable Example#

from fireworks.client import Fireworks
import instructor
from pydantic import BaseModel


# Enable instructor patches
client = instructor.from_provider("fireworks/llama-v3-70b-instruct")


class User(BaseModel):
    name: str
    age: int


# Extract multiple users from text
users = client.create_iterable(
    messages=[
        {
            "role": "user",
            "content": """
            Extract users:
            1. Jason is 25 years old
            2. Sarah is 30 years old
            3. Mike is 28 years old
        """,
        },
    ],
    response_model=User,
)

for user in users:
    print(user)

    # name='Jason' age=25
    # name='Sarah' age=30
    # name='Mike' age=28

Instructor Modes#

We provide several modes to make it easy to work with the different response models that Fireworks supports

  1. instructor.Mode.MD_JSON : This parses the raw text completion into a pydantic object
  2. instructor.Mode.TOOLS : This uses Fireworks’s tool calling API to return structured outputs to the client

Updates and Compatibility#

Instructor maintains compatibility with Fireworks’s latest API versions. Check the changelog for updates.

Note: Always verify model-specific features and limitations before implementing streaming functionality in production environments.

Link last verified June 7, 2026. View original ↗
Source: Instructor Docs
Link last verified: 2026-03-04