Programming Overview ↗
yesEditorial Notes
DSPy takes a fundamentally different approach to working with LLMs: instead of manually writing and tweaking prompts, you define typed signatures (input/output schemas) and let DSPy’s optimizers compile them into effective prompts automatically. This is a paradigm shift from traditional prompt engineering, and it means the skills you build here are more about program design than wordsmithing. Focus on understanding Modules and Signatures first, as they are the core abstractions everything else builds on. A common gotcha is expecting DSPy to work well without a good set of training examples — the optimizers need representative input-output pairs to produce high-quality compiled prompts, so invest in your dataset before tuning the optimizer.
Original Documentation
DSPy is a bet on writing code instead of strings. In other words, building the right control flow is crucial. Start by defining your task. What are the inputs to your system and what should your system produce as output? Is it a chatbot over your data or perhaps a code assistant? Or maybe a system for translation, for highlighting snippets from search results, or for generating reports with citations?
Next, define your initial pipeline. Can your DSPy program just be a single module or do you need to break it down into a few steps? Do you need retrieval or other tools, like a calculator or a calendar API? Is there a typical workflow for solving your problem in multiple well-scoped steps, or do you want more open-ended tool use with agents for your task? Think about these but start simple, perhaps with just a single dspy.ChainOfThought module, then add complexity incrementally based on observations.
As you do this, craft and try a handful of examples of the inputs to your program. Consider using a powerful LM at this point, or a couple of different LMs, just to understand what’s possible. Record interesting (both easy and hard) examples you try. This will be useful when you are doing evaluation and optimization later.
Beyond encouraging good design patterns, how does DSPy help here?
Conventional prompts couple your fundamental system architecture with incidental choices not portable to new LMs, objectives, or pipelines. A conventional prompt asks the LM to take some inputs and produce some outputs of certain types (a signature), formats the inputs in certain ways and requests outputs in a form it can parse accurately (an adapter), asks the LM to apply certain strategies like “thinking step by step” or using tools (a module’s logic), and relies on substantial trial-and-error to discover the right way to ask each LM to do this (a form of manual optimization).
DSPy separates these concerns and automates the lower-level ones until you need to consider them. This allow you to write much shorter code, with much higher portability. For example, if you write a program using DSPy modules, you can swap the LM or its adapter without changing the rest of your logic. Or you can exchange one module, like dspy.ChainOfThought, with another, like dspy.ProgramOfThought, without modifying your signatures. When you’re ready to use optimizers, the same program can have its prompts optimized or its LM weights fine-tuned.