Prompt engineering with Langchain and OpenAI

Prompt engineering is a technique used to design precise and effective questions or instructions for AI models to generate meaningful and accurate responses. With tools like LangChain and OpenAI, you can simplify this process and achieve impressive results. But how does it work in practice? Let’s break it down.

First, you need to install the necessary library:

pip install langchain_openai

Remember to choose your model and set your OpenAI API key:

OPENAI_API_KEY="paste your key here"
MODEL_ID="gpt-4o-mini"

LangChain’s integration with OpenAI allows you to create and interact with AI models easily. Here’s a simple example:

from langchain_openai import ChatOpenAI

model = ChatOpenAI(
    model=MODEL_ID,
    openai_api_key=OPENAI_API_KEY
)

response = model.invoke("What is the wider meaning of life? Explain it in one sentence with simple and understandable language.")
print(response.content)

In this example, the model generates a meaningful response:

The wider meaning of life often refers to the search for purpose and connection, where we find fulfillment through relationships, experiences, and contributions to something greater than ourselves.

Why is prompt engineering important? It helps to guide the model’s output by providing clear and specific instructions. What happens if you need more control? You can customize parameters like temperature, max_tokens, and timeout:

model = ChatOpenAI(
    model=MODEL_ID,
    openai_api_key=OPENAI_API_KEY,
    temperature=0,
    max_tokens=None,
    timeout=None,
    max_retries=2
    )

result = model.invoke("What is the wider meaning of life? Explain it in one sentence with simple and understandable language.")
result.content

This flexibility allows you to adjust the model’s behavior to fit different tasks. What’s the takeaway? Prompt engineering with LangChain and OpenAI makes it easier to get high-quality, relevant responses that align with your goals. Try it yourself! 🌟

One-shot in-context learning

One-shot in-context learning is a way to teach the model by giving it just one clear example of how to solve a task. Instead of training the model on many examples, you provide a single example directly in the prompt. This helps the model understand the task without extra effort or additional training.

How does it work? You first show the model an example of what you expect. For instance, if the task is to classify sentences as “happy” or “sad,” you might write:

model.invoke("""
Classify the following sentence as either 'happy' or 'sad'.

Sentence: I love my friend!
Feeling: happy

Sentence: I am feeling unhappy today.
Feeling: """).content

Here, the model is shown one completed example (“I love my friend!”) and asked to classify a new sentence (“I am feeling unhappy today”). The response would be:

sad

One-shot learning is useful when you don’t have enough data to provide multiple examples but still need the model to understand the task clearly. Isn’t it amazing how much the model can do with just one example? 😊

OBJECT or JSON output

Why would you want structured output from an AI model? Imagine you’re building an application where the output needs to be clear, organized, and easy to process by other parts of your code. Instead of returning freeform text, the model can output data in a structured format, like an object or JSON. This ensures the response is predictable and usable directly in your program.

For example, you can define a BaseModel in Python using the pydantic library to specify the exact structure of the output. Here’s how it works:

from pydantic import BaseModel, Field

class Feeling(BaseModel):
    feeling: str = Field(description="either happy or sad")
model_with_structure = model.with_structured_output(Feeling)

response = model_with_structure.invoke("""
Classify the following sentence as either 'happy' or 'sad'.

Sentence: I love my friend!
Feeling: happy

Sentence: I am feeling unhappy today.
Feeling: """).feeling
print(response)
# Output: sad

In this case, the model ensures that the output fits the Feeling structure. This makes it easier to integrate the result into other parts of your system, such as user interfaces or databases.

What about JSON? If you prefer, you can request the model to return the output in JSON format. JSON is widely used and easily parsed by many programming languages. Here’s an example:

from pydantic import BaseModel, Field

model_with_structure = model.with_structured_output(None, method="json_mode")
result = model_with_structure.invoke("""
Classify the following sentence as either 'happy' or 'sad'.
Return a JSON object.

Sentence: I love my friend!
Feeling: happy

Sentence: I am feeling unhappy today.
Feeling: """)
print(result['Feeling'])
# Output: sad

By using structured output, your application becomes more reliable. You no longer need to parse or guess what the model meant. Instead, you get exactly what you asked for – ready to use, clean, and clear.

Named Entity Recognition

Named Entity Recognition (NER) is a process where we identify specific types of information in text, such as names, dates, jobs, or places. Why is this important? Imagine you have a long document and need to find key facts quickly. NER helps by tagging these facts automatically, saving time and effort. But how does it work in practice?

For example, look at this prompt where the model is asked to identify and label entities in sentences. The allowed labels include NAME, YEAR, JOB, and CITY:

prompt = """
Identify and label the entities in the following sentence. Allowed labels are: NAME, YEAR, JOB, CITY.

Sentence: John Smith was born in New York. He started his own company in 2005.
Entities: NAME: John Smith, YEAR: 2005, JOB: self employed, CITY: New York

Sentence: Ann working on large datasets in modern logistic companies.
Entities: NAME: Ann, YEAR: None, JOB: data scientist, CITY: None

Sentence: Bob works in London since 2010. He owns a small shop with souvenirs.
Entities: """

model.invoke(prompt).content

The model’s response clearly identifies the relevant details:

NAME: Bob, YEAR: 2010, JOB: shop owner, CITY: London

This shows how NER simplifies information extraction. Can you imagine how useful this is for summarizing news articles, organizing data, or even helping customer service? With clear rules and examples, NER becomes an effective tool for understanding and structuring information. 🎯

Chain-of-thought

Chain-of-thought prompting is a method where the model solves problems step by step, breaking down complex tasks into smaller, easier-to-understand phases. This approach is like guiding the model to “think out loud,” helping it reach the correct answer by logically connecting each step. But why is this useful? 🤔 It mirrors how humans solve problems by organizing their thoughts, which makes it especially helpful for tasks involving reasoning or math.

For example, take the following prompt:

prompt = """
Problem: John has 11 books. He has lent one of them. How many books does John have now?

Phase1: John has 11 books.
Phase2: John has lent one of them.
Phase3: John has 11 - 1 = 10 books.

Answer: John has now 10 books.

Problem: Ann has 2 cars. She has rented one of them. After that she has bought one more. How many cars does Ann have now?
"""
model.invoke(prompt).content

The chain-of-thought reasoning helps the model solve the problem in a structured way:

Phase 1: Ann has 2 cars.  
Phase 2: Ann has rented one of them.  
Phase 3: Ann has 2 - 1 = 1 car left.  
Phase 4: Ann has bought one more car.  
Phase 5: Ann now has 1 + 1 = 2 cars.  

Answer: Ann has now 2 cars.

By dividing the problem into logical phases, the model ensures that every step is clear and easy to follow. This method is helpful for learners and professionals alike, as it mimics the way humans approach problem-solving – step by step, one phase at a time.

Multi-task Prompting

Have you ever wanted a model to do more than one thing at a time? Multi-task prompting makes this possible! It allows you to ask the model to complete multiple tasks in one single prompt. This not only saves time but also ensures that all tasks are connected to the same input. For example, imagine you want a summary of a text and also need key points from it. Why write two separate prompts when one can do the job? 🧠

Here’s how it works:

prompt = """
Below this is a sample text.
1. Resume this text in 100 chars
2. Write down the three points.

Text: String theory is a scientific idea that suggests that the smallest building blocks of the universe are not tiny particles, like atoms, but rather tiny, vibrating strings. These strings can vibrate in different ways, and the way they vibrate determines the type of particle they represent, such as electrons or quarks. Essentially, string theory aims to explain how everything in the universe is connected and how the fundamental forces of nature work together.
"""
model.invoke(prompt).content

The output might look like this:

1. String theory posits that the universe's building blocks are tiny, vibrating strings, not particles.

2. 
- Smallest building blocks are vibrating strings, not particles.
- Vibrations determine particle types (e.g., electrons, quarks).
- Aims to explain connections and fundamental forces in the universe.

Why is this approach effective? It ensures that the model produces clear and organized outputs while handling multiple tasks with ease. This can be helpful for summarizing articles, analyzing texts, or preparing information for presentations. 😊

ChatPromptTemplate

ChatPromptTemplate is a useful tool in LangChain that helps you create dynamic prompts by inserting variables into a template. But why is this important? Imagine you want to explain complex ideas, like “string theory,” in simple terms for children. Wouldn’t it be great if you could reuse the same structure and just change the word you’re explaining? That’s exactly what ChatPromptTemplate does!

Here’s an example. In this case, we are asking the model to define “string theory” in a way that kids can easily understand:

from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate.from_template("Write down the definition of the word {word} in simple and understandable language, suitable for kiddie level.")
chain = prompt | model
chain.invoke({"word": "the string theory"}).content

String theory is an idea in science that says everything in the universe, like stars, planets, and even tiny particles, is made up of tiny, wiggly strings. These strings are so small that we can't see them, but they vibrate in different ways, kind of like guitar strings. The way they vibrate helps to create all the different things we see around us. So, instead of thinking of particles as little dots, string theory tells us to think of them as tiny strings that can make all sorts of shapes and sizes!

As in the previous examples, we can get the stringified response thanks to StrOutputParser:

from langchain_core.output_parsers import StrOutputParser

prompt = ChatPromptTemplate.from_template("Write down the definition of the word {word} in simple and understandable language, suitable for kiddie level.")
chain = prompt | model | StrOutputParser()
chain.invoke({"word": "the string theory"})

String theory is an idea in science that says everything in the universe, like stars, planets, and even tiny particles, is made up of tiny, wiggly strings. These strings are so small that we can't see them, but they vibrate in different ways, kind of like guitar strings. The way they vibrate helps to create all the different things we see around us. So, instead of thinking of particles as little dots, string theory tells us to think of them as tiny strings that make up everything!

And if we want a structured result, such as a JSON object, we can add the JsonOutputParser:

from langchain_core.output_parsers import JsonOutputParser

prompt = ChatPromptTemplate.from_template("Write down the definition of the word {word} in simple and understandable language, suitable for kiddie level. Return the JSON object with the key: 'meaning'")
chain = prompt | model | JsonOutputParser()
response = chain.invoke({"word": "the string theory"})
response['meaning']

String theory is an idea in science that says everything in the universe is made up of tiny, tiny strings that are too small to see. These strings can vibrate, kind of like how a guitar string makes music when it vibrates. Depending on how they vibrate, they can create different things, like particles that make up atoms. So, string theory tries to explain how everything in the universe works by looking at these little strings!

Meta-prompting

What is Meta-prompting? It’s a clever way of improving a prompt by using the LLM itself to help! How does it work? First, you create an initial prompt, and then you ask the LLM to enhance it. The goal is to get a more detailed, structured, or specific version of the original prompt. This process uses a variable called meta_prompt, which includes the original prompt that needs improvement. Simple, right? 😊

Why use Meta-prompting? Sometimes, crafting the perfect prompt can be tricky. By letting the LLM refine it, you can save time and ensure better results. For example, you might want the improved prompt to guide the LLM in generating highly structured or in-depth answers. And here’s a smart tip: use a more advanced model to improve the prompt. This way, even a simpler model can handle the task well because the improved prompt does most of the heavy lifting.

Here’s an example of how to implement Meta-prompting in code:

from langchain.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser

def run_chain(prompt_template, model_name, inputs, key="article"):
    prompt = ChatPromptTemplate.from_template(prompt_template)
    model = ChatOpenAI(model=model_name, openai_api_key=OPENAI_API_KEY)
    chain = prompt | model | StrOutputParser()
    return chain.invoke(inputs)

simple_prompt = """
Summarize this news article: {article}
"""
meta_prompt = """
Improve the following prompt to generate a more detailed summary. 
Adhere to prompt engineering best practices. 
Make sure the structure is clear and intuitive and contains the type of news, tags, and sentiment analysis.

{simple_prompt}

Only return the prompt.
"""
inputs = {"simple_prompt": simple_prompt}
improved_prompt = run_chain(meta_prompt, "o1-preview", inputs, key="simple_prompt")

article = """String theory is an idea in science that says everything in the universe, like stars, planets, 
and even tiny particles, is made up of tiny, wiggly strings. These strings are so small that we can't see them, 
but they vibrate in different ways, kind of like guitar strings. The way they vibrate helps to create all the different things 
we see around us. So, instead of thinking of particles as little dots, string theory tells us to think of them as tiny strings 
that can make all sorts of shapes and sizes!"""

result = run_chain(improved_prompt, "gpt-4o-mini", {"article": article}, key="article")
print(result)

The simple prompt evolves from a basic request, such as:

Summarize this news article: {article}

to a more sophisticated and detailed one:

Please provide a detailed summary of the following news article, ensuring that the structure is clear and intuitive. Your summary should include the following elements:

- **Type of News**: Specify the category or genre of the news (e.g., Politics, Technology, Sports, etc.).
- **Tags**: List relevant keywords or topics associated with the article.
- **Sentiment Analysis**: Analyze the overall sentiment of the article (e.g., positive, negative, neutral) and provide a brief explanation for your assessment.

**Article:**

{article}

As a result, the final API call produces enriched content, like this:

**Type of News**: Science

**Tags**: String Theory, Physics, Universe, Particles, Vibrations, Theoretical Physics

**Sentiment Analysis**: The overall sentiment of the article is positive. This is due to the tone of wonder and fascination conveyed through the description of string theory, which presents an intriguing and innovative perspective on the composition of the universe. The language used evokes curiosity about the nature of reality, indicating an optimistic embrace of scientific exploration and understanding.

Meta-prompting turns a simple prompt into a powerful tool, enabling precise and structured outputs with minimal effort.

Conclusion

In my experience, using LangChain with OpenAI has completely changed how I approach prompt engineering. The biggest advantage? It simplifies creating and refining prompts, making the process not only faster but also much more effective. It fits seamlessly into any workflow, whether you’re brainstorming ideas or automating tasks.

Whether you’re a developer, a writer, or simply curious, these tools are designed for everyone. I encourage you to try them out and see the difference they can make in your projects! 😊