Skip to content

Prompt engineering fundamentals

Use case

Khanmigo is an AI-powered tool from Khan Academy that they call "Tutor for learners. Assistant for teachers.” It provides personalized learning experiences for students. It offers guidance to students on math, science, and humanities problems, a debate tool, and a writing tutor. It can support teachers in several tasks, like a teaching assistant. In this blog, we’ll explore how to replicate some of Khanmigo’s capabilities designed specifically to support teachers.

One of the most critical elements of an effective learning experience is the lesson hook. A lesson hook is a strategy teachers use to capture students’ attention and sustain their engagement, serving as an entry point into the lesson’s core content. A lesson hook can take many forms—such as a thought-provoking question, an interactive activity, a quick game, a short story, or a compelling analogy. Each topic offers multiple angles for engagement. A well-crafted hook not only grabs attention but also connects the subject matter to students’ lives, sparking curiosity and motivating them to dive deeper.

Creating lesson hooks

Creating unique hooks for every topic can be challenging, especially when managing multiple classes or addressing diverse sets of students. By leveraging LLMs, teachers can quickly generate creative, topic-specific hooks tailored to different learning levels.

Automating quizzes

Once the lesson is taught, the subsequent task for a teacher involves crafting a test to assess students retention. Generating quizzes can be a tedious endeavor. By leveraging LLMs, teachers can automate the quiz creation process.

Prompt Structure

When interacting with the OpenAI API, prompts have three key roles: system, user, and assistant.

Roles

  • system: Contains instructions defined by the application developer to guide the model’s behavior.
  • user: Represents the input provided by the end-user of the application.
  • assistant: The model's response, generated by applying the system instructions to the user input.

Large Language Models (LLMs) are optimized to interpret instructions within this role-based framework. The system prompt are the constant instructions provided to the model. While optional, this system prompt (e.g., “Classify the sentiment of the input sentence. Do not answer any other questions.”) is automatically pre-pended to every user message, eliminating the need to manually write it each time.

System messages helps to enforce consistent behavior and to restrict the model to a controlled set of instructions. Since end-users cannot modify system messages, the risk of prompt injection or misuse is significantly reduced.

User and assistant roles enable: - Multi-turn conversations for interactive experiences.
- Demonstrating ideal responses to guide the model’s output.

A typical prompt structure is illustrated in the figure below. png

Zero-Shot prompting

Zero-shot prompting is where you ask a language model to perform a task without providing any examples of how the task should be done. We rely solely on a clear and descriptive instruction.
A zero-shot prompt for creating lesson hooks will be as follows:

import config
API_KEY =  config.open_api_key
from openai import AzureOpenAI
model_name = "gpt-4o"
system_message = """Create a list of 3-5 engaging lesson hooks for a  lesson keeping in mind the grade of the students.
The hooks should be designed to capture students' attention and interest at the start of the lesson.
Consider the developmental needs and interests of students at this age when crafting the hooks."""
user_prompt = "Create a lesson hook for a lesson on newton's third law of motion for grade 8 students"
from openai import OpenAI

client = OpenAI(api_key = API_KEY)

response = client.chat.completions.create(
    model=model_name,  # Or another suitable model
    messages=[
        {"role": "system", "content":system_message},
        {"role": "user", "content": user_prompt}
    ]
)
print(response.choices[0].message.content)
Here are a few engaging lesson hooks specifically designed to capture the attention of grade 8 students as they learn about Newton's Third Law of Motion:

1. **Balloon Rocket Challenge**: Begin the lesson by showing a briefly inflated balloon. Secure the balloon to a straw that threads through a piece of string stretched between two points. Release the balloon and watch it zip along the string. Ask the students what they think is happening and how it relates to Newton's Third Law: "For every action, there is an equal and opposite reaction." Encourage predictions and immediate engagement with the law's concept in action.

2. **Interactive Tug-of-War**: Divide the students into two groups and have a fun, quick tug-of-war contest using a sturdy rope. Before they begin, ask them to think about what will happen when they pull, and why both teams feel a force acting on them. After the game, discuss how Newton's Third Law applies—the force each team exerts is met with an equal and opposite force from the other team.

3. **Water Rocket Experiment**: Demonstrate a small water rocket launch outdoors. As you prepare to launch the water rocket using a bike pump or hand pump, have students predict what will happen and why. Once launched, discuss how the expulsion of water results in the rocket moving in the opposite direction due to Newton's Third Law. This visual and dynamic demonstration will help solidify the law's principles.

4. **Mystery Box with Pulleys**: Prepare a mystery box with a pulley system on top, and without revealing too much, ask a volunteer to help pull one end of a string. As the string is pulled, the opposite reaction occurs, lifting a weight or revealing what's inside the box. Capture their curiosity and enthusiasm as they try to figure out how the law of motion explains the system.

5. **Synchronized Skaters Video**: Show a short, captivating video clip of synchronized skaters or dancers pushing off one another to move in opposite directions. Ask the students to think about what forces are at play and how Newton's Third Law is manifested in real life. This activity connects the concept to something familiar and visually striking, setting the stage for a more detailed exploration.

These hooks are developmentally appropriate for grade 8 students, incorporating curiosity, challenge, and real-life applications to encourage engagement with the physical principles of motion.

That's not a bad output at all. We can further modify parameters like temperature and top-k to improve the results.

Prompt parameters

Maximum tokens

The parameter (max_tokens) refers to the maximum number of tokens that can be generated in the chat completion. With this parameter, we can modify the output length such that the generation stops (sometimes abruptly) when the token limit hits.

response = client.chat.completions.create(
    model=model_name,
    messages=[
        {"role": "system", "content": system_message},
        {"role": "user", "content": user_prompt}
    ],
    max_tokens=256
)
print(response.choices[0].message.content)
1. **Balloon Rocket Demonstration:** Start the lesson by setting up a simple balloon rocket. Inflate a balloon and attach it to a straw that is threaded onto a long piece of string stretched between two points in the classroom. Release the balloon and let it zoom across the string. Ask students why the balloon moved in the opposite direction of the released air. This visual and interactive demo is a great way to introduce Newton's Third Law: "For every action, there is an equal and opposite reaction."

2. **Pop Culture Connection:** Kick off the lesson by showing a short clip from a popular superhero movie that features an epic battle scene. Focus on a moment when the hero uses their powers to send enemies flying backward. Pause the clip and challenge the students to connect these cinematic moments with real physics by discussing how Newton's Third Law is illustrated in these dramatic, exaggerated scenes.

3. **Tug-of-War Surprise:** Organize a quick classroom tug-of-war competition with a twist. Secretly have one side let go after a short pull. Once they fall back or the other side falls, discuss what happened in terms of both forces at play during the tug-of-war, using it as a metaphor for understanding action and reaction forces.

4. **Virtual Weigh Station

Temperature

This parameter controls the variability in the response. Using a higher temperature results in potentially lower probability tokens being chosen. With higher temperatures, we could prompt the model to make multiple completions for the same prompt.

With a temperature of 0, only the highest probability tokens are chosen so multiple completions should return the same response. This is not very useful for our feature. We want to give multiple options to the teachers to choose from.

response = client.chat.completions.create(
    model=model_name,
    messages=[
        {"role": "system", "content": system_message},
        {"role": "user", "content": user_prompt}
    ],
    n=2,# This will ask the model to present two choices for completion so that the teacher has many options to choose from
    temperature=0
)
print(response.choices[0].message.content)
print('--------------')
print(response.choices[1].message.content)
1. **Balloon Rocket Race**: Begin the lesson by setting up a simple balloon rocket race. Attach a string across the classroom and thread a straw through it. Inflate a balloon without tying it, tape it to the straw, and let it go. As the balloon propels itself along the string, ask students to observe and predict what is happening. This visual and interactive demonstration will pique their curiosity about the forces at play, leading into a discussion on Newton's Third Law of Motion.

2. **Action-Reaction Tug of War**: Organize a quick tug-of-war game using a rope. Divide the class into two teams and have them pull against each other. After a few rounds, pause the game and ask students to reflect on what they felt and observed. Use their experiences to introduce the concept of action and reaction forces, setting the stage for a deeper exploration of Newton's Third Law.

3. **Mystery Box Challenge**: Present the class with a sealed "mystery box" that contains a small object. Tell students that the box will move when they push it, but the object inside will react in a surprising way. Allow them to hypothesize what might happen when the box is pushed and then demonstrate it. This will spark their curiosity and lead into a discussion about how forces interact according to Newton's Third Law.

4. **Interactive Video Clip**: Show a short, engaging video clip of a real-world application of Newton's Third Law, such as a rocket launch or a skateboarder performing tricks. Pause the video at key moments and ask students to identify the action and reaction forces they observe. This visual and relatable example will help them connect the concept to real-life scenarios.

5. **Human Spring Experiment**: Have students pair up and stand facing each other with their hands pressed together. Ask them to push against each other's hands and observe what happens. Encourage them to describe the sensations and movements they experience. This simple activity will help them physically feel the action-reaction forces, making the concept of Newton's Third Law more tangible and memorable.
--------------
1. **Balloon Rocket Race**: Begin the lesson by setting up a simple balloon rocket race. Attach a string across the classroom and thread a straw through it. Inflate a balloon without tying it, tape it to the straw, and let it go. As the balloon propels itself along the string, ask students to observe and predict what is happening. This visual and interactive demonstration will pique their curiosity about the forces at play, leading into a discussion on Newton's Third Law of Motion.

2. **Action-Reaction Tug of War**: Organize a quick tug-of-war game using a rope, but with a twist. Have two students of similar strength pull on the rope while standing on skateboards or rolling chairs. As they pull, they will move towards each other, demonstrating the action-reaction forces. Ask the class to describe what they see and how it relates to forces acting in pairs, setting the stage for a deeper exploration of Newton's Third Law.

3. **Mystery Box Challenge**: Present the class with a sealed box labeled "Mystery Forces Inside!" and ask them to hypothesize what could be inside that demonstrates Newton's Third Law. After a brief discussion, reveal the contents: a set of magnets, a toy car, and a spring. Use these items to create simple demonstrations of action and reaction forces, engaging students' curiosity and encouraging them to think critically about the law in question.

4. **Video Clip of a Space Launch**: Show a short, exciting video clip of a rocket launch. As the rocket takes off, pause the video and ask students to discuss what they think is happening in terms of forces. Use this as a springboard to introduce the concept of action and reaction, explaining how the rocket's engines push down on the ground, and the ground pushes the rocket upwards, illustrating Newton's Third Law.

5. **Interactive Simulation**: Use an online simulation tool where students can manipulate variables to see how different forces interact. Start the lesson by allowing them to experiment with the simulation, such as launching virtual objects or colliding them. Encourage them to make observations and ask questions about the forces they see in action, leading into a discussion on how these observations relate to Newton's Third Law of Motion.

We can see that there is not that much variation between the responses. On the other hand, increasing the temperature increases the variability in the output.

response = client.chat.completions.create(
    model=model_name,
    messages=[
        {"role": "system", "content": system_message},
        {"role": "user", "content": user_prompt}
    ],
    n=2,# This will ask the model to present two choices for completion so that the teacher has many options to choose from
    temperature=1
)
print(response.choices[0].message.content)
print('--------------')
print(response.choices[1].message.content)
Certainly! Engaging middle school students, particularly those in grade 8, requires a dynamic and interactive approach that stimulates curiosity and connects with their everyday experiences. Here are a few captivating lesson hooks for introducing Newton's Third Law of Motion:

1. **Balloon Rocket Challenge:**
   Begin the class by challenging students to create a simple balloon rocket. Provide each group with a balloon, string, a drinking straw, and tape. Have them tape the straw to the blown-up (but not tied) balloon, thread the string through the straw, and let the balloon go. As they watch the balloon zip along the string, prompt them to think about what pushes the balloon forward and ask how it relates to Newton's Third Law.

2. **Push-Pull Tug-of-War:**
   Set up a quick tug-of-war game using a rope with equal teams on either side. As they pull, encourage students to observe what happens when they pull and what they feel. After a brief match, discuss how despite the teams exerting force against each other, they both remain in place, leading into an exploration of the action-reaction principle.

3. **Mystery Box Experiment:**
   Present the students with a closed, lightweight box. Inside, place balls of different weights on opposite sides connected by a spring. Shake the box lightly so they hear and feel the movement inside. Ask them to hypothesize what might be happening inside the box to make it behave in such a way. After some guesses, reveal the setup and link it to Newton's concept of action and reaction forces.

4. **Interactive Video Clip:**
   Start the lesson with a short, intriguing video clip showing dramatic and unexpected results of everyday actions, such as a skateboarder pushing off a wall or a rocket launch. Pause the video at relevant moments to ask students what they think causes the movements they see, then lead into a discussion about Newton's Third Law.

5. **Real-Life Action-Reaction Scenarios:**
   Pose a series of quick, relatable scenario questions like "What happens when you jump off a small boat onto a dock?" or "Why does a swimming fish move forward?" Have students discuss their thoughts in pairs, then use their observations to introduce the concept of equal and opposite forces.

Each of these hooks uses hands-on, relatable, or multimedia methods to engage students' imaginations and spark interest in the fundamental physics that governs their world.
--------------
1. **Rocket Balloon Launch:** Begin the lesson by inflating a balloon and holding the mouth closed. Ask the students to predict what will happen when you release it. Let the balloon go and observe its flight across the room. Engage the students by asking, "Why does the balloon shoot across the room when we let it go?" Use this demonstration to introduce the concept that for every action there is an equal and opposite reaction, a fun entry point into Newton's Third Law of Motion.

2. **Tug of War Challenge:** Organize a brief tug of war between two groups of students using a rope. After they’ve exerted their energy, pose the question, "What forces were at play here, and why didn’t one team fly backwards the moment they started pulling?" This physical activity can stimulate discussion on action and reaction forces, relevant to Newton’s Third Law.

3. **Mystery Box Push/Pull:** Before the lesson, place a mysterious box at the front of the class. Challenge students to move the box without directly touching it. Provide options like blowing on it through a straw, using a fan, or using a second object to push it. After they’ve tried different methods, discuss the unseen forces at play and relate it back to Newton's Third Law.

4. **Egg Drop Surprise:** Start by holding an egg and dropping it into a small glass of water with a plate and cardboard tube balancing on top. Ask students to explain why the egg lands in the water instead of breaking elsewhere. Use this engaging demonstration to introduce how balanced and unbalanced forces result in actions and reactions.

5. **Interactive Video Clip:** Show a short, exciting video of a space rocket launch, focusing on the exhaust of gases and the upward motion of the rocket. Pause the video at strategic moments and ask the students, "What exactly is propelling the rocket upwards?" Use this captivating imagery to hook students into understanding Newton's Third Law in the context of rocketry and space exploration.

Those are decent results but the responses are still similar.

Top-p

Top-p, also known as nucleus sampling, is a strategy that controls the diversity of model outputs (just like temperature). Instead of picking tokens based only on the highest probability, the model selects from the smallest set of tokens whose cumulative probability exceeds p (e.g., 0.9).
- Lower p (e.g., 0.8): More focused, deterministic responses.
- Higher p (e.g., 0.95): More creative, diverse outputs.

This parameter is often used alongside temperature to balance accuracy and creativity in generated text.

response = client.chat.completions.create(
    model=model_name,
    messages=[
        {"role": "system", "content": system_message},
        {"role": "user", "content": user_prompt}
    ],
    n=2,# This will ask the model to present two choices for completion so that the teacher has many options to choose from
    temperature=1,
    top_p=0.9
)
print(response.choices[0].message.content)
print('--------------')
print(response.choices[1].message.content)
1. **Balloon Rocket Race**: Begin the lesson by setting up a fun and interactive balloon rocket race. Attach strings across the classroom and thread a straw through each string. Inflate balloons without tying them off and tape them to the straws. Let the balloons go simultaneously and watch them zoom across the room. Ask the students to observe what happens and why. This demonstration will immediately capture their attention and provide a tangible example of Newton's Third Law: for every action, there's an equal and opposite reaction.

2. **Virtual Reality Experience**: If available, use a virtual reality setup to give students a first-person experience of an astronaut in space. Simulate scenarios where they push against the side of a spaceship and see themselves float in the opposite direction. After the experience, prompt students to discuss what they felt and how it relates to Newton's Third Law.

3. **Magic Trick Teaser**: Perform a simple "magic" trick where you sit on a swivel chair and use a heavy book to demonstrate motion. Hold the book and spin the chair in one direction while releasing the book in the opposite direction. Ask the students to predict what will happen to the chair when the book is released. Use their curiosity to delve into the concept that actions have equal and opposite reactions.

4. **Interactive Video Clip**: Start with a short, engaging video clip of different real-world examples of Newton's Third Law, such as a skateboarder pushing off a wall, a bird taking flight, or a rocket launch. Pause the video at key moments to ask students to predict what will happen next or to explain the observed reaction. This visual introduction sets the stage for an exciting and relatable exploration of the topic.

5. **Human Tug-of-War**: Engage students in a quick game of tug-of-war using a rope in the classroom. Have students form teams and pull the rope from opposite ends. As they exert force, ask them to notice what happens and how they feel the opposing force. This physical activity will not only energize them but also illustrate the concept of equal and opposite forces in a memorable way.
--------------
1. **Balloon Rocket Challenge:** Begin the class by showing a simple balloon rocket demonstration. Inflate a balloon, attach it to a straw on a string track, and release it. As the balloon propels along the string, ask students what they think is happening. This visual and interactive setup will spark curiosity and lay the groundwork for discussing action and reaction forces.

2. **Real-Life Tug-of-War:** Organize a quick, friendly tug-of-war contest in the classroom or outside. Pair students up and have them pull on a rope. After the activity, prompt them to think about how their pulling force is met with an equal force from their opponent, leading to a discussion about Newton's Third Law.

3. **Epic Fail Videos:** Start the lesson with a compilation of funny and relatable 'epic fail' videos where people experience action-reaction scenarios, such as slipping or falling. After watching, challenge students to explain the physics behind these mishaps using Newton's Third Law.

4. **Mystery Box Experiment:** Present students with a "mystery box" filled with various objects that react to forces (e.g., a bouncy ball, a spring, a rubber band). Have them make predictions about what will happen when they apply force to these objects and then test their hypotheses. This hands-on approach encourages inquiry and ties directly into the law of motion.

5. **Virtual Reality Experience:** If resources permit, use a virtual reality headset to take students on a simulated space journey. In zero gravity, demonstrate how astronauts push against surfaces to move, illustrating Newton's Third Law in an engaging and futuristic context. Afterward, discuss how these experiences are connected to the law of motion.

Penalties

There are two types of penalties that you can use in prompting: Presence Penalty and Frequency Penalty. Both parameters help control repetition and novelty in generated text:
1. Presence Penalty
- Encourages the model to introduce new topics or words by penalizing tokens that have already appeared.
- Higher values means more diverse content, less repetition.
- Example: Useful when you want creative brainstorming without sticking to the same ideas.
2. Frequency Penalty
- Reduces the likelihood of repeating the same words or phrases multiple times. - Higher values means less redundancy, more concise responses. - Example: Helpful for summarization or generating clean, non-repetitive text. We can use presence_penalty for topic diversity while frequency_penalty can reduce word-level repetition.

response = client.chat.completions.create(
    model=model_name,
    messages=[
        {"role": "system", "content": system_message},
        {"role": "user", "content": user_prompt}
    ],
    n=2,# This will ask the model to present two choices for completion so that the teacher has many options to choose from
    temperature=0.7,
    presence_penalty=0.8
)
print(response.choices[0].message.content)
print('--------------')
print(response.choices[1].message.content)
1. **Balloon Rocket Race**: Begin the lesson by setting up a simple balloon rocket race. Inflate several balloons and let them loose across the classroom, demonstrating how they zip around when released. Ask the students to observe what happens and brainstorm why the balloons move in the opposite direction of the air being expelled. This hands-on activity will introduce the concept of action and reaction forces in an engaging way.

2. **Action-Reaction Role Play**: Start the class with a quick role-play exercise. Pair up students and have them push against each other’s hands while standing face-to-face. Ask them to describe the forces they feel and discuss how they're both exerting and experiencing equal and opposite forces. This physical demonstration will help them visualize Newton's third law in action.

3. **Video Clip from Real Life**: Show a short, exciting video clip of a rocket launch or a skateboarder performing a trick. Pause at key moments to ask students what's happening in terms of forces. How is the rocket moving upwards, or how does a skateboarder propel themselves? This visual will capture their interest and set the stage for exploring how action and reaction forces work.

4. **Mystery Box Challenge**: Present the class with a "mystery box" containing various items that exemplify Newton's third law (e.g., a rubber band, a small toy car, a bouncy ball). Have students predict how the items demonstrate action and reaction forces. After predictions, reveal the items one by one, conducting mini-experiments to see the law in action.

5. **Interactive Simulation**: Use an interactive computer simulation or physics app that visually demonstrates Newton's third law. Allow students to manipulate variables like mass and force to see how changes affect motion. This tech-based approach can captivate tech-savvy eighth graders and offer a visual representation of abstract concepts.
--------------
Certainly! Here are some engaging lesson hooks for a Grade 8 lesson on Newton's Third Law of Motion:

1. **Balloon Rocket Challenge**: Begin the class by asking students to predict what will happen if you let go of an inflated balloon without tying it. Conduct a quick demonstration where you release a balloon and let it zoom across the room. Ask students to discuss in pairs why the balloon moved the way it did, leading them to realize that the air rushing out caused the balloon to move in the opposite direction.

2. **Tug-of-War Surprise**: Set up a mini tug-of-war challenge between two students using a rope. Once they are in position, have them pull against each other and freeze. Ask the rest of the class what they observe and why neither side is moving significantly. Use this as a springboard to introduce the concept that forces come in pairs.

3. **Action-Reaction Relay**: Organize a quick relay race where students use two skateboards or rollerblades. Have one student push against a wall to propel themselves backward across the room. As students watch, prompt them to think about what's causing the motion. This hands-on demonstration sets the stage for discussing action and reaction forces.

4. **Mystery Box Experiment**: Display a sealed box with a surprise inside and ask for a volunteer to push it across a table. Tell them to focus on how much force they apply and what happens to the box. Reveal the contents (perhaps something humorous like a small rock labeled "mass") and explain how their push is met with an equal force from the box, highlighting Newton's Third Law.

5. **Video Clip Analysis**: Show a short, exciting video clip of a rocket launch or a sports play where forces are clearly at work (like a soccer player kicking a ball). Pause the video at key moments and ask students what forces they see in action. Challenge them to identify the action-reaction pairs before diving into the lesson details.

These hooks are designed to capture the interest of eighth graders by connecting scientific concepts to real-world scenarios and interactive experiences.

Few-shot prompting

While system messages could be used to control the behaviour of LLMs, it becomes hard when we expect the output to follow a specific format (e.g., JSON). In such situations, few examples go a long way in specifying the behavior of the LLM (show, rather than tell). This technique is referred to as few-shot prompting. (If only one example is provided, it becomes one-shot prompting).
png

system_message = """Create a list of 3-5 engaging lesson hooks for a  lesson keeping in mind the grade of the students.
The hooks should be designed to capture students' attention and interest at the start of the lesson.
Consider the developmental needs and interests of students at this age when crafting the hooks."""
messages = [
    {"role": "system", "content": system_message},

    {"role": "user", "content": "Create a lesson hook on States of Matter for grade 5"},
    {"role": "assistant", "content": """Bring in a few different objects (e.g., ice cube, piece of metal, balloon) and ask students to describe their properties. Then reveal that these objects are all made of the same basic building blocks - atoms and molecules - just in different states."""},

    {"role": "user", "content": "Create a lesson hook on Multiplication for grade 3"},
    {"role": "assistant", "content": """Tell a simple story about going to the store and buying multiple items of the same kind (e.g., 3 bags of apples with 4 apples in each bag). Have students act it out with manipulatives to discover multiplication."""},

    {"role": "user", "content": "Create a lesson hook on civil war for grade 7"},
    {"role": "assistant", "content": """Hook: Read contrasting first-hand accounts from soldiers on opposite sides of a major battle. Discuss how perspectives differ."""},

    {"role": "user", "content": "Create a lesson hook on Photosynthesis for grade 7"},
    {"role": "assistant", "content": """The Incredible Shrinking Student: Tell students to imagine they are shrinking down to the size of a molecule to take a journey inside a leaf. Use this to introduce the structures involved in photosynthesis."""},

    {"role": "user", "content": "Create a lesson hook on Photosynthesis for grade 7"},
    {"role": "assistant", "content": """Divide the class into teams, each representing a part of the photosynthesis process. Have them race to complete the process in the correct order."""},

    {"role": "user", "content": "Create a lesson hook on Anatomy for grade 6"},
    {"role": "assistant", "content": """"Interview a Body Part": Assign each student a specific body part or system. Have them create interview questions and then act out an interview with their assigned part, discussing its functions, challenges, and fun facts."""},

    {"role": "user", "content": "Create a lesson hook on Ancient Civilizations for grade 7"},
    {"role": "assistant", "content": """""The Lost Civilization": Students are archaeologists who have discovered a long-lost civilization.
        As they uncover artifacts and decipher ancient texts, they learn about the culture, achievements, and eventual decline of this mysterious society, inspiring a deep dive into ancient history.
        Question Hook: "Why do some societies thrive while others collapse? Is it simply a matter of luck, or are there underlying patterns?" This hook is ideal for a unit on the rise and fall of ancient civilizations.
        """
    },
    {"role": "user", "content": user_prompt}
]
response = client.chat.completions.create(
    model=model_name,
    messages=messages,
    temperature=0.5
)
print(response.choices[0].message.content)
"Balloon Rocket Challenge": Begin the lesson by challenging students to create a simple balloon rocket. Provide each group with a balloon, a piece of string, and a straw. Have them thread the string through the straw and tape the inflated balloon to the straw without letting any air out. When they release the balloon, it will propel itself along the string. Use this activity to introduce Newton's Third Law of Motion by discussing how the air rushing out of the balloon creates an equal and opposite reaction, propelling the balloon forward. This hands-on activity will capture their interest and provide a tangible example of the law in action.

The performance of few-shot prompting is tied to the examples provided in the prompt and also on the nature of the task. For example, because we provided only one hook in every example, the model responded with only one example despite us asking for 3-5 examples in the prompt. In this task, few-shot is not significantly better than zero-shot. Better examples could lead to better results.

Using examples, you can drive guide the LLM's response in new directions. While we can use as many examples as we can, with each new example we are increasing the token consumption per API call which will lead to higher costs. The number of examples should be optimised by iteratively adding and removing examples and checking its effect on performance.

Chain-of-thought prompting

Chain-of-Thought prompting is a used to guide the model's response generation by providing a sequence of related prompts or questions. Instead of a single prompt, a CoT consists of multiple interconnected steps that build upon each other to guide the model's thinking process. These steps represent the "thinking" process that we want the model to follow.

The purpose of CoT prompting is to encourage the model to generate more coherent and contextually relevant responses by guiding its thought process in a structured manner. Each step in the chain serves as a stepping stone, providing additional context or constraints for the model to consider while generating the response.

CoT prompts could also be augmented with few-shot examples, so that the prompt guides the reasoning power of the model while examples guide expected output.
png

For example, a lesson hook should have the following qualities.
1. Grabbing Attention
2. Generating Curiosity
3. Creating Relevance
4. Building Enthusiasm
5. Establishing Connection

We can use enforce these qualities in the LLM's response by crafting a chain-of-thought prompt.

cot_prompt = """
 Create an engaging lesson hooks for a topic for a given grade level classroom.
 The hooks should be designed to capture students' attention and interest at the start of the lesson.
 Consider the developmental needs and interests of students at this age when crafting the hooks.

Below are 5 steps you need to follow to craft the lesson hook. At each step, you should check if any rewriting of the hook is necessary.
If so, add or remove text to align with the step. At the end, you should give me a lesson hook that follows all the guidelines.
At each step, evaluate if the current core of the hook is effective.
If at any point you feel like the current hook is ineffective, you should restart crafting the hook with a new core idea.

Step 1: Identify the lesson topic and key concepts
- What is the main subject or theme of the lesson?
- What are the most important ideas or skills students should learn?

Step 2: Consider the students' interests, experiences, and prior knowledge
- What do the students already know about this topic?
- What aspects of the topic might the students find most interesting or relatable to their lives?
- What current events, popular culture references, or real-world examples could help connect the lesson to the students' experiences?

Step 3: Brainstorm attention-grabbing opening techniques
- Could you start with a surprising fact, statistic, or statement related to the topic?
- Is there a thought-provoking question you could pose to the students?
- Could you present a mysterious or intriguing scenario that ties into the lesson?
- Would a short, engaging story or anecdote help capture the students' attention?

Step 4: Incorporate elements of curiosity and relevance
- How can you frame the hook to make students curious to learn more about the topic?
- What aspects of the hook will make the lesson feel relevant and meaningful to the students' lives or interests?
- Could you hint at an exciting activity, demonstration, or revelation to come later in the lesson?


At the end, evaluate all the ideas you have created and finally choose a single lesson hook that's best suited.
Explain why it's the best lesson hook and then write the best lesson hook.
"""
user_prompt = "Create a lesson hook for a lesson on newton's third law of motion for grade 8 students"

messages = [
    {"role": "system", "content": cot_prompt},
    {"role": "user", "content": user_prompt}
]
response = client.chat.completions.create(
    model=model_name,
    messages=messages,
    temperature=0.5
)
print(response.choices[0].message.content)
Step 1: Identify the lesson topic and key concepts
- The main subject of the lesson is Newton's Third Law of Motion.
- The most important idea is understanding that for every action, there is an equal and opposite reaction.

Step 2: Consider the students' interests, experiences, and prior knowledge
- Students may have some basic understanding of forces and motion from previous science lessons.
- They might find it interesting how Newton's Third Law applies to everyday activities, such as sports or riding a skateboard.
- Current events like space exploration or popular culture references like superhero movies could help connect the lesson to students' experiences.

Step 3: Brainstorm attention-grabbing opening techniques
- Start with a surprising fact: "Did you know that every time you take a step, you're actually pushing the Earth away from you?"
- Pose a thought-provoking question: "What do you think happens when a superhero punches a villain with all their might?"
- Present a mysterious scenario: "Imagine you're floating in space and you push against the wall of your spaceship. What do you think will happen to you?"
- Tell an engaging story: "Once upon a time, a group of astronauts found themselves in a zero-gravity environment where every tiny movement sent them spinning in unexpected directions..."

Step 4: Incorporate elements of curiosity and relevance
- Frame the hook to make students curious: "Today, we're going to explore how every action we take has a surprising reaction, and you'll get to see it in action!"
- Make the lesson relevant: "Understanding this law can help you improve in sports, solve puzzles, and even understand how rockets launch into space!"
- Hint at an exciting activity: "By the end of the lesson, you'll get to create your own mini rocket and see Newton's Third Law in action!"

Final Evaluation and Best Lesson Hook:

The best lesson hook combines an engaging scenario with a relatable and exciting activity. We will use the mysterious scenario and tie it to an exciting activity to capture students' interest:

"Imagine you're floating in space and you push against the wall of your spaceship. What do you think will happen to you? Today, we're diving into the world of Newton's Third Law of Motion, where you'll discover how every action you take has a surprising reaction. By the end of the lesson, you'll even get to create your own mini rocket and watch it launch, seeing this law in action firsthand!"

This hook is effective because it captures students' imaginations with a space scenario, relates to their interests in science and exploration, and promises an engaging hands-on activity.

We got lesson hooks that are more comprehensive but not necessarily better solution for teachers. While CoT provides a collection of creative hooks for teachers to choose from, Few-shot offers a single, more comprehensive lesson hook.

Rephrase and respond

In the Rephrase & Respond method of prompt engineering, we ask the LLM to rephrase the original user question to a format that will enable it to answer better.
png Lets use this method to create ten questions.

system_message = """Rephrase the question for clarity, then create 10 questions for testing the knowledge for 8 grade students."""
user_prompt = "Example: 3x + 5 = 17"
messages = [
    {"role": "system", "content": system_message},
    {"role": "user", "content": user_prompt}
]
response = client.chat.completions.create(model="gpt-4o", messages=messages)
print(response.choices[0].message.content)
Please solve the linear equation: 3x + 5 = 17.

Questions for 8th-grade students:

1. What is the first step to isolate the variable x in the equation 3x + 5 = 17?
2. After subtracting 5 from both sides of the equation 3x + 5 = 17, what is the resulting equation?
3. Once you have the equation 3x = 12, what should you do next to solve for x?
4. How would you check your solution once you find the value of x?
5. If 3x + 5 = 17, what does x equal?
6. What operation is used to eliminate the constant '5' from the left side of the equation 3x + 5 = 17?
7. In the equation 3x + 5 = 17, identify the coefficient of x.
8. What is the result of dividing both sides of the equation 3x = 12 by 3?
9. How would the equation change if the constant term (+5) on the left side were a -5 instead?
10. Solve the equation 3x + 5 = 17 and write your answer as a fraction, if necessary.

LLM as a judge

In this method, we use another LLM to rate the performance of the LLM used in the original task (see figure below for an example in case of summarization). This method of using LLMs to evaluate LLMs is usualy referred to as LLM-as-a-judge. When LLMs are used to evaluate output, the system message should clearly define the rubrics used for evaluation and the key aspects of the output that should be evaluated. The advantage of using LLMs as judges is that we do not need human baselines (that are costly to collect), while writing down the rubrics for assessment is usually an easier task.
png For an example of a chat response being evaluated, refer to the blog on RAG evaluation.

In our use case, it can be useful for grading quiz difficulty. First let us create a quiz question.

system_message = """Create 1 question for testing the knowledge for 8 grade students."""
user_prompt = "Example question: Solve 3x + 5 = 17 for x"
messages = [
    {"role": "system", "content": system_message},
    {"role": "user", "content": user_prompt}
]
response = client.chat.completions.create(model="gpt-4o", messages=messages)
print(response.choices[0].message.content)
Sure! Here's a question suitable for testing the knowledge of an 8th-grade student:

Question: Simplify the expression: 4(2x - 3) + 6. What is the value of x when the expression equals 10?

Please provide your answer in a step-by-step manner.

Let us evaluate the difficulty of the question using LLM.

RUBRIC = """
Rate K-12 math question difficulty for the specified grade (1–5) using this rubric:
1 = Very easy: direct recall, single-step arithmetic, no transformations, simple numbers.
2 = Easy: single-step equation or direct substitution, minimal operations, small integers.
3 = Moderate: two-step equation (e.g., 2x - 7 = 9), combine like terms, inverse operations; may include fractions or negatives.
4 = Challenging: multi-step manipulation, parentheses, mixed operations, rational coefficients, uncommon forms; careful error traps.
5 = Hard: multi-step with nontrivial structure, nested reasoning, edge cases, unfamiliar forms, or extension beyond grade level.

Always consider grade alignment, cognitive load (number of steps), symbol density, and common error traps.
"""

evaluator_system_prompt = f"""
You are a precise K-12 math difficulty judge. 
Follow this rubric strictly. 
{RUBRIC}
"""
evaluator_user_prompt = """
    Question: {0}
    Grade Target: {1}
    Task: Evaluate difficulty using the rubric. Highlight the final answer.
""".format(response.choices[0].message.content, 8)

messages = [
    {"role": "system", "content": evaluator_system_prompt},
    {"role": "user", "content": evaluator_user_prompt}
]
response = client.chat.completions.create(model="gpt-4o", messages=messages)
print(response.choices[0].message.content)
This 8th-grade math question has a few components:

1. Simplifying the expression: 4(2x - 3) + 6.
2. Solving the equation: Set the simplified expression equal to 10 and solve for x.

Breaking it down:

- Simplifying the expression involves distributing the 4 into the terms inside the parentheses and then combining like terms.
- Solving the equation after simplifying requires setting the resultant expression to 10 and isolating x, possibly involving inverse operations.

Analysis based on the rubric:

- Multi-step: Distribute, combine terms, set equal to 10, solve. 
- Mixed operations: Includes both distributing and solving equations.
- Rational process: Involves simplifying before solving for x.
- Careful error traps: Distribution and solving correctly.

Overall, this is challenging for an 8th grader but aligns with typical algebraic manipulation skills expected at this level.

**Difficulty Rating: 4 (Challenging)**

Self-consistency

In self-consistency, we generate multiple answers to the same question and pick the answer that is repeated the most across these occurrences. This is particularly valuable for factual and reasoning heavy questions.
png Students can answer a question in multiple ways. We can create sample answers using this method and validate if they reach to the same solution.

system_message = """Solve the equation using three different reasoning paths and pick the most common answer."""
user_prompt = "Solve: 2x - 7 = 9"
messages = [
    {"role": "system", "content": system_message},
    {"role": "user", "content": user_prompt}
]
response = client.chat.completions.create(model="gpt-4o", messages=messages, temperature=1, n=3)
for choice in response.choices:
    print(choice.message.content)
Let's solve the equation  2x - 7 = 9  using three different reasoning paths:

**Reasoning Path 1: Direct Algebraic Manipulation**

1. Start with the equation:
   2x - 7 = 9
2. Add 7 to both sides to get rid of the constant on the left side:
   2x - 7 + 7 = 9 + 7
   2x = 16
3. Divide both sides by 2 to solve for  x :
   x = 16/2
   x = 8

**Reasoning Path 2: Graphical Understanding**

1. Consider the equation  2x - 7 = 9  as two separate lines:  y = 2x - 7  and  y = 9 .
2. The solution to the equation is the x-coordinate of the intercept of the two lines.
3. For  y = 9 , draw a horizontal line at y = 9.
4. The line  y = 2x - 7  intersects the horizontal line when:
   2x - 7 = 9
   From our previous calculations, this occurs at  x = 8 .

**Reasoning Path 3: Substitution and Verification**

1. Guess a possible value for  x , say for example  x = 8 , and substitute it back into the original equation:
   2(8) - 7 = 16 - 7 = 9

2. Since the left-hand side equals the right-hand side of the equation,  x = 8  is a correct solution.

The most common answer from the three reasoning paths is:
x = 8
Let's solve the equation 2x - 7 = 9 using three different reasoning paths.

**Path 1: Algebraic Manipulation**

1. Start with the equation: 2x - 7 = 9.
2. Add 7 to both sides to isolate the term with x on one side of the equation:
   2x - 7 + 7 = 9 + 7
   Simplifying this gives:
   2x = 16
3. Divide both sides by 2 to solve for x:
   x = 16/2 = 8

**Path 2: Using Substitution to Check**

1. Assume x = 8, based on solving the equation.
2. Substitute x = 8 back into the original equation to check:
   2(8) - 7 = 16 - 7 = 9
   Since the left-hand side equals the right-hand side, x = 8 is indeed correct.

**Path 3: Visual Approach (Number Line or Logical Thinking)**

1. Recognize that 2x - 7 = 9 is asking for a value of x such that when it's doubled and reduced by 7, the result is 9.
2. Adding 7 to 9 gives the result before subtraction:
   9 + 7 = 16
3. The doubled value was 16, so dividing by 2 gives:
   x = 16/2 = 8

**Conclusion:**

All three reasoning paths lead to the same solution which is x = 8. Based on the consistency across methods, the most common and verified answer is x = 8.
Certainly! Let's solve the equation 2x - 7 = 9 using three different reasoning paths and see which solution is common among them.

### Reasoning Path 1: Algebraic Manipulation
1. Start with the equation: 2x - 7 = 9.
2. Add 7 to both sides to isolate the term with x: 
   2x - 7 + 7 = 9 + 7 -> 2x = 16
3. Divide both sides by 2 to solve for x:
   2x/2 = 16/2 -> x = 8

### Reasoning Path 2: Inspection Method
1. Notice that the term 2x needs to equal 16 for the equation to hold, since 2x - 7 = 9 simplifies to 2x = 16.
2. Therefore, solve for x by dividing 16 by 2:
   x = 16/2 = 8

### Reasoning Path 3: Graphical Interpretation
1. Consider the equation as two functions: y = 2x - 7 and y = 9.
2. Their intersection point gives the solution for x.
3. Rearrange the equation 2x - 7 = 9 to 2x = 16.
4. Divide by 2:
   x = 8
5. Thus, the graphical intersection occurs at x = 8.

### Conclusion
All three reasoning paths consistently lead to the solution x = 8. Therefore, the most common and correct answer is x = 8.

Tree-of-thought

Tree-of-thought prompting is a generalization of chain-of-thought prompting where the model is prompted to take multiple reasoning paths. This forces the LLM into a deliberate reasoning mode. png This technique breaks down complex tasks into structured steps, encouraging the model to plan before answering. It is ideal for multi-step reasoning like quiz generation.

tree_prompt = """Plan the quiz step-by-step:
1. Identify topic and difficulty
2. Generate 5 MCQs
3. Provide correct answers
4. Ensure age-appropriateness
Finally, output the quiz in JSON format."""
user_prompt = "Generate a math quiz on solving two-step equations for grade 7."
messages = [
    {"role": "system", "content": tree_prompt},
    {"role": "user", "content": user_prompt}
]
response = client.chat.completions.create(model="gpt-4o", messages=messages)
print(response.choices[0].message.content)
Sure! Let's create a math quiz on solving two-step equations specifically for grade 7 students.

### Step 1: Identify Topic and Difficulty
- **Topic:** Solving Two-Step Equations
- **Difficulty:** Appropriate for Grade 7

### Step 2: Generate 5 Multiple Choice Questions (MCQs)

1. **Question 1:**
   Solve the equation:  3x + 4 = 19 
   - A) 3
   - B) 5
   - C) 8
   - D) 6

2. **Question 2:**
   Solve the equation:  2x - 5 = 11 
   - A) 8
   - B) 3
   - C) 6
   - D) 9

3. **Question 3:**
   Solve the equation:  4x + 7 = 31 
   - A) 5
   - B) 6
   - C) 9
   - D) 8

4. **Question 4:**
   Solve the equation:  5x - 3 = 22 
   - A) 4
   - B) 6
   - C) 5
   - D) 3

5. **Question 5:**
   Solve the equation:  7x + 2 = 23 
   - A) 4
   - B) 3
   - C) 2
   - D) 1

### Step 3: Provide Correct Answers
- For **Question 1**, the correct answer is B) 5.
- For **Question 2**, the correct answer is C) 6.
- For **Question 3**, the correct answer is D) 6.
- For **Question 4**, the correct answer is C) 5.
- For **Question 5**, the correct answer is A) 4.

We can double-check each solution:
1.  3x + 4 = 19  ⟹  3x = 15  ⟹  x = 5 
2.  2x - 5 = 11  ⟹  2x = 16  ⟹  x = 8 
3.  4x + 7 = 31  ⟹  4x = 24  ⟹  x = 6 
4.  5x - 3 = 22  ⟹  5x = 25  ⟹  x = 5 
5.  7x + 2 = 23  ⟹  7x = 21  ⟹  x = 3

### Step 4: Ensure Age-Appropriateness
- The language and difficulty of the questions are appropriate for grade 7 students.

### Output the Quiz in JSON Format

```json
{
  "quiz": {
    "title": "Grade 7 Math Quiz: Solving Two-Step Equations",
    "questions": [
      {
        "question": "Solve the equation: 3x + 4 = 19",
        "options": ["3", "5", "8", "6"],
        "answer": "5"
      },
      {
        "question": "Solve the equation: 2x - 5 = 11",
        "options": ["8", "3", "6", "9"],
        "answer": "8"
      },
      {
        "question": "Solve the equation: 4x + 7 = 31",
        "options": ["5", "6", "9", "8"],
        "answer": "6"
      },
      {
        "question": "Solve the equation: 5x - 3 = 22",
        "options": ["4", "6", "5", "3"],
        "answer": "5"
      },
      {
        "question": "Solve the equation: 7x + 2 = 23",
        "options": ["4", "3", "2", "1"],
        "answer": "3"
      }
    ]
  }
}
```

Please note any errors in this quiz were corrected upon verifying each answer. Be sure to confirm all answers when presenting this quiz to students.

Other advanced methods

While these are some of the popular methods, there are many advanced prompting methods that can be used:
1. Role-based prompting: Assign roles to the model (e.g., “Act as a math teacher”).
2. Instruction + Context prompting: Combine explicit instructions with contextual data.
3. Generated Knowledge prompting: Ask the model to first generate relevant facts before answering.
4. Reflexion prompting: Encourage the model to critique and improve its own answer.
5. Meta-prompting: Ask the model to explain its reasoning or summarize its approach.
6. Contrastive prompting: Provide two different perspectives and ask for comparison.
7. Multimodal prompting: Combine text with images or diagrams for richer outputs.

Solving Math problems

While we looked at these methods, we can understand that language models (LLMs) will inevitably produce incorrect responses, especially for complex mathematical problems. This is because LLMs operate probabilistically on language and aren't designed for precise calculations. While some LLMs tailored for math might perform better, there's no guarantee of always getting the right answer.

The only reliable method for accurate responses from LLMs is to utilize agents.

Back to top