RPG Prompt Engineering Framework - output higher quality responses from LLMs

Unlock higher quality outputs with the RPG Prompt Engineering Framework

Prompt Engineering for Developers: Introducing the RPG Framework

Gareth Cull Profile Pic Gareth Cull
  • What is prompt engineering? Prompt engineering is the practice of designing instructions for large language models that elicit a desired response or output. It typically involves crafting specific instructions, questions, or context that guide a natural language model to produce more accurate, relevant, and useful outputs.
  • What is a prompt engineering framework? A prompt engineering framework is a structured approach or system designed to streamline the process of creating high-quality prompts, ensuring consistency, efficiency, and improved business outcomes when working with LLMs across a variety of applications and use cases.
  • What is the RPG Framework for Prompt Engineering? RPG stands for Role, Problem and Guidance. This framework was designed to align the LLM towards producing more relevant and higher quality outputs by clearly defining its Role, the problem the models needs to solve, along with guidance for how to solve the stated problem.
  • Are there any pre-requisites for this article? Having some prompt engineering, software or data engineering experience is helpful for getting the most out of this article.

Introduction

Developers unlocking higher quality outputs from LLMs often start with a good prompting framework.

But building an effective and repeatable framework takes a lot of deliberate practice and iteration.

Whether you're a beginner, an expert or even a non-developer, I believe this method can help you improve your results when working with models such as GPT-4o or Google Gemini Pro.

This framework is the result of my many years of iteratively prompting GPT models and I would love to know what you think.

But what is a prompt engineering framework? And why do i need one?

Well, the purpose of this guide is to walk you through how to get started building your own framework and why it is important. I'll also walk through a very specific use case that I've been working on at prototypr.ai to bring our learnings together.

As part of this tutorial, I assume you have experience with prompt engineering and some python programming for the example. So if this is you or you are interested in these areas, please read on!

Now let's get started!

Table of Contents

This guide will cover the following:

What is a prompt engineering framework?

Before we dive into the RPG framework, let's briefly touch on what a prompt engineering framework is.

A prompt engineering framework is a structured approach or system designed to streamline the process of creating high-quality prompts, ensuring consistency, efficiency, and improved business outcomes when working with LLMs across a variety of applications and use cases.

You can think of a prompt engineering framework as a toolbox that is equipped with techniques and best practices to help you "speak the language" of AI more effectively.

And while there are many different approaches to crafting better prompts, these frameworks often share some common features such as:

  • A Well Defined Structure: How prompts are formatted, including placeholders for dynamic content.
  • Clear Instructions with Objectives: as it is important to state the expected outcome to avoid ambiguity
  • Providing Contextual Information: so the model can better understand the problem it is solving
  • Incorporating Prompt Engineering Techniques - such as Few-shot Learning, Chain-of-thought Prompt Reasoning and Role-playing
  • Ethical Considerations: where developers put on a responsible AI hat and ensure that prompts do not lead to the generation of harmful, biased, or inappropriate content

These frameworks provide the necessary structure to help AI practitioners more effectively harness the capabilities of large language models.

Benefits of using a prompt engineering framework

Now let's quickly touch on the benefits of using a prompt engineering framework. Prompt engineering frameworks provide several key benefits. Here are 3 of the more prominent advantages:

  • Enhanced Clarity and Consistency: Frameworks provide a structured approach to prompt design, ensuring clarity in instructions given to the language model. This reduces ambiguity and leads to more consistent, predictable outputs that better align with your intended goals.
  • Improved Output Relevance and Quality: By incorporating techniques like few-shot learning and role-playing, frameworks enable you to guide the model's behavior more effectively. This results in outputs that are more relevant to the task, meet specific criteria, and exhibit higher quality overall.
  • Streamlined Workflow and Efficiency: A framework provides a systematic approach for prompt creation, experimentation, and refinement. This streamlines the entire process, saving you time and effort compared to ad-hoc prompt engineering, and especially so for more complex tasks.

By creating frameworks for your prompt engineering efforts, you and your organization can more effectively harness the power of large language models, leading to improved AI solutions and more positive business outcomes.

Now that we know a bit more about prompt engineering frameworks and their benefits, let's dive into the RPG Framework!

What is the RPG Framework?

RPG stands for Role, Problem and Guidance.

It’s been designed with the goal of setting the model up for success and I like to think its very much aligned with how people like to work.

So, let’s breakdown this RPG!

R = Role

It’s important to clearly define the role of the model, which helps set context and expectations for your output.

It also ensures that the model generates responses that are aligned with a desired expertise or perspective. Make sure the roles are specific enough to guide the model effectively. For instance, "software engineering assistant" is more precise than "assistant."

Example: You are an expert data engineer and world class data analyst specializing in building complete end-to-end measurement solutions on the Google Marketing Stack.

P = Problem

By clearly stating the problem, you can direct the model's efforts toward a specific task, which can be helpful in generating more accurate and relevant responses. Providing context here is key.

Example: A business stakeholder needs help answering an analytics question for an upcoming piece of analysis. You will need to build an API request to help fulfill that user need. This request will be then used to fetch data for analysis.

G = Guidance

Offering guidance on how the model can accomplish its task ensures that responses are not only correct but also actionable and in a desired format. This is probably where you’ll be spending most of your time iterating.

Example: When I’m offering guidance to a GPT model, I like to:

  • Format guidance as a bulleted list
  • Offer up examples of inputs and outputs to help steer the model
  • Define how the model response should be formatted (eg. HTML, JSON, a concise summary, etc)
  • Provide documentation as needed to help complete the task
  • Put on a responsible AI hat and define what is an acceptable request and refuse ones outside of that.

These are just some starting points, but there’s much more you can add here to build better outputs!

A practical example using the RPG Framework

So let's bring this all together and I'll walk you through a problem that most analysts experience on a day-to-day basis and how I applied the RPG framework to help me analyze Google Analytics data with GPT-4o.

To put this into context, I've been building an an AI powered Analytics workspace to help businesses rapidly generate dashboards with ai and explore data with leading large language models.

I’m particularly interested in understanding how models such as GPT-4o and Google Gemini can summarize and uncover insights into what is driving KPIs higher or lower vs. a previous period. And as an Analtyics professional, I am constantly looking at data and trying to understand what is causing data to fluctuate on a day to day basis.

I built a KPI Explorer feature into my Analytics AI workspace that allows customers to explore a KPI across 4 different dimensions. Having a tool like this has saved me countless hours and has been very helpful in supporting me with data analysis.

Here is a short video about how the KPI Explorer works:

This AI Analytics tool surfaces insights into your Google Analytics data using a GPT-4o RPG system prompt. I'll unpack this how this system prompt works in the section below.

But at a high level, the data pipeline for this product (built in python), works like this:

A data pipeline for a KPI data explorer powered by GPT-4o and the Google Analytics 4 API

KPI Data Explorer - Surface Insights with GPT-4o x GA4 API

And the RPG system prompt that i am currently exploring to help me analyze Google Analytics data with GPT-4o is as follows:

                    
system_message = f"""You are an expert data engineer and world class data analyst specializing in building complete end-to-end measurement solutions on the Google Analytics Platform.

# PROBLEM
A user has noticed a significant change in one of their KPI and needs some assistance in understanding key drivers behind its growth or decline.

# ADDITIONAL CONTEXT FOR THE PROBLEM
Here are the details of the KPI that was pulled from Google Analytics:
KPI Name: {kpi_name}
Timeframe: {timeframe} vs. the previous period
KPI Value: {kpi_value}
% Growth vs. Previous period: {kpi_diff}

You will be provided with additional Google Analytics data for this KPI in a follow-up prompt. It will contain data across 4 key segments and our stakeholder needs a short summary of the key findings that identify 3-5 key drivers which are moving this metric vs. a previous period.

# GUIDANCE
Please use the following guidelines to aid with your analysis and presentation of the data:

Analysis Guidelines:
1. Provide a short 1-2 sentence introductory statement which summarizes the KPI Name, Timeframe, KPI Value and % Growth vs. the previous period. Then follow-up with 3-5 key segments which are driving this KPI higher or lower.
2. Please keep your analysis short, to the point and focus on identifying key segments that are drivers of growth.
3. If the % Growth vs. the previous period is positive, please focus on finding key segments that are driving positive growth.
4. If the % Growth vs. the previous period is negative, please focus on finding key segments that are driving negative growth.

Guidelines for validating the accuracy of your output:
1. It is extremely important to double check and verify all of your statements and insights with the raw data.
2. All published analysis of data must exactly match the raw data that is provided and be factually correct.
3. After you have written your analysis, please double check your work again and ensure that all data presented is correct before responding.
4. Please make sure all numbers included in your analysis are factually accurate before publishing. Publishing wrong numbers hurts credibility of the model.

Formatting Your Response:
1. Your answer should be outputted as HTML.
2. Please use 3-5 bullet points to outline the key segments that driving increases or decreases in our KPI
3. Use bolded HTML to highlight key points which can be used to enhance readability and enable myself to more quickly scan the insights.

Formatting Output Example:
Here is a short example of the formatted output to follow when the % growth is positive.
<p class="analysis-copy">Yesterday, we saw 24 active users visit our website, which was an <b>increase of 19% in traffic</b> vs. the previous period. Further analysis highest the following key segments that contributed most to this growth:</p>
<ol class="analysis-copy">
    <li><b>'/products/ai-surveys' landing page</b> experienced a 250% increase in traffic, which is important to investigate further.</li>
    <li><b>The 'mobile' segment showed a significant 200% growth</b>, indicating a positive impact on overall active user growth.</li>
    <li><b>Users from 'google' and '(direct)' sources each had 50% growth </b>vs. the previous period, highlighting the influence of specific platforms.</li>
</ol>

Thank you for your assistance.
"""
                    
                

This prompt is then paired with a user prompt that simulates a data request, re-iterating the problem and requesting assistance. That prompt includes cleaned data from the Google Analytics API which is analyzed by GTP-4o.

                    
user_query = f"""I have to run an analysis of some company KPIs and need some assistance in understanding key drivers behind its growth.

Here are the details of the KPI that I pulled from Google Analytics:
KPI Name: {kpi_name}
Timeframe: {timeframe} vs. the previous period
KPI Value: {kpi_value}
% Growth vs. Previous period: {kpi_diff}

I further segmented this data across 4 key segments and need a short summary of the key findings that identify 3-5 key drivers which are moving this metric vs. the previous period.

Please do not include any markdown such as ```html or ``` in your response.

Thank you for your assistance. Here is the data:
{ga4_data}
"""
                    
                

The system prompt and user query then get packaged up as a message thread and sent to GPT-4o for analysis.

                    
import openai

# OpenAI GPT-4o Chat Completion Request
completion = openai.ChatCompletion.create(
    model='gpt-4o',
    messages=message_thread_data
)
                    
                

Breaking down the RPG System Prompt

Let's unpack the system prompt and learn how to use this framework to enhance your llm outputs.

R = Role

Here is how we defined the role for the model.

                        
# ROLE
You are an expert data engineer and world class data analyst specializing in building complete end-to-end measurement solutions on the Google Analytics Platform.
                        
                    

This is a pretty straightforward description of the role we want the model to undertake along with a specialization in all things Google Analytics.

P = Problem

Here is how the problem was defined, along with some additional context. Offering context to the problem is a very important detail and helps set the model up for success.

                        
# PROBLEM
A user has noticed a significant change in one of their KPI and needs some assistance in understanding key drivers behind its growth or decline.

# ADDITIONAL CONTEXT ABOUT THE PROBLEM
Here are the details of the KPI that I pulled from Google Analytics:
KPI Name: {kpi_name}
Timeframe: {timeframe} vs. the previous period
KPI Value: {kpi_value}
% Growth vs. Previous period: {kpi_diff}

You will be provided with Google Analytics data in a follow-up prompt. It will contain data across 4 key segments and our stakeholder needs a short summary of the key findings that identify 3-5 key drivers which are moving this metric vs. a previous period.
                        
                    

As you can see, I passed in data about the KPI into the prompt which helps set the context for the model's analysis. I also mentioned that the model should expect additional data as part of a follow-up prompt. And that its objective is to identify 3-5 key drivers or segments that are causing the KPI to increase or decrease vs. a previous period.

Providing context to the model is an essential ingredient for creating better outputs.

G = Guidance

In order to guide this model towards producing optimal outputs, i chose to focus on following areas:

  • How the model should conduct their analysis
  • The importance of reviewing and producing factually correct analysis
  • How the model should format their response

Let's go a little deeper into each of these, starting with guidance for model analysis. Here are the points I used to help guide the model when conducting analysis:

                        
Analysis Guidelines:
1. Provide a short 1-2 sentence introductory statement which summarizes the KPI Name, Timeframe, KPI Value and % Growth vs. the previous period. Then follow-up with 3-5 key segments which are driving this KPI higher or lower.
2. Please keep your analysis short, to the point and focus on identifying key segments that are drivers of growth.
3. If the % Growth vs. the previous period is positive, please focus on finding key segments that are driving positive growth.
4. If the % Growth vs. the previous period is negative, please focus on finding key segments that are driving negative growth.
                        
                    

The analysis overall is a pretty straightforward task for the model to complete. I find focusing the model on providing short answers that are to the point most helpful. I also offer guidance around where to focus effort when the model sees a KPI grow or decline.

Overall, I'd say I'm quite happy with the analysis that GPT-4o provides me using this framework.

Next, let's review how we can get more accurate results from our data. Obviously, these models are not going to be 100% accurate and there are plenty of examples out there that demonstrate this. And those examples are out of scope for this guide.

But, what I would say is to think of this analysis as a first draft that you can edit, before sending it out to a stakeholder. If you frame the analysis in this manner, I believe you've set the appropriate expectations and you should be able to quickly update the analysis because you have the raw data included below.

Now let's review the guidance around validating an LLM output.

                        
Guidelines for validating the accuracy of your output:
1. It is extremely important to double check and verify all of your statements and insights with the raw data.
2. All published analysis of data must exactly match the raw data that is provided and be factually correct.
3. After you have written your analysis, please double check your work again and ensure that all data presented is correct before responding.
4. Please make sure all numbers included in your analysis are factually accurate before publishing. Publishing wrong numbers hurts credibility of the model.
                        
                    

I believe these guidelines are just natural expectations you would set for any analysis, human or not. I'm not sure if I would explicitly state the last point to a person, but from my testing I did notice it helped steer towards providing more accurate responses.

I'd also love to know what you think? How do you guide the model towards producing more accurate responses?

Next, let's move on to the final piece of guidance: Formatting.

Here are the prompt formatting guidelines that I'm currently working with:

                        
Formatting Your Response:
1. Your answer should be outputted as HTML.
2. Please use 3-5 bullet points to outline the key segments that driving increases or decreases in our KPI
3. Use bolded HTML to highlight key points which can be used to enhance readability and enable myself to more quickly scan the insights.

Formatting Output Example:
Here is a short example of the formatted output to follow when the % growth is positive.
<p class="analysis-copy">Yesterday, we saw 24 active users visit our website, which was an <b>increase of 19% in traffic</b> vs. the previous period. Further analysis highest the following key segments that contributed most to this growth:</p>
<ol class="analysis-copy">
    <li><b>'/products/ai-surveys' landing page</b> experienced a 250% increase in traffic, which is important to investigate further.</li>
    <li><b>The 'mobile' segment showed a significant 200% growth</b>, indicating a positive impact on overall active user growth.</li>
    <li><b>Users from 'google' and '(direct)' sources each had 50% growth </b>vs. the previous period, highlighting the influence of specific platforms.</li>
</ol>

Thank you for your assistance.
                        
                    

From my experience, i've always found that providing an example and explicitly stating the format of your response leads to the best outputs. The more examples you can give, the better the model will respond. But there is a trade-off between length of your prompt and time to output, so use your best judgement when providing examples.

If you find your responses are too slow, reduce the number of examples or consider fine tuning this data. If you would like to learn more about fine tuning, i put together a beginner's guide outlining how to fine tune GPT-4o mini. So check it out and let me know what you think.

Conclusion

And that's the RPG Framework!

Thank you so much for reading! If you have any feedback on how I could improve this framework, or if you have alternative methods that work well for you, please feel free to reach out to me!

If you would like to try this new GenAI reporting tool with your own Google Analytics data, I put together a sign-up form in prototypr.ai. Currently, this tool is part of a GA4 x AI closed beta that I'm running. The goal is to find customers who are interested in using AI models such as GPT-4o to support their data analysis needs. I'm currently looking to build up a few case studies before i seek approval for this GA4 implementation from Google.

So, if you or someone in your organization would like to take this new tool for a spin, I’d love to hear from you! Please reach out to me either through the sign-up form or connect with me via LinkedIn.

That's all for now. Thank you so much for reading!

Happy Analyzing!

About the Author

Hi, my name is Gareth. I’m the creator of prototypr.ai and the founder of a startup called Data Narrative. If you enjoyed this post, please consider connecting with me on LinkedIn.

If you need help with your Analytics initiatives or are looking into building with Generative AI, please feel free to reach out. I love building GenAI and full stack measurement solutions that help businesses grow.

Until next time!