System Message versus User Message | Elements of prompt engineering

While almost all large language models started with the fundamental call-and-response structure of a prompt and completion, that simple structure has evolved over time. Today many models now incorporate two types of messages, which are then combined to form a single prompt.

The first type of message is the user message. This is the specific user question or comment that the model responds to. For example, when you interact with generative AI chatbots like ChatGPT, Gemini, Claude, or Bing, what you as the user type in is the “user message”.  

But hidden from your view as a user is another type of message: the system message. The system message determines what role the AI should play and how it should behave generally. When the Bing Chatbot first launched, it famously leaked its own system message, and the revelation demonstrated that system messages can be quite extensive, covering everything from how the model should respond, the tone it should use, when it should redirect the user to another topic, and much more.

We can use a few examples to demonstrate how user and system messages work together, and why combining them can be so useful. Let’s start with a user message, without any system message:  

User Message:

What color is the ocean?

Completion:

The ocean can appear to be many colors, depending on lighting, depth, and the presence of particles or organisms. However, it is most commonly perceived as blue.

Now, let’s introduce a simple system message and see what happens when we use the same user message:

‍System Message:

No matter what the user asks, just reply “HODOR”.

User Message:

What color is the ocean?

Completion:

HODOR

While an admittedly silly example, it shows you the power of system messages. You can really influence the completion with system messages!

More realistically, you might use a system message to control how users within your organization are able to interact with a model:

System Message:

You are an AI technology built for legal professionals. You only perform tasks related to legal work. For example, you can analyze contracts and draft memos because those are legal tasks. If you are asked to do something that doesn't seem related to legal work, respond with “I’m sorry but I am only able to perform tasks related to legal work.” ‍

User Message:

Tell me about the history of the Romans

Completion:

I'm sorry but I am only able to perform tasks related to legal work.

In contrast, if we provide a user message related to legal work, the model has no problem answering the question:

System Message:

You are an AI technology built for legal professionals. You only perform tasks related to legal work. For example, you can analyze contracts and draft memos because those are legal tasks. If you are asked to do something that doesn't seem related to legal work, respond with “I’m sorry but I am only able to perform tasks related to legal work.” ‍

User Message:

What are common areas of negotiation when redlining an NDA?

Completion:

When redlining a Non-Disclosure Agreement (NDA), there are several common areas of negotiation:

1. Scope of Confidential Information: Parties often negotiate what exactly constitutes confidential information. It's important to define this as clearly as possible to avoid future disputes.

2. Obligations of the Receiving Party: The obligations of the party receiving the confidential information can be a point of negotiation. This can include how the information can be used, who it can be disclosed to, and what steps need to be taken to protect it.

3. Duration of the NDA: The length of time the NDA is in effect can be a point of negotiation. Some parties may want a shorter duration, while others may want the NDA to last indefinitely [continued]

As you can see, system messages provide a powerful tool for designing a prompt.  

Not only can they be useful in controlling the tone and overall behavior of a model, system messages can also provide an easy way to segment your instructions from the content you want the model to apply those instructions to.

For example, we can enter the instructions for analyzing a contract as the system message and the contract we want analyzed as the user message:

System Message:

1. Carefully review each clause contained in the AGREEMENT TEXT.

2. For each clause, provide the following entities: "Clause Type", "Summary", "Favors Company".

3. The CLAUSE TYPE entity is the type of clause. For example, "Clause Type: Mutual Indemnity" or "Clause Type: Notice".

4. The SUMMARY provides an extremely short summary of the clause's contents.  

5. The FAVORS COMPANY entity indicates if the clause is favorable to the Company. This entity is either True or False. For example, "Favors Company: True" or "Favors Company: False".

6. If you are unsure or don't know about one or more entities for a clause, respond with "Unknown". For example, "Clause Type: Unknown". ‍

User Message:

AGREEMENT TEXT:

1. SAAS SERVICES AND SUPPORT

1.1 Subject to the terms of this Agreement, Company will use commercially reasonable efforts to provide Customer the Services. As part of the registration process, Customer will identify an administrative user name and password for Customer’s Company account.  Company reserves the right to refuse registration of, or cancel passwords it deems inappropriate.

1.2 Subject to the terms hereof, Company will provide Customer with reasonable technical support services in accordance with the terms set forth in Exhibit C.

2. CONFIDENTIALITY; PROPRIETARY RIGHTS

2.1 Each party (the “Receiving Party”) understands that the other party (the “Disclosing Party”) has disclosed or may disclose business, technical or financial information relating to the Disclosing Party’s business (hereinafter referred to as “Proprietary Information” of the Disclosing Party).  Proprietary Information of Company includes non-public information regarding features, functionality and performance of the Service.  Proprietary Information of Customer includes non-public data provided by Customer to Company to enable the provision of the Services (“Customer Data”). The Receiving Party agrees: (i) to take reasonable precautions to protect such Proprietary Information, and (ii) not to use (except in performance of the Services or as otherwise permitted herein) or divulge to any third person any such Proprietary Information.  The Disclosing Party agrees that the foregoing shall not apply with respect to any information after five (5) years following the disclosure thereof or any information that the Receiving Party can document (a) is or becomes generally available to the public, or (b) was in its possession or known by it prior to receipt from the Disclosing Party, or (c) was rightfully disclosed to it without restriction by a third party, or (d) was independently developed without use of any Proprietary Information of the Disclosing Party or (e) is required to be disclosed by law.  

2.2 Customer shall own all right, title and interest in and to the Customer Data, as well as any data that is based on or derived from the Customer Data and provided to Customer as part of the Services. Company shall own and retain all right, title and interest in and to (a) the Services and Software, all improvements, enhancements or modifications thereto, (b) any software, applications, inventions or other technology developed in connection with Implementation Services or support, and (c) all intellectual property rights related to any of the foregoing.      

2.3 Notwithstanding anything to the contrary, Company shall have the right collect and analyze data and other information relating to the provision, use and performance of various aspects of the Services and related systems and technologies (including, without limitation, information concerning Customer Data and data derived therefrom), and  Company will be free (during and after the term hereof) to (i) use such information and data to improve and enhance the Services and for other development, diagnostic and corrective purposes in connection with the Services and other Company offerings, and (ii) disclose such data solely in aggregate or other de-identified form in connection with its business. No rights or licenses are granted except as expressly set forth herein.   ‍

Completion:

Clause Type: SAAS Services and Support

Summary: The company will provide the customer with services and technical support, and reserves the right to refuse or cancel inappropriate passwords.

Favors Company: True

Clause Type: Confidentiality; Proprietary Rights

Summary: Both parties agree to protect each other's proprietary information. The customer owns all data provided to the company, while the company owns all rights to the services, software, and any improvements or modifications.

Favors Company: True

System messages can be particularly helpful when you plan on keeping your instructions static but changing the contents of your user message. To analyze another contract using the same framework, you would simply paste the text of the new contract in as another user message and press go.

One final note: OpenAI’s GPT 4 series of models is better at “listening” to system messages than prior series like GPT 3.5. For that reason, if you are getting poor results (particularly with GPT 3.5), you might try leaving the system message blank and putting instructions directly into the user message.