Prompt | Elements of prompt engineering

A “prompt” is text (usually an instruction or question) that you provide the AI model to generate a “completion” or “output”. The completion is the text that the model generates.  

In simplified terms, completions are composed of the words the model determines are most likely to come next – which is why many have taken to referring to LLMs as a “very advanced autocomplete”.

A quick example can help illustrate the concept of prompting. Let’s write a prompt using a tool you’ve probably used before: OpenAI’s ChatGPT (the GPT-4 version).

Prompt:

Write a haiku about how helpful prompt engineering can be for lawyers.

‍
Completion:

Case facts intertwine,

Prompt weaves clarity, truth's line,

Justice is defined.

Not bad! But large language models like the ones that power ChatGPT are useful for a lot more than a spontaneous poem or two. For example, let’s try a prompt that helps us analyze a piece of legal text:

Prompt:

CLAUSE:

The Customer will indemnify, defend, and hold harmless the Company, our affiliates, and our employees or agents from and against all claims, losses, and expenses (including attorneys’ fees) arising from or relating to your use of the Company’s Services.

QUERY:

Is the indemnity clause in the CLAUSE section above mutual?

RESPONSE:

Completion:

No, the indemnity clause provided is not mutual. It only stipulates that the "Customer" will indemnify the "Company," its affiliates, employees, or agents from claims, losses, and expenses arising from or relating to the customer's use of the company's services. For it to be mutual, there should also be a provision where the company agrees to indemnify the customer under specified circumstances.

Notice how we’re already getting more sophisticated with the prompt we’ve designed. In the prompt above, we’ve referenced text, asked a question, and even provided a structured set of headings to better cue the model into generating the completion we want.

We can continue to refine our prompt like this with even more detailed instructions. For example, maybe you want to analyze several sections of an agreement at once and then output responses in a more succinct, structured way:  

Prompt:

CLAUSE 1:

Clause: This agreement is governed by the laws of the state of Maryland.

Query: What is name of the state whose governing law applies to the agreement?

Response Format: Text

CLAUSE 2:

Clause: The Customer will indemnify, defend, and hold harmless the Company, our affiliates, and our employees or agents from and against all claims, losses, and expenses (including attorneys’ fees) arising from or relating to your use of the Company’s Services.

Query:  Is the indemnity clause mutual?

Response Format: True or False

CLAUSE 3:

Clause: The effective date shall be August 12, 2023.

Query:  What is the effective date?

Response Format: Date, in the YYYY-MM-DD format

INSTRUCTIONS:

1. Carefully review the clauses contained in the numbered CLAUSE sections above.

2. For each clause, answer the applicable question in the “Query” subsection based only on the text contained in the related “Clause” subsection.

3. The response to each query should be in the format specified in the “Response Format” subsection and should not contain any extraneous or filler text, only the answer to the query.

RESPONSES:

Completion:

Response to Clause 1: Maryland

Response to Clause 2: False

Response to Clause 3: 2023-08-12

Now imagine applying this to an entire agreement or to entire document sets, and you can start to see the promise of generative AI in legal. But it’s not just generative AI at work here: it’s up to us humans to carefully craft, test, and utilize prompts that get the desired output.

This is the power of prompt design!