Long Context Prompting Techniques

Claude can process vast technical documents or entire books through its 100,000 token long context window. To maximize Claude's potential, Anthropic proposes two prompting techniques that improve recall performance.

The key techniques are as follows:

• Extracting relevant reference quotes first before answering the question
• Adding examples of correctly answered questions about other sections of the document to the prompt

For the experiment, government documents created after Claude's training data cutoff were used. The document was divided into sections, and Claude was used to generate multiple-choice questions, which were then randomly recombined and tested using a 'randomized collage' method.

When building the evaluation dataset, a sophisticated prompt design process was carried out, avoiding common-sense questions the model would already know, steering clear of questions about word counts due to the token-based processing characteristics, and ensuring that answers did not contain hints. This evaluation was primarily conducted with the Claude Instant (version 1.2) model to clearly confirm the performance improvement effect.