How to Build a Financial Research Assistant with Cohere AI on AWS

• AWS and Cohere provide a tutorial for building a financial research assistant that uses Embed, Rerank, and Parse to handle complex documents like 10-K reports.
• Cohere's internal evaluations show that Rerank 5's Context Focus feature can reduce input tokens by 50-75% while retaining the vast majority of answer accuracy.
• The system supports 100+ languages and a 128K token context window, allowing multilingual search without a translation step.
• A demo using 928 pages from Bank of America, Morgan Stanley, and Wells Fargo 2024 annual reports demonstrates the pipeline on AWS infrastructure.

Architecture and Model Selection

• The recommended architecture uses Embed Pro for indexing and Embed Fast for queries to optimize speed and cost, as they share the same embedding space.
• Embedding vectors can be configured from 256 to 2,048 dimensions in float, int8, or binary formats; 1,024-dim int8 reduces storage by 8x compared to 2,048-dim float32.
• Parse converts page images to Markdown with bounding boxes, useful for scanned documents or complex tables, and is billed per hour on GPU endpoints.

Implementation and Deployment

• The demo uses ml.g5.xlarge instances in the us-east-1 region for GPU inference and OpenSearch Serverless for storage.
• The demo specifically uses Embed Pro for both indexing and querying, and Rerank Fast instead of Rerank Pro, to minimize GPU requirements.
• Deployment is available via an AWS CloudFormation template and code is published in the aws-samples/Cohere-on-AWS GitHub repository.
• Production deployment recommendations include VPC endpoints, private access, Identity Provider integration, and customer-managed AWS KMS keys for encryption.