FinDDR 2025

Financial Document Deep Research Grand Challenge

Nov 15-18, 2025, Singapore

Co-located with The 6th International ACM International Conference on AI in Finance (ICAIF)

ACM ICAIF 2025

Overview

Financial analysis is crucial for informed decision-making among stakeholders of public companies. Yet extracting insight from lengthy and complex annual reports remains a significant challenge. Utilizing the proven capabilities of Deep Research Agents, we propose the Financial Document Deep Research (FinDDR) Challenge, which adopts similar deep research methodologies in question design and evaluation frameworks. The FinDDR Challenge introduces a richly structured, industry-diverse dataset and requires participants to generate comprehensive, sectioned research reports. This is accomplished through a hierarchical, stepwise reasoning framework that closely emulates the analytical methodologies employed by professional financial analysts. In conclusion, the FinDDR Challenge seeks to establish new benchmarks for complex document-based deep research in financial AI applications, fostering progress and collaboration across both academic and industry communities.

Task and Dataset

The task requires participants to generate a comprehensive and deep research report based solely on a pair of multi-year annual reports of a listed company following a desired report structure. The use of external data sources other than provided annual reports (e.g., websites, news articles, financial data providers) is strictly prohibited. This structure is designed to emulate the analytical workflow of professional financial analysts, progressing from basic fact extraction, to calculation of financial indicators, to in-depth interpretation and summarization.
In the following, we provide the detailed description about the input raw annual report and the hierarchical structure of the desired output research report:

Diverse Input Raw Annual Report（Download）
- 4 Languages: English, Simplified Chinese, Traditional Chinese and Indonesia Bahasa
- 8 Markets: USA, UK, China, Hong Kong, Australia, Singapore, Indonesian, and Malaysia.
- 15+ Industries: Property & Real Estate, Healthcare & Medical Services, Consumer Goods & Services, Technology, Aerospace & Defense, Telecommunications, Oil & Gas, Transport & Logistics, Mining & Resources, Plantation, Utilities
Hierarchical and Comprehensive Output Report Structure (Download）
- Company Overview: This section provides a concise overview of the company, including:
- Financial Performance: This section presents a detailed analysis of the company’s financial health, including:
- Business Analysis: This section provides a summary and analysis of a company’s business performance and strategies, including:
- Risk Factors: This section identifies and discusses the principal risks the company faces, including:
- Corporate Governance: This section outlines the company’s governance framework, including:
- Future Outlook: This section provides management’s projections and strategic plans for the future, including:

FinDDR Dataset

For every market in the FinDDR Dataset, we provide an equal number of companies from various industries. Two annual reports (2023 and 2024) for each company are provided. The FinDDR Dataset is divided into sample, validation, and test sets. Participants can develop their methods using the sample set and evaluate performance on the validation and test sets. The raw annual reports for both the sample set and validation set are now available. For the sample set, we also provide the corresponding sample deep research reports (in both Markdown and Word formats). The raw annual reports for the test set will be released soon.

Market	Sample Set	Validation Set	Test Set
US	1	6	6
UK	1	6	6
China	1	6	6
Hong Kong	1	6	6
Singapore	1	6	6
Australia	1	6	6
Indonesia	1	6	6
Malaysia	1	6	6
Total	8	48	48

Evaluation

We apply two metrics to evaluate different sections, the actual metrics used for each section can be found in the defined structure of the deep research report）.

Accuracy: we will use advanced LLMs to judge if the submitted answers are factually correct by comparing them with the ground truth, where an LLM takes question, submitted answer, and ground truth as input and outputs the evaluation result.
DeepEval.SummarizationMetric: The summarization metric uses LLM-as-a-judge to determine whether your LLM (application) is generating factually correct summaries while including the necessary details from the original text. More details can be find via Link

The final score for a research report is a weighted sum of its section scores. The overall score of the submitted model is then calculated by averaging the scores of all reports.

Winner Verification: Top-performing teams will be invited to present their solutions at the FinDDR 2025 venue. To ensure fairness and reproducibility, winning teams will be required to provide supporting materials such as code, prompts, and relevant documentation to verify that their reports are AI-generated and their results can be reproduced.

Leaderboard

Leaderboard on Validation Set

Model	Team	Overall Score	US	UK	China	HK	Singapore	Australia	Indonesia	Malaysia	Last Update
GPT-4.5	-	-	-	-	-	-	-	-	-	-	-

Leaderboard on Test Set (Final)

Model	Team	Overall Score	US	UK	China	HK	Singapore	Australia	Indonesia	Malaysia	Last Update
GPT-4.5	-	-	-	-	-	-	-	-	-	-	-

Submission

To submit your predictions for the validation or test set, please send your results in Markdown format (Samples) to finddr2025@gmail.com. The predicted report should use the Case ID as the file name like the sample reports. For example, if the Case ID is "val001", the file name should be "val001.md".
Please use the following subject in your email: FinDDR2025-{Val/Test}-{Model Name}-{Team Name}

Team Member Information: Please include the names and affiliations of all team members directly in the body of your submission email. Each team may have up to 6 members. We will consider the team member list from your most recent submission as the final version for your team.

Important: Each team may submit up to 3 times per set per day. Please allow up to two days for leaderboard updates.