Amazon Aurora PostgreSQL Adds Direct Querying of Apache Iceberg and Parquet Data via Embedded DuckDB

Amazon Aurora PostgreSQL now allows direct querying of operational data alongside Apache Iceberg and Apache Parquet files in data lakes, eliminating the need for extract, transform, and load (ETL) pipelines. This capability is powered by DuckDB, which is embedded directly within Aurora following the acquisition of DuckLabs, enabling single-query joins between live transactional data and historical archives without data duplication.

The feature is supported on Aurora PostgreSQL 17 (starting with version 17.11) and 18 (starting with version 18.6). Users enable the aurora_analytics extension and attach an IAM role with the AuroraAnalytics feature to access data in Amazon S3 and the AWS Glue Data Catalog. Foreign tables can be created manually or bulk-imported using IMPORT FOREIGN SCHEMA, with schemas automatically inferred from Parquet metadata.

Aurora applies optimizations such as predicate pushdown and column pruning to minimize data reads from S3. Frequently accessed data is cached within the Aurora instance to speed up subsequent queries, and performance metrics can be monitored via aurora_analytics_stat_statements(). For workloads requiring single-digit-millisecond latency, users can materialize data lake results into native Aurora tables using standard SQL commands like CREATE TABLE AS SELECT.