Launching Qlik Open Lakehouse: A New Paradigm for Apache Iceberg Lakehouse
Launching Qlik Open Lakehouse: A New Paradigm for Building Iceberg Lakehouses
Open lakehouses powered by Apache Iceberg are transforming data management by combining the scalability of data lakes with the performance and reliability of data warehouses. This innovative architecture offers unmatched flexibility and control while embracing open standards. According to recent industry surveys, over 50% of large organizations expect to reduce analytics costs by more than 50%—some even up to 75%—by adopting lakehouse architectures. These savings come from eliminating data duplication, streamlining data movement, and independently scaling storage and compute.
Challenges in Realising Lakehouse Potential
Despite the promise of Apache Iceberg, organisations face significant challenges in maximizing the value of lakehouses:
- Lack of Native Ingestion: Apache Iceberg lacks built-in ingestion capabilities, requiring external tools to efficiently move data from operational systems into Iceberg tables.
- Complex Pipelines: Building robust data pipelines involves managing schema evolution, data quality, observability, and bursty workloads, which can be error-prone and inefficient.
- Manual Optimization Bottlenecks: Unoptimised Iceberg tables can increase storage costs and slow query performance, requiring manual tuning, compaction, and partitioning.
- Data Trust and Lineage: Without automated quality checks and end-to-end lineage, ensuring data accuracy for AI, analytics, or compliance is challenging.
Introducing Qlik Open Lakehouse
Qlik is thrilled to announce the launch of its Qlik Open Lakehouse, a new capability in Qlik Talend Cloud that simplifies ingestion, management, and optimisation of Apache Iceberg-based lakehouses. Following their acquisition of Upsolver, they’ve integrated its platform into Qlik Talend Cloud. Able to deliver a seamless, no-code experience for deploying lakehouses on Amazon S3.
With Qlik Open Lakehouse, users can load batch and real-time data into Iceberg tables with just a few clicks. There are no complex pipelines or manual configurations required. Available now in private preview.
Qlik Open Lakehouse automates schema mapping, type conflict resolution, intelligent partitioning, file compaction, schema evolution, and update/delete handling. Qlik’s Adaptive Iceberg Optimizer continuously monitors and optimises tables in real-time, delivering you:
- Up to 5x faster query performance through adaptive optimizations
- Up to 50% lower storage costs via efficient compaction and clean-ups
- A no-code experience for enterprise-scale Iceberg management
Watch the Qlik Open Lakehouse video
Qlik Open Lakehouse (see video) is included in the Standard Edition and above of Qlik Talend Cloud, making it easier and more affordable to adopt Apache Iceberg. It will be generally available (GA) in July 2025. Interested in early access? Contact us to sign up for the Early Access Program.
Key Features of Qlik Open Lakehouse
Real-Time High-Throughput Ingestion
Ingest massive volumes of batch and real-time data from over 200+ Qlik Talend Cloud sources—including databases, SaaS apps, SAP, mainframes, and CDC streams—directly into Iceberg tables on AWS S3. Qlik supports low-latency ingestion, Type 1 and Type 2 change history, and cost-efficient compute with AWS Spot Instances for bronze layer processing.
Qlik Adaptive Iceberg Optimizer
Qlik’s industry-leading Adaptive Iceberg Optimizer dynamically monitors and optimizes tables based on their unique characteristics, delivering 2.5x–5x faster query performance and up to 50% cost savings without manual tuning. Unlike other services requiring tedious configurations, Qlik automates optimizations for scalability and efficiency.
Learn how organisations like ironSource (500K events per second and over 20 billion events daily) and Cox Automotive achieved significant savings with Upsolver’s technology.
Data Warehouse Mirroring
Seamlessly replicate Iceberg table data into cloud data warehouses like Snowflake for querying or downstream transformations without duplicating data. Meanwhile, you can run ingestion and bronze layer processing with cost-effective Qlik compute and mirror data to Snowflake for further processing.
Open, Industry-Leading Integrations
Qlik Open Lakehouse is platform-agnostic, integrating with leading catalogs: (AWS Glue, Apache Polaris, Snowflake Open Catalog), and query engines: (Snowflake, Amazon Athena, Trino, Presto, Dremio, Apache Spark). It supports petabyte-scale data on Amazon S3, with plans for broader cloud storage compatibility, ensuring flexibility without vendor lock-in.
Secure and Flexible Data Management
Data stays within your AWS VPC, with Qlik-managed compute on Amazon EC2 Spot Instances for cost-effective, scalable processing. Configurable scaling strategies let you balance cost and latency, ensuring full control over performance, security, and cost.
A Unified Solution for Lakehouses
Qlik Talend Cloud with Open Lakehouse provides an end-to-end solution for ingesting, transforming, governing, optimizing, and managing Iceberg-based lakehouses. Unlike other managed Iceberg solutions that focus solely on ingestion or optimization, Qlik delivers a comprehensive pipeline from data sources to analytics and AI applications, ensuring data quality and trust.
Unlock the Full Potential of Iceberg
Qlik Open Lakehouse combines Qlik Talend Cloud’s low-latency ingestion with Upsolver’s adaptive optimization to deliver 5x faster queries and 50% lower costs. It integrates seamlessly with your existing AWS and Snowflake investments, enabling high-performance data reads and writes with zero manual effort.
Why Not Start building today!
Contact us to:
- Sign up for the Early Access Program
- Learn more about Qlik Open Lakehouse
- Download our whitepaper: The Iceberg Data Lakehouse Stack
Watch the on-demand webinar: Iceberg Ahead: The Future of Open Lakehouses
About Differentia Consulting
Qlik Elite Solutions Partner (since 2008) Differentia Consulting provides AI enabled Smarter.BI solutions. Advises, provisions, supports and maintains your end-to-end data needs; including the augmentation of data for the Information Superhighway, and Hyperautomation. Designing lean IT solutions for organisations to succeed.
To learn more about Qlik’s offerings contact us at [email protected]
