Notably, the new Lakeflow Declarative Pipelines capabilities allow data engineers to build end-to-end production pipelines in SQL or Python without having to manage infrastructure.

The second part is LakeFlow Pipelines, which is essentially a version of Databricks’ existing Delta Live Tables framework for implementing data transformation and ETL in either SQL or Python.

Prophecy has launched an integration for Databricks, one that will allow users of the lakehouse to build data pipelines more easily.

LakeFlow Pipelines: Simplifying and automating real-time data pipelines. Built on Databricks’ highly scalable Delta Live Tables technology, LakeFlow Pipelines allows data teams to implement data ...

Business users can create pipelines using visual tools, but behind the scenes, Databricks automatically embeds best practices – resilience, self-repairing capabilities, lineage tracking (which shows ...

Getting high-quality data to the right places accelerates the path to building intelligent applications," said Ali Ghodsi, Co-founder and CEO at Databricks. "Lakeflow Designer makes it possible for ...