Comparison
LakeFlow Connect ingests into Databricks, and only into Databricks. Etlworks loads Delta and Unity Catalog just as well, plus every other warehouse, database, API, and EDI partner you deal with.
The verdict
Feature breakdown
| Capability | Etlworks | Databricks LakeFlow |
|---|---|---|
| Pricing & commercial | ||
| Starting price (monthly) | $300 | Consumption — DBUs on top of Databricks spend |
| Pricing model | Fixed per tier | DBU consumption |
| Cost transparency | High — flat rate | Low — DBU rates vary by workload and tier |
| Vendor lock-in | Monthly or annual, no contract | Databricks-native |
| Integration scope | ||
| Sources | 260+ | Managed connectors for enterprise apps, databases, cloud storage, message buses |
| Destinations | Warehouses, databases, SaaS, NoSQL, files, APIs, queues, IoT, email | Databricks only |
| ETL capabilities | ETL, ELT, Reverse ETL, wildcard processing | Lakeflow Declarative Pipelines |
| API management | Full | |
| EDI processing | X12, EDIFACT, HL7, FHIR | |
| On-prem deployment | Databricks-hosted | |
| Embeddable | ||
| Transformations | ||
| Visual mapping | drag-and-drop designer with live preview | pipelines are SQL or Python |
| Scripting languages | SQL, JavaScript, Python, XSLT, shell | SQL, Python, Scala |
| Nested and hierarchical data | JSON, XML, Avro, Parquet — read, write, normalize, flatten by dragging | Partial — passes JSON through, unpacking happens downstream |
| Warehouse pushdown (ELT) | transform before load or in the warehouse, same engine | Declarative Pipelines in Databricks |
| Reusable logic | macros, templates, and 3,900+ prebuilt flow templates | notebooks and modules |
| Lookups and enrichment | Lookup Builder for cross-source lookups | |
| Data validation | validation rules with per-step error handling | expectations in Declarative Pipelines |
| Orchestration & workflow | ||
| Scheduling | Cron expressions and fixed intervals, with per-schedule parameters | Lakeflow Jobs scheduling |
| Event-driven triggers | HTTP listeners and webhooks, message queues, file and email events | Partial — file arrival triggers |
| Continuous execution | looping schedules for CDC and queue consumers | Partial — continuous sync on higher tiers |
| Visual workflow builder | Composer canvas, 200+ flow types | Partial — job task graph, not a general canvas |
| Nested workflows | nested flows with conditional and looped steps | job task dependencies |
| Run external tools | shell and SSH scripts, CLI, JavaScript, Python, SQL, HTTP calls | Partial — notebook, JAR, and Python tasks |
| Parallel execution | overlapping schedules run as independent, separately cancellable instances | parallel job tasks |
| Retries and error handling | per-step exception handling with notifications | automatic retry on sync failure |
| Run monitoring | per-schedule status, run history, automatic Flow Findings reports | sync logs and alerts |
| CDC & Streaming | ||
| CDC engine | Debezium-compatible, built-in (no Kafka required) | Log-based CDC via managed database connectors |
| Database CDC sources | MySQL, Postgres, SQL Server, Oracle, MongoDB, DB2, others | SQL Server, PostgreSQL, MySQL, Oracle |
| Streaming queues | Kafka, EventHubs, Kinesis, SQS, PubSub, ActiveMQ, RabbitMQ | Kafka, Kinesis, Event Hubs, Pub/Sub via Structured Streaming |
| IoT brokers | MQTT brokers | |
| Real-time replication | Log-based CDC, full, incremental | Incremental ingest into Delta |
| Change tracking modes | Log-based, trigger-based, timestamp/high-watermark | Log-based, incremental |
| Developer experience | ||
| REST API | full API for flows, connections, schedules, and runs | cloud API |
| CLI | full CLI with built-in SQL | cloud CLI |
| MCP server | built-in — connect Cursor, Claude, or ChatGPT to your instance | |
| Client libraries | Python, Bash, and PowerShell clients | cloud SDKs in many languages |
| Version control | built-in — automatic history, diff, and revert on every artifact | Partial — infrastructure-as-code, not artifact history |
| Embeddable / white-label | ||
| Compliance & security | ||
| SOC 2 Type 2 | audited; report under NDA, SOC 3 public | plus ISO 27001, PCI, FedRAMP |
| HIPAA | supported with a BAA | covered by the cloud BAA |
| GDPR / DPA | compliant, DPA available | |
| SSO and MFA | SAML SSO, optional 2FA, JWT stateless auth | cloud IAM and directory integration |
| Role and artifact-level access | six roles plus tag-based scoping of flows, connections, and schedules | fine-grained IAM policies |
| Encryption and data handling | TLS in transit, encrypted at rest, customer-managed PGP, SSH tunnels, IP allowlisting; rows are not persisted by default | platform KMS, customer-managed keys |
| Audit logging | admin actions logged, access logs monitored | cloud-native audit trail |
| Security testing | monthly vulnerability and penetration scans, static analysis blocking every build, periodic third-party audits | continuous, platform-wide |
| Gen AI | ||
| AI agent | Built-in agent (Simba) — builds and edits flows from chat | Partial — Databricks Assistant and Genie — code and analytics help, not cross-system flow building |
| Agent capabilities | Reads metadata, reads/samples data, writes JS & SQL, schedules, deploys, monitors | SQL and code generation, notebook assist |
| Natural-language flow building | ‘Vibe-build’ — create flows by describing what you want | Partial — Assistant drafts pipeline code |
| AI-driven mapping | Auto-suggests source-to-destination mappings | |
| Built-in analytics | Agent runs analysis on flow data and pipeline behavior | Genie, dashboards, SQL warehouse |
| Chat across product | Same agent context on every screen | Assistant across the workspace |
| CLI for agent | Full CLI access for run/deploy/monitor/manage | Databricks CLI, not agent-driven |
| Trains on customer data | Never | Per Databricks terms |