Eight layers, one decision
Roles, groups, permissions, tag views, and AI policy resolve into a single answer for every request, whether it arrives from the UI, REST, the CLI, or an AI agent.
Etlworks blog
AI and data integration, best practices that survive contact with production, lessons from real deployments, and the occasional strong opinion about things that matter. Written by the people building Etlworks.
Roles, groups, permissions, tag views, and AI policy resolve into a single answer for every request, whether it arrives from the UI, REST, the CLI, or an AI agent.
Every data tool shipped an AI feature in the last eighteen months. Most are demos. Here is the test that separates them.
Vendors raised prices in 2025, raised them again in 2026, and will raise them in 2027. We kept ours where they were and doubled the product instead.
Native Databricks flow types ship with three MERGE strategies, TEXT-only staging for schema drift, and an override for when the generated SQL is wrong.
Resumable snapshots, an offset editor with backup and restore, chunked parallel snapshots, and AS400 work that has no equivalent in stock Debezium.
A layered permission model enforced on every channel, an agent that creates any artifact, and parallel extraction for large tables.
A language model will confidently offer you a column that does not exist. The completion in our code editors reads your actual schema instead.
Most teams running Airflow maintain a Python framework in order to accomplish 'run this every morning'. There is a shorter path.
The in-app CLI runs SQL over the JSON that commands return, scripts with loops and captured variables, and doubles as the tool the agent uses to operate the platform.
No-license downloads for Linux and Windows, a Windows CLI to match the Linux one, and no form to fill in. For teams refreshing their tooling.
Interrupted CDC snapshots resume where they stopped, the offset editor gained backup and restore, and the host CLI reached Windows parity.
Building an order document with line items used to mean writing a loop. Two kinds of lookup replace the JavaScript.
Run history says what happened. It does not say whether your schedules collide, which flow is eating the box, or what breaks next. That is what Insights is for.
A translator, an AS2 gateway, and a mapper are usually three products and three invoices. Version 9.6.4 put all three on the engine that already runs your pipelines.
Native Databricks flow types, full AS2 support for EDI, a schema-driven X12 editor, and Etlworks itself became an MCP server.
Every engineer gets unlimited access to frontier models from both OpenAI and Anthropic. Nobody counts tokens, and nobody counts how much code the models wrote.
Half the founders I talk to are building agent meshes. We run three agents in total. The reason is accountability, and you cannot fire an agent.
A new runtime stack, CPython replacing Jython, built-in SSO, official Kubernetes support, and dbt. Nothing here has shipped yet.
The AI agent and the canvas-based flow builder both landed in this release, along with a rebuilt mapping editor.
Page, offset, cursor, nextLink, and time. The HTTP connector handles all five as connection settings, so paging an API stops being a scripting exercise.
No posts match those tags yet.