VAT IT

Introduction

VAT IT, a global leader in tax recovery solutions, uses Etlworks to stream transactional data from PostgreSQL into Azure Synapse and Microsoft Fabric in real time. Faced with the need to build low-latency, reliable pipelines across production and development environments, VAT IT turned to Etlworks for its flexibility, performance, and end-to-end automation. With support for Change Data Capture (CDC), dynamic scheduling, and enterprise-grade security, Etlworks delivered a scalable solution tailored to modern data engineering workflows.

The Challenge

VAT IT encountered several roadblocks with its previous approach:

• Real-Time Data Access: Required immediate access to transactional data across environments for reporting and analytics.

• Low-Latency Pipelines: Needed sub-minute data availability to support tax automation and customer dashboards.

• Cross-Environment Consistency: Pipelines had to work seamlessly across production and development with minimal reconfiguration.

• Security and Compliance: Handling sensitive financial data demanded secure, auditable workflows.

Existing solutions either lacked true CDC support or failed to scale reliably across cloud platforms like Synapse and Microsoft Fabric.

Why Etlworks

VAT IT selected Etlworks based on its ability to:

Support Native CDC: Seamlessly capture and stream changes from PostgreSQL without full table reloads.

• Enable Real-Time Streaming: Build low-latency pipelines that deliver data within seconds to multiple destinations.

• Handle Cross-Environment Use Cases: Use the same workflow across production and dev with variable configuration overrides.

• Ensure Secure Data Flows: Encrypt data in transit, enforce role-based access, and support hybrid deployment for additional control.

The Solution

Etlworks implemented a high-performance CDC-based framework optimized for cloud data delivery:

• Change Data Capture from PostgreSQL: Leveraged native CDC support to track inserts, updates, and deletes without impacting source systems.

• Streaming to Azure Synapse and Microsoft Fabric: Built pipelines that deliver consistent data across Microsoft’s ecosystem.

• Environment-Aware Deployment: Enabled VAT IT to replicate workflows across dev and prod with minimal changes.

• Secure and Monitored Integration: Integrated logging, alerting, and encryption to meet strict compliance requirements.

Results

• Near Real-Time Data Sync: Reduced data lag from hours to seconds, enabling timely reporting and analytics.

• Improved Reliability: Built pipelines that run consistently across staging and production.

• Reduced Maintenance: Reusable workflows and metadata-driven configs cut setup time and operational effort.

• Enhanced Data Security: Delivered encrypted, auditable data flows with full compliance controls.

Key Takeaways

• Real-Time CDC Integration: Streamlined change data capture from PostgreSQL to Azure platforms.

• Multi-Environment Consistency: Enabled development and production parity for reliable deployment.

• Cloud-Native Performance: Optimized delivery for Azure Synapse and Microsoft Fabric.

• Secure by Design: Protected sensitive data with encryption, RBAC, and detailed audit trails.

Ready to tackle your most complex data challenges? Discover how Etlworks can transform your data integration workflows. Start your free trial today or request a demo.