Cerebelly

Introduction

Cerebelly, a fast-growing direct-to-consumer baby food brand, uses Etlworks to automate order and invoice processing workflows. By integrating flat files stored in SharePoint with a Postgres-based backend, Cerebelly ensures accurate, enriched, and timely reconciliation of critical transactional data.

The Challenge

Cerebelly faced several operational and technical integration hurdles:

• Manual File Handling: Orders and invoices were received as flat files in SharePoint, requiring manual downloads and processing.

• Multiple Files per Batch: Needed to process a dynamic number of files daily, often varying in structure and volume.

• Data Enrichment: Required transformation and enrichment of raw order and invoice data before loading into the database.

• Reliable Reconciliation: Required automated, repeatable workflows to ensure data accuracy and minimize delays in financial reporting.

Why Etlworks

Cerebelly selected Etlworks for its out-of-the-box support for file-based ETL and ability to automate complex flows:

• Wildcard File Processing: Etlworks natively supports wildcard-enabled flows, allowing one pipeline to process many files without modification.

• SharePoint Integration: Seamless connectivity with Microsoft SharePoint enabled secure, automated file ingestion.

• Transformation and Enrichment: Robust mapping and scripting tools enabled business logic to be applied during ETL.

• Database Integration: Clean, transformed data was automatically loaded into Postgres with full control over structure and indexing.

• Monitoring and Alerts: Built-in logging and monitoring reduced risk and helped ensure data integrity.

The Solution

Etlworks implemented an automated integration pipeline for Cerebelly:

• SharePoint Watcher: Monitors designated folders for new order and invoice files.

• Wildcard File Loader: Ingests and parses multiple flat files using pattern-based logic.

• Transformation Rules: Applies business rules and data enrichment to standardize formats and compute derived values.

• Postgres Loader: Loads the final dataset into a structured Postgres schema for operational reporting and reconciliation.

• Automation & Alerts: Fully scheduled with real-time alerts for failures or anomalies.

Results

• End-to-End Automation: Entire order and invoice processing cycle automated from SharePoint to database.

• Improved Accuracy: Transformation and validation steps ensured consistent, reliable output.

• Operational Efficiency: Significantly reduced manual effort, enabling faster reconciliation and fewer errors.

• Scalability: Wildcard processing enabled the system to grow with Cerebelly’s increasing order volume.

Key Takeaways

• Wildcard File Ingestion: Etlworks processes multiple flat files in a single flow using pattern matching.

• Seamless SharePoint Integration: Orders and invoices are automatically retrieved from the cloud.

• Real-Time Transformation: Business rules applied during ETL streamline downstream processes.

• Reliable Data Delivery: Orders and invoices are loaded into Postgres for reporting and reconciliation without manual intervention.

Ready to tackle your most complex data challenges? Discover how Etlworks can transform your data integration workflows. Start your free trial today or request a demo.