Technique
Use Etlworks UI
Use Etlworks graphical user interface to create, test, schedule and
monitor connections and data integration flows.
|
Documentation
|
Why do you need it
- To create, test, schedule and monitor connections and data
integration
flows
- To manage users and tenants
- To manage your Etlworks account
- To personalize the development environment
- To monitor the health of the Etlworks instance
|
How To
Use Explorer
Etlworks Explorer helps you see and understand your data. Whether
it’s in a spreadsheet, an SQL database, a file, or a web service,
you can connect and visualize any data anywhere.
|
Documentation
|
Why do you need it
- To explore data sources and their structure, regardless of the
format
or location. Use it to develop and debug data integration flows
- To see data in a grid and raw (original) format. Discover
dependencies
by drilling down into the nested datasets, quickly filter out and
sort data
- To use SQL to query data in any format, including databases, web
services, structured and flat files. Leverage your natural ability
to spot visual patterns quickly. Share queries with a team
|
How To
Use built-in version control
Etlworks automatically tracks changes for all design artifacts:
Connections, Formats, Listeners, Flows, Macros, and Schedules. There
is nothing to configure except the retention policy.
|
Documentation
|
Why do you need it
- To view the history of changes or who-changed-what-and-when
- To compare any two versions
- To revert to any previous version
- To add comments to the commit when saving the artifacts
|
How To
Export and import design artifacts
In Etlworks, you can import a whole Flow (or multiple Flows) with all its Connections and Formats, then export it to any tenant or environment.
|
Documentation
|
Why do you need it
- To copy flows, connections and formats to different environments and tenants
- To duplicate flows, connections and formats
|
How To
Configure flow to resume from last step
It is possible to configure the ETL Flow which has multiple source-to-destination transformations or a single transformation configured with a wildcard source to retry failed or not yet executed steps (transformations) on restart.
|
Documentation
|
Why do you need it
- To save time when executing flows. A Flow configured to retry failed
or not yet executed
steps on restart (manually or by the scheduler) will skip the
successfully executed, retry failed, and execute not yet executed
steps
|
How To
Process data by wildcard
In Etlworks it is possible to process data sources (such as database
tables) and files by a wildcard.
|
Documentation
|
Why do you need it
- To save time and effort. Instead of configuring multiple source-to-destination transformations (which can be hundreds or even thousands), you can create a single transformation using wildcard patterns for the source and destination
|
How To
Build complex pipelines
In Etlworks there is an easy way to combine multiple Flows into a
single nested Flow (pipeline), add conditions, loops, and parallel
execution.
|
Documentation
|
Why do you need it
- Using this technique, you can create very complex data integration
pipelines, where one Flow is used as input for another, where Flow
is executed multiple times based on conditions, or where Flow is not
executed at all when conditions are not met
|