Building REST APIs

In Etlworks Integrator it is possible to build REST APIs with zero coding, using drag and drop interface. The following integration scenarios are supported:

Basic ideas

The core element of the infrastructure, which allows building REST APIs in Integrator is HTTP listener. HTTP listener is a special type of connector, constantly listening to the incoming HTTP requests, using configurable PATH and METHOD.

Depending on which side of the transformation (source or destination) HTTP listener sits, it can either push payload to the flow or return response to the client.

Authentication

Any API, built in Integrator, requires an authentication using either JWT token, received from the authentication API, or user name and password.

JWT Authentication

To configure API endpoint with JWT authentication, select JWT when defining Auth Type parameter for the HTTP listener. Select either Header (recommended and more secure), or Query Parameter (less secure) for the Auth Location.

Learn about authentication API.

Read how to call Integrator's REST API endpoints when using JWT authentication.

Basic Authentication

Important: generally speaking, basic authentication is less secure then JWT. Use it if it's impossible to use two-step authentication. For example, when third party application has a simple web hook that sends the payload to the API.

To configure API endpoint with basic authentication, select Basic when defining Auth Type parameter for the HTTP listener. Select either Header (recommended and more secure), or Query Parameter (less secure) for the Auth Location.

Read how to call Integrator's REST API endpoints when using basic authentication.

Defining endpoints

In Integrator you define endpoint by specifying URL Pattern parameter, when creating an HTTP listener.

Important: only one HTTP listener with the particular URL Pattern can exist at the same time. For example, you cannot create multiple HTTP listeners with the URL Pattern /patients. You can, however, created listener with the URL Pattern /patients and listener with URL Pattern /patient.

Within URL Pattern you can specify path parameters: /patients/{patientid} and query parameters: /patients?patientid={patientid}. They will be automatically converted to the variables, which can be referenced from the source SQL. From the functional standpoint path parameters=query parameters.

Example of the endpoint with path parameters:

/patients/{patientid} --> https://integrator.etlworks.com/plugins/schedules/rest/v1/httplistener/hl7/patients/123

Example of the endpoint with query parameters:

/?patientid={patientid} --> https://integrator.etlworks.com/plugins/schedules/rest/v1/httplistener/hl7/patients/?patientid=123

Example of the SQL with parameters:

select * from principal where principalid = {principalid}

Pull

Expose data source as an REST API endpoint

Problem: we have a database, which sits behind the firewall and is not accessible from the Internet. We would like to build an REST API that can be used to access some controlled portion of the data from the other side of the firewall.

Solution: any data source, which Integrator can read, can be exposed as an REST API endpoint. Read more about available data sources.

Steps to build an REST API

Step 1. Create new HTTP listener. Select GET as an HTTP method.

Users Listener

Step 2. Create format for the response. The following formats are supported: JSON, JSON Data Set, XML, XML Data Set, CSV and Fixed Length Text Format.

Step 3. Create flow, where the source is any supported data source and the destination is an HTTP listener, created in the step 1 and format, created in the step 2.

Users Flow

Step 4. Define source query, mapping and parameters if needed.

Source Query

Step 5. Schedule event-driven flow.

Working with databases

Follow the same steps as you would follow when the source is a database.

Working with data objects

Follow the same steps as you would when the source is a data object.

Important: you can define an SQL, which will be executes on the source data object, even if it is a file, web service or data in the key-value storage.

Testing GET API endpoint.

The simplest way to test GET API endpoint right from the browser is to use Postman plugin, for example for Chrome.

Step 1. Create POST authentication request in Postman:

URL: https://integrator.etlworks.com/rest/v1/token-auth/issue

Body: {"login":"user","password":"password"}. The user and password must be existing Integrator user and password.

Step 2. Click the Send button.

Step 3. Copy token body from the response.

Response example:

{ "token": "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzUxMiJ9.eyJpc3MiOiJldGx3b3Jrcy5jb20iLCJpYXQiOjE0OTMyNDMwNDQsImV4cCI6MTQ5MzI0MzY0NCwianRpIjoiMjJiMzQ5NjktOTU5MS00NGRlLTg5OTEtOWExMjZiMmNmNDIzIiwic3ViIjoiYWRtaW4ifQ.fgjJRqjPfVSXpgB7o7A6Q7ohYyoWovKZwdvatojx5xBXi1NLJapMVaPyIaJGg3jKdiQDaKPgwYk6daH_nut5vg" }.

Step 4. Create GET request in the Postman using configured in Integrator endpoint.

URL: https://integrator.etlworks.com/plugins/schedules/rest/v1/httplistener/patients/123.

Step 5. Specify Authorization header parameter with a Bearer prefix, and pass value from the response received in the step 3.

Example: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzUxMiJ9.eyJpc3MiOiJldGx3b3Jrcy5jb20iLCJpYXQiOjE0OTMxNjQwNjAsImV4cCI6MTQ5MzE2NDY2MCwianRpIjoiNjc2Nzk5NTgtODQ3OC00Y2MzLWFiOWMtYmJmNjYyYjQxOGNlIiwic3ViIjoiYWRtaW4ifQ.Loxfquv-qjuQRT15eRiunFCFWuWwtZTosrn86ZcWguoPbiiC00toXmFTJ4QqYYgdAHO7QFIEeqqNdgAXwED2lg.

Step 6. Click the Send button.

Push

In Integrator you can configure an API endpoint that will be receiving a payload from the third-party application which will be triggering execution of the data integration flow.

Steps to build an REST API

Read how to build event driven flows.

Testing API endpoint.

For the flow, configured to be executed on demand, you can create another flow, which will be sending triggering HTTP request right from Integrator.

Lets assume that there is a scheduled on demand flow, which is triggered by sending POST request to the https://integrator.etlworks.com/plugins/schedules/rest/v1/httplistener/patient endpoint, with a payload, which contains a json document representing patient.

For testing purposes, lets put the patient.json in the folder on the server or cloud storage.

Step 1. Create an HTTP connection with the following parameters:

URL: https://integrator.etlworks.com/plugins/schedules/rest/v1/httplistener/patient.

Authentication: token.

Authentication URL: https://integrator.etlworks.com/rest/v1/token-auth/issue.

HTTP Method for Token Authentication: POST.

Authentication Request Payload: {"login":"{user}","password":"{password}"}

Authentication Request Content Type: application/json.

User: any existing Integrator user which can run flows.

Password: password for the user.

Method: POST.

Step 2. Create flow, where the source connection is pointed to the patient.json, source format is json, destination connection is a connection created in the step 1, and the destination format is the same as the source format.

Step 3. Manually execute flow, created in the step 2, and watch it triggering execution of the on demand flow.

CRUD

In Integrator you create REST APIs for CRUD operations by creating event driven flows and selecting appropriate method, when defining HTTP listener parameters.

The following methods are supported: