Abstract –

Today, data plays an important role for any organization, every organization processes a large volume of data which can be in different formats. They may have many applications not necessarily supporting the same data format to process the data for their day-to-day activity. To solve this problem, they would need a translator/connector or a middleware to exchange the data within internal/external applications/systems and here we have a powerful ETL & integration tool HCL Link. HCL link is also used by other HCL software offerings such as UNICA, Commerce, Volt etc. to integrate with each other or the external systems.

The key points that can encourage the user to use HCL Link –

  • A GUI tool – very simple to use
  • HCL LINK Embedded Edition
  • REST based Custom connectors
  • Rich set of adapter’s support
  • Service Builder support for REST-based endpoints

 

  •  A GUI tool – very simple to use –

HCL Link – A robust platform that accelerates integration of the many applications and diverse data across organizations on-prem, cloud or hybrid environments​. It includes the Schema Designer, Map Designer, Flow Designer, Core, Runtime and Adapters. It’s a powerful web based ETL & integration tool and user friendly. It facilitates the simple way of uploading the files, creating connections/actions by using the rich set of adapters/connectors, creating maps for data transformation/transfer for the actions and orchestrate the maps and other nodes in the flow to execute together to get the desired goal.

The home UI page looks as below. It provides you the two banners i.e., Design and Deploy. Design can show the list of projects, create, and import projects and deploy banner can show the options to create the servers, packages, server groups, configuration variables and deploy packages.

Design Banner

Graphical user interface, text, application, email Description automatically generated

  • Under Design, click + icon to create a new project, enter the project name and click Ok. It will then show below screen

Graphical user interface, application Description automatically generated

Quick Links – It provides you the links to quickly access Files, Connections/Actions, Schemas, Maps and Flows. Also alternatively, you have the separate tabs at the bottom of the page to go through these options.

Link design options for creating/modifying the project–

Files – Use this option to upload the input files so that you can use in in the map/flow. You can also download the files on your local, specially required if you want to see the output on your local system.

Connections – A connection identifies what system / technology is being connected to – e.g., File, JDBC, FTP, REST etc.​

Actions – An action identifies what specifically is being done using that connection – e.g. Inserting rows in database table, update the Contact objects in any third-party systems/applications, invoke a specific REST API, transfer a file to the third-party system.​

Schema – A schema describes the structure of the data as returned by or sent to a connector/adapter​. You can use the schema importer to create a schema from CSV, Copybook, XSD and JSON template etc.

Map – A map provides the logic that produces the desired output results​. For example, if you would need to map the XML data to CSV, a map can be used to do this.

Flow – To provide a way to orchestrate multiple nodes together to achieve the goal. It can include one or more nodes.

Graphical user interface, application Description automatically generated

Deploy Banner

Graphical user interface, text, application, email Description automatically generated

Link Deploy options –

Server Groups – We can create the server groups such as Prod, Dev and Test and the server can be associated with the server group.

Servers – We can create a server that contains the connection string such as the platform details Windows or Linux, user credentials, and server URL or IP/FQDN and the server group.

Configuration Variables – A configuration variable defines an alias that resolves at run time to a value that is specific to a deployment environment (Server Group). By associating the values of a configuration variable with different server groups, you can run the same map or flow in different environments, servers, and platforms without changing the map or flow.

Packages – We can create a package by selecting the project to package. We can then add the maps, flows and files in that package.

Deploy – You can select the package and the server’s name to deploy the package on the selected server.

Build – You can build the package before you deploy it on the server to see if there is any compile time error.

HCL Link Components (Once link installed, you can check for below components) –

HIP Client – It is responsible to render the UI page in browser

HIP Server (Design-Time) – It facilitates the users to create schema, map, and flow.

HIP Rest (Runtime) – It facilitates to run the map & flow to meet the requirement.

Mongo – It’s used to store the artifacts that you create by using design time (HIP-Server)

Redis – HIP-Rest/runtime uses REDIS to execute the request

  • HCL LINK Embedded Edition

This is what we provide to other HCL Software teams to embed within their applications (e.g., Unica, Volt etc.), which includes HCL Link and Embeddability Extensions ​

Especially, UNICA applications i.e., Journey & Campaign combine HCL Link EE with their appropriate application-specific connectors (e.g., for Unica Link – Email connectors, AdTech, CRM, SMS, and the database connector) as part of their product packaging

HCL Link Embeddability Extensions (EmbedX) – An extension built on top of HCL Link to enable embedded usage by other HCL Software offerings, such as Unica, Volt, and Commerce etc. It enables partners and customers to easily create and use their own connectors​

 

 

 

 

 

 

The connectors consumed by Unica-Link Integration is depicted below–

Different hops involved to run Unica-Link request on HIP-REST(Runtime): –

Diagram Description automatically generated

  • REST based Custom connectors

HCL Link can be easily used by the customer to create the connectors as per their need. This is what the custom connector is. For example, Unica customers can leverage it to create their own connector. For example – if a Unica customer if gets a requirement to connect to the new third party system for data processing, let’s say the new marketing application, they can then leverage link to create such connector. They would need to have the requirements for that custom connector and then to verify that appropriate REST APIs are provided by the third-party application to be able to meet those requirements.

Once they are ready with the requirement, they can create a connector project, define the REST endpoints and the connector descriptor file to define the connection & action properties.

Artifacts required to create a custom connector package which can be installed on the server–

  1. Create a connect project file which contains maps/flows
  2. A service definition file/configuration package where you define the REST endpoints
  3. A connector descriptor file which defines the connection & action properties

Use packager command to create the connector package by packaging above artifacts. It can then be deployed on link hip-rest component

To package the connector into an output directory called ‘\output\test’, invoke the packager tool with -p (package) and -ep (export package) options:

C:\packager> packager \connectors\test \output\test -ep Test – p -h https://localhost:8443/ -u admin -pw ****

Above steps will create a connector package which can be deployed on the server and then installed so that the customer can start integrating Unica applications with the new connector for marketing through the new channel.

  • Rich set of adapter’s support

HCL link supports a rich set of adapters to connect to different applications/systems. It makes it robust which can meet the varying customer requirements when they intend to integrate with many applications using the different communication protocols.

  • Service Builder support for REST-based endpoints –

HCL Link includes Service Builder to create service definition file for the REST endpoints that will be used to connect to the other systems or applications. This is important for creating the UNICA-Link REST based connectors.

  • Go to Link home page and click the service builder link to create a server definition and endpoints

Graphical user interface, text, application, Word Description automatically generated

  • Create a new service by clicking on the ‘Create Service’.

A picture containing background pattern Description automatically generated

  • Provide the details such as service name, authentication type and parameters –

Graphical user interface, application Description automatically generated

  • You can go to the last stepper to save the service
  • In the service, click Create Endpoint

Graphical user interface, application Description automatically generated with medium confidence

  • Enter the endpoint name, method type and other parameters required for the request/response payload and save.

Graphical user interface, application Description automatically generated

  • You can click Call API button under request tab to check if endpoint Is working ok.

These endpoints can be used in the map and flow.

Link basic concepts explaining through an example for the new user

The scenario is to convert the CSV input file to JSON. The CSV input file structure, let’s assume is as below –

Employee Name,Designation

  • Go to the link UI and select + icon under Design banner to create a new project. You can then see many tabs at the bottom of the page which are explained below-

Files tab –

So, the step 1 is to upload the input file –

  • Click the Files tab then + icon to upload a new file –

Graphical user interface, text, application Description automatically generated

Connections tab –

  • Click the Connections tab

  • Click + icon to create a new connection. It will show you the list of adapters available to connect to the external or internal systems. Choosing File adapter as my source/target is file. Click next to provide the connection name and save it.

Graphical user interface, text, application Description automatically generated

Schemas tab-

Schema helps to use the importer to easily create the data structure from the template to store the file data.

We would create CSV followed by JSON.

  • Click on Schemas and then Import to Import from the file and select CSV from drop down box

Graphical user interface, application Description automatically generated

Application Description automatically generated with medium confidence

  • Select the input file name to create the structure and click Next to enter the schema name and select import.

Table Description automatically generated

Graphical user interface, application Description automatically generated

Now, import JSON schema –

Graphical user interface, application Description automatically generated

  • Click XSD/JSON and browse for the JSON file and click Ok.

Graphical user interface, text, application, email Description automatically generated

Connections tab for creating actions –

Creating a source action –

  • Hover the mouse on the connection name and click 3 dots to get below

Graphical user interface, text, application Description automatically generated

  • Click New Action option to create a new action

Chart Description automatically generated

  • Select Source and click Next and browse for the input file Graphical user interface, application Description automatically generated
  • Select the schema and schema type to match the input file, provide the name and save

Graphical user interface, text, application Description automatically generated

Now creating the target action –

  • Hover the mouse again on the connection name and click 3 dots

Graphical user interface, text, application Description automatically generated

  • Click New Action to create a target action and click Next

A picture containing graphical user interface Description automatically generated

  • Browse for the JSON template file and click Next. Enter the target action name and save.

Now, you can see two actions created i.e. the source action to read from the CSV file and the target action to write into JSON format. Graphical user interface, application Description automatically generated

Maps tab –

  • Click on Maps tab and then + icon to create a new map. Enter the map name and click Ok-

Graphical user interface, text, application Description automatically generated

Graphical user interface Description automatically generated with medium confidence

  • Now add the actions to the map in form of the input and output cards –

Graphical user interface, text, application Description automatically generated

  • After adding the actions – the map will look like below

Graphical user interface, text, application Description automatically generated

  • Create a functional map to map CSV to JSON

Graphical user interface, application Description automatically generated

Graphical user interface, text, application Description automatically generated

  • Click build icon to build the map. Once build is successful, click the run button to run the map

Graphical user interface, text, application Description automatically generated

  • The input and the output result data will be as follows

Graphical user interface, text, application Description automatically generated

Flows tab –

  • Click on the flows tab and then + icon to create a new flow. Enter the name and click Ok

Graphical user interface, text, application Description automatically generated

  • Creating a flow having one decision node and a map node. The decision node will execute true terminal if Flag_to_Run_the_Map flow valuable is set to True else the false terminal.
  • If Flag_to_Run_the_Map is True will execute the map node which will further result in JSON data. If it’s false, then map node won’t get executed.

Graphical user interface, text, application Description automatically generated

  • Executing the flow by setting Flag_to_Run_the_Map to True

Graphical user interface, text Description automatically generated

  • Executing the flow by setting Flag_to_Run_the_Map to False, would notice that the green tick mark doesn’t appear on the map node.

Graphical user interface, text Description automatically generated

 

 

Comment wrap
Further Reading
article-img
Cloud Native | July 20, 2021
Experience a Cloud Native integration with HCL Link
The world of integration is constantly changing. As we deal with such complexity, we tend to drive toward simpler integrations (REST, API’s & JSON). Data structures are changing more rapidly and data sources are being added at the highest frequencies. The evolution of data is leading to significant digital transformation in organizations across the globe, but along this path there is a pain point that we are seeing in the market.  We need connectivity. HCL Link makes connection between applications and the data needed to run and transform your business. But, can you imagine a software where demos, trials, and deployment to development and production environments were all through one containerized experience? It’s called HCL SoFy. Now, reimagine HCL Link on SoFy. This provides an easy path to get started with cloud native. It delivers richly capable, multi-cloud ready HCL Software products and microservices. This is designed to allow solutions to run in any Kubernetes environment. Thanks to HCL Now you can experience using HCL Link on SoFy. HCL Now is Cloud-Native-as-a-service. This approach enables a cloud-first architecture that allows freedom to deploy anywhere, power to innovate, and allows flexibility to scale on a dedicated environment. With experts on demand, you can rest easy and leave the burden to us. Reduce risk and cost as you move to cloud native with our Kubernetes and product experts.    If you’re searching for a simpler integration path, look no further. Experience a cloud native integration with HCL Link.
article-img
Data Management | February 4, 2021
What is the Supply Chain EDI Pack?
The HCL Link Supply Chain EDI (Electronic Data Interchange) Pack gives pre-assembled structures that convey different forms of the standards for use with HCL Link. The time and exertion to manually construct a portion of these structures would require an extended period of time due to the intricate nature of the guidelines.  
Close
Filters result by
Sort:
|