Data transformation is a process of converting data from one format to another. Depending on the infrastructure, and data volume the process of transforming your data may require a team of experts and substantial infrastructure costs. Data transformation is not a single process, it includes numerous stages of activities such as convert data types, and data normalization. This cleanses data by removing nulls or duplicate data, performs aggregations, and set transformation rules. The organization transforms their data for numerous reasons. For example, to make it compatible with other data, to upgrade the system, to move to other systems, or to combine two databases. While performing the data transformation an organization can face many challenges. However, only Link can address these challenges effectively. A modernized HCL tool, the HCL Link helps organizations to solve multifaceted data integration problems. Link provides a unified solution for data transformation and integration. Overcoming challenges with Link: Simplified Complexity Modelling, development, and testing of transformations and integration solutions via a code-free approach. Any-to-any transformation, Link can truly transform data from any format to any format. Save time on the maintenance of implementations by taking advantage of accessing meta-data or using pre-built standards. Many-to-many integration, Link reads data from multiple sources and writes to many destinations in one pass. Performs data validation and data formatting which improves the data quality and protects applications from potential landmines such as null values, unexpected duplicates, incorrect indexing, and incompatible formats. Link supports EDI, XML, SWIFT, HIPAA, and other standard-based B2B integration, as well as the real-time integration of data from multiple applications, databases, messaging middleware, and communications technologies across the organization.  Interoperability Link provides the ability to Connects applications, databases, processes within the organization Link support different classes of adapters. Eliminates the need for multiple data transformation and integration tools – run...

What is your company's strategy to managing the growing demands of continuous data and support cloud-native app development at today's rate? For many leaders they seek to modernize their data platform strategy to meet these challenges.  OneDB is a feature-rich and equally able to serve as the foundation for cloud solutions, embedded applications, and IoT or edge solutions. Whether you're ready to build brand cloud-native apps, rehost or re-platform applications to take advantage of the destination platform, HCL OneDB will set you at ease with it's multi-model, cloud-native capabilities, one step at a time.  HCL OneDB is well known for its reliability, performance, and simplicity. Whether deployed on premise, public cloud, or private cloud, clients will be able to gain further advantages.  Many of the unique advantages include: Always-on Transactions - Keep data available at all times, including zero downtime for maintenance and unplanned outages.  Increased Productivity - Stable multi-model data management allows you to focus and quickly deliver the right type of data for the business solutions you need.  Detecting Patterns - HCL OneDB is optimized to find anomalies and deviations for predictive analytics in Spatio-temporal data. Ease of Use - OneDB Explore, our modern graphical administration and monitoring tool for HCL OneDB gives you the ability to monitor what is critical, and take action on what is necessary to keep your business running smoothly.  To learn about OneDB's key capabilities visit our website or download our datasheet here. 

HCL Software is proud to announce the release of HCL Link version 1.1, part of HCL Software’s data management portfolio. HCL Link (formerly known as HCL Integration Platform) is a modernized tool to help enterprises solve multifaceted data integration problems. This code-free solution provides the ability to connect to the source or target application(s) of the relevant data and then perform complex data transformation.

Nowadays, having performance issues in the database has become a kind of cliché. As new data is getting piled up every minute, it is quite normal that the server is going to overload which ultimately affects the database performance. But what if a single command would direct you to the root cause and tackling such problems becomes easy?    This blog will explain more about the following command “onstat -p”. This single command gives all the details about engine statistics, based on its output you can decide where exactly the database performance is lagging. The command displays a performance profile that includes the number of reads and writes, the number of times that a resource was requested but was not available, and other miscellaneous information.   Below is a sample screenshot of the output: Onstat -p output varies as shared memory access disk memory. For example when we fire a simple SQL query let’s say “select * from abc” and if abc pages are not available in buffers then shared memory will access the disk to fetch “abc” table pages and that’s how to output data will change based on a number of disk access attempts. The first portion of the output describes reads and writes. Reads and writes are tabulated in three categories: from disk, from buffers, and the number of pages (read or written). Let’s understand the above output line by line: The status line provides the following information: The name and version of the OneDB server product The current mode of your server -> (Prim) stands for primary, each server modes ( quiescent, single-user, fast recovery, etc. ) serves a separate purpose. The length of time since shared memory was last initialized The size of the shared memory -> Detailed information about shared memory distribution can be acquired using “onstat -g seg “. 2. Whatever transactions are happening, all of them will be stored in buffer pools. Every data of the transaction goes through buffer pools. Let's say you...

What is Global Transaction? A global transaction is a transaction that involves more than one database server. HCL OneDB database servers support two types of global transactions: TP/XA with a transaction manager and two-phase commit. HCL OneDB uses a two-phase commit protocol to ensure that distributed queries are uniformly committed or rolled back across multiple database servers. Global Transaction needs to be terminated when your secondary server is stuck in Fast recovery mode and is not coming online. So, we need to locate and terminate global transactions. Sometimes, GT can be terminated gracefully or not, based on their FLAGs. Scenario 1: Restarting Updateable secondary after a crash will get stuck in fast recovery mode until all open transactions are processed. Global transaction can be terminated gracefully. In this scenario, your Updateable secondary was crashed due to several reasons and upon starting, it gets stuck in fast recovery mode. The message below is in online.log of sds node: 12:13:04 Started processing open transactions on secondary during startup The secondary will not be operational until all the global transactions were cleared. The message above shows it is incomplete. The secondary will allow new sessions only if you see the completed string in the log. 20:10:05 Finished processing open transactions on secondary during startup. Example In the below example SDS was stuck in FR mode for almost 8 hours. We should look for the output of onstat –G from both primary and secondary. They should have different addresses in memory, but they can be identified by the "data" column. The Flag should have ‘H’ at the 3rd position, which means it was heuristically rolling back or rolled back. We can zap them using onmode –H 0x61fbe988 and onmode -H 0x61fbecf0 on SDS node. Immediately you will see your SDS will be in operational...

What is Compression By minimizing the disk space that is used by your data and indexes, it’s easy to save money Helps improving I/O Ability to store data rows in compressed format on disk Saves up to 90% of row storage space Ability to estimate possible compression ratio Fits more data onto a page Fits more data into buffer pool Reduces logical log usage How IDS Storage Optimization works! By considering the entire row and all its columns IDS looks for repeating patterns and stores those patterns as symbols in a compression dictionary By considering the entire row and all its columns IDS looks for repeating patterns and stores those patterns as symbols in a compression dictionary Creating a compression dictionary Compressing the data in table After creating the dictionary, IDS starts a background process that goes through the table or fragment and compresses the table or fragment rows. The process compresses each row and leaves it in the page where it was compressed. Any new rows that are inserted or updated are also compressed. This compress operation runs while other transactions and queries are occurring on the table. Therefore, IDS performs the operation in small transactions and holds locks on the rows being actively compressed for only a short duration. Reclaiming free space After all the rows have been repacked, the shrink operation removes the unused table or fragment space and returns free space to the dbspace that contains the table or fragment. What we are using behind the scene! Lempel-Ziv (LZ) based algorithm – static dictionary, built by random sampling Frequently repeating patterns replaced with 12-bit symbol numbers Any byte that does not match a pattern is also replaced with a 12-bit reserved symbol number Patterns can be up to 15 bytes long Max possible compression = 90%...