One of the difficult tasks in performing an import of external data into a HCL Compass database is determining how to define the data type of a field within a Compass record.  If you define it incorrectly, the import of data will fail because of a type mismatch and you will need to modify your Compass schema to correct the issue.

The Compass Import Assistant is a simple Perl application that will parse an entire CSV file to be imported into Compass and will produce a recommendation for how to correctly define the Compass data types. 
 
For example, consider the following 10,000+ lines of data, each representing a record to be imported into Compass: 
 
“dbid”,”Headline”,”id”,”old_id”,”Owner”,”Priority”,”ratl_mastership”,”record_type”,”Resolution”,”Severity”,”State”,”Submit_Date”,”Submitter”,”version” 
“33554446”,”clerk allowed to charge too much on credit card”,”DB100000014″,”CLSIC00000014″,”engineer”,”3-Normal Queue”,”<local>”,”Defect”,”Fixed”,”4-Minor”,”Resolved”,”7/8/2000 0:00″,”lead”,”1″ 
“33554447”,”end-of-shift report fails if after midnight”,”DB100000015″,”CLSIC00000015″,”engineer”,”3-Normal Queue”,”<local>”,”Defect”,”Fixed”,”3-Average”,”Resolved”,”7/8/2000 0:00″,”lead”,”1″ 
… 
10,000 more records 
… 
“33554449”,”delete operation leaves blank line in form”,”DB100000017″,”CLSIC00000017″,”engineer”,”3-Normal Queue”,”<local>”,”Defect”,”Fixed”,”3-Average”,”Resolved”,”7/8/2000 0:00″,”lead”,”1″ 
“33554450”,”add item button not active after adding 3 other items”,”DB100000018″,”CLSIC00000018″,”lead”,”1-Resolve Immediately”,”<local>”,”Defect”,”Fixed”,”1-Critical”,”Resolved”,”7/8/2000 0:00″,”lead”,”1″ 
“33554451”,”sales tax incorrect for NH”,”DB100000019″,”CLSIC00000019″,””,””,”<local>”,”Defect”,””,”1-Critical”,”Submitted”,”7/8/2000 0:00″,”lead”,”1″ 

It would be difficult to visually parse this data to determine the data type for each of the 14 fields to be imported into Compass.  However, the Compass Import Assistant will parse then entire 10,000+ records and provide recommendations on how to define your Compass data types. 
 
Below is an example of running the Compass Import Assistant against our sample data: 

In this example, we see that the Compass Import Assistant has parsed the data, performed some analysis on it and has provided recommendations for how to define the right Compass data type for each field to be imported. 
 
By running this simple Perl application on your data prior to importing into Compass, you can save a great deal of time and effort in defining your Compass schema and importing your data into Compass. 
 
The source code for this application is provided below so that you can use it as-is or modify it to meet your specific needs. 

Source Code 

#!perl.exe 

# Compass Import Assistant 

# One of the most difficult things about bringing a Compass eval/install 

# up is defining the data types in the schema that will be used to hold 

# the data from a pre-existing CR Tracking tool.  It is trial-and-error  

# with mostly error. 

# This is a Perl script that will parse an *entire* csv file 

# to be used for the CQ import and will produce a recommendation for how to 

# define the data types. 

# Eventually, I may get around to putting a TK interface on it… 

# Note that it uses the “Date::Manip” Perl package, so you will need that in 

# your Perl installation. 

#   Author: Tom Milligan 

use Date::Manip; 

sub usage 

   die “Usage: $0 [-o outputfile] input.csv\n”; 

# Rudimentary check for correct number of args 

if ($#ARGV > 2) { 

 

Comment wrap
Further Reading
article-img
Secure DevOps | April 29, 2021
Getting Compass Single Sign-On Working with Okta
We have a simplified way to set up your Okta service and we're sharing tips to help! But what does this mean for Compass?
article-img
Secure DevOps | March 26, 2021
How to Enable Search in HCL Compass
As an end user, searching for information in a web application can be complicated. In Compass product, it would be great if we could search all attributes of records at once. This can be made possible through search configurations. 1. Why Compass Search? Searching various record types using ID is difficult. Search functionality allows the user to search for various Record types based on “Search Scope” for example Customer, Product, Iteration. Project etc. Suppose if user is interested to fetch all defects matching a project name, it will be possible to obtain all records matching the project name. User can also apply filters by combining multiple criteria like different Record Types and different attributes. 2. Advantages of Search functionality Text search is divided into two tasks: indexing and searching. The indexing stage will scan the text of all the records and build a list of search terms often called an index. In the search stage, only the index is referenced, rather than the text of the original records. Full-text search also supports advanced search options with operators using “AND” “OR” “NOT”. It also supports wild cards such as?*. 3. How to configure Full-text search in Compass web in old UI? Configuring and enabling the full-text search feature for HCL Compass Web consists of the following tasks: Configuration Steps: 1. Run the cpsearch.pl command, with the initSearch and -solrHome options, in a command prompt window: cqperl cpsearch.pl initSearch -username admin -password "" -dbset TextSearch -userdb SAMPL -searchHome C:\CPSearch\CPSearch.Home -solrHome C:\CPSearch\CPSearch.Solr -solrPort 8984 When this command completes, configuration data corresponding to your schema is placed in the following directory: C:\CPSearch\CPSearch.Home\TextSearch_SAMPL\ Below is the output user should get if there are no errors: 2. Edit the entity file associated with the deployment: C:\CQ.Search\TextSearch_SAMPL\Entity-TextSearch-SAMPL.txt. For each record type, select a display field to appear...
article-img
Secure DevOps | February 5, 2021
Protecting HCL Compass from Brute-force attacks
HCL Compass allows the administrator to protect Compass database repositories from brute-force attacks, here is the official documentation.  It limits the number of invalid login attempts of an attacker, who pretends as a user, before the lock out and also limits the login attempts from a certain host connecting to HCL Compass.  If an attacker is trying to hack and determine the password of a particular user, they might try many login attempts with different username and password combinations until they find one that works. As a defence against such attacks, HCL Compass can lock out the account or host used by the attacker when a certain number of failed attempts have occurred. When an account or a host is locked out, the login error message is the same as if the attacker entered an incorrect username and password. This prevents the attacker from knowing that the account has been locked out, causing them to waste resources on a continued fruitless attack. This severely limits the attacker to succeed in guessing the password.  Steps to protect the credential  HCL Compass enables the credential protection in just two simple steps.  First step to enable security is by creating a “.config” file with the below data. In this example we will create the config file with the name “SecureCredential.config”. This file can be created in any location; hence we have created the file in the HCL Compass default installation location, which is, “C:\Program Files\HCL\CCM\Compass”  # Comments are preceded by '#' and are ignored. # This enables lockouts lockout_enable 1 # Host lockout configuration # This sets the HOST lockout threshold to 10 tries lockout_threshold HOST 10  # This sets the HOST reset period to 60 seconds. lockout_reset HOST 60 # User lockout configuration # This sets the USER lockout threshold to 10 tries lockout_threshold USER 10  # This sets the USER reset period to 60 seconds. lockout_reset USER 60 # Whitelist / blacklists lockout_whitelist HOST whitelisthost lockout_whitelist USER whitelistuser1, whitelistuser lockout_blacklist HOST blacklisthost3 lockout_blacklist USER bl_user1 lockout_blacklist USER bl_user2,bl_user3 # Login attempt table cleanup # This says to cleanup any failed login attempts older # than one day (60s/m * 60m/hr * 24hr/day), and only # in...
Close