Event Rules are an extension of Workload Automation (WA) capabilities that enable events occurring external to the scheduling environment to trigger actions on scheduling objects within WA.  An ideal use of this capability is to detect the arrival of a file and then trigger an action to submit a Job Stream containing jobs to process the data contained in that file.  This capability has been available for a while and is widely used.  In this article, a hidden feature is explored where the name of the file and other properties related to the file are passed as variables to a Variable Table associated to the Job Stream being submitted as ac action where any Job within that Job Stream can retrieve those variables and process the data in the file. ​

Create an Event Rule with the Event, File Created, and Action, Submit Job Stream 

​Follow the steps below to create an Event Rule

  1. Login to the Dynamic Workload Console
  2. Navigate to Design à Create Event Rules
  3. Click on Create new à Event Rules
  4. Click on the field, <not_named> and the General Info panel is displayed on the right
  5. Click on the Name field, and enter the name, IF_FILE_EXISTS_PROCESS_FILE
  6. Click on Save as draft to disable it
  7. Click on Add Events à File Monitor à File created (click on + sign) à Close
  8. Click on the event just added, the Properties panel opens on the right
  9. Fill in the values in the fields as shown below:

​​​a. File name: /tmp/datafile.*.txt              An existing file with a matching name will not satisfy the condition, a new file with matching name must be created.
b. Sample interval: 300                           Indicates how often in seconds to check for the file
c. Workstation: EEL ​

10. Click on Add Actions à IBM Workload Scheduler actions à Submit job stream
11. Click on the Action just added, the properties panel opens on the right
12. Fill in the values in the fields as shown below:
a. Job stream name:         PROCESS_FILEThe name of the Job Stream to be submitted that contains jobs to process the
​                                                contents of the file.
b. Workstation name:       EEL
 c. ​ Custom parameter 1:
Click on the Variables icon on the right and choose fileCrtEvt1 à File name
FILE_NAME=%{fileCrtEvt1.FileName} will be entered
The value of File name will be passed to the Variable, FILE_NAME
d. Custom parameter 2:      Similarly, click on Variables icon on the right and choose any other variable that needs to be
passed as Variables, e.g.
e. Custom parameter 3:     WORKSTATION=%{fileCrtEvt1.Workstation}​
f. Custom parameter 4:      TIME_STAMP=%{fileCrtEvt1.TimeStamp}​​​
g. Custom parameter 5:     HOSTNAME=%{fileCrtEvt1.Hostname}​
h. Custom parameter 6:     IP_ADDRESS=%{fileCrtEvt1.IPAddress}
i. Custom parameter 7:      EVENT_RULE_ID=%{fileCrtEvt1.EventRuleId}
13. Click on the selection box next to the event Rule name and click on Save on the menu bar.
14. Click on the X to close the successful saved message.
15. The following is the Event Rule definition is composer format.
<?xml version=”1.0″?>
<eventRuleSet   xmlns:xsi=”https://www.w3.org/2001/XMLSchema-instance”
xmlns=”https://www.ibm.com/xmlns/prod/tws/1.0/event-management/rules”
xsi:schemaLocation=”https://www.ibm.com/xmlns/prod/tws/1.0/event-management/rules https://www.ibm.com/xmlns/prod/tws/1.0/event-management/rules/EventRules.xsd”>
<eventRule name=”IF_FILE_EXISTS_PROCESS_FILE” ruleType=”filter” isDraft=”no”>
<eventCondition name=”fileCrtEvt1″ eventProvider=”FileMonitor” eventType=”FileCreated”>
<scope>
/TMP/DATAFILE.*.TXT ON EEL
</scope>
<filteringPredicate>
<attributeFilter name=”FileName” operator=”eq”>
<value>/tmp/datafile.*.txt</value>
</attributeFilter>
<attributeFilter name=”SampleInterval” operator=”eq”>
<value>300</value>
</attributeFilter>
<attributeFilter name=”Workstation” operator=”eq”>
<value>EEL</value>
</attributeFilter>
</filteringPredicate>
</eventCondition>
<action actionProvider=”TWSAction” actionType=”sbs” responseType=”onDetection”>
<scope>
SBS EEL#PROCESS_FILE
</scope>
<parameter name=”JobStreamWorkstationName”>
<value>EEL</value>
</parameter>
<parameter name=”Parm_1″>
<value>FILE_NAME=%{fileCrtEvt1.FileName}</value>
</parameter>
<parameter name=”Parm_2″>
<value>SAMPLE_INTERVAL=%{fileCrtEvt1.SampleInterval}</value>
</parameter>
<parameter name=”Parm_7″>
<value>EVENT_RULE_ID=%{fileCrtEvt1.EventRuleId}</value>
</parameter>
<parameter name=”Parm_5″>
<value>HOSTNAME=%{fileCrtEvt1.Hostname}</value>
</parameter>
<parameter name=”Parm_6″>
<value>IP_ADDRESS=%{fileCrtEvt1.IPAddress}</value>
</parameter>
<parameter name=”Parm_3″>
<value>WORKSTATION=%{fileCrtEvt1.Workstation}</value>
</parameter>
<parameter name=”JobStreamName”>
<value>PROCESS_FILE</value>
</parameter>
<parameter name=”Parm_4″>
<value>TIME_STAMP=%{fileCrtEvt1.TimeStamp}</value>
</parameter>
</action>
</eventRule>
</eventRuleSet>

16. Select the Job Stream Name and choose Select an Action à Submit Job Stream into Current Plan

​Create a Variable Table, Job, and Job Stream with to process the file

Follow the steps below to create the objects

  1. Login to the Dynamic Workload Console
  2. Navigate to Design à Manage Workload Definitions
  3. Click on Create New à Variable Table
  4. In the General tab, enter in the Name field, PASS_VAR
  5. The variable to hold the properties of the file monitored are created and referenced internally, as such, no Variables need to be created.
  6. Click on the Save icon to save the Variable table.
  7. The following is the Variable Table definition is composer format.

 

VARTABLE PASS_VAR
MEMBERS
END

8. Click on Create New à Job Definition à UNIX
9. In General tab, enter in the Name filed, FILE_RECEIVED_N_PROCESSING
10. In the Task tab, Select Script and enter in the Script name field:

/bin/echo -e “\nThe properties of the file received are: \n\nFile Name=^FILE_NAME^ \nSample Interval=^SAMPLE_INTERVAL^ \nWorkstation=^WORKSTATION^ \nTime stamp=^TIME_STAMP^ \nHostname=^HOSTNAME^ \nIP Address=^IP_ADDRESS^ \nEvent Rule ID=^EVENT_RULE_ID^ \n\nThe next job in the Job Stream can now start processing the file.\n”
11. Notice that the same variable names are used to retrieve the values as they are assigned to the respective properties of the file monitored in the Event Rule, IF_FILE_EXISTS_PROCESS_FILE.

​12. Click on the save icon to save the Job
13. The following is the Variable Table definition is composer format.

EEL#FILE_RECEIVED_N_PROCESSING
SCRIPTNAME “/bin/echo -e \”\nThe properties of the file received are: \n\nFile Name=^FILE_NAME^ \nSample Interval=^SAMPLE_INTERVAL^ \nWorkstation=^WORKSTATION^ \nTime stamp=^TIME_STAMP^ \nHostname=^HOSTNAME^ \nIP Address=^IP_ADDRESS^ \nEvent Rule ID=^EVENT_RULE_ID^ \n\nThe next job in the Job Stream can now start processing the file.\n\””
STREAMLOGON iwadmin
DESCRIPTION “A Job to demo how to retrieve variables passed from an Event Rule”
TASKTYPE UNIX
RECOVERY STOP
14. Click on Create New à Job Stream
15. In General tab, enter in the Name filed, PROCESS_ FILE
16. Scroll down to the Variable table field and enter PASS_VAR to associate the variable table to the Job Stream.
17. In the Details view, right click on the Jobs and choose Add Jobs
18. Enter in the Job definition field, FILE_RECEIVED_N_PROCESSING, and click on Add
19. Click on the save icon to save the Job Stream
20. The following is the Variable Table definition is composer format.
SCHEDULE EEL#PROCESS_FILE
DESCRIPTION “A Job Stream to demo how to retrieve a variable passed from an Event Rule”
VARTABLE PASS_VAR
:
EEL#FILE_RECEIVED_N_PROCESSING
END

Monitor the execution of the Event Rule

​Follow the steps below to create the file, monitor the execution of the Event Rule, submission of the Job Stream, and the Job log to view the variable passed from the Event Rule to the Job Stream.

1. Login to server where the file is monitored and create a file to simulate the arrival of the file cd /tmp
touch datafile.100120201202.txt
2. On the MDM, monitor the contents of the messages.log cd /opt/ibm/iwa/iws/TWSDATA/stdlist/appserver/engineServer/logs
tail -f messages.log
3. Once messages are displayed about the Event Rule being executed, login to the Dynamic Workload Console and navigate to monitoring the Job stream.
4. The Job Stream, PROCESS_FILE, is submitted.
5. Go into Jobs view and monitor the status of the Job, FILE_RECEIVED_N_PROCESSING
​6. Once it is successful, view the Job Log.

7. Notice that the Variables are successfully passed from the Event Rule to the Job Stream and the Job is able to retrieve them from the Variable Table.
8. Also notice that the Variable Table has not been populated with the variables but were mapped to it internally.
VARTABLE PASS_VAR
MEMBERS
END
The same process can be used to create an Event Rule with Modification completed file monitor event.

Want to learn more? Write to me at Sajjad.Kabir@hcl.com or schedule a demo.
Comment wrap
Further Reading
article-img
Automation | June 20, 2022
The Dynamic Workload Console is the one-stop automation platform for users across the business
The Dynamic Workload Console (DWC) has become a core platform for workload automation, providing visibility into everything all in one place.“The designing of a job stream is a key operation for schedulers and application developers to or interconnect business applications and achieve governance and control,” Zaccone said. “Our idea with the new Workload Designer is to empower what we had and push it to an advanced level to provide everything is needed to our customers.” 
article-img
Automation | May 24, 2022
Ensuring Passwordless Job Schedulation with CyberArk Integration
CyberArk is an identity and access manager offering a wide set of identity security capabilities. You can use it to submit Workload Automation jobs without specifying any passwords, which are retrieved from the CyberArk vault.
article-img
Automation | May 19, 2022
Continuous Security and Quality Updates on Dynamic Workload Console 10.1
After the biggest launch of Workload Automation 10.0.1 release in 1Q of 2022 (see the Workload automation original Video), what can we expect in 2022? Big news! Our first release refreshing for Dynamic Workload Console 10.0.1 is ready. Let’s answer the 5 WH questions.
Close
Filters result by
Sort:
|