raincode rect

Raincode JCL User Guide


Version 4.2.325.0

1. Introduction to Raincode JCL

The Raincode JCL is an emulator of a part of the Job Entry Subsystem (JES) in z/OS. Its focus is to interpret JCLs (and hence lacks some of the features of JES such as, e.g. scheduling). It consists of a collection of executables and libraries, resulting in an emulation of a Mainframe Batch environment.

A key advantage of Raincode JCL is its ability to be programmatically customized, extended and integrated. Because of this flexibility through code, the intended audience of this document is twofold: on the one hand is operations staff and on the other hand is the development team.

The structure of this document reflects the twofold nature of the audience. After this introductory chapter, which is both for operations and development staff, the document is split in two parts. The first part is primarily targeted to operations, as it describes how to work with the different parts of Raincode JCL. The second part is exclusively targeted to the development team, as it explains how to customize and extend Raincode JCL through code.

The expectation is that readers of the second part will be familiar with the core elements laid out in the first part, as it is these functionalities that they will be extending and customizing through code.
image004
Figure 1. Overall Architecture Diagram

In the figure Overall Architecture Diagram, we show the overall architecture diagram. It illustrates that Submit is the point of entry to the Raincode JCL. Submit relies on the Catalog Manager, Parser and Batch Runtime to perform its functions, and which turn in multiple smaller modules. This section will present Submit before discussing the Catalog Manager, the Parser and the Batch Runtime.

The default installation directory for Raincode JCL on Windows, depending on the version, is C:\ProgramFiles\Raincode\JCL\net48 or C:\ProgramFiles\Raincode\JCL\net6.0. The environment variable %RCBATCHDIR% points to the directories holding the different executables, e.g. submit.exe. The location of the catalog, utilities, and logs are configurable (see section Catalog Configuration) for more details.

1.1. Submit

Submit is the equivalent of the Mainframe’s SUBMIT command, allowing users to submit a JCL for execution. This can be done using the command-line submit.exe executable or programmatically by calling the SubmitRunner.Execute() method. Hence, the users of Submit may be operational staff as well as developers.

All coordination with the filesystem, launched processes, interaction with the catalog, and processing of JCL is encapsulated in the call to Submit. This design allows for horizontal scaling by running jobs from different servers without a central coordinating server. The return code of Submit.exe is the highest return code for the job for easy integration with job schedulers. Editing the .NET configuration file for Submit.exe allows for custom configuration of various interfaces provided for extensibility.

For command-line options of Submit.exe, refer to Command-line options for Submit.exe. The most straightforward way to execute is to call Submit.exe by providing the JCL with the -File option, for example, as shown below:

simple

After execution, logs get created in C:\ProgramData\Raincode\Batch\SYSOUT. The results of an example job are shown in figure Logs SYSOUT.

image010
Figure 2. Logs SYSOUT

1.2. Catalog Manager

The Catalog Manager emulates the Catalog Manager of the file system on the Mainframe, i.e. it implements a view on the emulated file system that is part of Raincode JCL. The Catalog Manager maintains its information in an XML file, whose default path is C:\ProgramData\Raincode\Batch\Raincode.Catalog.xml(shown in figure Raincode JCL Catalog Location).

image008
Figure 3. Raincode JCL Catalog Location

The default volume location for data sets managed by the Catalog Manager is C:\ProgramData\Raincode\Batch\DefaultVolume, illustrated in figure Default Volume.

image009
Figure 4. Default Volume

It is an internal component of Raincode JCL that developers may access programmatically. Its architecture is shown in figure Catalog Manager.

Internally, the Catalog Manager relies on three subsystems: Data File Management, Lock Management and File System Support. Further, this document provides a more detailed explanation of these subsystems.

image003 1
Figure 5. Catalog Manager
  • Data File Management handles everything related to files (datasets) between the program and JCL per step. Data files are managed by dataset objects, which describe the record layout and other information required for running jobs.

  • The Lock Management is responsible for obtaining the necessary locks on datasets used by JCLs. This lock depends on the SHR option from the DD statement. Locks are made using the Win32 API function LockFileEx to lock a portion of the metadata file. This mechanism supports locking across network file shares and coordinates access between simultaneously running jobs.

  • The File System Support is responsible for doing the physical I/O on the reading, updating and writing operations files.

For more information on the Raincode’s approach to handling datasets, refer to Representation of the datasets.

1.3. Parser

The JCL parser is an internal component of Raincode JCL, as shown in figure Parser, it includes support for AutoEdit preprocessing and JCL preprocessing.

Parser
Figure 6. Parser

The AutoEdit preprocessing support includes date functionality, where calendar files can be specified in the Control-M format or programatically through the IAutoEditCalendar interface. Variables can be set externally using an environment variable CTM_xxxx, where xxxx is the AutoEdit variable name.

Considering JCL preprocessing, %%SET statements and &VARIABLE references are supported as on the Mainframe.

The parsers are built using the PEG parser generator. The generated parsers are in C# and use regular expression matching. After the input cards have been processed with the AutoEdit preprocessor, the JCL is parsed into a separate object model. This representation is tailored for parsing and is separated from the Batch Runtime’s job object model.

The parser may also be accessed programmatically by developers. This is as the input JCL stream can be modified at various processing points by using the IJclPreprocessor interface.

1.4. Batch Runtime

The Batch Runtime is responsible for the different steps in the execution of a JCL. The basic flow of execution involves Scan, Execute, and Cleanup, as shown in figure Batch Runtime.

image007
Figure 7. Batch Runtime

The process flows as follows:

  1. Scan is called once to scan the whole JCL and begin execution.

  2. Execute JOB is responsible for executing the full JCL, step by step. It checks all run condition by step execution with the related steps condition loop for all the steps.

    1. Execute STEP is responsible for executing a step in a given JCL. It will use the Legacy Runner rclrun.exe (see The Legacy Runner) to run the executable for that step.

    2. Cleanup STEP is executed at the end of the step execution, e.g. releasing datasets that are no longer used in the following steps.

  3. Cleanup JOB is executed at the end of each JCL.

When the runtime looks for a program to execute, it looks in the paths defined by:

  1. STEPLIB, which may be defined in the JCL (STEPLIB can be for all the steps or one step only)

  2. JOBLIB, which may be defined in the JCL (JOBLIB are generally for the whole JCL and are defined before the first step)

  3. The default location, defined in the catalog definition.

  4. The current directory where submit is called.

Lookup of executables occurs in the list order above.

Programmatically, the batch runtime consists of an object model representing a JCL job in-memory and classes that facilitate the execution of steps. The parser builds up the in-memory job representation, and the Submit runner calls the Scan and Execute methods on the job to direct the process.

The entry point into the Scan/Execution facility is Raincode.Batch.Submit.SubmitRunner, which is contained in the Submit executable. The static Execute method can be called with options that specify an actual file to parse into a Job object and execute or a pre-built Job object.

1.5. The Legacy Runner

The legacy runner rclrun.exe is the part of Raincode JCL that is ultimately responsible for running a program. Raincode COBOL, PL/I and HLASM applications are compiled to .dll files. Submit and IKJEFT01 use the Legacy Runner to run these DLLs.

The Legacy Runner will search for the RCDIR environment variable and execute from the given path. If the RCDIR environment variable does not exist, it will search the current executing directory.

The default legacy runner is %RCBIN%\rclrun.exe, but this may be changed to a user-defined runner by defining any of the below values:

  1. Content of Environment Variable RC_LegacyRunner

  2. Content of Environment Variable LegacyRunner

  3. Content of Catalog configuration XML attribute LegacyRunners

Lookup occurs in the list order above.

When SUBMIT starts a program, the environment variable RC_LegacyRunner is set to the path of the runner that is actually used.

Part I: Operations

2. Mainframe file system emulation

Mainframe Datasets are stored in Windows or Linux files. These files are managed by the catalog system, emulating the Mainframe file name structure using the DSN and syntax. In the Raincode Catalog, we have a directory for each qualifier part of the DSN (periods separate qualifiers in a DSN). Because the underlying file system cannot natively treat mainframe dataset attributes such as, e.g. record format and record length, datasets are stored as two files: one with a .meta and one with a .seq extension. The metadata of the dataset will be stored in the file .meta, and the data itself will be stored in .seq.

For example, consider the DSN XXX1.XXX2.FILE1. Inside the directory XXX1, we have a directory XXX2, and the rightmost part of the DSN (FILE1) yields the files FILE1.seq and FILE1.meta in that directory.

In addition to directly supporting plain datasets, Raincode also provides native support for Partitioned datasets (PDS) and Generation Data Group (GDG). PDS and GDG are stored as directories, with a special extension (._dir) to distinguish them from other folders.

2.1. File system emulation by example

We present a didactic example to show how the Raincode Catalog emulates the Mainframe file system. Consider the following structure of datasets on the Mainframe (also shown in Figure Datasets on Mainframe):

  • RAINCODE.PROJECT1: High-level qualifier.

  • PDSTEST: Partitioned Dataset.

  • GDMODEL1: Plain Dataset.

  • GDGROUP1: Generation Data Group Dataset.

  • GDGROUP1.G0001V00 and GDGROUP1.G0002V00 are the two files (versions) of the GDG Dataset.

    • G0001V00 is version 1.

    • G0002V00 is version 2. (Current Version)

datasets on mainframe
Figure 8. Datasets on Mainframe

On Windows, this yields the following structure (also shown in Figure Datasets on windows):

  • Two qualifiers (Folders):

    • RAINCODE

    • PROJECT1

  • Two directories for PDS and GDG (._dir):

    • GDGROUP1

    • PDSTEST

  • File Dataset:

    • GDMODEL1

datasets on windows
Figure 9. Datasets on windows

2.1.1. Partitioned Dataset

A Partitioned Dataset, also known as PDS, contains one or more members. These members are separate from each other. Considering our example, this is shown in Figure PDS on Mainframe Layout.

pds on mainframe
Figure 10. PDS on Mainframe Layout

In Windows, this gives the following structure: PDSTEST._dir that contains MEM1 and MEM2, shown in Figure PDS on Windows Layout.

pds on windows
Figure 11. PDS on Windows Layout

2.1.2. Generation Data Group

A Generation Data Group, also known as GDG, is the versioning of one dataset that is successive generations of historically-related data. Considering our example, this is shown in Figure GDG on Mainframe Layout.

GDG on mainframe
Figure 12. GDG on Mainframe Layout

In Windows, this gives a structure where the directory contains the two versions of the GDG: G0001V00 and G0002V00, as shown in Figure GDG on Windows Layout.

GDG on windows
Figure 13. GDG on Windows Layout

2.1.3. Metadata

In Mainframe, metadata is stored in the catalog, for example, the metadata of RAINCODE.PROJECT1.PDSTEST is shown in Figure Datasets metadata on Mainframe.

datasets mainframe
Figure 14. Datasets metadata on Mainframe

The metadata for the same file on Windows is as follows:

<dataSet
  name="RAINCODE.PROJECT1.PDSTEST"
  copies="0"
  isCataloged="TRUE"
  createDate="02/11/2020 09:41:47"
  dataSetType="PDS"
  expireDate="12/31/9999 23:59:59"
  fileFormat="EntrySequenced"
  markForDeletion="FALSE"
  recordFormat="FB"
  recordLength="80"
>
<FileConfig></FileConfig>
</dataSet>

2.2. Concatenated Datasets

On a Mainframe, a JCL can specify a concatenated dataset; multiple datasets that are presented to the program as a single concatenation, for example, as follows:

//STEP4   EXEC PGM=DUMPCBL
//OUTFILE DD DSN=MYDATA.SAMPLE.RESULT,DISP=(NEW,CATLG),
//           SPACE=(TRK,(5,5)),LRECL=80,RECFM=FB
//INFILE  DD DSN=MYDATA.SAMPLE.FILE3,DISP=(OLD,CATLG)
//        DD DSN=MYDATA.SAMPLE.FILE2,DISP=(OLD,CATLG)
//        DD DSN=MYDATA.SAMPLE.FILE1,DISP=(OLD,CATLG)

Raincode JCL has two ways to represent concatenated files: logical concatenation (the default) or concatenating into a temporary file.

There are two ways to control the choice of whether or not to use temporary files for concatenated datasets:

  1. Globally, by the usage of the SUBMIT parameter UseTempsForConcatenate=true|false

  2. For one specific program, by creating a .rcexe description (refer to rcexe xml description of executable) and include a tag < UseTempForConcatenated>true|false</UseTempForConcatenated>

The specific configuration overrides the global configuration.

We explain these two approaches in more detail now.

2.2.1. Using logical concatenation

Raincode JCL and Legacy runtime (COBOL, PL/I, and HLASM) have logical concatenation as a default. When a concatenated dataset is declared, submit passes a list of the data files to the program. The runtime system of Raincode has the ability to transparently read from the different files of the concatenation as needed.

In our sample, the DUMPCBL program will receive DD_INFILE=MYDATA/SAMPLE/FILE3.seq;MYDATA/SAMPLE/FILE2.seq;MYDATA/SAMPLE/FILE1.seq

The significant advantage of this approach is that we do not need additional disk space or CPU time to create a concatenated temporary file. The disadvantage is that the executed program must know and manage that list convention. This is true for any Raincode program but is probably not valid for third-party tools.

2.2.2. Using a temporary file

The alternate approach is creating a temporary file to concatenate data into one file. Then, that single file will be passed as a parameter for the INFILE DD.

The advantage is that any called program sees only one input file for any DD, while the disadvantage is that the temporary file needs disk space and time to build.

2.3. Dataset Migration

This section explains migrating datasets from the Mainframe to the Raincode Catalog.

2.3.1. Transfering Datasets

To transfer a given dataset from the Mainframe, use FTP in binary mode. If the dataset is Variable Block (VB), ensure to use literal site RDW to transfer the Variable Record Descriptors, e.g.:

ftp mainframe.mycompany.com
ftp> binary
200 Representation type is Image
ftp> literal site RDW
200 SITE command was accepted
ftp> get MYFILE

2.3.2. Sequential Dataset

To migrate a sequential dataset:

  1. Transfer the file from the Mainframe to the target machine, as detailed above in Transfering Datasets

  2. Write a JCL to copy the recently transferred file into the catalog using IEFBR14. Don’t forget to give the correct file attribute (LRECL and RECFM). For example:

//LOAD3    EXEC PGM=IEFBR14
//DD00     DD  DSN=GRP.GMINI$10.MGAA1,
//             DISP=(NEW,CATLG,DELETE),
//             DCB=(LRECL=80,BLKSIZE=27920,RECFM=FB),
//             PATH=('C:\myfile',COPY)

2.3.3. VSAM dataset

To migrate a VSAM dataset:

  1. On the Mainframe, copy the data from the VSAM file to a sequential one

  2. Migrate the sequential dataset, as detailed above in Sequential Dataset

  3. Write a JCL to create the empty VSAM file. For example:

    //STEP003  EXEC PGM=IDCAMS
    //SYSPRINT DD  SYSOUT=*
    //SYSIN    DD  *
     DEFINE    CLUSTER (NAME(DATA3.ACCOUNT) INDEXED -
                        RECORDSIZE (80 80)  -
                        KEYS (19 0) ) -
               DATA     (NAME(DATA3.ACCOUNT.DAT)) -
               INDEX    (NAME(DATA3.ACCOUNT.IDX))
  4. Write a JCL to copy the temporary sequential file into the VSAM one. For example:

    //STEP004  EXEC PGM=IDCAMS
    //SYSPRINT DD   SYSOUT=*
    //SORTIN   DD   DSN=TEMP.VSAM,DISP=SHR
    //SORTOUT  DD   DSN=DATA3.ACCOUNT,DISP=SHR
    //SYSIN    DD  *
      REPRO IFILE(SORTIN) OFILE(SORTOUT)

2.4. ASA print file vs ASCII print file and LSEQ vs VLSEQ

On the Mainframe, print files are stored in FBA files using the ASA printer control character as the first character in the record. Usually, these files are produced by the report writer of COBOL programs that will use WRITE ADVANCING statements.

On the other side, Windows and open system printers do not understand the ASA control char, and therefore, printing the resulting FBA requires a conversion. It is worth considering whether to use line sequential or variable line sequential files.

We now present the different aspects of handling print files.

2.4.1. Change the RECFM of the target file

Let’s suppose the following JCL step:

//STEP001  EXEC PGM=COB001
//RPT01    DD   DSN=A.B,DISP=(NEW,CATLG,DELETE),LRECL=133,RECFM=FBA

where COB001 produces a report in the A.B file, this will generate an FBA file containing an ASA character, a first character and 132 printable characters per line.

If you replace RECFM=FBA with RECFM=LSEQ

//STEP001  EXEC PGM=COB001
//RPT01    DD   DSN=A.B,DISP=(NEW,CATLG,DELETE),LRECL=133,RECFM=LSEQ

The produced file A.B will be an ASCII Print file where controls use LF/CR/FF ASCII control characters. The LRECL should not be changed as LSEQ will include 132 printables and 1 LF character at the end of the record.

The produced files are not FBA-compatible. Typically, the number of records will be greater, and the structure of the record is different. Therefore, this solution may only be used if the produced file will be printed as-is, and no other program will need to read and process the result while assuming it is an FBA.

2.4.2. Convert the FBA to an ASCII print file using IDCAMS

In some cases, the change of RECFM is unacceptable because further steps use the produced file to read it and process or add other information. Changing the format may break the logic, and the JCL has a chance to fail.

Another solution is to convert the FBA ASA print file to the LSEQ ASCII print file at the end of the job. To do so, use IDCAMS REPRO with the CONVERT verb:

//STEP001 EXEC PGM=COB001
//RPT01    DD   DSN=A.B,DISP=(NEW,CATLG,DELETE),LRECL=133,RECFM=FBA
…
//STEP04   EXEC PGM=IDCAMS
//INF      DD  DSN=A.B,DISP=SHR
//OUTF  DD   DSN=A.C,
//             DISP=(NEW,CATLG,DELETE),
//             DCB=(RECFM=LSEQ,LRECL=133)
//SYSIN    DD *
  REPRO    INFILE(INF) OUTFILE(OUTF) CONVERT
/*

In STEP04, the IDCAMS REPRO, using the CONVERT command, reads the FBA file and converts ASA characters to output the equivalent file as an ASCII PRINT FILE.

CONVERT only works if the input file is an FBA, VBA, FBM or VBM file and the target is an LSEQ or VLSEQ file.

2.4.3. LSEQ or VLSEQ

To handle print files, Raincode has support for two line sequential files usable in RECFM.

  • LSEQ is a fixed record size line sequential. The record is fixed size padded with space, and the last character is an LF (ASCII Line Feed ).

  • VLSEQ is a variable record size line sequential. The record is variable size trimmed: all trailing space is removed, and the last character is an LF (ASCII Line Feed). Unlike VB files, no size is written at the beginning of the record. The size is computed when the first LF is read.

3. Submit

This section describes the relevant workings of Submit, the equivalent of the Mainframe’s SUBMIT command, allowing users to submit a JCL for execution. We first describe various aspects of its use, and the end of this section contains the list of current command-line options to submit.exe (as generated by submit.exe itself).

3.1. Resolution of PGM=xxx

Each step of a JCL that includes PGM=<name> requires Submit to map the name to an external program to execute. To do so, Submit performs a lookup for a filename in the form <name>.<extension>, in the order of the following table:

Table 1. Executable lookup order by type

EXTENSION

FILE TYPE

REMARK

rcexe

XML

See rcexe xml description of executable

ps1

Powershell script

sh

Linux sh script

bat

Windows batch file

Windows only

js

VB Cscript

Windows only

vbs

VB Cscript

Windows only

wsh

VB Cscript

Windows only

dll

.Net assembly

Windows dll

exe

Binary executable

Windows only

No extension

Binary executable

Linux only

Submit looks for the file in the directories in the order listed in the following table.

Table 2. Executable lookup paths

LOCATION

REMARK

STEPLIB

The path defined in the steps' STEPLIB, if defined

JOBLIB

The path defined in the JCLs' JOBLIB, if defined

Executable Search Paths

The path list defined in the attribute executableSearchPaths of the catalog.configuration.xml

submit.exe path

The path of the running submit.exe

JCLLIB

JCLLIB definition allows coding a set of libraries to find procedures and includes.

3.1.1. rcexe xml description of executable

The .rcexe file allows for configuring what program to run. It is an XML file that allows the specification of an alternate executable, amongst other options. The root element of the file is an RCLaunch element, and it may contain a number of child elements. All of these child elements are optional.

The type of XML elements allowed in an .rcexe file are described in the following table:

Table 3. XML elements in RCLaunch
XML ELEMENT DESCRIPTION

ExecutablePath

The full path to the executable. If the executable path is empty, the name is derived from the .rcexe file name where .rcexe is replaced by .exe (on Windows) or .dll or no extension (the first match is used). If the path is relative, the full path of the .rcexe file will be used as a base.

Arguments

The arguments pass to the executable when it is called.

WorkingDirectory

Working directory: if not specified, the one of Submit is used.

ParametersAsEnvVar

Are the parameters PARM=(…​) only passed as environment variables in RC_PROC_PARM? If this is false (the default), they are also passed as command line parameters.

ExecMode

Execution Mode:

  • Direct: Call ExecutablePath directly (default).

  • RcUtility: similar to direct but assumes assembly inherited from Raincode.Batch.Utilities.Common.

  • RcInternal: This would be valid for assembly inheriting from Raincode.Batch.Utilities.Common, use a direct call to IntenalExec in the process of Submit. This should be only be used by Raincode utilities.

  • rclrun: This will be called using the rclrun executable (or the custom legacy runner defined in the Catalog configuration).

UseTempForConcatenated

Use Temporary files for concatenated DDs (see the section on Concatenated Datasets)

Alias

This command is just an alias of the name provided. For Example, IKJEFT01A is an alias of IKJEFT.

If present, no other elements are needed. This is as the actual executed command will be the alias.

DisableSysoutRedirect

Disable the capture of stdout of the executed command to store it in the SYSOUT Dataset. The default is false.

When false, Submit redirects the executing command’s stdout and stores the content into the file defined by the SYSOUT DD card.

When set to true, Submit does not redirect stdout and does not open SYSOUT. The application is responsible for managing the SYSOUT.

DisableSyserrRedirect

Same as DisableSysoutRedirect for SYSERR.

For example, the file below specifies that the program name Myprog is mapped to the Windows program c:\somepath\myexec.exe, and it will use temporary files for concatenation.

MYprog.rcexe
<?xml version="1.0" encoding="utf-8" ?>
<RCLaunch>
  <ExecutablePath>c:\somepath\myexec.exe</ExecutablePath>
  <UseTempForConcatenated>true</UseTempForConcatenated>
</RCLaunch>

3.1.2. Treatment of dll files

If Submit encounters a .dll, it is supposed to be a .Net assembly. Submit inspects the assembly to determine if it contains a Raincode legacy signature (e.g. it is generated by a Raincode legacy compiler).

If the legacy signature is found, Submit will use <The Legacy Runner to execute the dll. If the signature is not found, the dll will be run using the dotnet(.exe) command-line utility.

3.2. Job Scheduler Integration

To allow integration with an external scheduler, submit.exe has as return code the return code of the job that it executes. Consequently, to integrate with a job scheduler application, follow the instructions for executing a console application and use submit.exe as the executable.

3.3. Job outputs

By default the job outputs, i.e. the contents of SYSOUT for a job, are located in the C:\ProgramData\Raincode\Batch\Sysout Directory. Each job that has been run will have its specific folder containing the SYSOUT info.

Each folder in SYSOUT will contain:

  1. MSGLOG: The standard output from running a job.

  2. SYSLOG: Job information, job step statistics, and over job statistics display.

  3. SPECIFIED STEP: Specific sysout as defined by the job JCL.

  4. JCL LISTING: JCL for the job being run.

  5. StepActivity.json: Detailed information regarding the step activities of a job. Used for restart.

  6. StepDDActivity.json: Detailed information regarding the DDs of each step of a job. Used for restart.

3.4. JCL job artifacts repository

Submit includes the ability to build a database repository that archives several artifacts of the submitted JCL job, like job steps, conditions, procedures, etc. This repository can be used to analyze the portfolio and estimate the required workload for a migration effort.

Repository generation is available only when the -ScanOnly option is chosen, by using the command-line options -DBDriver and -DBConnectionString. DBDriver specifies the desired type of database driver, and DBConnectionString specifies the desired connection string for the database.

The first database driver that can be used is SQLite, e.g., as follows:

Submit.exe -File=.\JOB.JCL -ScanOnly -DBDriver=Sqlite

When using SQLite as the database driver, the repository will be created in the same directory where Submit.exe is executed under the default repository name: RC_JCL_INVENTORY.db. However, one can specify the name of the database file through the DBConnectionString option.

Alternatively, an ODBC driver can be used, e.g., as follows:

Submit.exe -File=.\JOB.JCL -ScanOnly -DBDriver=ODBC -DBConnectString="DSN=odbcConnect"

When using the ODBC driver, the target database is an SQL server. On the server-side, one needs to create a database named: RC_JCL_INVENTORY and define the ODBC data source for the connection before executing Submit.exe. This data source needs to be passed as part of the DBConnectionString option.

The created repository includes several SQL tables giving viable information about the submitted jobs. For more details, refer to the appendix JCL JOB artifacts repository.

3.5. JCL dataset Locking

Raincode JCL emulates the locking mechanism of Mainframe JCL on datatsets using two-step lock file acquisition. Consequently no changes must be made to the JCL batches to work with Raincode JCL. In order to prevent deadlocks, locks are obtained upfront at the beginning of the job, and taken in alphabetical order based on DSNAME.

3.5.1. Locks and the file system

Recall that Raincode JCL represents each dataset by a pair of files: the data file (.seq), and the meta file (.meta), as described in Mainframe file system emulation. When a lock is obtained on a dataset, Raincode JCL will only lock the meta file.

The lock files are located in a special directory structure that exists for each Volume defined in the catalog. The structure is in a directory named _lock that is a subdirectory of the root for the Volume. To lock a dataset, so-called lock files will be placed in the _lock directory.

The _lock directory is not addressable via JCL and should only contain lock files.
Spurious locks, i.e. files being considered locked when they are not, can be removed by removing the corresponding lock files in the _lock directory. Alternatively, all locks can be removed by deleting the _lock directory.

The locking mechanism is also accessible programmatically. When developing utilities that involve catalog manipulation, developers should use the lock facilities by referencing the Raincode.Batch.LockingClient assembly and using its exposed interface. Similarly, when developing applications not called from a JCL but still interacting with the catalog, the developer needs to take care to lock and unlock files that will be accessed. This can be done using using the LockingManger static methods from the Raincode.Batch.Catalog assembly.

3.5.2. Implementation of Locking

Raincode uses a lock file system as a semaphore to signal locking on the meta file. There will be two lock files for a meta. The shared lock file (.ls) and the exclusive lock file (.le).

The shared lock file represents that a process has a shared lock on a dataset. The exclusive lock file is the semaphore to the shared lock file, and both represent that an exclusive lock is owned or that a file is in the process of acquiring a shared lock.

The main principle is that all lock manipulation is a two-step process (as illustrated in figure Locking process):

  1. Obtaining a lock to exclusively lock the exclusive file.

  2. Once the lock on the exclusive file is obtained, try to acquire a lock on the shared file. The lock will be exclusive or shared, depending on the JCL level of locking needed.

image012
Figure 15. Locking process

Using the OS-provided locking mechanism avoids writing anything into the files to side step the need for complicated book keeping. All file opens are done with a Delete on Close mechanism usually provided by the OS.

3.6. Command-line options for Submit.exe

Below are the details of the command line options of Submit.exe. These tables are automatically generated and contain the same information produced by running Submit.exe without arguments.

3.6.1. General

Command-line option Default value Description

AllowInternalCall

Allow calls of Raincode utilities as internal (non process) call. This is a boolean value.

CatalogConfiguration

Path to the configuration file for the catalog. Default value is C:\ProgramData\Raincode\Batch\Raincode.Catalog.xml . If the option is not specified, SUBMIT check for the value of the environment variable RC_BATCH_CATALOG. If this variable is found the value is used as path to the configuration file for the catalog.

CM_NCT2

ControlM support for DISP=NEW Error RCS801 at restart (default = true). This is a boolean value.

Comment

User define comment that submit will pass as argument to raincode utilities.

ControlM

ControlM support see also EnhGDG and CM_NCT2 (default = true). This is a boolean value.

DBConnectString

The repository connection string. See also DBDriver option.

DBDriver

Sets the connection string to use for the repository persistence system as an ODBC connection string if ODBC persistence is used, or as a physical file name if SQLITE is used instead. See also DBConnectString option. Values can be any of the following:

  • ODBC

  • Sqlite

DDCheckSpace

check space parameter for DD with DISP=NEW/Mod if file is not existing (default = false). This is a boolean value.

DebugStep

Name of the step to debug. Causes a JIT debugger breakpoint to trigger when the step is being executed.

DebugSubmit

Causes a JIT breakpoint to be hit at the start of the Submit process. Useful for debugging custom hooks. This is a boolean value.

DisableResolve

Start preprocessing of JCL with a state equivalent to %%RESOLVE(OFF). This is a boolean value.

DispSHRPermissive

DISP=SHR allow file is not existing (default False). This is a boolean value.

DllLogLevel

LogLevel passed to rclrun.

DSN

Fully qualified dataset name for the JCL file to submit.

EnhGDG

ContorlM support for GDG adjustment at restart (default = true). This is a boolean value.

ExecNameCaseInsensitive

(Linux Only) Looking for a program file is case insensitive (default= True). This is a boolean value.

File

Path to a file containing JCL to submit. This is useful for bootstrapping an installation since it allows the user to submit JCL that is not in the catalog yet.

FileExistsTryOpenFile

When a file is tested for existence, use FileStream.Open to confirm if System.IO.File.Exists return false (default= false). This is a boolean value.

JclConverter

string to the configuration file of the JclConverter.xml If this parameter is set the given JCL will be translated into new format of the commands for the given utilities.

JobID

Define user own Job ID.

JobName

Define user own Job Name (Ignore Job name in JOB card).

KeepTempFiles

Retain temporary files on disk instead of deleting them (default behavior). This is a boolean value.

LogToConsole

Registers a log file listener that outputs to the console. This is a boolean value.

LogToTrace

Registers a log file listener that outputs to the system Tracer. This is a boolean value.

MetaNotFoundTry

Try count when a read meta fail, see also MetaNotFoundWait.

MetaNotFoundWait

Wait Time (ms) when a read meta fail, see also MetaNotFoundTry.

MsgLogFlushInterval

MSGLOG.txt and SYSOUT logfile flushing interval in seconds (Default is -1 disabled).

PortPool

Pool of ports which can be used for tcp.net binding (default = 500).

PortStart

Port number of the first port for tcp.net binding (default = 18000).

ProcLibs

Directory where to look for PROC not in catalog.

RecordFormat

Record format of the submitted file. Used only with -File option, as DSN will contain the record format. This option also requires the -RecordLength option to be specified. Values can be any of the following:

  • FB

  • VB

  • FBA

  • VBA

  • LSEQ

  • Unknown

  • VLSEQ

  • FBM

  • VBM

  • U

RecordLength

Record length of the submitted file. Used only with the -File option. Requires -RecordFormat option as well. This option must be specified if the RecordFormat is not line sequential.

RelLockAtStep

Release file lock at step level (default = true). This is a boolean value.

Restart

Name of the step to restart the job with. Overrides the RESTART= parameter in the JCL if present. If a proc-step is desired, use format STEP1.MYPROC.

RestartJobId

ID of the job which you are attempting to restart.

RunToStep

Name of the step to run to. When the step is encountered, the job will end after executing the RunToStep. If a proc-step is desired, use format STEP1.MYPROC.

SaveJobID

SaveJobID and SYSOUT folder full path to file.

ScanOnly

Parse and scan the JCL, but do not execute any of the steps. This is useful for checking JCL syntax for compatibility. This is a boolean value.

ServerIP

Server IP for LockingService to communicate to client if client is not executed on the same machine (default = localhost).

Silent

The JCL interpreter does not output anything to the console, log is still written. This is a boolean value.

StepProfilingInfo

Step profiling info in a separate file. This is a boolean value.

SubstvarXml

Path to the XML file to be used for SUBSTVAR processing. This overrides the setting in the catalog configuration for SubstitutionVariableFilePath.

SyslogXML

store SYSLOG data in XML format (Suffix .XML). File will be stored in the same directory as SYSLOG (SYSLOG.<jobname>.TXT).(default=false). This is a boolean value.

TruncateJCLAfter72

Truncate JCL stream after column 72 (default false). This is a boolean value.

UseTempsForConcatenate

Build a temporary file for concatenated input (default= false). This is a boolean value.

UtilityLogLevel

LogLevel passed to RC Utilities.

ValidateJobID

Validate that user Job ID does not exist. Return code is set to 16 if a job with same Job ID yet exist (RC=16) (default = false). This is a boolean value.

Volume

(optional) Specify the volume for DSN.

3.6.2. Miscellaneous

Command-line option Default value Description

DateTimeOffset

The datetime value used to initialize the runtime’s time from the offset.

DotNetConfigFile

An additional app.config file.

DumpLicense

Check and dump current License. This is a boolean value.

Help

Display the help message. This is a boolean value.

InitialDateTime

The datetime value used to initialize the runtime’s time.

LogLevel

INFO

Values can be any of the following:

  • SILENT

  • ERROR

  • WARNING

  • INFO

  • DEBUG

  • TRACE

  • PROGRAM_OUTPUT

  • DIAGNOSTIC

Version

Display version information. This is a boolean value.

3.6.3. Plugin

Command-line option Default value Description

Plugin

Loads plugin-providing assembly.

PluginPath

Path where plugins (and other assemblies) are searched.

3.6.4. Testing

Command-line option Default value Description

DisableAssignProcessToJobObject

Disable call to AssignProcessToJobObject. This is a boolean value.

4. Catalog Configuration

Submit relies on the Catalog Manager, among others, to perform its functions. The Catalog Manager emulates the Catalog Manager of the file system on the Mainframe, i.e. it implements a view on the emulated file system that is part of Raincode JCL. The working of the Catalog Manager can be configured using the catalog configuration file. It allows users to change default settings, such as units, volumes, record length, etc.

When Submit starts, a lookup of the catalog configuration is performed in the following order:

  1. If the command line option -CatalogConfiguration is present, the given parameter is the path to the catalog configuration file.

  2. The value of the environment variable RC_BATCH_CATALOG, if present, is used as a path to the catalog configuration file.

  3. If present, the file in the default location C:\ProgramData\Raincode\Batch\Raincode.Catalog.xml, or /var/lib/Raincode/Batch/ Raincode.Catalog.xml in Linux, is used.

If the lookup fails, a Default Catalog Configuration file will be created in the default location, and this file will be used.

4.1. Default Catalog Configuration

Below, we present the default configuration file (for Windows, at the time of writing this document) that is created if lookup fails. It consists of a top-level CatalogConfiguration element with a number of relevant attributes, and various nested entities. We describe all of these in the remainder of this chapter.

<catalogConfiguration
   enforceMemoryRestriction="TRUE"
   defaultRecordLength="0"
   defaultSysoutRecordLength="133"
   ebcdicCompareCodePage="870"
   defaultOutputClass="A"
   defaultCodePage="1252"
   sysoutFolderPath="C:\ProgramData\Raincode\Batch\SYSOUT"
   autoEditCalendarFolderPath="C:\ProgramData\Raincode\Batch\AutoEditCalendars"
   sysoutFolderNamingPattern="JOB{ID}.{NAME}"
   defaultRecordFormat="FB"
   defaultSysoutRecordFormat="LSEQ"
   instreamDataRecordFormat="LSEQ"
   instreamDataRecordLength="80"
   directoryTypeExtension="._dir">
  <units>
    <unit isDefault="TRUE" name="SYSDA">
      <volumes>
        <volume isDefault="TRUE" name="DEFAULT" path="C:\ProgramData\Raincode\Batch\DefaultVolume"/>
      </volumes>
    </unit>
  </units>
  <recordFormatExtensions>
    <ext>.dat</ext>
    <ext>.txt</ext>
  </recordFormatExtensions>
  <fileFormatExtensions>
    <ext>.seq</ext>
    <ext>.idx</ext>
    <ext>.rr</ext>
    <ext/>
  </fileFormatExtensions>
  <defaultProcLibraries>
    <dsn>SYS1.PROCLIB</dsn>
  </defaultProcLibraries>
  <executableSearchPaths>
    <path>c:\myfile1</path>
    <path>c:\myfile2</path>
  </executableSearchPaths>
 <programDBConnectionDefinitions>
    <dbConnection stepId="STEP1.*">PLAN01</dbConnection>
    <dbConnection pgm="PRGB1">PLANA1</dbConnection>
    <dbConnection pgm="PRGA.*">PLANA2</dbConnection>
    <dbConnection >PLAN02</dbConnection>
  </programDBConnectionDefinitions>
</catalogConfiguration>

4.2. Catalog configuration description

The top-level CatalogConfiguration element can have a number of attributes, the meaning of which is described in the table below.

Table 4. Catalog configuration attributes
Tags Default Values Description

autoEditCalendarFolderPath

"Configuration_folder\AutoEditCalendars"

Path to the folder containing named calendar XML files. For more details, refer to AutoEdit Calendar configuration.

batchCustomPluginPath

Path to the folder containing the Batch Plugins.

batchCustomPlugins

Names of the Plugins that are loaded when executing Submit. For more details, refer to Plugin.

catalogDbConnection

If catalogFormat is db, the connection string for the database.

catalogFormat

disk

The catalog storage format. Accepted values: disk and db.
disk: Stored in the file system.
db: Stored in a SqlServer DB.

dbConnectionDataProvider

RcDbConnections.csv

Path to the CSV file to map the PLAN name to the actual connection string. For more details, see section, Map a PLAN name to a connection string.

defaultAsciiCodePage

The default code page to use for interpreting ASCII data. If nothing is set, the DefaultCodePage will be used. For more details, see section, JCL character encoding.

defaultCodePage

1252

The default code page to use for interpreting text data. This is particularly useful for testing jobs that need to execute under a code page different from the current System. For more details, see section, JCL character encoding.

defaultEbcdicCodePage

The default code page to use for interpreting EBCDIC data. For more details, see section, JCL character encoding.

defaultFileConfig

<defaultFileConfig> <MainFrameFBA>false</MainFrameFBA>
   <LineEndCRLF>false</LineEndCRLF>
   </defaultFileConfig>

Default File Config parameters

defaultOutputClass

A

The default output class for SYSOUT=*, if an MSGCLASS is not specified on the JOB card.

defaultRecordFormat

FB

Default record format. At the moment, only FB is supported.

defaultRecordLength

0

The default record length, if not specified, should be set to 0. If set to 0, it will be changed to 80

defaultSysoutRecordFormat

LSEQ

The default record format for SYSOUT (LSEQ=Line Sequential).

defaultSysoutRecordLength

133

The default record length for SYSOUT datasets.

defaultSysprintRecordFormat

LSEQ

The default record format for SYSPRINT (LSEQ=Line Sequential).

defaultSysprintRecordLength

133

The default record length for SYSPRINT datasets.

directoryTypeExtension

._dir

Extension for directory type datasets. (Only used if catalogFormat is disk.)

ebcdicCompareCodePage

870

The EBCDIC code page to use for comparing strings. This is used by AutoEdit conditional processing statements.

enforceMemoryRestriction

TRUE

This flag indicates whether or not a memory limit should be enforced for the process. Accepted Values: TRUE or FALSE.

enforceUpdateLastReference

FALSE

Enforce that the lastReferenceDate will be updated at any access to the data file. Accepted values: TRUE or FALSE.

instreamDataRecordFormat

FB

The default instream data record format, used if the JCL does not specify RECFM options.

instreamDataRecordLength

80

The default instream data record length, used if no DD parameters are set overriding the instream data record format in JCL.

legacyRunner

%RCDIR%/bin/rclrun.exe

Path to the legacy runner needed to run a COBOL, PL/I or HLASM program. This allows the use of a custom runner.

SDSN_Wait

"3600"

Specifies the installation policy for batch jobs that must wait to enqueue the dataset (Locking). Values can be NO, YES, or 1-999999. For more details, see section SDSN Wait.

substitutionVariableFilePath

Optional path to a file containing substitution variables, see section Substvar section

sysoutFolderNamingPattern

"JOB{ID}.{NAME}"

Naming pattern to use when generating the SYSOUT folder name. For more details, see section Sysout Folder Naming Convention.

sysoutFolderPath

"Configuration_folder\SYSOUT"

Path to a folder where job-specific SYSOUT files should be written.

sysoutMetaKeep

TRUE

TRUE: The SYSOUT meta file will be kept at the end of the step.
FALSE: The SYSOUT meta file will be deleted at the end of the step.

sysoutSyslogVersion

1

The version of the SYSLOG format in the SYSOUT folder.
1 = V1 OLD (status until 2022/04).
2 = V2 NEW (STEP logging changed PROCNAME).

sysoutVolumeName

#SYSOUT

Name of the Volume used to store SYSOUT files.

truncateRedirectedSysoutRecord

TRUE

TRUE: Truncate SYSOUT record to the length of SYSOUT when redirected to a file.
FALSE: SYSOUT record is split over several records if needed.

The CatalogConfiguration element can have various nested entities, which we describe in the following table.

Table 5. Catalog configuration elements
Tags Values Description

autoDDCardDefinitions

Definition of DD cards for the step that is not explicitly declared in the JCL. For more details, see section JCL Automatic DD card definition.

datasetSqlMapping

Datasets data that are stored into a database. For more details, see section Dataset mapping with tables.

defaultProcLibraries

<defaultProcLibraries>
    <dsn>SYS1.PROCLIB</dsn>
  </defaultProcLibraries>

List of PDS libraries to search for PROCs. SYS1.PROC is always added as the last element.

environmentVariables

Environment variable values that Submit will set at the beginning of each step. For more details, see Environment Variables.

executableSearchPaths

Paths to use when looking for an executable in an EXEC statement

fileFormatExtensions

<fileFormatExtensions>
     <ext>.seq</ext>
     <ext>.idx</ext>
     <ext>.rr</ext>
     <ext></ext>
   </fileFormatExtensions>

List of file extensions for file formats, indexed by file format enum cast as int.

noLocking

List of DSN patterns on which locking must not be applied. The pattern is a simple IDCAMS pattern a.c.C.**.

programDBConnectionDefinitions

Define rules to associate programs and jobs to a default PLAN (by extension a connection String). For more details, see section DB connection definition

recordFormatExtensions

List of file extensions for file types, indexed by file type enum cast as int.

Units

Units that are defined in the catalog, see section Unit and Volume Configuration.

codePages

<codePages jclAlternate="1252"
jclRewrite="Default"
console=""
sysout=""
sysprint="" />

The default code page to use for interpreting ASCII data. This is particularly useful for testing jobs that need to execute under a code page different from the current System. See section JCL character encoding.

4.3. Unit and Volume Configuration

Units are simply a named collection of volumes. The default unit is SYSDA.

Table 6. Unit Attributes

Attribute

Default Value

Description

name

Name of the default unit

isDefault

FALSE

The unit is the default unit

Volumes map to drives or folders on disk. They can be UNC shares or local folders.

In a distributed environment, the volumes should reside on a UNC share.
Table 7. Volume Attributes

Attribute

Default Value

Description

name

Name of the volume.

path

Folder that contains the data of the volume. It can be omitted if the catalogFormat is db and all the data are mapped to VsamSql.

isDefault

FALSE

Is it the default volume.

To add new volumes to the catalog configuration, manually create a new folder on disk and add a new <Volume> element (under the <Volumes> element) with the path to the newly created folder.

4.4. Sysout Folder Naming Convention

The unique job folder under the sysoutFolderPath can be configured by editing the sysoutFolderNamingPattern option in the catalog configuration. Two components have a specific meaning: ID and NAME. The ID string is replaced with the unique job ID, and the NAME string is replaced with the job name. The ID string is required, and the NAME string is optional.

For Example:

 sysoutFolderNamingPattern="JOB{ID}-{NAME}"

When the job name is not yet known, such as when a job is submitted, but there are errors in parsing, the job name is set to UNKNOWN. Once the job name is known, the sysout folder will be renamed accordingly.

This directory contains different files and directories:

  • JCLListing.JCL: The listing of the JCL with all the included items resolved.

  • MSGLOG.txt: The messages displayed on the console.

  • PROCESS.txt: The list of processes executed by this job.

  • RETURNCODE.TXT: The return code of the job.

  • SYSLOG.<job name>.txt: The syslog of the job. Information about the job and its different steps.

  • SerializedJob.json: Raincode internal usage for restart.

For each PROC/STEP, there is a directory containing the files generated by the job, e.g. SYSOUT, SYSPRINT. These files are only present if they are not empty.

4.5. SDSN Wait

Specifies the installation policy for batch jobs that must wait to enqueue the dataset (Locking). Possible values are NO, YES, 1-999999 with a default of 3600 seconds (One hour)

Specifies whether to cancel jobs that must wait to enqueue the dataset:

  • NO: The system cancels the job, releases resources, and issues the message RC904.

  • YES: When YES is specified, and a batch job’s enqueue request cannot be satisfied, it will wait forever until we get the resource

  • 1-999999: A timeout value in seconds to wait. If the enqueue request cannot be satisfied in the defined time, the system cancels the job and issues message RC904.

Use caution when specifying YES or a timeout, as this can cause deadlocks with other jobs in the system.

4.6. Environment variables for steps

The catalog configuration defines environment variable values that Submit will set at the beginning of each step such that the executing program can retrieve this information if needed. Below is an example.

<environmentVariables>
    <envvar name="Name1" stepId="StepRegEx">Test1Value</envvar>
    <envvar name="Test2" pgm="PRGB1">Test2Value</envvar>
    <envvar name="Test3" pgm="PRGA.*">Test3PRGA</envvar>
    <envvar name="Test3" >Test3OTHER</envvar>
</environmentVariables>

<environmentVariables> is a list of <envvar> elements. These have one mandatory attribute: Name, which is the name of the variable, and a set of optional attributes, as defined in the table below.

Table 8. List of optional envvar attributes
Attribute name Function Typical Value

jobID

Regular Expression to be matched on current JobID

00X1..

jobName

Regular Expression to be matched on current Job Name

TSK01.*

pgm

Regular Expression to be matched on current PGM=clause

PRG001

stepID

Regular Expression to be matched on the current StepID

0001.*

The value of the environment variable is the character data inside the <envvar> element.

The name attribute is mandatory, and the other attributes are optional.

The definition of environment variables and their values is conditional, based on regular expressions on Program Name, StepID, and JobID. If several regular expressions are defined for one <envvar> (e.g., stepId= "0001.*" and PGM="PRG01") all of them must match to define the environment variable.

If no regular expression is given, the environment variable is always defined.

The list of <envvar> is evaluated in the order it appears in the .xml. If the same name attribute exists multiple times, only the first match is used.

4.7. JCL Automatic DD card definition

The Mainframe can define a DD card for a step not explicitly declared in the JCL. The catalog supports this through the <autoDDCardDefinitions> section that containings <ddCard> elements. An example is below.

  <autoDDCardDefinitions>
    <ddCard name="AUTO1" RECFM="FB" LRECL="120"/>
    <ddCard name="AUTO2" />
  </autoDDCardDefinitions>

The ddCard element defines a dataset in the SYSOUT directory. It has one mandatory attribute: Name, which is the DD card name, and a set of optional attributes, as defined in the table below.

Table 9. List of optional ddCard attributes

Attribute name

Function

Typical Value

jobID

Regular Expression to be matched on current JobID

00X1..

jobName

Regular Expression to be matched on current Job Name

TSK01.*

pgm

Regular Expression to be matched on current PGM=clause

PRG001

stepID

Regular Expression to be matched on the current StepID

0001.*

RECFM

RECFM= format to use for the DD CARD (FB, VB …) Default is the format of SYSOUT

FB

LRECL

LRECL= value to use for the DD CARD. Default is the value of `SYSOUT`120

If several regular expressions are defined for one <ddCard> (e.g., stepId= "0001.*",PGM="PRG01"), all of them must match to define the DD card.

If no regular expression is defined, the DD Card is always defined.

4.7.1. Example

If <ddCard name="AUTO1" RECFM="FB" LRECL="120"/> is defined and the JCL is:

//SIMPLE   JOB  CLASS=A,MSGCLASS=C
//STEP1    EXEC  PGM=TST01

The JCL would be equivalent to the following:

//SIMPLE   JOB  CLASS=A,MSGCLASS=C
//STEP1    EXEC  PGM=TST01
//AUTO1  DD RECFM=FB,LRECL=120,SYSOUT=*

This will create a file:

For JCL 4.1:

SYSOUT/JOB0000000001.SIMPLE/STEP1.AUTO1.meta
SYSOUT/JOB0000000001.SIMPLE/STEP1.AUTO1.txt

For JCL 4.2:

SYSOUT/JOB0000000001-SIMPLE/STEP1/AUTO1.meta
SYSOUT/JOB0000000001-SIMPLE/STEP1/AUTO1.seq

4.8. JCL Connection string retrieval

When a program using a DB is used, it requires a connection string and a DB Type to pass to rclrun. On the mainframe, this is typically defined by a configuration that associates a Job or Program Name to a Plan Name, while in IKJEFT01, PLAN (xxx) lets a user define the Plan name that will determine the connection to the DB. This section provides insights into how this is done in Raincode JCL.

4.8.1. Global DB connection definition

For development and debugging or demo purposes, the simple way to define the connection string is to use environment variables. Setting them defines a global connection string that will be used by Submit and rclrun.

Define:

RC_DB_TYPE = [Sqlserver/DB2/DB2HIS/Postgres]
RC_DB_CONNECTION = <The connection string that will correspond to the DB engine defined by RC_DB_TYPE>

Example:

RC_DB_TYPE = SqlServer
RC_DB_CONNECTION = "Data source=sqlsrv2017; Initial Catalog=PLICompilerTestDB; Uid=****; Pwd=*******"

When calling rclrun, IKJEFT01 will build the command line using the connection string:

RCLRUN.exe -SqlServer="Data Source= sqlsrv2017; Initial Catalog=PLICompilerTestDB; Uid=****; Pwd=*******"
If PLAN is resolved, the new connection string will be used; otherwise, the environment variable will be used.

4.8.2. Local DB connection definition

In contrast to a global connection definition, a local connection definition is more like the mainframe production environment.

Raincode JCL emulates the mainframe connection definition and PLAN by using two attributes of the catalog configuration: dbConnectionDataProvider and programDBConnectionDefinitions. We present them next.

Map a PLAN name to a connection string

The dbConnectionDataProvider attribute in the catalog configuration defines a mapping from PLAN names to connection strings. In this file, PLAN names are C# regular expressions linked to a connection string.

Lookup for this file is performed in the following order:

  1. In the catalogConfiguration element of the catalog configuration, the value of the attribute dbConnectionDataProvider, if it exists.

  2. An environment variable RC_DB_FILE containing the path, if it exists.

  3. The default file name: RcDbConnections.csv

The format of the file is comma separated fields, with one field per line in the following form:

<PLAN RegEx Pattern>, <DB Type>, <Connection String>

Supported values for <DB Type> are: SqlServer,DB2,DB2HIS and Postgres.

Example entries are as follows:

 PLANA01, SqlServer, Data Source=sqlsrv2017; Initial Catalog=PlICompilerTestDB
 PLANB.*, SqlServer, Data Source=sqlsrv2017; Initial Catalog=CPTTestDB

The list of mappings is evaluated in the order the entries are written in the file. The first entry that matches the regular expression on the PLAN Name is used.

Before execution of any step after the change of the connection string by IKJEFT01, the environment variable is set to actual connection values:

RC_DB_TYPE=<DB Type>
RC_DB_CONNECTION=<Connection String>
Map a Job or program to a PLAN

The programDBConnectionDefinitions element in the catalog configuration contains a list of dbConnections elements. Each of these allows to define a mapping from jobId and/or ProgramId, and/or stepId using a C# regular expression to a PLAN Name. This is done through the definition of attributes on dbconnection, all of which are optional.

Table 10. dbConnection attributes

Attributes

Description

Type

jobId

Regular Expression to be matched on current JobID

Regular Expression String

jobName

Regular Expression to be matched on current Job Name

Regular Expression String

pgm

Regular Expression to be matched on current PGM=clause

Regular Expression String

stepId

Regular Expression to be matched on the current StepID

Regular Expression String

The value of the plan name is the character data inside the element.

The elements are evaluated in the order of their appearance in the configuration file, and the first match is used. If an entry does not specify a given attribute, matching will succeed for any value of that attribute.

An example list of definitions is as follows:

< programDBConnectionDefinitions >
    <dbConnection pgm="PRGB1">PLANA01</dbConnection >
    <dbConnection pgm="PRGA.*">PLANA02</dbConnection >
    <dbConnection JobId=".*" >PLANOTHER</dbConnection >
 </programDBConnectionDefinitions >

Since <dbConnection pgm="PRGB1">PLANA01</dbConnection> only specifies pgm, any jobId, any JobName, and any StepId are valid, i.e. all executions of PRGB1 will match the plan PLANA01.

Another PLAN definition source is the IKJEFT01 PLAN (xxx) option.
The DB connection string lookup logic can also be customized programmatically, see Custom Plan Mapping and Password Plugin for more information.

4.9. Use XML include in Raincode.Catalog.xml

Using <environmentVariables> or <programDBConnectionDefinitions> entities in the catalog configuration file may make this file large and unwieldy. So, you may want to store that information in separate files, possibly with different access permissions. This is possible since the catalog manager implements the W3C XML Inclusions (XInclude) 1.0 Recommendation for XML inclusion.

To allow this, the catalogConfiguration element needs to contain a xmlns:xi="http://www.w3.org/2003/XInclude" attribute. Inclusion of a file is then done by using an xi:include element, e.g. <xi:include href="file name" />.

xmlns:xi="http://www.w3.org/2003/XInclude" is a hardcoded signature that is recognized by the catalog manager. Even though it refers to a URL, your system does not need to be connected to the internet, there will be no connection to download anything. This signature is used simply to conform to the w3.org standard.

Sample usage of XML Include

File: Raincode.Catalog.xml

<catalogConfiguration
 xmlns:xi="http://www.w3.org/2003/XInclude"

><xi:include href="./envvars.xml" />
</catalogConfiguration>

File: envvar.xml

  <environmentVariables>
    <envvar name="Test1" stepId="STEP1.*">Test1Value</envvar>
    <envvar name="Test2" pgm="PRGB1">Test2Value</envvar>
    <envvar name="Test3" pgm="PRGA.*">Test3PRGA</envvar>
    <envvar name="Test3" >Test3OTHER</envvar>
  </environmentVariables>

The file envvar.xml will be included, and the actual configuration used will be equivalent to the following:

<catalogConfiguration
 xmlns:xi="http://www.w3.org/2003/XInclude"

><environmentVariables>
    <envvar name="Test1" stepId="STEP1.*">Test1Value</envvar>
    <envvar name="Test2" pgm="PRGB1">Test2Value</envvar>
    <envvar name="Test3" pgm="PRGA.*">Test3PRGA</envvar>
    <envvar name="Test3" >Test3OTHER</envvar>
  </environmentVariables>
</catalogConfiguration>

4.10. AutoEdit Configuration

Raincode JCL includes support for Autoedit of JCLs. It uses environment variables to hold replacement values for AutoEdits. If replacements, e.g. %%foo, are not set, it looks for the CTM_FOO environment variable. Additionally, some Control-M replacement variables are specified through environment variables. They are listed below:

Table 11. AutoEdit replacement variables

Parameter

Environment variable passed by Control-M

%%APPL

CTM_APPL

%%JOBNAME**

CTM_JOBNAME

%%ORDERID**

CTM_ORDERID

%%RN**

CTM_RN

%%TIME

CTM_TIME

%%$CENT

CTM_$CENT

%%DATE

CTM_DATE

%%DAY

CTM_DAY

%%MONTH

CTM_MONTH

%%YEAR

CTM_YEAR

%%WDAY

CTM_WDAY

%%ODATE

CTM_ODATE

%%ODAY

CTM_ODAY

%%OMONTH

CTM_OMONTH

%%OYEAR

CTM_OYEAR

%%OWDAY

CTM_OWDAY

%%RDATE

CTM_RDATE

%%RDAY

CTM_RDAY

%%RMONTH

CTM_RMONTH

%%RYEAR

CTM_RYEAR

%%RWDAY

CTM_RWDAY

%%JULDAY

CTM_JULDAY

%%OJULDAY

CTM_OJULDAY

%%RJULDAY

CTM_RJULDAY

%%$DATE

CTM_$DATE

%%$YEAR

CTM_$YEAR

%%$ODATE

CTM_$ODATE

%%$OYEAR

CTM_$OYEAR

%%$RDATE

CTM_$RDATE

%%$RYEAR

CTM_$RYEAR

%%$OJULDAY

CTM_$OJULDAY

%%JOBID

CTM_JOBID

Autoedit calendars are also supported and by default, should be saved to the following folder location: C:\ProgramData\Raincode\Batch\AutoEditCalendars. This location, however, is customizable in the catalog configuration using the autoEditCalendarFolderPath attribute.

The example configuration file below shows the expected structure of such a calendar file. The important part is the NAME= property since it maps to the calendar name that AutoEdit uses.

<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE DEFCAL SYSTEM "defcal.dtd">
<DEFCAL>
	<CALENDAR DATACENTER="enterprise" NAME="MYCALENDAR" TYPE="Relative">
	  <YEAR NAME="1999" DAYS="NNNNYYYYNYYYYYYYYNNNNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYNYYYYYYYYYYYYYNYYYYYYYYYYYYYYYYYYYNYYYYYYYYYYYYYYYYYYYYYYY" DESCRIPTION="This is year 2000"/>
	  <YEAR NAME="2000" DAYS="YYNYYYYYNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYNYYYYYYYYYYYYYNYYYYYYYYYYYYYYYYYYYNYYYYYYYYYYYYYYYYYYYYYYY" DESCRIPTION="This is year 2000"/>
	  <YEAR NAME="2001" DAYS="YYNYYYYYNNNNNYYYYYYYYYYNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYNYYYYYYYYYYYYYNYYYYYYYYYYYYYYYYYYYNYYYYYYYYYYYYYYYYYYYYYYY" DESCRIPTION="This is year 2001"/>
	</CALENDAR>
</DEFCAL>

Alternatively, the IAutoEditCalendar interface can be implemented and registered in the submit.exe.config file to perform the supported calendar operations. The default implementation uses the XML file format from Ctrl-M. It only supports the Relative type format shown above.

4.11. Substvar

The JCL parser supports substituting variables that start with the # @ characters (e.g., # @VAR1). The values for these variables are stored in an XML file specified in the catalog configuration attribute substitutionVariableFilePath. The format of the file is given in the example below:

<Substvars>
	<Variable Name="# @MYVAR" Value="VAL1"/>
	<Variable Name="# @TEST" Value="HELLO"/>
</Substvars>

Variables can also be passed through the IJCLPreprocessor interface when the UpdateSubstvarVariables method is called. It is simply a dictionary of key/value pairs.

4.12. Dataset mapping with tables

When using the VsamSql, VsamLite, and VsamOracle drivers (see chapter VsamSql), we have to define the dataset that will use the VsamSql driver to store data in a database instead of on the file system. This mapping information is stored in the <datasetTemplate> element under <datasetSqlMapping>.

The <datasetTemplate> element contains several attributes:

  • connection: the plan for the connection string, see section Map a PLAN name to a connection string

  • tableName: the name of the table in which the data is stored.

  • commitRate: the number of inserts to be done before a commit. If it is set to 0, the commit is only done when the file is closed. Default value = 0.

  • prefetch: the number of rows that are prefetched. Default value = 10.

  • driver: the driver to be used. Default value = VsamSql.

<datasetTemplate> contains one or more <pattern> elements, one <parameters> element and zero or more <driverParameter> elements. An example is below:

  <datasetTemplate tableName="REC_AREA" connection="VSAMDEAL">
    <pattern>
        <DSN>VSAM\.AREA\..*</DSN>
    </pattern>
    <pattern>
      <volume>DEFAULT</volume>
      <DSN>VSAM\.RECAREA</DSN>
    </pattern>
    <parameters length="316">
      <key start="1" length="10"/>
      <altKey start="305" length="5" unique="Y"/>
    </parameters>
  </datasetTemplate>

The different entities in the datasetTemplate entity are as follows:

  • <pattern> describes the dataset name that must be stored in the table. It contains at most one <volume> element that gives the name of the volume. Use the default volume if there is no <volume> element. It contains one <DSN> element, a regex representing the DSN stored in this table.

  • <parameters> describes the maximum size of the records and their keys. The attribute length is the maximum length of the records. The optional element <key> defines 0 based position (start) and the size (length) of the key. And zero or more elements <altKey> define the 0 based position (start) and the size (length) of the alternated key. The unique attribute is set to Y or YES if the alternate key is unique, otherwise it is set to N or NO.

  • <driverParameter> is used to give parameters to the driver. These additional parameters can be specific to a job, a step or a program. The additional parameters are conditional, based on a matching regular expression on Program Name, StepID, and JobID. This is done through the definition of attributes on driverParameter, all of which are optional. See the table below.

Table 12. driverParameter attributes
Attribute name Function Type Typical Value

jobID

Regular Expression to be matched on the current JobID

Regular Expression String

00X1..

jobName

Regular Expression to be matched on the current Job Name

Regular Expression String

TSK01.*

pgm

Regular Expression to be matched on the current PGM=clause

Regular Expression String

PRG001

stepID

Regular Expression to be matched on the current StepID

Regular Expression String

0001.*

Considering the contents of driverParameter: it should be a valid XML and is passed to the file driver as it is. The VsamSql allowed elements are:

  • connectionGroup: a group can be associated with a file. All the files that belong to the same group use the same DB transaction, i.e., all the tables storing the data are committed together.

  • commitReadNext: if it is set to 1 for a file, each time a record of this file is read, the connection group to which this file belongs is committed.

  • commitRate: the number of inserts to be done before a commit. If it is set to 0, the commit is only done when the file is closed. It overrides the value of the attribute of datasetTemplate.

  • prefetch: the number of rows that are prefetched. It overrides the value of the attribute of <datasetTemplate>.

Tables creation

Before using the VsamSql/VsamOracle files, tables need to be created. VsamLite creates the tables on the fly. The driver does not do the table creation because the people executing a program usually don’t have the right to create tables; that’s why there is a program, VsamSql.DbGenerator reads the catalog configuration and generates the table creation script. This script needs to be executed by somebody (DBA, for example) that has the right to create tables.

4.13. Bootstrapping your Environment: An Example

Bootstrapping your environment typically consists of creating one or multiple PDS libraries and importing existing JCL files into the PDS libraries created. Below is an example of such a JCL bootstrap job.

//BOOTSTRAP  JOBID 
//* 
//* This job is used to bootstrap the batch environment 
//* 
//* First we create the PDS Library 
//STEP1A  EXEC PGM=IEFBR14 
//DD1     DD DSN=NIMBLE.TEST.JCL,DSNTYPE=PDS, 
//        DISP=(NEW,CATLG,DELETE),RECFM=LSEQ,VOL=SER=DEFAULT 
//* 
//* Second, we import the JCL files using a DD Path option 
//STEP2   EXEC PGM=IEFBR14 
//DD1     DD DSN=NIMBLE.TEST.JCL(CONDEXEC),DISP=(,CATLG,DELETE), 
//        PATH=(&#39;.\JCL\CONDEXEC.JCL&#39;,COPY) 
//* 
//DD2     DD DSN=NIMBLE.TEST.JCL(CONDJOB),DISP=(,CATLG,DELETE), 
//        PATH=(&#39;.\JCL\CONDJOB.JCL&#39;,COPY) 
//* 
//DD3     DD DSN=NIMBLE.TEST.JCL(DCBINFO),DISP=(,CATLG,DELETE), 
//        PATH=(&#39;.\JCL\DCBINFO.JCL&#39;,COPY) 
//* 

4.14. Setting up File Shares and UNC path locations

Setting up a file share and accessing it with its UNC path means having a centralized file share location, allowing a multi-batch server environment. Each batch server will access volumes and their datasets via the respective UNC path.

To enable this, first, share the folder on the server and note the remote path to the share. Then, modify the catalog config file to use the new share (for sysout, volumes, etc.) using the remote path.

4.15. Catalog Explorer

Catalog Explorer is a graphical user interface for accessing Raincode JCL Catalog. Catalog Explorer helps manage datasets of type File, PDS, and GDG.

This tool makes it easy to differentiate different datasets and see the metadata about the datasets without having to parse the XMLs manually.

For more details, refer to the Catalog Explorer Manual.

5. Utility

Raincode JCL provides a number of utility programs that emulate the functionality of their mainframe counterparts, e.g. the IDCAMS file manipulation utility. All these programs are located in the installation directory (where submit.exe can be found) and have the same name as their mainframe counterpart for seamless treatment of JCLs. In this chapter, we present the specifics of a number of utilities provided by Raincode JCL.

5.1. IDCAMS

IDCAMS (Integrated Data Cluster Access Method Services) is a file manipulation utility. It allows for the creation, modification, and deletion of datasets.

5.1.1. General Assumptions

Standard IDCAMS entry name specifications are used. Lowercase characters are converted to uppercase. Invalid characters outside of the 0-255 range are silently ignored.

Windows-reserved file names CON, PRN, AUX, NUL, COM[1-9], and LPT[1-9] cannot be used as a dataset name qualifier.

5.1.2. Additional command line arguments

The following command-line arguments have been added to the Raincode implementation:

  • ScanOnly: performs only scanning/parsing of passed input

  • PreProcessDump: dumps preprocessed input to a file

  • ScanToDb: works in pair with ScanOnly to dump information about the statements after parsing by creating a database table of statements, their parameters, and the indication of a successful parsing.

5.1.3. Modal commands

IF-THEN-ELSE, NULL, DO-END, SET, and CANCEL modal commands are supported.

5.1.4. Supported functional commands

Commands are listed in the unabbreviated form. Standard IDCAMS abbreviations can be used.

Allocate datasets
ALLOCATE DATASET (dsname)
[FILE(ddname)]
[NEW|OLD|SHR|MOD]
[KEEP|CATALOG|DELETE|UNCATALOG]
[DIR(integer)]

Allocates dataset dsname. DIR must be specified for a new partitioned dataset.

This command is only useful for allocating a new partitioned dataset.

Some of the parameters file(ddname), old, shr, mod, keep, delete, uncatalog are supported by the parser but are currently ignored.
Rename
ALTER entryname NEWNAME(newname)

Rename entryname to newname.

Define alias
DEFINE ALIAS (NAME(aliasname) RELATE(entryname))
Only parsing of define alias is supported.
Define Cluster

Defines a VSAM cluster. Supported attributes are NAME, DATA, INDEX, RECORDSIZE, KEYS, ERASE/NOERASE, INDEX, CYLINDERS, KILOBYTES, MEGABYTES, RECORDS, TRACKS, ALTERNATEINDEX. Only KSDS indexed files are supported at the moment.

Implementation of the VSAM KSDS for DEFINE ALTERNATEINDEX has a few limitations:

  • The definition ALTERNATEINDEX must be done before any actual creation of the datafile (open output and add record)

  • The attributes used by DEFINE ATLERNATEINDEX are KEY and UNIQUE/NOUNIQUE. All other attributes are ignored

  • Added ALTERNATEINDEX are always UPDATE

  • DELETE of ALTERNATEINDEX is not supported

  • BLDINDEX is not supported, as ALTERNATEINDEX are always updated

Define GDG (Generation Data Group)
DEFINE GENERATIONDATAGROUP
LIMIT(limit)
[EMPTY|NOEMPTY]
[SCRATCH|NOSCRATCH]

Define a GDG (Generation Data Group).

Delete datasets or GDG (Generation Data Group)
DELETE (entryname[ entryname …])
[ALIAS|
ALTERNATEINDEX|
CLUSTER|
GENERATIONDATAGROUP|
LIBRARYENTRY|
NONVSAM|
NVR|
PAGESPACE|
PATH|
TRUENAME|
USERCATALOG|
VOLUMEENTRY|
VVR]
[ERASE|NOERASE]
[FILE(ddname)]
[FORCE|NOFORCE]
[PURGE|NOPURGE]
[RECOVERY|NORECOVERY]
[SCRATCH|NOSCRATCH]

Deletes a dataset or a GDG. Wildcards are supported in the entryname. Partial datasets are deleted only if they contain no members.

Some of the parameters like alias, cluster, libraryentry, nvr, pagespace, path, truename, usercatalog, volumeentry, vvr, recovery, norecovery, scratch, noscratch are supported by parser but currently ignored.
List catalog entries
LISTCAT ENTRIES(entryname [ entryname...])|LEVEL(level)
[USERCATALOG]
[NONVSAM]
[OUTFILE(ddname)]
[NAME|HISTORY|VOLUME| ALLOCATION|ALL]

List catalog entries. Listing non-VSAM datasets option is supported. Wildcards are supported in the entryname. Listing to an output file is supported.

Some of the parameters usercatalog, history, volume, allocation, are supported by the parser but are currently ignored.
Print
PRINT {INFILE(ddname)|INDATASET(entryname)} [COUNT(number)]
[CHARACTER|HEX|DUMP]
[OUTFILE(ddname)]

Printing in hex/dump or character formats is supported - print dataset as text records. Only sequential datasets are supported. Printing to an output file is supported.

Copy
REPRO {INFILE(ddname)|INDATASET(entryname)}
{OUTFILE(ddname)|OUTDATASET(entryname)}
[SKIP(number)]
[COUNT(number)]

Copy dataset to a new dataset. Only sequential datasets are supported.

5.2. IEBGENER

IEBGENER is a copy utility that copies records from a sequential dataset or converts a dataset from a sequential organization to a partitioned organization.

There are four main use cases for IEBGENER:

  1. Creating a copy without editing - creates an identical copy of the source dataset.

  2. Creating a copy with editing - creates an edited copy of the source dataset by affecting the length, size, and organization of source records.

  3. Creating a partitioned dataset or PDS member without editing - creates the partitioned dataset and/or creates PDS members identical to the source dataset.

  4. Creating a partitioned dataset or PDS member with editing - creates the partitioned dataset and/or PDS members from the source dataset by affecting source records' length, size, and organization.

The last use case, creating a partitioned dataset or PDS member with editing, is currently not supported.

5.2.1. General Assumptions

When creating a PDS member, the PDS directory needs to exist (see the chapter on mainframe file system emulation), or the DD=SYSUT2 data definition needs to contain information that the dataset is partitioned.

5.2.2. Utility control statements

There are five utility control statements: GENERATE, MEMBER, RECORD,EXITS and LABELS. Of these, EXITS and LABELS are not supported

GENERATE

The GENERATE statement is required when output is to be partitioned, or editing is to be performed.

The GENERATE statement needs to be the first statement in the instream-data content.
[label] GENERATE    [,MAXNAME=n]
                    [,MAXFLDS=n]
                    [,MAXGPS=n]
                    [,MAXLITS=n]
                    [,DBCS={YES|NO}]

All GENERATE statement parameters are supported by the parser; however, only MAXNAME and MAXFLDS parameters have their functionality implemented.

MEMBER

The MEMBER statement is used when the output dataset is to be partitioned.

[label] MEMBER      NAME=(name)
RECORD

The RECORD statement is used to define a record group and to supply editing information.

If the RECORD statement appears after the MEMBER statement, it defines editing information for that member, case: Creating a PDS dataset (or PDS member) with editing.

If the RECORD statement appears just after the GENERATE statement, no MEMBER statements are allowed. When doing this, the RECORD statement defines editing information for the use case Creating a copy with editing.

[label] RECORD      [{IDENT|IDENTG}=(length,'name',input-location)]
                    [,FIELD=([length],
                            [{input-location|'literal'}],
                            [conversion],
                            [output-location])]
                    [,FIELD=...]
                    [,LABELS=n]

The parser supports all RECORD statement parameters. However, only the FIELD parameter has its functionality implemented, with the exception of its literal and conversion options.

Record groups and labels are currently NOT supported.

5.3. Sort Runner

The Raincode Sort Runner is designed for compatibility with the existing mainframe SORT utility; there is no change needed to the sort commands. Datasets are sorted, copied, and merged as on the mainframe.

The Sort Runner implements sort functionality both for the calls of SORT from a COBOL or PL/I program as for standalone SORT executions from a JCL. The code of the Raincode Sort Runner is in a .dll. The Raincode runtime uses that .dll for all the sort programs called directly from COBOL or PL/I. SORT.exe, also relies on this .dll to perform its work.

5.4. IKJEFT01

Raincode provides a bare-bone implementation of the IKJEFT01 utility. Only the following commands are partially implemented: RUN, OCOPY, and LISTCAT.

IKJEFT01 on the mainframe is often used to launch applications and connect them to the database. The Raincode implementation of IKJEFT01 relies on an underlying runner, which by default is The Legacy Runner rclrun.exe. IKJEFT01.exe will invoke rclrun.exe and pass the required command-line arguments. If an alternative implementation of a legacy runner should be used, this is configured by adding an environment variable LegacyRunner and providing the full path to the executable for the alternative runner.

To provide connection strings to the utility, the user must provide an environment variable RC_DB_FILE containing the path to a file providing the connection strings. This file must be in the following format:

  • One connection string per line

  • Each line consists of three comma-separated parameters: <System>, <DB Type>, <Connection String>

Supported values for DBType are SqlServer, DB2 and DB2HIS.

For more information on database connections and connection strings, see section JCL Connection string retrieval.

5.5. DSNUTILB

Raincode provides an implementation of the DSNUTILB load and unload functionality. It obtains the connection string from the environment variables RC_DB_CONNECTION and RC_DB_TYPE. Both are set up by the JCL interpreter when the DSN parameter is provided to the EXEC command. For more details on how to specify the database to be used, see section JCL Connection string retrieval.

The utility understands UNLOAD commands of the following form:

UNLOAD TABLESPACE spacename FROM TABLE tablename LIMIT amount
       WHEN condition

And LOAD statements of the form:

LOAD DATA RESUME YES INDDN name
INTO TABLE tablename WHEN condition (columndescription, *)

5.6. DSNTIAUL

Raincode provides an implementation of the DSNTIAUL utility. It obtains the connection string from the environment variables RC_DB_CONNECTION and RC_DB_TYPE. Both are set up by the JCL interpreter when the DSN parameter is provided to the EXEC command. For more details on how to specify the database to be used, see section JCL Connection string retrieval.

By default, DSNTIAUL executes the SQL queries without any modification. However, in some cases, for example, when the target DB type changes, it is desirable to translate the queries before executing them.

DSNTIAUL can be extended with a plugin that will translate the queries, and it is delivered with a plugin that translates DB2 queries into SqlServer queries. This plugin uses cobrc to perform the query translation, and the result is stored in a cache (Sqlite) Database.

To use this plugin, DSNTIAUL must be called with two additional options -PluginPath=<DSNTIAUL.exe path> -Plugin=queryTranslation.dll

The plugin can be parametrized using different environment variables:

Table 13. DSNTIAUL plugin environment variables
Variable Default value Description

RC_SQL_REWRITE_CACHE_DIR

Temporary folder

The folder where the cache DB will be stored

RC_SQL_REWRITE_COBRC_OPTION

:IgnoreClassification :ScriptPaths={ScriptPath} :RewriteSQLProc=CastSubstrings.Rewrite :SQLSemanticRulesFile={SemanticRules} :SQLRewritingRulesFile={RewritingRules} :SQLSupportSelectStar :SQL=sqlserver :SQLServerConnectionString = "{connData.ConnectionString}"

The options passed to cobrc.exe

If needed, a custom version of this plugin can be created. Its sources are available inside the samples directory of the installation: \samples\UtilitiesSamples\DSNTIAUL\QueryTranslation

5.7. DFSRRC00

DFSRRC00 is the utility to access IMS through a JCL. Raincode only implements a subset of the original utility:

  • BMP/DLI: execute a program that uses IMS

  • ULU: load/unload data

DFSRRC00 takes only one argument: the PARM taken from the JCL. Recall that on the mainframe this PARM consists of a comma-separated list of elements. The length of the list and the meaning of the elements depend on the value of the first element of the list.

We now provide details of what subset of DFSRRC00 is implemented.

5.7.1. BMP/DLI

If the first element of the parameter list is BMP or DLI, the program executes a batch program (COBOL or PL/I). The list of parameters is 21 elements long; we only provide support for a subset of the elements, as detailed in the table below.

Table 14. Supported parameters in DFSRRCC00 BMP and DLI
List position Name or value Description

1

BMP or DLI

Execution mode

2

MBR

Specify the name of the program

3

PSB

Specify PSB name

19

CKPTID

Specify checkpoint for program restart

20

IMSID

Override IMS subsystem identifier

21

DEBUG

Set this parameter to TRUE to debug the IMSql plugin

In Raincode IMSql, IMSID is the plan used to connect to the database. If the execution mode is BMP, IMSID is the region name, and the plan config_<IMSID> is used to get the connection string to the TM database.

5.7.2. ULU

If the first element of the parameter is ULU, the utility unloads or loads a DBD depending on the value of the second element: DFSURGU0 to unload or DFSURGL0 to load.

Table 15. Supported parameters in DFSRRCC00 ULU

List Position

Value

Description

1

ULU

2

DFSURGU0 or DFSURGL0

Unload or load

3

The DBD name

The name of the DBD to unload/load is in the third position. When loading, the DFSUINPT DD is the dataset that will be loaded into the DBD. When unloading, the DFSURGU1 is the dataset in which the DBD will be unloaded. To get the connection to the database, the prefix (until the last .) of the IMS DSN is used as a key to search in the JCL Connection string retrieval specification.

5.8. FTP

Raincode provides an implementation for the mainframe FTP functionality. It accepts both FTP and FTPS protocols. If FTPS is used, all certificates are accepted.

The connection information is read from the NETRC DD. The environment variable USERID contains the default user id that is used to find FTP.DATA

5.8.1. Implemented options and commands

The utility has support for two options: Exit and TRACe. The list of implemented commands is given in the table below.

Table 16. Supported commands in FTP

Command

Notes

APpend

AScii

Uses the encoding defined in the defaultAsciiCodePage property of the catalog configuration. For more details, refer to Catalog Configuration.

BINary

BLock

CWd

CD

CDUp

CLose

DELEte

DIr

EBcdic

Uses the encoding defined in the defaultEbcdicCodePage property of the catalog configuration. For more details, refer to Catalog Configuration.

FIle

Get

Not for VsamSql

LCd

LS

MDelete

MGet

MKdir

MOde

Only B and S mode are supported

MPut

NOop

Open

PAss

PUt

PWd

QUIt

RECord

REName

RMdir

STREam

STRuctu

Only F and R are supported

TYpe

Only A and E are supported

User

Verbos

5.8.2. Command line options for FTP

Below is the detail of the command line options of FTP, as generated by ftp.exe itself.

General
Command-line option Default value Description

CatalogConfiguration

Path to the configuration file for the catalog.

Comment

User define comment pass as argument.

Debug

Causes a JIT breakpoint to be hit. Useful for debugging custom hooks. This is a boolean value.

f

Path to FTP.DATA config file.

KeepTempFiles

Retain temporary files on disk instead of deleting them (default behavior). This is a boolean value.

LogToConsole

Registers a log file listener that outputs to the console. This is a boolean value.

ScanInventory

The repository inventory file name to proceed.

ScanOnly

Indication that input should only be parsed. This is a boolean value.

t

Ftp timeout in second (default is 15).

Miscellaneous
Command-line option Default value Description

DotNetConfigFile

An additional app.config file.

Help

Display the help message. This is a boolean value.

LogLevel

INFO

Values can be any of the following:

  • SILENT

  • ERROR

  • WARNING

  • INFO

  • DEBUG

  • TRACE

  • PROGRAM_OUTPUT

  • DIAGNOSTIC

Version

Display version information. This is a boolean value.

Plugin
Command-line option Default value Description

Plugin

Loads plugin-providing assembly.

PluginPath

Path where plugins (and other assemblies) are searched.

6. JCL Character Encoding

Character encoding (and decoding) refers to the process of transforming a set of characters to and from a sequence of bytes. The mainframe uses EBCDIC encoding, while most open systems, such as Windows or Linux, work using ASCII encoding. This implies that migrating from the mainframe also includes deciding how to treat character encoding.

For more information on character encoding and codepages in .Net, we refer to the Microsoft documentation on the API for System.Text.Encoding.

In the remainder of this section, we will use the following two codepages as examples: number 870 is IBM EBCDIC (Multilingual Latin-2), and number 1252 is Western European (Windows). Note that the former is not supported in .Net

6.1. Configuration settings and autodetection

There are two configuration settings in the Raincode Catalog that determine how character encoding is treated: defaultCodePage and codePages. The latter also determines autodetection of codepage behaviour, so we discuss this here as well.

6.1.1. defaultCodePage attribute

The defaultCodePage attribute is mandatory; it defines the code page used by the system to store data files and source files.

  • Instream data found in JCLs are stored on disk using the encoding specified in the defaultCodePage

  • Unless overridden by a codePages element configuration, JCLs are read using the defaultCodePage

  • Submit assumes the encoding specified in defaultCodePage is used by the stack runtime

  • All newly created datasets [DISP=NEW] will use the defaultCodePage

  • The RC_CODEPAGE environment variable is set to this value before calling step programs. The effect of this is to enforce defaultCodePage as the default code page for any Raincode utility.

  • The output files produced by utilities like DSNTIAUL/DSNUTILB will use defaultCodePage to encode data.

6.1.2. codePages element

The codePages element in the catalog configuration file contains two attributes, jclAlternate and jclRewrite, as described in the table next.

Table 17. codePages element attributes
Attribute Type Format

jclAlternate

String

<Codepage identifier>

jclRewrite

String

`"Alternate"' or `"Default"'

jclRewrite is the codepage used to write JCL and PROCLIB on disk in the SYSOUT folder. If this is set to "Alternate" , the codepage specified in jclAlternate will be used. Otherwise, i.e. if undefined or "Default", the defaultCodePage will be used.

For example, if defaultCodePage="870", the value jclAlternate="1252" means the user expects JCL and PROCLIB to be written as ASCII on the disk when SUBMIT reads the file.

jclAlternate is also the auto-encoding detection alternative codepage when reading a JCL or a PROCLIB, as discussed next.

6.1.3. Autodetection

In some situations, a user may need as many JCLs as possible to be in the ASCII encoding, yet some remain in EBCDIC for a specific reason (automatic conversion of the catalog, part of PROCLIB written by an application program, …). To allow this, Raincode provides Autodetection of Encoding of JCL and PROCLIB between the default encoding and the alternate encoding, both as specified above.

With jclAlternate set, when reading a JCL or a PROCLIB, SUBMIT will test for the “//” pattern as the beginning of the first line of the JCL. It will read the first two bytes of the file and use both encodings to determine which one translates those bytes to “//”. This encoding is then used to read the entire JCL.

6.2. JCL Character Encoding Strategies

There are three big strategies on how to treat character encoding: purely in ASCII, purely in EBCDIC and a mixed encoding. We now present them in a bit more detail.

6.2.1. Pure ASCII Encoding

When using purely ASCII, the JCL, PROCLIB and Data files are encoded in an ASCII codepage. To do this, programs are compiled with StringRuntimeEncoding set to ASCII and the defaultCodePage in the Raincode Catalog is ASCII , e.g. 1252.

The ASCII code page is the default for Raincode tools as it is the native encoding to Windows and Linux servers.

Advantages:

  • All third-party tools use the same ASCII encoding; it is more natural for an open system.

Disadvantages:

  • Files must be converted when transferred (from and to the Mainframe).

  • The collating sequence is different to EBCDIC, and so the result of the sort may be different.

  • When validating the migration, comparing the result file is more complicated due to the difference in encoding and collating sequence.

6.2.2. Pure EBCDIC Encoding

When using purely EBCDIC, i.e. the native encoding of the mainframe, the JCL, PROCLIB and Data Files remain encoded in an EBCDIC codepage. Programs are compiled with StringRuntimeEncoding set to EBCDIC, and the defaultCodePage in the Raincode Catalog is EBCDIC , e.g. 870.

Advantages:

  • Files do not need to be converted.

  • The collating sequence is identical as on the mainframe.

  • Validation of the migration by comparing the result file is more effortless.

Disadvantages:

  • Third-party tools need to be able to use EBCDIC, which complicates data manipulation.

  • Source editing requires a text editor that supports the EBCDIC codepage.

6.2.3. Mixed Encoding

In a mixed encoding, data files are stored in EBCDIC and JCL and PROCLIB files are stored in ASCII or EBCDIC.

Programs are compiled with StringRuntimeEncoding set to EBCDIC, and the defaultCodePage in the Raincode Catalog is EBCDIC , e.g. 870. In addition, the codePages jclAlternate needs to be set to ASCII, e.g. jclAlternate="1252" (see the section on Autodetection for more information on the effect of jclAlternate).

This setting fundamentally allows data to be processed and stored in EBCDIC while sources are stored and edited in ASCII for convenience. In addition, the autodetection process allows JCL and PROCLIB files to be also stored in EBCDIC, permitting JCL and PROGLIB files to be generated by a program (which produces its data in EBCDIC format).

The stack compiler has options (for more details, refer to Encoding) to define the Encoding of runtime and sources.

Advantages:

  • Data files do not need to be converted.

  • The collating sequence is identical as on the mainframe.

  • Validation of the migration by comparing the result file is more effortless.

  • Editing of JCL may be done with standard third-party tools.

Disadvantages:

  • Third-party tools need to be able to use EBCDIC, which complicates data manipulation.

  • To be editable in standard third-party tools, JCL files need to be converted to ASCII.

6.3. DD CARD Codepage Enforcing

For some operations, you may need to be able to enforce the encoding format, overriding the default format. This as some third-party utilities may require JCL instream data to be encoded in a specific codepage, or the encoding of the output file of the DSNTIAUL and DSNUTILB utilities may need to in a specific codepage.

Raincode JCL provides two extensions to the JCL DD statement: CODEPAGE=<integer value> and AUTOENCODE=<YES/NO>. The CODEPAGE setting overrides the default codepage that is specified in the catalog configuration file. AUTOENCODE=YES will allow the runtime to transparently convert the content of the file to the actual encoding that is being used by the runtime, if needed.

For example, in the JCL below we use CODEPAGE=1252 to enforce the encoding of the instream commands on disk to be 1252. This allows third-party tools to easily work with the files. If the runtime uses EBCDIC (e.g. when using Pure EBCDIC Encoding or Mixed Encoding) then AUTOENCODE=YES ensures that it will transform the contents of the file to EBCDIC encoding before passing it to the program MYPROG.

//STEPXXX EXEC PRG=MYPROG
//INFILE DD CODEPAGE=1252,AUTOENCODE=YES,*
 Instream command 1
 Instream command 2
 ...
/*

Similarly, the CODEPAGE argument in the DD statement of the output file for the programs DSNUTILB and DSNTIAUL will modify the encoding of the output file to the specified codepage.

Auto encoding supposes that the file is a pure text file and will blindly transform its contents. Consequently, if the file contains binary data, then this data will also be remapped, which will corrupt the file.

7. VsamSql

7.1. Introduction

The idea behind VsamSql is to store the dataset data and meta information (i.e. properties of a dataset, see also Mainframe file system emulation) in a database instead of on a file system.

vsamsql
Figure 16. VsamSql is a file driver

All the Raincode tools and compiled programs use the file driver to access the files (i.e. datasets). To store the data in a database instead of on disk, the VsamSql file driver is provided that redirects input-output instructions to the database. Consequently, all the programs that use the file driver interface can store data in the database seamlessly, as illustrated in the figure above.

The VsamSql driver is designed such that one table can contain the data of multiple files, as long as each of them has the same properties, i.e. record length, keys position, record type and record structure.

7.2. Catalog configuration

The Raincode Catalog is configured to use VsamSql for specific datasets through the datasetSqlMapping element of the catalog configuration file. For more details, refer to the Catalog configuration section.

The datasetSqlMapping element holds datasetTemplate child elements that map the files to the corresponding table in which the data must be stored, as described in Dataset mapping with tables. For each table, a datasetTemplate element declares the name of the table and the plan to use to connect to the database. One or more pattern elements use regular expressions to identify the files (Unit and DSN) mapped to the table. The parameters element describes the record: its length and keys.

Additionally, the catalog configuration file contains information about where the catalog should be stored (on the file system or in a database) and whether the file should be stored in the database or the file system. The attribute catalogFormat describes the location of the catalog: disk – the catalog is stored on the file system; db – the catalog is stored in the database; if the catalogFormat is db, catalogDbConnection is the plan/connection string. For more details, refer to Map a PLAN name to a connection string to connect to the database. If the catalog is stored in the database, two tables are used to store the catalog information: CATALOG_META and CATALOG_LOCK.

7.3. Tables creation

The table VSAM_META contains meta-information about the files stored in the database.

Each data table contains three columns:

  • RID: an SQL Server identity column

  • FILE: the foreign key to the VSAM_META table

  • Data: the actual data as a binary array (VARBINARY)

If the table is used to represent an indexed file, then the table has one additional column: KEY. It holds the key for each record of the dataset. It is a computed-persistent column as a substring of the Data column.

The creation script of the tables is generated by VsamSql.DbGenerator. This program reads the catalog configuration file, whose path is given as an argument, and generates a SQL file that contains the tables creation SQL script. This SQL script then needs to be executed on the database server.

An example table script generation is as follows:

VsamSql.DbGenerator.exe -CatalogConfiguration="C:\ProgramData\Raincode\Batch\Raincode.Catalog.xml" -OutputFile="output.sql"

7.4. View creation

The Data column is binary data: the array of bytes that correspond to the COBOL or PL/I program record. In those programming languages the records are read as one array of bytes and transparently mapped to a variable. This variable is usually defined in a copybook or include, and is composed by a hierarchy of sub-variables meaningful to the application. The application then accesses the sub-variables directly instead of working with the array of bytes that is the record.

To aid in the development of programs that use VsamSql but are not written in COBOL or PL/I, VsamSql can define views. The idea here is that these views decompose the Data column based on the variables defined in a COBOL or PL/I program, as shown in the following figure.

viewcreation
Figure 17. View creation

Triggers are associated with the view, enabling them to be used to both query and update the data.

The view can be created as below:

  1. Use the COBOL or PL/I compiler to extract variable declarations from the program source code into an xml file:

    cobrc.exe :DeclDescriptors= "output_xml_file.xml" :MaxMem=1G "cobol_program_or_copybook" :SQL=SqlServer
  2. Use CopybookViewGenerator to generate the view and trigger the creation script from this xml file:

    CopybookViewGenerator.exe -xml="xml_file.xml" -struct=NAME_OF_COBOL_VAR -table=TABLE_NAME -output="OUTPUT_FILE.sql" -conn="connection string to the DB"
  3. Before executing the SQL script created from the above step, ensure the following SQL scripts are executed: EbcdicFuncs.sql and Functions.sql (these are part of the compiler installation and can be found in "%RCBIN%\sql").

8. JclConversion

JclConversion is a functionality that converts JCL into a new format, typically such that the converted JCL uses a different utility than what was originally specified in PGM. To activate this functionality, the option JclConverter of Submit is used.

In itself, JclConversion does not perform the conversion; instead, it delegates the core of this work to external programs. These programs are not meant to be written by the user, they are supplied by Raincode. The converted output is placed in an output directory, as specified by the configuration file discussed below.

The result of a conversion is not automatically used by Submit in future calls of the JCL. Instead, it is the responsibility of the user to ensure that the converted JCL, which has been placed in the output directory, is used where appropriate.

8.1. JclConverter config file

The JclConverter config file is an XML file that describes which PGM= must be converted and how to convert them. An example file is below, and it specifies that calls to SORT need to be replaced with calls to SYNCSORT and that the SYSIN DD needs to be converted by the XSORT conversion program.

<JclConverterInfo
  JclConvSaveDir="outputDir">
  <JclConverterPGMs>
    <JclConverterPGM
      PGM_ConvScript="XSORT"
      PGM_ConvDD="SYSIN"
      PGM_ConvExeCur="SORT"
      PGM_ConvExeNew="SYNCSORT" />
  </JclConverterPGMs>
</JclConverterInfo>

There is one JclConverterPGM element for each PGM to be converted. The meaning of the different attributes are:

XML Attribute

Description

JclConvSaveDir

Defines where the new JCL will be saved with the same name as the original one

PGM_ConvExeCur

The name of the PGM to match on

PGM_ConvExeNew

The new name of the PGM, after conversion

PGM_ConvDD

DD name with the input that will be converted into the new syntax by the conversion program. This DD will have the new commands in the newly created JCL

PGM_ConvScript

The script that will perform the conversion

If the converted JCL already exists in the JclConvSaveDir, this file will be overwritten.
The JCLConverter supports only inline DD for the parameter PGM ConvDD, as it will place the result of conversion there.

Below, we include three sample JclConverterPGM elements.

Sample 1: JclConverterPGM to convert a SORT step

<JclConverterPGM
 PGM_ConvScript="XICETOOL"
 PGM_ConvDD="TOOLIN"
 PGM_ConvExeCur="ICETOOL"
 PGM_ConvExeNew="XXXTOOL" />

Sample 2: JCLConverterPGM to convert DB2 LOAD/UNLOAD (DSNUTILB) step

<JclConverterPGM
 PGM_ConvScript="XDSNUTILB"
 PGM_ConvDD="SYSIN"
 PGM_ConvExeCur="DSNUTILB"
 PGM_ConvExeNew="RCDSNUTILB" />

Sample 3: JCLConverterPGM to convert HPU UNLOAD (INZUTILB) step

<JclConverterPGM
 PGM_ConvScript="XINZUTILB"
 PGM_ConvDD="SYSIN"
 PGM_ConvExeCur="INZUTILB"
 PGM_ConvExeNew="RCINZUTILB" />

8.2. Considerations for JCLConverter

The use of the JCLConverter has the following prerequisites:

  • The translation tools must be installed with the Raincode Stack.

  • The JCL to be converted must be executed in a working environment where the file catalog is available; files defined in JCL will be accessed within this JCL run.

  • Files will be created or updated depending on the use of their DD statements in the JCL, just like a normal execution.

  • RestartJobID can not be used.

In the case of a Sort SYSIN or ICETOOL TOOLIN to be converted into multiple steps (SYSINs and TOOLINs), the following changes to the JCL will apply:

  • In the JCL, we will now have SYSIN1 to SYSINn depending on the generated SYSINs from the SortTranslator for SORT or ICETOOL.

  • For the SORT, the EXEC PGM statement will get an additional parameter for the execution:

PARM='SYSIN=n,TEMP=n'

where SYSIN=n, is the number of SYSINs and TEMP=n is the number of used temporary files of the SYNCSORT execution.

For the ICETOOL, the EXEC PGM statement will get an additional parameter for the execution:

PARM='TOOLIN=n,TEMP=n'

where TOOLIN=n is the number of TOOLINs and TEMP=n is the number of used temporary files of the Syncsort execution.

  • The Powershell script Syncsort or ICETOOL will read these parameters and execute the SYNCSORT/ICETOOL.exe as required.

  • All used $ENV:DD_variables will be replaced in this Powershell script. The DD_TMP files will be in the SYSLOG directory of the given JOB.

8.3. Sample Sort

8.3.1. Input

//STEP03   EXEC PGM=SORT

//SYSIN    DD   *
SORT FIELDS=(1,12,CH,A)
/*

8.3.2. Output

Below is the output produced from the SORT command: submit -Jclconverter="c:\convertedJcl.xml"

//STEP03   EXEC PGM=SYNCSORT

//SYSIN    DD   *
/FIELDS Src_0 1 12 CHARACTER
/INFILE $env:DD_SORTIN FIXED 80
/KEYS Src_0 ASCENDING
/COLLATINGSEQUENCE DEFAULT EBCDIC
/OUTFILE $env:DD_SORTOUT FIXED 80 OVERWRITE
/*
Environment variables $env:DD_SORTIN etc. will be replaced with the actual content at execution before the SYNCSORT is called.

8.4. Sample SORT for multiple SYSIN files

8.4.1. Input

//STEP02   EXEC PGM=XSORT
//SYSIN    DD   *
  JOINKEYS F1=INPUT1,FIELDS=(3,4,A,7,4,A)
  JOINKEYS F2=INPUT2,FIELDS=(1,4,A,5,4,A)
  JOIN UNPAIRED,F1,ONLY
  SORT FIELDS=(3,20,CH,A,68,10,ZD,A)
/*
//*       END

8.4.2. Output

//STEP02   EXEC PGM=SYNCSORT,PARM='SYSIN=4,TEMP=3'
//SYSIN1   DD   *
/FIELDS Src_0 3 4 UINTEGER BIGENDIAN, Src_1 7 4 UINTEGER BIGENDIAN
/INFILE $env:DD_INPUT1 STREAM CRLF
/KEYS Src_0 ASCENDING, Src_1 ASCENDING
/COLLATINGSEQUENCE DEFAULT EBCDIC
/OUTFILE $env:DD__TMP1 STREAM CRLF OVERWRITE
/END
/*
//SYSIN2   DD   *
/FIELDS Src_0 1 4 UINTEGER BIGENDIAN, Src_1 5 4 UINTEGER BIGENDIAN
/INFILE $env:DD_INPUT2 STREAM CRLF
/KEYS Src_0 ASCENDING, Src_1 ASCENDING
/COLLATINGSEQUENCE DEFAULT EBCDIC
/OUTFILE  $env:DD__TMP2 STREAM CRLF OVERWRITE
/END
/*
//SYSIN3   DD   *
/FIELDS Src_2 1 4, Src_3 5 4
/FIELDS Src_0 3 4, Src_1 7 4
/INFILE  $env:DD__TMP1 VARIABLE 80
/JOINKEYS SORTED Src_0, Src_1
/INFILE  $env:DD__TMP2 VARIABLE 80
/JOINKEYS SORTED Src_2, Src_3
/JOIN UNPAIRED LEFTSIDE ONLY
/OUTFILE  $env:DD__TMP3 VARIABLE 152 OVERWRITE
/END
/*
//SYSIN4   DD   *
/FIELDS Src_0 3 20 CHARACTER, Src_1 68 10 ZD
/INFILE  $env:DD__TMP3 VARIABLE 152
/KEYS Src_0 ASCENDING, Src_1 ASCENDING
/COLLATINGSEQUENCE DEFAULT EBCDIC
/OUTFILE $env:DD_SORTOUT FIXED 80 OVERWRITE
/END
/*
//*       END

8.5. Possible error messages

  • JclConverter JclConvDD is not set

  • JclConverter JclConvExeCur is not set

  • JclConverter JclConvExeNew is not set

  • JclConverter JclConvPgmName not found as an EXE in this JCL

  • JclConverter JclConvSaveDir must be different from jclFilePath

  • JclConverter JclConvSaveDir is not set or empty

  • JclConverter JclConvScript is not set

  • JclConverter JclConverterPgms is not set or empty

  • JclConverter JclConverterXML file not found or error

  • JclConverter JclFilePath not set. We have to abend, submit must be called with a file parameter

  • JclConverter RestartJobID must not be used if JclConvert is used

9. JCL Error codes

Raincode JCL introduces a few Return codes, as mentioned in this section.

Table 18. System completion Error codes

Error codes

Description

RC016Error = 16

Error return code value to be used to determine if a higher return code should be set in a certain error condition.

RC020OutOfMemory = 20

The return code of the process when it is out of memory due to a REGION parameter violation.

RC08DefaultJobSkipped = 8

Default return code when ShouldRun indicates the job should not run.

RC021NStepSkipped = 21

STEP was skipped by condition

RC027NPgmNotFound = 27

PGM not found when CONTROLM is set, report error -27 and continue. Also, refer to RCS806PgmNotFound = 806

Table 19. System Abends, internal +8192, and throw messages
Error codes Description

RCS097DCBError = 97

DCB argument Error

RCS098ArgError = 98

Argument exception error

RCS099DefaultAbend = 99

The default ABEND return code when triggered by error conditions controlled by batch - not used for program ABENDs

RCS122 = 122+8192

CTRL = C abend S122

RCS801 = 801

JCL Error Dataset already exists

RCS802 = 802

Access error PDS Member

RCS806PgmNotFound = 806

PGM not found abend S806 when CONTROLM is not set issue this error and stop processing

RC875ProcError = 875

The default return code for PROC must be different when correctly set in all cases

RC876StepError = 876

The default return code for STEP must be different when correctly set in all cases

RC987JobError = 987

The default return code for JOB must be different when correctly set in all cases

RCS901JclNoStep = 901

JCL ERROR JOB NOT RUN - JCL ERROR JOB HAS NO STEPS

RCS902JclErrorDSN = 902

Invalid dataset name specified

RCS903JclErrorDISP = 903

Invalid DISP=SHR IEF286I DISP FIELD INCOMPATIBLE WITH DSNAME

RCS904SDSN_Wait = 904

SDSN_WAIT Abend Timer

RCS905NoStepExe = 905

No steps have been executed for this job

RCS906DDError = 906

Non existent GDG or PDS

RCS907DCBError = 907

Conflicting DCB Parameters

RCS949DSNMeta = 949

DSN Metadata file missing or corrupt

RCS950DSNDup = 950

DSN duplicate already exiting

RCS951JclConvert = 951

JclConverter Error see the message

RCS996Error = 996

Error in the STEP StartTime/EndTime maybe some fields are not set correctly

RCS998Error = 998

Return code for SUBMIT Return code not handled somewhere else

RCS999SubmitError = 999

Return code for Submit to return if an internal error occurs

RCS9000 = 9000

Rclrun error 9000 - 8192 = S808

RCS9080 = 9080

Abend by compiler COBOL / PLI / HLASM 9080 - 8192 = S888

RCS9081 = 9081

Command line parsing error 9081 - 8192 = S889

RCReturnCodeMask = 0x0FFF

RCUserAbendMask = 0x1000

RCSystemAbendMask = 0x2000

Part II: Development

10. Submit For Developers

Some elements of Submit are targeted towards developers, hence we present them in this section.

10.1. Environment variables defined by Submit

For each step that is executed, Submit defines several environment variables, which are available to the program that is executed during that step.

Table 20. Environment variables defined by Submit
Environment Variable Description

RC_JOB_ID

Contains the value of the option JobID. See section Command-line options for Submit.exe.

RC_STEP_ID

Current executing StepID

RC_JOB_NAME

Current executing Job Name

RC_LegacyRunner

Legacy runner path for COBOL, PL/I, Assembler DLL. The default value is $(RCDIR)/bin/rclrun.exe.

RCBATCHDIR

Path to Raincode JCL binary from where Submit.exe was started.

RC_DB_CONNECTION

Connection string defined. Utilities like IKJEFT01 may overwrite this.

RC_DB_TYPE

DB Type associated with the Connection String

RC_JOB_RESTARTED_ID

Contains the value of the option RestartJobID. See section Command-line options for Submit.exe.

…​ < From Raincode.Catalog.xml >

All Environment variables defined in Raincode.Catalog.xml match the conditions. For more details, see Environment Variables.

10.2. Debugging batch jobs

To enable the debugging of the execution of a JCL, i.e. a batch job, Submit includes the ability to increase execution trace information, as well as to use the .Net SDK debugger.

10.2.1. Increasing log information: use Trace

Increase the LogLevel to increase details in the log messages produced by the execution of the program. -LogLevel=TRACE provides very detailed information on what Submit is doing. Add -LogToConsole to output logs to the console instead of MSGLOG.txt.

10.2.2. Debug at .Net Level

If the Trace logs do not help, you may consider debugging a batch job at the .Net level. This, of course, presupposes that the full SDK is installed. Two levels of debugging are possible: the job level and the step level.

Debug Job

Running submit with the -DebugSubmit=true flag will cause execution to break before any steps are executed. An SDK debugger window will pop up and open. Select the source code you wish to debug and attach the debugger. The breakpoint will hit when the source is executed for any of the steps.

Debug Step

Running submit with the -DebugStep=STEPNAME will cause the execution to break just before the given step. An SDK debugger window will pop up and open. Select the source code for the step executable you wish to debug and attach the debugger.

10.3. Timeshift

Timeshift testing is needed to test how a job behaves in the future or the past. The Raincode legacy runtime provides a way to perform Timeshift testing by letting you modify the date and time retrieved by the job when executing.

Timeshift can be:

  • absolute by defining an initial date and time for the run or

  • relative by defining a date time offset relative to the current time.

If RC_DATETIMEOFFSET is defined (in the format of C# TimeSpan), this offset is added to DateTime.Now when the Date/Time is read by all the steps.

If RC_DATETIMEOFFSET is not defined, the runtime checks (only once) the environment variable RC_INITIALDATETIME.

Suppose RC_INITIALDATETIME is defined (in the format of C# DateTime). In that case, the runtime computes the DateTime offset between now and the provided value and sets the RC_DATETIMEOFFSET environment variable to ensure that the called program/step uses the same offset.

Submit program accepts the command line parameter -InitialDateTime to set up the Initial DateTime/offsets.

These rules and environment variables also apply to Raincode legacy stack runtime. For more details, refer to Timeshift.

11. Plugins

An important aspect of Raincode JCL is the ability to customize its workings through the creation of plugins. In this section we discuss a few plugins in detail, and list plugin interfaces to be used with Autofac.

11.1. DB connection plugin

Some use cases require sending specific commands to the DB when opening or closing a database connection. To allow for this, Raincode JCL lets you implement a C# plugin that will be called just after the call of Open function and just before the call of Close. It will be given the instance of System.Data.Common.DbConnection that represents the connection.

11.1.1. Plugin hooks

There are three hooks into the plugin:

Raincode.Batch.Utilities.Database.DatabaseConnection.BatchSQLConnectionCreate

  • Called after DbConnection creation

  • Input parameter the created DbConnection

  • Returns a (possibly updated) DbConnection

Raincode.Batch.Utilities.Database.DatabaseConnection.BatchSQLConnected

  • Called after DbConnection Open

  • Input parameter the open DbConnection

  • Returns a (possibly updated) DbConnection

Raincode.Batch.Utilities.Database.DatabaseConnection.BatchSQLDisconnect

  • Called before DbConnection Close

  • Input parameter the DbConnection to be closed

  • Returns a (possibly updated) DbConnection

11.1.2. Sample Plugin implementation

using System;
using System.Data.Common;
using RainCode.Core.Logging;
using RainCode.Core.Plugin;

[assembly: PluginProvider(typeof(Raincode.Tests.DbConnectionPlugin), nameof(Raincode.Tests.DbConnectionPlugin.Register))]
namespace Raincode.Tests
{
    public class DbConnectionPlugin
    {
        private static LogSource LogSource = new LogSource("DbConnectionPlugin");
        public static void Register()
        {
            Logger.LogInfo(LogSource, "Registering QueryTranslation");
            Raincode.Batch.Utilities.Database.DatabaseConnection.BatchSQLConnectionCreate.Provide(10, ConnectionCreate);
            Raincode.Batch.Utilities.Database.DatabaseConnection.BatchSQLConnected.Provide(10, ConnectionOpen);
            Raincode.Batch.Utilities.Database.DatabaseConnection.BatchSQLDisconnect.Provide(10, ConnectionClose);
        }

        private static DbConnection ConnectionCreate(DbConnection arg)
        {
            // manage the connection after create and before open
            Console.WriteLine("ConnectionCreate");
            return arg;
        }

        private static DbConnection ConnectionOpen(DbConnection arg)
        {
            // manage the connection after open
            Console.WriteLine("ConnectionOpen");
            return arg;
        }

        private static DbConnection ConnectionClose(DbConnection arg)
        {
            // manage the connection Before close
            Console.WriteLine("ConnectionClose");
            return arg;
        }
    }
}

11.2. Custom Plan Mapping

In some situations, the dbConnectionDataProvider file (see Map a PLAN name to a connection string) will be insufficient to map a PLAN name to the actual connection string.

To address that limitation, Raincode JCL lets you implement a C# plugin that implements a custom mapping scheme. It is called right after the resolution of the PLAN name with two parameters:

  1. The PLAN Name

  2. Then proposed connection string, as resolved by Submit.

It returns a string, which is the connection string that will be used.

Example uses for this plugin are:

  1. Implementing a custom PLAN to connection string mapping

  2. Retrieving a password and patch the connection string with the actual password to use

Sample plugin implementation

A sample for the C# plugin: DbConnectionMapperSample is available in the samples directory in the Raincode JCL installation directory. The code of this sample is as follows:

using RainCode.Core.Plugin;
using System;

[assembly: PluginProvider(typeof(DbConnectionMapperSample.DbConnectionMapperPasswordSample), nameof(DbConnectionMapperSample.DbConnectionMapperPasswordSample.Register))]
namespace DbConnectionMapperSample
{
     public class DbConnectionMapperPasswordSample
     {
         public static void Register()
         {
             Raincode.Batch.Common.DbConnectionDataProvider.ConnectionMappingPlugin.Provide(10, MapPassword);
         }

         private static string MapPassword(string plan, String proposed_connectionString)
         {
             /* Here retrieve the connection string */
             /* return null if no patch is possible                    */
             if (plan == "test")
               return  = "Your special connection string";
             else return proposed_connectionString;
         }
     }
}

11.3. Password Plugin

Another problem with the dbConnection provider file (see Map a PLAN name to a connection string) is that the file storing the connection strings will contain passwords in plaintext.

To address this limitation, Raincode JCL lets you implement a C# plugin that will be called just before executing the connection call to the database layer. Its purpose is to retrieve the actual password, and as this is called at the latest moment, the actual connection string (including the password) will not appear in any log file.

The basic usage scheme is to create a dbConnection provider file (or a custom plan mapping plugin) that stores the full connection string where the password is a pattern that is easy to match. These patterns will then be replaced at the latest moment by the actual password retrieved via your security process in the plugin.

If used please ensure to add -Plugin=<assembly> -PluginPath=<path> in your custom rclrun.args file and add these to .args files to batch utilities that would need to connect the database (like DNSUTILB, …​).

Sample Plugin implementation

A sample implementation of this plugin is as follows:

using RainCode.Core.Plugin;
using RainCodeLegacyRuntimeUtils;

[assembly: PluginProvider(typeof(OwnPasswordRewriter.OwnPasswordRewriterSample), nameof(OwnPasswordRewriter.OwnPasswordRewriterSample.Register))]
namespace OwnPasswordRewriter
{
    public class OwnPasswordRewriterSample
    {
        public static void Register()
        {
            RainCodeLegacyRuntimeUtils.PasswordMapping.PasswordRewriter.Provide(10, MapPassword);
        }

        private static string MapPassword(string connectionString)
        {
            /* Here patch the Connection String with correct password */
            /* return null if no patch is possible                    */
            string res = connectionString.Replace ("#PASSWORD#", "myPassword");
            return res;
        }
    }

}

11.4. Custom Interface Configurations

The behavior of Raincode JCL can also be customized programmatically by providing C# implementations of a number of interfaces.

The default application configuration is located in the installation directory, submit.exe.config. The JCL runtime uses Autofac to register types for the various interfaces that are supported.

11.4.1. Raincode.Batch.Runtime.IExecutionController

  • This interface is called before and after steps, and jobs are executed.

  • This interface is useful to implement if custom logging, tracing, or individual step inspection is required.

  • The submit.exe.config contains the registration for the DefaultExecutionController, which does nothing.

  • Replace the class specified in the <autofac>/<components>/<component> element for IExecutionController to register a custom class. (e.g., <component type="Raincode.Batch.Runtime.DefaultExecutionController,Raincode.Batch.Runtime" service="Raincode.Batch.Runtime.IExecutionController,Raincode.Batch.Runtime">)

11.4.2. Raincode.Batch.Jcl.Interfaces.IJclPreprocessor

  • This interface is called before and after AutoEdit variables are replaced in the input stream, and called to set/modify/update the SUBSTVAR dictionary variables.

  • The submit.exe.config contains the registration for the DefaultPreProcessor, which does nothing.

  • Replace the class specified for IJclPreprocessor to register a custom class.(e.g., <component type = "Raincode.Batch.Jcl.DefaultPreProcessor,Raincode.Batch.Jcl" service= "Raincode.Batch.Jcl.Interfaces.IJclPreprocessor,Raincode.Batch.Jcl">;)

11.4.3. Raincode.Batch.Runtime.IExecuteSecurity

  • This interface is called to determine if a user can run a job, submit a particular dataset, or submit a particular file.

  • It operates independently of standard Windows security ACLs that can be set on volumes/files.

  • The submit.exe.config contains the registration for the DefaultExecuteSecurity class, which returns true for each of the methods allowing any job or dataset to be submitted.

  • Replace the class specified for the IExecuteSecurity service in the submit.exe.config file with a custom class.(e.g., <component type="Raincode.Batch.Runtime.DefaultExecuteSecurity,Raincode.Batch.Runtime" service="Raincode.Batch.Runtime.IExecuteSecurity,Raincode.Batch.Runtime">;)

11.4.4. Raincode.Batch.Runtime.IPrintHandler

  • This interface is called to control the spooling of outputs to a printer/file/other destination.

  • This element uses the name attribute to specify the name of the output class to map to the custom print handler.

  • There are three separate print handlers that are provided, which are documented in the submit.exe.config file. (e.g., <component type="Raincode.Batch.Runtime.DefaultPrintHandler,Raincode.Batch.Runtime" service="Raincode.Batch.Runtime.IPrintHandler, Raincode.Batch.Runtime" name="A">; Registers the default print handler (will print to the default printer on the machine for output class A. )

  • When a SYSOUT item is spooled with the default print handler, it will read the ASA control character and advance the appropriate number of lines/pages before writing the output.

12. Writing utilities

Raincode provides support for developers writing JCL utilities with the following:

  1. A JCL grammar for the PEG parser generator,

  2. Two classes that implement useful functionality for the utilities

  3. Three utility implementations as samples.

The class Raincode.Batch.Utilities.Common.BaseOptions manages command-line options common to all batch utilities.

Table 21. Attributes of BaseOptions
Attributes Description

protected ArgumentSet SupportedArguments

A recognized Argument set is needed to add your arguments.

public Manager CatalogManager

Catalog Manager builds on the given - CatalogConfiguration parameter or on RC_BATCH_CATALOG

public bool ScanOnly {get; set;}
public bool ScantoDB
public string DBConnectString {get; set;}
public string DBDriver {get; private set;}

Some Raincode utilities can run in Scan mode and log information into a repository.

The class Raincode.Batch.Utilities.Common.BaseUtility is the base root class for batch utilities.

Table 22. Attributes of BaseUtility
Attributes Description

protected ActualOptions Options

Options parsed: This class contains a property CatalogManager to be used to access DSN

public static LogSource LogSource

LogSource to be used when using RccoreLib log facilities.

Table 23. Methods of BaseUtility
Method Description

BaseUtility (bool needRuntime)

The class constructor uses a boolean parameter that informs the base class if the utility will use Raincode Stack assembly. If true, path %RCDIR%/bin is added as the assembly resolution path.

protected abstract void Execute()

This is the entry point of your utility. This method is called once all parameters are parsed and CatalogManager is created. This method is abstract and must be defined by the user.

protected static void BasicMain (string[] args)

The method is to be called (most of the time trivially) by the main entry point of the executable. This method takes care of parsing parameters and calls the Execute.

The main class of a utility should look like this:

using Raincode.Batch.Utilities.Common;
using RainCode.Core.CommandLine;
using RainCode.Core.Logging;
using RainCodeLegacyBatchDataset;
using System;
using System.IO;

namespace Raincode.Batch.Utilities
{
    [RCProductGroupAttribute("Raincode.Batch.Utilities")]
    public class MyOptions : BaseOptions
    {
        public MyOptions()
        {
            // Your own options
            SupportedArguments.SupportArg(new StringArg("MyFlag")
            {
                Description = "Your own Option.",
                Action = val => { MyFlag = val; },
                Category = "General"
            });

        }
        public string MyFlag { get; set; }
    }

    public class MyUtility : BaseUtility<MyUtility, MyOptions>
    {
        // Main program entry call BasicMain unless you need special processing
        static void Main(string[] args) => BasicMain(args);

        public MyUtility() : base(needRuntime: false) { }
        protected override void Execute()
        {
            // This is the main procedure of your Utility.
            // The return code should be placed in Environment.ExitCode
            Logger.LogInfo(LogSource, "MyUtility starting");

            // Retrieve your Input in SYSIN
            Logger.LogTrace(LogSource, "Reading SYSIN");
            DataSet systsin = UtilityOptions.CatalogManager.FindKnowDDName("SYSIN");

            if (systsin == null)
            {
                throw new FileNotFoundException("Input file SYSIN not found");
            }
			//TODO add your utility code here
            throw new NotImplementedException("TODO not implemented!");
        }
    }
}

The Sample directory provides you with the implementation of three utilities you may use as a template for your utilities.

13. Samples

13.1. Creating and running a Job from C#

The following sample describes how to create a job from C# code and execute it using the SubmitRunner class from the Raincode.Batch.Submit assembly.

13.1.1. Job Setup and Execution

The first step in creating a job is to allocate a Catalog Manager object. This is used to generate the unique Job ID and is used throughout the execution of the job to query and manage datasets. Next, the job object is built up, and finally the SubmitRunner.Execute() method is called to execute the job.

Below is a sample job being constructed:

var catMan = new Manager();

Job job = new Job("MYJOB",catMan,catMan.AllocateJob());
var step1 = ProgramStep.CreateProgramStep("STEP1","IEFBR14",job.ID);
job.AddStep(step1);
step1.AddDD(new DD()
{
 RecordFormat = RecordFormat.FB,
 DSN = "TEST.FILE",
 Disposition = new Disposition()
    {
		Mode = DispositionStatus.MOD,
		Normal = DispositionType.CATLG,
		Abnormal = DispositionType.DELETE
	},
RecordLength = 80,
Volume = "DEFAULT"
});

var options = new Submit.Options();
var ret = Submit.SubmitRunner.Execute(new Submit.Options(), job);

It is important to remember that the step is added to the job object by calling AddStep. This is so the step can be configured to run in the job correctly. Also, note that AddDD is called on the step to add a DD object instead of adding directly to the collection. This is required so the DD can be properly configured for the step.

13.2. Serializing a Job for Remote Execution

To serialize a job for remote execution, call the Job.SerializeJob() method passing in a constructed Job object that has not been executed yet. It will return a JSON encoded string that can be sent to another machine (via web service, file, or other IPC mechanism). Deserialization is handled by Job.DeserializeJob(), however additional setup is required to make it ready to execute.

The code below shows an example:

// Job is created from C# or from parsed JCL

var serialized = Job.SerializeJob(job);

// The serialized variable now contains the string representing the job

//On the other machine/instance:

var deserialized = Job.DeserializeJob(serialized);

var catMan = new Manager();

jobId = catMan.AllocateJob();

deserialized.ResetState(catMan,jobId);

Submit.SubmitRunner.Execute(new Submit.Options(), deserialized);

13.3. Customizing Submit

The submit process can be customized by implementing a console application that creates an instance of the Options() class to parse the command-line parameters, then call SubmitRunner.Execute() and return the return code.

Below is a sample implementation:

namespace Raincode.Batch.Submit
{
  internal class Program
    {
        private static int Main(string[] args)
        {
            var opts = new Options();
			opts.Parse(args);
            return SubmitRunner.Execute(opts);
		}

    }
}

13.4. Customizing Logging

The batch runtime process uses the RainCode.Core.Logging.Logger class to route log messages to the appropriate source. By default, a logger is registered that outputs messages to the default Trace listener. Additionally, the -LogToConsole option for submit will register a ConsoleLogger that will output log events to the console. Output can be captured by adding a .NET configuration section to the submit.exe.config (or appropriate application configuration file), such as:

<system.diagnostics>
    <trace autoflush="true" indentsize="4">
      <listeners>
     <add name="myListener" type="System.Diagnostics.TextWriterTraceListener" initializeData="TextWriterOutput.log"/>
				<remove name="Default"/>
			</listeners>
	</trace>
</system.diagnostics>

Alternatively, a custom logging class can be registered in a custom Submit executable. For instance, this implementation of Submit.exe adds a ConsoleLogger source:

namespace Raincode.Batch.Submit
{
  internal class Program
  {
	private static int Main(string[] args)
	 {
		RainCode.Core.Logging.Logger.CreateAndRegister<RainCode.Core.Logging.ConsoleLogger>();
		  var opts = new Options();
		  opts.Parse(args);
		  return SubmitRunner.Execute(opts);
	 }
   }

}

To control the log level that is set, refer to the -LogLevel command-line option.

13.5. JCL Containerization

In the sample RaincodeJCLWrapper, we will see the steps to containerize a JCL. Here, we create a REST API which accepts JobName, JobId, etc., as input parameters and returns the JobId of the submitted job as an output.

apicontroller
Figure 18. ApiController

As shown below, we define jclOptions, the path for the file storage, the path for the catalog manager and jobid. In the later part, we are calling the submitRunner.Execute with the jclOptions to run a JCL.

invokesubmitrunner
Figure 19. Invoke Submit Runner

For more details, refer to Submit.

13.5.1. Prerequisites

  • Raincode JCL compiler image

  • Raincode JCL runner image

  • Raincode license - Must be added to the RaincodeJCLWrapper project.

  • Docker Desktop

For more details on loading the Raincode JCL compiler and JCL runner image, refer to load the desired image.

13.5.2. Build Docker Image

The screenshot below is a dockerfile that creates an image to containerize the JCL runner.

dockerfilescreenshot
Figure 20. A docker file

Through the Visual Studio solution, open the developer command prompt and navigate to the location of the dockerfile and issue the command docker build, as shown in the screenshot below:

dockerbuild
Figure 21. Docker Build

Once the build is successful, it will generate an image. The image can be seen through the command prompt, as shown in the screenshot below.

images
Figure 22. Images
Make sure that the docker desktop is running.

13.5.3. Run Docker container

The docker run command runs a container from the image and runs on the port specified (8080).

The file volume mount option lets us mount a local machine volume to a particular path inside the container so that the container can access the file volume.

As the screenshot below shows, we have successfully mounted the FileStorage volume from the local system to the container’s app/FileStorage location.

Mountvolume
Figure 23. Container running on port 8080 - Command Prompt
joblogonthecontainer
Figure 24. Container running on port 8080 - Docker Desktop

After the file mount, we let the container access the file from your local machine. The container can access the Catalog, Jcls and Program executables.

contentsofthecontainer2
Figure 25. Contents of the container

Below is the sample Raincode.Catalog.xml that describes the configuration for the SUBMIT runner. For more details, refer to Catalog Configuration.

raincodecatalogxml
Figure 26. Raincode Catalog XML

13.5.4. Run a sample JCL

The above running docker container is called to execute JCLHELLO.JCL, which invokes HELLO.cob to print a SIMPLE HELLO WORLD message.

JCL
Figure 27. A JCL file
COBOL
Figure 28. A COBOL file

Since we are running the jclwrapper container on port 8080, we must invoke this API using an HTTP client such as postman. The screenshot below shows that the request has been sent to execute JCLHELLO.JCL and received job id JOB0000000001 as a response.

JCLincontainer
Figure 29. JCL in postman

We can also see the job log from the container.

jobloginterminal
Figure 30. Joblog of the terminal

You can examine the output on the local system from the folder SYSOUT. You can check MSGLOG.txt for the details.

joblogonthelocalsystem
Figure 31. Joblog on the local system

If we look at the app/Filestorage location inside the container, we see the necessary folder created by the SUBMIT runner and the job log for the JCL, as shown in the screenshot below.

contentsofthecontainer1
Figure 32. Content of the container

Appendix A: JCL JOB artifacts repository

RC_JCL_JOB Table

The RC_JCL_JOB table holds information about the submitted JCL job.

Table 24. Artifacts Repository database RC_JCL_JOB
Column Description

RC_ID

The record identifier for JCL job. RC_ID is the key of this record

RC_JOB_NAME

Name of submitted JCL job

RC_JOB_PATH

JCL job path

RC_JOB_COND

JCL job conditions. For example:(0,GT)

RC_JOB_TIME

Time when the job was submitted

RC_JCL_PGM_STEP Table

The RC_JCL_PGM_STEP table holds information about the program steps of submitted jobs.

Table 25. Artifacts Repository database RC_JCL_PGM_STEP
Column Description

RC_ID

Foreign key to entry in RC_JCL_JOB. Identifies the program on which these data apply

RC_SEQ

The couple RC_ID, RC_SEQ serves as the primary key for this record

RC_STEP_NAME

Name of JCL job step

RC_PROG_NAME

Name of the program to be executed in JCL step

RC_PROG_PARAMS

Program parameters

RC_PROG_COND

Program conditions

RC_PROC_RUNNER

The utility used to run the program. For example, IKJEFT, DFSRRC00.

RC_JCL_PROC_STEP Table

The RC_JCL_PROC_STEP table holds information about the procedure steps of submitted jobs.

Table 26. Artifacts Repository database-RC_JCL_PROC_STEP
Column Description

RC_ID

Foreign key to entry in RC_JCL_JOB. Identifies the program on which these data apply

RC_SEQ

The couple RC_ID, RC_SEQ serves as the primary key for this record

RC_STEP_NAME

Name of JCL job step

RC_PROC_NAME

Name of the procedure to be executed in JCL step

RC_PROC_PARAMS

Procedure parameters

RC_PROC_COND

Procedure conditions

RC_PROC_INPUT_CARDS

Procedure source code

RC_JCL_DD Table

The RC_JCL_DD table holds information about data definitions of submitted jobs.

Table 27. Artifacts Repository database RC_JCL_DD
Column Description

RC_ID

Foreign key to entry in RC_JCL_JOB. Identifies the program on which these data apply

RC_SEQ

The couple RC_ID, RC_SEQ serves as the primary key for this record

RC_DD_NAME

Data definition name

RC_DSN_NAME

Dataset name

RC_IS_DUMMY

Indication if DD is used as DUMMY dataset

RC_INSTREAM_DATA

Instream data content

RC_PROG_STEP_REF

DD reference to parent program step RC_SEQ number (-1 means no reference)

RC_PROC_STEP_REF

DD reference to parent procedure step RC_SEQ number (-1 means no reference)

RC_JCL_ERR Table

The RC_JCL_ERR table holds information about errors of submitted jobs.

Table 28. Artifacts Repository database RC_JCL_ERR
RC_ID Foreing key to entry in RC_JCL_JOB. Identifies the program on which these data apply

RC_SEQ

The couple RC_ID, RC_SEQ serves as the primary key for this record

RC_ERR_TYPE

JCL error type

RC_ERR_MSG

Error message including the cause of an error

RC_ERR_LINE

The line number where the error occurs

Currently, only parsing errors are supported.

RC_JCL_VAR Table

The RC_JCL_VAR table holds information about the variables used in the submitted jobs.

Table 29. Artifacts Repository database RC_JCL_VAR
Column Description

RC_ID

Foreign key to entry in RC_JCL_JOB. Identifies the program on which these data apply

RC_SEQ

The couple RC_ID and RC_SEQ serves as the primary key for this record

RC_VAR_NAME

The name of the variable

RC_VAR_FOUND

This column contains only 1 or 0. 1 if the value of the variable is known, 0 otherwise

RC_VAR_VALUE

The value of the variable. If the variable is not found, the value is the name of the variable prefixed by v

RC_VAR_CONTROLM

1 if it’s a Control-M variable

RC_UTILITIES_PARSING_STATUS Table

The RC_UTILITIES_PARSING_STATUS table holds information about the SYSIN parsing done by the utilities (SORT, IKJEFT, …​)

Table 30. Artifacts Repository database RC_UTILITIES_PARSING_STATUS
Column Description

RC_ID

Foreign key to entry in RC_JCL_JOB. Identifies the program on which these data apply

RC_SEQ

The couple RC_ID and RC_SEQ serves as the primary key for this record

RC_STEP_NAME

The name of the step

RC_UTILITY_NAME

The name of the utility

RC_COMMAND_TEXT

The command sent to the utility (usually the SYSIN)

RC_STATUS_TEXT

The error message, if any

RC_COLUMN_NUMBER

The column number where the error occurs

RC_COMMAND_PARSE_STATUS

1 if there is no parsing error, 0 otherwise

Appendix B: Lists of tables and figures in document

This appendix includes the automated lists of the tables and figures available in this document; placed in this location, to allow you to gain access to core information regarding Raincode JCL more readily up front, and as an aide-memoire.