Hybrid Scripting: Running Python from JCL and JCL from Python on z/OS

Mainframes (specifically those running the z/OS operating system) are renowned for their robust job scheduling capabilities, often driven by Job Control Language (JCL). On the other hand, Python is a modern, versatile language, increasingly popular in data science and automation. Interestingly, these two worlds can be bridged, allowing seamless interaction between Python and JCL on z/OS.

Submitting JCL from Python

If you have the zoautil_py package installed within your z/OS Unix System Services (USS) environment, submitting JCL from Python becomes surprisingly easy. Let's look at an example:

from zoautil_py import jobs, datasets

jcl_template = """
//Z35240T JOB (ACCT#1) 'JCL FROM PYTHON',
//     CLASS=X,
//     MSGCLASS=X
//* 
//STEP1     EXEC PGM=SORT
//SYSPRINT  DD  SYSOUT=* 
//SYSOUT    DD  SYSOUT=*
//SORTIN    DD PATH='/z/z35240/files/mydatafile.dat'
//SORTOUT   DD DSN=Z35240.SORTOUT1,
//          DISP=(NEW,CATLG),
//          SPACE=(TRK,(10,10)),
//          DCB=(RECFM=FB,LREC=80,BLKSIZE=800),
//          UNIT=3390
//*
//SYSIN     DD *
 SORT FIELDS=COPY
 OUTREC FIELDS=(1,50,,30X)
/*
//* 
"""

mypds = datasets.hlq() + "TEMP.JCLS"
this_job = mypds + "(TESTJOB)"
datasets.write(dataset=this_job, content=jcl_template)
job_info = jobs.submit(this_job)

print(f"JobId : {job_info.id}")
print(f"Job Name : {job_info.name}")
print(f"Job Owner : {job_info.owner}")

job_info.wait()
job_info.refresh()
print(f"Job Status : {job_info.status}")
print(f"Job RC : {job_info.rc}")        

In this code, we demonstrate how to:

• Construct a JCL template in Python: Using f-strings for convenient formatting.
• Create a dataset: Storing the JCL with the datasets module.
• Submit and Execute the JCL: Using the jobs module, we get a job object back to monitor its status.

Power of 'zoautil_py': This package empowers us to interact with z/OS resources—datasets, jobs, and more—programmatically from Python, simplifying management.

Running Python from JCL

Let's flip the script! Here are a few techniques to run Python directly from JCL:

1. BPXBATCH: Use the BPXBATCH program to directly execute a shell script (.sh). This shell script can contain your command to run python - python /z/z35240/files/python_script.py

//STEP1???EXEC PGM=BPXBATCH
//STDERR??DD?SYSOUT=*?
//STDOUT??DD?SYSOUT=*
//STDPARM??DD?*
sh /z/z35240/files/runpyfromjcl.sh
/*        

2. Direct Echo and Pipe: Here we just echo the python script file to the USS shell to execute.

//STEP1   EXEC PGM=BPXBATCH
//STDERR  DD SYSOUT=* 
//STDOUT  DD SYSOUT=*
//STDPARM  DD *
sh echo "python /z/z35240/files/python_script.py" | sh
/*        

3. Inline Commands ('dcat'): Create a PDS member and put your call to python script as we usually do in terminal. This is a slight variation from our method 1. We just place the script call in a PDS member and use the dcat command to fetch this member and pipe its contents to a shell for further execution.

//STEP1???EXEC PGM=BPXBATCH
//STDERR??DD?SYSOUT=*?
//STDOUT??DD?SYSOUT=*
//STDPARM??DD?*
SH dcat "Z35240.SOURCE.SCRIPTS(CALLPY)" | sh 
/*        

4. Direct Echo and Pipe: This gets interesting now. Ditch those USS files altogether. Create a PDS member containing your Python script. Use the dcat command to fetch this member and pipe its contents to Python for execution.

//STEP1???EXEC PGM=BPXBATCH
//STDERR??DD?SYSOUT=*?
//STDOUT??DD?SYSOUT=*
//STDPARM??DD?*
SH dcat "Z35240.SOURCE.SCRIPTS(PYSCRPT)" | python 
/*        

In this case "Z35240.SOURCE.SCRIPTS(PYSCRPT)" has my full python script as sample below.

from datetime import datetime
print(f'Test Script Runtime : {datetime.now()}')        

Real-World Applications

Why integrate Python and JCL? Here are a few compelling use cases:

• Automation & Orchestration: Python scripts can submit, monitor, and control JCL jobs, streamlining complex workflows.
• Modernizing Legacy Systems: Introduce Python for data analysis or web interactions within established JCL jobs.
• Data Pipelining: Python excels at data manipulation; use it to preprocess or postprocess data managed through JCL.

Further Exploration

I encourage the community to do further experiments! Remember, running Python on z/OS provides access to a rich array of Python libraries. Consider more complex scenarios of interactions between JCL and Python to fully unlock the potential of this hybrid approach!

Let me know your comments!

References:

1. https://github.com/IBM/zoau-samples/blob/main/samples/jobs.py
2. https://www.ibm.com/docs/en/zoau/1.2.x