z/VM Pipeline (Filter)
Eduardo M. Crivelaro
Technology Risk Management Specialist | Mainframe Security Expert | Cybersecurity | IAM Driving Innovation, Enhancing Compliance, and Safeguarding Critical Systems
By now, you should have a fairly good idea about some of the?device drivers?provided by?CMS Pipelines.?We will now study some?filter?stages in more detail.?We'll develop a couple of small applications and explain the possibilities of the filters as they come along.
CMS Pipelines?has many filter stage commands.?A filter reads data from its input stream, does some work using that data, and writes the results to its output stream.?The difference between a filter and a device driver is that a filter does not interact with devices or other system resources, as device drivers do.
The results that filters write can be far different from the data they read.
The?CHANGE?stage command, for example, writes a record to its output stream for every record it reads.??LOCATE, on the other hand, reads all the records, but writes only those that match a search string.??COUNT?reads every record in its input stream, but writes only a single record to its output stream.?By stringing filters together, you can transform raw data into useful results.
This chapter describes some commonly used filters?:
Some of the filters are similar in function and format to XEDIT subcommands.?These similarities are intentional, and are intended to help you learn?CMS Pipelines?quickly.
The CP command?QUERY NAMES?displays information about users logged on to your system.?CP normally writes the response to your terminal screen.?But, the response can also be captured and processed by a pipeline.?It will allow us to "enhance" the possibilities of the command and to produce nicer output where possible.?This application serves mainly education purposes, and is initially not necessarily of much practical use.
Our very first step is to get the CP response in the pipeline. This goes via the?CP?device driver that we already encountered in previous chapter?:
pipe cp query names | console
MVS55 - DSC , MVS56 - DSC , MVS54 - DSC , MAINTJDV - 006C
TOOLS - DSC , BIBLIO - DSC , DISKACNT - DSC , EREP - DSC
LANPRINT - DSC , PORTMAP - DSC , RTMESA - DSC , FTPSERVE -L0000
SQLMACH - DSC , VTAM2 - DSC , IECCNM - DSC , OPVM1 - 0065
FSCMVS - 0063, FSCIPOE - DSC , VMBARS - DSC , CALLUP - DSC
OPERATOR - 00ED, RUNTOOLS - DSC
VSM - VTAM
OP1 -PBRS0525, DECEULAE-FBIS8C53, MAINTIP -AB992611, BUELENSC-FBWS8J00
ACFMVS2 -ABBNAV08, ACFMVS5 -ABBNAV06, ACFMVS1 -ABBNAV07, ATESNA -ABBNAV05
ARTS38 -AB$2003L, MAINTP -ABBNAV16, MAINTCD -FBWSXM02, NL03710 -FNKSTT5
VSM - TCPIP
The output is sent to the screen via our well known?CONSOLE?device driver.
You may know that the?CP?stage must allocate a buffer to accept the results of CP (the same is true for?EXECIO?and the REXX?DIAG()?function).?The default size of the buffer is 8K.?This size may not be enough to accommodate the complete answer, especially for some QUERY commands.?For example, around 103 lines of CP Q RDR ALL can fit in the 8K buffer.?Since VM/ESA Release 2.1?CMS Pipelines?will detect the buffer overflow condition, allocate a larger buffer and retry the query.?If however you can know the size of the answer, you should specify the size of the buffer to the?CP?stage to avoid this overhead.?For example, this solution calculates the size of the buffer?:
/* Issue CP stage with calculated buffer */
address command
parse value diag(8,'QUERY USERS') with nlogon .
buffersize=format((nlogon/4 +5)*80,,0)
'PIPE CP 'buffersize' QUERY NAMES ! ...'
While we produce this text, it is easy to eliminate a series of records via XEDIT to limit the number of lines needed for the example above, but when you're developing your pipelines, it may be nice to limit the number of records handled by the pipeline so that it becomes surveyable what happens to the pipeline.
TAKE?and?DROP?make it easy to select records based on their positions in the input stream.
The?TAKE?filter picks the first or last n lines (where n is zero or positive) for output to the pipeline.?By default,?TAKE?selects the first records (not the last).?Next figure demonstrates taking the first 3 records.
pipe cp query names | take 3 | console
MVS55 - DSC , MVS56 - DSC , MVS54 - DSC , MAINTJDV - 006C
TOOLS - DSC , BIBLIO - DSC , DISKACNT - DSC , EREP - DSC
LANPRINT - DSC , PORTMAP - DSC , RTMESA - DSC , FTPSERVE -L0000
Ready;
To take the last records, specify the?LAST?operand on?TAKE.?This example shows how to take the last 4 records.
pipe cp query names | take last 4 | console
OP1 -PBRS0525, DECEULAE-FBIS8C53, MAINTIP -AB992611, BUELENSC-FBWS8J00
ACFMVS2 -ABBNAV08, ACFMVS5 -ABBNAV06, ACFMVS1 -ABBNAV07, ATESNA -ABBNAV05
ARTS38 -AB$2003L, MAINTP -ABBNAV16, MAINTCD -FBWSXM02, NL03710 -FNKSTT5
VSM - TCPIP
Ready;
The?DROP?stage command is the converse of?TAKE. It lets you delete the first or last n lines (where n is 0 or a positive number).
The pipe in next figure drops the first six records from the output?:
pipe cp query names | drop 6 | console
VSM - VTAM
OP1 -PBRS0525, DECEULAE-FBIS8C53, MAINTIP -AB992611, BUELENSC-FBWS8J00
ACFMVS2 -ABBNAV08, ACFMVS5 -ABBNAV06, ACFMVS1 -ABBNAV07, ATESNA -ABBNAV05
ARTS38 -AB$2003L, MAINTP -ABBNAV16, MAINTCD -FBWSXM02, NL03710 -FNKSTT5
VSM - TCPIP
Ready;
To drop the last records in the pipeline, use the?LAST?operand as shown in next example?:
pipe cp query names | drop last 6 | console
MVS55 - DSC , MVS56 - DSC , MVS54 - DSC , MAINTJDV - 006C
TOOLS - DSC , BIBLIO - DSC , DISKACNT - DSC , EREP - DSC
LANPRINT - DSC , PORTMAP - DSC , RTMESA - DSC , FTPSERVE -L0000
SQLMACH - DSC , VTAM2 - DSC , IECCNM - DSC , OPVM1 - 0065
FSCMVS - 0063, FSCIPOE - DSC , VMBARS - DSC , CALLUP - DSC
Ready;
By combining?TAKE?and?DROP, you can get records in the middle.
We've already mentioned that sometimes the records have to be buffered in storage to be able to process them by the pipeline.??TAKE LAST?and?DROP LAST?are examples of that.?Indeed,?CMS Pipelines?doesn't know how many records will flow through the pipeline, and so can't know how to take the last 4 records, unless it first gathers all records till the end of the file is reached.
As CP replies with 1 to 4 users per output line, it is difficult to process these records, for example to extract only those users that are actually connected to a terminal and to exclude those that run disconnected.?We therefore need a way to split the records so that each user is on a separate line.
The?SPLIT?filter is exactly what we need.?And while we're discussing it, we may as well discuss its opposite filter,?JOIN.
By default,?SPLIT?creates an output record for each blank-delimited word in its input records?; here is an example.
pipe literal A phrase with five words | split | console
A
phrase
with
five
words
Ready;
SPLIT?has operands to define other kinds of splitting.?For our application, we are not interested in splitting at blank-delimited words, but at the separation between the userids, hence at the comma sign.?This is how we can do it?:
pipe CP query names | take 2 | split , | console
MVS55 - DSC
MVS56 - DSC
MVS54 - DSC
MAINTJDV - 006C
TOOLS - DSC
BIBLIO - DSC
DISKACNT - DSC
EREP - DSC
You see that the TAKE 2 comes in handy here to limit the output while testing.?We could have done it the other way round, first split and then limit the output via a?TAKE?command.?There is however a subtle difference that may adversely impact the performance of the pipeline.?Indeed, coding the?TAKE?after the?SPLIT?will make?SPLIT?process?all records.?If the input is huge, it results in unnecessary overhead.?This is yet another example of?pipe-thinking.
As you can see, the output is not properly aligned.?This is because there is an extra space after the comma signs.?This space becomes part of the split records.?For the first user-ids, there is however no leading space.?We'll handle this problem after we're done with explaining the?JOIN?stage.
Note:?We don't cover all possibilities of all stages in this lesson. We could for example have used?SPLIT AFTER STRING /,?/?to avoid the alignment problem.?Try this out at the terminal... You could also have a look at the HELP for?SPLIT?to learn about other possibilities.
JOIN?creates a single record from one or more input records.?By default,?JOIN?puts together pairs of input records, as shown in next figure?:
pipe literal A phrase with five words | split | join | console
Aphrase
withfive
words
Ready;
LITERAL?puts a record into the pipeline.??SPLIT?puts each of the five words on a separate record.?Then?JOIN?combines pairs of records.?Because there weren't an even number of input records,?JOIN?puts the last word by itself in an output record.
Notice that?JOIN?puts records together without intervening blanks.?To put a blank between the joined records, specify /?/ as shown in this example?:
pipe literal A phrase with five words | split | join / / | console
A phrase
with five
words
Ready;
Between the delimiters (/) you can put any string you want to insert between the joined records, not only blanks.?And, as usual, you can use any character not in the string itself as the delimiter.
You can also join more than two records with?JOIN?by telling it the number of records to put together.?Put the number after the?JOIN?keyword before any delimited string.??The number itself is the number of records to be joined to the first one.?If you want to join every three records, specify 2 not 3.?Next figure shows how to join every three input records into one output record.
pipe literal A phrase with five words | split | join 2 / / | console
A phrase with
five words
Ready;
To put all input records in a single output record, specify an asterisk (*) instead of a number.
?2Do you have any idea of the danger of using?JOIN *?when the number of records is very large??
Remember, our output was not nicely aligned after the?SPLIT?stage.?We can however use the?STRIP?stage command to remove characters from the beginning and the end of records.?The figure shows a simple?STRIP?example.
pipe CP query names | split , | strip | take 5 | console
MVS55 - DSC
MVS56 - DSC
MVS54 - DSC
MAINTJDV - 006C
TOOLS - DSC
By default,?STRIP?removes both leading and trailing blanks, but you can add the?LEADING?or?TRAILING?options to remove only leading or only trailing blanks.?This is very similar to the REXX?STRIP()?function.
You can also use?STRIP?to remove characters other than blank.?Use the?STRING?operand to identify the string to be stripped.
pipe literal 0000120| strip leading string /0/ | console
120
Ready;
pipe literal bread meat bread| strip string /bread/ | console
meat
Ready;
pipe literal bread meat bread lettuce bread| strip string /bread/ | console
meat bread lettuce
Ready;
Now that we have our users nicely separated and aligned, we can go a step further and select only specific users, for example those that are connected to a terminal address.
Several stage commands select pipeline records.?That is, they read all records in the pipeline, but write only those that meet some selection criteria.?These filters will probably be the ones you'll use the most when writing pipelines. This section describes filters that select records based on the content of the record itself.?Those filters include?:
LOCATE?and?NLOCATE
FIND?and?NFIND
TOLABEL?and?FRLABEL
There are a lot more filters that let you select records by content.?We'll probably encounter them later when there is a specific need.
The filters we'll study here are case sensitive.?This means that only those records that match exactly the string(s) you're locating will be selected.?The words "Apple" and "apple" are thus not considered equal.
If you want case insensitive filtering, then you can use the?CASEI?filter, or use techniques based on?XLATE?(see later), to uppercase (parts of) the records.
Since VM/ESA Version 2, many selection filters are enhanced and now carry a?ANYCASE?option.?This is better performing than the?CASEI.
The?LOCATE?stage command selects records having a particular string?;?NLOCATE?selects records?not?containing a particular string.?By default, both filters look everywhere in the record for the string you supply.?You can specify column ranges to limit the scope of the search.
LOCATE stage command
The?LOCATE?stage command writes only the records containing a specific string.?If we want to select only those users that are disconnected (they are indicated by the DSC characters), then we would code?:
pipe cp query names | take 2 | split , | strip | locate /- DSC/ | console
MVS55 - DSC
MVS56 - DSC
MVS54 - DSC
TOOLS - DSC
BIBLIO - DSC
DISKACNT - DSC
EREP - DSC
Ready;
If the record contains the string,?LOCATE?writes it to its output stream.?If it does not,?LOCATE?discards it.
Note that we located the string?/- DSC/?to avoid finding a user like?LANDSCAP
What would happen if you looked for?/- DISC/?instead of?/- DSC/??
pipe cp q n|take 2|split ,|strip|locate /- DISC/|cons
Ready;
No record in the output from CP matches the search string, so?LOCATE?does not write anything to its output stream.?Consequently,?CONSOLE?has nothing to display.?It is perfectly acceptable for?LOCATE?(or any other filter that selects records) to find no matching record.??CONSOLE?does not give an error when there aren't any records in its input stream.?The pipeline remains intact?; no error has occurred.
In the previous examples of?LOCATE?slashes (/) are used to delimit the string you are searching for.?You can use any character to delimit the string as long as it does not appear in the string itself.?
To select records containing?x/y?for example, you might use commas as delimiters, as shown in next pipeline?:
pipe literal z=x/y | locate ,x/y, | console
z=x/y
Ready;
When entering a PIPE command next to a file in the list displayed by commands like FILELIST and RDRLIST, do not use the slash (/) as the delimiter.?The slash has a special meaning to these filelist environments.
Using column ranges on LOCATE
We selected the disconnected users by searching for the string?/- DSC/.?This way we are sure to search for the disconnected users because it is impossible to have a userid name containing a space and a dash.?But, there may be cases where we have to limit the scan to specific columns (or zones).?In that case, it is possible to limit the range of columns inspected by?LOCATE.?Simply enter a column range before the string operand.?The first example displays records with the letter o in column 2.?The second example looks for o anywhere in columns 2 through 3.
pipe literal shoe|literal sock| locate 2 /o/ | console
sock
Ready;
pipe literal shoe|literal sock| locate 2-3 /o/ | console
sock
shoe
Ready;
Notice in the second PIPE command example above that?sock?is displayed before?shoe.?Given the order of the?LITERAL?stages in the pipeline, this is not what you might expect.
When?LITERAL?is first in a pipeline, it simply writes a record to its output stream.?That record contains whatever you type as the?LITERAL?operand.?When?LITERAL?is not first, it writes a record containing the operand,?and then copies any records in its input stream to its output stream.?This is a very important characteristic of?LITERAL, and one that is easily forgotten.??Remember:?LITERAL?writes before copying.
In the example, the first?LITERAL?stage writes a record containing?shoe?to its output stream.?The second?LITERAL?stage writes a record containing its operand (sock) to its output stream, and then copies any records in its input stream to its output stream.?So the?sock?record travels through the pipeline before the?shoe?record.
To be even more precise, the?LITERAL?stage dissappears from the pipeline once it has written his record to its output stream.?It becomes as if the stage is no longer there.?The dispatcher will no longer dispatch it.?It is said the stage?shorts?itself (think about a short-circuit).
Several filters let you specify column ranges.?You can specify a single column or a range of columns.?When specifying a range of columns, use a dash (-) to identify the range.?For example, to identify a column range from 1 through 10 inclusive, specify?1-10?as range.?This is?from-to?notation.
You can also specify a column range as a starting column followed by the number of columns in the range.?Separate these numbers with a period (.).??22.5?indicates a range of 5 columns, beginning at column 22. This is the?from-for?notation. Do not put a blank between the dash or period and the number.
To specify multiple column ranges, separate each specification with one or more blanks.?Multiple column ranges must always be enclosed by parentheses.??(2-10 14 72.7)?identifies three column ranges?: columns 2-10, column 14, and columns 72-78.
An asterisk (*) in the second position of a column range means end of record.?So a column range of?3-*?means from column 3 until the end of record.
The?LOCATE?stage command does?not?let you specify multiple strings like you can do with XEDIT's?LOCATE?command.?Instead, use multiple?LOCATE?stage commands.?For example, to verify if user TOOLS is disconnected, we could enter?:
pipe cp query names!split ,!strip! locate /- DSC/ ! locate /TOOLS/ !console
TOOLS - DSC
RUNTOOLS - DSC
Ready;
This is not totally the answer we would expect.?We get 2 records, as the word TOOLS appears in 2 userids.?We could have searched for?/TOOLS???/?to be sure, but we'll learn about a better approach a bit later.
If you want to verify if either?DECEULAE?or?BUELENSC?are logged on, then you'll need to use a?multistream pipeline?which we will study in a later lesson.
VM/ESA Release 2.1 added the?ALL?filter which is very similar to XEDIT's?ALL?command and allows to locate multiple strings.?For example, to find both users, the pipeline becomes?:
pipe cp query names ! ... ! ALL /DECEULAE/ !! /BUELENSC/ / ...
Remark the double vertical bar between the 2 names to indicate the OR character.?This is needed to avoid that?CMS Pipelines?interprets this as a stage separator.?
We've already mentioned the possibility to change the stage separator character via the?STAGESEP?option of the?PIPE?command.?
Since VM/ESA Release 2.1 there is however the possibility to use?self-escaping?by specifying two adjacent stage separators, or to define explicitly an escape character.?It tells the?CMS Pipelines?parser to ignore the special signification of the character that follows.?
So, to display all lines of the INPUT FILE that contain the character string A|B, you can use?:
pipe (escape %) < INPUT FILE A ! locate /A%!B/ ! console
or
pipe < INPUT FILE A ! locate /A!!B/ ! console
Selecting records by length
You can select records that have some minimum length by using?LOCATE?without a string.?Instead, just specify a column.?Next figure selects records not shorter than 2 characters.
pipe literal a! locate 2 ! console
Ready;
pipe literal ab! locate 2 ! console
ab
Ready;
Be careful about trailing blanks when you use?LITERAL.?Notice that a stage separator immediately follows the strings in above example?- there are no trailing blanks.?If you put a blank between the character?a?and the stage separator,?LITERAL?puts the trailing blank in the output.?The blank is counted as a character, which means the entire string is 2 characters long.??LOCATE?would then select the string.
NLOCATE stage command
NLOCATE?does the opposite of the?LOCATE?stage command.?It writes all records that do not contain the string specified as the operand.
Until now we searched for the disconnected users, but you'll probably be more interested in the really connected users.?NLOCATE, as it is the opposite to?LOCATE?is the only thing we have to change in our pipe to have the output we want.
pipe CP query names!split ,!strip! nlocate /- DSC/ !console
MAINTJDV - 006C
FTPSERVE -L0000
OPVM1 - 0065
FSCMVS - 0063
OPERATOR - 00ED
VSM - VTAM
OP1 -PBRS0525
DECEULAE-FBIS8C53
MAINTIP -AB992611
BUELENSC-FBWS8J00
ACFMVS2 -ABBNAV08
ACFMVS5 -ABBNAV06
ACFMVS1 -ABBNAV07
ATESNA -ABBNAV05
ARTS38 -AB$2003L
MAINTP -ABBNAV16
MAINTCD -FBWSXM02
NL03710 -FNKSTT5
VSM - TCPIP
We deliberately didn't limit the output to show you that we still have a few problems to solve to produce a nice output.?There are 2 VTAM Service Machines (VTAM and TCPIP(2)) and the LU-names are differently aligned from the local or logical terminals.?We'll learn other stages to handle these conditions, but for now, let's continue the discussion for?LOCATE?and Co..
You can select short records with?NLOCATE.?The next figure shows how to select all records having a length less than 5.
pipe literal too long! literal okay! nlocate 5 ! console
okay
Ready;
The record okay is selected because it has fewer than 5 characters.
NLOCATE?(and?NFIND?which we will see later) are especially useful to eliminate unneeded records from the input before doing the real processing.?Examples of records that one probably wants to eliminate are comment records.?
By eliminating unneeded records early in the pipeline, you also improve the performance of your pipeline as later stages have to process less records.?It is good practice to apply some?pipe-thinking?before coding the pipeline to improve its performance.
CMS Pipelines?includes stage commands that look only at the beginning of a record.?This section describes four of them?:?FIND,?NFIND,?TOLABEL, and?FRLABEL.
FIND and NFIND stage commands
These stage commands select records according to whether the leading characters match the argument string.
A while back, we wanted to verify if user TOOLS was disconnected and we got 2 output records.??FIND?will help us to solve that little problem?:
pipe cp query names!split ,!strip!find tools!console
TOOLS - DSC
Ready;
Remark that?FIND?does not use delimited strings.
But, we can still have problems if we don't pay attention.?If we would look for user MAINT instead of TOOLS, then we would get this?:
pipe cp query names!split ,!strip!find MAINT!console
MAINTJDV - 006C
MAINTIP -AB992611
MAINTCD -FBWSXM02
Ready;
pipe cp query names!split ,!strip!find MAINT !console
MAINTJDV - 006C
MAINTIP -AB992611
MAINTCD -FBWSXM02
Ready;
Remark that in the second?PIPE?we inserted an extra blank after the word MAINT and that we still get the same output.
For?FIND?(and?NFIND), blanks in the argument string indicate columns whose contents are arbitrary, not columns containing only a blank character.
This differs from the way?LOCATE?works.?To?LOCATE, a blank is a blank.
How do you find a blank?? Exactly as you would do with XEDIT's FIND, use an underscore (_) in the argument string to indicate that a column must have a blank.?See how the results change in next example?:
pipe cp query names!split ,!strip!find MAINT_!console
Ready;
No output, as MAINT is not logged on at all.
Because an underscore indicates a required blank, you cannot search for underscores directly.?Instead, use?XLATE?to transpose underscore and some other character before the?FIND?or?NFIND?stage command and restore them afterward.?We will explain?XLATE?later in this chapter.
Be careful with blanks before the stage separator.?If there are one or more blanks between the string you want to find and the stage separator, these positions must be present in the input record even if their contents are ignored.?This is why next example shows responses you might not expect.?The input record is only one character long, but the argument to?FIND?(and?NFIND) is two bytes.
pipe literal a! find a ! console
Ready;
pipe literal a! nfind a ! console
a
Ready;
NFIND?is the converse of?FIND?; it selects records that do not match the argument.
We could use the?NFIND?filter to eliminate the?VSM?entries of the list of connected users?:
pipe CP query names!split ,!strip!nlocate /- DSC/! nfind VSM_!console
MAINTJDV - 006C
FTPSERVE -L0000
OPVM1 - 0065
FSCMVS - 0063
OPERATOR - 00ED
OP1 -PBRS0525
DECEULAE-FBIS8C53
...
You can use?FIND?with a suitable number of blanks to select records of a certain size (or longer), but it is simpler to use?LOCATE?with a column number to do this.?In a similar way, select short records with?NLOCATE.
Attention also for variable length records.?Even though CMS files can not contain records with a length of zero (e.g. XEDIT will place at least 1 blank in an empty record), zero length records (or null records) can appear in a pipeline.?For example, reading a variable file with some "blank" records and using the?STRIP?stage would result in?null records?traveling through the pipeline.
pipe < input file a ! strip ! ...
Thus, if you want to eliminate the null records, it will?not work?when you use this?:
pipe < input file a ! strip ! find _! console
as those records don't even contain a blank.
?3Fill in the next pipe with the proper stage(s) to effectively eliminate the null records.
pipe < input file a ! strip ! ... ! console
XEDIT, EXECIO?and?CMS Pipelines?all have both the?LOCATE?and?FIND?(sub)commands.?From experience, we know that the differences are not well understood, so let's expand a little bit?:
LOCATE?:
This subcommand will search for a given,?delimited?character string in that part of the records that is called the?zone.?The string can be located anywhere inside the boundaries of the zone, which by default spans the complete record.
The different tools let you?LOCATE(3)?a string in a zone as follows?:
XEDIT
[SET ZONE scol ecol]
LOCATE /string/
EXECIO
EXECIO nlines DISKR fileid ([ZONE scol ecol] LOCATE /string/
PIPE
PIPE ...!LOCATE [scol-ecol] /string/!...
Notes?:
XEDIT has possibilities for more complex and combined targets (AND, OR), and can also perform a case insensitive search.
CMS Pipelines?allows for multiple separate zones to be specified, while XEDIT and EXECIO allow only one zone.
CMS Pipelines?has also the?ALL?stage that is similar to XEDITs' ALL macro, and thus allows complex targets.?Case insensitive search is possible via the?CASEI?stage, or using the?ANYCASE?option available to many selection stages since VM/ESA Version 2.
The string delimiter has not to be the '/', any character not appearing in the search string will do.
FIND?:
This subcommand looks for a given string?that starts at the beginning of the record or at a given column.?As you can understand, this search needs less machine processing, so that you should prefer this function to?LOCATE?whenever possible.
The different command formats are?:
XEDIT
[SET IMAGE ON]
[SET TABS scol]
FIND string
EXECIO
EXECIO nlines DISKR fileid ([ZONE scol ecol] FIND /string/
PIPE
PIPE ...!FIND string!...
Notes?:
XEDITs?FIND?starts looking in the first?tab?column when?IMAGE?is set ON, else it searches for the string at column 1 (see?HELP XEDIT FIND, Usage notes).
'FIND USER MAINT?'
'LOCATE /USER MAINT?/'
'FIND USER MAINT_'
Let's come back to our little application.?We saw that when we list all connected users, we get a mix of local and network (VTAM) users.?Maybe we are only interested in the users connected to the VTAM service machine.?As these users follow the?VSM - VTAM?record,?FRLABEL?will help us here.
FRLABEL?skips records until one is met with the required beginning?; the record with the matching string?and?the remaining records are copied to the output.?It doesn't matter what is in the rest of the file, even if there is another matching record.
So, our pipeline becomes?:
pipe CP query names!split ,!strip!nlocate /- DSC/! frlabel VSM !console
VSM - VTAM
OP1 -PBRS0525
DECEULAE-FBIS8C53
MAINTIP -AB992611
BUELENSC-FBWS8J00
ACFMVS2 -ABBNAV08
ACFMVS5 -ABBNAV06
ACFMVS1 -ABBNAV07
ATESNA -ABBNAV05
ARTS38 -AB$2003L
MAINTP -ABBNAV16
MAINTCD -FBWSXM02
NL03710 -FNKSTT5
VSM - TCPIP
?4By means of what stage(s) would you eliminate the?VSM - VTAM?record??
TOLABEL?copies records to its output stream until it meets a record that begins with the argument string.??TOLABEL?does?not?write the matching record to its output stream.
We still have the?VSM -TCPIP?record (and there might be users listed logged on to TCPIP as well).?All records from the TCPIP entry on should thus be eliminated, but we propose that you do this as an exercise...
Exercise : Select only users connected to VSM VTAMTask
Complete our application in order to display only the users connected to the VSM VTAM.?Thus you have to eliminate the users connected to any other VSM and also eliminate the VSM records themselves.
Remarks
If you want to do this exercise on your system and you don't have more than one VSM service machine (or even none), then create a file with a simulated response from QUERY NAMES with multiple VSM virtual machines (with different names of course), and use that as input instead of the CP command.?You should be able to do this quickly with a little pipeline and a bit of XEDIT.
If you have to spend more than 30 minutes to find the solution to this question, then leave it and go on with next topic.
Yet another exercise will learn you to use selection filters:
Exercise : Select only users connected to VSM VTAM (enhanced).Task
Now complete the previous VSMUSER procedure so that you can pass a parameter and list the connected users to one of the different VSM machines you (can) have on a system, thus not only the first one.
领英推荐
Please note that?FRLABEL?and?TOLABEL, unlike?FIND, do not allow to search for blanks via the underscore character.
CMS Pipelines?includes stage commands that change the records passing through them.?You can?:
REXX programmers will notice that many of these stage commands have counterparts in the REXX language.?The difference is that these stage commands work on all data flowing through the stage, while in REXX they operate on a single expression.
We will spend some more time on these filters in order to further enhance our little application, and also to give you enough input to be able to complete other exercises.
The?CHANGE?stage command replaces one character or group of characters with another.?It is similar to the XEDIT?CHANGE?subcommand.?For example, suppose we want to change the string?-?DSC?into?Disconnected, we might enter?:
/* PIPCHNGE - Change example */
address command
'PIPE cp query names' ,
'!take 2' , /* limit the output */
'!split ,' , /* split users */
'!locate /- DSC/' , /* extract DSC users*/
'!change /- DSC/Disconnected/', /* change string */
'!console' /* output to screen */
exit rc
pipchnge
MVS55 Disconnected
MVS56 Disconnected
MVS54 Disconnected
TOOLS Disconnected
BIBLIO Disconnected
DISKACNT Disconnected
EREP Disconnected
Ready;
Note:?As our pipeline became rather long, we switched to the REXX solution.
The?CHANGE?stage command replaces the string wherever it occurs in every record in the pipeline. The strings don't have to be the same length.
Be careful when using?CHANGE.
pipe ... ! change /John/Martin/ ! ...
would change?:
John Smith and Bob Johnson ate johnnycake.
to
Martin Smith and Bob Martinson ate johnnycake.
Both John and Johnson were changed because there isn't a blank after John in the?CHANGE?stage command.?This may or may not be what you intended.?The word johnnycake was not altered because?CHANGE?is case-sensitive.
The default is to change all occurrences in every record.?You can limit the number of substitutions per record by writing a number after the delimited strings.?For example, write 1 after the delimited strings to change only the first occurrence in every record.
You can also limit?CHANGE?by specifying a range of columns.?In next figure,?CHANGE?searches for John only in the first four columns of each record.?Consequently, Johnson is not changed.
pipe < story script ! change 1-4 /John/Martin/ ! console
Martin Smith and Bob Johnson ate johnnycake.
Ready;
The column range tells?CHANGE?where to look.?It does not cause?CHANGE?to limit replacements to the first four columns.?Martin still replaced John even though Martin goes beyond the fourth column.
Next figure shows two column ranges.??CHANGE?looks for John in columns 1 through 4 and in columns 20 through 23.?Use parentheses as shown when entering more than one column range.?This time Johnson is changed because it begins in column 20?:
pipe < story script ! change (1-4 20-23) /John/Martin/ ! console
Martin Smith and Bob Martinson ate johnnycake.
Ready;
CHANGE?looks in all column ranges before changing the record.?So, the fact that Martin is longer than John does not prevent?CHANGE?from finding the second John starting in column 20.?In the output, Martinson begins in column 22.
You can specify up to 10 column ranges.?Make sure they are in ascending order and do not overlap.
If you are working with structured data and want to avoid changing the length of the record, simply make the search or replacement strings the same length as you would do in XEDIT?:
pipe < yourid netlog ! change /MIKE /BARBARA / ! console
The?SPECS?stage command rearranges contents of records.?It is one of the most versatile stage commands.?The HELP counts over 1000 records?!?This section covers many?SPECS?operands, but it does not cover all of them.?Some of the possibilities are difficult to understand for a beginning plumber, so we'll have to take a step-by-step approach.?We'll surely have to come back to other possibilities in later lessons.
SPECS?creates output records by piecing together data from various sources.?An output record might contain snippets from an input record, a literal string that you supply, and a record number that?SPECS?itself generates.?For each piece of data you want in the output records, you must tell?SPECS?where it is to get the data and where it should be placed.
For example, suppose your input records contain names and addresses.?Each record contains a surname in columns 20 through 40, inclusive.?The rest of each input record contains the first name and the address of the person.?For each input record read, you want to create an output record containing only the surname.?What's more, you want that surname to start in column 1 of the output record.?Here is the?SPECS?stage command to do it?:
...! specs 20-40 1 !...
The first?SPECS?operand (20-40) tells it where to get the data?: from columns 20 through 40 of each input record.?The second?SPECS?operand (1) tells it where to put that data in each output record: starting at column 1.?
Now suppose you also want to extract the zip code from each input record and put it after the surname in each output record.?You need to add a few more operands to?SPECS.?Assuming that the zip code is in columns 70 through 79 of each input record, here is a?SPECS?stage command to do it?:
...! specs 20-40 1 70-79 22 !...
We've added the operands 70-79 22 to?SPECS.?The operand 70-79 tells?SPECS?to get a second data item from columns 70 through 79 of each input record.?The operand 22 tells?SPECS?to put that item in each output record starting at column 22 (immediately after the 21-column surname).
The above two examples illustrate two very important aspects of?SPECS?:
To further describe?SPECS, we'll first show you various input operands.?That is, we'll show you how to tell?SPECS?where to get data.?Then we'll show you various ways to tell?SPECS?where to put that data in the output record.?In all cases, the input and output operands must be in the following order?:
input output
Input operands--Columns numbers and column ranges
We've already seen isolated?SPECS?stage commands that put pieces of input records in the output records.?Now let's look at a complete PIPE command?:
pipe literal abcdefghij ! specs 3-5 1 ! console
cde
Ready;
The?SPECS?operands indicate that columns 3 through 5 of each input record should be written to each output record starting at column 1.?As a result, the string cde is extracted from the sole input record and is written in columns 1 through 3 of the output record, which?CONSOLE?then displays.?
In the example, none of the other letters in the input record are in the output record.?The output records start out empty.?The only data in the output records is what you tell?SPECS?to put in them.?To put additional data from each input record into each output record, use additional groups of operands, as shown now?:
pipe literal abcdefghij ! specs 3-5 1 1 5 ! console
cde a
Ready;
We've added the operands 1 and 5 to?SPECS.?The operand 1 tells?SPECS?that you want the data in column 1 of the input record.?The operand 5 tells?SPECS?to put that data in column 5 of the output record.
By default,?SPECS?fills unassigned columns of the output records with blanks.?In the example, we've put data in the output record in columns 1 through 3 and in column 5, but not in column 4.??SPECS?puts a blank in column 4 for you.?The output record length is determined by the last assigned column.?In the example, the output record has a length of 5 characters.
You can use any of the usual formats for specifying column ranges.?Next figure uses the operand 4.5 to identify a column range of 5 characters starting with column 4.?It also uses the operand 4-6 to identify columns 4 through 6.
pipe literal 000Television ! specs 4.5 5 4-6 1 ! console
Tel Telev
Ready;
The figure also shows that you do not need to build the output record in any particular order.?First we put data starting in column 5, then we put data starting in column 1.?There isn't anything wrong with doing this.?It's also valid to refer to columns of the input record more than once.?Notice that we refer twice to columns 4, 5, and 6.
An asterisk (*) in the second position of a column range means end of record, just as it does on other stage commands.?The next example uses an asterisk in the column range.?It removes sequence numbers from columns 1 through 8 of the input (no matter how long it is).?All characters from column 9 to the end of the input record are copied to the output record starting at column 1.
pipe literal 12345678An input record ! specs 9-* 1 ! console
An input record
Ready;
Using numbers to identify columns or column ranges on input records is useful when the data is structured.?In inventory records, for example, all part numbers are likely to occur within a specific column range.?However, not all data is structured in this way.?For processing other forms of data, the?WORDS?operand is often more useful.?
Input operands--WORDS
The?WORDS?operand of?SPECS?lets you select data from input records by word number (or word range) rather than column number (or column range).?To?SPECS, a word is any blank-delimited string of characters (the same meaning word has in the REXX language).
Next figure shows two examples of?WORDS.?In both examples, the third word of the input record is put in the output record starting at column 1.
pipe literal See Joe compute. ! specs words 3 1 ! console
compute.
Ready;
pipe literal See Joe compute. ! specs w3 1 ! console
compute.
Ready;
In the first example, the input operand?WORDS 3?identifies the third word in the input record.?Even though the input is expressed in words, the output is expressed in columns.?The output operand?1?tells?SPECS?that you want that word placed in the output record starting in column 1.?In the second example, the input operand?W3?also indicates the third word.?W is an abbreviation for?WORDS, and the space between?WORDS?and the number is optional.
You can also specify a range of words.?Indicate word ranges in the same way that you have been indicating column ranges.?This is an example.
pipe CP query names ! specs w1 1 w5 10 w9 19 w13 28 ! console
MVS55 MVS56 MVS54 MAINTJDV
TOOLS BIBLIO DISKACNT EREP
LANPRINT PORTMAP RTMESA FTPSERVE
SQLMACH VTAM2 IECCNM OPVM1
FSCMVS - - -
OPERATOR -
...
Ready;
You see we have a problem in the last records of the output.?Indeed, a word is a character string separated by blanks from the rest of the input.?As connected users show their terminal address in the form?:
FSCMVS - 0063, FSCIPOE - DSC , VMBARS - DSC , CALLUP - DSC
there is no space anymore between the terminal address and the comma, and this counts for one word...
You can mix references to columns and to words in a single?SPECS?stage command.?Use whatever format is most appropriate for the task at hand.?There are also ways to specify data that is not from the input records.
Inputs--Literal strings
The preceding examples all show how data from the input records can be placed in the output records.?Data can come from sources other than the input records.?One of those sources is a literal string specified on the?SPECS?stage command itself.
To specify a literal on?SPECS, use a delimited string as the input operand instead of a column or word reference to the input record.?Delimit the string the same way you delimit strings on other stage commands (such as?LOCATE).?Use any character that is not in the string itself and that does not have a special meaning (such as a stage separator).??SPECS?will put the string on every output record.?For example, this pipe will modify our output to full phrases?:
/* REFORM - Reformat the output records */
address command
'PIPE cp query names' , /* query CP */
'!split ,' , /* split users */
'!strip' , /* strip blanks */
'!nlocate /- DSC/' , /* find connected */
'!tolabel VSM' , /* only local users */
'!specs /User/ 1' , /* literal in col 1 */
'Words 1 6' , /* userid in col 6 */
'/is logon at terminal/ 15' , /* literal at col 15 */
'11-* 37' , /* termid at col 37 */
'!console' /* output */
exit rc
reform
User MAINTJDV is logon at terminal 006C
User FTPSERVE is logon at terminal L0000
User OPVM1 is logon at terminal 0065
User FSCMVS is logon at terminal 0063
User OPERATOR is logon at terminal 00ED
Ready;
As before, read the?SPECS?arguments in groups (the REXX coding style allowed us to separate the groups)?:
specs /User/ 1 words 1 6 /is logon at terminal/ 15 11-* 37
---+---- ----+---- -----------+------------- ---+---
! ! ! !
v v v v
Group 1 Group 2 Group 3 Group 4
The input operand in the first group tells?SPECS?that the data to be put on the output record is the string?User.?Notice that slashes (/) are used to delimit the string.?The output operand 1 of that group puts the literal in the output record starting at column 1.?The second operand group puts the first word (the userid) in the output record starting at column 6, which happens to be after the?User?string with one intervening space.?Then another literal string is put on the output record, and finally, all characters from column 11 till the end of the input record (the terminal id) is added starting at column 37.?
Usually you will use literal strings with regular characters.?
Occasionally you might need to use hexadecimal characters (perhaps because you cannot type the desired characters on your terminal).?To do it, prefix the string of hexadecimal digits with the character x or h.?A blank marks the end of the literal, so there cannot be blanks in the string itself.?The number of digits in the string must be even.
Next example shows how to put a vertical bar (!) in the output record without it being interpreted as a stage separator when the pipeline is processed.?The contents of the input record are put after the vertical bar and another vertical bar is put after the record.?Remark the trailing blank for?LITERAL?belongs to the string.?
pipe literal a line ! specs x4f 1 1-* 2 x4f next! console
!a line !
Ready;
The next few sections describe various operands you can use to position data in output records.
Output operands--Columns and column ranges
We've already seen how to use column numbers.?Let's give one extra example where we miscalculate the numbers?:
pipe literal Raymond Jones ! specs 9-13 1 1-8 5 ! console
JoneRaymond
Ready;
In this case, Jones is copied to columns 1 through 5, and Raymond is copied to column 5.?Raymond overlays the s in Jones.?Because you might want to do something like this intentionally, no error message is produced.
You can also specify a column range for the output operand.?In this case, the input field is truncated or padded on the right to fill the output range.
Output operands--NEXT
Often you'll want to put data on the output record in the next available column.?Use the?NEXT?output operand to do it.?When?NEXT?is used, you don't have to count columns.?We could therefore change our previous pipeline into?:
/* REFORM2 - Reformat the output records */
address command
'PIPE cp query names' , /* query CP */
'!split ,' , /* split users */
'!strip' , /* strip blanks */
'!nlocate /- DSC/' , /* find connected */
'!tolabel VSM' , /* only local users */
'!specs /User / 1' , /* literal in col 1 */
'Words 1 next' , /* userid in next col*/
'/is logon at terminal / nw', /* literal next word */
'11-* n' , /* termid next word */
'!console' /* output */
exit rc
By adding an extra space to our literal strings, we provide for the space in the output.?We can then use the?NEXT?operand to let?CMS Pipelines?calculate where to put the output.?Remark that?NEXT?can be abbreviated to simply the letter?N.
?5Will the output of this pipeline be exactly the same as our previous solution?? If you can't find the answer, try it on the system.?What would you change to make it identical again??
Output operands--NEXTWORD
NEXTWORD?is similar to?NEXT.?The difference is that?NEXTWORD?puts a blank before the input data.??NEXTWORD?does not insert a blank if there isn't yet data in the output record, but copies the input in column 1 instead.
So, our procedure can be simplified even more?:
/* REFORM3 - Reformat the output records */
address command
'PIPE cp query names' , /* query CP */
'!split ,' , /* split users */
'!strip' , /* strip blanks */
'!nlocate /- DSC/' , /* find connected */
'!tolabel VSM' , /* only local users */
'!specs /User/ 1' , /* literal in col 1 */
'Words 1 nextword' , /* userid next word */
'/is logon at terminal/ nextw' , /* literal next word */
'11-* nw' , /* termid next word */
'!console' /* output */
exit rc
We don't need extra spaces in our literal strings anymore.?Remark also that?NEXTWORD?can be abbreviated to?NEXTW?or even?NW.?
We stop our discussion about?SPECS?here for the moment.?There is a lot more to say about it, but we leave it for later lessons.
Let's now have an overview of some other filters that you might find useful to complete later exercises.
XLATE?translates data passing through the pipeline on a character-by-character basis.?You can translate characters to lowercase or to uppercase.?You can also replace all occurrences of one character with another.?The figure below shows how to translate characters to uppercase or lowercase?:
pipe literal Asterix & Obelix!xlate upper!console!xlate lower!console
ASTERIX & OBELIX
asterix & obelix
Ready;
To limit the translation to certain columns, specify a column range after the?XLATE, as in this example?:
pipe literal Play piano, not forte. ! xlate 17.5 upper ! console
Play piano, not FORTE.
Ready;
The 17.5 defines a range of 5 columns starting on column 17.?(Specifying 17-21 would yield the same result).
With the?UPPER?and?LOWER?operands, we've been translating many characters with a single?XLATE?stage command.?It's also possible to translate individual characters.?Next figure shows how.?Instead of specifying?UPPER?and?LOWER?operands, specify pairs of characters after?XLATE.?The example changes all e's to X's.
pipe literal Extra eggplants free. ! xlate e X ! console
Extra Xggplants frXX.
Ready;
The order of the translation parameters is the opposite of the order in the?TRANSLATE()?function of REXX, hence input before output.
Notice also that the capital E in Extra was not changed.??XLATE?is case sensitive.?To change both small and capital E's to X's, specify two pairs of characters after?XLATE?as follows?:
pipe literal Extra eggplants free. ! xlate e X E X ! console
Xxtra Xggplants frXX.
Ready;
You can specify many pairs of characters after?XLATE.?The characters are not limited to those in the alphabet.?You can also specify numbers, punctuation, and hexadecimal values.?
It is possible to define a zone (or columnrange) on which?XLATE?must work.?This range has to be specified before the translate table.?For example?:
pipe literal Extra eggplants free. ! xlate 1-10 e X ! console
Extra Xggplants free.
Ready;
As it is possible to translate numeric digits, these digits in the translate table can be confused with a column range.?In that case, you must specify a column range (e.g. 1-* to indicate the complete record).
pipe literal 370 ! xlate 1-* 3 E 7 S 0 A ! console
ESA
Ready;
You can specify the range between parenthesis if you want.
Instead of specifying characters directly, you can use hexadecimal values.?
This is useful when you want to specify characters that you cannot type on your keyboard.?It is also useful when you want to specify a blank, which is the hexadecimal value X'40'.?Specify hexadecimal values by typing the two characters that make up the byte.?Here, we change all parentheses to blanks.
pipe literal x(x)x ! xlate 1-* ( 40 ) 40 ! console
x x x
Ready;
As the column range could have been coded in parenthesis, and as the first character of the translate table is also a left parenthesis, the specification of a columnrange is needed here too, to ensure that?CMS Pipelines?parses the stage command properly.?If you don't, you get?:
DMSXLA2698E Missing character range
DMSPSR2653I ... Issued by stage number 2 of pipeline number 1
DMSPSR2651I ... Running stage: xlate ( 40 ) 40
You can also specify a range of characters (as opposed to a column range) to be translated instead of individual characters.?For example, suppose you want to translate the numbers 3 through 5 to equal signs.?One way to do it is by typing each character?:
...! xlate 1-* 3 = 4 = 5 = ! ...
Next figure shows another way.?It uses a character range to do the same translation.?The second?PIPE?command in the figure also shows a character range.?It removes punctuation from a record by translating the punctuation to blanks.?Hexadecimal values are used in the second example.
pipe literal 123456789 ! xlate 1-* 3-5 = ! console
12===6789
Ready;
pipe literal (In parentheses.) ! xlate 1-* 41-7f 40 ! console
In parentheses
Ready;
You can specify ranges for both the input and the output.?Next example translates characters 1 through 9 into the corresponding letters A through I.
pipe literal 123 12389 ! xlate 1-* 1-9 A-I ! console
ABC ABCHI
Ready;
To translate all but a few of the characters in a range, specify the range of characters and also the characters you wish to omit.?Before?XLATE?does any translation, it determines whether any character is specified more than once in the operands.??XLATE?uses only the last occurrence when translating the data.?Here is an example.?The characters from c to g are translated to equal signs, except for the character e.?The character e is translated to itself.
pipe literal abcdefghi ! xlate c-g = e e ! console
ab==e==hi
Ready;
You can use the same technique to augment the built-in translations to uppercase or lowercase with your own translations.
Before you jump to conclusions and start to create translation tables for conversion from ASCII to EBCDIC for example, notice that?CMS Pipelines, again, provides you the necessary functions to do that?:
... ! XLATE 1-* E2A ! ... /* translates from EBCDIC to ASCII */
... ! XLATE 1-* A2E ! ... /* translates from ASCII to EBCDIC */
... ! XLATE 1-* TO CODEPAGE 037 ! ...
/* translate from codepage 500 (international) to 037 (USA) */
The?COUNT?stage command counts characters (bytes), words, or records in its input stream.?It writes a single record containing the count to its output.
When counting words, the?COUNT?stage command considers any blank-delimited string to be a word.?To?COUNT, the TEST DATA file contains 10 words.
========================================
Don't worry about me--I can take
care of myself.
The string of equal signs (=) counts as one word, and the string?me--I?also counts as one (not two).
pipe < test data a ! count bytes ! console
89
Ready;
pipe < test data a ! count words ! console
10
Ready;
pipe < test data a ! count lines ! console
3
Ready;
?6What would be the output of the pipeline below, where we combined the 3 previous pipes into one??
pipe < test data a!count bytes!console
!count words!console
!count lines!console
Note?: we suppose you use still the same input.
COUNT?lets you count several things at a time, as shown in next figure.?The operand?CHARS?is a synonym for?BYTES
pipe < test data a ! count chars words lines ! console
89 10 3
Ready;
When you specify more than one item,?COUNT?always returns results in this order?: characters, words, lines.?The order in which you specify the operands does not change the order of the results.
VM/ESA 2.2 adds the options?MAXLINE?and?MINLINES, returning the lenght of the largest, resp.?shortest record passing along the stage.?This can be useful to pad all records to the size of the largest one when creating Fixed format files.
The?SORT?stage command orders pipeline records.??SORT?buffers (or delays, to use the term used in the command reference) the pipeline records?; that is, it reads all its input records before writing output records.?The sorting is done in virtual storage.??SORT?gives a message and a nonzero return code if the input is too large to fit in storage.
For example, the following pipeline sorts our users connected via a VTAM service machine?
/* PIPSORT - Sort connected VTAM users */
address command
'PIPE cp query names' ,
'!split ,' , /* split users */
'!strip' , /* strip blanks */
'!frlabel VSM' , /* only via VSM */
'!nfind VSM_' , /* eliminate VSM cards */
'!sort' , /* sort output */
'!console' /* output to screen */
exit rc
pipsort
ACFMVS1 -ABBNAV07
ACFMVS2 -ABBNAV08
ACFMVS5 -ABBNAV06
ARTS38 -AB$2003L
ATESNA -ABBNAV05
BUELENSC-FBWS8J00
DECEULAE-FBIS8C53
MAINTCD -FBWSXM02
MAINTIP -AB992611
MAINTP -ABBNAV16
NL03710 -FNKSTT5
OP1 -PBRS0525
Ready;
Using column ranges when sorting
You can also specify column ranges to be used in the sort.?The records are sorted by the contents of the column ranges.?For example, the pipeline in next figure sorts the names of the files on file mode A by file type, and displays the first ten file names.
pipe command LISTFILE * * A ! sort 10-17 ! take 10 ! console
NOT AUTHORIZ A1
TEMP DATA A1
TEST DATA A1
TEST1 DATA A1
TEXT DATA A1
DIRADD EXEC A1
DIRCOUNT EXEC A1
DISKSPAC EXEC A1
DUAL EXEC A1
FEXAM EXEC A1
Ready;
To sort in descending order when a column range is used, use?DESCENDING?after the column range?:
pipe command LISTFILE * * B ! sort 10-17 descending ! take 4 ! cons
MYBOOK SCRIPT B1
ROLLUP MODULE B1
DMSC5MST LIST3820 B1
TEST DATA B1
Ready;
Discarding duplicates when sorting
Use?SORT UNIQUE?to discard duplicate records during a sort.?The figure shows how to display a list of unique words in a file.
pipe < input file ! split ! sort unique ! console
The?<?stage command reads the file INPUT FILE.??SPLIT?puts each blank-delimited word on a separate record.?The records are then sorted in alphabetic order.?The?UNIQUE?operand on?SORT?causes duplicate records to be discarded.?Finally, the unique words are displayed on the console.
Counting and discarding duplicates while sorting
The?COUNT?operand of?SORT?counts the number of duplicates of each record.
It discards duplicates, but adds a count of the number of duplicates to the beginning of each remaining record.?The count is right-justified in columns 1 through 10 of each output record.?The original input record follows, beginning in column 11.
Next figure shows two?PIPE?commands.?The first command shows the contents of the file that is sorted in the second?PIPE?command.
pipe < sample data ! console
one
two
two
three
three
three
four
four
four
four
Ready;
pipe < sample data ! sort count ! specs 1-10 1 11-* nw ! console
4 four
1 one
3 three
2 two
Ready;
This ends the theory part of this lesson.?You should now take the time to code as much as possible of the proposed exercises to put the text in practice.?But, before that, we'll give you an extra productivity tool that can help you in solving the exercises.
When you build a pipeline, it would be handy to trace what's flowing through the pipe at specific places in the pipeline.
The first solution that may come to your mind is to add a?CONSOLE?stage at the specific moments where you want to trace the flowing data.?For example?:
PIPE COMMAND Q DISK ! LOCATE /19/ ! > 19X DISKS A
will collect all your accessed disks with virtual addresses in the range 190 to 19F.?Now if you want to see what's happening to the pipe, then add some?CONSOLE?stages like in this example:
PIPE COMMAND Q DISK ! CONSOLE ! LOCATE /19/ ! CONSOLE ! > 19X DISKS A
Right, but when you run this, you'll see that the output is not always what you would like to see.?Indeed, due to the two?CONSOLE?stages, you will get a mix of output and you will not be able to distinguish what stage produced the output.?See, what we mean?:
LABEL VDEV M STAT CYL TYPE BLKSIZE FILES BLKS USED-(%) BLKS LEFT...
- DIR A R/W - - 4096 148 - -...
- DIR B R/W - - 4096 26 - -...
- DIR C R/W - - 4096 39 - -...
- DIR D R/W - - 4096 53 - -...
GOODIE 193 E R/O 20 3380 4096 614 1983-66 1017...
GOODIE 193 E R/O 20 3380 4096 614 1983-66 1017...
- DIR F R/W - - 4096 72 - -...
- DIR G R/W - - 4096 25 - -...
- DIR H R/O - - 4096 15 - -...
MNT190 190 S R/O 96 3390 4096 333 7967-46 9313...
MNT190 190 S R/O 96 3390 4096 333 7967-46 9313...
MNT19E 19E Y/ R/O 50 3390 4096 379 2999-33 6001...
MNT19E 19E Y/ R/O 50 3390 4096 379 2999-33 6001...
You see?? You get the outputs intermixed.?This is because at each moment there are only a few records going through the pipeline and all stages are almost working at the same time...
OK, what is the solution then?? Use the?TRACE REXX?pipe stage listed below.?This is an example of a user-written stage.?We will learn more about it in a later lesson so we won't comment on the way it works?; we just give its contents here?:
/***********************************************************************/
/* Pipe-stage TRACE REXX */
/* Show data in the middle of the pipeline. Add the tag passed to */
/* TRACE, display each record, and delete the tag before passing */
/* the record to the output stream. */
/***********************************************************************/
parse arg id /* get argument in mixed case */
if id = '' then do
'stagenum' /* get the stage number */
id = 'Stage' rc /* use it as tag */
end
id = id':' /* append a colon */
'callpipe',
'*:', /* Read from input */
'! change //'id'/', /* Insert id first */
'! console', /* Type */
'! specs' length(id)+1'-* 1', /* Remove id */
'! *:' /* Pass on */
exit rc
Now, the only thing you have to do is replace the?CONSOLE?stages by?TRACE?stages, like here (note the different parameters):
PIPE COMMAND Q DISK!TRACE sta!LOCATE /19/!TRACE end!> 19X DISKS A
and, look at the difference of the output (note that the TRACE stage only temporarily modifies the records and restores them afterwards so that we can say that the labels are not part of the records themselves)?:
sta:LABEL VDEV M STAT CYL TYPE BLKSIZE FILES BLKS USED-(%)...
sta:- DIR A R/W - - 4096 150 - ...
sta:- DIR B R/W - - 4096 26 - ...
sta:- DIR C R/W - - 4096 40 - ...
sta:- DIR D R/W - - 4096 53 - ...
sta:GOODIE 193 E R/O 20 3380 4096 614 1983-66 ...
end:GOODIE 193 E R/O 20 3380 4096 614 1983-66 ...
sta:- DIR F R/W - - 4096 72 - ...
sta:- DIR G R/W - - 4096 25 - ...
sta:- DIR H R/O - - 4096 15 - ...
sta:MNT190 190 S R/O 96 3390 4096 333 7967-46 ...
end:MNT190 190 S R/O 96 3390 4096 333 7967-46 ...
sta:MNT19E 19E Y/S R/O 50 3390 4096 379 2999-33 ...
end:MNT19E 19E Y/S R/O 50 3390 4096 379 2999-33 ...
One last hint, choose labels of same length if you want to keep the output aligned, like we did here.?
You are ready now to work out the exercises described in?Exercises for Lesson 1..
Footnotes:
(2)?Network Service Machine (NSM) would have been a better choice here, but in the beginning, there was only VTAM...
(3)?ISPFs' editor has the FIND command, which is very similar to the LOCATE command we are discussing here.