Python versus Perl (Redux)

In July of 2019, I wrote an article which compared Perl and Python. At the time, I still found use for Perl, primary because of the power of the regular expression engine, and some programmatic shortcuts which were cumbersome in Python. As of version 3.10, my objections have ended. It is with some sadness and lots of fond memories that I admit that it is now time to bid a fond farewell to Perl and its derivatives. Python now has all the useful programmatic features that are needful for nearly any type of high-level scripting, including text processing. Additionally with tools such as TextFSM available, I would argue that for more complex processing, Python has taken the lead in being the “go to” language for text.

So what changed?

Let me address a couple examples of things that have either changed, or how behavioral differences can be overcome.

The walrus!

Two key changes have happened in the last couple versions. The first was the “walrus” operator. This greatly simplified regular expression match syntax by reducing a needless step. In Perl and C this functionality has been assumed from the early days. Essentially, it was always possible to make an assignment as part of a comparison statement. The comparison, then, is against the result of the assignment. This simplifies and clarifies the operation. However, in Python this has not been possible until version 3.8 with the walrus operator. Here is an example of <3.8 code.

import re
my_var=re.match(r'matchstring',targetstring)
if my_var is not None:
	do stuff with my_var        

Now, with the walrus operator in 3.8, it can be done this way.

import re
if (my_var:=re.match(r'matchstring',targetstring):
	do stuff with my_var        

As long as the assignment does not equal a numeric 0 or the special type None (something like Null in other languages), the expression matches, and the contained code snippet will be executed. This is very useful and a much cleaner form of the same thing.

You may say, “But really, it’s ONE extra line! Does it really matter?” It isn’t much more in overall script length per expression, but in my opinion, it makes a lot of difference in tying the assignment to the condition and keeping the code block cohesive.

Additionally, the regular expression engine in Python is much more powerful than it used to be. Almost (if not all) of the regular expression features are available, including even all the intermediate features such as capturing, grouping, back reference, and look around, and even advanced features such as conditional matches, atomic matches, etc. There are few situations which cannot be solved with regular expressions. A full reference to Python regex can be found here, along with other references.

Python Regex Documentation

Python-style "online" syntax tester

Thorough regex feature documentation

When combining these features with TextFSM, large multi-line text blocks with complex formatting can be matched and placed into a dictionary variables.

The match/case clause

Both Perl and C have support for a switch/case statement. In Perl, it is done via a standard module “Switch”. In C, it is built in. Although the execution is slightly different, the concept of multiple match cases applies to both.

In Python this has not been possible… until version 3.10. Now, we can! Consider this code section.

targettest=3

if targettest==1:
	print('matched 1')
elif targettest==2:
	print('matched 2')
elif targettest==3:
	print('matched 3')
elif targettest==4:
	print('matched 4')
else:
	print('matched nothing')        

Although this certainly works, it makes a very ugly block of code. Now, we can express the same thing this way.

targettest=3

match targettest:
	case 1:
		print('matched 1')
	case 2:
		print('matched 2')
	case 3:
		print('matched 3')
	case 4:
		print('matched 4')
	case _:
		print('matched nothing')        

This works as long as the primary comparison of each match condition is against a single tested element which in this case is "testtarget". In C this is somewhat limited because of type casting. However, in Python, this can be done with multiple types, including dictionaries or lists. For example:

targettest={
    'keylevel1a': {
        'field1':1,
        'field2':2,
    },
    'keylevel1b': {
        'field1':3,
        'field2':4,
    }
}
anothervalue=10
match targettest:
    case {'keylevel1a':{'field1':value}}:
        print(f'level1a with field1:{value}')
    case {'keylevel1b':value} if anothervalue==10:
        print(f'level1b with field1 matches 3')
    case {'keylevel1c':value}|{'keylevel1d':value}:
        print(f'matched level 1c or 1d with {value}')
    case _:
        print('matched nothing')        

This will match the first “case”. However, this code section would allow several matches, additional validation steps like the second case, and including “or” blocks like the third case. All three cases above show a method of recording the matching values inline, which otherwise would take additional steps.

Flexibility

One of the most powerful aspects of Python is the flexibility of the language. I have written previously about code portability, and how easy it is to bring previously written code into current projects. However, Python has a unique level of flexibility in allowing you to modify core default behavior. For example, in Perl if you define a multilevel hash in which the higher levels are not yet defined, Perl will automatically fill in the missing levels. For example:

$hash{'level1'}{'level2'} = 'value';
print $hash{"level1"}{"level2"}        

Also, if you reference a hash value which does not exist, you will simply receive nothing. However, in Python, referencing a non-existing value, or attempting to perform the above directly will result in a KeyError exception. There are times when this protection can be helpful. However, at times it can create cumbersome levels of "if" and/or "try" blocks to ensure that no such exceptions arise. However, Python also provides a way to modify the default behavior. For example, consider this.

class testdict(dict):
    def __getitem__(self,args):
        try:
            var=super().__getitem__(args)
        except:
            self[args]=testdict()
            return self[args]
        else:
            return var

test=testdict()
test['level1']['level2']['level3']='some value'        

Now, using the new “testdict” class, you can not only reference non-existing values, but you can make “on the fly” multi-level assignments. The flexibility in modifying the default behavior does not come without a performance cost because the type-handling code is now script rather than compiled, but as long as the use case can tolerate the hit on performance, the flexibility is nearly limitless.

Conditional Operators

In both C and Perl, it is possible make assignments based on a condition. This is shorthand, and sometimes simplifies code expressions. Here is an example from Perl:

$var=($othervalue==2)? 1 : 2;
print $var        

In this example, the value assigned to var depends on whether othervalue is equal to 2. This exact feature is not available in Python, but a similar result can be created with this code snippet which uses a tuple and index.

var= (2, 1)[othervalue==2]        

It is possible to expand this option to include a comparison of length. For example:

var= (1, 2)[bool(len(stringvalue))]        

Some Complaints

This is not to imply that Python is the perfect language... yet. There are several things I would like to see added and/or updated. For example, Python currently does not support the C-like "increment before" or "increment after" features of variable usage (++x, or x++ respectively).

Another example would be C structures. The Python struct module does allow a user to make something similar in result, but because Python is somewhat removed from direct memory access, link-lists (as an example) are much harder to create and process efficiently. This is a significant impediment if you wish to write a high-performance application which has a large memory stack. This will be a limitation in native Python as a low-level socket programming language. That is unfortunate, but hopefully will be addressed in the future.

Although there other aspects which I believe should be updated, they are minor. Overall, the language is becoming very mature.

Summary

While low-level languages like C are still faster when executing intense memory operations, Python has become the best general, multipurpose high-level scripting language (in my opinion). It is a language which everyone should learn if they have any engineering interaction within a production environment. While it has not yet replaced BASH or Expect scripting for specific uses, Python is well suited to handle nearly every other circumstance.

Specific engineering disciplines which I believe should learn at least a rudimentary level of Python: Network Engineers, Linux Engineers, Virtualization Engineers, Storage Engineers, NOC Engineers, Monitoring Engineers, and likely many more.

Thanks for reading. Happy coding!