Regular Expressions Are Your Friend

I hope your reaction to my title isn’t “WTF?”. Let me hasten to explain…

TL;DR:?Plug for learning how to use regular expressions, incrementally.?

I suspect regular expressions are something some/many of us either don’t know much about, or, for those who do know about them, we consider regexes painful. As in “make my head hurt”. (And I’ll grant the head hurt thing has happened to me a few times over the years.)

However, I claim regular expressions are something it behooves network engineers to?start?learning and working with at some point. Say 3 years into a career, maybe?

Emphasis on “start”.

Regexes aren’t something someone new to networking (aka “newbie”) needs to deal with. They may not be something you want to poke a big hole into your certification plans for. But if you learn a little and start trying to use them, they can be helpful in the job, and you can start building more serious regex skills.?

So the key initial skill is understanding where a regex might help you. Then when you have a task they might help with, you can expend a little time fiddling to see if you can come up with a regex that fits. You’ll quickly learn to spot problems a short/simple regex can help with, and as your skills build, you can do fancier ones. I’ve seen some really long ugly ones lately, and I personally draw the line when the obscurity level gets too great.

Why Regexes?

1)????CLI. In Mac or Linux systems, you can do all sorts of mad things with regexes on the command line. Like find files matching a pattern. Or change all file names in a folder to pure lower case. Just last week, I used them to look up IP addresses. I have a folder of show output (including show run) from a customer, and I did “grep ‘ip address 100.101.102.103’ results/*” while troubleshooting using traceroutes. Ok, that’s a rather basic example – but with egrep you could match on a regex, and the filename wildcard could be fancier.?

2)????You just thought “but I run Windows”. Didn’t you? Well, the WSL (Windows Subsystem for Linux) isn’t hard to install, and I’ve read that the latest version of Windows includes the Linux by default (I see online that it has to be enabled). Apparently with Windows 10, you needed the Pro version to install WSL. So with minimal effort, you can jazz up your Windows CLI. Doing so allows you to run some useful tools as well, like a fast multi-address ping tool.?

3)????Programming. Python loves regular expressions. Well, sort of. Very useful if you’re processing (parsing) show output or almost anything else.?

4)????Regex-aware editors. Notepad++ on Windows, SublimeText on Mac are the ones I’m familiar with. (Been a while since I did Notepad++ though.) Regex patterns are amazing for transforming text data. (And Undo is your friend when you’re trying to get the regex right!)

For instance, as you might have noticed (prior blogs), I’ve been fiddling with DNS lookups and flow data. Some egrep CLI work let me quickly remove a lot of junk from ASA firewall flow data, getting it down to the format <source IP, source port, destination IP, destination port, byte count>. I could then easily read the data into some quick Python code or an Excel spreadsheet. For more regular use, I’d take the winning regex and build it into the Python code.?

Related Linux Tools

There are some Linux tool programs that I find myself using with regexes.

I generally use regexes with ‘ls’ (to find files), ‘grep’ and ‘egrep’ to match patterns in files or in file names, and ‘sed’ if I want to do some CLI-based text replacement.?

Tips

In my parsing coding lately, I’ve been using complex regular expressions. Naming the components helps.

I use r_ as a prefix for a named regular expression, and rp_ for the same regex with parentheses around it. The latter being a field I want the parsed value of for other use. That lets me write things like?‘^router\s+’ + rp_routeproto + ‘\s+’ + rp_anything + ‘\s*$’, to extract the routing protocol in question, and any stuff following it. The ‘^’ says it starts at the beginning of the line, the ‘router’ is literally the word ‘router’, the ‘\s’ is a space character, ‘+’ means one or more,??‘*’ means 0 or more, and ‘$’ means end of line. Yes, I could have named those too, but then the regex patterns get rather long.?

The rp_routeproto might be something like ‘(eigrp|ospf|bgp|isis)’ – an OR pattern. The rp_anything might be ‘(\S+)’ for anything but whitespace.?

I think I got this idea from something I saw online, and am thinking there might even be some online documents with standard canned regular expressions for networking like the above. I did some quick google searching and didn’t find anything, though.?

So here are some handy named regexes I’ve been using in parsing a lot of Cisco show output:

• r_non_digits='[^0-9]+'
• r_digits='[0-9]+'
• rp_digits='('+r_digits+')'
• r_alphadigits='[A-Za-z0-9]+'
• rp_alphadigits='('+r_numdigits+')'
• rp_mac='([A-Fa-f0-9\.]+)'
• r_comma_text='[A-Za-z\,]+'
• rp_comma_text='('+r_comma_text+'+)'
• r_alpha='[A-Za-z]+'
• rp_alpha='('+r_alpha+')'
• r_spaces='\s+'
• r_spaces_maybe='\s*'
• rp_nonspaces='(\S+)'
• r_ipaddress='[0-9\.]+'?
• rp_ipaddress='('+r_ipaddress+')'
• r_vrf='[A-Za-z0-9\-\_]+'
• rp_vrf='('+r_vrf+')'
• rp_interface='([A-Za-z0-9\/\-\:\.]+)'

I have more, but didn’t want you to fall asleep reading this. Also, some are ‘heuristic’, as in matched what I had examples of – Cisco interface names for instance (more variants than I’d expected).?

Note:?The ipaddress pattern could be more precise (4 blocks of 1 to 3 digits separated by periods, etc.), but that hasn’t been necessary in parsing actual router/switch output because the CLI would have caught truncated addresses and typos. See also?https://www.oreilly.com/library/view/regular-expressions-cookbook/9780596802837/ch07s16.html?for various patterns for this, with differing degrees of precision (accuracy).?

Resources

Google search turned up some courses and links to tutorials. Free is good.?

There’s an interesting-looking regex learning site at?https://github.com/ziishaned/learn-regex

It links to a site that lets you test regexes against text you supply:?https://regex101.com/

There seem to be lots of comparable resources available.?

For someone else’s take on my theme here, I ran across the following:?

https://ttl255.com/regex-and-unix-tools-for-networking-basics/????????????????

Conclusion

Regular expressions are too darn handy to ignore. Build a little proficiency with them to the point where they’re providing you some value, and then you’ll be in a position to grow your skills.?

As usual, I hope this blog was helpful.?

Comments

Comments are welcome, both in agreement or constructive disagreement about the above. I enjoy hearing from readers and carrying on deeper discussion via comments. Thanks in advance!?

Hashtags:?#NetCraftsmen #CiscoChampion #CCIE1773 #Regex #RegularExpression

Disclosure statement

Twitter:?@pjwelcher

LinkedIn:?Peter Welcher