Yet Another CLI Trick: grep for multiple matches

Let's start this one off with a fun little factoid: grep is an acronym for General Regular Expression Parser. If you haven't figured this out about me by now, you're gonna - I would like for you to read the man page.

I bet you already know you can search for more than one "element" at a time using the -e command line operator. Using -e to specify multiple elements is an OR operation, not an AND.

grep -e element1 -e element2 -e element3 filename        

This invocation matches any line in filename that contains any one of the elements. How do you make use of that? Do you script a constructor to put together a massive invocation with a long train of -e combined together?

grep `awk '{print "-e",$1}' elements` filename        

If you create a file with all the elements you're searching for, one element per line, and being as creative as I am, you name the file elements, then the invocation above will generate a listing of every line in filename that matches any one of the elements in your elements file.

But that's ridiculous. Wanna know why? Because the authors of the grep utility already got you covered, fam! The invocation below is equivalent to the invocation just above.

grep -f elements filename        

I mentioned in a previous article that many Unix utilities will use the hypen [-] as a token to pass in for a filename when the operator intends to read from STDIN or write to STDOUT.

As it turns out, grep is right there with the Unix standard on this one.

cat elements | grep -f - filename        

Why do it that way? Consider the situation where you don't have a pre-existing list of elements to search for - what then? Try chaining together multiple operations across multiple systems:

ssh ipam show-most-recent-leases -maconly | grep -f - /var/log/radius        

A bit contrived, sorry - but let's say you have some command that you run against your ipam management console, yielding a list of mac addresses from the past <whatever> timeframe. Yikes, that's a terrible example - you can think of something more applicable, right? I'm trying to convey a situation where you reach out to some remote host to generate a report, then prune the report to yield a column of only the attributes you need to search the local log system for.

But wait - the report generates MAC addresses with [0-9a-f] and your local filesystem logs MAC addresses as [0-9A-F] - how do you cope with case sensitive matches? Combine another flag with -f: use -i for case insensitive matches. [Note: many Unix utilities are able to parse compound options (-i -f and -if are parsed as equivalent invocations).]

ssh ipam show-most-recent-leases -maconly | grep -if - /var/log/radius        

One word of warning - take care how many elements are in your search string, since your runtime is now exposed to a possible O(n^2) - the number of elements in your search definition, multiplied by the number of lines in your target to be searched. (Further reading: Big O on Wikipedia)

What are some crazy helpful options you've discovered in grep? How do you elevate your search game? Light me up in the comments!

Happy hunting!