Multi-Segment Analysis in LiveWire

Isn't that a great looking screenshot? I knew it would get some eyeballs so I decided to write about it. Those are packets traveling from one segment to the next, and very visually showing the increasing time, and thus the latency between each segment of the network.

This is a new version of a feature in LiveAction LiveWire called Multi-Segment Analysis or MSA. LiveAction has had MSA for years in Omnipeek for Windows, but what's new is that now it is all built into the LiveWire server software and the UI is all web based in Omnipeek for Web. The Omnipeek for Web client is really amazing, great looking, easy to use, and is quickly ramping up to provide all of the features of Omnipeek for Windows, and in many ways has surpassed it. This new version of MSA is a great example of that.

But let's back up a bit, and lay some groundwork. What is Multi-Segment Analysis or MSA, and why is it more than just a pretty picture? MSA is the ability to find the same flows captured across different segments of the network at the same time, and stitch them together to see all kinds of information about the individual packets traveling back and forth between the client and server. This type of analysis is useful for understanding where along the path of a packet there is network latency, or where a packet gets dropped.

MSA is great for network latency in particular, because application latency would be the amount of time it took a request to respond back with the data. At the flow level you can see the overall application and network latency with just one capture point, because the analysis is 2-way analysis, or the amount of time a request packet goes out and a response packet comes back. MSA on the other hand, is 1-way analysis because it measures the amount of time it takes for each packet to reach the next capture point. MSA does though measure 1-way latency of packets going in both directions. This makes it possible to analyze whole transactions, like the SYN-SYN-ACK three way hand shake shown at the top of the screenshot above.

To perform Multi-Segment Analysis there must be at least two capture points along the path of the flow so the same packets can be captured at each point. But in the screenshot above, you can see there are five capture points. You can also see that there is very little latency between each capture point. What you can't see is how interactive the MSA UI is and how it integrates nicely with the rest of the Omnipeek for Web views and features. Hmm, I should do a video!

So how did I do this? Well, I just so happen to be one of the developers on the LiveWire team so I have lots of LiveWire Edges at my house for development and testing. Actually, I have recently migrated to product management, but I still dabble on the dev side, and enjoy doing all kinds of interesting experiments with LiveWire, and the Edges in particular. In this case, I have daisy chained five Edges using the Bridge ports between my PC and the router connected to the internet. On the LiveWire Edge, the two Bridge ports allow you to tap into the network without the need for a span port, and the necessary configuration or permission required for that.

Once it was all connected, I used the LiveAdmin Web UI on each LiveWire Edge to make sure that the NTP server settings were the same on all of them, and tested the time using the date command to make sure they were all in sync. This is critical, since the whole point is to measure time and the delta between timestamps. But even with NTP servers, the time on each LiveWire can be a bit different so the MSA feature has an option to automatically calculate a time offset between devices or manually specify that offset yourself.

Next, I used one of the Edges as the master, and added all of the Edges to it. This is a great feature in Omnipeek for Web, for accessing large numbers of LiveWire servers, and it is necessary to perform MSA because it defines the set of LiveWire Servers to find the packets on. Once I had the list, I could use it to click on the entries to hop to each LiveWire, and configure all of the LiveWire servers.

The only configuration I had to do on each LiveWire was to create a capture on the Bridge port with the Capture to Disk (CTD) feature enabled. As you can see in my infinite creativity I called them all Bridge Capture. Once it was all set up, and packets were being captured, I started some YouTubes on the PC and let it run. After letting it run for a while it was time to try MSA and see how it looked.

Searching for packets across multiple devices is called a Distributed Forensic Search or DFS. DFS can be used by itself to search for packets across any number of LiveWire Servers using a time range and a filter. With DFS, which LiveWire servers to search are chosen from the master list. There is a group feature which I used so a single click selects them all. So easy! When used alone, the results of the DFS are saved into pcap files which can be downloaded separately or merged into a single pcap file.

When DFS is used as part of the MSA workflow, the resulting packets are pulled into the master server and stitched together to display the interactive ladder diagram displayed in the screenshot above. The analysis can take a bit of time, and the more packets get returned from the search, the longer it can take. But it is definitely worth the wait, and since we all multi-task as a way of life, there are plenty of other things to do in the meantime.

When the analysis is done, the MSA UI is displayed. At the top of the UI is the list of flows that were found based on the time range and filter provided. Clicking on a flow displays the analysis for that flow in the bottom window. The default view is the Flow Map, which displays high level aggregated average, min, and max latency across each capture point.

The Flow Ladder is where the real good stuff is. In the screenshot above the SYN ACK packet is selected, and the cursor is hovering over it which triggers the display of even more information about that packet. The default information shown in each packet is TTL and TCP Flags, but other information about the packet and its relationship to the other packets can be displayed as well. In my setup the TTL does not change, but if the Edges were spread across a larger network, with routers and switches in between the TTL would change, and of course there would be more latency in between each hop.

In the upper right corner of the screen is an Analysis Options button. In the Options Window you can reorder the capture points and change other configurations about the analysis, like the time offset settings.

And remember, all of the analysis happens on the LiveWire Servers, and is stored on the master LiveWire server. This means any number of your colleagues can see it, click it, and use it. Also, any number of MSA projects can be run simultaneously and stored on the server, depending of course on the cpu, disk, and memory resources of the machine running the LiveWire server software.

So that's it for now. I hope you as excited as I am about this feature, and interested enough to want to try it. Oh, one more thing. One of the features I really like about the MSA is the ability to change the time scale from milliseconds to microseconds. By doing this, the latency between the packets becomes more visible and obvious. But there are so many other cool features in MSA and the rest of Omnipeek for Web that you are just going to have to try it out for yourself. So give us a call and lets talk about how LiveWire Edges can dramatically increase the visibility you have to the segments in the datacenter, cloud, on the edge, or anywhere in between. But also free free to just download LiveWire Virtual and spin it up in your favorite VM.