What the heck is Application Performance Monitoring!

What is APM so called Application Performance Monitoring

It is all about gathering data, actually gathering millions of data points from your live production software and use it to change and improve the way your application and business operate. This data will act as feedback for your analysis and you can make changes accordingly in your software. Inshort it is full visibility into:

• Performance of your Application (Mobile App/Web App/ Website) covers issues with application code, application dependencies, key transactions
• Customer Experience Analytics – Any lag, delay while browsing the site and much more
• Business Metrics - CTR, Sessions, Traffic, Conversions, Cart checkout rate, Total orders, Profit/Loss etc.

Why we need APM?

1. For Incident resolution and Management

IT Teams purchase APM on the grounds that they have an excessive number of incidents with their product that lead to downtime that business have to face and suffer, and these incidents take too long to even consider resolving.

APM significantly reduces the incidents that happen, and lessens the time it takes to resolve them when they take place. The mean (average) time taken to resolve an incident is called Mean Time To Resolution (MTTR). This is the only metrics that matters in APM world. The entirety process and APM solution is implemented to diminish MTTR for clients. In IT time driven world, MTTR plays a major role because in few seconds, companies lose thousands of dollars of revenue because of downtime.

In 2017, Southwest Airlines’ website gone down for 4 days and their internal IT systems going down for a day — 900 flights canceled leaving passengers stranded, stock price taking a hit – that how much critical is IT dependency in today’s time.

There are three main Steps to resolve any possible incident:

1. Monitoring: Recognizing that there is an issue with a product (Application).
2. Problem Isolation: Narrowing down in which component(s) of the application is causing the issue.
3. Root Cause Analysis: Doing a noteworthy speculation about what's going on with the application or possible root cause of issue.

2. For monitoring and measuring your business:

Peter Drucker is one of the greatest Management Thinker of all time and he said

“If you can’t measure it, you can’t improve it.” 

For example, it’s nearly impossible to lose weight without stepping on a scale once in a while to measure your results – if you don’t, you have no idea if you are succeeding or not.

You have to ask the most import questions like:

• Do you know the number of new website visitors you received in the last 30 days?
• Do you know what percentage of them turned into new paying customers?
• Which is the most visited Page/Product on our website?
• What is the average time that your users are spending on which page of your website?
• What are the issues your customers are facing while navigating your website?
• How much time, it takes to load a certain page of site/app?
• Is my website is responding well to my customers or not. If not, why?
• Why there is certain decline in number of visitors on my site?
• How much time does it take to execute the checkout transaction?

That is why there is need of Analytics and to answer these complex questions and to do the root cause analysis, we need APM.

The reason why APM solutions exist because to understand WHY your application or website is slow or producing errors while navigating.

Once you understand WHY, you can fix that problem permanently.

Slowness and Errors in any application can be caused by coding errors or bugs in the app, database calls slowdowns, hosting or network issues, or from end user OS or device or browser issues and APM solution does not only identify these issues but tell you the root cause so that your Dev team can fix it without taking much time. For more, you can always reach out to me, I would be happy to help you.

So here are the top benefits of implementing APM in your organization:

• Satisfy your customers by improving seamless Customer Experience (UX) on your website and App.
• Make Better Software and applications that does not easily breakdown with casual bottlenecks.
• Improve and Optimize the resources you are using for your application like Cloud usage, hosting, team members.
• Foster Collaboration across the Dev team, Product team, Operation team, Support team of your organization.
• Derive great business outcomes by better understanding your customers and improving customer experience.

Let’s Dive and see how Performance Monitoring works. For that let's understand, what are types of monitoring that you can do and what problem you can solve with these monitoring.

Types of Monitoring:

1. Application Performance monitoring
2. End User Monitoring (RUM)
3. Synthetic Monitoring ‘
4. Infrastructure Monitoring
5. DB Monitoring
6. Network monitoring
7. Monitoring External endpoints (API)
8. Monitoring for Mobile Applications
9. Log Monitoring

Application Performance Monitoring

Monitoring a system still means to monitor the 4 Golden Signals. APM give you complete end to end visibility of your Application Performance, bottlenecks and dependencies.

Whether you want to fix simple bug or do complex refactoring of one of your bottleneck, APM guides you to where to look for the problem and help you to focus your time and money on right cause of the problem.

What Metrics you can see in your Dashboard: Here are the basic ones

• HTTP Response Time or Web Response Time
• Throughput or number of page views per second/min (User Traffic)
• Number and type of Transactions and Transaction details like Most time consuming transactions, or slowest response time transactions etc.
• You can even drill down a specific transaction to code level from transaction trace. - This will help you to find if there is issue with specific code or Database call or external API.
• Error Analytics - To check, what are the current issues with your Application and which error is creating most problems.

You can get data in Table form, you can create Area charts and line charts, Bar charts and Pie Charts, Gauge charts and can set even warnings and alerts. You can also create other charts like Histogram, Funnel charts, Heat maps, Percentile charts and trend comparisons.

End User Monitoring (RUM)

Now, the first interaction between you and your customers is through your website, once you acquire that visitor (from organic or paid source), If they come to your website and experience poor performance and lags and bad User interface (UX) and User Experience (UX), they just abandon your website and directly go to your competition. So it is very critical and important to monitor, how your website is performing.

End User Monitoring or Real user monitoring is way to understand and analyze, how your customers engage with your website so that you can troubleshoot and fix any front end issues that your customers are facing.

Real User Monitoring allows you to see how actual users are experiencing your site in real time. You can monitor their activities like:

• Pages they visit – Web Pages or App Pages
• Throughput - Website traffic
• Page load time
• HTTP request time
• SPA (Single Page Application) Load Time
• SPA ThroughPut
• AJAX response time
• Page with JS Errors
• Recent Session Traces
• Time of day they access your website
• User Data based on – Geographical location, Browser Type, App version, Device type and
• Operating System and Connection Type (2G, 3G, 4G, Wi-Fi, ISDN etc.)

Page Load Time Graph

SPA Load time Graph: Single Page Application load time captures both the page load time and time taken to complete any other activity after the initial page load – including asynchronous elements like JAVAScript and AJAX calls.

Recent Session Traces waterfall model Each session trace is a detail recording of single user actions on a page. They show: a) What user did on website b) How the page has performed in second by second detail. For help, you can always reach out to me,

Synthetic Monitoring

Synthetic Monitoring is an artificial simulation of real user traffic to your website, where you kind of test your application and website, critical business transactions and API End points to make sure that they are ready to go production environment and you can address the issues before they affect your end users.

If you have big event or sale coming up planned for your website and application, and you know there will be significant increase in traffic, I advise you to always use synthetics to verify that your website can take load of that magnitude and that critical functions are working correctly.

Synthetic Monitoring dashboard

With Scripted browser monitors, you can simulate real users and doing anything from basic load page to more complex multi-steps actions like cart check out, website search features and logging into users account.

You can do synthetic monitoring to keep tabs on the health of endpoints your App relies on, even 3rd party endpoints. Knowing immediately when 3rd party is experiencing issues can give you time to take actions and prevent cascading problems within your architecture.

Types of Synthetic monitoring:

-         Availability Monitoring (also called uptime monitoring)

-         Web Performance Monitoring

-         Transaction Monitoring

You can know about lot of metrics and come to conclusion as per your site behavior for the simulated users and take actions like troubleshooting and fixing the issues that you found out while doing synthetic monitoring.You can see things like Page Load time, Availability, Slowest results or Slowest URLs and Failures on your APM Software. For help, reach out to me,

Database Monitoring

Here you can monitor the database performance from database dashboard of your APM Software. Here you can find:

1.     How much your App spends making database calls

2.     How long the DB calls take to complete?

3.     How many calls are being made in total or per operation.

4.     Your slowest DB Queries

Monitoring External Application (APIs)

You can monitor the performance of the external services that your app relies on from the API Monitoring. This will show the transaction performance details for the request made from your app to any outside service. For example: Your Social handles like Facebook page, Twitter handle, Calendly.com or any third party app or service - depends upon your application. These can be anything from API Calls to other websites or social media platforms or Apps within your own architecture.

External API Monitoring graph

Infrastructure Monitoring

Infrastructure Monitoring gives you visibility how your app is performing on hosts, servers and supporting network and storage resources by displaying data about Response time, Throughput, CPU Usage and Memory per host.

You get a real time data of entire host ecosystem from your own data centers to public cloud services to hybrid and private Cloud. You can even optimize your Cloud as per your application requirement and that keep your Cloud Cost down. You can fine-tune your databases, modify your storage usage, better configure your load balancer, or even re-architecture your applications.

Infrastructure monitoring

You can also know if your all hosts are running on correct software versions and packages. You can see complete list of the events that happened and changed in your infrastructure so that you can analyze what changes in infrastructure have made what impact on whole system.

You can easily integrate your cloud provider with your APM to monitor the services on Cloud. You can also easily integrate external services like Kubernetes, MySQL, Redis, Kafka and other too.

This also shows Service Maps where you can see your app’s connections and dependencies including databases and external services.

Service Maps for application infrastructure

Monitoring for Mobile Applications

In these times, for most of the business, the mobile traffic comprises of average of 50 to 70 Percent of overall traffic. For this, having your mobile app user friendly that provide great customer experience is essential part of your business.

In order to make sure this you can monitor any type of native mobile application (Android and iOS) on an APM performance Tool. You can do testing and monitoring of apps which are in testing phase or in production environment and live.

You can monitor metrics like:

1.      Execution Time or App response Time

2.      Recent Crashes and Crash Rate

3.      Crash reports and Stack Traces

4.      Page load time by App pages

5.      Interaction trail or Session recording of each user section

6.      Slowest interactions with drill down ability to code level.

7.      Traffic by OS, App version, geographical location, devices, carrier and a lot more.

Dashboard showing app crashes and crash rate

Log Monitoring and Management

Logs provide you detailed description of every event happening the system. For example, an incident happened and suddenly one of the page of your website is not loading for any reason. Now with system logs, you can check and investigate the cause of incident, by looking into the logs for that particular time when incident happened.

Logs give you complete visibility into application and infrastructure performance events and errors so that it will help you to reduce mean-time-to-resolve (MTTR) and quickly troubleshoot production incidents. You can search specific keywords (could be related to incident or name of page or module, API etc), with time constraints, alerts, and co-location of application, infrastructure, and log data. For more reach out to me, happy to help.

Log Management Dashboard

Why should you do Log Monitoring:

• Instantly search through your all logged events so that you can do RCA of incident.
• Visualize your log data directly from the Logs UI.
• Use logging data to create custom charts, dashboards, and alerts.
• Troubleshoot performance issues without switching between tools.

Performance Alerts

Performance Alerts are the way to get notified of the performance issues happening in your application or website so that you can address them immediately and keep seamless experience for your customers.

 You can create alerts anything from JS Errors, Ajax response time, Key transactions, Error rates and external service calls, CPU utilization rates to the custom metrics alerts that you can create of your own on any metrics that APM monitors. You can even get alerts to more advanced metrics like AWS Usage, Estimated AWS Billing Costs and Kubernetes Cluster Performance and many more.

Performance Alerts 

In order to create an alert, you have to choose a threshold value (Static or dynamic) for the given metric and system will alert you when metric deviate from that threshold baseline. 

You can decide, who, when and for what will be notified from your team for the alerts Notification. These alert notifications can send via Email address, Slack Channel, or event management systems like Pagerduty, Victorops and Webhooks etc.

Custom Metrics and Reports

In addition to all this, APM Tools also help you to create custom Metrics and Report as per your own Dev team and business team requirement. For example, you can also add new metrics like Product Shipping status and details, Inventory Status and Alerts, etc. Custom reports also include the comparison of measured data with any external data or parameters. Custom reports can totally be built on Client requirements with suitable filters and representations. You can create multiple custom reports in your APM Dashboard. These reports can also help identify under and over utilized severs and storage, understand performance trends, predict future trends based on historical data and thereby plan resource reallocation and infrastructure upgrades.

APM Installation:

Well, the procedure of installing APM varies and depends upon the technologies you have used to create your App or website and if you are using microservices and on what frameworks your Application or website is based on.

However generally for native mobile application, you just directly install the agent or install the SDK file in your Application server that will be connected to your monitoring Dashboard. For website monitoring, either APM agent or Javascript snippet will be added on your website page.

How can I get started?

If you wanted to implement APM for your website or mobile application, it typically depends upon few factors like, what business you are into, what is the frameworks/technologies your website is based on and what current issues you are facing with your application or website and what is your general approach for Software/ Application life cycle, are you using waterfall approach, Agile approach or Devops approach and few more specific factors. For this, you can always reach out to me, I would be happy to help you :)

If you find this valuable, please share with your network and also let me know if you want me to write about specific topic or concern. For more content on APM, please connect with me.