How do you troubleshoot performance issues in a Type 1 hypervisor?

This may also vary on other aspects as well, Let say if on the specific patterns VM performance is compromising and rest all the day or week it's performance is good, may be at the backend there are some scheduled jobs running like backup which comsumes most of the memory, and CPU. VM running on high latency disks.. That is HDDs not SDDs Bad Network architect design.

> What operation is slow? Read/write/access > Was it working as expected ever before? Performance is all about expectations - so how slow is it now ? How fast was it before? > Exact timestamps of the slowness seen? how are they actually measuring the slowness ? > What errors are matching the timestamp of actual slowness seen? > Is the slowness actually reported for any operation or it is being complained just by observing some stats that look high like ( high latency or cpu usage ) in any of the reports? More questions accordingly will help narrow down the issue. Performance issues are never straight forward ??

As a VMware guy, I would focus on ESXi type 1 hypervisor. Troubleshooting ESXi hosts performance are some of the hardest nuts to crack due to several different issues such as network latency, memory over-commitment, CPU constraints or storage overused/latency. To Identify the symptoms! You need to check the 4 most important aspects: CPU, Memory, Network & Storage.

This is a pretty general question. If performance is poor are ypo running something besides the hypervisor on that machine? The speed of the hypervisor machine hardware can slow everything down. One of the things I know is better questions , better answers. give us motr sympyoms, configuration and we could maybe answer it.

First, you should check the "performance monitoring" for heavy hitters which is built-in vSphere client. Negotiability VMware performance issues can show up on services or applications running on VMs with slow respond or unresponsive. For a more in-depth assessment, use the "esxtop" command to verify if the ESXi host is overloaded. PowerCLI for VMware vSphere is another powerful tool, based on Microsoft PowerShell. Also, there is a great "vSCSIStats" utility if storage system is working too slow. Moreover, "VMware Aria Operations" (formerly vRealize Operations) gathers a deep and detailed metrics to monitor, discover and analyze networks and applications across clouds. – covering everything from hosts to VMs and Applications.

VMware Performance Monitoring: Try VM’s virtual disk’s read and write latencies, I would recommend to use alarms wherever possible so you’re always on top of any performance issue. The esxtop utility: CPU, memory, storage adapter, storage device, VM storage, network, interrupt, and CPU power panels. [root@localhost:~] esxtop will open the interactive screen of esxtop, by default. VMware Aria Operations: Create one or more nodes in a cluster. The primary node and the HA/CA replica node as the data node. Deploy only from vCenter Server. Use vscsiStats for Storage: Results are reported per VMDK IO size, Seek distance, Outstanding IOs, Latency (in microseconds) Two steps; (1) Start statistics collection. (2) View accrued statistics.

Before deploying a type 1 hypervisor, I would recommend the following: * Check that all hardware in the cluster is on the "HCL". * The hardware must meet the minimum config supported by VMware. * Test given & available CPU, Memory, Network & Storage resources for 3 days. * Check CPU compatibility for vMotion/Storage vMotion, Fault Tolerance and DRS. * Check Persistent Memory (PMem) configuration "Intel Optane DC" & "NVDIMM-N". * Check Storage performance: WLD, hardware, vendor, RAID level, cache size, stripe size. * For VMware vSAN deployment consider the advantages of all-flash & not a hybrid vSAN. * For VM encryption get hardware supporting AES-NI processor. * Consider VLANs/VPNs to provide a way to logically configuring networks "QoS"

A lot of admins ignore this rule "documentation" what this info should consist of: - ESXi names/IP addresses. - ESXi hardware description. - Storage and datastore configuration. - NIC vendor, driver and version. - Root/Admin/Service password accounts. - Storage & Switches IP, Firmware, PW/UN. - VLAN settings and network configuration. - Storage Array LUN, RAID, # of drives. Make sure you install VMware Tools on the Virtual Machine (VM) ~ Improves mouse response and video performance. ~ Synchronize time with the ESXi host. ~ Enables Guest Introspection for NSX/agentless antivirus. ~ Installs the VMXNET3 driver for improved network. ~ Allows copy and paste between the VM and desktop. ~ Virtual Machine (VM) heartbeat.