Enhancing Oracle DBA Disk Management with a StorCLI-Based Script

Operating system administrators consistently utilize remote management consoles for efficient oversight of physical servers. Access to these consoles is typically provided through web interfaces or command-line interfaces, secured by unique credentials.

Various manufacturers offer their proprietary solutions, including:

• SuperMicro's IPMI (Intelligent Platform Management Interface),
• Hewlett Packard's iLO (Integrated Lights-Out),
• DELL's iDRAC (Integrated Dell Remote Access Controller).

These advanced platforms grant administrators the capability to conduct a wide range of remote server management tasks such as system monitoring, remote power management, real-time console access, and firmware updates. An additional crucial feature common to all the aforementioned consoles is their sophisticated interface for disk configuration.

For database administrators (DBAs), the ability to manage RAID configurations and virtual disks is particularly crucial. DBA involvement in the architectural design of a server's disk system plays a pivotal role in:

• Maximizing Performance: Selecting the appropriate RAID configuration for various disk groups, determining strip size, and configuring caching are all critical factors in constructing the ASM disk layer efficiently.
• Planning for Future Expansion: It is essential to map out the complete disk space, considering the database's growth and potential fluctuations in load.
• Ensuring System Robustness: Assigning hot spare disks to RAID virtual groups with the correct replacement options is vital for maintaining system reliability.
• Managing Firmware Versions of Controllers: DBAs may review the firmware description of RAID controllers for bugs and constraints that could impact disk performance.

Data center administrators might employ specific server templates for Oracle databases; however, production servers invariably require close collaboration with the database architect/administrator to achieve an optimally tuned setup. Throughout my experiences in various organizations, I have observed instances where Oracle DBAs were not provided access to these critical remote server management technologies. Such restrictions highlight a considerable gap in integrating database management with server administration, potentially compromising operational efficiency and strategic system planning.

An alternative to remote consoles could be the use of operating system-level administration tools, such as the MegaRaid package. MegaRAID for Linux encompasses a suite of software tools and drivers aimed at managing RAID storage configurations on Linux-based systems, especially on servers outfitted with LSI (now a Broadcom subsidiary) MegaRAID hardware RAID controllers. These tools allow administrators to create, monitor, manage, and maintain RAID configurations, vital for ensuring data redundancy, performance, and reliability in enterprise and data center settings.

I've considered developing a simple script that would display the most crucial parameters for overseeing server and disk provisioning based on storcli. Running storcli requires "root" privileges, but obtaining these is generally more feasible, or one could request a Unix administrator to send the script's output instead of dozens of screenshots from IPMI/iLO/iDRAC. This approach is particularly useful when specific options are scattered across various interface windows.

Please find the script code below:

#!/bin/bash
cOracleHome="/oracle/GRID/19.0.0";

lvCommand1="/opt/MegaRAID/perccli/perccli64";
lvCommand2="/sbin/storcli";
if [[ -f $lvCommand1 ]]; then 
  cCommand=$lvCommand1; 
  echo "[INFO] Script will be using 'perccli' for RAID console operations."
else
  if [[ -f $lvCommand2 ]]; then 
    cCommand=$lvCommand2; 
      echo "[INFO] Script will be using 'storcli' for RAID console operations."
  else
    echo "[ABORT] MegaRAID commands (storcli/perccli) were not found. Exiting...";
    exit 1;
  fi
fi

lvASMCheck=$($cOracleHome/bin/asmcmd afd_state 2>&1 | grep ASMCMD-9526 | wc -l);
lvASMLibCheck=$(oracleasm status 2>&1 | grep "Checking if ASM is loaded" | wc -l);

if [[ $lvASMCheck -eq 1 ]]; then
  echo "[INFO] Oracle AFD was found to be loaded.";
  lvASMDisksAll=$($ORACLE_HOME/bin/asmcmd afd_lsdsk | awk '{print $2" "$1}' | tail -n +4); 
  lvLib="AFD" ;
else
  if [[ $lvASMLibCheck -eq 1 ]]; then
    echo "[INFO] Oracle ASM lib was found to be loaded.";
    lvASMDisksAll=$(oracleasm listdisks | xargs oracleasm querydisk -p | grep LABEL | awk '{print $1" "$2}' | sed -e 's/: LABEL="/ /g' -e 's/"https://g'); 
    lvLib="ASMlib" ; 
  else
    echo "[ERROR] Neither AFD, neither ASM lib were found to be loaded. Script will show no ASM disk info.";
  fi  
fi

printf -- '-%.0s' {1..160};printf '\n'
printf " %-3s | %-3s | %-7s | %-9s | %-10s | %-9s | %-11s | %-9s | %-20s | %-20s | %-20s \n" "GID" "DID" "RAID" "ReadCache" "WriteCache" "DiskCache" "Size" "StripSize" "Name" "Physical Disk" "$lvLib Disk";
printf -- '-%.0s' {1..160};printf '\n'
while read -r lvDiskID lvRAID lvCache lvSize lvSizeUnit lvName; do
  if [[ -z $lvDiskID ]]; then
    continue;
  fi
  lvDG=$(echo $lvDiskID | grep -oP '^[0-9]*');
  lvVD=$(echo $lvDiskID | grep -oP '(?<=\/)[0-9]*');
  lvWriteCache=$(echo $lvCache | grep -oP 'WT|FWB|WB');
  lvReadCache=$(echo $lvCache | grep -oP 'NR|R');
  lvDetails=$($cCommand /c0/v${lvVD} show all);
  lvStripSize=$(echo "$lvDetails" | grep -oP '(?<=Strip Size \= ).*');
  lvPhysDisk=$(echo "$lvDetails" | grep -oP '(?<=OS Drive Name \= ).*');
  lvDiskCaching=$(echo "$lvDetails" | grep -oP '(?<=Disk Cache Policy \= ).*'  | sed -e 's/Disabled/-/g' -e 's/Enabled/+/g');
  lvASMDisk=$(echo "$lvASMDisksAll" | grep "$lvPhysDisk" | awk '{print $2}');
  if [[ $lvLib == "ASMlib" ]]; then
    lvPartition=$(echo "$lvASMDisksAll" | grep "$lvPhysDisk" | awk '{print $1}' | grep -oP '[0-9]*$');
    if [[ ! -z "$lvPartition" ]]; then lvPhysDisk="${lvPhysDisk}(${lvPartition})"; fi;
  fi
  printf " %-3s | %-3s | %-7s | %-9s | %-10s | %-9s | %-11s | %-9s | %-20s | %-20s | %-20s \n" "$lvDG" "$lvVD" "$lvRAID" "$lvReadCache" "$lvWriteCache" "$lvDiskCaching" "$lvSize $lvSizeUnit" "$lvStripSize" "$lvName" "$lvPhysDisk" "$lvASMDisk"
done < <($cCommand /c0/vall show | grep -P '^[0-9]\/[0-9]{1,3}.*' | sed -e 's/Virtual Disk/VirtualDisk/g' | sort -t'/' -k1,1n -k2,2n | awk '{print $1" "$2" "$6" "$9" "$10" "$11}');
printf -- '-%.0s' {1..160};printf '\n'

printf "%s\n" "ReadCache: R=Read Ahead Always | NR=No Read Ahead;"
printf "%s\n" "WriteCache: WB=WriteBack | FWB=Force WriteBack | WT=WriteThrough;"        

Sample output:

Script Workflow Logic:

1. Specifies the location of the Oracle GRID home.
2. The script checks for the presence of storcli/perccli in typical locations and selects the first one available.
3. The script identifies the current disk provisioning technology (ASMlib, AFD).
4. Iteratively gathers important attributes of all disks, such as:

• RAID group number
• Virtual disk ID
• RAID type
• Read cache type
• Write cache type
• Disk cache
• Disk size
• Strip size
• Virtual disk name
• Physical disk at the OS level
• ASM disk name

(*) It's important to note that not all versions of storcli display the “OS Drive Name” parameter, so instead of using a simple command:

lvPhysDisk=$(echo "$lvDetails" | grep -oP '(?<=OS Drive Name = ).*');        

It may be necessary to use mapping based on SCSI ID:

lvSCSIID=$(echo "$lvDetails" | grep -oP '(?<=SCSI NAA Id = ).*'); lvPhysDisk=$(lsblk -o NAME,SERIAL | grep "${lvSCSIID}" | awk '{print $1}');"        

I use this script to monitor the status of disks and check critical system parameters after changes. It also reduces the possibility of errors by comparing ASM disks with the corresponding virtual disks, meaning there's a lower chance of incorrectly assigning a disk to the wrong group.

(!) Please note that all scripts are written for conditions that may not be relevant to the standards of a specific technical implementation. Always check the code for compliance and first test in a non-productive environment.

Conclusion:

This article illustrates the pivotal role of a StorCLI tool in enhancing Oracle DBA's capabilities for disk and server management. By automating the monitoring and alignment of disk parameters, the script not only streamlines the provisioning process but also significantly mitigates the risk of misconfigurations. This ensures optimized system performance, future-proof expansion planning, and robust system reliability.