Linux: Megaraid SAS
From ReceptiveIT
Contents |
Introduction
These instructions relate to the LSI Megaraid SAS based raid controllers, and my adventures with them using Ubuntu Linux 64-bit.
The driver for these cards is quite stable, and is in the mainstream kernel (megaraid_sas). There are two management tools available for these cards, the command line MegaCli and the graphical Raid Web Console 2.
Monitoring
Monitoring your raid arrays can be done quite easily by using the command line tool, and if you combine this with a good monitoring solutions, like Zabbix, it can give you really good peace of mind.
Common Tasks
How many adapters do we have?
Before we go any further, we should make sure that we have a supported, and recognised card.
> ./MegaCli64 -adpCount Controller Count: 1. Exit Code: 0x01
There, we have 1 controller in our machine
Firmware
From time to time, you need to upgrade the firmware on your adapter. It is quite easy to do this with the command line tool.
Finding Current Firmware Version
> ./MegaCli64 -AdpAllInfo -aAll -NoLog
Adapter #0
==============================================================================
Versions
================
Product Name : Intel(R) RAID Controller SRCSASPH16I
Serial No : P268152008
FW Package Build: 7.0.1-0045
Mfg. Data
================
Mfg. Date : 06/15/08
Rework Date : 00/00/00
Revision No : (
Battery FRU : N/A
Image Versions in Flash:
================
Boot Block Version : R.2.3.12
BIOS Version : MT33
MPT Version : MPTFW-01.18.78.00-IT
FW Version : 1.12.120-0394
WebBIOS Version : 1.1-33d-e_11-Rel
Ctrl-R Version : 1.04-019A
Preboot CLI Version: 01.00-011:#%00001
Flashing Firmware
> ./MegaCli64 -adpfwflash -f 68_fw718.rom -aAll Adapter 0: Intel(R) RAID Controller SRCSASPH16I Vendor ID: 0x1000, Device ID: 0x0411 Package version on the controller: 7.0.1-0045 Package version of the image file: 7.0.1-0074 Download Completed. Flashing image to adapter... Adapter 0: Flash Completed. Exit Code: 0x00
This process puts the new flash into a queue, and it will perform an upgrade at the next boot.
Do we have any degraded drives?
> ./MegaCli64 -AdpAllInfo -aAll -NoLog
Device Present
================
Virtual Drives : 1
Degraded : 0
Offline : 0
Physical Devices : 9
Disks : 7
Critical Disks : 0
Failed Disks : 0
Command Line Help
Intel(R) RAID Command Line Utilities Version 2
Ver 4.00.14 July 01, 2009
(c)Copyright 2009, LSI Corporation, All Rights Reserved.
MegaCli -v
MegaCli -help|-h|?
MegaCli -adpCount
MegaCli -AdpSetProp {CacheFlushInterval -val}|{ RebuildRate -val}
|{PatrolReadRate -val}|{BgiRate -val}|{CCRate -val}
|{ReconRate -val}|{SpinupDriveCount -val}|{SpinupDelay -val}
|{CoercionMode -val}|{ClusterEnable -val}|{PredFailPollInterval -val}
|{BatWarnDsbl -val} |{EccBucketSize -val} | {EccBucketLeakRate -val}
|{AbortCCOnError -val} | AlarmEnbl | AlarmDsbl | AlarmSilence
|{SMARTCpyBkEnbl -val} | NCQEnbl | NCQDsbl -aN|-a0,1,2|-aALL
|{SSDSMARTCpyBkEnbl -val} -aN|-a0,1,2|-aALL
|{MaintainPdFailHistoryEnbl -val} -aN|-a0,1,2|-aALL
|{EnblSpinDownUnConfigDrvs -val} -aN|-a0,1,2|-aALL
|{EnblSSDPatrolRead -val} -aN|-a0,1,2|-aALL
MegaCli -AdpSetProp -AutoDetectBackPlaneDsbl -val -aN|-a0,1,2|-aALL
val - 0=Enable Auto Detect of SGPIO and i2c SEP.
1=Disable Auto Detect of SGPIO.
2=Disable Auto Detect of i2c SEP.
3=Disable Auto Detect of SGPIO and i2c SEP.
MegaCli -AdpSetProp -CopyBackDsbl -val -aN|-a0,1,2|-aALL
val - 0=Enable Copyback.
1=Disable Copyback.
MegaCli -AdpSetProp -LoadBalanceMode -val -aN|-a0,1,2|-aALL
val - 0=Auto Load balance mode.
1=Disable Load balance mode.
MegaCli -AdpSetProp -UseFDEOnlyEncrypt -val -aN|-a0,1,2|-aALL
val - 0=FDE and controller encryption (if HW supports) is allowed.
1=Only support FDE encryption, disallow controller encryption.
MegaCli -AdpGetProp CacheFlushInterval | RebuildRate | PatrolReadRate | BgiRate | CCRate | ReconRate | SpinupDriveCount | SpinupDelay | CoercionMode | PredFailPollInterval | EccBucketSize | EccBucketLeakRate | EccBucketCount | ClusterEnable | BatWarnDsbl | AbortCCOnError | AlarmDsply | AutoDetectBackPlaneDsbl| CopyBackDsbl | LoadBalanceMode | SMARTCpyBkEnbl | SSDSMARTCpyBkEnbl | MaintainPdFailHistoryEnbl | EnblSpinDownUnConfigDrvs | EnblSSDPatrolRead | NCQDsply -aN|-a0,1,2|-aALL | UseFDEOnlyEncrypt | WBSupport -aN|-a0,1,2|-aALL
MegaCli -AdpAllInfo -aN|-a0,1,2|-aALL
MegaCli -AdpGetTime -aN|-a0,1,2|-aALL
MegaCli -AdpSetTime yyyymmdd hh:mm:ss -aN
MegaCli -AdpSetVerify -f fileName -aN|-a0,1,2|-aALL
MegaCli -AdpBIOS -Enbl |-Dsbl | -SOE | -BE | -Dsply -aN|-a0,1,2|-aALL
MegaCli -AdpBootDrive {-Set -Lx}|-Get -aN|-a0,1,2|-aALL
MegaCli -AdpAutoRbld -Enbl|-Dsbl|-Dsply -aN|-a0,1,2|-aALL
MegaCli -AdpCacheFlush -aN|-a0,1,2|-aALL
MegaCli -AdpPR -Dsbl|EnblAuto|EnblMan|Start|Stop|Info| SSDPatrolReadEnbl | SSDPatrolReadDsbl
|{SetDelay Val}|{-SetStartTime yyyymmdd hh}|{maxConcurrentPD Val} -aN|-a0,1,2|-aALL
MegaCli -AdpCcSched -Dsbl|-Info|{-ModeConc | -ModeSeq [-ExcludeLD -LN|-L0,1,2]
[-SetStartTime yyyymmdd hh ] [-SetDelay val ] } -aN|-a0,1,2|-aALL
MegaCli -AdpCcSched -SetStartTime yyyymmdd hh -aN|-a0,1,2|-aALL
MegaCli -AdpCcSched -SetDelay val -aN|-a0,1,2|-aALL
MegaCli -FwTermLog -BBUoff|BBUoffTemp|BBUon|BBUGet|Dsply|Clear -aN|-a0,1,2|-aALL
MegaCli -AdpAlILog -aN|-a0,1,2|-aALL
MegaCli -AdpShutDown -aN|-a0,1,2|-aALL
MegaCli -PDList -aN|-a0,1,2|-aALL
MegaCli -PDGetNum -aN|-a0,1,2|-aALL
MegaCli -pdInfo -PhysDrv[E0:S0,E1:S1,...] -aN|-a0,1,2|-aALL
MegaCli -PDOnline -PhysDrv[E0:S0,E1:S1,...] -aN|-a0,1,2|-aALL
MegaCli -PDOffline -PhysDrv[E0:S0,E1:S1,...] -aN|-a0,1,2|-aALL
MegaCli -PDMakeGood -PhysDrv[E0:S0,E1:S1,...] -aN|-a0,1,2|-aALL
MegaCli -PDHSP {-Set [-Dedicated [-ArrayN|-Array0,1,2...]] [-EnclAffinity] [-nonRevertible]}
|-Rmv -PhysDrv[E0:S0,E1:S1,...] -aN|-a0,1,2|-aALL
MegaCli -PDRbld -Start|-Stop|-ShowProg |-ProgDsply
-PhysDrv [E0:S0,E1:S1,...] -aN|-a0,1,2|-aALL
MegaCli -PDClear -Start|-Stop|-ShowProg |-ProgDsply
-PhysDrv [E0:S0,E1:S1,...] -aN|-a0,1,2|-aALL
MegaCli -PdLocate {[-start] | -stop} -physdrv[E0:S0,E1:S1,...] -aN|-a0,1,2|-aALL
MegaCli -PdMarkMissing -physdrv[E0:S0,E1:S1,...] -aN|-a0,1,2|-aALL
MegaCli -PdGetMissing -aN|-a0,1,2|-aALL
MegaCli -PdReplaceMissing -physdrv[E0:S0] -arrayA, -rowB -aN
MegaCli -PdPrpRmv [-UnDo] -physdrv[E0:S0] -aN|-a0,1,2|-aALL
MegaCli -EncInfo -aN|-a0,1,2|-aALL
MegaCli -PhyInfo -phyM -aN|-a0,1,2|-aALL
MegaCli -PdFwDownload -PhysDrv[0:1,1:2,...] -f <filename> -aN|-a0,1,2|-aALL
MegaCli -LDInfo -Lx|-L0,1,2|-Lall -aN|-a0,1,2|-aALL
MegaCli -LDSetProp {-Name LdNamestring} | -RW|RO|Blocked | WT|WB [-Immediate] |RA|NORA|ADRA
| Cached|Direct | -EnDskCache|DisDskCache | CachedBadBBU|NoCachedBadBBU
-Lx|-L0,1,2|-Lall -aN|-a0,1,2|-aALL
MegaCli -LDGetProp -Cache | -Access | -Name | -DskCache -Lx|-L0,1,2|-LALL
-aN|-a0,1,2|-aALL
MegaCli -LDInit {-Start [-full]}|-Abort|-ShowProg|-ProgDsply -Lx|-L0,1,2|-LALL -aN|-a0,1,2|-aALL
MegaCli -LDCC {-Start [-force]}|-Abort|-ShowProg|-ProgDsply -Lx|-L0,1,2|-LALL -aN|-a0,1,2|-aALL
MegaCli -LDBI -Enbl|-Dsbl|-getSetting|-Abort|-ShowProg|-ProgDsply -Lx|-L0,1,2|-LALL -aN|-a0,1,2|-aALL
MegaCli -LDRecon {-Start -rX [{-Add | -Rmv} -Physdrv[E0:S0,...]]}|-ShowProg|-ProgDsply
-Lx -aN
MegaCli -LdPdInfo -aN|-a0,1,2|-aALL
MegaCli -LDGetNum -aN|-a0,1,2|-aALL
MegaCli -CfgLdAdd -rX[E0:S0,E1:S1,...] [WT|WB] [NORA|RA|ADRA] [Direct|Cached]
[CachedBadBBU|NoCachedBadBBU] [-szXXX [-szYYY ...]]
[-strpszM] [-Hsp[E0:S0,...]] [-AfterLdX] [-Force]|[FDE|CtrlBased] -aN
MegaCli -CfgEachDskRaid0 [WT|WB] [NORA|RA|ADRA] [Direct|Cached]
[CachedBadBBU|NoCachedBadBBU] [-strpszM]|[FDE|CtrlBased] -aN|-a0,1,2|-aALL
MegaCli -CfgClr -aN|-a0,1,2|-aALL
MegaCli -CfgDsply -aN|-a0,1,2|-aALL
MegaCli -CfgLdDel -LX|-L0,2,5...|-LALL -aN|-a0,1,2|-aALL
MegaCli -CfgFreeSpaceinfo -aN|-a0,1,2|-aALL
MegaCli -CfgSpanAdd -r10 -Array0[E0:S0,E1:S1] -Array1[E0:S0,E1:S1] [-ArrayX[E0:S0,E1:S1] ...]
[WT|WB] [NORA|RA|ADRA] [Direct|Cached] [CachedBadBBU|NoCachedBadBBU]
[-szXXX[-szYYY ...]][-strpszM][-AfterLdX]|[FDE|CtrlBased] -aN
MegaCli -CfgSpanAdd -r50 -Array0[E0:S0,E1:S1,E2:S2,...] -Array1[E0:S0,E1:S1,E2:S2,...]
[-ArrayX[E0:S0,E1:S1,E2:S2,...] ...] [WT|WB] [NORA|RA|ADRA] [Direct|Cached]
[CachedBadBBU|NoCachedBadBBU][-szXXX[-szYYY ...]][-strpszM][-AfterLdX] | -aN
[FDE|CtrlBased] -aN
MegaCli -CfgAllFreeDrv -rX [-SATAOnly] [-SpanCount XXX] [WT|WB] [NORA|RA|ADRA]
[Direct|Cached] [CachedBadBBU|NoCachedBadBBU] [-strpszM]
[-HspCount XX [-HspType -Dedicated|-EnclAffinity|-nonRevertible]]| -aN
[FDE|CtrlBased] -aN
MegaCli -CfgSave -f filename -aN
MegaCli -CfgRestore -f filename -aN
MegaCli -CfgForeign -Scan | [-SecurityKey sssssssssss] -aN|-a0,1,2|-aALL
MegaCli -CfgForeign -Dsply [x] | [-SecurityKey sssssssssss] -aN|-a0,1,2|-aALL
MegaCli -CfgForeign -Preview [x] | [-SecurityKey sssssssssss] -aN|-a0,1,2|-aALL
MegaCli -CfgForeign -Import [x] | [-SecurityKey sssssssssss] -aN|-a0,1,2|-aALL
MegaCli -CfgForeign -Clear [x]|[-SecurityKey sssssssssss] -aN|-a0,1,2|-aALL
x - index of foreign configurations. Optional. All by default.
MegaCli -AdpEventLog -GetEventLogInfo -aN|-a0,1,2|-aALL
MegaCli -AdpEventLog -GetEvents -f <fileName> -aN|-a0,1,2|-aALL
MegaCli -AdpEventLog -GetSinceShutdown -f <fileName> -aN|-a0,1,2|-aALL
MegaCli -AdpEventLog -GetSinceReboot -f <fileName> -aN|-a0,1,2|-aALL
MegaCli -AdpEventLog -IncludeDeleted -f <fileName> -aN|-a0,1,2|-aALL
MegaCli -AdpEventLog -GetLatest n -f <fileName> -aN|-a0,1,2|-aALL
MegaCli -AdpEventLog -GetCCIncon -f <fileName> -LX|-L0,2,5...|-LALL -aN|-a0,1,2|-aALL
MegaCli -AdpEventLog -Clear -aN|-a0,1,2|-aALL
MegaCli -AdpBbuCmd -aN|-a0,1,2|-aALL
MegaCli -AdpBbuCmd -GetBbuStatus -aN|-a0,1,2|-aALL
MegaCli -AdpBbuCmd -GetBbuCapacityInfo -aN|-a0,1,2|-aALL
MegaCli -AdpBbuCmd -GetBbuDesignInfo -aN|-a0,1,2|-aALL
MegaCli -AdpBbuCmd -GetBbuProperties -aN|-a0,1,2|-aALL
MegaCli -AdpBbuCmd -BbuLearn -aN|-a0,1,2|-aALL
MegaCli -AdpBbuCmd -BbuMfgSleep -aN|-a0,1,2|-aALL
MegaCli -AdpBbuCmd -BbuMfgSeal -aN|-a0,1,2|-aALL
MegaCli -AdpBbuCmd -SetBbuProperties -f <fileName> -aN|-a0,1,2|-aALL
MegaCli -AdpFacDefSet -aN
MegaCli -AdpFwFlash -f filename [-NoSigChk] [-NoVerChk] -aN|-a0,1,2|-aALL
MegaCli -AdpGetConnectorMode -ConnectorN|-Connector0,1|-ConnectorAll -aN|-a0,1,2|-aALL
MegaCli -AdpSetConnectorMode -Internal|-External|-Auto -ConnectorN|-Connector0,1|-ConnectorAll -aN|-a0,1,2|-aALL
MegaCli -PhyErrorCounters -aN|-a0,1,2|-aALL
MegaCli -DirectPdMapping -Enbl|-Dsbl|-Dsply -aN|-a0,1,2|-aALL
MegaCli -PDCpyBk -Start -PhysDrv[E0:S0,E1:S1] -aN|-a0,1,2|-aALL
MegaCli -PDCpyBk -Stop|-ShowProg|-ProgDsply -PhysDrv[E0:S0] -aN|-a0,1,2|-aALL
MegaCli -PDReprovision -PhysDrv[E0:S0,E1:S1,...] | [-Force] -aN|-a0,1,2|-aALL
MegaCli -DestroySecurityKey | [-Force] -aN
MegaCli -CreateSecurityKey -SecurityKey sssssssssss | [-Passphrase sssssssssss] |[-KeyID kkkkkkkkkkk] -aN
MegaCli -ChangeSecurityKey -OldSecurityKey sssssssssss | -SecurityKey sssssssssss|
[-Passphrase sssssssssss] | [-KeyID kkkkkkkkkkk] -aN
MegaCli -GetKeyID [-PhysDrv[E0:S0]] -aN
MegaCli -SetKeyID -KeyID kkkkkkkkkkk -aN
MegaCli -VerifySecurityKey -SecurityKey sssssssssss -aN
MegaCli -GetPreservedCacheList -aN|-a0,1,2|-aALL
MegaCli -DiscardPreservedCache -Lx|-L0,1,2|-Lall -aN|-a0,1,2|-aALL
sssssssssss - It must be between eight and thirty-two
characters and contain at least one number,
one lowercase letter, one uppercase
letter and one non-alphanumeric character.
kkkkkkkkkkk - Must be less than 256 characters.
Note: The directly connected drives can be specified as [:S]
Wildcard '?' can be used to specify the enclosure ID for the drive in the
only enclosure without direct connected device or the direct connected
drives with no enclosure in the system.
The following options may be given at the end of any command above:
[-Silent] [-AppLogFile filename] [-NoLog] [-page[N]]
N - Number of lines per page.
Exit Code: 0x00
