![首頁 Logo Keith ]p !!](/themes/mytheme/logo.png){kind=logo}
最後更新: 2020-05-26
目錄
- Installation (Centos, Debian)
- List Adapter Information
- Logical Drive Information
- Physical Disk Information
- Show Enclosure Info
- RAID Health & Rebuild
- Foreign Configuration
- Cause the front LED of the drive to blink
- Events Log
- BBU
- Terminal Logging(firmware terminal)
- Display configuration
- 時間與時間
- Number of X
- Check Disk SMART
- IT 與 IR Mode
- Disk Cache Policy
- I/O Policy
- Read Ahead
- Stripe Width 與 Stripe Size
- Cheat List
- See Other
Installation
Centos
Download (2020-05)
https://docs.broadcom.com/docs/12351582
unzip Lin_CliKL_8.04.07.zip
unzip MegaCliKLLin.zip
*.rpm
Lib_Utils-1.00-09.noarch.rpm Lib_Utils2-1.00-01.noarch.rpm MegaCliKL-8.04.07-1.noarch.rpm
安裝後 megacli 會放在
/opt/MegaRAID/MegaCli/
check version
./MegaCli64 -v
MegaCLI SAS RAID Management Tool Ver 8.04.07 May 28, 2012
Debian
alien -k MegaCliKL-x.xx-x.noarch.rpm
dpkg -i megaclikl_x.xx.xx-x_all.deb
ln -s /opt/MegaRAID/MegaCli/MegaCli64 /usr/sbin/MegaCli64
Testing
MegaCli64 -V
MegaCLI SAS RAID Management Tool Ver 8.04.07 May 28, 2012
(c)Copyright 2011, LSI Corporation, All Rights Reserved.
Exit Code: 0x00Help
MegaCli64 -h
Troubleshoot
./MegaCli64:
error while loading shared libraries: libncurses.so.5: cannot open shared object file: No such file or directory
[Fix]
ldconfig
Or
apt-get install libncurses5
List Adapter Information
MegaCli64 -adpCount
Controller Count: 1.
MegaCli64 -AdpAllInfo -a0 | less # 亦可以用 "-aALL"
Adapter #0 ============================================================================== Versions ================ Product Name : RAID Ctrl SAS 6G 5/6 512MB (D2616) Serial No : FW Package Build: 12.15.0-0239 # Firmware Package 來, 它已經是 9261-8i 最後的 FW ........ Image Versions in Flash: ================ FW Version : 2.130.403-4660 BIOS Version : 3.30.02.2_4.16.08.00_0x06060A05 ........ PCI Info ================ ........ HW Configuration ================ ... BBU : Present ... Memory Size : 512MB TPM : Absent Upgrade Key : Absent ... Settings ================ Current Time : 13:23:37 5/26, 2020 Predictive Fail Poll Interval : 300sec Interrupt Throttle Active Count : 16 Interrupt Throttle Completion : 50us Rebuild Rate : 30% PR Rate : 20% BGI Rate : 30% Check Consistency Rate : 30% Reconstruction Rate : 30% Cache Flush Interval : 4s Max Drives to Spinup at One Time : 2 Delay Among Spinup Groups : 6s Physical Drive Coercion Mode : Disabled Cluster Mode : Disabled Alarm : Disabled Auto Rebuild : Enabled Battery Warning : Enabled Ecc Bucket Size : 15 Ecc Bucket Leak Rate : 1440 Minutes Restore HotSpare on Insertion : Enabled Expose Enclosure Devices : Disabled Maintain PD Fail History : Enabled Host Request Reordering : Enabled Auto Detect BackPlane Enabled : SGPIO/i2c SEP Load Balance Mode : Auto Use FDE Only : No Security Key Assigned : No Security Key Failed : No Security Key Not Backedup : No Default LD PowerSave Policy : Automatic Maximum number of direct attached drives to spin up in 1 min : 20 Auto Enhanced Import : No Any Offline VD Cache Preserved : No Allow Boot with Preserved Cache : No Disable Online Controller Reset : No PFK in NVRAM : No Use disk activity for locate : No POST delay : 90 seconds Capabilities ================ RAID Level Supported : RAID0, RAID1, RAID5, RAID6, RAID00, RAID10, RAID50, RAID60, PRL 11, PRL 11 with spanning, SRL 3 supported, PRL11-RLQ0 DDF layout with no span, PRL11-RLQ0 DDF layout with span Supported Drives : SAS, SATA ... Status ================ ECC Bucket Count : 0 Limitations ================ Max Arms Per VD : 32 Max Spans Per VD : 8 Max Arrays : 128 Max Number of VDs : 64 Max Parallel Commands : 1008 Max SGE Count : 80 Max Data Transfer Size : 8192 sectors Max Strips PerIO : 42 Max LD per array : 16 Min Strip Size : 8 KB Max Strip Size : 1.0 MB Max Configurable CacheCade Size: 0 GB Current Size of CacheCade : 0 GB Current Size of FW Cache : 0 MB Device Present ================ Virtual Drives : 0 Degraded : 0 Offline : 0 Physical Devices : 1 Disks : 0 Critical Disks : 0 Failed Disks : 0 Supported Adapter Operations ================ Rebuild Rate : Yes CC Rate : Yes BGI Rate : Yes Reconstruct Rate : Yes Patrol Read Rate : Yes Alarm Control : Yes Cluster Support : No BBU : Yes Spanning : Yes Dedicated Hot Spare : Yes Revertible Hot Spares : Yes Foreign Config Import : Yes Self Diagnostic : Yes Allow Mixed Redundancy on Array : No Global Hot Spares : Yes Deny SCSI Passthrough : No Deny SMP Passthrough : No Deny STP Passthrough : No Support Security : No Snapshot Enabled : No Support the OCE without adding drives : Yes Support PFK : Yes Support PI : No Support Boot Time PFK Change : No Disable Online PFK Change : No PFK TrailTime Remaining : 0 days 0 hours Support Shield State : No Block SSD Write Disk Cache Change: No Supported VD Operations ================ Read Policy : Yes Write Policy : Yes IO Policy : Yes Access Policy : Yes Disk Cache Policy : Yes Reconstruction : Yes Deny Locate : No Deny CC : No Allow Ctrl Encryption: No Enable LDBBM : Yes Support Breakmirror : No Power Savings : No Supported PD Operations ================ Force Online : Yes Force Offline : Yes Force Rebuild : Yes Deny Force Failed : No Deny Force Good/Bad : No Deny Missing Replace : No Deny Clear : No Deny Locate : No Support Temperature : Yes Disable Copyback : No Enable JBOD : No Enable Copyback on SMART : Yes Enable Copyback to SSD on SMART Error : Yes Enable SSD Patrol Read : No PR Correct Unconfigured Areas : Yes Enable Spin Down of UnConfigured Drives : Yes Disable Spin Down of hot spares : No Spin Down time : 30 T10 Power State : No Error Counters ================ Memory Correctable Errors : 0 Memory Uncorrectable Errors : 0 ........ Default Settings ================ Phy Polarity : 0 Phy PolaritySplit : 0 Background Rate : 30 Strip Size : 64kB Flush Time : 4 seconds Write Policy : WT Read Policy : None Cache When BBU Bad : Disabled Cached IO : No SMART Mode : Mode 6 Alarm Disable : No Coercion Mode : None ZCR Config : Unknown Dirty LED Shows Drive Activity : No BIOS Continue on Error : Yes Spin Down Mode : None Allowed Device Type : SAS/SATA Mix Allow Mix in Enclosure : Yes Allow HDD SAS/SATA Mix in VD : No Allow SSD SAS/SATA Mix in VD : No Allow HDD/SSD Mix in VD : No Allow SATA in Cluster : No Max Chained Enclosures : 16 Disable Ctrl-R : Yes Enable Web BIOS : Yes Direct PD Mapping : No BIOS Enumerate VDs : Yes Restore Hot Spare on Insertion : Yes Expose Enclosure Devices : No Maintain PD Fail History : Yes Disable Puncturing : No Zero Based Enclosure Enumeration : No PreBoot CLI Enabled : Yes LED Show Drive Activity : Yes Cluster Disable : Yes SAS Disable : No Auto Detect BackPlane Enable : SGPIO/i2c SEP Use FDE Only : No Enable Led Header : No Delay during POST : 0 EnableCrashDump : No Disable Online Controller Reset : No EnableLDBBM : Yes Un-Certified Hard Disk Drives : Allow Treat Single span R1E as R10 : No Max LD per array : 16 Power Saving option : All power saving options are enabled Default spin down time in minutes: 30 Enable JBOD : No TTY Log In Flash : No Auto Enhanced Import : No BreakMirror RAID Support : No Disable Join Mirror : No Enable Shield State : No Time taken to detect CME : 60s
縮寫
- -WT (Write through), WB (Write back): Selects write policy.
- -NORA (No read ahead), RA (Read ahead): Selects read policy.
- -Access: -RW, -RO, Blocked: Displays access policy.
- -DskCache: Displays drive cache policy.
- -PSPolicy: Displays the default and current power savings policy of the virtual drive.
- -Consistency: Displays if the physical drive is consistent or not.
BGI rate:
A controller property indicating the rate at which the background initialization of virtual drives will be carried out.
copyback
The procedure used to copy data from a source drive of a virtual drive to a destination drive that is not a part of the virtual drive.
The copyback operation is often used to create or restore a specific physical configuration for a drive group
(for example, a specific arrangement of drive group members on the device I/O buses).
The copyback operation can be run automatically or manually.
Typically, a drive fails or is expected to fail, and the data is rebuilt on a hot spare. The failed drive is replaced with a new drive.
Then the data is copied from the hot spare to the new drive, and the hot spare reverts from a rebuild drive to its original hot spare status.
The copyback operation runs as a background activity, and the virtual drive is still available online to the host.
Logical Drives Information
Command to list all Logical Drives
MegaCli64 -LDInfo -LALL -aALL
正常的 Logical Drives:
Adapter 0 -- Virtual Drive Information: Virtual Drive: 0 (Target Id: 0) Name : RAID Level : Primary-1, Secondary-0, RAID Level Qualifier-0 Size : 1.818 TB Mirror Data : 1.818 TB State : Optimal Strip Size : 64 KB Number Of Drives per span:2 Span Depth : 2 Default Cache Policy: WriteBack, ReadAdaptive, Direct, No Write Cache if Bad BBU Current Cache Policy: WriteBack, ReadAdaptive, Direct, No Write Cache if Bad BBU Default Access Policy: Read/Write Current Access Policy: Read/Write Disk Cache Policy : Disk's Default Encryption Type : None Bad Blocks Exist: No Is VD Cached: Yes Cache Cade Type : Read Only
RAID Level
- ['Primary-0, Secondary-0, RAID Level Qualifier-0'] = RAID-0
- ['Primary-1, Secondary-0, RAID Level Qualifier-0'] = RAID-1
- ['Primary-5, Secondary-0, RAID Level Qualifier-3'] = RAID-5
- ['Primary-6, Secondary-0, RAID Level Qualifier-3'] = RAID-6
- ['Primary-1, Secondary-3, RAID Level Qualifier-0'] = RAID-10
有問題時會出
State : Degraded
Span Depth
1: RAID0, RAID1 ...
2: RAID10, RAID50, RAID60
Cache Policy
MegaCli64 -LDInfo -LAll -aALL | grep Policy
Default Cache Policy: WriteThrough, ReadAheadNone, Direct, No Write Cache if Bad BBU Current Cache Policy: WriteThrough, ReadAheadNone, Direct, No Write Cache if Bad BBU Default Access Policy: Read/Write Current Access Policy: Read/Write Disk Cache Policy : Disabled
Physical Information
MegaCli64 -PDGetNum -a0
Number of Physical Drives on Adapter 0: 3
Command to list ALL Physical Information
MegaCli64 -PDList -aAll
Output Example: 正常的 HardDisk
Adapter #0 Enclosure Device ID: 32 Slot Number: 0 Drive's postion: DiskGroup: 0, Span: 0, Arm: 0 Enclosure position: N/A Device Id: 0 WWN: ????????????? Sequence Number: 2 Media Error Count: 0 Other Error Count: 0 Predictive Failure Count: 0 Last Predictive Failure Event Seq Number: 0 PD Type: SATA Raw Size: 465.761 GB [0x3a386030 Sectors] Non Coerced Size: 465.261 GB [0x3a286030 Sectors] Coerced Size: 465.25 GB [0x3a280000 Sectors] Firmware state: Online, Spun Up Device Firmware Level: 1S02 Shield Counter: 0 Successful diagnostics completion on : N/A SAS Address(0): 0x4433221107000000 Connected Port Number: 2(path0) Inquiry Data: WD-????????? WD5003ABYX-???????? 01.01S02 FDE Capable: Not Capable FDE Enable: Disable Secured: Unsecured Locked: Unlocked Needs EKM Attention: No Foreign State: None Device Speed: 3.0Gb/s Link Speed: 3.0Gb/s Media Type: Hard Disk Device Drive Temperature : N/A PI Eligibility: No Drive is formatted for PI information: No PI: No PI Port-0 : Port status: Active Port's Linkspeed: 3.0Gb/s Drive has flagged a S.M.A.R.T alert : No
單一隻碟 Info.
MegaCli -pdinfo -PhysDrv[E0:S0,E1:S1,...] -aN
ie.
MegaCli -pdinfo -PhysDrv[252:2] -a0
Output Example: HardDisk 出事時
Enclosure Device ID: 32 Slot Number: 1 Drive's postion: DiskGroup: 0, Span: 0, Arm: 1 Enclosure position: N/A Device Id: 1 WWN: 50014ee00327a42c Sequence Number: 3 Media Error Count: 333 Other Error Count: 909 Predictive Failure Count: 0 Last Predictive Failure Event Seq Number: 0 PD Type: SATA Raw Size: 465.761 GB [0x3a386030 Sectors] Non Coerced Size: 465.261 GB [0x3a286030 Sectors] Coerced Size: 465.25 GB [0x3a280000 Sectors] Firmware state: Failed Device Firmware Level: 1S02 Shield Counter: 0 Successful diagnostics completion on : N/A
更換 HardDisk 之後 Rebuild Raid 中:
Coerced Size: 465.25 GB [0x3a280000 Sectors]
Firmware state: Rebuild
Device Firmware Level: PA07
Shield Counter: 0
Successful diagnostics completion on : N/APlug Disk Event Log
seqNum: 0x0000cdb1 Time: Wed May 27 10:35:39 2020 Code: 0x0000005b Class: 0 Locale: 0x02 Event Description: Inserted: PD 01(e0xfc/s2) Event Data: =========== Device ID: 1 Enclosure Index: 252 Slot Number: 2 seqNum: 0x0000cdb2 Time: Wed May 27 10:35:39 2020 Code: 0x000000f7 Class: 0 Locale: 0x02 Event Description: Inserted: PD 01(e0xfc/s2) Info: enclPd=fc, scsiType=0, portMap=00, sasAddr=4433221101000000,0000000000000000 Event Data: =========== Device ID: 1 Enclosure Device ID: 252 Enclosure Index: 1 Slot Number: 2 SAS Address 1: 4433221101000000 SAS Address 2: 0
Disk Size
Raw Size: 931.512 GB [0x74706db0 Sectors] Non Coerced Size: 931.012 GB [0x74606db0 Sectors] Coerced Size: 931.0 GB [0x74600000 Sectors] Emulated Drive: Yes ...
coerced capacity:
A drive property indicating the capacity to which a drive has been coerced (forced) to make it compatible with other drives that are nominally the same capacity. For example, a 4-GB drive from one manufacturer might be 4,196 MB, and a 4-GB from another manufacturer might be 4,128 MB. These drives could be coerced to a usable capacity of 4,088 MB each for use in a drive group in a storage configuration.
Show Enclosure Info
MegaCli64 -EncInfo -aAll
Number of enclosures on adapter 0 -- 1
Enclosure 0:
Device ID : 32
Number of Slots : 4
Number of Power Supplies : 0
Number of Fans : 0
Number of Temperature Sensors : 0
Number of Alarms : 0
Number of SIM Modules : 0
Number of Physical Drives : 4
Status : Normal
Position : 0
Connector Name : Unavailable
Enclosure type : SES
FRU Part Number : N/A
Enclosure Serial Number : N/A
ESM Serial Number : N/A
Enclosure Zoning Mode : N/A
Partner Device Id : 65535
Inquiry data :
Vendor Identification : DP
Product Identification : BACKPLANE
Product Revision Level : 1.07
Vendor Specific : ???????
Exit Code: 0x00
RAID Health & Rebuild
# 查看有無死先.
MegaCli64 -PDList -aAll | grep "Firmware state"
# 有Disk 壞左 (有 Error)
Firmware state: Online, Spun Up # 0 Firmware state: Online, Spun Up # 1 Firmware state: Failed # 2 Firmware state: Online, Spun Up # 3
# 有 Disk 死左 (沒有反應)
Firmware state: Online, Spun Up Firmware state: Online, Spun Up Firmware state: Online, Spun Up
# 查看 rebuild 到幾多 %
Usage:
MegaCli64 -pdrbld -showprog -physdrv\[E:S\] -aALL
# 必須填的資料:
- E = Enclosure Device ID
- S = Slot Number
Example
MegaCli64 -PDRbld -ShowProg -PhysDrv [32:2] -a0
- Enclosure Device ID: 32
- Slot Number: 2
Device(Encl-32 Slot-2) is not in rebuild process Exit Code: 0x00
OR
Rebuild Progress on Device at Enclosure 32, Slot 2 Completed 1% in 1 Minutes.
更換 HDD & Rebuild
<1> Add the disk back into the array
Get the disk [32:2] back into array 1, as disk 2:
megacli -PdReplaceMissing -PhysDrv[252:4] -array1 -row2 -a0
<2> 人手 start rebuilt
megacli -PDRbld -Start -PhysDrv[E:S] -a0
i.e.
# Started rebuild progress on device(Encl-32 Slot-2)
MegaCli64 -PDRbld -Start -PhysDrv[32:2] -a0
Started rebuild progress on device(Encl-32 Slot-2)checking
MegaCli64 -PDRbld -ShowProg -PhysDrv [32:2] -a0
Auto rebuild
# enable automatic rebuild on adapter 0
MegaCli64 -AdpAutoRbld -Dsply -a0
Adapter 0: AutoRebuild is Enabled.
另有
- MegaCli64 -AdpAutoRbld -Dsbl -a0
- MegaCli64 -AdpAutoRbld -Enbl -a0
Rebuild rate
# 查看
MegaCli64 -AdpGetProp RebuildRate -a0
Adapter 0: Rebuild Rate = 30%
# 設定
MegaCli64 -AdpSetProp RebuildRate 60 -a0
Adapter 0: Rebuild Rate = 60%
Foreign Configuration Disk Re-add to RAID & Rebuild
<1> Make the drive online again:
MegaCli64 -PDMakeGood -PhysDrv[32:2] -a0
* The controller will now recognise the disk as being a "foreign" one.
(It means it has detected some RAID informations on it)
<2>
MegaCli64 -PDList -aAll | grep -e Slot -e Foreign
Slot Number: 0 Foreign State: Foreign Foreign Secure: Drive is not secured by a foreign lock key Slot Number: 1 Foreign State: Foreign Foreign Secure: Drive is not secured by a foreign lock key Slot Number: 2 Foreign State: None
<3> ask the controller to scan for foreign configuration
MegaCli64 -CfgForeign -Scan -a0
There are 1 foreign configuration(s) on controller 0.<4>
MegaCli64 -CfgForeign -Dsply -a0
Foreign configuration 0 on controller 0: Number of DISK GROUPS: 1 DISK GROUP: 0 Number of Spans: 1 SPAN: 0 Virtual Drive Information: Virtual Drive: 0 (Target Id: 0) Name :20161110 RAID Level : Primary-6, Secondary-0, RAID Level Qualifier-3 Size : 1.089 TB ... Physical Disk: 1 This Physical Disk is Missing. Physical Disk: 2 Enclosure Device ID: 252 ... Physical Disk: 3 This Physical Disk is Missing.
<5> Drop foreign configuration
MegaCli64 -CfgForeign -Clear -a0 # !! 此 Action 係沒有 Confirm 的
Foreign configuration 0 is cleared on controller 0.
Import
<1>
MegaCli64 -CfgForeign -Preview -a0
Foreign configuration on controller 0(After import): Number of DISK GROUPS: 1 DISK GROUP: 0 Number of Spans: 1 SPAN: 0 Virtual Drive Information: Virtual Drive: 0 (Target Id: 0) Name :20161110 RAID Level : Primary-6, Secondary-0, RAID Level Qualifier-3 Size : 1.089 TB ...
<2>
MegaCli64 -CfgForeign -Import -a0
Remark: Import Fail
The MegaRAID Utility may display a “Failed to Import Foreign Configuration” error.
There are two ways the error may occur:
In a server system, if the HDDs are offline and the system is power cycled, the RAID configuration will be removed.
If the user selects for the HDDs to come back online, they can import the original RAID configuration.
If the user chooses to import an individual RAID configuration (for example, “Configuration 1” or “Configuration 2”), the import will fail.
如果是更新了新 RAID Card, 那可能是新不支援舊
<3> Figure out how that disk was identified inside the RAID array [Import 了的 Foreign 才 Display]
MegaCli64 -CfgDsply -a0
Cause the front LED of the drive to blink
Start blinking
MegaCli -PdLocate -start -physdrv[E:S] -aALL
i.e.
./MegaCli64 -PdLocate -start -physdrv[32:0] -a0
Adapter: 0: Device at EnclId-32 SlotId-0 -- PD Locate Start Command was successfully sent to Firmware Exit Code: 0x00
Stop blinking
MegaCli -PdLocate -stop -physdrv[E:S] -aALL
Event Log
# Output RAID controller events to a file(raid_events.log):
MegaCli64 -AdpEventLog -GetEvents -f raid_events.log -aALL
The information shown consists of total number of entries available at firmware side since the last clear and
details of each entries of the error log.
Start_entry specifies the initial event log entry when displaying the log.
# 一定要加 "-f" 因為 console 係無 output 的
# Create a file called megacli_events_since_shutdown that will contain all the events logged by all the controllers,
# this will include warnings, info messages and errors since last shutdown.
megacli -AdpEventLog -GetSinceShutdown -f megacli_events_since_shutdown -aALL
Log 的 S/N
MegaCli64 -AdpEventLog -GetEventlogInfo -aALL
Displays overall event information such as
- total number of events
- newest sequence number
- oldest sequence number
- clear sequence number
- shutdown sequence number
- reboot sequence number
Adapter #0 Newest Seqnum: 0x0000cd8a Oldest Seqnum: 0x00000001 Clear Seqnum: 0x0000cd6d Shutdown Seqnum: 0x0000cd6e Reboot Seqnum: 0x0000cd6f Success in AdpEventLog
clear 後 reboot
# Consistency check log
MegaCli64 -AdpEventLog -GetCCIncon -f raid_events.log -LAll -aALL
-GetCCIncon: Displays the events relating to inconsistent data found during a consistency check.
# Clear Event Log
MegaCli64 -AdpEventLog -Clear -a0
Success in AdpEventLog
BBU
https://datahunter.org/raid_bbu
Terminal Logging(firmware terminal)
# pulls all of the events
MegaCli64 -FwTermLog -Dsply -aALL
03/22/13 8:36:56: EVT#15171-03/22/13 8:36:56: 103=Rebuild progress on PD 03(e0x20/s3) is 23.98%(2052s) 03/22/13 8:38:23: EVT#15172-03/22/13 8:38:23: 103=Rebuild progress on PD 03(e0x20/s3) is 24.98%(2139s) 03/22/13 8:39:51: EVT#15173-03/22/13 8:39:51: 103=Rebuild progress on PD 03(e0x20/s3) is 25.98%(2227s) 03/22/13 8:40:42: DM: Chip 0 Paused 03/22/13 8:40:42: Chip <0> Slots: Cur=[74] 03/22/13 8:40:42: [4c]= 8 [51]= 1 03/22/13 8:40:44: DM: Timing wheel expired - Chip 0 Slot 4c 03/22/13 8:40:44: Chip <0> Slots: Cur=[76] 03/22/13 8:40:44: [4c]= 8 [51]= 1 03/22/13 8:40:44: Timedout RDM: a0566600, Cmd 2 DevId[3], State 20 Flags f1400005 03/22/13 8:40:44: Timedout RDM: a0574e00, Cmd 2 DevId[3], State 20 Flags f1480005 03/22/13 8:40:44: Timedout RDM: a057ac00, Cmd 2 DevId[3], State 20 Flags f1480005 03/22/13 8:40:44: Timedout RDM: a063c400, Cmd 2 DevId[3], State 20 Flags f1480005 03/22/13 8:40:44: Timedout RDM: a065ea00, Cmd 2 DevId[3], State 20 Flags f1480005 03/22/13 8:40:44: Timedout RDM: a066fa00, Cmd 2 DevId[3], State 20 Flags f1480005 03/22/13 8:40:44: Timedout RDM: a06a1600, Cmd 2 DevId[3], State 20 Flags f1480005 03/22/13 8:40:44: Timedout RDM: a0752400, Cmd 2 DevId[3], State 20 Flags f1480005 03/22/13 8:40:44: DM_TMWheelScanAllMsg (slot 4c):Scanned 9, Timeout 8 Msg mask 21000 03/22/13 8:40:44: EVT#15174-03/22/13 8:40:44: 267=Command timeout on PD 03(e0x20/s3) Path 1221000003000000, CDB: 2a 00 1e c2 24 00 00 00 80 00 03/22/13 8:40:44: Raw Sense for PD 3: 03/22/13 8:40:44: EVT#15175-03/22/13 8:40:44: 267=Command timeout on PD 03(e0x20/s3) Path 1221000003000000, CDB: 2a 00 1e c2 24 80 00 00 80 00 03/22/13 8:40:44: Raw Sense for PD 3: 03/22/13 8:40:44: EVT#15176-03/22/13 8:40:44: 267=Command timeout on PD 03(e0x20/s3) Path 1221000003000000, CDB: 2a 00 1e c2 23 80 00 00 80 00 03/22/13 8:40:44: Raw Sense for PD 3: 03/22/13 8:40:44: EVT#15177-03/22/13 8:40:44: 267=Command timeout on PD 03(e0x20/s3) Path 1221000003000000, CDB: 2a 00 1e c2 26 00 00 00 80 00
# Clear BBU terminal log
MegaCli64 -FwTermLog -Clear -a0
Clear log 後會有 log "11/26/14 12:23:44: TTY History buffer cleared" 寫入
電池 與 Log
# 查看電池的 Log
MegaCli64 -FwTermLog -BbuGet -a0
Battery is ON for TTY history on Adapter 0
# Temporarily turn off BBU firmware logging
MegaCli64 -FwTermLog -BbuoffTemp -a0
# Battery On / Off for terminal log
# MegaCli64 -FwTermLog -Bbuoff -a0
# MegaCli64 -FwTermLog -Bbuon -a0
Battery is set to ON for TTY history on Adapter 0
P.S.
- FwTermLog(firmware terminal logging)
- BBU(Battery Backup Unit)
Display configuration
MegaCli64 -CfgDsply -aAll | less # 有 Disk 時才有 Config, 因為 Config 係保存在碟的
Output
============================================================================== Adapter: 0 Product Name: PERC H700 Adapter Memory: 512MB BBU: Present Serial No: ??????? ============================================================================== Number of DISK GROUPS: 1 SPANNED DISK GROUP: 0 Number of Spans: 2 -------------------------------------------------------------------------- SPAN: 0 SPAN: 0 Span Reference: 0x00 Number of PDs: 2 Number of VDs: 1 Number of dedicated Hotspares: 0 Virtual Drive Information: Virtual Drive: 0 (Target Id: 0) Name :20120315 RAID Level : Primary-1, Secondary-0, RAID Level Qualifier-0 Size : 930.5 GB Mirror Data : 930.5 GB State : Degraded Strip Size : 64 KB Number Of Drives per span:2 Span Depth : 2 Default Cache Policy: WriteBack, ReadAdaptive, Direct, No Write Cache if Bad BBU Current Cache Policy: WriteBack, ReadAdaptive, Direct, No Write Cache if Bad BBU Default Access Policy: Read/Write Current Access Policy: Read/Write Disk Cache Policy : Disk's Default Encryption Type : None Bad Blocks Exist: No Is VD Cached: Yes Cache Cade Type : Read Only Physical Disk Information: ---------------------------------------------- Physical Disk: 0 Physical Disk: 0 Enclosure Device ID: 32 Slot Number: 0 Drive's postion: DiskGroup: 0, Span: 0, Arm: 0 Enclosure position: N/A Device Id: 0 WWN: 50014ee0587ce9b0 Sequence Number: 2 Media Error Count: 0 Other Error Count: 0 Predictive Failure Count: 0 Last Predictive Failure Event Seq Number: 0 PD Type: SATA Raw Size: 465.761 GB [0x3a386030 Sectors] Non Coerced Size: 465.261 GB [0x3a286030 Sectors] Coerced Size: 465.25 GB [0x3a280000 Sectors] Firmware state: Online, Spun Up Device Firmware Level: 1S02 Shield Counter: 0 Successful diagnostics completion on : N/A SAS Address(0): 0x4433221107000000 Connected Port Number: 2(path0) Inquiry Data: ???? 01.01S02 FDE Capable: Not Capable FDE Enable: Disable Secured: Unsecured Locked: Unlocked Needs EKM Attention: No Foreign State: None Device Speed: 3.0Gb/s Link Speed: 3.0Gb/s Media Type: Hard Disk Device Drive Temperature : N/A PI Eligibility: No Drive is formatted for PI information: No PI: No PI Port-0 : Port status: Active Port's Linkspeed: 3.0Gb/s Drive has flagged a S.M.A.R.T alert : No ---------------------------------------------- Physical Disk: 1 Physical Disk: 1 Enclosure Device ID: 32 Slot Number: 1 Drive's postion: DiskGroup: 0, Span: 0, Arm: 1 Enclosure position: N/A Device Id: 1 WWN: 5000c5004d647155 Sequence Number: 2 Media Error Count: 0 Other Error Count: 0 Predictive Failure Count: 0 Last Predictive Failure Event Seq Number: 0 PD Type: SATA Raw Size: 465.761 GB [0x3a386030 Sectors] Non Coerced Size: 465.261 GB [0x3a286030 Sectors] Coerced Size: 465.25 GB [0x3a280000 Sectors] Firmware state: Online, Spun Up Device Firmware Level: PA07 Shield Counter: 0 Successful diagnostics completion on : N/A SAS Address(0): 0x4433221106000000 Connected Port Number: 3(path0) Inquiry Data: ???? PA07 FDE Capable: Not Capable FDE Enable: Disable Secured: Unsecured Locked: Unlocked Needs EKM Attention: No Foreign State: None Device Speed: 3.0Gb/s Link Speed: 3.0Gb/s Media Type: Hard Disk Device Drive Temperature : N/A PI Eligibility: No Drive is formatted for PI information: No PI: No PI Port-0 : Port status: Active Port's Linkspeed: 3.0Gb/s Drive has flagged a S.M.A.R.T alert : No -------------------------------------------------------------------------- SPAN: 1 SPAN: 1 Span Reference: 0x01 Number of PDs: 2 Number of VDs: 1 Number of dedicated Hotspares: 0 Virtual Drive Information: Virtual Drive: 0 (Target Id: 0) Name :20120315 RAID Level : Primary-1, Secondary-0, RAID Level Qualifier-0 Size : 930.5 GB Mirror Data : 930.5 GB State : Degraded Strip Size : 64 KB Number Of Drives per span:2 Span Depth : 2 Default Cache Policy: WriteBack, ReadAdaptive, Direct, No Write Cache if Bad BBU Current Cache Policy: WriteBack, ReadAdaptive, Direct, No Write Cache if Bad BBU Default Access Policy: Read/Write Current Access Policy: Read/Write Disk Cache Policy : Disk's Default Encryption Type : None Bad Blocks Exist: No Is VD Cached: Yes Cache Cade Type : Read Only Physical Disk Information: ---------------------------------------------- Physical Disk: 0 Physical Disk: 0 Enclosure Device ID: 32 Slot Number: 2 Drive's postion: DiskGroup: 0, Span: 1, Arm: 0 Enclosure position: N/A Device Id: 2 WWN: 50014ee0592ab20c Sequence Number: 22 Media Error Count: 0 Other Error Count: 1 Predictive Failure Count: 0 Last Predictive Failure Event Seq Number: 0 PD Type: SATA Raw Size: 465.761 GB [0x3a386030 Sectors] Non Coerced Size: 465.261 GB [0x3a286030 Sectors] Coerced Size: 465.25 GB [0x3a280000 Sectors] Firmware state: Failed Device Firmware Level: 1S03 Shield Counter: 0 Successful diagnostics completion on : N/A SAS Address(0): 0x4433221105000000 Connected Port Number: 1(path0) Inquiry Data: ???? 01.01S03 FDE Capable: Not Capable FDE Enable: Disable Secured: Unsecured Locked: Unlocked Needs EKM Attention: No Foreign State: None Device Speed: 6.0Gb/s Link Speed: 6.0Gb/s Media Type: Hard Disk Device Drive: Not Certified Drive Temperature : N/A PI Eligibility: No Drive is formatted for PI information: No PI: No PI Port-0 : Port status: Active Port's Linkspeed: 6.0Gb/s Drive has flagged a S.M.A.R.T alert : No ---------------------------------------------- Physical Disk: 1 Physical Disk: 1 Enclosure Device ID: 32 Slot Number: 3 Drive's postion: DiskGroup: 0, Span: 1, Arm: 1 Enclosure position: N/A Device Id: 3 WWN: 50014ee0587ce09a Sequence Number: 2 Media Error Count: 1 Other Error Count: 0 Predictive Failure Count: 0 Last Predictive Failure Event Seq Number: 0 PD Type: SATA Raw Size: 465.761 GB [0x3a386030 Sectors] Non Coerced Size: 465.261 GB [0x3a286030 Sectors] Coerced Size: 465.25 GB [0x3a280000 Sectors] Firmware state: Online, Spun Up Device Firmware Level: 1S02 Shield Counter: 0 Successful diagnostics completion on : N/A SAS Address(0): 0x4433221104000000 Connected Port Number: 0(path0) Inquiry Data: ???? 01.01S02 FDE Capable: Not Capable FDE Enable: Disable Secured: Unsecured Locked: Unlocked Needs EKM Attention: No Foreign State: None Device Speed: 3.0Gb/s Link Speed: 3.0Gb/s Media Type: Hard Disk Device Drive Temperature : N/A PI Eligibility: No Drive is formatted for PI information: No PI: No PI Port-0 : Port status: Active Port's Linkspeed: 3.0Gb/s Drive has flagged a S.M.A.R.T alert : No
時間
MegaCli64 -AdpGetTime -aALL
Adapter 0:
Date: 03/22/2013
Time: 10:37:11
Exit Code: 0x00
設定時間
MegaCli64 –AdpSetTime yyyymmdd HH:mm:ss -a0
e.g. 1: 人手設定
MegaCli64 -AdpSetTime 20141126 11:00:00 -a0
New date/time is set on Adapter 0 Exit Code: 0x00
e.g. 2: 用系統的時鍾
MegaCli64 -AdpSetTime `date +%Y%m%d` `date +%H:%M:%S` -aALL
Number of X
# Total Number of Drives
MegaCli64 -PDGetNum -aAll
Number of Physical Drives on Adapter 0: 4
# Display the Number of Virtual Drives
MegaCli64 -LDGetNum -aAll
Number of Virtual Drives Configured on Adapter 0: 1
# Virtual Drive Information
MegaCli64 -LDInfo -Lall -aAll
Virtual Drive: 0 (Target Id: 0) Name :20120315 RAID Level : Primary-1, Secondary-0, RAID Level Qualifier-0 Size : 930.5 GB Mirror Data : 930.5 GB State : Degraded Strip Size : 64 KB Number Of Drives per span:2 Span Depth : 2 Default Cache Policy: WriteBack, ReadAdaptive, Direct, No Write Cache if Bad BBU Current Cache Policy: WriteBack, ReadAdaptive, Direct, No Write Cache if Bad BBU Default Access Policy: Read/Write Current Access Policy: Read/Write Disk Cache Policy : Disk's Default Encryption Type : None Bad Blocks Exist: No Is VD Cached: Yes Cache Cade Type : Read Only
# 某 Logical Volume 的 info.
# detailed information on the drives in the array
MegaCli64 -LDPDInfo -aAll | less
Number of Virtual Disks: 1 Virtual Drive: 0 (Target Id: 0) Name :20130311 RAID Level : Primary-1, Secondary-3, RAID Level Qualifier-0 Size : 1.818 TB Mirror Data : 1.818 TB State : Degraded Strip Size : 64 KB Number Of Drives per span:2 Span Depth : 2 Default Cache Policy: WriteBack, ReadAheadNone, Direct, No Write Cache if Bad BBU Current Cache Policy: WriteBack, ReadAheadNone, Direct, No Write Cache if Bad BBU Default Access Policy: Read/Write Current Access Policy: Read/Write Disk Cache Policy : Disk's Default Encryption Type : None Is VD Cached: No Number of Spans: 2 Span: 0 - Number of PDs: 2 PD: 0 Information Enclosure Device ID: 32 Slot Number: 0 Drive's postion: DiskGroup: 0, Span: 0, Arm: 0 ... PD Type: SATA ... PD: 1 Information ... Span: 1 - Number of PDs: 2 PD: 0 Information Enclosure Device ID: 32 Slot Number: 2 Drive's postion: DiskGroup: 0, Span: 1, Arm: 0 PD: 1 Information ...
Check 在 RAID 上的 Disk SMART
# 找 Device Id
MegaCli64 -PDList -aAll
# 有問題的 Disk Enclosure Device ID: 32 Slot Number: 2 Drive's postion: DiskGroup: 0, Span: 0, Arm: 2 Enclosure position: N/A Device Id: 2 WWN: 50014ee0adde4846 Sequence Number: 3 Media Error Count: 6 Other Error Count: 6 Predictive Failure Count: 0 Last Predictive Failure Event Seq Number: 0 PD Type: SATA Raw Size: 465.761 GB [0x3a386030 Sectors] Non Coerced Size: 465.261 GB [0x3a286030 Sectors] Coerced Size: 465.25 GB [0x3a280000 Sectors] Firmware state: Failed Device Firmware Level: 1S02 Shield Counter: 0 Successful diagnostics completion on : N/A SAS Address(0): 0x4433221105000000 Connected Port Number: 1(path0) Inquiry Data: WD-WMAYP3444742WDC WD5003ABYX-18WERA0 01.01S02 FDE Capable: Not Capable FDE Enable: Disable Secured: Unsecured Locked: Unlocked Needs EKM Attention: No Foreign State: None Device Speed: 3.0Gb/s Link Speed: 3.0Gb/s Media Type: Hard Disk Device Drive Temperature : N/A PI Eligibility: No Drive is formatted for PI information: No PI: No PI Port-0 : Port status: Active Port's Linkspeed: 3.0Gb/s Drive has flagged a S.M.A.R.T alert : No ................................
查看未建立 Volume 的 Disk 的 SMART
smartctl --scan
... /dev/bus/0 -d megaraid,1 # /dev/bus/0 [megaraid_disk_01], SCSI device /dev/bus/0 -d megaraid,12 # /dev/bus/0 [megaraid_disk_12], SCSI device /dev/bus/0 -d megaraid,13 # /dev/bus/0 [megaraid_disk_13], SCSI device
smartctl -d megaraid,1 -i /dev/bus/0
smartctl -d megaraid,1 -A /dev/bus/0
不支援 SMART 的 Disk
smartctl -d megaraid,12 -a /dev/bus/0
... SMART support is: Unavailable - device lacks SMART capability. ...
Physical drive management
Step
- 找出 HDD
- Set HDD State to Offline
- Mark as missing
- 換碟
1. 找出 HDD
MegaCli64 -PDList -aAll | grep -e 'Enclosure Device' -e Slot -e Inquiry -e Raw
Enclosure Device ID: 252 Slot Number: 0 Raw Size: 558.911 GB [0x45dd2fb0 Sectors] Inquiry Data: SEAGATE ST3600057SS ???? Enclosure Device ID: 252 Slot Number: 1 Raw Size: 558.911 GB [0x45dd2fb0 Sectors] Inquiry Data: SEAGATE ST3600057SS ???? Enclosure Device ID: 252 Slot Number: 2 Raw Size: 931.512 GB [0x74706db0 Sectors] Inquiry Data: Z9A1P2K1ST1000DM003-1SB10C CC43
2. Set HDD State to Offline (offline 不代表可以 unplug 佢)
# Set the drive offline, if it is not already offline due to an error
MegaCli64 -PDOffline -PhysDrv [E:S] -aN
i.e.
MegaCli64 -PDOffline -PhysDrv [252:0] -a0
Adapter: 0: EnclId-252 SlotId-0 state changed to OffLine.
MegaCli64 -PDList -aAll | grep -i -e slot -e firmware
Slot Number: 0 Firmware state: Unconfigured(good), Spun down
3. Mark as missing
MegaCli -PDMarkMissing -PhysDrv [E:S] -aN
* If you want to make a configured drive as missing
* "-PdMarkMissing" works only on offline drives.
* When -PdReplaceMissing is run, the drive becomes offline,
and rebuild does not start automatically. You have to start it explicitly.
View missing
MegaCli64 -PdGetMissing -aAll
Adapter 0 - Missing Physical drives
No. Array Row Size Expected
0 1 1 953344 MB4. 換碟
抽出壞 Disk 並插入新 Disk
5. Set New State to Online
MegaCli -PDOnline -PhysDrv [E:S] -aN
6. Re-add the new drive to virtual drive
MegaCli -PdReplaceMissing -physdrv[E0:S0] -arrayA, -rowB -a0
OR
MegaCli -PDHSP -Set -PhysDrv [E:S] -a0 # Global Hot Spare
MegaCli -PDHSP -Set -Dedicated -ArrayN -PhysDrv [E:S] -a0 # Dedicated
Remark
MegaCli -CfgDsply -a0
7. Start the rebuilding
MegaCli -PDRbld -Start -PhysDrv [E:S] -a0
--------
Prepare Unconfigured Drives for Removal (Spun down disk)
Remove a non-failed drive that is connected to the controller
=> Firmware spins down this drive.
=> The drive state is set to Unaffiliated, which marks it as offline even though it is not a part of a configuration.
# -Undo: This option undoes this operation. If you select undo, firmware marks this drive as Unconfigured Good.
MegaCli64 -PDPrpRmv [-Undo] -PhysDrv[E:S] -a0
i.e.
MegaCli64 -PDPrpRmv -PhysDrv[252:0] -a0
Prepare for removal Success
--------
Unconfigured(bad)
Drives stuck in the Unconfigured(bad) state need to be manually set back to Unconfigured(good) before they will be importable.
Failure to import arrays usually means that the drives were unexpectedly powered off or disconnected from the RAID controller,
which puts them in the Unconfigured(bad) state.
原因:
This is done as a precautionary measure to prevent drives with bad connections from joining the array,
as sudden disconnects can result in data corruption.
MegaCli64 -PDList -a0
... Firmware state: Unconfigured(bad) ...
MegaCli64 -PDMakeGood -PhysDrv[E0:S0,E1:S1,...]
i.e.
MegaCli64 -PDMakeGood -PhysDrv[252:1] -a0
Adapter: 0: EnclId-252 SlotId-1 state changed to Unconfigured-Good.
建立 Virtual Drive
Step1: List 可用的 Disk
# 可用的 Disk = Firmware state: Unconfigured(good), Spun Up
MegaCli64 -PDList -a0
Step2: 建立 VD
MegaCli -CfgLdAdd R0|R1|R5|R6[E0:S0,E1:S1,...]\ [WT|WB] [NORA|RA] [Direct|Cached] [CachedBadBBU|NoCachedBadBBU] \ [-szXXXXXXXX [-szYYYYYYYY [... ]]] \ [-strpszM] [-Hsp[E5:S5,...]] -aN
i.e.
MegaCli64 -CfgLdAdd R1[252:0,252:1] -a0
MegaCli64 -CfgLdAdd R5[32:1,32:2,32:3] WB RA Cached -a0
* Default: WT, NORA
-Rx[E0:S0,...]
Specifies the RAID level and the drive enclosure/slot numbers to construct a drive group.
-strpszM
Specifies the stripe size, where the stripe size values are
8 KB, 16 KB, 32 KB, 64 KB, 128 KB, 256 KB, 512 KB, or 1024 KB.
-Hsp[E5:S5,...]
Creates hot spares when you create the configuration.
IO Policy (Direct|Cached)
Direct(default):
reads are not buffered in cache memory.
Data is transferred to the cache and the host concurrently.
If the same data block is read again, it comes from cache memory.
Cached:
all reads are buffered in cache memory.
* 此 Setting 不會影響 RA
Drive Cache (Enable|Disable)
Specify the drive cache policy.
- Enable: Enable the drive cache.
- Disable(default): Disable the drive cache.
i.e.
[A]
MegaCli64 -cfgldadd R1[252:0,252:1] -a0
Adapter 0: Created VD 1 Adapter 0: Configured the Adapter!!
[B]
MegaCli64 -cfgldadd R1[252:0,252:1] WB RA -a0
Step3: 查看建立出來的 VD Info.
MegaCli64 -LDInfo -LAll -a0 | less
Adapter 0 -- Virtual Drive Information: Virtual Drive: 1 (Target Id: 1) Name : RAID Level : Primary-1, Secondary-0, RAID Level Qualifier-0 Size : 465.25 GB Is VD emulated : No Mirror Data : 465.25 GB State : Optimal Strip Size : 64 KB Number Of Drives : 2 Span Depth : 1 Default Cache Policy: WriteThrough, ReadAheadNone, Direct, No Write Cache if Bad BBU Current Cache Policy: WriteThrough, ReadAheadNone, Direct, No Write Cache if Bad BBU Default Access Policy: Read/Write Current Access Policy: Read/Write Disk Cache Policy : Disabled Ongoing Progresses: Background Initialization: Completed 0%, Taken 0 min. Encryption Type : None Bad Blocks Exist: No Is VD Cached: No
Step4: 設定(-LDSetProp)
MegaCli -LDSetProp -Name LdNamestring -LN -aN MegaCli -LDSetProp SETTING -LN -aN
Allows you to change the following SETTING:
- Write policy: WT (Write through), WB (Write back)
- Specifies whether to use write cache when the BBU is bad: ForcedWB | CachedBadBBU | NoCachedBadBBU
- Read policy: NORA (No read ahead), RA (Read ahead)
- IO cache policy: Cached, Direct
- Access policy: RW, RO, Blocked, RemoveBlocked
- Drive(Disk) cache: EnDskCache | DisDskCache
- Immediate
ForcedWB # To enable cache even without BBU
i.e.
# Rename
MegaCli64 -LDSetProp -NAME R1 -L1 -a0
Set name to R1 on Adapter 0, VD 1 (target id: 1) success
# Set Cache
MegaCli64 -LDSetProp WB Immediate -L1 -a0
MegaCli64 -LDSetProp RA Immediate -L1 -a0 # 只可以一行行設定
Set BGI Rate
MegaCli64 -AdpSetProp BgiRate -100 -a 0
Adapter 0: Set BGI Rate to 100% success.
MegaCli64 -AdpGetProp BgiRate -a0
Adapter 0: BGI Rate = 100%
Remove Virtual Drive
* 此 CMD 係沒有 Comfirm 的 !!
MegaCli64 -CfgLdDel -L1 -a0
Adapter 0: Deleted Virtual Drive-1(target id-1)
All Disk To RAID0 (CfgEachDskRaid0)
MegaCli64 -CfgEachDskRaid0 -a0
Adapter 0: Created VD 0 Configured physical device at Encl-252:Slot-2. Adapter 0: Created VD 1 Configured physical device at Encl-252:Slot-1. Adapter 0: Created VD 2 Configured physical device at Encl-252:Slot-0. 3 physical devices are Configured on adapter 0.
dmesg
[11182.567632] scsi 0:2:0:0: Direct-Access LSI RAID 5/6 SAS 6G 2.13 PQ: 0 ANSI: 5 [11182.570212] sd 0:2:0:0: Attached scsi generic sg3 type 0 [11182.570334] sd 0:2:0:0: [sdd] 1952448512 512-byte logical blocks: (1000 GB/931 GiB) [11182.570336] sd 0:2:0:0: [sdd] 4096-byte physical blocks [11182.570430] sd 0:2:0:0: [sdd] Write Protect is off [11182.570439] sd 0:2:0:0: [sdd] Mode Sense: 1f 00 10 08 [11182.570493] sd 0:2:0:0: [sdd] Write cache: disabled, read cache: disabled, supports DPO and FUA [11182.593818] sd 0:2:0:0: [sdd] Attached SCSI disk [11182.629587] megaraid_sas 0000:02:00.0: scanning for scsi0... [11220.626376] scsi 0:2:1:0: Direct-Access LSI RAID 5/6 SAS 6G 2.13 PQ: 0 ANSI: 5 [11220.626737] sd 0:2:1:0: Attached scsi generic sg4 type 0 [11220.626783] sd 0:2:1:0: [sde] 1171062784 512-byte logical blocks: (600 GB/558 GiB) [11220.626806] sd 0:2:1:0: [sde] Write Protect is off [11220.626807] sd 0:2:1:0: [sde] Mode Sense: 1f 00 10 08 [11220.626875] sd 0:2:1:0: [sde] Write cache: disabled, read cache: disabled, supports DPO and FUA [11220.633490] sd 0:2:1:0: [sde] Attached SCSI disk [11241.554459] megaraid_sas 0000:02:00.0: scanning for scsi0... ....
一隻隻碟加
MegaCli64 -cfgldadd R0[252:2] -a0
Adapter 0: Created VD 1 Adapter 0: Configured the Adapter!!
MegaCli64 -LDInfo -L1 -a0
MegaCli64 -CfgLdDel -L1 -a0 # 是沒有 Confirm 的
Clear Drive (PDClear)
MegaCli -PDClear -Start -PhysDrv [E:S] -aN
i.e.
MegaCli -PDClear -Start -PhysDrv [252:2] -a0
Started clear progress on device(Encl-252 Slot-2)
MegaCli -PDClear -Stop -PhysDrv [E:S] -aN
MegaCli -PDClear -ShowProg -PhysDrv [E:S] -aN
i.e.
MegaCli -PDClear -ShowProg -PhysDrv [252:2] -a0
Clear Progress on Device at Enclosure 252, Slot 2 Completed 0% in 0 Minutes.
Other
MegaCli64 -AdpCacheFlush -a0
Cache Flush is successfully done on adapter 0.
IT 與 IR Mode
IT mode stands for "initiator target".
It presents each drive individually to the host.
IR mode is in whatever raid formats the HBA supports.
Some allow you to present drives individually as JBODs or raid0 with one drive each.
=============================
crossflash
D2616 HBA which is based on LSI SAS2108
Disk Cache Policy
如果 raid 控制器為 writeback, Disk Cache 是否開啟對系統的寫性能影響很小
* Default: Disable
* RAID 卡的 BBU 特性則不能保護磁盤緩存的數據.
I/O Policy
The I/O Policy determines whether the RAID controller will keep data in the cache,
which can reduce the access time if subsequent read requests are made to the same data blocks.
Direct I/O (Recommended in most cases)
直接從磁盤讀取數據, 不使用 raid cache 功能.
Recommended 原因:
Most file systems and many applications have their own cache and
do not require caching data at the RAID controller level.
Cached I/O
The controller caches both read and write requests.
If there are subsequent read requests to the same data blocks,
they are read from the RAID cache instead of the hard drives.
This mode may be required if the application or file system does not cache read requests.
Read Ahead
The RAID controller will read the whole stripe containing the requested data block and will keep it in cache.
Note:
This setting will only make difference if the typical read request size is smaller than the stripe width.
Stripe Width 與 Stripe Size
The properties of a distributed array
Stripe Width
The number of parallel stripes that can be written to or read from simultaneously
Which equals the number of drives in a RAID array
作用: 決定了同時讀寫能用
Strip Size
The smallest chunk of data within a RAID array that can be addressed.
It also determines how many drives are accessed to service a single I/O request.
Increasing & Decreasing the Stripe Size
Increasing Stripe Size: Fewer drives are required to store files of a given size.
=> allows the drives not needed for a particular access to be used for another one
=> improving positioning performance.
Decreasing Stripe Size: theoretically increasing transfer performance, but decreasing positioning performance
Note:
Matching the strip size to the file system cluster size does not usually provide any benefit.
Data block or file sizes used by the application are usually more important.
However, setting the strip size smaller than the cluster size is not recommended.
larger stripe sizes enable more I/O operations per second, and I/O performance increases even more if large stripe sizes are used.
( 對於 Read / Write throughput 沒大影響)
PatrolRead and ConsistencyChecking
Cheat List
- MegaCli64 -AdpAllInfo -aALL | less
- ---------------------
- MegaCli64 -PDList -a0 # 找出所有 Phyical Disk
- MegaCli64 -PDList -a0 | grep "Firmware state" # check disk status
- ---------------------
- MegaCli64 -LDInfo -LAll -a0 #
- ---------------------
- smartctl --scan
- smartctl -d megaraid,1 -a /dev/sda # 某 Disk 的 S.M.A.R.K
- ---------------------
- MegaCli64 -PDRbld -ShowProg -PhysDrv [32:2] -a0 # % rebuild
- ---------------------
- MegaCli64 -CfgForeign -Scan -a0
- MegaCli64 -CfgForeign -Dsply -a0
- MegaCli64 -CfgForeign -Import -a0
- ---------------------
- MegaCli64 -AdpSetTime `date +%Y%m%d` `date +%H:%M:%S` -aALL
- MegaCli64 -AdpGetTime -aALL
- ---------------------
- MegaCli64 -AdpEventLog -GetEvents -f raid_events.log -aALL
