MegaCli

最後更新: 2020-05-26

目錄

  • Installation (Centos, Debian)
  • List Adapter Information
  • Logical Drive Information
  • Physical Disk Information
  • Show Enclosure Info
  • RAID Health & Rebuild
  • Foreign Configuration
  • Cause the front LED of the drive to blink
  • Events Log
  • BBU
  • Terminal Logging(firmware terminal)
  • Display configuration
  • 時間與時間
  • Number of X
  • Check Disk SMART
  • IT 與 IR Mode
  • Disk Cache Policy
  • I/O Policy
  • Read Ahead
  • Stripe Width 與 Stripe Size
  • Cheat List
  • See Other

 


Installation

 

Centos

Download (2020-05)

https://docs.broadcom.com/docs/12351582

unzip Lin_CliKL_8.04.07.zip

unzip MegaCliKLLin.zip

*.rpm

Lib_Utils-1.00-09.noarch.rpm  Lib_Utils2-1.00-01.noarch.rpm  MegaCliKL-8.04.07-1.noarch.rpm

安裝後 megacli 會放在

/opt/MegaRAID/MegaCli/

check version

./MegaCli64 -v

MegaCLI SAS RAID Management Tool  Ver 8.04.07 May 28, 2012

Debian

alien -k MegaCliKL-x.xx-x.noarch.rpm

dpkg -i megaclikl_x.xx.xx-x_all.deb

ln -s /opt/MegaRAID/MegaCli/MegaCli64 /usr/sbin/MegaCli64

Testing

MegaCli64 -V

      MegaCLI SAS RAID Management Tool  Ver 8.04.07 May 28, 2012

    (c)Copyright 2011, LSI Corporation, All Rights Reserved.

Exit Code: 0x00

Help

MegaCli64 -h

Troubleshoot

./MegaCli64:

error while loading shared libraries: libncurses.so.5: cannot open shared object file: No such file or directory

[Fix]

ldconfig

Or

apt-get install libncurses5


List Adapter Information

 

MegaCli64 -adpCount

Controller Count: 1.

MegaCli64 -AdpAllInfo -a0 | less          # 亦可以用 "-aALL"

Adapter #0

==============================================================================
                    Versions
                ================
Product Name    : RAID Ctrl SAS 6G 5/6 512MB (D2616)
Serial No       :
FW Package Build: 12.15.0-0239     # Firmware Package 來, 它已經是 9261-8i 最後的 FW

........

                Image Versions in Flash:
                ================
FW Version         : 2.130.403-4660
BIOS Version       : 3.30.02.2_4.16.08.00_0x06060A05

........

                PCI Info
                ================
........

                HW Configuration
                ================
...
BBU              : Present
...
Memory Size      : 512MB
TPM              : Absent
Upgrade Key      : Absent
...

                Settings
                ================
Current Time                     : 13:23:37 5/26, 2020
Predictive Fail Poll Interval    : 300sec
Interrupt Throttle Active Count  : 16
Interrupt Throttle Completion    : 50us
Rebuild Rate                     : 30%
PR Rate                          : 20%
BGI Rate                         : 30%
Check Consistency Rate           : 30%
Reconstruction Rate              : 30%
Cache Flush Interval             : 4s
Max Drives to Spinup at One Time : 2
Delay Among Spinup Groups        : 6s
Physical Drive Coercion Mode     : Disabled
Cluster Mode                     : Disabled
Alarm                            : Disabled
Auto Rebuild                     : Enabled
Battery Warning                  : Enabled
Ecc Bucket Size                  : 15
Ecc Bucket Leak Rate             : 1440 Minutes
Restore HotSpare on Insertion    : Enabled
Expose Enclosure Devices         : Disabled
Maintain PD Fail History         : Enabled
Host Request Reordering          : Enabled
Auto Detect BackPlane Enabled    : SGPIO/i2c SEP
Load Balance Mode                : Auto
Use FDE Only                     : No
Security Key Assigned            : No
Security Key Failed              : No
Security Key Not Backedup        : No
Default LD PowerSave Policy      : Automatic
Maximum number of direct attached drives to spin up in 1 min : 20
Auto Enhanced Import             : No
Any Offline VD Cache Preserved   : No
Allow Boot with Preserved Cache  : No
Disable Online Controller Reset  : No
PFK in NVRAM                     : No
Use disk activity for locate     : No
POST delay                       : 90 seconds

                Capabilities
                ================
RAID Level Supported             : RAID0, RAID1, RAID5, RAID6, RAID00, RAID10, RAID50, RAID60, 
                                   PRL 11, PRL 11 with spanning, SRL 3 supported, 
                                   PRL11-RLQ0 DDF layout with no span, PRL11-RLQ0 DDF layout with span
Supported Drives                 : SAS, SATA
...

                Status
                ================
ECC Bucket Count                 : 0

                Limitations
                ================
Max Arms Per VD          : 32
Max Spans Per VD         : 8
Max Arrays               : 128
Max Number of VDs        : 64
Max Parallel Commands    : 1008
Max SGE Count            : 80
Max Data Transfer Size   : 8192 sectors
Max Strips PerIO         : 42
Max LD per array         : 16
Min Strip Size           : 8 KB
Max Strip Size           : 1.0 MB
Max Configurable CacheCade Size: 0 GB
Current Size of CacheCade      : 0 GB
Current Size of FW Cache       : 0 MB

                Device Present
                ================
Virtual Drives    : 0
  Degraded        : 0
  Offline         : 0
Physical Devices  : 1
  Disks           : 0
  Critical Disks  : 0
  Failed Disks    : 0

                Supported Adapter Operations
                ================
Rebuild Rate                    : Yes
CC Rate                         : Yes
BGI Rate                        : Yes
Reconstruct Rate                : Yes
Patrol Read Rate                : Yes
Alarm Control                   : Yes
Cluster Support                 : No
BBU                             : Yes
Spanning                        : Yes
Dedicated Hot Spare             : Yes
Revertible Hot Spares           : Yes
Foreign Config Import           : Yes
Self Diagnostic                 : Yes
Allow Mixed Redundancy on Array : No
Global Hot Spares               : Yes
Deny SCSI Passthrough           : No
Deny SMP Passthrough            : No
Deny STP Passthrough            : No
Support Security                : No
Snapshot Enabled                : No
Support the OCE without adding drives : Yes
Support PFK                     : Yes
Support PI                      : No
Support Boot Time PFK Change    : No
Disable Online PFK Change       : No
PFK TrailTime Remaining         : 0 days 0 hours
Support Shield State            : No
Block SSD Write Disk Cache Change: No

                Supported VD Operations
                ================
Read Policy          : Yes
Write Policy         : Yes
IO Policy            : Yes
Access Policy        : Yes
Disk Cache Policy    : Yes
Reconstruction       : Yes
Deny Locate          : No
Deny CC              : No
Allow Ctrl Encryption: No
Enable LDBBM         : Yes
Support Breakmirror  : No
Power Savings        : No

                Supported PD Operations
                ================
Force Online                            : Yes
Force Offline                           : Yes
Force Rebuild                           : Yes
Deny Force Failed                       : No
Deny Force Good/Bad                     : No
Deny Missing Replace                    : No
Deny Clear                              : No
Deny Locate                             : No
Support Temperature                     : Yes
Disable Copyback                        : No
Enable JBOD                             : No
Enable Copyback on SMART                : Yes
Enable Copyback to SSD on SMART Error   : Yes
Enable SSD Patrol Read                  : No
PR Correct Unconfigured Areas           : Yes
Enable Spin Down of UnConfigured Drives : Yes
Disable Spin Down of hot spares         : No
Spin Down time                          : 30
T10 Power State                         : No

                Error Counters
                ================
Memory Correctable Errors   : 0
Memory Uncorrectable Errors : 0

........

                Default Settings
                ================
Phy Polarity                     : 0
Phy PolaritySplit                : 0
Background Rate                  : 30
Strip Size                       : 64kB
Flush Time                       : 4 seconds
Write Policy                     : WT
Read Policy                      : None
Cache When BBU Bad               : Disabled
Cached IO                        : No
SMART Mode                       : Mode 6
Alarm Disable                    : No
Coercion Mode                    : None
ZCR Config                       : Unknown
Dirty LED Shows Drive Activity   : No
BIOS Continue on Error           : Yes
Spin Down Mode                   : None
Allowed Device Type              : SAS/SATA Mix
Allow Mix in Enclosure           : Yes
Allow HDD SAS/SATA Mix in VD     : No
Allow SSD SAS/SATA Mix in VD     : No
Allow HDD/SSD Mix in VD          : No
Allow SATA in Cluster            : No
Max Chained Enclosures           : 16
Disable Ctrl-R                   : Yes
Enable Web BIOS                  : Yes
Direct PD Mapping                : No
BIOS Enumerate VDs               : Yes
Restore Hot Spare on Insertion   : Yes
Expose Enclosure Devices         : No
Maintain PD Fail History         : Yes
Disable Puncturing               : No
Zero Based Enclosure Enumeration : No
PreBoot CLI Enabled              : Yes
LED Show Drive Activity          : Yes
Cluster Disable                  : Yes
SAS Disable                      : No
Auto Detect BackPlane Enable     : SGPIO/i2c SEP
Use FDE Only                     : No
Enable Led Header                : No
Delay during POST                : 0
EnableCrashDump                  : No
Disable Online Controller Reset  : No
EnableLDBBM                      : Yes
Un-Certified Hard Disk Drives    : Allow
Treat Single span R1E as R10     : No
Max LD per array                 : 16
Power Saving option              : All power saving options are enabled
Default spin down time in minutes: 30
Enable JBOD                      : No
TTY Log In Flash                 : No
Auto Enhanced Import             : No
BreakMirror RAID Support         : No
Disable Join Mirror              : No
Enable Shield State              : No
Time taken to detect CME         : 60s

縮寫

  • -WT (Write through), WB (Write back): Selects write policy.
  • -NORA (No read ahead), RA (Read ahead): Selects read policy.
  • -Access: -RW, -RO, Blocked: Displays access policy.
  • -DskCache: Displays drive cache policy.
  • -PSPolicy: Displays the default and current power savings policy of the virtual drive.
  • -Consistency: Displays if the physical drive is consistent or not.

BGI rate:

A controller property indicating the rate at which the background initialization of virtual drives will be carried out.

copyback

The procedure used to copy data from a source drive of a virtual drive to a destination drive that is not a part of the virtual drive.

The copyback operation is often used to create or restore a specific physical configuration for a drive group

(for example, a specific arrangement of drive group members on the device I/O buses).

The copyback operation can be run automatically or manually.

Typically, a drive fails or is expected to fail, and the data is rebuilt on a hot spare. The failed drive is replaced with a new drive.

Then the data is copied from the hot spare to the new drive, and the hot spare reverts from a rebuild drive to its original hot spare status.

The copyback operation runs as a background activity, and the virtual drive is still available online to the host.

 


Logical Drives Information

 

Command to list all Logical Drives

MegaCli64 -LDInfo -LALL -aALL

正常的 Logical Drives:

Adapter 0 -- Virtual Drive Information:
Virtual Drive: 0 (Target Id: 0)
Name                :
RAID Level          : Primary-1, Secondary-0, RAID Level Qualifier-0
Size                : 1.818 TB
Mirror Data         : 1.818 TB
State               : Optimal
Strip Size          : 64 KB
Number Of Drives per span:2
Span Depth          : 2
Default Cache Policy: WriteBack, ReadAdaptive, Direct, No Write Cache if Bad BBU
Current Cache Policy: WriteBack, ReadAdaptive, Direct, No Write Cache if Bad BBU
Default Access Policy: Read/Write
Current Access Policy: Read/Write
Disk Cache Policy   : Disk's Default
Encryption Type     : None
Bad Blocks Exist: No
Is VD Cached: Yes
Cache Cade Type : Read Only

RAID Level

  • ['Primary-0, Secondary-0, RAID Level Qualifier-0'] = RAID-0
  • ['Primary-1, Secondary-0, RAID Level Qualifier-0'] = RAID-1
  • ['Primary-5, Secondary-0, RAID Level Qualifier-3'] = RAID-5
  • ['Primary-6, Secondary-0, RAID Level Qualifier-3'] = RAID-6
  • ['Primary-1, Secondary-3, RAID Level Qualifier-0'] = RAID-10

有問題時會出

State               : Degraded

Span Depth

1: RAID0, RAID1 ...

2: RAID10, RAID50, RAID60

Cache Policy

MegaCli64 -LDInfo -LAll -aALL | grep Policy

Default Cache Policy: WriteThrough, ReadAheadNone, Direct, No Write Cache if Bad BBU
Current Cache Policy: WriteThrough, ReadAheadNone, Direct, No Write Cache if Bad BBU
Default Access Policy: Read/Write
Current Access Policy: Read/Write
Disk Cache Policy   : Disabled

 


Physical Information

 

MegaCli64 -PDGetNum -a0

Number of Physical Drives on Adapter 0: 3

Command to list ALL Physical Information

MegaCli64 -PDList -aAll

Output Example: 正常的 HardDisk

Adapter #0

Enclosure Device ID: 32
Slot Number: 0
Drive's postion: DiskGroup: 0, Span: 0, Arm: 0
Enclosure position: N/A
Device Id: 0
WWN: ?????????????
Sequence Number: 2
Media Error Count: 0
Other Error Count: 0
Predictive Failure Count: 0
Last Predictive Failure Event Seq Number: 0
PD Type: SATA

Raw Size: 465.761 GB [0x3a386030 Sectors]
Non Coerced Size: 465.261 GB [0x3a286030 Sectors]
Coerced Size: 465.25 GB [0x3a280000 Sectors]
Firmware state: Online, Spun Up
Device Firmware Level: 1S02
Shield Counter: 0
Successful diagnostics completion on :  N/A
SAS Address(0): 0x4433221107000000
Connected Port Number: 2(path0)
Inquiry Data:      WD-????????? WD5003ABYX-????????                  01.01S02
FDE Capable: Not Capable
FDE Enable: Disable
Secured: Unsecured
Locked: Unlocked
Needs EKM Attention: No
Foreign State: None
Device Speed: 3.0Gb/s
Link Speed: 3.0Gb/s
Media Type: Hard Disk Device
Drive Temperature : N/A
PI Eligibility:  No
Drive is formatted for PI information:  No
PI: No PI
Port-0 :
Port status: Active
Port's Linkspeed: 3.0Gb/s
Drive has flagged a S.M.A.R.T alert : No

 

單一隻碟 Info.

MegaCli -pdinfo -PhysDrv[E0:S0,E1:S1,...] -aN

ie.

MegaCli -pdinfo -PhysDrv[252:2] -a0

 

Output Example: HardDisk 出事時

Enclosure Device ID: 32
Slot Number: 1
Drive's postion: DiskGroup: 0, Span: 0, Arm: 1
Enclosure position: N/A
Device Id: 1
WWN: 50014ee00327a42c
Sequence Number: 3
Media Error Count: 333
Other Error Count: 909
Predictive Failure Count: 0
Last Predictive Failure Event Seq Number: 0
PD Type: SATA

Raw Size: 465.761 GB [0x3a386030 Sectors]
Non Coerced Size: 465.261 GB [0x3a286030 Sectors]
Coerced Size: 465.25 GB [0x3a280000 Sectors]
Firmware state: Failed
Device Firmware Level: 1S02
Shield Counter: 0
Successful diagnostics completion on :  N/A

更換 HardDisk 之後 Rebuild Raid 中:

Coerced Size: 465.25 GB [0x3a280000 Sectors]
Firmware state: Rebuild
Device Firmware Level: PA07
Shield Counter: 0
Successful diagnostics completion on :  N/A

Plug Disk Event Log

seqNum: 0x0000cdb1
Time: Wed May 27 10:35:39 2020

Code: 0x0000005b
Class: 0
Locale: 0x02
Event Description: Inserted: PD 01(e0xfc/s2)
Event Data:
===========
Device ID: 1
Enclosure Index: 252
Slot Number: 2


seqNum: 0x0000cdb2
Time: Wed May 27 10:35:39 2020

Code: 0x000000f7
Class: 0
Locale: 0x02
Event Description: Inserted: PD 01(e0xfc/s2) Info: enclPd=fc, scsiType=0, portMap=00, sasAddr=4433221101000000,0000000000000000
Event Data:
===========
Device ID: 1
Enclosure Device ID: 252
Enclosure Index: 1
Slot Number: 2
SAS Address 1: 4433221101000000
SAS Address 2: 0

Disk Size

Raw Size: 931.512 GB [0x74706db0 Sectors]
Non Coerced Size: 931.012 GB [0x74606db0 Sectors]
Coerced Size: 931.0 GB [0x74600000 Sectors]
Emulated Drive: Yes
...

coerced capacity:

A drive property indicating the capacity to which a drive has been coerced (forced) to make it compatible with other drives that are nominally the same capacity. For example, a 4-GB drive from one manufacturer might be 4,196 MB, and a 4-GB from another manufacturer might be 4,128 MB. These drives could be coerced to a usable capacity of 4,088 MB each for use in a drive group in a storage configuration.

 


Show Enclosure Info

 

MegaCli64 -EncInfo -aAll

    Number of enclosures on adapter 0 -- 1

    Enclosure 0:
    Device ID                     : 32
    Number of Slots               : 4
    Number of Power Supplies      : 0
    Number of Fans                : 0
    Number of Temperature Sensors : 0
    Number of Alarms              : 0
    Number of SIM Modules         : 0
    Number of Physical Drives     : 4
    Status                        : Normal
    Position                      : 0
    Connector Name                : Unavailable
    Enclosure type                : SES
    FRU Part Number               : N/A
    Enclosure Serial Number       : N/A
    ESM Serial Number             : N/A
    Enclosure Zoning Mode         : N/A
    Partner Device Id             : 65535

    Inquiry data                  :
        Vendor Identification     : DP
        Product Identification    : BACKPLANE
        Product Revision Level    : 1.07
        Vendor Specific           : ???????


Exit Code: 0x00

 


RAID Health & Rebuild

 

# 查看有無死先.

MegaCli64 -PDList -aAll | grep "Firmware state"

# 有Disk 壞左 (有 Error)

Firmware state: Online, Spun Up                      # 0
Firmware state: Online, Spun Up                      # 1
Firmware state: Failed                               # 2
Firmware state: Online, Spun Up                      # 3

# 有 Disk 死左 (沒有反應)

Firmware state: Online, Spun Up
Firmware state: Online, Spun Up
Firmware state: Online, Spun Up

# 查看 rebuild 到幾多 %

Usage:

MegaCli64 -pdrbld -showprog -physdrv\[E:S\] -aALL

# 必須填的資料:

  • E = Enclosure Device ID
  • S = Slot Number

Example

MegaCli64 -PDRbld -ShowProg -PhysDrv [32:2] -a0

  • Enclosure Device ID: 32
  • Slot Number: 2
Device(Encl-32 Slot-2) is not in rebuild process

Exit Code: 0x00

OR

Rebuild Progress on Device at Enclosure 32, Slot 2 Completed 1% in 1 Minutes.

更換 HDD & Rebuild

<1> Add the disk back into the array

Get the disk [32:2] back into array 1, as disk 2:

megacli -PdReplaceMissing -PhysDrv[252:4] -array1 -row2 -a0

<2> 人手 start rebuilt

megacli -PDRbld -Start -PhysDrv[E:S] -a0

i.e.

# Started rebuild progress on device(Encl-32 Slot-2)

MegaCli64 -PDRbld -Start -PhysDrv[32:2] -a0

    Started rebuild progress on device(Encl-32 Slot-2)

checking

MegaCli64 -PDRbld -ShowProg -PhysDrv [32:2] -a0

Auto rebuild

# enable automatic rebuild on adapter 0

MegaCli64 -AdpAutoRbld -Dsply -a0

Adapter 0: AutoRebuild is Enabled.

另有

  • MegaCli64 -AdpAutoRbld -Dsbl -a0
  • MegaCli64 -AdpAutoRbld -Enbl -a0

Rebuild rate

# 查看

MegaCli64 -AdpGetProp RebuildRate -a0

Adapter 0: Rebuild Rate = 30%

# 設定

MegaCli64 -AdpSetProp RebuildRate 60 -a0

Adapter 0: Rebuild Rate = 60%

 


Foreign Configuration Disk Re-add to RAID & Rebuild

 

<1> Make the drive online again:

MegaCli64 -PDMakeGood -PhysDrv[32:2] -a0

* The controller will now recognise the disk as being a "foreign" one.

(It means it has detected some RAID informations on it)

<2>

MegaCli64 -PDList -aAll | grep -e Slot -e Foreign

Slot Number: 0
Foreign State: Foreign
Foreign Secure: Drive is not secured by a foreign lock key
Slot Number: 1
Foreign State: Foreign
Foreign Secure: Drive is not secured by a foreign lock key
Slot Number: 2
Foreign State: None

<3> ask the controller to scan for foreign configuration

MegaCli64 -CfgForeign -Scan -a0

There are 1 foreign configuration(s) on controller 0.

<4>

MegaCli64 -CfgForeign -Dsply -a0

Foreign configuration 0 on controller 0:
Number of DISK GROUPS: 1

DISK GROUP: 0
Number of Spans: 1
SPAN: 0
Virtual Drive Information:
Virtual Drive: 0 (Target Id: 0)
Name                :20161110
RAID Level          : Primary-6, Secondary-0, RAID Level Qualifier-3
Size                : 1.089 TB
...
Physical Disk: 1
This Physical Disk is Missing.
Physical Disk: 2
Enclosure Device ID: 252
...
Physical Disk: 3
This Physical Disk is Missing.

<5> Drop foreign configuration

MegaCli64 -CfgForeign -Clear -a0           #  !! 此 Action 係沒有 Confirm 的

Foreign configuration 0 is cleared on controller 0.

Import

<1>

MegaCli64 -CfgForeign -Preview -a0

Foreign configuration on controller 0(After import):
Number of DISK GROUPS: 1

DISK GROUP: 0
Number of Spans: 1
SPAN: 0
Virtual Drive Information:
Virtual Drive: 0 (Target Id: 0)
Name                :20161110
RAID Level          : Primary-6, Secondary-0, RAID Level Qualifier-3
Size                : 1.089 TB
...

<2>

MegaCli64 -CfgForeign -Import -a0

Remark: Import Fail

The MegaRAID Utility may display a “Failed to Import Foreign Configuration” error.

There are two ways the error may occur:

In a server system, if the HDDs are offline and the system is power cycled, the RAID configuration will be removed.

If the user selects for the HDDs to come back online, they can import the original RAID configuration.

If the user chooses to import an individual RAID configuration (for example, “Configuration 1” or “Configuration 2”), the import will fail.

如果是更新了新 RAID Card, 那可能是新不支援舊
 

<3> Figure out how that disk was identified inside the RAID array [Import 了的 Foreign 才 Display]

MegaCli64 -CfgDsply -a0

 


Cause the front LED of the drive to blink

 

Start blinking

MegaCli -PdLocate -start -physdrv[E:S] -aALL

i.e.

./MegaCli64 -PdLocate -start -physdrv[32:0] -a0

Adapter: 0: Device at EnclId-32 SlotId-0  -- PD Locate Start Command was successfully sent to Firmware

Exit Code: 0x00

Stop blinking

MegaCli -PdLocate -stop -physdrv[E:S] -aALL

 


Event Log

 

# Output RAID controller events to a file(raid_events.log):

MegaCli64 -AdpEventLog -GetEvents -f raid_events.log -aALL

The information shown consists of total number of entries available at firmware side since the last clear and

details of each entries of the error log.

Start_entry specifies the initial event log entry when displaying the log.

# 一定要加 "-f" 因為 console 係無 output 的

# Create a file called megacli_events_since_shutdown that will contain all the events logged by all the controllers,

# this will include warnings, info messages and errors since last shutdown.

megacli -AdpEventLog -GetSinceShutdown -f megacli_events_since_shutdown -aALL

Log 的 S/N

MegaCli64 -AdpEventLog -GetEventlogInfo -aALL

Displays overall event information such as

  • total number of events
  • newest sequence number
  • oldest sequence number
  • clear sequence number
  • shutdown sequence number
  • reboot sequence number
Adapter #0

Newest Seqnum: 0x0000cd8a
Oldest Seqnum: 0x00000001
Clear Seqnum: 0x0000cd6d
Shutdown Seqnum: 0x0000cd6e
Reboot Seqnum: 0x0000cd6f

Success in AdpEventLog

clear 後 reboot

# Consistency check log

MegaCli64 -AdpEventLog -GetCCIncon -f raid_events.log -LAll -aALL

-GetCCIncon: Displays the events relating to inconsistent data found during a consistency check.

# Clear Event Log

MegaCli64 -AdpEventLog -Clear -a0

Success in AdpEventLog

 


BBU

 

https://datahunter.org/raid_bbu

 


Terminal Logging(firmware terminal)

 

# pulls all of the events

MegaCli64 -FwTermLog -Dsply -aALL

03/22/13  8:36:56: EVT#15171-03/22/13  8:36:56: 103=Rebuild progress on PD 03(e0x20/s3) is 23.98%(2052s)
03/22/13  8:38:23: EVT#15172-03/22/13  8:38:23: 103=Rebuild progress on PD 03(e0x20/s3) is 24.98%(2139s)
03/22/13  8:39:51: EVT#15173-03/22/13  8:39:51: 103=Rebuild progress on PD 03(e0x20/s3) is 25.98%(2227s)
03/22/13  8:40:42: DM: Chip 0 Paused
03/22/13  8:40:42: Chip <0> Slots: Cur=[74]
03/22/13  8:40:42: [4c]= 8 [51]= 1
03/22/13  8:40:44: DM: Timing wheel expired - Chip 0 Slot 4c
03/22/13  8:40:44: Chip <0> Slots: Cur=[76]
03/22/13  8:40:44: [4c]= 8 [51]= 1
03/22/13  8:40:44: Timedout RDM: a0566600, Cmd 2 DevId[3], State   20 Flags f1400005
03/22/13  8:40:44: Timedout RDM: a0574e00, Cmd 2 DevId[3], State   20 Flags f1480005
03/22/13  8:40:44: Timedout RDM: a057ac00, Cmd 2 DevId[3], State   20 Flags f1480005
03/22/13  8:40:44: Timedout RDM: a063c400, Cmd 2 DevId[3], State   20 Flags f1480005
03/22/13  8:40:44: Timedout RDM: a065ea00, Cmd 2 DevId[3], State   20 Flags f1480005
03/22/13  8:40:44: Timedout RDM: a066fa00, Cmd 2 DevId[3], State   20 Flags f1480005
03/22/13  8:40:44: Timedout RDM: a06a1600, Cmd 2 DevId[3], State   20 Flags f1480005
03/22/13  8:40:44: Timedout RDM: a0752400, Cmd 2 DevId[3], State   20 Flags f1480005
03/22/13  8:40:44: DM_TMWheelScanAllMsg (slot 4c):Scanned 9, Timeout 8 Msg mask 21000
03/22/13  8:40:44: EVT#15174-03/22/13  8:40:44: 267=Command timeout on PD 03(e0x20/s3) Path 1221000003000000, CDB: 2a 00 1e c2 24 00 00 00 80 00
03/22/13  8:40:44: Raw Sense for PD 3:
03/22/13  8:40:44: EVT#15175-03/22/13  8:40:44: 267=Command timeout on PD 03(e0x20/s3) Path 1221000003000000, CDB: 2a 00 1e c2 24 80 00 00 80 00
03/22/13  8:40:44: Raw Sense for PD 3:
03/22/13  8:40:44: EVT#15176-03/22/13  8:40:44: 267=Command timeout on PD 03(e0x20/s3) Path 1221000003000000, CDB: 2a 00 1e c2 23 80 00 00 80 00
03/22/13  8:40:44: Raw Sense for PD 3:
03/22/13  8:40:44: EVT#15177-03/22/13  8:40:44: 267=Command timeout on PD 03(e0x20/s3) Path 1221000003000000, CDB: 2a 00 1e c2 26 00 00 00 80 00

# Clear BBU terminal log

MegaCli64 -FwTermLog -Clear -a0

Clear log 後會有 log "11/26/14 12:23:44: TTY History buffer cleared" 寫入

電池 與 Log

# 查看電池的 Log

MegaCli64 -FwTermLog -BbuGet -a0

Battery is ON for TTY history on Adapter 0

# Temporarily turn off BBU firmware logging   

MegaCli64 -FwTermLog -BbuoffTemp -a0

# Battery On / Off for terminal log

# MegaCli64 -FwTermLog -Bbuoff -a0

# MegaCli64 -FwTermLog -Bbuon -a0

Battery is set to ON for TTY history on Adapter 0

P.S.

  • FwTermLog(firmware terminal logging)
  • BBU(Battery Backup Unit)

 


Display configuration

 

MegaCli64 -CfgDsply -aAll | less       # 有 Disk 時才有 Config, 因為 Config 係保存在碟的

Output

==============================================================================
Adapter: 0
Product Name: PERC H700 Adapter
Memory: 512MB
BBU: Present
Serial No: ???????
==============================================================================
Number of DISK GROUPS: 1

SPANNED DISK GROUP: 0
Number of Spans: 2
-------------------------------------------------------------------------- SPAN: 0
SPAN: 0
Span Reference: 0x00
Number of PDs: 2
Number of VDs: 1
Number of dedicated Hotspares: 0
Virtual Drive Information:
Virtual Drive: 0 (Target Id: 0)
Name                :20120315
RAID Level          : Primary-1, Secondary-0, RAID Level Qualifier-0
Size                : 930.5 GB
Mirror Data         : 930.5 GB
State               : Degraded
Strip Size          : 64 KB
Number Of Drives per span:2
Span Depth          : 2
Default Cache Policy: WriteBack, ReadAdaptive, Direct, No Write Cache if Bad BBU
Current Cache Policy: WriteBack, ReadAdaptive, Direct, No Write Cache if Bad BBU
Default Access Policy: Read/Write
Current Access Policy: Read/Write
Disk Cache Policy   : Disk's Default
Encryption Type     : None
Bad Blocks Exist: No
Is VD Cached: Yes
Cache Cade Type : Read Only
Physical Disk Information:
---------------------------------------------- Physical Disk: 0
Physical Disk: 0
Enclosure Device ID: 32
Slot Number: 0
Drive's postion: DiskGroup: 0, Span: 0, Arm: 0
Enclosure position: N/A
Device Id: 0
WWN: 50014ee0587ce9b0
Sequence Number: 2
Media Error Count: 0
Other Error Count: 0
Predictive Failure Count: 0
Last Predictive Failure Event Seq Number: 0
PD Type: SATA

Raw Size: 465.761 GB [0x3a386030 Sectors]
Non Coerced Size: 465.261 GB [0x3a286030 Sectors]
Coerced Size: 465.25 GB [0x3a280000 Sectors]
Firmware state: Online, Spun Up
Device Firmware Level: 1S02
Shield Counter: 0
Successful diagnostics completion on :  N/A
SAS Address(0): 0x4433221107000000
Connected Port Number: 2(path0)
Inquiry Data:      ????                  01.01S02
FDE Capable: Not Capable
FDE Enable: Disable
Secured: Unsecured
Locked: Unlocked
Needs EKM Attention: No
Foreign State: None
Device Speed: 3.0Gb/s
Link Speed: 3.0Gb/s
Media Type: Hard Disk Device
Drive Temperature : N/A
PI Eligibility:  No
Drive is formatted for PI information:  No
PI: No PI
Port-0 :
Port status: Active
Port's Linkspeed: 3.0Gb/s
Drive has flagged a S.M.A.R.T alert : No
---------------------------------------------- Physical Disk: 1
Physical Disk: 1
Enclosure Device ID: 32
Slot Number: 1
Drive's postion: DiskGroup: 0, Span: 0, Arm: 1
Enclosure position: N/A
Device Id: 1
WWN: 5000c5004d647155
Sequence Number: 2
Media Error Count: 0
Other Error Count: 0
Predictive Failure Count: 0
Last Predictive Failure Event Seq Number: 0
PD Type: SATA

Raw Size: 465.761 GB [0x3a386030 Sectors]
Non Coerced Size: 465.261 GB [0x3a286030 Sectors]
Coerced Size: 465.25 GB [0x3a280000 Sectors]
Firmware state: Online, Spun Up
Device Firmware Level: PA07
Shield Counter: 0
Successful diagnostics completion on :  N/A
SAS Address(0): 0x4433221106000000
Connected Port Number: 3(path0)
Inquiry Data:             ????                                 PA07
FDE Capable: Not Capable
FDE Enable: Disable
Secured: Unsecured
Locked: Unlocked
Needs EKM Attention: No
Foreign State: None
Device Speed: 3.0Gb/s
Link Speed: 3.0Gb/s
Media Type: Hard Disk Device
Drive Temperature : N/A
PI Eligibility:  No
Drive is formatted for PI information:  No
PI: No PI
Port-0 :
Port status: Active
Port's Linkspeed: 3.0Gb/s
Drive has flagged a S.M.A.R.T alert : No


-------------------------------------------------------------------------- SPAN: 1
SPAN: 1
Span Reference: 0x01
Number of PDs: 2
Number of VDs: 1
Number of dedicated Hotspares: 0
Virtual Drive Information:
Virtual Drive: 0 (Target Id: 0)
Name                :20120315
RAID Level          : Primary-1, Secondary-0, RAID Level Qualifier-0
Size                : 930.5 GB
Mirror Data         : 930.5 GB
State               : Degraded
Strip Size          : 64 KB
Number Of Drives per span:2
Span Depth          : 2
Default Cache Policy: WriteBack, ReadAdaptive, Direct, No Write Cache if Bad BBU
Current Cache Policy: WriteBack, ReadAdaptive, Direct, No Write Cache if Bad BBU
Default Access Policy: Read/Write
Current Access Policy: Read/Write
Disk Cache Policy   : Disk's Default
Encryption Type     : None
Bad Blocks Exist: No
Is VD Cached: Yes
Cache Cade Type : Read Only
Physical Disk Information:
---------------------------------------------- Physical Disk: 0
Physical Disk: 0
Enclosure Device ID: 32
Slot Number: 2
Drive's postion: DiskGroup: 0, Span: 1, Arm: 0
Enclosure position: N/A
Device Id: 2
WWN: 50014ee0592ab20c
Sequence Number: 22
Media Error Count: 0
Other Error Count: 1
Predictive Failure Count: 0
Last Predictive Failure Event Seq Number: 0
PD Type: SATA

Raw Size: 465.761 GB [0x3a386030 Sectors]
Non Coerced Size: 465.261 GB [0x3a286030 Sectors]
Coerced Size: 465.25 GB [0x3a280000 Sectors]
Firmware state: Failed
Device Firmware Level: 1S03
Shield Counter: 0
Successful diagnostics completion on :  N/A
SAS Address(0): 0x4433221105000000
Connected Port Number: 1(path0)
Inquiry Data:      ????                   01.01S03
FDE Capable: Not Capable
FDE Enable: Disable
Secured: Unsecured
Locked: Unlocked
Needs EKM Attention: No
Foreign State: None
Device Speed: 6.0Gb/s
Link Speed: 6.0Gb/s
Media Type: Hard Disk Device
Drive:  Not Certified
Drive Temperature : N/A
PI Eligibility:  No
Drive is formatted for PI information:  No
PI: No PI
Port-0 :
Port status: Active
Port's Linkspeed: 6.0Gb/s
Drive has flagged a S.M.A.R.T alert : No
---------------------------------------------- Physical Disk: 1
Physical Disk: 1
Enclosure Device ID: 32
Slot Number: 3
Drive's postion: DiskGroup: 0, Span: 1, Arm: 1
Enclosure position: N/A
Device Id: 3
WWN: 50014ee0587ce09a
Sequence Number: 2
Media Error Count: 1
Other Error Count: 0
Predictive Failure Count: 0
Last Predictive Failure Event Seq Number: 0
PD Type: SATA

Raw Size: 465.761 GB [0x3a386030 Sectors]
Non Coerced Size: 465.261 GB [0x3a286030 Sectors]
Coerced Size: 465.25 GB [0x3a280000 Sectors]
Firmware state: Online, Spun Up
Device Firmware Level: 1S02
Shield Counter: 0
Successful diagnostics completion on :  N/A
SAS Address(0): 0x4433221104000000
Connected Port Number: 0(path0)
Inquiry Data:      ????                  01.01S02
FDE Capable: Not Capable
FDE Enable: Disable
Secured: Unsecured
Locked: Unlocked
Needs EKM Attention: No
Foreign State: None
Device Speed: 3.0Gb/s
Link Speed: 3.0Gb/s
Media Type: Hard Disk Device
Drive Temperature : N/A
PI Eligibility:  No
Drive is formatted for PI information:  No
PI: No PI
Port-0 :
Port status: Active
Port's Linkspeed: 3.0Gb/s
Drive has flagged a S.M.A.R.T alert : No

 


時間

 

MegaCli64 -AdpGetTime -aALL

Adapter 0:
    Date: 03/22/2013
    Time: 10:37:11

Exit Code: 0x00

 

設定時間

MegaCli64 –AdpSetTime yyyymmdd HH:mm:ss -a0

e.g. 1: 人手設定

MegaCli64 -AdpSetTime 20141126 11:00:00 -a0

New date/time is set on Adapter 0

Exit Code: 0x00

e.g. 2: 用系統的時鍾

MegaCli64 -AdpSetTime `date +%Y%m%d` `date +%H:%M:%S` -aALL

 


Number of X

 

# Total Number of Drives

MegaCli64 -PDGetNum -aAll

Number of Physical Drives on Adapter 0: 4

# Display the Number of Virtual Drives

MegaCli64 -LDGetNum -aAll

Number of Virtual Drives Configured on Adapter 0: 1

# Virtual Drive Information

MegaCli64 -LDInfo -Lall -aAll

Virtual Drive: 0 (Target Id: 0)
Name                :20120315
RAID Level          : Primary-1, Secondary-0, RAID Level Qualifier-0
Size                : 930.5 GB
Mirror Data         : 930.5 GB
State               : Degraded
Strip Size          : 64 KB
Number Of Drives per span:2
Span Depth          : 2
Default Cache Policy: WriteBack, ReadAdaptive, Direct, No Write Cache if Bad BBU
Current Cache Policy: WriteBack, ReadAdaptive, Direct, No Write Cache if Bad BBU
Default Access Policy: Read/Write
Current Access Policy: Read/Write
Disk Cache Policy   : Disk's Default
Encryption Type     : None
Bad Blocks Exist: No
Is VD Cached: Yes
Cache Cade Type : Read Only

# 某 Logical Volume 的 info.

# detailed information on the drives in the array

MegaCli64 -LDPDInfo -aAll | less

Number of Virtual Disks: 1
Virtual Drive: 0 (Target Id: 0)
Name                :20130311
RAID Level          : Primary-1, Secondary-3, RAID Level Qualifier-0
Size                : 1.818 TB
Mirror Data         : 1.818 TB
State               : Degraded
Strip Size          : 64 KB
Number Of Drives per span:2
Span Depth          : 2
Default Cache Policy: WriteBack, ReadAheadNone, Direct, No Write Cache if Bad BBU
Current Cache Policy: WriteBack, ReadAheadNone, Direct, No Write Cache if Bad BBU
Default Access Policy: Read/Write
Current Access Policy: Read/Write
Disk Cache Policy   : Disk's Default
Encryption Type     : None
Is VD Cached: No
Number of Spans: 2
Span: 0 - Number of PDs: 2

PD: 0 Information
Enclosure Device ID: 32
Slot Number: 0
Drive's postion: DiskGroup: 0, Span: 0, Arm: 0
...
PD Type: SATA
...

PD: 1 Information
...

Span: 1 - Number of PDs: 2

PD: 0 Information
Enclosure Device ID: 32
Slot Number: 2
Drive's postion: DiskGroup: 0, Span: 1, Arm: 0

PD: 1 Information
...

 


Check 在 RAID 上的 Disk SMART

 

# 找 Device Id

MegaCli64 -PDList -aAll

# 有問題的 Disk

Enclosure Device ID: 32
Slot Number: 2
Drive's postion: DiskGroup: 0, Span: 0, Arm: 2
Enclosure position: N/A
Device Id: 2
WWN: 50014ee0adde4846
Sequence Number: 3
Media Error Count: 6
Other Error Count: 6
Predictive Failure Count: 0
Last Predictive Failure Event Seq Number: 0
PD Type: SATA

Raw Size: 465.761 GB [0x3a386030 Sectors]
Non Coerced Size: 465.261 GB [0x3a286030 Sectors]
Coerced Size: 465.25 GB [0x3a280000 Sectors]
Firmware state: Failed
Device Firmware Level: 1S02
Shield Counter: 0
Successful diagnostics completion on :  N/A
SAS Address(0): 0x4433221105000000
Connected Port Number: 1(path0)
Inquiry Data:      WD-WMAYP3444742WDC WD5003ABYX-18WERA0                  01.01S02
FDE Capable: Not Capable
FDE Enable: Disable
Secured: Unsecured
Locked: Unlocked
Needs EKM Attention: No
Foreign State: None
Device Speed: 3.0Gb/s
Link Speed: 3.0Gb/s
Media Type: Hard Disk Device
Drive Temperature : N/A
PI Eligibility:  No
Drive is formatted for PI information:  No
PI: No PI
Port-0 :
Port status: Active
Port's Linkspeed: 3.0Gb/s
Drive has flagged a S.M.A.R.T alert : No

................................

查看未建立 Volume 的 Disk 的 SMART

smartctl --scan

...
/dev/bus/0 -d megaraid,1 # /dev/bus/0 [megaraid_disk_01], SCSI device
/dev/bus/0 -d megaraid,12 # /dev/bus/0 [megaraid_disk_12], SCSI device
/dev/bus/0 -d megaraid,13 # /dev/bus/0 [megaraid_disk_13], SCSI device

smartctl -d megaraid,1 -i /dev/bus/0

smartctl -d megaraid,1 -A /dev/bus/0

不支援 SMART 的 Disk

smartctl -d megaraid,12 -a /dev/bus/0

...
SMART support is:     Unavailable - device lacks SMART capability.
...

 


Physical drive management

 

Step

  1. 找出 HDD
  2. Set HDD State to Offline
  3. Mark as missing
  4. 換碟

1. 找出 HDD

MegaCli64 -PDList -aAll | grep -e 'Enclosure Device' -e Slot -e Inquiry -e Raw

Enclosure Device ID: 252
Slot Number: 0
Raw Size: 558.911 GB [0x45dd2fb0 Sectors]
Inquiry Data: SEAGATE ST3600057SS     ????
Enclosure Device ID: 252
Slot Number: 1
Raw Size: 558.911 GB [0x45dd2fb0 Sectors]
Inquiry Data: SEAGATE ST3600057SS     ????
Enclosure Device ID: 252
Slot Number: 2
Raw Size: 931.512 GB [0x74706db0 Sectors]
Inquiry Data: Z9A1P2K1ST1000DM003-1SB10C     CC43

2. Set HDD State to Offline (offline 不代表可以 unplug 佢)

# Set the drive offline, if it is not already offline due to an error

MegaCli64 -PDOffline -PhysDrv [E:S] -aN

i.e.

MegaCli64 -PDOffline -PhysDrv [252:0] -a0

Adapter: 0: EnclId-252 SlotId-0 state changed to OffLine.

MegaCli64 -PDList -aAll | grep -i -e slot -e firmware

Slot Number: 0
Firmware state: Unconfigured(good), Spun down

3. Mark as missing

MegaCli -PDMarkMissing -PhysDrv [E:S] -aN

 * If you want to make a configured drive as missing

 * "-PdMarkMissing" works only on offline drives.

 *  When -PdReplaceMissing is run, the drive becomes offline,

     and rebuild does not start automatically. You have to start it explicitly.

View missing

MegaCli64 -PdGetMissing -aAll

    Adapter 0 - Missing Physical drives

    No.   Array   Row   Size Expected
    0     1       1     953344 MB

4. 換碟

抽出壞 Disk 並插入新 Disk

5. Set New State to Online

MegaCli -PDOnline -PhysDrv [E:S] -aN

6. Re-add the new drive to virtual drive

MegaCli -PdReplaceMissing -physdrv[E0:S0] -arrayA, -rowB -a0

OR

MegaCli -PDHSP -Set -PhysDrv [E:S] -a0                                # Global Hot Spare

MegaCli -PDHSP -Set -Dedicated -ArrayN -PhysDrv [E:S] -a0    # Dedicated

Remark

MegaCli -CfgDsply -a0

7. Start the rebuilding

MegaCli -PDRbld -Start -PhysDrv [E:S] -a0

--------

Prepare Unconfigured Drives for Removal (Spun down disk)

Remove a non-failed drive that is connected to the controller

=> Firmware spins down this drive.

=> The drive state is set to Unaffiliated, which marks it as offline even though it is not a part of a configuration.

# -Undo: This option undoes this operation. If you select undo, firmware marks this drive as Unconfigured Good.

MegaCli64 -PDPrpRmv [-Undo] -PhysDrv[E:S] -a0

i.e.

MegaCli64 -PDPrpRmv -PhysDrv[252:0] -a0

Prepare for removal Success

--------

Unconfigured(bad)

Drives stuck in the Unconfigured(bad) state need to be manually set back to Unconfigured(good) before they will be importable.

Failure to import arrays usually means that the drives were unexpectedly powered off or disconnected from the RAID controller,

which puts them in the Unconfigured(bad) state.

原因:

This is done as a precautionary measure to prevent drives with bad connections from joining the array,

as sudden disconnects can result in data corruption.

MegaCli64 -PDList -a0

...
Firmware state: Unconfigured(bad)
...

MegaCli64 -PDMakeGood -PhysDrv[E0:S0,E1:S1,...]

i.e.

MegaCli64 -PDMakeGood -PhysDrv[252:1] -a0

Adapter: 0: EnclId-252 SlotId-1 state changed to Unconfigured-Good.

 


建立 Virtual Drive

 

Step1: List 可用的 Disk

# 可用的 Disk = Firmware state: Unconfigured(good), Spun Up

MegaCli64 -PDList -a0

Step2: 建立 VD

MegaCli -CfgLdAdd R0|R1|R5|R6[E0:S0,E1:S1,...]\
 [WT|WB] [NORA|RA] [Direct|Cached] [CachedBadBBU|NoCachedBadBBU] \
 [-szXXXXXXXX [-szYYYYYYYY [... ]]]  \
 [-strpszM] [-Hsp[E5:S5,...]] -aN

i.e.

MegaCli64 -CfgLdAdd R1[252:0,252:1] -a0

MegaCli64 -CfgLdAdd R5[32:1,32:2,32:3] WB RA Cached -a0

 * Default: WT, NORA

-Rx[E0:S0,...]

Specifies the RAID level and the drive enclosure/slot numbers to construct a drive group.

-strpszM

Specifies the stripe size, where the stripe size values are

8 KB, 16 KB, 32 KB, 64 KB, 128 KB, 256 KB, 512 KB, or 1024 KB.

-Hsp[E5:S5,...]

Creates hot spares when you create the configuration.

IO Policy (Direct|Cached)

Direct(default):

reads are not buffered in cache memory.

Data is transferred to the cache and the host concurrently.

If the same data block is read again, it comes from cache memory.

Cached:

all reads are buffered in cache memory.

 * 此 Setting 不會影響 RA

Drive Cache (Enable|Disable)

Specify the drive cache policy.

  • Enable: Enable the drive cache.
  • Disable(default): Disable the drive cache.

i.e.

[A]

MegaCli64 -cfgldadd R1[252:0,252:1] -a0

Adapter 0: Created VD 1

Adapter 0: Configured the Adapter!!

[B]

MegaCli64 -cfgldadd R1[252:0,252:1] WB RA -a0

Step3: 查看建立出來的 VD Info.

MegaCli64 -LDInfo -LAll -a0 | less

Adapter 0 -- Virtual Drive Information:
Virtual Drive: 1 (Target Id: 1)
Name                :
RAID Level          : Primary-1, Secondary-0, RAID Level Qualifier-0
Size                : 465.25 GB
Is VD emulated      : No
Mirror Data         : 465.25 GB
State               : Optimal
Strip Size          : 64 KB
Number Of Drives    : 2
Span Depth          : 1
Default Cache Policy: WriteThrough, ReadAheadNone, Direct, No Write Cache if Bad BBU
Current Cache Policy: WriteThrough, ReadAheadNone, Direct, No Write Cache if Bad BBU
Default Access Policy: Read/Write
Current Access Policy: Read/Write
Disk Cache Policy   : Disabled
Ongoing Progresses:
  Background Initialization: Completed 0%, Taken 0 min.
Encryption Type     : None
Bad Blocks Exist: No
Is VD Cached: No

Step4: 設定(-LDSetProp)

MegaCli -LDSetProp -Name LdNamestring -LN -aN
MegaCli -LDSetProp SETTING -LN -aN

Allows you to change the following SETTING:

  • Write policy: WT (Write through), WB (Write back)
  • Specifies whether to use write cache when the BBU is bad: ForcedWB | CachedBadBBU | NoCachedBadBBU
  • Read policy: NORA (No read ahead), RA (Read ahead)
  • IO cache policy: Cached, Direct
  • Access policy: RW, RO, Blocked, RemoveBlocked
  • Drive(Disk) cache: EnDskCache | DisDskCache
  • Immediate

ForcedWB        # To enable cache even without BBU

i.e.

# Rename

MegaCli64 -LDSetProp -NAME R1 -L1 -a0

Set name to R1 on Adapter 0, VD 1 (target id: 1) success

# Set Cache

MegaCli64 -LDSetProp WB Immediate -L1 -a0

MegaCli64 -LDSetProp RA Immediate -L1 -a0           # 只可以一行行設定

Set BGI Rate

MegaCli64 -AdpSetProp BgiRate -100 -a 0

Adapter 0: Set BGI Rate to 100% success.

MegaCli64 -AdpGetProp BgiRate -a0

Adapter 0: BGI Rate = 100%

Remove Virtual Drive

 * 此 CMD 係沒有 Comfirm 的 !!

MegaCli64 -CfgLdDel -L1 -a0

Adapter 0: Deleted Virtual Drive-1(target id-1)

 


All Disk To RAID0 (CfgEachDskRaid0)

 

MegaCli64 -CfgEachDskRaid0 -a0

Adapter 0: Created VD 0
Configured physical device at Encl-252:Slot-2.
Adapter 0: Created VD 1
Configured physical device at Encl-252:Slot-1.
Adapter 0: Created VD 2
Configured physical device at Encl-252:Slot-0.

3 physical devices are Configured on adapter 0.

dmesg

[11182.567632] scsi 0:2:0:0: Direct-Access     LSI      RAID 5/6 SAS 6G  2.13 PQ: 0 ANSI: 5
[11182.570212] sd 0:2:0:0: Attached scsi generic sg3 type 0
[11182.570334] sd 0:2:0:0: [sdd] 1952448512 512-byte logical blocks: (1000 GB/931 GiB)
[11182.570336] sd 0:2:0:0: [sdd] 4096-byte physical blocks
[11182.570430] sd 0:2:0:0: [sdd] Write Protect is off
[11182.570439] sd 0:2:0:0: [sdd] Mode Sense: 1f 00 10 08
[11182.570493] sd 0:2:0:0: [sdd] Write cache: disabled, read cache: disabled, supports DPO and FUA
[11182.593818] sd 0:2:0:0: [sdd] Attached SCSI disk
[11182.629587] megaraid_sas 0000:02:00.0: scanning for scsi0...
[11220.626376] scsi 0:2:1:0: Direct-Access     LSI      RAID 5/6 SAS 6G  2.13 PQ: 0 ANSI: 5
[11220.626737] sd 0:2:1:0: Attached scsi generic sg4 type 0
[11220.626783] sd 0:2:1:0: [sde] 1171062784 512-byte logical blocks: (600 GB/558 GiB)
[11220.626806] sd 0:2:1:0: [sde] Write Protect is off
[11220.626807] sd 0:2:1:0: [sde] Mode Sense: 1f 00 10 08
[11220.626875] sd 0:2:1:0: [sde] Write cache: disabled, read cache: disabled, supports DPO and FUA
[11220.633490] sd 0:2:1:0: [sde] Attached SCSI disk
[11241.554459] megaraid_sas 0000:02:00.0: scanning for scsi0...
....

一隻隻碟加

MegaCli64 -cfgldadd R0[252:2] -a0

Adapter 0: Created VD 1
Adapter 0: Configured the Adapter!!

MegaCli64 -LDInfo -L1 -a0

MegaCli64 -CfgLdDel -L1 -a0             # 是沒有 Confirm 的

 


Clear Drive (PDClear)

 

MegaCli -PDClear -Start -PhysDrv [E:S] -aN

i.e.

MegaCli -PDClear -Start -PhysDrv [252:2] -a0

Started clear progress on device(Encl-252 Slot-2)

MegaCli -PDClear -Stop -PhysDrv [E:S] -aN

MegaCli -PDClear -ShowProg -PhysDrv [E:S] -aN

i.e.

MegaCli -PDClear -ShowProg -PhysDrv [252:2] -a0

Clear Progress on Device at Enclosure 252, Slot 2 Completed 0% in 0 Minutes.

 


Other

 

MegaCli64 -AdpCacheFlush -a0

Cache Flush is successfully done on adapter 0.

 


IT 與 IR Mode

 

IT mode stands for "initiator target".

It presents each drive individually to the host.

IR mode is in whatever raid formats the HBA supports.

Some allow you to present drives individually as JBODs or raid0 with one drive each.

=============================

crossflash

D2616 HBA which is based on LSI SAS2108

 


Disk Cache Policy

 

如果 raid 控制器為 writeback, Disk Cache 是否開啟對系統的寫性能影響很小

 * Default: Disable

 * RAID 卡的 BBU 特性則不能保護磁盤緩存的數據.

 


I/O Policy

 

The I/O Policy determines whether the RAID controller will keep data in the cache,

which can reduce the access time if subsequent read requests are made to the same data blocks.

Direct I/O (Recommended in most cases)

直接從磁盤讀取數據, 不使用 raid cache 功能.

Recommended 原因:

Most file systems and many applications have their own cache and

do not require caching data at the RAID controller level.

Cached I/O

The controller caches both read and write requests.

If there are subsequent read requests to the same data blocks,

they are read from the RAID cache instead of the hard drives.

This mode may be required if the application or file system does not cache read requests.

 


Read Ahead

 

The RAID controller will read the whole stripe containing the requested data block and will keep it in cache.

Note:

This setting will only make difference if the typical read request size is smaller than the stripe width.

 


Stripe Width 與 Stripe Size

 

The properties of a distributed array

Stripe Width

The number of parallel stripes that can be written to or read from simultaneously

Which equals the number of drives in a RAID array

作用: 決定了同時讀寫能用

Strip Size

The smallest chunk of data within a RAID array that can be addressed.

It also determines how many drives are accessed to service a single I/O request.

Increasing & Decreasing the Stripe Size

Increasing Stripe Size: Fewer drives are required to store files of a given size.

 => allows the drives not needed for a particular access to be used for another one

 => improving positioning performance.

Decreasing Stripe Size: theoretically increasing transfer performance, but decreasing positioning performance

Note:

Matching the strip size to the file system cluster size does not usually provide any benefit.

Data block or file sizes used by the application are usually more important.

However, setting the strip size smaller than the cluster size is not recommended.

larger stripe sizes enable more I/O operations per second, and I/O performance increases even more if large stripe sizes are used.
( 對於 Read / Write throughput 沒大影響)

 


PatrolRead and ConsistencyChecking

 

Link

 


Cheat List

 

  • MegaCli64 -AdpAllInfo -aALL | less
  • ---------------------
  • MegaCli64 -PDList -a0                                            # 找出所有 Phyical Disk
  • MegaCli64 -PDList -a0 | grep "Firmware state"          # check disk status
  • ---------------------
  • MegaCli64 -LDInfo -LAll -a0                                     #
  • ---------------------
  • smartctl --scan
  • smartctl -d megaraid,1 -a /dev/sda                           # 某 Disk 的 S.M.A.R.K
  • ---------------------
  • MegaCli64 -PDRbld -ShowProg -PhysDrv [32:2] -a0    # % rebuild
  • ---------------------
  • MegaCli64 -CfgForeign -Scan -a0
  • MegaCli64 -CfgForeign -Dsply -a0
  • MegaCli64 -CfgForeign -Import -a0
  • ---------------------
  • MegaCli64 -AdpSetTime `date +%Y%m%d` `date +%H:%M:%S` -aALL
  • MegaCli64 -AdpGetTime -aALL
  • ---------------------
  • MegaCli64 -AdpEventLog -GetEvents -f raid_events.log -aALL

 

 

 

Creative Commons license icon Creative Commons license icon