storcli

由 datahunter 在五, 17/11/2017 - 15:46 發表

最後更新: 2019-04-01

介紹

* storcli replaced megacli

Check Version
設定 RAID 卡時間
Help
Basic Info
PD 與 DG
Info. about controller
Locate a physical disk by blinking it's LED
Phyical Disk info
HDD SMART
Firmware Update
Activation of Features
JBOD
Creating a Virtual Drive
查看 RAID 內有什麼 Disk
Virtual Drives
Rebuild Status
Consistency Check Commands
Initialization
電池
Cache Setting
Set Drive State Commands (online/offline)
spindown & spinup
Remove Foreign Config
Import Foreign Configuration
實習1~4

Check Version

storcli64 -v

StorCli SAS Customization Utility Ver 1.23.02 Mar 28, 2017

設定 RAID 卡時間

storcli64 /c0 show time

------------------------------
Ctrl_Prop Value
------------------------------
Time      2019/04/01 09:54:41
------------------------------

storcli64 /c0 set time=yyyymmdd hh:mm:ss | systemtime

ie.

storcli64 /c0 set time=systemtime

Help

storcli64 help

/cx        Controller specific commands
/ex        Enclosure specific commands
/sx        Slot/PD specific commands
/vx        Virtual drive specific commands
/dx        Disk group specific commands
/px        Phy specific commands

Basic Info

storcli64 show

-----------------------------------------------------------------------------------------
Ctl Model                        Ports PDs DGs DNOpt VDs VNOpt BBU  sPR DS EHS ASOs Hlth
-----------------------------------------------------------------------------------------
  0 RAIDCtrlSAS6G5/6512MB(D2616)     8   2   0     0   0     0 Msng On  -  Y      2 Opt
-----------------------------------------------------------------------------------------

說明

PDs=Physical drives
DGs=Drive groups
DNOpt=DG NotOptimal
VDs=Virtual drives
VNOpt=VD NotOptimal
BBU=Battery backup unit
sPR=Scheduled Patrol Read
DS=DimmerSwitch
EHS=Emergency Hot Spare
ASOs=Advanced Software Options
Hlth=Health

Health

Fld=Failed
Opt=Optimal
Msng=Missing
Dgd=Degraded
NdAtn=Need Attention
Unkwn=Unknown
Safe=Safe-mode boot

PD 與 DG

PD 組成 DG

DG 分割成 VD

Info. about controller

# Show all the info related to controller 0

storcli64 /c0 show all

Basics :
======

Version :
=======

Bus :
===

Pending Images in Flash :
=======================

Status :
======

Supported Adapter Operations :
============================

Supported PD Operations :
=======================

Supported VD Operations :
=======================

Advanced Software Option :
========================

HwCfg :
=====

Policies :
========

Policies Table :
==============

Boot :
====

High Availability :
=================

Defaults :
========

Capabilities :
============

Scheduled Tasks :
===============

TOPOLOGY :
========

VD LIST :
=======

PD LIST :
=======

Locate a physical disk by blinking it's LED

storcli64 /c0 show activityforlocate

-----------------------------------
Ctrl_Prop                    Value
-----------------------------------
Use Disk Activity For Locate ON
-----------------------------------

* 必須要 ON 才用到 locate 功能

# 把它設定成 ON

storcli64 /c0 set activityforlocate=on

# s5 閃

storcli64 /c0/e252/s5 start locate

Phyical Disk Info.

# 例出碟 Info.

storcli64 /c0 /eall /sall show

Drive Information :
=================

----------------------------------------------------------------------------------
EID:Slt DID State DG     Size Intf Med SED PI SeSz Model                  Sp Type
----------------------------------------------------------------------------------
252:0     3 Onln   0 5.457 TB SATA HDD N   N  512B WDC WD6002VRYZ-02FXTB1 U  -
252:1     2 Onln   0 5.457 TB SATA HDD N   N  512B WDC WD6002VRYZ-02FXTB1 U  -
252:2     1 UGood  - 5.457 TB SATA HDD N   N  512B WDC WD6002VRYZ-02FXTB1 U  -
252:3     0 UGood  - 5.457 TB SATA HDD N   N  512B WDC WD6002VRYZ-02FXTB1 U  -
----------------------------------------------------------------------------------

說明

UGood = Unconfigured Good
Onln = Online
U-Up|D-Down

# 查看某一隻 Disk

storcli /cx/px show

ie. storcli64 /c0/eall/s0 show

Drive Information :
=================

----------------------------------------------------------------------------------
EID:Slt DID State DG     Size Intf Med SED PI SeSz Model                  Sp Type
----------------------------------------------------------------------------------
252:0     3 Onln   1 3.637 TB SATA HDD N   N  512B WDC WD4002FYYZ-01B7CB0 U  -
----------------------------------------------------------------------------------

# Information regarding the available disks and their current state (Serial Number, Model Number, Temperature etc.)

storcli64 /c0 /eall /sall show all

HDD SMART

# Checking for hard disk access

cat /proc/scsi/scsi

Attached devices:
Host: scsi0 Channel: 02 Id: 00 Lun: 00
  Vendor: LSI      Model: RAID 5/6 SAS 6G  Rev: 2.13 # 即使是 RAID 1 也會用此 Model Name
  Type:   Direct-Access                    ANSI  SCSI revision: 05
Host: scsi0 Channel: 02 Id: 01 Lun: 00
  Vendor: LSI      Model: RAID 5/6 SAS 6G  Rev: 2.13
  Type:   Direct-Access                    ANSI  SCSI revision: 05
Host: scsi0 Channel: 02 Id: 02 Lun: 00
  Vendor: LSI      Model: RAID 5/6 SAS 6G  Rev: 2.13
  Type:   Direct-Access                    ANSI  SCSI revision: 05
Host: scsi1 Channel: 00 Id: 00 Lun: 00
  Vendor: ATA      Model: KINGSTON SA400S3 Rev: 71E0
  Type:   Direct-Access                    ANSI  SCSI revision: 05

# Scans for devices and prints each device name, device type and protocol ([ATA] or [SCSI])

smartctl --scan

...
/dev/bus/0 -d megaraid,0 # /dev/bus/0 [megaraid_disk_00], SCSI device
/dev/bus/0 -d megaraid,1 # /dev/bus/0 [megaraid_disk_01], SCSI device
/dev/bus/0 -d megaraid,2 # /dev/bus/0 [megaraid_disk_02], SCSI device
/dev/bus/0 -d megaraid,3 # /dev/bus/0 [megaraid_disk_03], SCSI device
/dev/bus/0 -d megaraid,12 # /dev/bus/0 [megaraid_disk_12], SCSI device
/dev/bus/0 -d megaraid,13 # /dev/bus/0 [megaraid_disk_13], SCSI device
/dev/bus/0 -d megaraid,14 # /dev/bus/0 [megaraid_disk_14], SCSI device
/dev/bus/0 -d megaraid,15 # /dev/bus/0 [megaraid_disk_15], SCSI device

storcli64 /c0/dall show | grep DRIVE

----------------------------------------------------------------------------
DG Arr Row EID:Slot DID Type  State BT      Size PDC  PI SED DS3  FSpace TR
----------------------------------------------------------------------------
 0 0   0   252:4    12  DRIVE Onln  N   5.457 TB dsbl N  N   dflt -      N
 0 0   1   252:5    13  DRIVE Onln  N   5.457 TB dsbl N  N   dflt -      N
 1 0   0   252:0    3   DRIVE Rbld  Y   3.637 TB dsbl N  N   dflt -      N
 1 0   1   252:1    2   DRIVE Onln  N   3.637 TB dsbl N  N   dflt -      N
 1 0   2   252:2    1   DRIVE Onln  N   3.637 TB dsbl N  N   dflt -      N
 1 0   3   252:3    0   DRIVE Onln  N   3.637 TB dsbl N  N   dflt -      N
 2 0   0   252:6    14  DRIVE Onln  N   7.276 TB dsbl N  N   dflt -      N
 2 0   1   252:7    15  DRIVE Onln  N   7.276 TB dsbl N  N   dflt -      N

# <N> stands for the device ID on the RAID controller.

smartctl -a -d megaraid,N /dev/bus/0

Firmware Update

storcli /cx download file=/path/to/firmware.rom

Activation of Features

e.g. CacheCade, FastPath,...

storcli /cx set aso key=AAAAAAAABBBBBBBBCCCCCCCC

JBOD

JBOD means the individual disks are presented (to a server) with no amalgamation, pooling or structure applied.

The term is in widespread use, especially in the context of computers that have software volume management,

such as LVM (AIX, HP-UX, Linux), DiskSuite (Solaris), ZFS (Solaris), Veritas Volume Manager (Unixes), Windows and so on.

功能: passthrough (to unraid) <-- disks untouched directly to the operating system without configuration

# JBOD (just a bunch of disks or just a bunch of drives)

# Check raid card 是否支援 JBOD

storcli64 /c0 show all | grep -i jbod

Support JBOD = No
Support SecurityonJBOD = No
Support JBOD Write cache = No
Enable JBOD = No

Creating a Virtual Drive

storcli64 /cx add vd r[0|1|5|6|00|10|50|60] name=xxxxxxxx \
drives=[EnclosureID:SlotID|:SlotID-SlotID|:SlotID,SlotID] [WT|WB]

* Default: Write back & Read ahead

i.e.

# 用 "storcli64 /c0 /eall /sall show" 找出要用來建立 RAID1 的 DISK

storcli64 /c0 add vd type=r1 drives=252:2-3 wt

Controller = 0
Status = Success
Description = Add VD Succeeded

查看 RAID 內有什麼 Disk

storcli64 /c0/dall show

TOPOLOGY :
========

---------------------------------------------------------------------------
DG Arr Row EID:Slot DID Type  State BT     Size PDC  PI SED DS3  FSpace TR
---------------------------------------------------------------------------
 0 -   -   -        -   RAID1 Optl  N  5.457 TB dsbl N  N   dflt N      N
 0 0   -   -        -   RAID1 Optl  N  5.457 TB dsbl N  N   dflt N      N
 0 0   0   252:0    3   DRIVE Onln  N  5.457 TB dsbl N  N   dflt -      N
 0 0   1   252:1    2   DRIVE Onln  N  5.457 TB dsbl N  N   dflt -      N
 1 -   -   -        -   RAID1 Dgrd  N  5.457 TB dsbl N  N   dflt N      N
 1 0   -   -        -   RAID1 Dgrd  N  5.457 TB dsbl N  N   dflt N      N
 1 0   0   252:2    1   DRIVE Onln  N  5.457 TB dsbl N  N   dflt -      N
 1 0   1   -        -   DRIVE Msng  -  5.457 TB -    -  -   -    -      N
---------------------------------------------------------------------------

Row 顯示有 0, 1 的就代表 Disk. 當它的 EID:Slot 時, 代表 Msng (Missing)

Virtual Drives

# 查看所有 VD

storcli64 /c0 /vall show

Virtual Drives :
==============

--------------------------------------------------------------------
DG/VD TYPE  State Access Consist Cache Cac sCC     Size Name
--------------------------------------------------------------------
0/0   RAID1 Optl  RW     Yes     NRWTD -   OFF 5.457 TB 20171031_R1
1/1   RAID1 Optl  RW     No      NRWTD -   OFF 5.457 TB 20171117_R1
--------------------------------------------------------------------

注意: Consist 是 No, 因為佢在 INIT 中

"NRWTD"

NR=No Read Ahead
WT=WriteThrough
WB=WriteBack
D=Direct IO

# 看某一 VD

storcli64 /c0/v1 show all

Controller = 0
Status = Success
Description = None


/c0/v1 :
======

--------------------------------------------------------------------
DG/VD TYPE  State Access Consist Cache Cac sCC     Size Name
--------------------------------------------------------------------
1/1   RAID1 Optl  RW     No      NRWTD -   OFF 5.457 TB 20171117_R1
--------------------------------------------------------------------

Cac=CacheCade|Rec=Recovery|OfLn=OffLine|Pdgd=Partially Degraded|Dgrd=Degraded
Optl=Optimal|RO=Read Only|RW=Read Write|HD=Hidden|TRANS=TransportReady|B=Blocked|
Consist=Consistent|R=Read Ahead Always|NR=No Read Ahead|WB=WriteBack|
AWB=Always WriteBack|WT=WriteThrough|C=Cached IO|D=Direct IO|sCC=Scheduled
Check Consistency


PDs for VD 1 :
============

----------------------------------------------------------------------------------
EID:Slt DID State DG     Size Intf Med SED PI SeSz Model                  Sp Type
----------------------------------------------------------------------------------
252:2     1 Onln   1 5.457 TB SATA HDD N   N  512B WDC WD6002VRYZ-02FXTB1 U  -
252:3     0 Onln   1 5.457 TB SATA HDD N   N  512B WDC WD6002VRYZ-02FXTB1 U  -
----------------------------------------------------------------------------------

EID-Enclosure Device ID|Slt-Slot No.|DID-Device ID|DG-DriveGroup
DHS-Dedicated Hot Spare|UGood-Unconfigured Good|GHS-Global Hotspare
UBad-Unconfigured Bad|Onln-Online|Offln-Offline|Intf-Interface
Med-Media Type|SED-Self Encryptive Drive|PI-Protection Info
SeSz-Sector Size|Sp-Spun|U-Up|D-Down|T-Transition|F-Foreign
UGUnsp-Unsupported|UGShld-UnConfigured shielded|HSPShld-Hotspare shielded
CFShld-Configured shielded|Cpybck-CopyBack|CBShld-Copyback Shielded


VD1 Properties :
==============
Strip Size = 64 KB
Number of Blocks = 11719933952
VD has Emulated PD = Yes
Span Depth = 1
Number of Drives Per Span = 2
Write Cache(initial setting) = WriteThrough
Disk Cache Policy = Disabled
Encryption = None
Data Protection = None
Active Operations = Background Initialization (5%)
Exposed to OS = Yes
Creation Date = 17-11-2017
Creation Time = 08:33:18 AM
Emulation type = default
Is LD Ready for OS Requests = Yes
SCSI NAA Id = 6003005700cc623021a15b4e2f2e02ec

為 VD 改名 (不會影響 Data)

storcli64 /c0/v2 set name=20190329_R1

Rebuild Status

storcli64 /c0 /eall /sall show rebuild

Consistency Check Commands

storcli64 /c0/v0 show cc

-----------------------------------------------------------
VD Operation Progress% Status          Estimated Time Left
-----------------------------------------------------------
 1 CC        -         Not in progress -
-----------------------------------------------------------

-------------------------------------------------------
VD Operation Progress% Status      Estimated Time Left
-------------------------------------------------------
 1 CC                0 In progress 10 Hours 23 Minutes
-------------------------------------------------------

storcli64 /cx/vx <start|stop> cc

storcli64 /cx/vx <pause|resume> cc

storcli64 /c0 show ccrate

----------------
Ctrl_Prop Value
----------------
CC Rate   30%
----------------

Scheduled Check Consistency(sCC) # 不是每 Model 都有此功能

i.e.

storcli64 /c0 set CC=on starttime=2019/04/06 10

seq: Sequential mode.

conc: Concurrent mode.

off: Turns off the consistency check

delay( 24 X 7 X 4 = 672 )

Delay a scheduled consistency check. The value is in hours.

A value of 0 makes the CC runs continuously with no delay

starttime

A valid date and hour in format (yyyy/mm/dd hh)

Initialization

storcli64 /c0/v1 show bgi

VD Operation Status :
===================

-------------------------------------------------------
VD Operation Progress% Status      Estimated Time Left
-------------------------------------------------------
 1 BGI               0 In progress 7 Hours 27 Minutes
-------------------------------------------------------

電池

# 查看電池資料

storcli64 /c0/bbu show

----------------------------------------------
Ctrl Status Property ErrMsg             ErrCd
----------------------------------------------
   0 Failed -        Battery is absent!    34
----------------------------------------------

# awb: always write back

storcli64 /cx/vx set wrcache=<wt|wb|awb>

# turn off Disk Cache (PD cache)

storcli /c0/v0 set pdcache=off

說明

pdcache = physical drives cache

Cache Setting

cacheflushint

# show cache flush interval in seconds

storcli64 /c0 show cacheflushint

---------------------------
Ctrl_Prop            Value
---------------------------
Cache Flush Interval 4 sec
---------------------------

# Sets

Flushes the controller cache

storcli /c0 flushcache

Set write Cache

#write back, write through

storcli64 /c0/v0 set wrcache=wt

Set Read Cache

# read ahead, no read ahead, or adaptive read ahead.

storcli64 /c0/v0 set rdcache=nora

Set Disk Cache

storcli /cx/vx set pdcache=<on|off|default>

Set Drive State Commands (online/offline)

storcli /cx[/ex]/sx set jbod

storcli /cx[/ex]/sx set good [force]

storcli /cx[/ex]/sx set offline

storcli /cx[/ex]/sx set online

spindown & spinup

prepare the physical drive for removal

storcli /cx[/ex]/sx spindown

storcli /cx[/ex]/sx spinup

Remove Foreign Config

storcli64 /c0/fall show

---------------------------------------
DG EID:Slot Type  State     Size NoVDs
---------------------------------------
 0 -        RAID1 Frgn  5.457 TB     1
---------------------------------------

* Individual foriegn configuration display not supported. Use /fall

# deletes ALL foreign configuration

storcli64 /c0/fall delete

Import Foreign Configuration

# show all config

storcli64 /c0 show

# preview

storcli64 /c0/fall import preview

# import

storcli64 /c0/fall import

Auto import feature

# Imports foreign configuration automatically, at boot.

storcli64 /c0 show foreignautoimport

實習1: 更換 RAID 內壞了的碟

1. Set the failed drive as Offline

./storcli64 /cx/ey/sz set offline

x = Controller number
y = Encolsure Device ID (Seen above as EID)
z= Controller defined slot number

2. Set the failed drive as Missing

./storcli64 /cx/ey/sz set missing

3. Spindown the failed drive

./storcli64 /cx/eall/sy spindown

4. Remove the failed drive and replace it with a new drive

* HDD 要 same model

5. The rebuild should start automatically (If configured) and can be monitored

./storcli64 /cx/eall/sall show rebuild

If auto rebuild isn’t turned on

# dg, array, row correspond to the Topology table from "./storcli64 /cx show all"

./storcli64 /cx/ey/sz insert dg=0 array=0 row=0

Then start the rebuild manually (Topology table will have state ‘Rbld’ for that drive)

./storcli64 /cx/ey/sz start rebuild

Rebuild rate default is set to 30%

6. 把 rebuild rate 設定高一點

# N = 0-100

./storcli64 /cx set rebuildrate=N

實習2 - Failed Disk

storcli64 /c0/e252/sall show

-----------------------------------------------------------------------------------
EID:Slt DID State  DG     Size Intf Med SED PI SeSz Model                  Sp Type
-----------------------------------------------------------------------------------
252:0     3 Failed  1 3.637 TB SATA HDD N   N  512B WDC WD4002FYYZ-01B7CB0 U  -

storcli64 /c0/e252/s0 set offline

storcli64 /c0/e252/s0 start rebuild

storcli64 /c0/e252/s0 show rebuild

------------------------------------------------------
Drive-ID    Progress% Status      Estimated Time Left
------------------------------------------------------
/c0/e252/s0         0 In progress -
------------------------------------------------------

實習3 - UBad Disk

If an improperly removed device is reconnected to the RAID controller it will be recognized as UBAD (Unconfigured Bad)

當將它設定成 UGood 時, 就會有 foreign config

# 查看

storcli64 /c0 /eall /sall show

----------------------------------------------------------------------------------
EID:Slt DID State DG     Size Intf Med SED PI SeSz Model                  Sp Type
----------------------------------------------------------------------------------
252:2     1 Onln   1 5.457 TB SATA HDD N   N  512B WDC WD6002VRYZ-02FXTB1 U  -
252:3     0 UBad   - 5.457 TB SATA HDD N   N  512B WDC WD6002VRYZ-02FXTB1 U  -

# Set 隻 dsk good (UBad -> UGood)

storcli64 /c0 /e252 /s3 set good

# Now the device is foreign in the pool

storcli64 /c0 /eall /sall show

----------------------------------------------------------------------------------
EID:Slt DID State DG     Size Intf Med SED PI SeSz Model                  Sp Type
----------------------------------------------------------------------------------
.....
252:2     1 Onln   1 5.457 TB SATA HDD N   N  512B WDC WD6002VRYZ-02FXTB1 U  -
252:3     0 UGood  F 5.457 TB SATA HDD N   N  512B WDC WD6002VRYZ-02FXTB1 U  -

storcli64 /c0 /fall show

---------------------------------------
DG EID:Slot Type  State     Size NoVDs
---------------------------------------
 0 -        RAID1 Frgn  5.457 TB     1

# To clear / erase the status of a disk (UGood 是不能 Erase 的)

storcli64 /c0/e252/s3 start erase simple

simple: erases data once
normal: erases data three times
thorough: erases data nine times

------------------------------------------------------------
Drive       Status  ErrCd ErrMsg
------------------------------------------------------------
/c0/e252/s3 Failure   255 Start Drive erase is not allowed.

實習4 - UGood Disk

UGood 代表不在任何 RAID 的 Disk, 它有機會是 foreign Disk

#1 - 查找 UGood Disk

storcli64 /c0 /eall /sall show

----------------------------------------------------------------------------------
EID:Slt DID State DG     Size Intf Med SED PI SeSz Model                  Sp Type
----------------------------------------------------------------------------------
252:3     0 UGood  - 5.457 TB SATA HDD N   N  512B WDC WD6002VRYZ-02FXTB1 U  -

#2 - 將 UGood Disk 加到 RAID

storcli64 /c0/dall show

---------------------------------------------------------------------------
DG Arr Row EID:Slot DID Type  State BT     Size PDC  PI SED DS3  FSpace TR
---------------------------------------------------------------------------
 1 -   -   -        -   RAID1 Dgrd  N  5.457 TB dsbl N  N   dflt N      N
 1 0   -   -        -   RAID1 Dgrd  N  5.457 TB dsbl N  N   dflt N      N
 1 0   0   252:2    1   DRIVE Onln  N  5.457 TB dsbl N  N   dflt -      N
 1 0   1   -        -   DRIVE Msng  -  5.457 TB -    -  -   -    -      N

# to add the disc into the degraded array

storcli64 /c0/e252/s3 insert dg=1 array=0 row=1

# 3 - 查看 Disk 是否被加到 RAID 內

storcli64 /c0 /eall /sall show

----------------------------------------------------------------------------------
EID:Slt DID State DG     Size Intf Med SED PI SeSz Model                  Sp Type
----------------------------------------------------------------------------------
252:2     1 Onln   1 5.457 TB SATA HDD N   N  512B WDC WD6002VRYZ-02FXTB1 U  -
252:3     0 Offln  1 5.457 TB SATA HDD N   N  512B WDC WD6002VRYZ-02FXTB1 U  -

# 4 - 將 Offline 的 Disk rebuild (不用 online 佢)

storcli64 /c0/e252/s3 start rebuild

# 5 - checking

storcli64 /c0/e252/s3 show rebuild

------------------------------------------------------
Drive-ID    Progress% Status      Estimated Time Left
------------------------------------------------------
/c0/e252/s3         0 In progress -
------------------------------------------------------

storcli64 /c0 show

----------------------------------------------------------------------------------
EID:Slt DID State DG     Size Intf Med SED PI SeSz Model                  Sp Type
----------------------------------------------------------------------------------
252:2     1 Onln   1 5.457 TB SATA HDD N   N  512B WDC WD6002VRYZ-02FXTB1 U  -
252:3     0 Rbld   1 5.457 TB SATA HDD N   N  512B WDC WD6002VRYZ-02FXTB1 U  -

chkraid.sh

#!/bin/bash

HOST="MyServer"
ADMIN="X@y"

/usr/sbin/storcli64 /c0/vall show | mail -s "$HOST weekly RAID status" $ADMIN

瀏覽次數： 6836

夢想家

storcli

介紹

目錄

Check Version

設定 RAID 卡時間

Help

Basic Info

PD 與 DG

Info. about controller

Locate a physical disk by blinking it's LED

Phyical Disk Info.

HDD SMART

Firmware Update

Activation of Features

JBOD

Creating a Virtual Drive

查看 RAID 內有什麼 Disk

Virtual Drives

Rebuild Status

Consistency Check Commands

Initialization

電池

Cache Setting

Set Drive State Commands (online/offline)

spindown & spinup

Remove Foreign Config

Import Foreign Configuration

實習1: 更換 RAID 內壞了的碟

實習2 - Failed Disk

實習3 - UBad Disk

實習4 - UGood Disk