{"id":68,"date":"2008-08-02T21:33:17","date_gmt":"2008-08-03T05:33:17","guid":{"rendered":"http:\/\/greg.porter.name\/wordpress\/?p=68"},"modified":"2009-12-02T08:25:56","modified_gmt":"2009-12-02T16:25:56","slug":"sun-3510-and-3511-disk-arrays","status":"publish","type":"post","link":"https:\/\/greg.porter.name\/wordpress\/?p=68","title":{"rendered":"Sun 3510 and 3511 Disk Arrays"},"content":{"rendered":"

We have 2 Sun StorEdge 35XX disk arrays. Two Solaris hosts are directly attached to a Sun StorEdge 3510 Fibre Channel Array<\/span><\/a>. One linux host has a similar Sun StorEdge 3511 Fibre Channel Array<\/span><\/a>. The 3510 uses 10K RPM FC disks (fast), the 3511 uses slower (but larger) 7.2K RPM SATA disks. Other than that, the arrays are very similar. I didn’t think much of them (why don’t we have a *REAL* SAN?) but they are decent little arrays…<\/p>\n

Supposedly, you can talk to these arrays with a serial cable, but I have tried on repeated occasions with various cables and terminals. I have yet to date to actually talk to one of these arrays with serial. Sun must use some funky RS-232 handshaking signals that regular old serial cables don’t present correctly.<\/p>\n

These arrays have an ethernet management port on them… But of course, these were never configured.<\/p>\n

Sun makes array management software that you can optionally load on a directly attached host. The software actually talks to the array across the FC connection (they call this in-band). Of course, none of our hosts had this software loaded.<\/p>\n

I always make sure that I have the latest firmware, and the latest management software. The firmware on disk arrays is especially important, as old firmware has bugs that cause I\/O to hiccup, controllers to hang, etc.<\/p>\n

Of course, our arrays have the firmware they shipped with (never been updated). It is so old now that a direct upgrade is not possible. You have to do an incremental upgrade. Follow these directions<\/a>.<\/p>\n

There are 6 different array components that have firmware on them, you have to do them all, and in the right order. If you fail to follow the directions, you could easily kill the array. Even if you do follow the directions, you may lose your disk configuration, FC configuration, etc. So make sure you either have it all backed up, or you are willing to reset the array to factory defaults.<\/p>\n

Assuming you have the latest firmware and management software, then you can talk to the array from an attached host with the sccli command line tool. It is a little tricky, but it works.<\/p>\n

Basic Disk Operations <\/a><\/h3>\n

Use this book to see sccli command syntax<\/a>. Basically to use the array to add disks to your hosts you have to:<\/p>\n

    \n
  1. See if there are physical disks available in the array<\/li>\n
  2. Create a logical drive<\/li>\n
  3. Map the logical drive to a host FC channel<\/li>\n
  4. Reboot the host, and use host OS commands to manipulate the new disk<\/li>\n<\/ol>\n

    See if there are physical disks available in the array <\/a><\/h4>\n

    So for this example, we are starting with an unconfigured array, no partitions, no maps, etc. You can see this stuff with sccli commands, like so:<\/p>\n

    [root@jedi ~]# sccli\r\nsccli: selected device \/dev\/sg1 [SUN StorEdge 3511 SN#008AAD]\r\nsccli> show disks\r\nCh     Id      Size   Speed  LD     Status     IDs                      Rev  \r\n----------------------------------------------------------------------------\r\n2(3)   0  372.61GB   200MB  NONE   FRMT       HITACHI HDS7240SBSUN400G A47A \r\nS\/N KRFS06RAGYPPDC  \r\nWWNN 22E4000A33007849\r\n2(3)   1  372.61GB   200MB  NONE   FRMT       HITACHI HDS7240SBSUN400G A47A \r\nS\/N KRFS06RAGWHDAC  \r\nWWNN 22E4000A3300784A\r\n2(3)   2  372.61GB   200MB  NONE   FRMT       HITACHI HDS7240SBSUN400G A47A \r\nS\/N KRFS06RAGYWS4C  \r\nWWNN 22E4000A3300784B\r\n2(3)   3  372.61GB   200MB  NONE   FRMT       HITACHI HDS7240SBSUN400G A47A \r\nS\/N KRFS06RAGYWH7C  \r\nWWNN 22E4000A3300784C\r\n2(3)   4  372.61GB   200MB  NONE   FRMT       HITACHI HDS7240SBSUN400G A47A \r\nS\/N KRFS06RAGY7HYC  \r\nWWNN 22E4000A3300784D\r\n2(3)   5  372.61GB   200MB  NONE   USED       HITACHI HDS7240SBSUN400G A47A \r\nS\/N KRFS06RAGYM8YC  \r\nWWNN 22E4000A3300784E\r\n2(3)   6  372.61GB   200MB  NONE   FRMT       HITACHI HDS7240SBSUN400G A47A \r\nS\/N KRFS06RAGYPY6C  \r\nWWNN 22E4000A3300784F\r\n2(3)   7  372.61GB   200MB  NONE   FRMT       HITACHI HDS7240SBSUN400G A47A \r\nS\/N KRFS06RAGYKE3C  \r\nWWNN 22E4000A33007850\r\n2(3)   8  372.61GB   200MB  NONE   FRMT       HITACHI HDS7240SBSUN400G A47A \r\nS\/N KRFS06RAGYPZ4C  \r\nWWNN 22E4000A33007851\r\n2(3)   9  372.61GB   200MB  NONE   FRMT       HITACHI HDS7240SBSUN400G A47A \r\nS\/N KRFS06RAGXT6AC  \r\nWWNN 22E4000A33007852\r\n2(3)  10  372.61GB   200MB  NONE   FRMT       HITACHI HDS7240SBSUN400G A47A \r\nS\/N KRFS06RAGYPS0C  \r\nWWNN 22E4000A33007853\r\n2(3)  11  372.61GB   200MB  NONE   FRMT       HITACHI HDS7240SBSUN400G A47A \r\nS\/N KRFS06RAGYPM2C  \r\nWWNN 22E4000A33007854\r\nsccli> show logical-drives\r\nsccli> show maps\r\nsccli><\/pre>\n
    Create a logical drive <\/a><\/h4>\n
    The syntax is fairly straight forward. In this example, my array has 12 disks in it. I’m going to set one as the global hot spare (this step is optional), and then use all of the rest for one big logical drive of 11 physical drives in RAID-5. Note that you specify physical disks in sccli in <channel>.<disk number>, so for example I’ll use disk 11 for the spare below, so I’ll specify 2.11.<\/p>\n
    sccli> configure global-spare 2.11\r\nsccli> show disk\r\nCh     Id      Size   Speed  LD     Status     IDs                      Rev  \r\n----------------------------------------------------------------------------\r\n<snip>\r\n2(3)  11  372.61GB   200MB  GLOBAL STAND-BY   HITACHI HDS7240SBSUN400G A47A \r\nS\/N KRFS06RAGYPM2C  \r\nWWNN 22E4000A33007854\r\nsccli> create logical-drive raid5 2.0-10\r\nsccli: created logical drive 651F6A8B\r\nsccli> show logical-drive\r\nLD    LD-ID        Size  Assigned  Type   Disks Spare  Failed Status     \r\n------------------------------------------------------------------------\r\nld0   651F6A8B   3.64TB  Primary   RAID5  11    1      0      Good  I \r\nWrite-Policy Default          StripeSize 128KB\r\nsccli> show disks\r\nCh     Id      Size   Speed  LD     Status     IDs                      Rev  \r\n----------------------------------------------------------------------------\r\n2(3)   0  372.61GB   200MB  ld0    ONLINE     HITACHI HDS7240SBSUN400G A47A \r\nS\/N KRFS06RAGYPPDC  \r\nWWNN 22E4000A33007849\r\n2(3)   1  372.61GB   200MB  ld0    ONLINE     HITACHI HDS7240SBSUN400G A47A \r\nS\/N KRFS06RAGWHDAC  \r\nWWNN 22E4000A3300784A\r\n2(3)   2  372.61GB   200MB  ld0    ONLINE     HITACHI HDS7240SBSUN400G A47A \r\nS\/N KRFS06RAGYWS4C  \r\nWWNN 22E4000A3300784B\r\n2(3)   3  372.61GB   200MB  ld0    ONLINE     HITACHI HDS7240SBSUN400G A47A \r\nS\/N KRFS06RAGYWH7C  \r\nWWNN 22E4000A3300784C\r\n2(3)   4  372.61GB   200MB  ld0    ONLINE     HITACHI HDS7240SBSUN400G A47A \r\nS\/N KRFS06RAGY7HYC  \r\nWWNN 22E4000A3300784D\r\n2(3)   5  372.61GB   200MB  ld0    ONLINE     HITACHI HDS7240SBSUN400G A47A \r\nS\/N KRFS06RAGYM8YC  \r\nWWNN 22E4000A3300784E\r\n2(3)   6  372.61GB   200MB  ld0    ONLINE     HITACHI HDS7240SBSUN400G A47A \r\nS\/N KRFS06RAGYPY6C  \r\nWWNN 22E4000A3300784F\r\n2(3)   7  372.61GB   200MB  ld0    ONLINE     HITACHI HDS7240SBSUN400G A47A \r\nS\/N KRFS06RAGYKE3C  \r\nWWNN 22E4000A33007850\r\n2(3)   8  372.61GB   200MB  ld0    ONLINE     HITACHI HDS7240SBSUN400G A47A \r\nS\/N KRFS06RAGYPZ4C  \r\nWWNN 22E4000A33007851\r\n2(3)   9  372.61GB   200MB  ld0    ONLINE     HITACHI HDS7240SBSUN400G A47A \r\nS\/N KRFS06RAGXT6AC  \r\nWWNN 22E4000A33007852\r\n2(3)  10  372.61GB   200MB  ld0    ONLINE     HITACHI HDS7240SBSUN400G A47A \r\nS\/N KRFS06RAGYPS0C  \r\nWWNN 22E4000A33007853\r\n2(3)  11  372.61GB   200MB  GLOBAL STAND-BY   HITACHI HDS7240SBSUN400G A47A \r\nS\/N KRFS06RAGYPM2C  \r\nWWNN 22E4000A33007854<\/pre>\n
    Map the logical drive to a host FC channel <\/a><\/h4>\n
    OK, so now the array knows it has a logical drive. However the controller doesn’t know who to give it to (where to present it). So a map says basically to present a given logical drive to a particular host.<\/p>\n
    sccli> show channels\r\nCh  Type    Media   Speed   Width  PID \/ SID\r\n--------------------------------------------\r\n0  Host    FC(L)   2G      Serial  40 \/ N\/A\r\n1  Host    FC(L)   2G      Serial  N\/A \/ 42\r\n2  DRV+RCC FC(L)   2G      Serial  14 \/ 15\r\n3  DRV+RCC FC(L)   2G      Serial  14 \/ 15\r\n4  Host    FC(L)   2G      Serial  44 \/ N\/A\r\n5  Host    FC(L)   2G      Serial  N\/A \/ 46\r\n6  Host    LAN     N\/A     Serial  N\/A \/ N\/A<\/pre>\n
    The “host channels” basically correspond to the physical FC ports on the back of the controller. In my example, my one and only host is plugged into port 0 on the back of the controller. In sccli-speak, that is 0.40 – <channel>.<PID>. I will present my new logical drive as LUN 0 to 0.40.<\/p>\n
    sccli> show partition\r\nLD\/LV    ID-Partition        Size\r\n-------------------------------------\r\nld0-00   651F6A8B-00       3.64TB\r\nsccli> map 651F6A8B-00 0.40.0\r\nsccli: mapping ld 651F6A8B-00 to 0.40.0\r\nsccli> show map\r\nCh Tgt LUN   ld\/lv   ID-Partition  Assigned  Filter Map\r\n---------------------------------------------------------------------\r\n0  40   0   ld0     651F6A8B-00   Primary<\/pre>\n
    Now reboot the host to get it to see the newly presented drive.<\/p>\n
    Use host OS commands to manipulate the new disk <\/a><\/h4>\n
    These commands are specific to your OS.  Google around for a howto.<\/p>\n
    [root@jedi ~]# fdisk -l\r\nDisk \/dev\/sda: 73.4 GB, 73407868928 bytes\r\n255 heads, 63 sectors\/track, 8924 cylinders\r\nUnits = cylinders of 16065 * 512 = 8225280 bytes\r\nDevice Boot      Start         End      Blocks   Id  System\r\n\/dev\/sda1   *           1          13      104391   83  Linux\r\n\/dev\/sda2              14        8924    71577607+  8e  Linux LVM\r\nDisk \/dev\/sdb: 3998.1 GB, 3998146887680 bytes\r\n255 heads, 63 sectors\/track, 486080 cylinders\r\nUnits = cylinders of 16065 * 512 = 8225280 bytes\r\nDisk \/dev\/sdb doesn't contain a valid partition table\r\n[root@jedi ~]# parted \/dev\/sdb\r\nGNU Parted 1.8.1\r\nUsing \/dev\/sdb\r\nWelcome to GNU Parted! Type 'help' to view a list of commands.\r\n(parted)<\/pre>\n","protected":false},"excerpt":{"rendered":"
    We have 2 Sun StorEdge 35XX disk arrays. Two Solaris hosts are directly attached to a Sun StorEdge 3510 Fibre Channel Array. One linux host has a similar Sun StorEdge 3511 Fibre Channel Array. The 3510 uses 10K RPM FC… Continue Reading →<\/span><\/a><\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-68","post","type-post","status-publish","format-standard","hentry","category-content"],"_links":{"self":[{"href":"https:\/\/greg.porter.name\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/68","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/greg.porter.name\/wordpress\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/greg.porter.name\/wordpress\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/greg.porter.name\/wordpress\/index.php?rest_route=\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/greg.porter.name\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=68"}],"version-history":[{"count":3,"href":"https:\/\/greg.porter.name\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/68\/revisions"}],"predecessor-version":[{"id":246,"href":"https:\/\/greg.porter.name\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/68\/revisions\/246"}],"wp:attachment":[{"href":"https:\/\/greg.porter.name\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=68"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/greg.porter.name\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=68"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/greg.porter.name\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=68"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}