From Greg Porter's Wiki

Revision as of 07:48, 17 February 2010 by Greg (Talk | contribs)
(diff) ← Older revision | Current revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Need shared storage for virtualization, like XenServer or ESX? Use Openfiler to turn an old server into an iSCSI filer for a storage repository!


Openfiler, the Missing Manual

I've been doing a lot of work with Openfiler, a well regarded Open Source Storage Appliance Software package. With Openfiler, you can take "any old" server you have laying around and make a iSCSI, CIFS or NFS file server easily and quickly.

I don't intend for this to duplicate the contents of the Openfiler manual. The Openfiler people made the basic install and configuration dead easy. Really. If you've ever loaded an operating system before (especially Linux, like Red Hat) it's dead easy, a no-brainer. If you aren't comfortable with this, do what I and many others have done. Try it over and over again until you get it. If you want a screenshot by screenshot manual, buy one from Openfiler. The Openfiler folks make their living selling manuals and support. I bought one, because I think they are doing a great job. You should, too. Their manual covers basic OpenFiler operations, but doesn't cover some of the cooler advanced features, like replication.

As I run across configuration issues not covered in their manual, and figure out some of the advanced features, I'll post notes here. Stay tuned.

I'm also going to do some benchmarks with OpenFiler running head to head against dedicated iSCSI arrays.

Openfiler does all sorts of filer duties, like iSCSI, NFS, CFS, FTP, etc. I'm using it as an iSCSI backend for VMware ESX and Citrix XenServer. This document will focus on using the iSCSI features first, and then maybe I'll add other sections later.

What's a filer? Why should I care?

"Filer" means different things to different people, but in general a "filer" is a server dedicated to providing storage. This term came into common use with the products of the storage vendor NetApp. "Filer" originally meant a "networked attached storage" or NAS unit. Now it's common for "filer" to mean a storage device capable of offering up storage by either file level (NAS with NFS, CIFS, AFP) or block level (SAN with iSCSI or Fibre Channel).

Properly done, using shared storage can be cheaper, faster and better than using local direct attached storage. If you have shared storage available, you no longer need to buy servers with disk drives (you can boot from SAN) or you can get one or two very small drives instead of a lot of expensive drives.

Often filers offer advanced snapshot capabilities, which make it easy to backup and restore files. They also often offer replication, which allows a filer at one location to send blocks or snapshots to another filer at a remote location for redundancy and disaster tolerance.

One important use case for filers and shared storage is with virtualization. Most virtualization products like VMware ESX or Citrix XenServer require shared storage in order to use virtualization features like high availability or workload balancing. If you don't have shared storage, you can't use these features that come with your virtualization product.

Why Openfiler?

Dedicated filers are expensive. Prices vary widely depending on configuration and vendor, but an entry level filer costs in the neighborhood of $20,000 and up.

Under the hood, most filers are just a server, running a stripped down operating system, with a lot of disks. Often filer operations are not particularly CPU intensive, so the CPUs used are not necessarily the fastest available.

You could certainly "roll your own" filer, by stripping down an operating system by hand. Linux is the obvious choice for this. However, the folks at Openfiler have done this for you already, and just to sweeten the pot, they have included a web based administration GUI. So Openfiler is easy to load, configure and use, and it runs well.

So if you have a suitable host server, and you're willing to spend a few hours fiddling with OpenFiler, you can put together a decent filer for little or no cost, perfect as iSCSI storage for XenServer or ESX.

Installing Openfiler

Basically any old server will do. If it'll run Linux, it'll run Openfiler. 32 bit or 64 doesn't seem to matter. The main thing is that is that the server have disks, the more the better.

Find a suitable Openfiler host

Hardware Requirements

The manual (you know, the one you ought to buy) says:

Recommended Specifications:

  • 64-bit 1.6 GHz or higher performance processor
  • 1GB or higher RAM
  • 1GB disk space for memory swap area
  • 2GB disk space for OpenFiler OS installation
  • 1Gb Etherner network interface
  • Separate storage volumes/disks for data export
  • Hardware RAID controller

You can see from these specs that Openfiler itself doesn't need much. 3GB of disk space, that's it. My grandma's PC meets these specs (Well, most of them, she doesn't have hardware RAID).

What to look for

Basically you should look at what you have already, and find a server that has a lot of disks (if you have disks, why buy more?). Ideally, the disks should be "enterprise class" (SCSI or SAS), you should have a number of them (more disk spindles means more IOs), and hopefully they are connected via a hardware RAID controller. So an ideal candidate for an Openfiler host would be a "yesterday's model" server, a couple of years old perhaps, that was bought originally with a hardware RAID controller and a number of larger SCSI disks. Examples of servers like this would be a HP Proliant DL380 G3 or Dell 2950.

If you don't have a suitable server laying around, then consider buying a used one. Maybe a couple off of eBay (you can cannibalize one for parts to keep the other one going). You should be able to spend very little money. Servers like this (a couple of years old, large form factor) go for pennies on the dollar. You could buy a couple of suitable ones for a couple of hundred dollars. You'll probably pay more to ship them than the purchase price. I sell ones like this every now and then at Servers4Linux.

If you're not sure if your server has a dedicated RAID controller, reboot it and watch the prompts during boot. A hardware RAID controller will "say hello" during the boot process.

For sake of example, I'll use a Dell PowerEdge 6950. Here's what one looks like.
Dell 6950
Reboot it, watch the BIOS.
6950 Initial BIOS
Here's the RAID controller identifying.
6950 RAID Identification
Usually there's a "hot key" to go into the RAID setup. Hit it. On this example server, it's Control+R. In a few seconds, you'll be in the RAID setup screen.
6950 RAID Setup

My example server has five SCSI 285GB disks. These are presented to the operating system as "one" disk of about 1TB.

Virtual Disk Config

Get media

Easy. Decide if you want 32 bit or 64 bit (try 64 if you can run it) and download the corresponding .iso disk image. Burn it to disk.


Even if you didn't buy the manual, the Openfiler people have posted excerpts of it for both the text based installation and the graphical installation. By default you get the graphical installation. There doesn't appear to be any difference between the two. Might as well go graphical if you can, and see the pretty graphics.

Since they have good install guides, complete with screenshots, there's no point in duplicating the effort. See their install guides.

The Openfiler Installer

Install "gotcha"

This is a typical Linux based installation. All in all, it's easy and like any other Linux you may have loaded before. There is one big gotcha, though, that you need to pay attention to:


If you fail to do this and use automatic partitioning (the default), you'll see that you will wind up with an Openfiler system where you are UNABLE TO USE THE GUI TO MAKE VOLUME GROUPS. It'll work, but you'll be unable to do a VERY IMPORTANT task with the GUI. Follow their install guide, and manually partition the disks as they specify.

Partition the boot disk

Use manual partitioning. You need to make three partitions: /boot, / (root), and swap. Make them all on the first disk. If you're following my simplistic example, you only have one disk (all the real disks are made to look like one by the hardware RAID controller).

Read the install guides or the manual. To summarize, make them like so:

Partition for the /boot file system:

  • Mount Point: /boot
  • Filesystem Type: ext3
  • Allowable Drives: Select one disk only. This should be the first IDE (hda) or first SCSI disk (sda)
  • Size(MB): 100 (This is the size in Megabytes, allocate 100MB by entering "100")
  • Additional Size Options: Select Fixed Size radio button from the options.
  • Force to be a primary partition: Checked (select this check box to force the partition to be created as a primary partition)

Partition for the / (root) file system:

  • Mount Point: / (root partition)
  • Filesystem Type: ext3
  • Allowable Drives: Select one disk only. This should be the first IDE (hda) or first SCSI disk (sda)
  • Size(MB): 1024 or 2048 (This is the size in Megabytes, I used 2048, allocate 2048MB by entering "2048")
  • Additional Size Options: Select Fixed Size radio button from the options.
  • Force to be a primary partition: Checked (select this check box to force the partition to be created as a primary partition)

Partition for swap (no file system or mount point):

  • Mount Point: none (will gray out when type of swap is selected, swap partitions don't have a mount point)
  • Filesystem Type: swap
  • Allowable Drives: Select one disk only. This should be the first IDE (hda) or first SCSI disk (sda)
  • Size(MB): 1024 or 2048 (This is the size in Megabytes, I used 2048, allocate 2048MB by entering "2048")
  • Additional Size Options: Select Fixed Size radio button from the options.
  • Force to be a primary partition: Checked (select this check box to force the partition to be created as a primary partition)

Other than that, no big surprises.

You'll be prompted for the user root's password. Remember what you entered, you'll need it soon.

User names and passwords

An account for the user root will be made with the password you specify.

Web based GUI operations should be done as the user named openfiler, which is created automatically. The default password for the user named openfiler is password.

Logging in after install

The console will come up in a text mode. You can log in on the console as root. Ssh is enabled by default. You can ssh to your new filer, and log in as root. (Ssh and console logins as the user openfiler don't work).

For most common operations, you should probably use the web GUI. This management interface is at Use the user openfiler, and the password password. (Of course, use *YOUR* filer's name or IP address).

Change the user openfiler's password

Probably the easiest way to change the password is to log in the GUI as the user openfiler, password is password. Go to the accounts tab. On the far right, click on "Admin Password" and use the dialog to change the user openfiler's password to something better.

Alternately you could log in via ssh or on the console and use the Linux passwd command like so:

[glporter@name ~]$ ssh's password: 
Last login: Thu Dec  3 13:15:50 2009 from
[root@my_filer ~]# passwd openfiler
Changing password for user openfiler.
New UNIX password: 
Retype new UNIX password: 
passwd: all authentication tokens updated successfully.

Patch the new install

Once the install is complete, then reboot. Once the Openfiler server is back up, patch it. Like most things, the easiest way to do this is from the web GUI. Log in as the user openfiler, go to the System tab, click on System Update on the far right.

Alternately you can use the command line (logged in as root) like so:

[root@my_filer ~]# conary updateall

It's possible that some patches require a reboot to take effect. Kernel patches are like this, and there's at least one for Openfiler 2.3 that needs a reboot to take effect as of this writing (Nov 2009). It's possible if it's been a while since you patched that there's patches for the patches. So the safest thing to do is patch, reboot, attempt to patch again. Patch, reboot, patch, reboot until no more patches are available. Patch until you see something like:

System Update

or (command line):

[root@my_filer ~]# conary updateall
no new troves were found 

Make required tweaks to address the Openfiler 2.3 reboot issue

A default install of Openfiler 2.3 has an issue when rebooted. On a default installation of Openfiler 2.3, clients cannot use iSCSI target disks after a reboot. In other words, it’ll work fine until you reboot it. This has been discussed at some length in Openfiler forums, and has an open bug filed against it.

Also note that it has nothing to do with using XenServer. The iscsi-target service will not properly restart after a reboot, period, whether you are using XenServer as a iSCSI client against the Openfiler target or not.

Luckily, it’s trivial to fix with just a couple little tweaks.

What the 2.3 reboot issue looks like and how to replicate it

This is “just the way it is” on 2.3. I replicated the issue for testing by:

  • Do a default install of Openfiler 2.3 from media (I used the 32 bit, I’ve seen it on 64, too).
  • Patch, reboot, patch… Do this until no further patches are required. (I always patch to the latest, but unpatched or patched, you’ll see the reboot issue in 2.3)
  • Set the iSCSI target server to enabled in the Openfiler GUI.
  • Make one or more iSCSI targets in the GUI. The name or size doesn’t matter.
  • Connect to the iSCSI target with a client. It will work. The log file at /var/log/messages will have no errors, something like:
Jan 14 10:39:26 aqua kernel: [  201.153905] iSCSI Enterprise Target Software - version 1.4.18
Jan 14 10:39:26 aqua kernel: [  201.154716] iscsi_trgt: Registered io type fileio
Jan 14 10:39:26 aqua kernel: [  201.154750] iscsi_trgt: Registered io type blockio
Jan 14 10:39:26 aqua kernel: [  201.154756] iscsi_trgt: Registered io type nullio
Jan 14 10:39:26 aqua iscsi-target: ietd startup succeeded
  • Reboot the Openfiler server.
  • Attempt to connect to the target with a client. It won’t work.
  • If you look in the log file, you’ll see entries like:
Jan 14 11:19:39 aqua ietd: Can't create a logical unit 16 1 0 Path=/dev/test_vg/test_lv,Type=blockio,ScsiSN=P5e333-XQ1k-phoS,ScsiId=P5e333-XQ1k-phoS,IOMode=wt
Jan 14 11:19:39 aqua kernel: [18.084347] iscsi_trgt: blockio_open_path(167) Can't open device /dev/test_vg/test_lv, error -16
Jan 14 11:19:39 aqua kernel: [18.084626] iscsi_trgt: blockio_attach(355) Error attaching Lun 0 to Target

Quick one time fix (fixes it until the next reboot)

If you are stuck here, then you can do a one time fix to clear the error, and get the lun(s) to attach to the target (which will allow clients to use them again). This is the fix mentioned in the forums discussion by the user jirikanicky (Look at his post that starts “I believe that I found out the root cause”, down about 2/3 of the way on the first page). Note that this procedure *IS NOT* a permanent fix, the behavior will manifest again after the next reboot.

  • Log in to the Openfiler server as root.
  • Use the pvscan command to display the exact name of the volume group(s) affected. It’ll look something like:
PV /dev/dm-0   VG VG_XenStorage-ef48414d-76e7-cb74-a1cb-43bddd590643   lvm2 [2.86 GB / 2.86 GB free]
  • Deactivate this volume group with the vgchange -an <VG NAME> command. (Put your volume group name here).
[root@aqua log]# vgchange -an VG_XenStorage-ef48414d-76e7-cb74-a1cb-43bddd590643
0 logical volume(s) in volume group "VG_XenStorage-ef48414d-76e7-cb74-a1cb-43bddd590643" now active
  • Restart the iscsi-target service with the command service iscsi-target restart. (You could also disable/enable the iSCSI Target Server in the GUI).
  • Try the client again, you should be able to connect now.
  • The iscsi-target service will stop logging errors in /var/log/messages

Like I said, this works fine until the next reboot. If you reboot, you have to do this again.

Permanent fix

To permanently fix this on OpenFiler 2.3, make three tweaks at the command line, logged in as root.

1. Comment out the following lines (lines 333-337) in /etc/rc.sysinit (add a # to the beginning of each line).

#       if [ -x /sbin/lvm.static ]; then
#               if /sbin/lvm.static vgscan --mknodes --ignorelockingfailure > /dev/null 2>&1 ; then
#                       action $"Setting up Logical Volume Management:" /sbin/lvm.static vgchange -a y --ignorelockingfailure
#               fi
#       fi

2. Modify /etc/lvm/lvm.conf, line 53 and make the line look like so (add some additional filters):

filter = [ "r|/dev/dm-*|", "r|/dev/VG_XenStorage*|", "r|/dev/sr/iscsi|", "a/.*/" ]

3. Turn the ATA over Ethernet service off, and make sure it doesn't start again (most people are *NOT* using this...)

[root@my_filer ~]# service aoe force-stop
                                                           [  OK  ]
[root@my_filer ~]# service aoe status

[root@my_filer ~]# chkconfig aoe off

After making these 3 changes, reboot the Openfiler, and your clients will see their targets again.

Again, these tweaks are straight from the Openfiler forum post on this problem. Thanks to the forum users jirikanicky, phil_websurfer and others who originally posted the fixes.

Using Openfiler for iSCSI

Enable the iSCSI target server

If you want to use Openfiler for a iSCSI target, you have to turn the iSCSI target server on. Go to the Service tab, and click on "Enable" for the iSCSI target server.

Enable the iSCSI target server

Prepare the physical disk

So to recap, for my simplistic example, my Openfiler server has 5 285 GB disks on a hardware RAID controller. In the RAID controller, I told it to bind all 5 disks together into one RAID-5 virtual disk. So as far as Openfiler is concerned, it has one disk of a little more than 1TB.

If you followed the Openfiler installation guide, then you have 3 slices on that disk (/boot, root, and swap) that takes a little less than 5GB. The remainder of the disk is available for use by Openfiler to give to clients (we'll use iSCSI, but you could also use NFS, CIFS, rsync, etc.).

Immediately after install, you can look at your disk(s) and see how they are partitioned. Go to Volumes, Block Devices, and click on View in the Partitions column.

View Partitions

So the first thing you need to add a partition for Openfiler to use to hold the client storage. On the same page, click on the disk name (/dev/sda, say) in the "Edit Disk" column.

Edit Disk

You want to add at least one partition. You'll use this to hold the volume group you make next. Scroll down to "Create a partition". Create a partition of type "Physical volume" (pull down the list and select "Physical volume"). Hit Create.

Create partition

Now the list of partitions should be four items long. The partition you just added will occupy the remaining space on the disk.

New partition

So the point is, you now have a partition on the physical disk in which to build a volume group for client use.

Make a volume group

Now that you have a partition to make a volume group in, make a volume group. Go to volumes. You don't have any volume groups for client use, so the list is empty, and the dialog to add one is displayed. The only thing you get to do here is name the new volume group. I suggest that the name be useful, not vg1 but something that indicates what it is for, and where it is hosted.

Add volume group

Now you have a volume group that will hold to hold logical volumes, the actual "fake" drives or LUNs that the iSCSI target server will offer to clients.

New volume group

Make one or more logical volumes

Now you are ready to add logical volumes to the volume group. These are the actual "disks" that the iSCSI target server will offer to clients, and that clients will use. They are also referred to as LUN's which is an old SCSI term for Logical Unit or Logical Unit Number.

How many do you need? How large should they be?

That's a good question. It depends. Have one giant LUN that everything goes into could cause disk contention, if the client machines has lots of simultaneous processes writing to it. Having lots and lots of tiny LUNs is a management nightmare, and they will be easy to outgrow.

If making logical volumes for virtualization use, you will be storing virtual machines in these. Say the "average" vm is about 20GB in size (including snapshots, etc.). A reasonable number of vm's to store together is ten. So LUNs or logical volumes about 200GB (10 20GB vms) sized are reasonable.

So for my example, with 1TB available, I could make 5 200GB lvols. I will only make one for now. If I need another later I'll have room. I could also use the remaining free room to make snaphots and do other fancy tricks with logical volumes in Openfiler. Go to Volumes, Add volume.

Add volume

Name the volume, give a description, pick a size, choose iSCSI for "File System Type". (Note I am not aware of an actual Linux file system type called "iSCSI" but whatever, that's what the GUI wants you to do.)

Pick iSCSI as FS Type

Now you have a LUN to offer to clients as a target.

New volume

Make an iSCSI target

An iSCSI target is a "name" for disks that Openfiler will offer to clients. Clients will connect to Openfiler, and ask it to "SendTargets". Openfiler will respond with a list of available targets.

In a simple case, all you need is one target. Say you are offering up one "disk" (LUN or lvol) to one XenServer. You only need one target.

Look at the targets. Go to Volumes, iSCSI Targets.

iSCSI Targets

On a new Openfiler, there will be no targets, and Openfiler will offer to make a new one. It'll even suggest a name for you. iSCSI target names are called iSCSI Qualified Names (IQNs). Openfiler will generate a valid IQN for you. You could use this IQN as is, if desired. It is valid, however, to add a colon and some text to make the IQN more meaningful to humans (iSCSI doesn't care about readability).

So for example, Openfiler may suggest an IQN like:

It is valid to put :Openfiler_host_name on the end like so:

That way when you see the IQN later, you'll know what that IQN is for.

Add a target. I suggest adding a :Openfiler_host_name on the end for readability.

Add target

That will generate a target IQN ("name") for you to use. Once you have at least one target, you will see a long list of target settings you could optionally modify. Don't. The default options work fine for general use. If you decide to delete a given target, there's a delete button at the bottom of the page.

Target settings

Map a LUN to a target

So in the simplest case, you have one disk (LUN) you want to offer to a client. You have one target to use. So now you have to map the LUN to the target. Go to LUN Mapping. You will see your disk (LUN) and Openfiler will suggest mapping it to the one and only target. Take the defaults for "R/W Mode" (write-thru) and "Transfer Mode" (blockio). Click Map.

LUN Mapping

Now you can see that your LUN is mapped to the target.

LUN is mapped

Note that it is valid to have a number of LUNs mapped to one target. You could have also have a number of targets with one LUN each. You can mix and match like this, whatever makes sense to you. I don't think it matters from a performance or technical point of view. I'd probably opt to have multiple targets with meaningful names, and then map one LUN to each target. Since the client doesn't see the name of the LUN, only the target IQN, having one LUN on one target gives me the opportunity to use the target IQN to keep them straight. Whatever floats your boat.

Configure network access control

You're almost done. Hang in there.

The next required task is to set up network access control. Basically you need to tell Openfiler the IP address (or name) of the client machine that will use the target. Openfiler will only let that client access the target. If some other client issues a SendTargets command to Openfiler, Openfiler will return a null list (that client is not allowed to access a target).

Click on the Network ACL tab. You'll see:

No access
A list of networks have not been created yet.
You cannot configure network access control unless
you create a list of networks in the Local Networks section.
Until that time, this iSCSI target will be unavailable.

By default, Openfiler will deny access to the target by anyone. In other words, it's in a "default deny" stance, and you have to specify exceptions to the deny rule.

Click on the link and go to Local Networks.

At this point, you probably don't have groups of hosts that need access to LUNs. Probably just one or two hosts. The easiest thing to do is add these hosts, host IP address by host IP address. Examples in the Openfiler admin manual for a host entry use a mask of, so for a host entry, try that mask.

Add the hosts you need to grant access to. Just fill in the IP address information for the host and hit Update.

Network Access Configuration

Now go back to Volumes, iSCSI targets. You have told the Openfiler about your client hosts, but Openfiler is set to default deny. It will still not trust them. Change the clients from deny to allow and hit Update.

iSCSI host access configuration

Function check

From a valid iSCSI initiator (iSCSI client) on one of the client hosts you enabled access for you should be able to issue a SendTargets command, and get the IQN for the LUN you've created back. You should be able to select the LUN, and read and write to it.

If you want to "prove" things a bit more, try writing like crazy to it for a while. This will exercise the whole iSCSI connection, the client initiator, the network between initiator and target, and Openfiler. A quick way to is to get a disk benchmark utility, and set it to do a bunch of writes against the iSCSI mounted disk. I've used iometer on Windows and bonnie++ on Linux.

Note that since like most of us, you are probably using a software initiator (you don't have a fancy TCP Offload or hardware initiator NIC that does some of the work for you). What this means is that both the client and the Openfiler filer host will consume CPU when doing a lot of iSCSI work. This is "normal", hosts doing iSCSI show CPU load if using software initiators.

If this turns out to be a hassle either on a client or on the Openfiler host, then think about getting hardware initiators that do iSCSI packet processing on the NIC itself. If you want initiators for your Openfiler host, make sure that the ones you buy are supported under Linux/Openfiler.

Basic adminstration tasks


Make sure that nothing is using any iSCSI target, all is quiet. Turn off all virtual machines, for example. Then use the Openfiler GUI to turn off the filer.


Make sure that nothing is using any iSCSI target, all is quiet. Turn off all virtual machines, for example. Then use the Openfiler GUI to reboot the filer.


Make sure that nothing is using any iSCSI target, all is quiet. Turn off all virtual machines, for example. Then use the Openfiler GUI to update the filer. Reboot it after the update. Patch, reboot, patch, reboot, until no patches are needed.

Making a volume larger

You can easily make a volume larger. You must have free space in the volume group it is housed in in order to do this. This free space may also be used for snapshots, so if you ever want to try snapshots, don't use up all the room. My rule of thumb is to leave something like 25% of the volume group empty for snapshot use. Snapshots can come in very handy to instantly recover to a point in time.

Here's some other things to think about when preparing to make a volume bigger. You can make the volume bigger at any time, even when the volume is in use. However iSCSI clients will not see the new larger size until you restart the iscsi-target service on the Openfiler server. Restarting the service interrupts all iSCSI I/O's. So in order for clients to see the new larger size, all iSCSI disk clients must be off (or quiet).

Also note that this is a one way operation. It's easy to make the volume larger, but you can't easily shrink it again. If you make a mistake, your only option to make the volume is to copy the contents somewhere else, destroy the volume, make a new replacement with the correct smaller size, and copy the contents back. BE VERY SURE THAT THE NEW LARGER SIZE IS THE SIZE YOU WANT. Since it's so easy to make volumes larger, perhaps make it just a bit bigger and try it. Then if you need still more room, make it a bit bigger again.

Use the GUI to make a volume larger. Log into the Openfiler GUI. Click on the Volumes tab. Select the volume group that the logical volume you want to expand is housed in and click Change. Look at your volume group. Make sure that you have some free room to expand the logical volume into. Like I said, don't use it all.

Picture soon

Find your volume and click the Edit link in the Properties column. The GUI opens an Edit properties dialog.

Picture soon

Move the slider to select a larger size or type in the new size. Click Change.

Picture soon

The volume is quickly, almost instantly made larger. However, the iscsi-target service is still reporting to iSCSI disk clients the former smaller size. In order to have clients see the new size, you must restart the iscsi-target service. First, make sure that no iSCSI target is in use by clients. For example, if you're using the disk for storage of virtual machines, turn all virtual machines off.

Once all iSCSI disks are not in use, then restart the iscsi-target service. You can use the GUI or the command line. From the GUI, go to the Services tab. Disable, the re-enable the iSCSI target server.

If you want to use the command line then try:

[root@of0 ~]# service iscsi-target restart
Stopping iSCSI target service: ......                      [  OK  ]
Starting iSCSI target service:                             [  OK  ]

Alternately, you could just reboot the Openfiler server.

Even though the iscsi-target server is now reporting the new larger size, clients may still not see it. For example, XenServer doesn't see the new size until you detach/re-attach the storage repository (or you reboot it).

Backup and recovery

Coming soon, stay tuned.

Backing up the Openfiler system

The GUI has backup and recovery tools in it. I'm not sure yet what they do. If you're like me, I keep my filers very basic. I can load a filer from media and configure it in maybe an hour.

Backing up logical volumes used by clients for storage

Ahh, that's a good question. Openfiler has built in snapshots, and some fancy command line tools like drbd and zumastor. I'm practicing using some of these. When I know more, I'll post more information. A good first step is to run Openfiler (the system) on one small hard drive, and client storage on other disks. That means that problems with the Openfiler operating system won't affect the data disks and vice versa.

Basic troubleshooting

The usual issue you come across is that you think you've done all these steps, but you can't see the targets or get to a LUN. The usual culprit is that you missed a step along the way. A very common step to miss is changing "deny" to "allow" after you make the client host network entries. Recheck your work, go through each step and verify that you did it correctly. I always seem to fat-finger the network access entries, and then spin my wheels for a while until I realize I've done it again, *SIGH*. Make sure a host mask entry is, for example. If you are sure that every is done correctly , every now and then it still might not work. I've seen this in a few instances. It seems like Openfiler get "confused" sometimes, and needs a kick in the pants.

Try restarting the iSCSI target service. Sometimes just disabling, and then immediately re-enabling the service seems to do the trick.

I've seen a number of times where Openfiler "didn't like" the first target name I used. I did all the steps, everything *SHOULD* work, and the client can't talk to the target to see the LUNs. There's nothing special about the target name (assuming no other client is currently using it). Try another. Unmap the LUN, delete the target, generate another target, remap the LUN, maybe restart the iSCSI target service. I've had to do this maybe one in three times on new Openfiler installs. I'm not sure what's going on behind the scenes (there a lot of moving parts the GUI is hiding from you), but this isn't so hard. I've usually don't have to do this more than once, usually just the first target name on a brand new Openfiler install. Once a target is working, they seem to stay working.

On version 2.3 of Openfiler, after you get a target and some LUNs behind it, everything often goes kerflooey after an Openfiler reboot. I mentioned this above. Make sure that you made the tweaks I suggested above to keep this from happening.

Those are really the only issues I've seen in practice. The initial target name is sometimes flaky, and Openfiler doesn't like reboots. Note that you should very rarely reboot your storage filer anyway, so that isn't as big of a deal as it sounds. My test environments have literally written terabytes to iSCSI disks on Openfilers day in and day out for weeks, no hiccups or badness noted.

Good luck.

Personal tools