Never grow a disk if the machine has snapshots. Even if the GUI lets you.
This morning I got a request to grow a virtual disk on a vm. What you should do is look at the machine and see if it has any snapshots. If it does, then you should delete them before attempting any disk grow operations. In fact, what should happen is is if you try it, the GUI will give you an error stating that the “machine has snapshots, grow is not allowed”. I was going too fast (and didn’t have enough coffee) and tried growing it with the machine on. It failed. I didn’t check for snapshots, I just turned the machine off. Then I grew the disk. It allowed me to do this. (Really, the GUI should have refused to do this.) This did *NOT* do what I expected…
Behind the scenes what happened was that the grow operation messed up the reference to the snapshot. So the disk I grew was basically the right size but missing the active snapshot. In effect, it looked like the disk from “way back when” (before the snapshot was made) and was missing a lot of contents. We tried various fiddling around with vmkfstools, etc., but it was never happy again. We wound up recovering the file system on the affected disk from tape.
If I get bored someday, I’m going to see if this happens reliably with a test machine. In other words:
- Make a VM, put some stuff in it.
- Make a snapshot
- Change the vm, perhaps add some stuff into a file system.
- Attempt to grow a disk with it on (the GUI should stop you)
- Turn the machine off, attempt to grow a disk with it off (the GUI should stop you)
If it reliably behaves like it did this morning (i.e. allows you to grow the disk, thereby destroying the vm), then it probably should be reported to VMware as a bug.
See this VMware KB article for the gory details:
Warning: In current versions of ESX 2.5, 3.x and 4.x, do not extend a disk that has snapshots. Commit or discard snapshots before increasing the size of the virtual disk. Extending the size of a virtual disk that has snapshots invalidates the snapshots.
