Monday, April 22, 2013

How to handle the 'disk space full' problem in your Virtual Machine in VMware Workstation ?

Good to write a blog after a long break. This post is about how I  came out of a big headache recently. Currently I run a fedora 14-64 bit virtual machine in VMWare Workstation 7.1.4 build-385536. This is mainly created for the purpose of installing and maintaining oracle 11g database management system. Since I’m doing my College assignments in the virtual machine, I used to create separate databases for each assignment for ease of understanding but meanwhile, I ended up with a trouble because of running out of disk space in my home directory. I couldn't continue doing anything with the Virtual machine without a proper disk re size to allocate some more disk space at least for /dev/sda4 partition of the VM where the home directory is also located.

First of all, I tried little tricks to get some space in the disk to continue my work. You may guess what they might be. What I basically thought is to do something similar to performing disk clean up and running a Defragmentation for the disk as same as we perform such in the familiar Windows environment. My actions are listed below



Since the above tasks can be performed independently, I have made them available in a set of posts. If you want to perform any task, please click on the action you are interested in to view the specific post. Since these types of issues are an obvious headache for a beginner, I thought to make the things understand in an easier manner in those posts.

Increase the size of the disk partition(s) of the required drive of the virtual machine using GParted

In my previous post, I described how to expand the virtual disk size in VMWare Workstation 7. But that is not enough for you to increase the usable disk size of your virtual machine. The size of the disk partition wont increase as observed inside the guest operating system. You must follow steps related to your specific operating system to increase the size of the disk partition or partitions. 

I recommend you that always before you meddle with your disks, partitions, memory related activities, please have a backup of your virtual machine. What I mean here as a 'backup' is a Full Clone of your virtual machine. I don't recommend having just a copy of your virtual machine folder in some other location in your computer.

Now, the problem we have in our hand is that how to add the expanded disk size to the partitions of our vmware to work with it. The tool that I have used here is GParted. Though it took some time for me to understand the concepts behind it and how it works, finally I was in the correct track. 

First of all, I would like to recommend some reference materials that I used to learn about GParted. I think they'll be help for you too. 

You'll definitely need to use the GParted website to find out the correct version of this tool which supports to the platform that you work with. here is the link for that


If you would like to watch a video to have a quick understanding of what we are supposed to do with Gparted and how it works, you can watch these two videos as reference. They helped me to learn the tool for a certain extent.
Following one shows how it works in Ubuntu platform


The video shown below is also taken from YouTube but little unclear to understand without speech demonstration. Actually It solved half of the unclear points that I had about using GParted.




Anyways, I'll explain what the tool is and how to use it as far as I learned about it. Since I too learned about Gparted and its usage as a self learner, I think I can give some help for a beginner to work with it. That is the main intention of these series of posts about virtual machines and their disk space management.

What is GParted?



GParted is a free partition editor for graphically managing your disk partitions.With GParted you can resize, copy, and move partitions without data loss, enabling you to:

  • Grow or shrink your drives
  • Create space for new operating systems
  • Attempt data rescue from lost partitions

What are the Requirements to use GParted in your machines?


GParted can be used on x86 and x86-64 based computers running Linux, Windows, or Mac OS X by booting from media containing GParted Live. A minimum of 128 MB of RAM is needed to use all of the features of the GParted application.


Installation

Though I have titled it as Installation, there should not be an installation for GParted. This is the point where I got confused and spend lot of time to search what went wrong and why. 

Actually, Gparted is a disk partitioning tool and it should not be installed and used from a disk what we are trying to partition. I did this mistake by installing GParted in my fedora 14 virtual machine using yum and tried to partition by disk. I observed from the Gparted GUI that all my disk partitions were marked with a key icon (which means, the drives are busy) and under that state, it wont allow you to resize your partitions. That's quite obvious. 

Then how should it be run? Actually I tried out three ways to run Gparted to re size partitions of my Virtual machine.

  • create a bootable USB flash drive with GParted Live
  • create a bootable CD/ DVD with GParted Live
  • Simply browse the ios image of Gparted Live in VMWare CD/DVD option 

Download GParted Live 


To install GParted Live on CD, download the .iso file and burn it as an image to a CD. I downloaded gparted-live-0.14.1-6-i486.iso which is about 133MB and burnt in to a CD. If you don't know how to make a bootable CD you can google it. If you have Windows 7 in your machine and feeling lazy to google for it, then follow the link below :) . It's simple and easy


of course you also can make a bootable USB flash drive using some tool like Universal-USB-Installer within few seconds with the same iso image that I downloaded above. Yoou can get it from here,


I had problems in setting my USB flash drive as the first in System BIOS boot sequence. So ultimately, I had to go for the CD option. You can select which ever the way you want. 

The point that you should take to your consideration here is, you have to run GParted on top of your OS. not running within it. All what we need to do is, boot your OS from some bootable device  with GParted and observe the currently available partitions you have in a GUI. Then you can select  a partition at a time and expand, shrink, move or what ever the option that is available for you in an easier manner. 

So, that's the simple magic behind GParted partitioning tool. But always keep in your mind that the tasks that you perform via GParted is so critical. If you used it without care, even your OS may crash. Believe my words because I already experienced such scenarios many times when experimenting with the tool :). Luckily, I always had backups of my virtual machine in both my desktop computer and laptop (because I'm dead scared of loosing all my efforts in a flash :p ). 

But before starting the partitioning task, let me explain what my problem is. 

Currently I am running a Fedora 14- 64 bit virtual machine in VMWare Workstation 7.1.4. I have oracle 11g installed in it and as I work with many databases within my fedora OS, the size of the disk expanding very fast. I was in a great trouble from the moment my OS prompted a warning saying my home directory is almost full. I could verify it as below. 













note that /dev/sda5 is almost 90% in the above scenario. What I wanted to do is to allocate some more space at least for that partition. Before doing that, I performed some other tasks like Performing a disk defragmentation, Compacting the virtual hard disk, expanding the size of the virtual hard disk . You can refer them if you too want to know how I did those tasks but for the re sizing of the disk partition  you just have to know how you can add some space to your virtual hard disk in VMWare. you can refer only that post if you are interested.

I believe, now you are ready with the bootable CD of GParted. Our first task is to let VMWare workstation know that we are required to boot from that CD if it is available. to do so, follow the below steps.

1.  power off your virtual machine if it is already running
2.  make sure that you are in the correct tab of your virtual machine that you are going to perform the disk partitioning



3. select VM-->settings-->CD/DVD(IDE) under Hardware tab-->select Auto Detect under Use physical drive option






















4. Make sure you have the GParted bootable CD inserted in to your CD/DVD Drive

How to change boot sequence of your virtual machine in VMWare Workstation?

1. follow the above four steps
2. Select VM-->Power-->Power On to BIOS






















3.Once you see the BIOS screen as below just click inside the screen using the mouse and there onward you have to use only the keys.


4. Use Right arrow key and select Boot tab




























5. Use up and down arrow keys to select CD-ROM Drive and click + key from the keyboard to move it up. Place in as the first in the boot sequence as shown below


6. Press F10 to save the changes. select Yes and hit Enter























How to boot the virtual machine from GParted Live CD?

1. Follow the above steps to reorder the boot sequence from the BIOS settings of your virtual machine. If you have set the CD-ROM as the first one in the list.

2. Power on your virtual machine and it will boot up in GParted as shown below























3. Select GParted Live (Default Settings) and hit Enter key























4.Select Don't touch keymap and hit Enter key



5.type 02 as the Language option and hit Enter key



6.Enter 0 as the preferd mode and hit Enter key



7.You should end up with a screen as shown below



8.wait until it auto detects the disk partitions and displays in GParted GUI





9.you can observe the partitions in different colors. Used space in yellow, unused space in white, unallocated space in grey and so on

10. select the partition that you want to re-size from the below list




11. Right click on it and select Resize/Move



12. You can either use the increment arrows to set a specific value for the parttion or just drag it to the prefered location towards the unallocated space as shown below





13. Click Resize/Move

14. note that your action has added in the bottom of the list with a green icon.It says, you have a set of action planned so far. But until you apply them, the changes won't be done. So, click on the Apply button (you can see it with a green correct mark labeled as Apply) to apply the changes



15. If the task is completed, you will see a success screen as below. click Close button to close the window (It's not mandatory to save the details in a file)



16. You should observe the applied changes as shown below




17. You can perform any task with a selected partition  but make sure that anytime any action can cause your data in the virtual machine to be corrupted. Make sure your are working with a virtual machine that has a backup already

once you have done with the resizing in GParted, you can select GParted-->Quit option to quit from the GParted GUI and then click on the Exit button on the GParted Desktop to exit. When it asks the shutting down options, select shutdown. Then it'll automatically shutdown your virtual machine meanwhile ejecting your GParted Live CD from the CD/DVD drive.

You  can power on your virtual machine , open  a new terminal and type df -h  to check whether the disk partitions are re sized as you applied.

Note: I could repartiiton my home directory's partition successfully in this way. It is not that difficult because whenever you add space to your virtual disk, the unallocated space positions just after the /dev/sda5 partition  where it's easy for you to add the unallocated space to /dev/sda5. but think that you want to add more space to your root partition  It's not that easy, right. You have to move the unallocated space just beside the root partition and add to it.

As far as I tried, that was a bit tedious task because moving partitions can end up with data loss. Moving partitions take more time and while you are waiting, the process may stop and the partition may become inconsistent.

I am not discouraging you not to use those features but be really careful with the actions that you are performing when repartitioning. you don't need to worry if your partitions doesn't contain any data but if it has sensitive data, they have to be handled with more care.

OK, that is the end of this long post :) Hope it was helpful and please leave a comment if  you have any suggestions about my posts.

Sunday, April 21, 2013

Expand the size of the virtual disk in VMWare Wokstation


I write this series of posts in order to explain you how I followed some steps to overcome the problem I faced because of running out of disk space in my fedora 14 virtual machine. So far I have explained how I could defragment the virtual hard disk  and compact the virtual hard disk to obtain some space from the unused space. If you require further details, please refer the posts in advance but it doesn't mean that you must follow those steps in order to add some more space to your virtual hard disk to expand the size of it. As I have already mentioned, the tasks that I performed are independent from each other.
Before expanding your VM disk size, please note that even though this provides steps to increase the size of your virtual disk, the size of the disk partition does not increase as observed inside the guest operating system. You must follow steps related to your specific operating system to increase the size of the disk partition or partitions. Don't worry, I'll be explaining how I achieved that task through a very useful tool called GParted in one of my future posts.

I recommend you that always before you meddle with your disks, partitions, memory related activities, please have a backup of your virtual machine. What I mean here as a 'backup' is a  Full Clone of your virtual machine. I don't recommend having just a copy of your virtual machine folder in some other location in your computer.

Whatever the version of VMWare you are using, confirm the following before increasing the size of the virtual disk.

  • Check whether you have existing snapshots of your virtual machine. if so, remove all of them
  • Make sure your virtual machine is powered off.

Steps that should be followed in VMWare Workstation 7.1.4 

  1. Power off the virtual machine if it is already running
  2. select the proper tab of your virtual machine from the VMWare Home screen

3. Select VM tab from the top and select Settings



4. Under Hardware tab, click on Hard Disk (SCSI) and select Expand from the Utilities drop down list.




5.Enter the new size that you want to allocate as more space to your disk and then click Expand.                   



6. The disk expand starts automatically and displays a success message if it completes




It would take some time to complete the disk expanding task. When it completes, it will display a success message. You can also notice that the new size of your virtual machine disk has changed to the value that you provided. 

if you came this far to solve the 'low disk space' in your virtual machine, even now you will notice your problem has not solved yet. What actually happens here is that VMWare has just added some space to your virtual disk as you requested but it will just remain as unallocated space in your virtual disk without actually adding them to your disk partition or partitions. 

The next challenge is how we are going to add this 'added but unallocated space' to our virtual machine disk partition to make use of it.

See you with the solution in the next post :)











Compact the Virtual Disk in VMware Workstation

My previous post was about how you can defragment your virtual disk in VMWare. Though I already mentioned there that you cannot reclaim your virtual hard disk's  unused space on a virtual disk, it is best practice to run a disk defragmentation before compacting the disk.

What really happens if you compact the virtual Disk?

If you have a virtual disk that grows as data is added, you can shrink it as describes in this post. Note that if you have allocated all the space for you virtual disk at the time you created it, you cannot shrink it. In my case, I am running a fedora 14 virtual machine (VMWare workstation 7.1.4) which has oracle 11g installed in it. As I create databases and maintain them, the disk size grows fast and ultimately I had to perform some tasks to come out of the disk full issue by following some actions. 

When you delete files from your virtual machine, the disk space occupied by those files is not immediately returned to your host system. If a virtual disk has such empty space, you can use the Clean up disks command to return that space to the hard drive on a Microsoft Windows host.


The Clean up disks command is similar to the Compact command in the Workstation virtual machine settings and the shrink command provided by VMware Tools. 


If you want to know how to Defragment the virtual hard disk in VMWare, refer the post under Defragmenting the virtual Disk in VMWare because it is best [raactice to defragment your virtula hard disk before you compact or shrink it

Before shrinking the virtual disk, consider the following restrictions and requirements

  • Shrinking requires free disk space on the host equal to the size of the virtual disk you are shrinking.
  • Shrinking applies only to virtual disks. You cannot shrink physical disks or CD-ROMs.
  • You cannot shrink the virtual disk if,
    •  you have already created Snapshots of your virtual machine
    • If your VM is a parent of a linked clone 
    • If your VM is a linked clone (because linked clone depends on its parent clone
Note that there are two types of clones that you can create using a Virtual machine. They are Full clones and Linked Clones. For the moment, remember that Full clone is similar to a Copy of a file that you create in any Operating system and  a Linked clone is like a shortcut that you create for a file. Shortcuts are always dependent on the actual file whereas copies of files can work independently. I'll explain these concepts in some other post but keep in mind you have to consider the above points before performing a Shrinking of disk.

Steps to Shrink a virtual hard disk

Please note that I run  my virtual machine in VMWare Workstation 7.1.4 in which the Shrink task is refers as Compact. follow the below steps to perform the task



  1. Power off the virtual machine if it is already running
  2. select the proper tab of your virtual machine from the VMWare Home screen
  3. Select VM tab from the top and select Settings
  4. Under Hardware tab, click on Hard Disk (SCSI) and select Compact from the Utilities drop down list.
  5. The disk compact starts automatically and displays a success message if it completes






















It would take some time to complete the disk compact task. When it completes, it will display a success message.


Defragmenting the virtual Disk in VMWare

Actually what does fragmentation and defragmentation mean?

These two terms refer to the condition of a disk in which files are divided into pieces scattered around the disk. Fragmentation happens when we use disks frequently in creating, deleting and modifying files. At some point, the operating system needs to store parts of a file in non-contiguous clusters. Though this is entirely invisible to users, it can slow down the speed at which data is accessed because the disk drive must search through different parts of the disk to put together a single file. 

Just think of it in a simple scenario. If you put all your books around a table, will you be able to find a specific book quickly? But it is easier and quicker to find it if you have organized and stored the books in a proper way. This is the state that I want to explain here. All what I am going to do is similar to rearranging the books on a table to make finding things faster :)

Like physical disk drives, virtual disks also can become fragmented. Defragmenting disks rearrange files, programs and unused space on the virtual disks so that programs run faster and files quicker. Also note that defragmenting does not reclaim unused space on a virtual disk. to do so, you can shrink the virtual disk. But it is best practice to run a disk defragmentation before shrinking the disk 

Let me explain the steps you need to follow to defragment the virtual hard disk through your VMWare Workstation

  1. Power off the virtual machine if it is already running
  2. select the proper tab of your virtual machine from the VMWare Home screen

  3. Select VM tab from the top and select Settings
  4. Under Hardware tab, click on Hard Disk (SCSI) and select Defragment from the Utilities drop down list. 























 


  5.The disk defragmentation starts automatically and displays a success message if the defragmentation      completes



























Remember that Disk defragmentation may take a considerable time based on how frequently you run this action.


Note: The defragmentation process requires free working space on the host computer's disk. If your virtual disk is contained in a single file, for example, you need free space equal to the size of the virtual disk file. Other virtual disk configurations require less free space.