Naming Convention

By convention, IDE drives will be given device names /dev/hda to /dev/hdd. Hard Drive A (/dev/hda) is the first drive and Hard Drive C (/dev/hdc) is the third.

A typical PC has two IDE controllers, each of which can have two drives connected to it. For example, /dev/hda is the first drive (master) on the first IDE controller and /dev/hdd is the second (slave) drive on the second controller (the fourth IDE drive in the computer).

You can write to these devices directly (using cat or dd). However, since these devices represent the entire disk, starting at the first block, you can mistakenly overwrite the master boot record and the partition table, which will render the drive unusable.

Once a drive has been partitioned, the partitions will represented as numbers on the end of the names. For example, the second partition on the second drive will be /dev/hdb2. The partition type (primary) is listed in the table above for clarity 

SCSI drives follow a similar pattern; They are represented by 'sd' instead of 'hd'. The first partition of the second SCSI drive would therefore be /dev/sdb1. In the table above, the drive number is arbitraily chosen to be 6 to introduce the idea that SCSI ID numbers do not map onto device names under linux.

Name Assignment

Under (Sun) Solaris and (SGI) IRIX, the device name given to a SCSI drive has some relationship to where you plug it in. Under linux, there is only wailing and gnashing of teeth.

Before

SCSI ID #2        SCSI ID #5       SCSI ID #7        SCSI ID #8
 /dev/sda          /dev/sdb         /dev/sdc          /dev/sdd

After

SCSI ID #2                         SCSI ID #7        SCSI ID #8
 /dev/sda                           /dev/sdb          /dev/sdc

SCSI drives have ID numbers which go from 1 through 15. Lower SCSI ID numbers are assigned lower-order letters. For example, if you have two drives numbered 2 and 5, then #2 will be /dev/sda and #5 will be /dev/sdb. If you remove either, all the higher numbered drives will be renamed the next time you boot up.

If you have two SCSI controllers in your linux box, you will need to examine the output of /bin/dmesg in order to see what name each drive was assigned. If you remove one of two controllers, the remaining controller might have all its drives renamed. Grrr...

There are two work-arounds; both involve using a program to put a label on each partition. The label is persistent even when the device is physically moved. You then refer to the partition directly or indirectly by label.

Logical Partitions

The table above illustrates a mysterious jump in the name assignments. This is due to the use of logical partitions.

This is all you have to know to deal with linux disk devices.

Table 6. Device file attributes

Foreign Partition Types

The partition type codes have been arbitrarily chosen (you can't figure out what they should be) and they are particular to a given operating system. Therefore, it is theoretically possible that if you use two operating systems with the same hard drive, the same code might be used to designate two different partition types. OS/2 marks its partitions with a 0x07 type and so does Windows NT's NTFS. MS-DOS allocates several type codes for its various flavors of FAT file systems: 0x01, 0x04 and 0x06 are known. DR-DOS used 0x81 to indicate protected FAT partitions, creating a type clash with Linux/Minix at that time, but neither Linux/Minix nor DR-DOS are widely used any more.

OS/2 marks its partitions with a 0x07 type and so does Windows NT's NTFS. MS-DOS allocates several type codes for its various flavors of FAT file systems: 0x01, 0x04 and 0x06 are known. DR-DOS used 0x81 to indicate protected FAT partitions, creating a type clash with Linux/Minix at that time, but neither Linux/Minix nor DR-DOS are widely used any more.

Logical Partitions

One primary partition of a hard drive may be subpartitioned. These are logical partitions. This effectively allows us to skirt the historical four partition limitation.

The primary partition used to house the logical partitions is called an extended partition and it has its own file system type (0x05). Unlike primary partitions, logical partitions must be contiguous. Each logical partition contains a pointer to the next logical partition, which implies that the number of logical partitions is unlimited. However, linux imposes limits on the total number of any type of partition on a drive, so this effectively limits the number of logical partitions. This is at most 15 partitions total on an SCSI disk and 63 total on an IDE disk.

Swap Partitions

Every process running on your computer is allocated a number of blocks of RAM. These blocks are called pages. The set of in-memory pages which will be referenced by the processor in the very near future is called a "working set." Linux tries to predict these memory accesses (assuming that recently used pages will be used again in the near future) and keeps these pages in RAM if possible.

If you have too many processes running on a machine, the kernel will try to free up RAM by writing pages to disk. This is what swap space is for. It effectively increases the amount of memory you have available. However, disk I/O is about a hundred times slower than reading from and writing to RAM. Consider this emergency memory and not extra memory.

Everything in your linux file system can go in the same (single) partition. However, there are circumstances when you may want to restrict the growth of certain file systems. For example, if your mail spool was in the same partition as your root fs and it filled the remaining space in the partition, your computer would basically hang.

Simple Invocation

mke2fs /dev/hdb1

mkfs.ext2 /dev/hdb1

both of which make an ext2 file system on the first partition of the second drive, and

mke2fs -j /dev/hdb1

mkfs.ext3 /dev/hdb1

make an ext3 file system.

Reserved blocks

The -m option is probably the one of most use to non-experts. If the file system becomes filled and there is no more space to write, it is basically unusable because the operating system is constantly writing to disk. By default, five percent of the partition is reserved for use by the root user. This allows root to conduct administrative activities on the partition and perhaps move some data off. However, this is most critical when the partition contains / or home directories. For pure data partitions, this is just lost space. Five percent of a 250Gb partition is 12.5 Gb. Especially in the case of large partitions, it is safe to set the reserved space to the minimum, which is one percent.

mkfs.ext3 -m 1/dev/hdb1

creates a file system with only 1% of its space reserved for the root user. tune2fs -m can be used to adjust the reserved blocks after data is loaded on the partition.

The Linux Logical Volume Manager (LVM) is a mechanism for virtualizing disks. It can create "virtual" disk partitions out of one or more physical hard drives, allowing you to grow, shrink, or move those partitions from drive to drive as your needs change. It also allows you to create larger partitions than you could achieve with a single drive.

Traditional uses of LVM have included databases and company file servers, but even home users may want large partitions for music or video collections, or for storing online backups. LVM and RAID 1 can also be convenient ways to gain redundancy without sacrificing flexibility.

This article looks first at a basic file server, then explains some variations on that theme, including adding redundancy with RAID 1 and some things to consider when using LVM for desktop machines.

LVM Installation

An operational LVM system includes both a kernel filesystem component and userspace utilities. To turn on the kernel component, set up the kernel options as follows:

Device Drivers --> Multi-device support (RAID and LVM)

    [*] Multiple devices driver support (RAID and LVM)
    < >   RAID support
    <*>   Device mapper support
    < >     Crypt target support (NEW)

You can usually install the LVM user tools through your Linux distro's packaging system. In Gentoo, the LVM user tools are part of the lvm2package. Note that you may see tools for LVM-1 as well (perhaps named lvm-user). It doesn't hurt to have both installed, but make sure you have the LVM-2 tools.

LVM Basics

To use LVM, you must understand several elements. First are the regular physical hard drives attached to the computer. The disk space on these devices is chopped up into partitions. Finally, a filesystem is written directly to a partition. By comparison, in LVM, Volume Groups (VGs) are split up into logical volumes (LVs), where the filesystems ultimately reside (Figure 1).

Each VG is made up of a pool of Physical Volumes (PVs). You can extend (or reduce) the size of a Volume Group by adding or removing as many PVs as you wish, provided there are enough PVs remaining to store the contents of all the allocated LVs. As long as there is available space in the VG, you can also grow and shrink the size of your LVs at will (although most filesystems don't like to shrink).

The Linux Logical Volume Manager (LVM) is a mechanism for virtualizing disks. It can create "virtual" disk partitions out of one or more physical hard drives, allowing you to grow, shrink, or move those partitions from drive to drive as your needs change. It also allows you to create larger partitions than you could achieve with a single drive.

Traditional uses of LVM have included databases and company file servers, but even home users may want large partitions for music or video collections, or for storing online backups. LVM and RAID 1 can also be convenient ways to gain redundancy without sacrificing flexibility.

This article looks first at a basic file server, then explains some variations on that theme, including adding redundancy with RAID 1 and some things to consider when using LVM for desktop machines.

LVM Installation

An operational LVM system includes both a kernel filesystem component and userspace utilities. To turn on the kernel component, set up the kernel options as follows:

Device Drivers --> Multi-device support (RAID and LVM)

    [*] Multiple devices driver support (RAID and LVM)
    < >   RAID support
    <*>   Device mapper support
    < >     Crypt target support (NEW)

You can usually install the LVM user tools through your Linux distro's packaging system. In Gentoo, the LVM user tools are part of the lvm2package. Note that you may see tools for LVM-1 as well (perhaps named lvm-user). It doesn't hurt to have both installed, but make sure you have the LVM-2 tools.

LVM Basics

To use LVM, you must understand several elements. First are the regular physical hard drives attached to the computer. The disk space on these devices is chopped up into partitions. Finally, a filesystem is written directly to a partition. By comparison, in LVM, Volume Groups (VGs) are split up into logical volumes (LVs), where the filesystems ultimately reside (Figure 1).

Each VG is made up of a pool of Physical Volumes (PVs). You can extend (or reduce) the size of a Volume Group by adding or removing as many PVs as you wish, provided there are enough PVs remaining to store the contents of all the allocated LVs. As long as there is available space in the VG, you can also grow and shrink the size of your LVs at will (although most filesystems don't like to shrink).

A Basic File Server

A simple, practical example of LVM use is a traditional file server, which provides centralized backup, storage space for media files, and shared file space for several family members' computers. Flexibility is a key requirement; who knows what storage challenges next year's technology will bring?

For example, suppose your requirements are:

400G  - Large media file storage
 50G  - Online backups of two laptops and three desktops (10G each)
 10G  - Shared files

Ultimately, these requirements may increase a great deal over the next year or two, but exactly how much and which partition will grow the most are still unknown.

Disk Hardware

Traditionally, a file server uses SCSI disks, but today SATA disks offer an attractive combination of speed and low cost. At the time of this writing, 250 GB SATA drives are commonly available for around $100; for a terabyte, the cost is around $400.

SATA drives are not named like ATA drives (hda, hdb), but like SCSI (sda, sdb). Once the system has booted with SATA support, it has four physical devices to work with:

/dev/sda  251.0 GB
/dev/sdb  251.0 GB
/dev/sdc  251.0 GB
/dev/sdd  251.0 GB

Next, partition these for use with LVM. You can do this with fdiskby specifying the "Linux LVM" partition type 8e. The finished product looks like this:

# fdisk -l /dev/sdd

Disk /dev/sdd: 251.0 GB, 251000193024 bytes
255 heads, 63 sectors/track, 30515 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device            Start   End      Blocks      Id  System
/dev/sdd1         1       30515    245111706   8e  Linux LVM

Notice the partition type is 8e, or "Linux LVM."

Creating a Virtual Volume

Initialize each of the disks using the pvcreatecommand:

# pvcreate /dev/sda /dev/sdb /dev/sdc /dev/sdd

This sets up all the partitions on these drives for use under LVM, allowing creation of volume groups. To examine available PVs, use the pvdisplaycommand. This system will use a single-volume group named datavg:

# vgcreate datavg /dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1

Use vgdisplayto see the newly created datavgVG with the four drives stitched together. Now create the logical volumes within them:

# lvcreate --name medialv  --size 400G
# lvcreate --name backuplv --size  50G
# lvcreate --name sharelv  --size  10G

Without LVM, you might allocate all available disk space to the partitions you're creating, but with LVM, it is worthwhile to be conservative, allocating only half the available space to the current requirements. As a general rule, it's easier to grow a filesystem than to shrink it, so it's a good strategy to allocate exactly what you need today, and leave the remaining space unallocated until your needs become clearer. This method also gives you the option of creating new volumes when new needs arise (such as a separate encrypted file share for sensitive data). To examine these volumes, use the lvdisplaycommand.

Now you have several nicely named logical volumes at your disposal:

/dev/datavg/backuplv     (also /dev/mapper/datavg-backuplv)
/dev/datavg/medialv      (also /dev/mapper/datavg-medialv)
/dev/datavg/sharelv      (also /dev/mapper/datavg-sharelv)