Showing posts with label resize2fs. Show all posts
Showing posts with label resize2fs. Show all posts

How to reduce AWS EBS root volume size

Jephe Wu - http://linuxtechres.blogspot.com

Objective:  reduce the AWS EBS root volume size from 8G to 4G since file system only used less than 3G.
Environment:  Amazon Linux AMI 2015.03 (HVM), SSD Volume Type - ami-fd9cecc7, t2.macro instance type with 8G gp EBS volume as root volume (default creation steps with launch instance wizard)

Diagram

original instance: jephe-base, with 8G ebs root volume, need to shrink it to 5G 
lab1 instance: used for the whole shrink process test
ebs volume 'source': created from snapshot of root volume from original instance.
ebs volume 'dest':  manually created new volume with 5G 



Steps:
1.  create another small instance named lab1
Amazon Linux AMI 2015.03 (HVM), SSD Volume Type - ami-fd9cecc7 , t2.macro, 8G gp ebs volume.

2.  make snapshot backup and create volume from it
stop the original base instance, detach its 8G volume, make snapshot of it, then create test volume named 'source' from snapshot, so we can work on the source volume instead of original base volume 

3. create another new volume named 'dest' with expected size (5g) and within same AZ as instance lab1
Note: the actual file system size of base volume must be less than 5G

4. attach volume F and G to small instance lab1 as /dev/xvdf and /dev/xvdg respectively
so the Linux OS will see /dev/xvdf, /dev/xvdf1 and /dev/xvdg as follows

5. while lab1 instance is running, attach both volume source and dest to it as /dev/sdf and /dev/sdg.

[ec2-user@ip-10-0-11-185 ~]$ more /proc/partitions
major minor  #blocks  name

 202        0    8388608 xvda
 202        1    8386543 xvda1
 202       80    8388608 xvdf
 202       81    8386543 xvdf1
 202       96    5242880 xvdg


6. fsck and resize2fs to minimum 

[root@ip-10-0-11-185 mnt]# e2fsck -f /dev/xvdf1
e2fsck 1.42.12 (29-Aug-2014)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/: 38013/524288 files (0.2% non-contiguous), 606002/2096635 blocks

[root@ip-10-0-11-185 mnt]# resize2fs -M -p /dev/xvdf1
resize2fs 1.42.12 (29-Aug-2014)
Resizing the filesystem on /dev/xvdf1 to 944604 (4k) blocks.
Begin pass 2 (max = 139004)
Relocating blocks             XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Begin pass 3 (max = 64)
Scanning inode table          XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Begin pass 4 (max = 4518)
Updating inode references     XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
The filesystem on /dev/xvdf1 is now 944604 (4k) blocks long.

[root@ip-10-0-11-185 mnt]# resize2fs -M -p /dev/xvdf1
resize2fs 1.42.12 (29-Aug-2014)
Resizing the filesystem on /dev/xvdf1 to 940777 (4k) blocks.
The filesystem on /dev/xvdf1 is now 940777 (4k) blocks long.

[root@ip-10-0-11-185 mnt]# resize2fs -M -p /dev/xvdf1
resize2fs 1.42.12 (29-Aug-2014)
Resizing the filesystem on /dev/xvdf1 to 940770 (4k) blocks.
The filesystem on /dev/xvdf1 is now 940770 (4k) blocks long.

7. calculate the actual file system size after resize2fs -M, round it up a bit to 300
[root@ip-10-0-11-185 mnt]# echo "scale=5; 940770*4/(1024*16)" | bc
229.68017

8.  make partition for /dev/xvdg by fdisk
[root@ip-10-0-11-185 mnt]# fdisk /dev/xvdg
Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0xaa83e894.

Command (m for help): p

Disk /dev/xvdg: 5368 MB, 5368709120 bytes, 10485760 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0xaa83e894

    Device Boot      Start         End      Blocks   Id  System

Command (m for help): n
Partition type:
   p   primary (0 primary, 0 extended, 4 free)
   e   extended
Select (default p): p
Partition number (1-4, default 1):
First sector (2048-10485759, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-10485759, default 10485759):
Using default value 10485759
Partition 1 of type Linux and of size 5 GiB is set

Command (m for help): wq
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
[root@ip-10-0-11-185 mnt]# fdisk -l /dev/xvdg

Disk /dev/xvdg: 5368 MB, 5368709120 bytes, 10485760 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0xaa83e894

    Device Boot      Start         End      Blocks   Id  System
/dev/xvdg1            2048    10485759     5241856   83  Linux

[root@ip-10-0-11-185 mnt]# cat /proc/partitions 
major minor  #blocks  name

 202        0    8388608 xvda
 202        1    8386543 xvda1
 202       80    8388608 xvdf
 202       81    8386543 xvdf1
 202       96    5242880 xvdg
 202       97    5241856 xvdg1

9.  use dd to copy file system 
[root@ip-10-0-11-185 mnt]# dd if=/dev/xvdf1 of=/dev/xvdg1 bs=16M count=230 
230+0 records in
230+0 records out
3858759680 bytes (3.9 GB) copied, 113.625 s, 34.0 MB/s

Note: you can run kill -USR1 pidofdd from another terminal to get the dd progress

10. mount /dev/xvdg1 to /mnt/g and bind mount /dev to /mnt/g/dev 

[root@ip-10-0-11-185 mnt]# mount /dev/xvdg1 g
[root@ip-10-0-11-185 mnt]# cd g
[root@ip-10-0-11-185 g]# ls
bin  boot  dev  etc  home  lib  lib64  local  lost+found  media  mnt  opt  proc  root  run  sbin  selinux  srv  sys  tmp  usr  var

[root@ip-10-0-11-185 g]# mount --bind /dev /mnt/g/dev/


11. install grub on /dev/xvdg

[root@ip-10-0-11-185 g]# chroot .
[root@ip-10-0-11-185 /]# ls
bin  boot  dev  etc  home  lib  lib64  local  lost+found  media  mnt  opt  proc  root  run  sbin  selinux  srv  sys  tmp  usr  var
[root@ip-10-0-11-185 /]# grub
Probing devices to guess BIOS drives. This may take a long time.


    GNU GRUB  version 0.97  (640K lower / 3072K upper memory)

 [ Minimal BASH-like line editing is supported.  For the first word, TAB
   lists possible command completions.  Anywhere else TAB lists the possible
   completions of a device/filename.]
grub> find /boot/grub/stage1
find /boot/grub/stage1
 (hd0,0)
 (hd1,0)
 (hd2,0)
grub> root (hd2,0)
root (hd2,0)
 Filesystem type is ext2fs, partition type 0x83
grub> setup (hd2)
setup (hd2)
 Checking if "/boot/grub/stage1" exists... yes
 Checking if "/boot/grub/stage2" exists... yes
 Checking if "/boot/grub/e2fs_stage1_5" exists... yes
 Running "embed /boot/grub/e2fs_stage1_5 (hd2)"...  29 sectors are embedded.
succeeded
 Running "install /boot/grub/stage1 (hd2) (hd2)1+29 p (hd2,0)/boot/grub/stage2 /boot/grub/grub.conf"... succeeded
Done.
grub> quit
quit
[root@ip-10-0-11-185 /]# exit
exit
[root@ip-10-0-11-185 g]# cd /

[root@ip-10-0-11-185 /]# umount /mnt/g/dev
[root@ip-10-0-11-185 /]# umount /mnt/g/
[root@ip-10-0-11-185 /]# df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/xvda1      7.8G  919M  6.8G  12% /
devtmpfs        490M   72K  490M   1% /dev
tmpfs           499M     0  499M   0% /dev/shm


[root@ip-10-0-11-185 ec2-user]# e2fsck -f /dev/xvdg1
e2fsck 1.42.12 (29-Aug-2014)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/: 38013/237568 files (0.3% non-contiguous), 587515/940770 blocks

Now stop instance lab1, detach all 3 volumes including os base volume for lab1, attach dest volume to lab1 with root device as /dev/xvda. start up instance lab1 again to make sure it can start up successfully.

Now, stop test instance lab1, detach volume named 'dest' , stop original instance and detach its volume,  attach volume 'dest' to original instance to start up.

Note: during ec2 instance startup, it will automatically resize file sytem to the maximum.

Here is the df -h result after startup the 'dest' volume with original instance jephe-base.

[root@ip-172-31-13-17 ec2-user]# df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/xvda1      4.8G  2.2G  2.7G  46% /
devtmpfs        490M   56K  490M   1% /dev
tmpfs           499M     0  499M   0% /dev/shm



How to expand array under Linux for HP Proliant server

Jephe Wu - http://linuxtechres.blogspot.com

Environment: HP Proliant DL360G6, CentOS 5.4 64bit, 6x300G hard disk with RAID5 (one for spare), hot added 2 more 300G hard disk, default disk partition layout by OS installation.
Objective: expand the existing RAID5 array with 2 more new added hard disk online

Steps:
1. Install HP Proliant support pack for RHEL 5

2. start up HP array configuration utility online for Linux
cd /opt/compaq/cpqacuxe/bld
./cpqacuxe -R
note: after finishing online configuration utility, you should stop it by running
./cpqacuxe -stop

3. expand array and logical drive

access https://log.domain.com:2381 then click on array configuration utility link
click on expand array, this will take long time to finish, after that, another button which is 'expand logical drive' will appear, come back to click on that also after finishing expanding array

4. make Linux kernel to recognize the new size of hardware raid5

reboot Linux server until the 'fdisk -l /dev/cciss/c0d0' shows the new size.
You can try 'partprobe' or 'sfdisk -R /dev/cciss/c0d0' first.

5. enlarge partition with fdisk

fdisk /dev/cciss/c0d0  (you might conside to use fdisk -u /dev/cciss/c0d0 to use sector instead of cyclinder)
p
press d then 2 to remove partition
press n then primary partition to use the full space
Make sure the old and new partition start at the same cylinder or sector position, otherwise, data will be destroyed.
press t to change partition type to LVM
w
q

note: you need to reboot again

6. resize physical volume size, logical volume size and file system online increase

after reboot, check again the new size

[root@log ~]# fdisk -l /dev/cciss/c0d0


Disk /dev/cciss/c0d0: 1799.7 GB, 1799797127168 bytes
255 heads, 63 sectors/track, 218812 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

           Device Boot      Start         End      Blocks   Id  System
/dev/cciss/c0d0p1   *           1          13      104391   83  Linux
/dev/cciss/c0d0p2              14      218813  1757509959+  8e  Linux LVM


[root@log ~]# pvresize /dev/cciss/c0d0p2
  Physical volume "/dev/cciss/c0d0p2" changed
  1 physical volume(s) resized / 0 physical volume(s) not resized
 
  note: use vgdisplay -v to check the number of free PE, let's say it's 17879.
 
[root@log ~]# lvextend -l +17879 /dev/VolGroup00/LogVol02

  Extending logical volume LogVol02 to 1.60 TB
  Logical volume LogVol02 successfully resized

[root@log ~]# resize2fs /dev/VolGroup00/LogVol02

resize2fs 1.39 (29-May-2006)
Filesystem at /dev/VolGroup00/LogVol02 is mounted on /data; on-line resizing required
Performing an on-line resize of /dev/VolGroup00/LogVol02 to 430276608 (4k) blocks.
The filesystem on /dev/VolGroup00/LogVol02 is now 430276608 blocks long.