Sunday 13 January 2013

iPXE CloneZilla

CloneZilla is a linux toolset that allows you to clone either a partition or whole disk to another location;  either a connected storage device,  or remotely to the network.   This is a great tool for imaging systems before you work on them and lets you have a copy in case the worst should happen.    It has a variety of bundled tools in order to read from most of the popular filesystems in use,  falling back to DD to copy each disk sector if you're using some obscure or proprietorial filer.   This is the FOSS alternative to Norton Ghost!

The great thing about CloneZilla is that its quick and easy to get it booting via iPXE,  so is worth investing a small amount of time in setting up so that you have it ready to go should you need it.

These instructions are based on release clonezilla-live-20121217-quantal.iso which seems to be versioned 2.0.1-15.  

Download the ISO from the CloneZilla site.  Use 7zip or your favourite image opening tool to open the ISO.  You need to extract the following files:

  • vmlinuz
  • initrd.img
  • filesystem.squashfs

and put them onto your boot webserver.  In this example,  I have created a folder called CloneZilla.


############ CloneZilla ############
:Clonezilla
echo Starting CloneZilla with default options
kernel http://boot.server/CloneZilla/vmlinuz
initrd http://boot.server/CloneZilla/initrd.img

imgargs vmlinuz boot=live config noswap nolocales edd=on nomodeset ocs_live_run="ocs-live-general" ocs_live_extra_param="" ocs_live_keymap="" ocs_live_batch="no" ocs_daemonon="ssh" usercrypted=Kb/VNchPYhuf6 ocs_lang="" vga=788 nosplash noprompt fetch=http://boot.server/CloneZilla/filesystem.squashfs
boot || goto failed
goto start



And that is really about it! You'll notice we pass a few arguments which set various options. The most important option is the 'fetch=' command which tells the image where to download the main file system from. The other option I set was 'usercrypted=' which uses the Linux mkpasswd command to set the root password on boot - in this example iloveclonezilla.

A really easy one this week, but one worth trying. I'm fighting to get Backtrack5 booting over iPXE without using the ISO method, but this is proving troublesome. I think the image simply isn't able to cope with being booted from a http network source.

Thursday 3 January 2013

SysRescueCD v3.1.2

So to kick off, we'll start with booting SysRescueCD from http://www.sysresccd.org/.

From Wikipedia:
SystemRescueCd is an operating system for the x86 computer platform, though the primary purpose of SystemRescueCD is to repair unbootable or otherwise damaged computer systems after a system crash. SystemRescueCD is not intended to be used as a permanent operating system. It runs from a Live CD or a USB flash drive. It was designed by a team led by François Dupoux, and is based on the Gentoo Linux distribution.

For this activity, I used the download versioned v3.1.2 which I got from http://goo.gl/F36zV 

My issue was that the machine I was trying to boot from didn't seem to have enough memory to use the memdisk/iso boot option common for most installs, which meant I had to try and boot the ISOLINUX image.

The Software:
Open the ISO in your favorite ISO opening tool (I use 7zip).  
Extract the following files into your web boot server.  I used a sub-directory called SysRescueCD
  • sysrcd.dat
  • sysrcd.md5
  • ISOLINUX/rescue32 (or 64)
  • ISOLINUX/initram.igz
Note,  I assume rescue64 is the 64 bit version of the kernel, and rescue32 is the 32bit kernel.  There is also altkrn32 and altkrn64 which are referenced in ISOLINUX as alternative kernel builds.  They all seem to work.


The Webserver Config:
This is the menu display section of the config:
item SysRescueCD32 SysRescueCD - 32bit

And this is the execution program required to boot it.

############ SYSRESCUECD ############
:SysRescueCD32
echo Starting Sys RescueCD (32bit) with default options
initrd http://boot.server/SysRescueCD/initram.igz
chain http://boot.server/SysRescueCD/rescue32 cdroot docache dodhcp setkmap=uk netboot=http://boot.server/SysRescueCD/sysrcd.dat
boot || goto failed
goto start

Note, you can change setkmap= to your preferred keyboard mapping; I'm in the UK so that is the one I use.  If you leave this option unset,  it will prompt you when you boot the server.
If you change
rescue32 to rescue64 or one of the alternate kernel images,  the same commands seem to work.  There doesn't seem to be any difference in using netboot= or boothttp= to locate the main disk image.

Finally, I'm using a Thecus NAS as my boot webserver,  using FaJo's Apache Webserver module.  For some reason,  whilst the initrd and kernel load perfectly well,  the image refused to boot, freezing at 'null' in the download.  Another Apache webserver didn't exhibit the same condition, but its something to be aware of.  If I find the cause,  I'll update this post.


iPXE Booting - An Introduction

If you've arrived at this blog,  you likely already know what iPXE booting is,  but for those who don't,  I want to give a brief introduction to what iPXE is and how useful it is;  both for IT engineers and computer hobbyists.

So, What is it?

Quite simply,  its an open-source bootloader, primarily for the x86/amd64 machine architecture.  Its aim is to allow you to boot software and images across a network connection, using common network protocols such as tftp, http, ftp, iSCSI and AoE.

iPXE can be called over the network, started from bootable media like USB sticks or CD-Roms and even injected into the ROM of network cards where available.  This makes it an ideal tool for IT 'fixers' who often need to load a variety of images rather than just the standard Windows install on a machine.


A Brief History of PXE Booting.

PXE (Or pixie) booting is nothing new - in fact, pretty much all of the x86 type computers I have come across built in the last 10 years allow PXE booting from the network card, although sometimes the option needs to be turned on in the BIOS or the NIC.  Thin Clients,  VoIP phones and other 'low touch' devices also often support downloading configurations, software patches and other such updates over the network from a central source.   But PXE booting is not without its problems.  Its not setup to allow user interaction, meaning whatever the DHCP config is set to delivered is delivered.  Secondly, it uses tftp to transfer files - perfectly adequate for small text based config files,  but SLLLOOOOOWWWW when sending much over a few megabytes, even on fast lan connections.  Finally, PXE is only available with the functionality built into the firmware, so if it doesn't do what you need it to,  you're pretty much stuck with what you've got.

iPXE has had a bit of a convoluted development path in its reasonably short life.  It began life as 'Etherboot', which started in around about 1995.  Etherboot evolved into gPXE (Nothing to do with Google) and then after a bit of a falling out within the gPXE team,  gPXE was forked by some members of that team into the iPXE we have today.  Note,  iPXE doesn't have anything to do with Apple and their iDevices.   Both gPXE and iPXE continue to this day, doing pretty much the same thing.  However,  from my experience,  gPXE is better documented,  whilst iPXE receives more regular development activity.  And development does not happen in parallel, so you may want to check out both in case one has a feature set the other doesn't.

A Google Tech-Talk about gPXE and Network Booting.

I've moved from gPXE to iPXE in 2012, mainly because gPXE wouldn't support a NIC on a Dell machine I was trying to boot,  and had been considering making the switch at some point anyway.  iPXE resolved the issue for me,  so its now become a bit like Doc Martins were in the 80s and 90s - The boots of choice.

But why use iPXE Booting?

For me, it was a means to an end - I needed to test a motherboard and RAM combo, but didn't have a spare hard-disk to boot from.  I also didn't have any blank CD's to burn a live-cd image to in order to boot from a CD drive.  Finally, I didn't have any large (>64mb) USB thumb drives that I could write an ISO to, so I was kind of out of options.  Then I wondered if there was a way of using the small stick to chain boot something from the network and well,  it turns out there was.

I've stuck with network booting because once you get it working, your life as an engineer becomes much easier.  No more messing around burning CD's, or digging through CD wallets trying to find the version of the particular utility you need.

Now I can connect up a network cable, boot from the network, choose the ISO, utility, or test tool you require from a menu,  and stand back and watch it boot from my NAS filer.  This is often quicker than waiting for a silver platter of plastic and metal spin up and be read using the power of laser light.

The process is a little complicated, but once you've set it up the booting piece is pretty much hands off;  the only maintenance is adding new images and updating the menu.

The way I've configured it is
  • PXE requests an IP Address from the DHCP Servers
  • IP Address is returned,  along with instructions on getting the iPXE 'software' from a TFTP server.
  • The PXE downloads iPXE from the network,  and chainloads into this.
  • iPXE re-requests an IP Address from the DHCP Servers, along with iPXE specific instructions - i.e. where to go next.
  • iPXE reads through this config and initiates a http connection to the web server containing the main iPXE config files and menu etc.
  • The end user/operator (me) chooses what to boot from the menu.
  • iPXE parses the settings supplied in the menu, downloads the necessary components (again, over http) and boots the image.
gPXE took me about a day to get working,  because of the need to install and configure a DHCP server with various options.  Because this is a home network, I had to create this new by learning ISC DHCP,  plus I took the opportunity to install ISC BIND as a DNS server - further learning required.  Note,  you don't *need* these for iPXE to work, but it makes life easier in the long run.   I may blog more about setting up iPXE at some point,  but there are plenty of documented guides on how to do it.   

And what is the point of this blog?


So, as I've already mentioned above,  iPXE doesn't have *that* great a set of documentation,  particularly in regard to configuring images to boot.  Most of these comes through gleaning facts from various sources,  putting them together through trial, error and a little bit of prior experiencing and seeing if it works.   Whilst I manage to get things to boot,  I think its useful to share solutions with the wider community to make things easier to use.  This is particularly true when trying to convert Linux live images into an iPXE boot set up, where you don't necessarily want to or can't load an entire ISO into memory.  Hopefully others will find this a useful resource,  and I'm going to try and use it as a library of knowledge on my own experiences of network booting.