Booting Windows 2016 on HP G8 Microserver MicroSD Card

As good as FreeNAS has been, most of the clients on my home network are Windows based and speak CIFS/SMB,  and I've not had great success with FreeNAS reliably/stably serving these protocols.   Under load, the shares sometimes lock up and stop responding, and permissions can be a bit hit and miss.

FreeNAS support forums drink their own special brand of cool aid, so I've decided to try Windows, which, whilst being part of their own borg collective has a much wider base of users and obviously native integration with my client base.  So I'm piloting Windows Server 2016 with its various storage capabilities to see how it compares.

I've got a HP Microserver G8 which as well as 4 disk trays, supports a fifth SATA device via an additional ODD port, an internal USB and a MicroSD port, as well as various external USBs.

My FreeNAS is a previous N54L Microserver, which installs and boots easily to a USB drive, but Windows is a bit more pig-headed at booting from USB/MicroSD devices.

However, with the help of Daniels Tech Blog https://www.danielstechblog.info/how-to-deploy-windows-server-2016-tp3-onto-an-sd-card/  I have managed to get my Microserver booting from the MicroSD Card

Daniels instructions are more or less spot on, except for one change.  

diskpart
list disk
select disk x
clean
create partition primary
format quick fs=ntfs label="SD"
active
assign letter=C
exit
dism /Apply-Image /ImageFile:D:\sources\install.wim /index:2 /ApplyDir:C:
bootsect /nt60 C: /force /mbr
bcdboot C:\Windows

I couldn't get that final line to write to the MicroSD. I kept getting errors about BCDBOOT not being able to write the files, or unable to find the source location. However, I read the documentation about BCDBOOT at Microsofts MSDN site https://msdn.microsoft.com/en-gb/windows/hardware/commercialize/manufacture/desktop/bcdboot-command-line-options-techref-di and happened upon the command for writing to USB devices.

bcdboot C:\Windows /s C: /f ALL

This seems to work fine, and a reboot allows Windows 2016 to boot.

Moving Wordpress to a subsite in Azure Websites

Deploying Wordpress in Azure is relatively straightforward.  Deploy a Resource Group, with a DB, and a Web Server using the Wordpress template and bob is your fathers brother.

However, I wanted to move WP to a subdomain which all in all should be pretty straight forward. All the instructions online are out of date or for different systems though, so I wanted to note what I had to set to get it all working.

General Steps are:

1) Deploy your Wordpress site.

2) Go into the WP site and perform the configuration to get it running bare-bones

3) Go back into the Azure Portal and find the FTP configuration

4) Log into the Website with the FTP credentials (I used WinSCP)

5) Copy the entire Wordpress site to your local machine

6) Delete everything from your site except for
              index.php
              azuredeploy.json
              web.config
I had a few problems deleting the wp-admin and wp-content folders - it kept blocking access, but went eventually.

7) Create the name of the subfolder you want to use (in my case /d/)

8) Copy the backup of your Wordpress into the subfolder.

9) Edit the root web.config file.  Mine is setup to redirect all traffic to the /d/ folder.  You many a different setup, but this is mine:
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<system.webServer>
    <rewrite>
        <rules>
            <rule name="Root Hit Redirect" stopProcessing="true">
                <match url="^$" />
                <action type="Redirect" url="/d/" />
            </rule>
        </rules>
    </rewrite>
</system.webServer>
</configuration>

 10) Edit the root index.php file and change the require
require( dirname( __FILE__ ) . 'wp-blog-header.php' );
to include the sub folder you want to use.
require( dirname( __FILE__ ) . '/d/wp-blog-header.php' );

11) Azure Wordpress seems to ignore the SQL database setting for the site and home name.  Therefore this must be hardcoded in the wp-config.php file.
Replace the existing paths BELOW the Stop Editing Line! as follows.

define('WP_HOME','http://example.site/d');
define('WP_SITEURL','http://example.site/d');
define('WP_CONTENT_URL', 'http://example.site/d/wp-content');

I also had to define the WP_CONTENT_URL string as my Salient theme wasn't picking up the subpath for some reason, so I hard coded it.

12) The only thing you may have to do is edit the subfolders web.config file.   I was having loads of errors, until I deleted it, then it magically started working.  However, it does seem to have been replaced with an empty web.config file, so its contents are here:
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
  <system.webServer>
    <rewrite>
      <rules/>
    </rewrite>
  </system.webServer>

</configuration>

I think that was about it, but it was quite the pain to get it configured.  Nb, if you change the website name or add a custom domain, don't forget you'll have to change the wp-config.php with the new domain name as Azure won't do this for you automatically.

I'm also not sure how things will go when there is a new version available, but hopefully this will get you going.

HP EliteDesk 800 G1 - DisplayPort to HDMI Audio

I've been fighting for several days to get the audio working from a HP EliteDesk 800 G1 SFF device into my Panasonic TV using a DisplayPort to HDMI Cable.  The PC is running Windows 10 build 10586.   I tried lots of things, including BIOS patches, various driver builds from MS and Intel, but all to no avail.

However, I have managed to get it working, by changing the Default Format for output to FM Radio Quality, down from DVD or CD in the Advanced TV Properties.   I'm not sure if this is a TV or PC/Driver problem, but I suspect its the latter as I had the same problem with another TV,  but the cable works on a Surface Pro 3 Mini-DP to DP to HDMI adapter, so I think this is PC related.

Hopefully this helps someone else out.

Saving Images to AWS S3 Scriptomagically

Whilst I've been messing around creating boot images, I've hit against the problem of needing to archive off some large images for later use.  Now I've finally got access to a high-speed bandwidth internet link, I'm can back stuff off to Amazon's AWS S3 cloud in a reasonably timely fashion.

s3cmd does a great job of interfacing with AWS from a Linux CLI, but is designed to deal with precreated files, not stuff that is dynamically made.    When you're talking about multi-gigabyte files, it isn't always an option to make a local archive file before pushing it to the remote storage location.

I'm used to using pigz, dd and ssh to copy files like this, and wanted to achieve something similar to s3, however there doesn't seem to be many guides to achieving this.  I have however made it work on my debian based distro relatively easily.

Tooling

This is the tooling I combined

s3cmd

You need a recent version of s3cmd to make this work - v1.5.5 or above is apparently what supports stdin/stdout which you'll need.
As of writing, this can be obtained from the s3tools git repository @ https://github.com/s3tools/s3cmd
You'll need git and some python bits and pieces but building was straightforward in my case.

Before you start, make sure you setup s3cmd using the command s3cmd --configure

pigz

I use pigz, although you can use gzip to achieve the same thing.  For those that don't know, Pigz is a multi-threaded version of gzip - it offers much better performance than gzip on modern systems.

tar

tar is pretty much on every linux system, and helps deal with folder contents in a way that gzip/pigz can't.

Usage

The command I built is as follows:

tar cvf - --use-compress-prog=pigz /tmp/source/directory/* --recursion --exclude='*.vswp' --exclude='*.log' --exclude='*.hlog' | /path/to/s3cmd put - s3://bucket.location/BackupFolder/BackupFile.tar.gz --verbose -rr

I think its pretty self explaintory, but I'll run through the code anyway...

tar cvf = tar compress verbose next option is a file
- = stands for stdout in tar parlance 
--use-compress-prog=pigz = self explaintory, but you can probably swap this for any compression app which supports stdout. 
/tmp/source/directory/* = the directory or mount point where your source files are coming from
--recursion = recurse through the directorys to pickup all the files
--exclude='*.vswp' --exclude='*.log' --exclude='*.hlog' = exclude various file types (in this instance, I was backing up a broken VMFS storage array 
| = breaks the input to the next app
/path/to/s3cmd = the directory where s3cmd resides - in my instance, Id installed git repository version
put = send to s3; put works with a single file name.
- = use stdout as the source  s3://bucket.location/BackupFolder/BackupFile.tar.gz = the s3 bucket and path where you want the output stored
--verbose = debugging verbose output and status tracking
-rr = reduced redundancy storage - less expensive than full redundancy, and you can include/exclude this based on your needs.

 The biggest problem with this is you don't really get an idea of how long a backup will take. s3cmd splits the file into chunks, but you don't know how many chunks it is until the process has completed.  I average around 6 MB/s but a multi-gigabyte file can still take several hours to upload.  Whilst I didn't time it exactly,  a 70GB file, compressed to 10GB, took around 90mins to send to s3.
You may want to leave your backup running in a screen session.

Azure Self-Signed Cert

I've been messing around with some of the Azure services, and am about to try some of the desktop utilities such as the Hyper-V converter to try publishing services.  One thing you'll come across if you're trying to do this is the need to create a self-signed management certificate to allow these apps to authenticate, and you'll see all the technet articles mention the makecert.exe tool.

The problem is that makecert is bundled into the Windows 8.1 SDK and Visual Studio 2013 Express downloads, both of which are several hundred megabytes - overkill for what we need.

Well, the easy way to get makecert and only use about 9mb of storage space - download the 8.1 SDK installer from http://go.microsoft.com/fwlink/p/?linkid=84091  and run the installer.  When prompted to select the tools, you only need to install the MSI Tools.  This gives you makecert (as well as a few other bits and bobs) but in a much more compact format than having all of the developer tools installed.

Wierd goings on with a Live CD and libdvdcss.so.2

I've been messing around trying to make a live-CD with some transcoding/ripping utilities built in to utilize some of the spare hardware I've got lying around. More on this later, but I've been reworking the guide @ http://willhaley.com/blog/create-a-custom-debian-live-environment/ with my own utilities and tools.

One problem I've been challenged with over the last couple of days is HandBrake-Cli bombing out with the message:

root@av3-build2108141:/mnt/Videos/Movies/dvdrip/91# HandBrakeCLI -i BHD.iso -o BHD.mkv --preset="High Profile"
[20:41:41] hb_init: starting libhb thread
HandBrake 0.9.9 (2014070200) - Linux x86_64 - http://handbrake.fr
4 CPUs detected
Opening BHD.iso...
[20:41:41] hb_scan: path=BHD.iso, title_index=1
index_parse.c:191: indx_parse(): error opening BHD.iso/BDMV/index.bdmv
index_parse.c:191: indx_parse(): error opening BHD.iso/BDMV/BACKUP/index.bdmv
bluray.c:2341: nav_get_title_list(BHD.iso) failed
[20:41:42] bd: not a bd - trying as a stream/file instead
libdvdnav: Using dvdnav version 4.1.3
libdvdread: Missing symbols in libdvdcss.so.2, this shouldn't happen !
libdvdread: Using libdvdcss version  for DVD access
Segmentation fault

This has been bugging me, as it worked before I converted the image to a livecd.  I wondered if it was some kind of problem with the lack of 'real' disk space, or a lack of memory or something like that, but nothing I could find would identify it.

Finally, I started looking into libdvdcss rather than HandBrake itself.  I think what confused me is the symbols error looks like a warning, especially given that there is a follow-on message which looks like libdvdcss is continuing.  Anyway, eventually!   I ran an md5sum on the libdvdcss.so.2 file to see if it matched a non-live machine (to a virtually identical build).

root@av3-build2108141:/# md5sum /usr/lib/x86_64-linux-gnu/libdvdcss.so.2
4702028ab20843fd5cb1e9ca4b720a72  /usr/lib/x86_64-linux-gnu/libdvdcss.so.2

N.b. libdvdcss.so.2 is symlinked to libdvdcss.so.2.1.0 in my current Debian sid based build.

On the donor machine
root@mediasvr:/usr/lib# md5sum x86_64-linux-gnu/libdvdcss.so.2
c9b314d9ed2688223c427bc5e5a39e6f  x86_64-linux-gnu/libdvdcss.so.2

So I've SCPd the source file into the live machine, checked the md5sum matched the donor machine (it did), and repeated the HandBrake job.  Lo and behold it worked!  So I've restreamed the two files into the filesystem and success, it just works.

So I don't know if something funky happens when the image is created using the link, but actually its quite easy to fix once you understand the problem.

Hope this helps someone,  and I'll be back soon with more details about building a live image, then booting it using iPXE.

iPXE Booting OpenElec

Open Embedded Linux Entertainment Center (OpenELEC) is a small Linux distribution built from scratch as a platform to turn your computer into an XBMC media center. OpenELEC is designed to make your system boot fast, and the install is so easy that anyone can turn a blank PC into a media machine in less than 15 minutes.

This is a great live image for getting up and running with XBMC, or testing it before committing to installing to a harddisk.   I've set it up today to boot from the network to see how well it works on a machine I'm thinking about using for a media centre.  It was a bit of a pain to get it working,  but now that it is,  it works fine.

First of all, download a copy of OpenElec from http://www.openelec.tv/get-openelec/download - I got a the tarballed version entitled OpenELEC-Generic.x86_64-devel-20131026131436-r16293 from the developer sources, but I think stable versions will equally well.

This was copied to my NAS server, and untarred using the command.
 tar -xvf OpenELEC-Generic.x86_64-devel-20131026131436-r16293.tar
This then spat out what I presume to be an OpenElec live-cd or some such (but who cares - we don't do CD's do we? :)  ).    Within the created folder, there is a 'target' folder, which contains the images you need to boot from.  

Make sure the target folder is in a location where it is accessible from both HTTP and NFS.  Note,  I've not been able to make this boot using HTTP, and I'm not sure its possible, because it seems to use NFS as a persistent storage location for your configuration.

Next, create a folder for storing your persistent information (I created a folder called persistent within my target folder.

Now update your iPXE menu.

:OpenElec
echo Booting OpenElec Media Centre
echo HTTP and NAS Method 
kernel http://boot.server/openelec/OpenELEC-Generic.x86_64-devel-20131026131436-r16293/target/KERNEL boot=NFS=10.222.222.50:/boot.server/openelec/OpenELEC-Generic.x86_64-devel-20131026131436-r16293/target/ disk=NFS=10.222.222.50:/boot.server/openelec/persistent/ netboot=nfs ssh ip=dhcp
boot 

So this loads the kernel using http from the server, and passes the boot partition nfs and persistent nfs location.  Note, neither of the latter two define the files,  just the folder paths.  The Kernel knows what its looking for when it boots.
The final variables tell the kernel that it is being booted with nfs required,   to enable ssh (if you want it) and to get the IP using DHCP.    There are a number of other modes for debugging, text only mode, that sort of thing, but that is not discussed here.

Anyway,  other than configuring the iPXE menu to call :OpenElec,  that's all there is too it.