Upgrading a version of Xubuntu (16.x to 17.x)

Well I had hoped to keep my mediapc running 16.04 with Qgis 313ec55 but it tells me I should update to a new version so I have followed the procedure to update to 16.10 and then 17.04. Unfortunately it automatically uninstalled Qgis as part of this and can't reinstall from the Xenial repository for 16.10 so I have let it go off this machine. I have decided that mediapc may as well go to artful like the other computers as this is fairly stable. It is being updated quite a lot, there were a lot of new packages for the bedroom computer when I checked last night. 

Fortunately a virtual machine running 16.04 and 313ec55 on my mainpc has proved to be quite usable for nearly everything in Qgis and can be set to have two virtual displays making it relatively easy to keep editing maps on older versions of Qgis as will be necessary to continue with the project work and having the advantage of being on the mainpc which is the best ergonomically with the placement of screens in my desk layout.

I have had quite a few virtualbox VMs running all sorts of different Qgis versions because of various issues with different editions of the software but I can't go back to 2.18 or earlier from 2.99 because of changes in the project format which would see a lot of stuff that would have to be put into the 2.18 project that would be a lot of work I would rather not have to do.

However because of some issues in this combination I still need a VM running 2.18 to run the composers for outputting pages of maps so that one will be used as well. The project version issues don't affect the composers so it continues to work quite well.

Virtualisation and Qgis

UPDATE: The below comment does not make a lot of sense as I have since used Virtualbox VMs very extensively to run Qgis and edit maps. Right now I don't have need to do this, but in the past it has certainly been a major part of the Qgis work that I have done.

(Unfortunately after further testing the use of VMs, at least under VirtualBox, to run different Qgis versions, has been found to be unsatisfactory. It is easy to see from a technical viewpoint how this would be possible in an environment where video, keyboard and mouse environments are emulated, rather than native. 

The solution that has been arrived at for now is to use the VM to edit project properties like the rule based styles in an older version of Qgis, while continuing to use the development version for everything else. However should the development version have too many issues to make this work I would be forced to revert to the current release version, since in Linux it appears to be much more difficult to have multiple versions on the same computer simultaneously)

---

One of the perils of running a beta version of software is that the bugs may make it unusable. This has happened from time to time with Qgis but it has never been as much of a problem as it is with the current version 2.17-master. A bug has been found that freezes Qgis when attempting to edit rule-based styles in data layers. Since all of my layers use rule-based styles (there is a "type" field in each record which is used in all the rules) obviously I have a problem with this version.

For now I have set up two VMs to use the current release version of Qgis. One of these is running the same version of Xubuntu as the host and the other is running Windows 10. Since the host now has 24 GB of RAM (3/4 of the maximum of 32 GB the board supports) it is easily possible to run one or even both of these VMs at the same time. I won't actually need to do this but I did run the Xubuntu one and set up the Win10 one at the same time.

Here you can see the Xubuntu VM running Qgis 2.16. The Maps folder on the host has been shared and after a fashion I was able to work out how to permanently mount this to a Maps folder in the home folder of the user on the VM. So when I save stuff on the VM, it goes back into the right folder on the host (using network shares as the shared folder capability in VirtualBox ran into permissions problems).

And here we have Qgis running on the Windows 10 VM. Again using a mapped network drive, this time drive M mapped to the share. As you can see a key difference is that the Windows 10 guest additions don't support full automatic resizing of the VM's display, which becomes a window centred inside the VirtualBox window. 

So at least I can continue working with Qgis (probably preferring the Xubuntu VM, but may need the Win10 one to run IrfanView when needed) until the bug in 2.17 is addressed.

Both these VMs are using the host's folders to access the map content. This also means the overGrive app running on the host will synchronise changes to the NZ Rail Maps Google Drive.

The third VirtualBox VM I have running on this machine is a Windows 7 one. One of the reasons for having it is to give me access to a different Google Drive - this being the one linked to my main email account - which in this case is used for a whole lot of personal stuff. So for example my budget spreadsheet and all of my study materials are on that one. There is also software I need for those purposes which is not available on Linux.

I also had a play with VirtualBox on one of my work computers which has a Wolfdale Celeron E3300 dual core and only 4 GB of RAM. This particular CPU was specially selected at the time of purchase because it supports VT-x (and 64 bit) so it just manages to meet the requirement for hardware assisted virtualisation. The computer is 6 years old and long since ceased to be of any use at home because the 4 GB memory limitation, driven by having only 2 memory slots and unavailability of DIMMs with more than 2 GB on them, means it runs out of steam pretty quickly. Still, with Xubuntu on it which has lower memory footprint than a lot of other Ubuntu based distros, it has managed to set up and run a Windows 10 VM in 1 GB of memory. This has been pretty slow work, but it looks like it should be able to achieve what is needed - and it's possible I might be able to increase its memory allocation if it can run some of the tasks I currently put on my Xubuntu desktop. It worked surprisingly well with just 1 GB allocated and the host also worked very well with a number of other resource-intensive applications such as Google Earth and Thunderbird running at the same time. It so happens that if I put the VM up to full screen then it is able to adjust the resolution to 1920x1080, but on my home computer where 1680x1050 is the screen resolution, that VM can't be made to go up to anything higher than 1152x864, so it would seem the Vbox Guest Additions have limited adaptability to different host screen sizes.

Subsequent to this I discovered issues in the Xubuntu VM that were peculiar to it, that were not seen in the Win10 VM, so I switched to the Win10 VM and then investigated whether the screen resolution could be bumped up. It turns out this is possible with the VBoxManage command on the host. Specifically the command VBoxManage setextradata "VM Name" CustomVideoMode1 1680x1050x32 did the trick, and after restarting the VM, the new fullscreen resolution appeared on the display settings. VBoxManage is basically the command line tool for changing all the VM's settings, although it obviously appears to be a superset of what is available in the GUI. 

Today's Linux Experience: Virtualisation Tools

One of the great things about the major desktop OSs (Windows, OSX and Linux) is that they all have software based hypervisors available to let you run other operating systems as virtual instances in a window on your desktop. When I was getting into PC computing in a big way, the only option we had for running multiple OSs on the x86 platform was by setting up a PC to dual boot. This is a messy system that is hard to maintain and can cause some problems with compatibility of different OSs with a boot manager. At the Polytech there would be dedicated suites for teaching a particular class that had the specific operating systems needed for that class installed on the PCs in that classroom and the class could only be taught in that classroom. But then along came viable virtualisation platforms. The Polytech became a Microsoft school, and I was told they used two VMWare servers to run all of their instructional virtual servers on for all of the different classes. When we went to do a class, we would just grab a pre-configured virtual machine instance and copy it to our local workstation's hard drive and then we could use it in any classroom on the campus or even at home with the free VMWare Player.

The basic types of virtualisation are running in a bare metal hypervisor, like a Hyper-V server, in which everything else on the server consists of instances of different virtual machines, or software based hypervisors which run in a window on a desktop PC. Even the Mac platform can be virtualised these days, although the hypervisor can only run on Mac hardware. Virtualisation is one possible option for getting around the limitations of your favourite software package not being available for the host OS of your computer. On the x86 platform we have more choice than on OSX, so I am not going to look at the latter at all here. The options for Windows have changed from Virtual PC (last produced for Windows 7, which used it to provide the "XP Mode" VM) into a specific Hyper-V platform on higher spec editions of the desktop OS. Unlike earlier instances of VPC, Hyper-V requires specific hardware support, e.g. VT-x or AMD-V implemented in CPU and motherboard. Hyper-V obviously isn't an option for Linux, but here we can draw on two other software products that are also available for Windows. These are VMWare and VirtualBox.  

I have tested both VMWare and VirtualBox on my Linux computers. VMWare is a commercial product that has a free "Player" edition available for various platforms. Due to competition the so-called player has been enhanced to include VM creation capabilities. Installing it onto Linux can be tricky as at times this will require additional modules to be compiled into the kernel, which means you need to have the appropriate minimum version of GCC and the source header files installed as well. The other issue I found was that after using it for a while it would refuse to start up at all. I can't recall the error message but it was not easy to work out a resolution so I gave up on the product at this point. VMWare requires hardware virtualisation support in your computer like Hyper-V Platform and the first edition of Virtual PC for Windows 7 (previous VPC editions did not require this, and the Windows 7 version was later altered to remove this minimum requirement). I noted this when I tested it on one of my AMD E350 systems and unexpectedly got the message that there was no hardware support for virtualisation. At this stage since neither of those systems is now running any Linux edition (both having been pressed into use as backup Windows computers) I have no idea if a Bios upgrade from Gigabyte fixes the problem since the CPU supposedly should supporting AMD-V.

So now I am using VirtualBox due to the tricky to resolve VMWare error messages. Unlike VMWare, VirtualBox is completely free. In fact, it is FOSS, and it can run on many hardware platforms, including OSX, and can also run many guest OSs, also including OSX (on Apple hosts only). Hardware virtualisation support is not always required, however Windows 8 and later and some other OSs as guest mandate it as do 64 bit guests. Multiple monitors are supported. One disappointment has been USB support. I had hoped to have a VM able to run my camera's download software but the VirtualBox manager was unable to detect the camera when plugged in, although I have since learned that there is an extension package that may be needed. So that is where virtualisation has let me down, but maybe this USB thing will work better in future editions of VirtualBox. Other than that I have found it to work very well with a Windows 7 guest and a Lubuntu guest tested out so far. 

Linux users also have additional bare metal hypervisor options available but that is outside the scope of this posting.

How Many Mints?

In my household I have four computers for various purposes plus another one out the back. I have been thinking a lot about which computers will run which OSs. The main issue is I still need a Windows 7 system for some stuff that is easier to do on a physical computer than by running something in a Wine layer or virtual machine. But there only needs to be one single computer in the whole house that does this. I also need a Windows 10 system for testing some stuff out but not for day to day use and that is the one out the back, but getting the right balance for the house computers has been tricky.

The main killer apps are things like IrfanView and Office plus the backup system but that isn't to say it would be impossible to run some of that stuff in a virtual machine and store the files on a virtual hard disk necessarily so it could be feasible to run things in a Windows VM hosted on a Linux desktop, or the photos could be on a Linux host system served by Samba. The key issue is I want to be able to do most of my stuff in one location and the second location essentially be no more than a backup. Perhaps it is a kind of server or live backup. I am sort of working towards a future understanding that living in a smaller premises than the previous one guarantees that that fourth computer could be using up valuable space where it is. However it is the newest of my computers and it won't be let go of in a hurry. Running it on Linux and having it as some kind of storage location or server seems to be the best use of it right now.

I have had a lot of fun with VMWare on Linux of late with silly error messages stopping it from running which is why I am taking a serious look at VirtualBox as of now. VB won't run on the AMD E350 computers that I have apparently because the Gigabyte Mini-ITX motherboards in these PCs don't support it. Although I was sure I had had VMWare Player running under Lubuntu on one of these systems, I am not so confident now but I seem to recall that was the case because it was quite apparent that the system struggled with low resources to run a virtual machine. Since I have got VB ready to set up a Win7 VM on my main computer then I will see if it works better in this system than VMware, which tended to freeze the system. If all else fails I have got a Windows 7 system next to the main PC that can be used as an actual physical computer to do some stuff. I also need to rearrange the networking here so that the computers are all running across gigabit as at the moment some of them are connected through a fast ethernet switch so they don't all have the full speed capability.

The spare PC cum server is going to be set up as a RAID-1 system just like my main PC which means I will be applying my newfound skills with MDADM having completed the commissioning of the RAID-1 array on the latter in the last week. In this case the data to be migrated has been backed up through the Cobian Backup software I used on the computer when it was running Windows, and it will be migrated back to the array once it has been set up. Due to finding that it is not possible to connect a camera via USB to the VirtualBox VM on the main PC, the third computer in the living room (one of the AMD E350s) is being reinstalled to Windows. This turned out to be surprisingly difficult to do with the installation of Windows 7 SP1 being unable to install updates, or Windows 7 without SP1 able to install updates fine until SP1 was installed, when the VM using the same ISO image had no such difficulties. Being unable to find any solution to the problem, including flashing a BIOS update, I have had to install Windows 8.1 instead. It is very difficult to understand what the problem could be as this computer has had Windows 7 on it before. However with 7 now out of mainstream support it is going to be more and more difficult to resolve problems with it and eventually MS will force everyone onto 10 as the only supported version.