General computing update - week 34, 2020.

 Since writing early July about about the need to replace the disks in mainpc I have finally progressed towards this stage by purchasing the first of two disks. As noted in the earlier post, this was prompted by an I/O failure of one of the disks that resulted in it dropping out of the RAID array. However I was eventually able to add it back into the array and it has worked properly ever since. The first disk has been added to the existing array which is currently a three disk mirror. As soon as mdadm has synced this disk completely, I can then remove one of the existing disks off the array and then take it out of the computer, as I need backup disks right now. As noted at previous article the disk array will be the same size as current. I am expecting to buy the second disk in about another month and install it in the same way so that when it is also installed, the old disks can be removed completely, and all without actually copying any files. Last time when I did an installation I manually copied the files, but that was mainly because I doubled the disks from 1 TB to 2 TB; this time I will be staying at 2 TB.

Another techical task I have been working on around the house is erecting a TV aerial on the outside of the house. This has meant installing a cable duct and two faceplates between the inside wall and the underside of the soffit to take the aerial cable. That was the hard part as running the cable through the duct was very simple as was putting the aerial itself up. The TV itself is wall mounted and the cable comes through directly behind it. As I am in a prime reception area, the aerial does not need actual line of sight to the transmitter – there is actually another house in the direct line, but the signal is so strong that there is enough signal to get a good picture on every channel.

A few weeks ago I had some problems with a new package-based version of Qgis that would crash every time I tried to run it, on more than one computer. I had to install a flatpak-based version of Qgis, although I could have also run it in a virtual machine. The project developers built a new release after several weeks and it started working again, as did the next edition. Due to the disruption, I have implemented an extra precaution of testing every new release on a VM before installing it onto a working computer.

Google has again heaped glory all over themselves by implementing an arbitrary and dictatorial change to the Youtube platform. As of last week, they stopped making it possible for Youtube to send out notification emails whenever a new video is uploaded or streamed onto a channel. This has left a lot of people, including me, very annoyed by the decision. It is similar to the nonsense of the new Blogger user interface that is badly broken and which has forced me to copy and paste all my posts from another editor like LibreOffice. As I am doing right now with this post. Contrast that with the approach taken by Wordpress.com with a new editor system introduced a few months back; it was pushed through, but they did a lot of consultation with their users. In this case, Google has just announced the changes, but if there are any issues, it’s a one way conversation filing a problem report, just like the way you cannot actually have a conversation with Facebook about bugs or problems on their platform most of the time.

PDF Editors For Linux

This post is brought to you by the new Blogger interface Google has forced on everyone! Which is dreadful! The only way I keep my sanity editing posts on Blogger is to create the post in a separate editor (LibreOffice Writer in this case) and then paste the completed article into Blogger. There is much angst in the Blogger users’ community over the broken functionality in the new interface and the fact it has been forced into use with what they believe to be insufficient testing. My experiences with it exemplify those experiences, but this is the only one of my regular blogs that is still hosted on Blogger, and the workarounds for me are sufficient until such time as they fix all the bugs.

Today’s post is about editing a PDF using free/open source software. The PDF format as we are generally aware has historically been an Adobe thing, and so has the main editor software package. Everyone knows and uses Adobe Reader on various platforms, but relatively few people use Acrobat, the expensive commercial package that can edit PDF documents. Hence, a few alternative solutions have been developed, and the abilities of these are improving all the time. Here are my takes on a few of them.

My particular requirement here is filling out a PDF form and inserting my signature. It’s fine to be able to fill out the form in Okular (KDE’s in house PDF viewer) but inserting a graphic is impossible. So I looked at some of these alternatives:

LibreOffice Draw is part of the LibreOffice suite and can read and edit PDFs as files made up of individual elements. In my brief examination of Draw, the main concern I had was that it would be able to output the document looking like the original after editing; it seemed to have difficulty converting all of the text to typefaces that would fit cleanly into the original format. Because of this, I have not explored Draw further for my particular requirement at this stage.

Inkscape is a well known graphics editor that has a lot of features and is one of a few favourite graphical editors I have installed on my computer. I haven’t looked very deeply into its capabilities because the major limitation I have observed so far is that it can only handle a single page PDF; there is no obvious way of working with multi page documents.

Most of the full editors that are available are paid only. PDFSam and MS Word 2019 are examples that are Windows only. I have no desire at all to spend money on any type of Windows computer, or even a virtual machine, just to run these solutions. PDFstudio is an alternative that is available on Linux. The Pro edition that is capable of PDF editing costs $129 to buy and is licensed for 2 computers. It would be interesting to evaluiate this product at some stage to see if it is worth purchasing in future. Master PDF Editor is another product I might evaluate, it just puts a watermark on each page but it might be possible to remove that with one of the free editors.

Scribus is a FOSS desktop publishing package that also can open PDF files. Version 1.5 which is currently a development edition and only supported on most distros as an AppImage. I found however it has the same issue as some other packages of being unable to render fonts in the previously filled out PDF form.

Ultimately for this particular situation, needing a quick and easy solution to create my PDF and get it useful for my requirement, I have used Gimp which will import each page as either a layer or a separate image according to a selection choice when opening the document. It imports the pages as graphics, but you can fill in a form in something like Okular, save it to a new document, and then inserting a signature as a graphic can be done in Gimp, then export each image to a new file and paste them into a new document and export it back to PDF. A complex process for just one form but it lets me send my document completely filled out complete with signature because Okular cannot do the insertion of a graphic into the appropriate place on a PDF. I think this Gimp solution will be the best short term but I will still be interested in evaluating other possibilities in future.

General update week 30, 2020

I wrote about a lot of fun getting a system to work with static IP addressing (it is running Debian 10 with LXQt). Since it was able to be made to statically configure its IP with no issues, I configured the other Debian10-LXQt system with a static configuration also. The second computer only has one network adapter, but the benefit is that since the network router and wireless broadband router are only switched on when needed, I do not need to wait for the network router to come online each time and be ready to supply DHCP addresses to the client computers when they start up. Everything is working well now.

At the present time the backups are all being done locally on each computer as previously referenced using rsync since I have yet to do further work to make an rsync daemon function as planned for trial for backups. Another option I could look at for backups is the BackInTime application, however I have been generally wary of using anything that does not do straight file by file backups to the destination but instead creates its own archive format. This is not such an issue with commercially supported proprietary software, but I am very insistent that I do not get locked into archive formats with open source software that may not be supported forever. Probably this is the reason why lots of people prefer just to use rsync.

mainpc had an issue a couple of weeks ago when some sort of timeout on one of the RAID array disks caused mdadm to push it out of the array. It was then coming on and off line seemingly randomly for a time, but then stabilised and stayed online. Smartctl testing showed there were no errors on it. After a week it was readded to the array which is now functioning at 2 disks again seemingly with no problems. However as these disks are 4 years old (last replaced 2016) I am currently planning to buy 2 new disks for the array as soon as I can, this will be at the rate of 1 disk per month with the old ones being available for backups although it would still be preferred to buy completely new backup disks. 

The disks in the array currently are WD Black and that is the proposed replacement at $250 each. SSD is considerably more expensive but by the time I next have to replace any of these disks the cost of SSD could well have come down and be more affordable. WD Black SSDs in the size needed are  very expensive at around $1000 regular pricing each (four times the HDD price) and are only M.2 format which is problematic since it is rare for a board to have more than one M.2 slot at least in the price range I have. I have checked and the H-97 and B250M-D3H boards do have one M.2 slot each but not two. There are adapters available but another option is simply to switch to another SSD brand such as Intel which is considerably cheaper although with lower performance, if the performance at least equals standard HDD it would be a big reduction in cost to around $400 each. As I think, in a few more years getting a pair of SSDs for the array will be within the same price range as the regular HDD. I am still keeping the lid on the amount of storage on this computer resisting the temptation to buy bigger disks as in spite of growing photo and general data storage, so far it has proved possible to manage existing storage sufficiently well to avoid running out of space.

Rsync Backup System [6]: Latest Progress / Using The Rsync Daemon

Because of issues with Rsync over ssh I am looking into using the Rsync daemon instead. The commentary on that is below. Until I can get that working properly I am backing up each computer individually using its own removable drive bay. This can be achieved by changing the ownership of the backup folder on the removable backup drive to backupuser and then  logging into a virtual terminal as backupuser and then running the backup under that user account. We use that account rather than root or our regular user account on the system being backed up in order to ensure the account has only read and execute permissions on the backed up computer so that it cannot in any way risk deleting files from the computer. This is very important because I have used the --delete flag on rsync to ensure files removed from the source are also removed from the backup. When doing an incremental backup that flag is important to ensure the disk does not end up filling up with removed files and running out of space. But it is also useful to have the safeguard that you can't accidentally copy files from the backup disk over the source files, especially with incremental where your backup disk is already full of files.

The simple means for an incremental backup that rsync supports is using an existing backup disk that has previously had full backup onto it. This enables rsync to compare the source and backup without any additional system to record file details that it can then work out if they have changed. In other words with each source file it compares to the existing backup then only backs up incremental changes. This is quite different from other backup solutions I have used that have a separate incremental backup disk.

To implement this system I will need at least 3 disks for each computer if I want to keep one full backup and two incrementals. In other words each disk will start off as a full backup, then two of them will be alternating incrementals, and a full backup from scratch will be done every few months and kept separately from the incrementals.

I would still have preferred separate incrementals, and in fact it is possible with two removable drive bays, one contains a full backup disk and one contains an incremental disk, then the incremental can be written to the incremental disk instead of the full backup. However the problem with this solution is that rsync cannot produce progressive incrementals since it is always comparing with the last full backup, that means the incrementals will not be a snapshot at a particular point since the last incremental, but will always be incremented from the date of the last full backup. It is also messy working with the extra paths to the different disks in the backup command, and therefore, more likely to risk mistakes like backing up to the wrong disk.

So in summary the best solution is using a separate disk for each backup, which will be an incremented full backup in the case of 2 disks and a separate full-only backup in the case of one disk.

At the moment I am down to 1 backup disk for each computer and so I need to buy some more disks along with the removable bay caddies and storage cases. I am also working on secure storage for the disks in the storage location which is away from the house. At the moment I have only been doing one backup about every 3 months because the RAID-1 array system in the computer is so reliable for day to day backup that I don't need backups too often. But I will attempt to schedule a monthly backup cycle in future.

Since I have been using Rsync as a backup solution, it has been very reliable, but there are occasional difficulties, and one I am having at the moment is the backup terminating with rsync error 12 partway through. I have done some fault finding but haven't been able to narrow down what is occurring that causes this termination but it is probably something to do with the ssh client in the source computer, which rsync connects to. This means I have to either try and find out what is happening with ssh on the source, or try the alternative, which is using the rsync daemon (service) on the source and connecting rsync to that from the backup server.

I'll be starting with this guide and applying the steps in it: https://linuxconfig.org/how-to-setup-the-rsync-daemon-on-linux
Also https://www.atlantic.net/vps-hosting/how-to-setup-rsync-daemon-linux-server/

Firstly as I am on Debian, rsyncd is already installed with rsync. Next step is to configure the /etc/rsyncd.conf file which I will do from this sample provided in that article:

uid = 1001

gid = 1001

pid file = /var/run/rsyncd.pid
lock file = /var/run/rsync.lock
log file = /var/log/rsync.log

[backupuser]
path = /home/patrick/
comment = backup patrick
read only = true
timeout = 300

There are some changes I am doing from their sample, the most notable being the uid and gid, which are set to the values for backupuser, which is the user that has permissions to do the backup on the source computer. Assuming rsyncd is run as root, this will ensure the permissions are correct. Also I am specifying the module name as backupuser, and it will automatically connect to the backup path for my home folder.

Start rsync with systemctl start rsync and set it for startup with systemctl enable rsync .

So next time I will list the result of testing with the daemon and whether it worked successfully.

Reinstall KDE system with LXQt [3]

This is the third article in a series about problems with KDE leading to a reinstallation with LXQt. Both running on top of Debian 10.

The computer concerned is running two NICs because it needs to access both networks that I have here. I have the main one based on a corporate wireless link from next door, which is filtered and throttled, and a second networtk based on a Skinny cellular broadband network modem, which is not filtered or throttled, although speed is moot since there are limitations inherent with 4G cellular data. The Skinny modem is connected to the local devices through a Ubiquiti AirRouter, which includes a DHCP server and network switch. It also provides the wireless network for phones or tablets. The capabilities of the AirRouter are much better than those built into the Huawei Skinny modem for internal networks and that is why I am using it.

There are some issues with LXQt mainly in panels but apart from that the reinstallation of software has gone well on the platform. One issue with this computer in general is that it needs more installed RAM and I am planning to address that in the next week or two, since I need it to be able to run a virtual machine at the same time as the range of stuff I normally use it for. One issue was reinstalling Gimp, I had difficulty in working out where Gimp keeps its config data as I wanted to copy over the config from my existing computer. It turns out that Gimp uses ~/.config/Gimp as a path, but apparently it also used a Flatpak specific path in ~/.var/ to store some data and there is considerable confusion over why it maintains these two paths especially as the user interface of the Preferences dialog doesn't actually point to the ~/.config path when it appears it should.

One of the interesting experiments this week has been to work with the mounting arms that I use to hold some displays. I have used Brateck wall mount arms with VESA adapters on them for this for some years. The key has been to modify these to allow fine control of the height so that when displays are stacked vertically, it is easy to put them close together. 

This photo shows how the attachment of the arm to the "wall" (in this case a vertical post attached to the desk) has been modified to allow its height to be adjustable. This is done with two right angle brackets which are bolted together using 80 mm length M6 gutter bolts, which gives theoretically an adjustment range of 80 mm.

I have to work around issues with the backup system from one of my computers at the moment. rynsc has not been able to connect over the network between the computers and running the backup locally on a computer seems to be the way to go at the moment.

Setting up sudo on Debian

Debian is a bit different from Ubuntu and that is the reason I like it. Debian is much more focused for technically advanced people and, for example, has hibernation enabled by default, which contrasts with Ubuntu that makes it difficult to use hibernation, and there are many other differences in Debian that I prefer.

One thing with Debian is that sudo is not set up by default for the first user account and it's necessary to undertake a few steps to set it up. sudo is installed so all you have to do is add your user account to the configuration. This also underlines that under Debian 10, /sbin is excluded from the standard user's path in a terminal window and many commands will not work unless they are prefixed with the full path, something I have not yet got to a full understanding of the rationale for. When you go into a virtual terminal, that limitation is removed and it's hard to know why the restrictions have been put into the terminal window running under the desktop environment, when Debian 9 did not have those issues.

Anyway the steps to set up sudo are pretty straightforward:

• Use the su command to enter superuser mode in a terminal shell
• Install sudo if not already installed: apt install sudo
• Run the adduser command to add your user account to the sudo group: /sbin/adduser <username> sudo
• Log out and then log back in again

And that's it.

Reinstall KDE system with LXQt [2]; KDE network and display configuration extremely difficult with numerous bugs

As related yesterday I found various issues with KDE on a computer and decided to reinstall Debian 10 with LXQt as the user interface. After completing the installation I attempted to use ConnMan, which is the network configuration manager that comes with LXQt on Debian, to do a manual configuration of the 1.x network adapter to set its IP address manually without using DHCP. However, my experience has repeated that found in another computer where I had used ConnMan in the past, where it has continued to send DHCP requests for this network adapter despite the manual setting and overridden the blank "gateway" setting specified in the manual settings, with the DHCP settings. In this case, the setting that was overridden was the DNS server; it was configuring the system to use the 1.x network's DNS server address obtained via DHCP instead of the 4.x network's DNS server obtained via DHCP for the 4.x network adapter. 

As I have noted the Lubuntu network configuration manager does not have this problem and is definitely the best out of all three. However as I am not sure at this stage of being able to install this on the computer, my next step was to uninstall ConnMan and set the network parameters in the /etc/network/interfaces file. After uninstalling ConnMan, dhclient could not complete any DHCP requests for any interfaces; it seems that dhclient is configured to use settings provided by ConnMan and would have to be reconfigured to work with it. 

The interfaces file entries ended up looking similar to this

auto enp0s31f6

allow-hotplug enp0s31f6

iface enp0s31f6 inet static

           address 192.168.4.173/24

           gateway 192.168.4.11

auto enx00500b60c1eb7

allow-hotplug enx00500b60c1eb7

iface enx00500b60c1eb7 inet static

           address 192.168.1.173/24

In addition, /etc/resolv.conf contains the entry

nameserver 192.168.4.11

After completing these entries the system was rebooted and further checking confirmed the settings had been applied. In addition it was confirmed there had been no further lease requests to any DHCP servers, the lease for the 1.x network adapter (the problematic one that was pushing through unwanted default gateway and DNS server settings) had expired and had not been renewed. 

It therefore appears I have resolved so far the display issues and also the network issues. The only remaining issue out of 3 mentioned in the original article to be resolved is to get x11vnc running. Then there are the separate matters of reinstalling the software that was on this computer before I reinstalled it.

LXQt has some obvious differences from KDE and one of those is the panel configuration is different and at present it will not support more than one panel on the desktop and will not support a panel being in the top position on the lower display, which in the configuration I have with my displays is the ideal place to have just one panel that is shared between both screens. Although a panel can be placed there and used normally, the panel cannot be right clicked and configured using any of the usual menu options for configuration because the menu is truncated. The only fix being to move the panel to the bottom using the panel.conf file, stop and restart the panel and then configure at the bottom then move back to the top when finished configuring. I will have to check with the LXQt project to see if they have any fixes for this problem.

The other key issue of course is fewer widgets available yet for LXQt. The main one I do not have is the disk free space. I can simulate the CPU monitor bars like I am using on KDE (they show CPU usage, memory usage and swap usage) with some of the built in widgets. So apart from some niggling issues related to the differences between the desktop environments I expect to proceed quickly to having this system up and running again fairly soon and to be able to have a more reliable and easier to use system than I was experiencing with KDE.

Reinstall KDE system with LXQt [1]; KDE network and display configuration extremely difficult with numerous bugs

KDE has some great reputation as a desktop environment in Linux and has won considerable plaudits. It however has numerous unresolved bugs which I have observed in the display and network configurations that are leading me to ditch KDE on one of my desktop computers in favour of LXQt. I have had three computers running KDE and one running LXQt all on top of Debian but I will be taking two computers onto LXQt and therefore only having two running KDE.

The concerning issues that are being observed with KDE mainly concern non standard configurations in both display and network, and also the inability to use x11vnc on KDE which somehow blocks it from running. The problem can be summed up as KDE not being sufficiently versatile to allow different configurations where the standard ones are insufficient. For example the VNC client designed for KDE, Krfb, will only buffer one display on a two display computer, making it inferior compared to x11vnc.

The summary of the issues experienced to date is:

• Networking. Unable to configure a static IP address and route for a network adapter and stop the automatic configuration of an adapter via DHCP. Unable to disable a network adapter and stop it from connecting as KDE ignores the disable setting and creates another new adapter with the same settings as the disabled one (the only option then is to disable or remove the adapter at hardware level).
• Display. Unable to save the configuration of two displays on my computer. The displays are stacked vertically. KDE is unable to remember these settings and after each reboot, defaults to side by side displays.
• Remote Frame Buffer. As mentioned above, KDE stops x11vnc from working. Krfb will only buffer one display in a dual display system.

My networking requirement is special as the computer has two network adapters due to being required to connect with two different networks. On LXQt this is relatively easy to set up. On KDE I have experienced the problems mentioned that it keeps ignoring the manual settings. Only one of the adapters can be a default gateway because there can only be one default route to the internet. I have made it clear that the manually configured adapter will not have a default gateway but KDE ignores the instruction to use the manual settings and keeps sending DHCP requests on that network. The net result is it has become impossible to enforce the required network configuration of the computer. 

To illustrate that this is not difficult on other desktop environments, I have set up a virtual machine on the same computer which is set up in VirtualBox settings with the two network adapters with the appropriate configuration in LXQt (the VM is running Lubuntu). This has worked flawlessly without all the dramas that KDE is creating. The network configuration on LXQt on the virtual machine is implemented as the option "Shared to other computers" which allows the specification of an IP address and netmask without gateway settings. In other words that adapter does not have a gateway specified and must not send DHCP requests on the network.

Another difference between LXQt and KDE with regard to networking is the list of configuration methods and their implementation. In the connection manager that is provided with Lubuntu, which is admittedly different from LXQt on Debian, "Shared to other computers" lets a manual IP address and netmask be specified. But this information can't be put in when selecting the same configuration method in KDE. The KDE network manager in this case automatically creates its own IP address on the 10.x.x.x network and you obviously have to configure the rest of your network to match.

The key problem with KDE is it takes over too many things in a computer and makes it difficult to work around its default settings. In LXQt it is extremely straightforward to configure a non standard dual display and save the settings in a config file, extremely straightforward to install and manage x11vnc and it can be more straightforward to configure a network adapter. However ConnMan is a bit more tricky to configure than the Lubuntu network manager - the Lubuntu one is one of the best ones I have used on Linux overall.

I have started a complete reinstall of Debian 10 on the computer despite my misgivings over having to reinstall all the software (including the tricky iscan software for the Epson scanner), because KDE has such an impact on the system, that it is difficult just to put LXQt over the top of it.

Free Linux video editors [3]

Well I have written about free video editors for Linux a number of times, this is the third article in this series. It’s taken me a few years to get to this point of discovering good quality software for Linux because there are so many open source projects that people have started and then abandoned, and finding the really good ones, which are mostly being produced by professional companies, has taken a while.

Here is an article from Linx Magazine and it lists a few options

https://www.linux-magazine.com/Issues/2015/171/Video-Editor-Roundup

For my setup, AppPC which has potentially sufficient storage and RAM, will be the system that long term is set up for tasks like this. Obviously it doubles at the moment with Gimp editing mosaic projects for NZ Rail Maps but this won’t be happening forever and so, it is natural to look at using its ample resources for video production, and possibly in the future, audio editing as well.

There are some really good high quality video editors out there. Last time I wrote I was taking a look at Blender and Handbrake among others. I am currently working on a tricky project that has involved bringing together a number of clips shot in two different formats. About half of it was shot at 640x480 in MOV format and the other half in 1920x1080 MP4. The problem is melding together these two resources that are greatly different in size as well as aspect ratio. After a lot of mucking around I finally used Avidemux with two video filters to put black borders onto the original video clip (changing its resolution to 1920x1440, which gives it a 4:3 aspect ratio) and then scale this to 640x480. A little trick to remember is that video filters in Avidemux will not work unless you choose a different video mode other than “Copy”, in other words you tell it to re-render the video output. This also gives you a chance to downsample the video at the same time. The combination of these settings brought the final clip down from 14 GB to 0.5 GB in size. Handbrake was useful for converting the MOV clips into M4V but I could not work out how to re-aspect the HD clips in it so that ended up being what I used Avidemux for, as well as joining a number of separate clips into one.

So I now have two separate clips each about 80 minutes in length and the same format, to put together in a video editor, along with a few still shots. I am trying out several different software applications to see what is going to work best. Here is what I have attempted so far:

• Blender is FOSS and has a lot of support for it, but not as a video editor. This function is not well supported in the wider community. When I tried using it, I found it was clunky and not very intuitive. This led me to look into other alternatives.
• Da Vinci Resolve is a free version of a commercial product. It is complex to install, and wouldn’t work out of the box for me. Because of this I haven’t looked further into it.
• Lightworks is also a free version of a commercial product. It looks promising, but so far I haven’t tried installing it.
• Cinelerra is another of the few that are FOSS. There are two versions, HV which is the full deal, and GG. If you want to use HV, you have to compile it yourself from the source code. GG is a whole lot easier, with native packages in a repository that install directly onto Debian (and a few other distros). So I have installed it, for testing, and will see how it goes. The software looks reasonably good but the interface so far is confusing, it has also crashed once. 
• Kdenlive is another fully FOSS package. It is officially part of KDE, and is produced as an AppImage. This makes installation very simple and straightforward as it involves just a download. So I have this installed as well, for testing alongside Cinelerra-GG. At the moment Kdenlive is the one I am trying the most.

So we will see how things go with the last two packages with this project and which option is the easiest to use and gives the best results. I hope to have this project finished by the end of this week.

Ubuntu LTS vs non LTS releases

Good day. As I install various versions of Ubuntu on a few virtual machines and computers and also use Debian on some as well, there is naturally an important comparison to be made between the Debian and Ubuntu release cycles. Ubuntu has stabilised on a twice yearly release cycle but the LTS releases are every two years and as I am now finding, the 5 years of support for an LTS release is much preferable to only a few months for intermediate releases. Recently I have had to deal with systems installed with Disco releases that are now not being supported for updates as what I understood to be about two years of support has been dropped to just nine months resulting in 404 errors when apt attempts to contact the servers for updates. This in turn affects the ability to install new software as well as update existing packages.

This realisation leads me to conclude that as far as any version of Ubuntu is concerned, it is best to stick with LTS unless you are intending to keep updating every year or twice a year to the latest release. For my laptop that is only used occasionally I am depending on it being able to be installed with new software at any time and not have the situation I had this week where I was unable to install Zoom because the supporting packages for the installation were not available from the official repositories. I was then forced to update to a later version before being able to complete the installation.

For this type of situation we must therefore conclude that it is best to rely on the Ubuntu LTSs which like Debian come out less frequently but are supported for a longer period. The latest LTS is 20.04 (Focal) and hopefully it won't need to be updated until at least 2 years when the next LTS comes out for the software I use on the laptop because I don't want to be caught the same way again. The best thing that can be said is that you can streamline a reinstall by remounting /home to a separate partition and then install as much software as you can to your home folders but this is not possible to achieve for software that relies on apt, dpkg or snap to install. I have only been able to achieve this with portable type software such as AppImage that can be placed into any path as a single file and run there, as well as Firefox Developer and Thunderbird which are designed as portable applications that can run from any folder path and consist of multiple files and folders that can be extracted to any path from an archive.

Linux video editing, revisited

Some way back, I don't recall exactly, I talked about Linux video editing. When I set up mediapc the idea was that was going to be its key function. Then I ran into some roadblocks trying to find FOSS software that could do this, and just gave up and stuck with Avidemux, which can do a few basic things but does tend to crash from time to time (latest is it crashing every time I try to extract 14 seconds from a 50 minute clip). To be true, I also did look at some audio mixing software that was Linux shareware, had a quick play with it and left it because I didn't have the need at that time (Ardour). It might be completely free now, I'm not sure, but I expect something like it will get more use from me in the future the way it looks like things are going for me. 

Anyway, back to video editing. This blog post by a Red Hat guy was really enlightening, because his experiences with some of the FOSS packages out there really mirrored mine.

• https://opensource.com/life/15/1/current-state-linux-video-editing

Clearly he reached a conclusion out of many editors he looked at (I looked at most of them as well), Blender is the only one worth bothering with, all the rest are cr@p. So at least I can be sure it isn't just me that many of these programs crash non stop, etc.

mediapc and playerpc will be getting Blender installed in them pretty soon for various tasks and it looks like once I get the hang of it, it should be able to do everything I need. We shall see. 

Rolling back a Flatpak application to a previous version

Today on my main graphics editing PC, I decided to update Gimp from 2.10.16 to 2.10.18. Which should have been OK. Except that it wasn't. I have spent all day dealing with Gimp crashing during saves and trying to work around with ever smaller files to be saved. It did crash last night as well but crashes during saves like that are normally relatively infrequent, and in this case I observed that the system resources were almost completely exhausted, which hasn't been the case at any time today - one save was only half of the file, another was only 3%, and in neither case were the system resources anywhere near excluded.

As the evening approached I have decided enough is enough, let's try to roll back Gimp to the previous version. Well, as it turns out, provided a previous version is available on the server, this is certainly possible.

Flatpak can interrogate a repository for you to see what versions are available by using the following command:

flatpak remote-info --log flathub org.gimp.GIMP

Obviously the last parameter is the path to the repository which for Gimp is the one shown in the example above.

What will come out is a list of commits, for example here is some of the output from a command issued today:

Commit: 62578b6f3b1262ceab9fc5d8a93d1b034cda51636a88bce9de6036ae6c5f038c
   Subject: Release GIMP 2.10.18. (bbbf1b4e)
      Date: 2020-02-25 00:53:47 +0000

    Commit: 57036c515dd653f828b8c6574e4b56fca6c29b6ac04f5e84c328cc6fc5e7987f
   Subject: Release GIMP 2.10.16. (796cb329)
      Date: 2020-02-18 22:43:45 +0000

    Commit: 798d8cd8a6a2ccf1bf21c0f5e9749dd4d97ecd511a3af1922d6fedf6c26e3a7c
   Subject: Release GIMP 2.10.12! (1366fa63)
      Date: 2019-06-13 02:25:52 +0000

In this case I have decided to roll back to 2.10.12 because of being uncertain about whether I had 2.10.12 or 2.10.16 previously installed (there was no release for 2.10.14 for some reason).

To achieve this the following command is required:

flatpak update --commit=798d8cd8a6a2ccf1bf21c0f5e9749dd4d97ecd511a3af1922d6fedf6c26e3a7c 
  org.gimp.GIMP

Note the slightly erroneous command "update" even though you are downgrading in this instance.

One blessing at least is that flatpak, at least on Debian, will not automatically update when you use apt upgrade. So we can use apt to update everything else and flatpak applications will not be affected. I suppose Synaptic or KDE's software manager might one day try to update flatpaks itself - I have no idea if that is the plan the developers have in mind. Every time I see the message there are updates ready I just go to the command window and use apt to install the updates.

However, my other computer has 2.10.18 installed and hasn't been having crashes when I edited large Gimp projects on it so I am not sure what is going on with my graphics computer but I have wasted too much time in the last couple of days to want to continue with 2.10.18 at present. Both computers are fairly up to date with 15 new packages released, most of them being a new Qgis build, so it isn't really obvious right now why the systems appear to behave differently, unless it is this particular file.

Subsequent testing however has shown there is nothing wrong with Gimp and the issue is down to the particular characteristics of this file. Gimp does seem to have internal limits as to the amount of file data it can deal with and it is rare for me to be able to edit a project larger than about 30 GB because they tend to crash on save, probably due to the additional resources needed to compress the file before saving it. There doesn't seem to be any issue with canvas size as such - as one would expect a canvas does not require every pixel to be saved. The software simply needs to save the data in each layer along with its coordinates within the canvas. So I have been able to work with very large canvases of over 10 Gpx without difficulty - as long as they are sparsely populated with graphics layers.

As part of my testing I reinstalled the graphics pc completely which was very easy with only a few applications installed on this special purpose computer, but especially because of being able to reuse the previous home drive which saves a great deal on reinstallation because all of the user settings are kept. The reality is a new installation can be up and running in an hour. I also noted that Debian 10 ships with its own Gimp 2.10 package pre installed (2.10.8) but for some reason this version could not open anything so I have uninstalled it and only use the Flatpak version.

Stopping desktop notification popups on LXQt

I see a fair bit of LXQt, whether it is on my one desktop that is running it on top of a Debian install, or my preference for laptops and virtual machines which is to install Lubuntu for fast pain-free installations, especially with non-free firmware and low resource use. Debian remains my preference for my desktops due to its versatility and feature set being more oriented to hard core geeks like me. Hence the one desktop that is running Debian/LXQt rather than Lubuntu, which is annoying in having disabled hibernation, among other things.

One thing that itself is a nuisance on a computer for screen recording is the popup desktop notifications in LXQt, which for example take over the browser message popups, or the on screen feedback to up and down volume buttons being pressed on the keyboard, and as far as I can tell, always appear on screen 1. There is only one way to get rid of these, but you can't configure them by application as you can on some desktop environments. Nor is there much control over them in the LXQt config center. So it is just easier to turn them off altogether.

Just go to LXQt configuration center, open Session Settings then on the first screen "Basic Settings" in the "LXQt Modules" window, find "Notification Daemon" and stop it, then untick its entry as well.

You will still have to look at browser notifications - I recommend turning them completely off as well.

Rsync Backup System [5]: Ongoing Backup Journey

Since writing the first four articles on this topic, I have not yet figured out how to implement an incremental backup strategy, and so I am still doing only the full backup every few months. Hence right now I am doing my first full backup of mainpc for three months.

The first issue that has come up is zfs saying there are no pools available on the server that mounts the backup disks and runs the backup. This apparently is because zpool is not designed with removable disks in mind, even though they have been unmounted prior to removal. After a lot of head scratching trying to figure out how to get zfs to recognise the previously created pool on the removable disk I had inserted for the backup, I eventually stumbled across the zpool import command. Once that was issued the pool was said to be online and the zfs set mountpoint command could be issued:

zfs set mountpoint=/mnt/backup/fullbackup mpcbkup2

At that point I could go to the mount path and work with the contents of the disk.

One possibility for this issue could be that the disk is physically in a different path on the computer than the one it was created in. A key factor is that zfs appears to be designed primarily to work with /dev/sdx device paths. We already know that for regular disk mounting we can use UUIDs to get around the problem of the device path /dev/sdx changing at the whim of the operating system at boot time, which actually does happen. For some reason or other the backup disk in this instance was at /dev/sdh when it may have originally been in a zpool mounted at /dev/sdb according to the earlier articles I wrote. At any rate, using the import command can bring the zpool back to life. Perhaps there is a need to have a command that suspends a zpool when the disk is taken offline, since it appears to be necessary to do more than unmount the disk.

I looked into this further and after considering the options, the best command to use is
zpool export mpcbkup2

This command basically closes down the pool for it to be removed from the system. The system will then report that no pools are available. The next time we need to use it, it can be imported back into the system and then mounted as above. 

I also need to buy another backup disk for ensuring each major server has two full backup volumes. At the moment I don't have enough disks to assure this. It will only cost about $100 to get another 2 TB disk.

The lingering question of course is how to detect which files were backed up on which date and therefore which files need to be backed up incrementally. To make any progress with this, the very first step is to find out if rsync can log the files that have been successfully backed up, and find some way of automatically scanning the log for the names of files, and then store them somewhere (for example, an SQLite database). Another option is to find some way of having rsync only back up files that have been modified after the date of the last full backup. So far I actually haven't spent nearly any time at all thinking too much about these options, because the simplest solution by far would be to have some sort of command or script that handles this completely automatically. Linux lacks the file modification flag that is implemented in Windows (the archive bit). People explain this away by saying the FS has superior capabilities and that a file modification bit is a very crude capability, but it still isn't possible to get around the fact that being able to reset that flag after each backup, and then being able to scan for files where it has been set again, are very easy to implement. In Linux there is no easy way of being able to record that information unless you store a date somewhere for each file and then scan against those dates. I explored the possibility of writing an extended attribute for a file, the question there is where this data is stored as we do not want to modify the file itself. So I hope to spend more time over the next month or so exploring these issues further.

I am currently trialling having a log file produced and using this form of the rsync command:

rsync -arXvz --progress --delete backupuser@192.168.x.y:/home/patrick/ /mnt/backup/fullbackup/patrick --log-file=/home/patrick/rsync.log --log-file-format "mainpc|%f|%M|%l|%b|%o|%U"

The only issue to date with it is there is supposed to be a %a option but that is not being recognised so having the remote IP address logged is not available so the script has been customised to output the actual remote computer name as a literal.  The rest of the information is logged in the format and the pipe characters can be used as delimiters to separate the parameters. So I have made some progress on this issue and now the question is how to use the information to analyse what is needed for future backups.

With the backup of mainpc I wiped the disk first, but for serverpc I chose to send the data to the existing backup which means it can run a lot faster because it is not transferring every single piece of data to the disk. So that option should speed up the backup and using the --delete option will remove files on the disk that are no longer present in the source directory.

I have to set up mediapc to be able to back itself up, which will be implemented as a "backupuser" account that is logged on to in the terminal virtual console, and then runs the backup for mediapc locally with read permissions to the source files/folders.

An option for progressing the backups is simply to use the full backup disks to create the incrementals as well, but this is possibly going to need bigger disks. The advantage is that rsync can handle this by default with the incrementals pushed into a separate directory. I would want to have possibly three backup disks for each comptuer, which brings the need to buy more disks.

At any rate this will take some time to devise. Another alternative is separate incremental disks, with a second removable disk caddy installed in this computer, so that it can backup to a completely separate disk. This does have the advantage of not requiring new disks, and keeping the full and incremental backups separate.

Debugging Linux startup issues with JournalCtl

My computers running Linux are very reliable but every so often a spanner will be thrown in the works and the computer will refuse to come up beyond a command prompt. When this happens you have to look at the logs to try to work out what is happening in the system and how to get it going again.

This happened to me a couple of days ago when I turned on the computer and instead of resuming after hibernation, it threw up a number of error messages and then asked for the root password for maintenance (or there was the option to press Ctrl-D to continue). The Ctrl-D option simply put the computer into a loop where it ran a FSCK only of the boot drive, and then failed to resolve any issues, and came back to the same message. Or the option was there to enter the password which took it to a terminal to see what could be done.

Since the advent of systemd, its companion journalctl (with the -xb parameter) is what is recommended to look through to try to determine what the problem is. This took quite a while and it wasn't particularly obvious what was actually happening. The main issue I noticed immediately was no networking; the wired network adapter in the computer appeared not to be functioning, indicating that part of the startup of getting the adapter up and running had not happened. I tried a whole lot of things but eventually concluded I would have to try a live CD to see if this was a hardware issue or if it was something with a driver. 

The network turned out to be a red herring once it was realised that there was no home path in the file tree (i.e. /home was completely missing) and fairly soon running fsck over /dev/md1 revealed some errors. Once these had been fixed, rebooting the system brought up KDE normally and suddenly everything was working again like usual.

It isn't the first time this has occurred and I think last time we just threw in the towel and reinstalled. At the time of trying to resolve this issue I was downloading netinst and live images to burn to a pen drive and try to reinstall, but finding this relatively simple solution sorted the issue without having to take that complex, lengthy and very inconvenient route of putting everything back together. And what seems an obvious issue may actually not be the problem at all. The network adapter not working turned out to be a non issue as soon as the disk issue was taken care of and a reboot initiated.

Change Monitor Configurations Permanently in LXQt for Screen Recording

As we have discussed in the past, I use LXQt in two different configurations with my various computers. Lubuntu, which now uses LXQt as its desktop environment, is what most of my non-desktop computers use. For example a media playback computer and a laptop are normally set up with Lubuntu which is quick and easy to install as it has all the drivers included in the setup.

For my desktops I am using Debian, with LXQt on one computer in particular because this computer needs stuff like hibernation that is better supported in Debian, but also with LXQt it gets around some of the bugs in KDE which I use on other desktops, as well as ensuring the resources that KDE uses aren't wasted in its lower spec. So I am testing these settings on that desktop because it has two screens and it is in fact the main computer that I have at the moment that these settings are important for.

Because I use this computer to do screen recording (using SimpleScreenRecorder), the screens have to be nearly at diagonal placement to each other. That is, the left hand screen and the right hand screen are to the left and right of each other, but in a vertical sense there is only a tiny overlap at the top right corner of the left screen and the bottom left corner of the right screen. This means the mouse can only be moved from one screen to the other by passing through a very small area of overlap that is in the top right corner of the left screen and the bottom left corner of the right screen, and also windows needing to be dragged from one screen to the other can be passed through this small area.

The reason for this configuration is that for screen recording, we have the software that is doing the recording running on the left (control) screen, recording the right (target) screen. We want to ensure what we need to do on the left screen does not cause anything unwanted to be displayed on the right screen. Some examples of this are:

• accidentally moving the mouse from the control screen onto the target screen
• any action on the control screen which causes pop up notifications, such as operating keyboard volume up and down. Even if the popup is on the control screen, if it is adjacent to an area of the target screen it can cause unwanted display of information on the target screen.

We ensure this by making sure as described above, minimum overlap between the two screens, just enough to get the mouse from one to the other and drag windows from one to the other. We still have to be careful not to run the control screen applications maximised to ensure the mouse is kept out of the top right corner of the control screen at all times other than actually going from one screen to the other.

The LXQt monitor settings GUI doesn't actually position the screens exactly as we would like to. They can be aligned vertically and horizontally close to the suggested outcome, but with a small amount of unwanted superimposition of the two screens. Fortunately you aren't limited to using that GUI to set positions. After changing the positions in the GUI and closing applying or saving settings, you can further tweak these with manual editing in the file ~/.config/lxqt/lxqt-config-monitor.conf. These seem to be just the same sort of parameter specs as running xrandr will spit out for you. Under the [currentConfig] section are the ones presently being used for the computer and with a few adjustments we can eliminate that superimposition and ensure the overlap is as small as possible. For example, with the control screen being screen 1 on HDMI-1 (settings\1) at a resolution of 1440x900, xPos=0 yPos=745 are the settings and for the target screen being screen 2 on HDMI-2 (settings\2) resolution 1360x768, xPos=1440 ypos=0. 

The names of the displays xrandr generates for this system are interesting as it has one HDMI port, one DVI port and one VGA port, yet the DVI port appears as an HDMI port in the list of outputs. It seems this initial configuration of the displays happens just after logon. Also, the settings are saved in the user's profile (~/.config folder) which means when reinstalling, these settings should be reused into the new installation of LXQt.

OverGrive seems to work again...for now.

OverGrive is a Google Drive client for Linux. It has been around for several years. I used (and licensed) an earlier version of the product, which I was forced to stop using last year because it stopped working (this quite often happens when Google updates their API to a new version which requires software changes). At that point with no apparent communication or updates from the developer, I switched to the free "grive" package, which however is command line only and lacks the GUI configuration interface, system tray icon and other features of OverGrive. However that also stopped working earlier this year, probably for similar reasons.

It so happened I was reading a Linux website today and saw an article on Tech Republic showcasing OverGrive, so I decided to try reinstalling it, and was surprised to find that it works. The version listed on their website is still the same version as I had installed before. It seems to work OK on Buster despite this (the free grive package had to have specific versions for each release of Ubuntu, which was a trial and error installation onto Debian because of course, Ubuntu uses a different release schedule).

I currently have two licenses for two different versions, on two different computers. One of these is for the NZ Rail Maps project and provides for a daily backup of the core project files and layers. It doesn't, of course, include all the aerial photo layers which would not fit into a 15 GB Google Drive cloud storage space. However at present these layers are stored on two different computers as well as being backed up on removable disk.

So with very minimal documentation on OverGrive it's hard to understand why it stopped working on my computer and whether there really has been an update to it. The homepage still links to the highly out of date GitHub and Launchpad repositories, without any explanation of why these areas have not been updated for several years. If this outfit actually still maintains their product and take an active interest in it, they haven't exactly made it obvious.

Rsync Backup System [4]: Linux File and Directory Security Using ACLs

As per our series on Rsync backups, we desire to use a different user from the one that owns the home directory in order to ensure they have only the permissions they need when logging in remotely. At this point whilst by default the backup user could access many things on the computer, there were some files that they received permission denied errors for.

Whilst there is a simple file and directory permission scheme built into Linux, the acl extensions are very worthwhile as they give a lot more control, especially with the all-important ability to have inheritable permissions on a parent folder. In this case we'd want to give the backup user read-only permissions on the home directory that are inherited by all subdirectories and files as they are created.

Getting ACL is pretty simple, firstly you need to install the acl package using apt. The second requirement is to see if a volume is mounted with acl support. Use a command called tune2fs -l /dev/xxx (part of e2fsprogs package) and look for "Default mount options". If it says acl in there then the volume will be mounted by default with ACLs enabled. If this is not a default then you'd want to go into the volume mount entry in /etc/fstab and you can add acl to the mount options there. In this case because in fstab I used "defaults" as the mount options, acl is automatically turned on at mount time. If I needed to specify acl then I could change the word "defaults" to "defaults,acl" in the fstab entry to enable ACLs.

ZFS also has its own (more extensive) ACL support which is not covered by this post as we are referring to the existing filesystems which are ext4 on the computers being backed up in this case. I am not planning to change the existing RAID arrays from MDADM to ZFS but it could be an interesting consideration for the future should a disk array be replaced in any system.

Now how to configure ACLs for our requirement? Commands used to set and get ACLs are setfacl and getfacl respectively. To set up read access for backupuser on /home/patrick we need to use a command that looks like this:

setfacl -dR -m u:backupuser:rx /home/patrick

If we then use a ls -l (or ls -al) command in /home/patrick we can see next to the standard permissions a + sign which tells us that ACLs are set. In this case I can see whilst all directories have ACLs set, the files that are in each directory are not set with ACLs. So I would need to run another command to set these:

setfacl -R -m u:backupuser:rx /home/patrick/* /home/patrick/.* /home/patrick/*.*

The two different commands are as follows:

The first one using 

-d setting the default ACL for each directory (the default ACL to be applied to any new file or directory)

-R operating recursively on all subdirectories

-m modifying the ACLs

u:backupuser:r is user backupuser granting read permission.

The second one is practically the same except there is no default applied because it is for existing files and the paths following it are the different filespecs to match. 

Whenever a new file or directory is created in future it will inherit the default ACLs that have been defined using the first command which means backupuser will get the permissions it needs. Multiple paths to apply the ACLs to can be specified but only one user specification can be put in at a time.

ACLs override the default permissions system when they come into effect and so you need to have good knowledge of how Linux permissions work. In particular you need to ensure your user has x (execute) as well as r (read) permissions specified in ACLs for directories in order to be able to traverse directories, otherwise you will keep getting permissions issues. It took me a bit of work to get the ACLs set up but once they are done properly, the advantage is that inheritable permissions are possible, so for my backups, the backup user will have automatic rights to new files and folders that are created in future. In fact for any computer the way to make it happen is to set a default ACL on /home that gives the backup user rx permissions on everything below in future.

With the ACLs properly set the test backup was able to access all the files it needed to and complete the backup. Since I knew there were roughly 114,000 items waiting for backup, it was interesting to watch rsync at work discovering new files (a big increase from the roughly 7000 it had access to the first time). 

The means of doing the incrementals is still being worked out as it is not as straightforward as I had hoped. The log file is not consistent in its format (at least not so far) that would make it easy to determine which files successfully transferred. The option I am tending towards would probably consist of getting a list of files that have changed since the last backup, then getting rsync to copy one at a time, looking at the result code that it returns, and if there is no error then writing an extended attribute of the source file to indicate a successful backup result. The extended attributes will probably be backup.full = <date> and backup.incr = <date> and these two extended attributes will be written by the appropriate backup script. A full backup simply updates the attribute at backup completion without checking it. Whereas for an incremental, we compare the last modified date in the file with the date stored (backup.incr if set, otherwise backup.full) and do the backup if the file has been modified since the last backup date. So I expect to start more testing reasonably soon with developing scripts to perform these tasks.

But so far as the full backups go, it's doing pretty good. I have just set up another full backup and it's working very well. The second time setting it all up was so much smoother. It also looks like ZFS disk compression works very well and has achieved some useful saving but I will check out exactly what sort of improvement I am getting on these disks.

Rsync Backup System [3]: Using Rsync For Full Backup

So last time we talked about how to set up a single disk with ZFS to use compression. Having got our backup disks sorted, the next step is to work out how to use rsync to do the actual backups.

rsync is written by the same people that devised samba and is a very powerful piece of software. What we need to do is to first of all set up a dedicated backup user on the system that is to be backed up, that has only read access to the filesystem. Having this is extremely important in case we make a mistake and accidentally overwrite the files on the source filesystem (which is quite possible with such a powerful system).

Using useradd we can set up backupuser in this case with a very simple password consisting of four consecutive digits and then by default it will have read only access to other users' home drives.  This means it can read my home directory. backupuser in this case is used with SSH to access the home drive over the network for creating the backup.

The next step is to look at the required form of the rsync command. There are certainly a lot of options for this. Assuming the backup destination zpool has been mounted to /mnt/backup/fullbackup and the source is on 192.168.x.y at /home/patrick then a good starting point for the backup would be along the following lines:

rsync -arXvz --progress --delete backupuser@192.168.x.y:/home/patrick/ /mnt/backup/fullbackup/patrick --log-file=/home/patrick/rsync.log

This looks to cover all bases needed. The options are -a which means copy in archive mode (preserving all source file attributes. -r which means recurse into source directories, -v which means verbose and -z which means compress during transfers (which can speed things up). --progress means during each file transfer you will see the bytes transferred, speed, percentage and ETA which is useful for large files. --delete will remove extraneous files from destination directories (which means files that aren't in the source). This option is obviously useful when a file gets deleted or moved in the source. -X means to preserve extended file attributes if any exist.

At the moment I am testing this with a full backup of serverpc with this running in a terminal window. There are some issues with some directories not having permission for backupuser which so far has only affected a few hidden folders but will have to be looked into further. Previous backups always used patrick as the user but it is pretty important to have a special backup user with restricted permissions which is really a best practice for any kind of professional computer setup. An example is because a mistake with the rsync command could wipe out the source directory if there was read-write access to it.

After looking at log file options there is this option that we can set the log file format for rsync using an extra parameter and this is what I came up with in deciding what format of information would be useful in each line of the rsync log:

rsync -arXvz --progress --delete backupuser@192.168.x.y:/home/patrick/ /mnt/backup/fullbackup/patrick --log-file=/home/patrick/rsync.log --log-file-format "%a|%f|%M|%l|%b|%o|%U"

Using in particular the log file format option, it seems to be usefully logging the information we need for each file. However the %a logfile option does not actually seem to be recognised by rsync. 

There is one thing that rsync does not do by itself and that is incremental backups to a separate disk. rsync is designed to be able to by default create incremental backups in a separate directory from the one where the full backup is stored, but the full backup directory must be online at the time when the incremental is done so that it knows which files have changed. This is a problem when, as I intend, you want to be able to use separate disks for the incrementals. Here you see the fundamental issue with the Linux file system (at least ext4) that does not have the separate archive flag for a file that NTFS has. The argument goes that it isn't necessary but that archive bit is the way you can tell that a file has been modified since the backup last ran, and increment it. 

There are several possible solutions to this and one of them is to create an extended attribute for each source file. This is possible using the setfattr command. So we could create this on each of the source files following the full backup (it would have to be a separate script process following the execution of rsync). Maybe the script would be part of a verify process that we run after the backup is carried out, to verify that the source and destination files both exist, and then write the extended attribute to the source file. The issue is that it may well prove difficult to be sure the source file was backed up unless we can have a look at a log file and prove that it lists the source files and then feed that into a script that produces the extended attribute. Anyway, the point of this is to write an extended attribute to each source file that says when that file was last backed up. This could be a part of an incremental script that uses find to get all files that were last backed up since a certain date. Or we can just use a range of dates for the find command to get the incremental file list using the last modified file time. This will all be looked at in more detail when I start working on incrementals, because for now I am just doing a full backup like I did before with rdiff-backup.

HOWTO: Set font size in virtual consoles

In Debian you can have virtual terminal consoles, which are different from a terminal emulator window. Before Buster came along, a lot of commands could be run in a terminal emulator, but these days, more often than not, commands have to be run in a virtual console instead. The difference is that a virtual console is fullscreen, not a window, and doesn't run under the GUI, which goes into the background at that point.

You can have six separate consoles that you switch between by pressing Ctrl-Alt-Fx where x is 1 to 6, i.e. the F1 to F6 keys. To switch back to the GUI you press Ctrl-Alt-F7.

The next issue is the font size which I tend to find rather too small. Fortunately this is easy to fix. Use nano to edit /etc/default/console-setup and change the following lines to read as follows:

CODESET="Lat15"

FONTFACE="TerminusBold"

FONTSIZE="16x32"

This gives us quite a large easy to read font. 

Run the command setupcon to apply it immediately, it should become the permanent setting.

The fonts are all stored in /usr/share/consolefonts directory so you can look up what is available if you want something different. What I haven't done yet is work out if I can change the colour from white.