Rsync Backup System [6]: Latest Progress / Using The Rsync Daemon

Because of issues with Rsync over ssh I am looking into using the Rsync daemon instead. The commentary on that is below. Until I can get that working properly I am backing up each computer individually using its own removable drive bay. This can be achieved by changing the ownership of the backup folder on the removable backup drive to backupuser and then  logging into a virtual terminal as backupuser and then running the backup under that user account. We use that account rather than root or our regular user account on the system being backed up in order to ensure the account has only read and execute permissions on the backed up computer so that it cannot in any way risk deleting files from the computer. This is very important because I have used the --delete flag on rsync to ensure files removed from the source are also removed from the backup. When doing an incremental backup that flag is important to ensure the disk does not end up filling up with removed files and running out of space. But it is also useful to have the safeguard that you can't accidentally copy files from the backup disk over the source files, especially with incremental where your backup disk is already full of files.

The simple means for an incremental backup that rsync supports is using an existing backup disk that has previously had full backup onto it. This enables rsync to compare the source and backup without any additional system to record file details that it can then work out if they have changed. In other words with each source file it compares to the existing backup then only backs up incremental changes. This is quite different from other backup solutions I have used that have a separate incremental backup disk.

To implement this system I will need at least 3 disks for each computer if I want to keep one full backup and two incrementals. In other words each disk will start off as a full backup, then two of them will be alternating incrementals, and a full backup from scratch will be done every few months and kept separately from the incrementals.

I would still have preferred separate incrementals, and in fact it is possible with two removable drive bays, one contains a full backup disk and one contains an incremental disk, then the incremental can be written to the incremental disk instead of the full backup. However the problem with this solution is that rsync cannot produce progressive incrementals since it is always comparing with the last full backup, that means the incrementals will not be a snapshot at a particular point since the last incremental, but will always be incremented from the date of the last full backup. It is also messy working with the extra paths to the different disks in the backup command, and therefore, more likely to risk mistakes like backing up to the wrong disk.

So in summary the best solution is using a separate disk for each backup, which will be an incremented full backup in the case of 2 disks and a separate full-only backup in the case of one disk.

At the moment I am down to 1 backup disk for each computer and so I need to buy some more disks along with the removable bay caddies and storage cases. I am also working on secure storage for the disks in the storage location which is away from the house. At the moment I have only been doing one backup about every 3 months because the RAID-1 array system in the computer is so reliable for day to day backup that I don't need backups too often. But I will attempt to schedule a monthly backup cycle in future.

Since I have been using Rsync as a backup solution, it has been very reliable, but there are occasional difficulties, and one I am having at the moment is the backup terminating with rsync error 12 partway through. I have done some fault finding but haven't been able to narrow down what is occurring that causes this termination but it is probably something to do with the ssh client in the source computer, which rsync connects to. This means I have to either try and find out what is happening with ssh on the source, or try the alternative, which is using the rsync daemon (service) on the source and connecting rsync to that from the backup server.

I'll be starting with this guide and applying the steps in it: https://linuxconfig.org/how-to-setup-the-rsync-daemon-on-linux
Also https://www.atlantic.net/vps-hosting/how-to-setup-rsync-daemon-linux-server/

Firstly as I am on Debian, rsyncd is already installed with rsync. Next step is to configure the /etc/rsyncd.conf file which I will do from this sample provided in that article:

uid = 1001

gid = 1001

pid file = /var/run/rsyncd.pid
lock file = /var/run/rsync.lock
log file = /var/log/rsync.log

[backupuser]
path = /home/patrick/
comment = backup patrick
read only = true
timeout = 300

There are some changes I am doing from their sample, the most notable being the uid and gid, which are set to the values for backupuser, which is the user that has permissions to do the backup on the source computer. Assuming rsyncd is run as root, this will ensure the permissions are correct. Also I am specifying the module name as backupuser, and it will automatically connect to the backup path for my home folder.

Start rsync with systemctl start rsync and set it for startup with systemctl enable rsync .

So next time I will list the result of testing with the daemon and whether it worked successfully.

Setting up sudo on Debian

Debian is a bit different from Ubuntu and that is the reason I like it. Debian is much more focused for technically advanced people and, for example, has hibernation enabled by default, which contrasts with Ubuntu that makes it difficult to use hibernation, and there are many other differences in Debian that I prefer.

One thing with Debian is that sudo is not set up by default for the first user account and it's necessary to undertake a few steps to set it up. sudo is installed so all you have to do is add your user account to the configuration. This also underlines that under Debian 10, /sbin is excluded from the standard user's path in a terminal window and many commands will not work unless they are prefixed with the full path, something I have not yet got to a full understanding of the rationale for. When you go into a virtual terminal, that limitation is removed and it's hard to know why the restrictions have been put into the terminal window running under the desktop environment, when Debian 9 did not have those issues.

Anyway the steps to set up sudo are pretty straightforward:

• Use the su command to enter superuser mode in a terminal shell
• Install sudo if not already installed: apt install sudo
• Run the adduser command to add your user account to the sudo group: /sbin/adduser <username> sudo
• Log out and then log back in again

And that's it.

HOWTO: Set laptop screen brightness in Lubuntu

Some laptops have function keys that can be used to adjust screen brightness. These may not be supported on all laptops, however. In that case, there may be the controls in monitor settings or power management to adjust the brightness.

In my case I wasn't able to find anything, so I discovered how to adjust this using a command line setting. Install the xbacklight package and then invoke it with a command like this

xbacklight

By itself, this will give you the current reading.

xbacklight -set <x>

Pass some number after the -set parameter and it will set the brightness. I found 50 to be a good number after discovering the default is just 12.5.

Rolling back a Flatpak application to a previous version

Today on my main graphics editing PC, I decided to update Gimp from 2.10.16 to 2.10.18. Which should have been OK. Except that it wasn't. I have spent all day dealing with Gimp crashing during saves and trying to work around with ever smaller files to be saved. It did crash last night as well but crashes during saves like that are normally relatively infrequent, and in this case I observed that the system resources were almost completely exhausted, which hasn't been the case at any time today - one save was only half of the file, another was only 3%, and in neither case were the system resources anywhere near excluded.

As the evening approached I have decided enough is enough, let's try to roll back Gimp to the previous version. Well, as it turns out, provided a previous version is available on the server, this is certainly possible.

Flatpak can interrogate a repository for you to see what versions are available by using the following command:

flatpak remote-info --log flathub org.gimp.GIMP

Obviously the last parameter is the path to the repository which for Gimp is the one shown in the example above.

What will come out is a list of commits, for example here is some of the output from a command issued today:

Commit: 62578b6f3b1262ceab9fc5d8a93d1b034cda51636a88bce9de6036ae6c5f038c
   Subject: Release GIMP 2.10.18. (bbbf1b4e)
      Date: 2020-02-25 00:53:47 +0000

    Commit: 57036c515dd653f828b8c6574e4b56fca6c29b6ac04f5e84c328cc6fc5e7987f
   Subject: Release GIMP 2.10.16. (796cb329)
      Date: 2020-02-18 22:43:45 +0000

    Commit: 798d8cd8a6a2ccf1bf21c0f5e9749dd4d97ecd511a3af1922d6fedf6c26e3a7c
   Subject: Release GIMP 2.10.12! (1366fa63)
      Date: 2019-06-13 02:25:52 +0000

In this case I have decided to roll back to 2.10.12 because of being uncertain about whether I had 2.10.12 or 2.10.16 previously installed (there was no release for 2.10.14 for some reason).

To achieve this the following command is required:

flatpak update --commit=798d8cd8a6a2ccf1bf21c0f5e9749dd4d97ecd511a3af1922d6fedf6c26e3a7c 
  org.gimp.GIMP

Note the slightly erroneous command "update" even though you are downgrading in this instance.

One blessing at least is that flatpak, at least on Debian, will not automatically update when you use apt upgrade. So we can use apt to update everything else and flatpak applications will not be affected. I suppose Synaptic or KDE's software manager might one day try to update flatpaks itself - I have no idea if that is the plan the developers have in mind. Every time I see the message there are updates ready I just go to the command window and use apt to install the updates.

However, my other computer has 2.10.18 installed and hasn't been having crashes when I edited large Gimp projects on it so I am not sure what is going on with my graphics computer but I have wasted too much time in the last couple of days to want to continue with 2.10.18 at present. Both computers are fairly up to date with 15 new packages released, most of them being a new Qgis build, so it isn't really obvious right now why the systems appear to behave differently, unless it is this particular file.

Subsequent testing however has shown there is nothing wrong with Gimp and the issue is down to the particular characteristics of this file. Gimp does seem to have internal limits as to the amount of file data it can deal with and it is rare for me to be able to edit a project larger than about 30 GB because they tend to crash on save, probably due to the additional resources needed to compress the file before saving it. There doesn't seem to be any issue with canvas size as such - as one would expect a canvas does not require every pixel to be saved. The software simply needs to save the data in each layer along with its coordinates within the canvas. So I have been able to work with very large canvases of over 10 Gpx without difficulty - as long as they are sparsely populated with graphics layers.

As part of my testing I reinstalled the graphics pc completely which was very easy with only a few applications installed on this special purpose computer, but especially because of being able to reuse the previous home drive which saves a great deal on reinstallation because all of the user settings are kept. The reality is a new installation can be up and running in an hour. I also noted that Debian 10 ships with its own Gimp 2.10 package pre installed (2.10.8) but for some reason this version could not open anything so I have uninstalled it and only use the Flatpak version.

Stopping desktop notification popups on LXQt

I see a fair bit of LXQt, whether it is on my one desktop that is running it on top of a Debian install, or my preference for laptops and virtual machines which is to install Lubuntu for fast pain-free installations, especially with non-free firmware and low resource use. Debian remains my preference for my desktops due to its versatility and feature set being more oriented to hard core geeks like me. Hence the one desktop that is running Debian/LXQt rather than Lubuntu, which is annoying in having disabled hibernation, among other things.

One thing that itself is a nuisance on a computer for screen recording is the popup desktop notifications in LXQt, which for example take over the browser message popups, or the on screen feedback to up and down volume buttons being pressed on the keyboard, and as far as I can tell, always appear on screen 1. There is only one way to get rid of these, but you can't configure them by application as you can on some desktop environments. Nor is there much control over them in the LXQt config center. So it is just easier to turn them off altogether.

Just go to LXQt configuration center, open Session Settings then on the first screen "Basic Settings" in the "LXQt Modules" window, find "Notification Daemon" and stop it, then untick its entry as well.

You will still have to look at browser notifications - I recommend turning them completely off as well.

Wireless broadband [2]

So in my last post I wrote about Skinny 4G wireless broadband. I have had this for about 10 days and it's been really good. I have two computers connected to it, one in each main room of the house, and using a proper computer with it instead of a limited Raspberry Pi is really the key to getting the most out of it. But just as relevant is the observation that it is way better than the Vodem that I used to plug into one of my PCs via the USB port. There were plenty of times when that wouldn't connect reliably and it was often very slow especially in daytime. I was half expecting the same sort of challenges with Skinny but like my earlier evaluation of the new Spark wireless broadband for a relative, I was left pleasantly surprised by the performance and reliability of Skinny Wiress Broadband.

It so happens the same product is available as Skinny Jump at a much cheaper cost for people who have various forms of economic or social constraints that make it difficult for them to pay for a full wireless product, and certainly paying $10 for 60 GB is quite a lot cheaper than my $39 plan for the same volume. However although I can qualify for the Jump product and would not have to pay upfront for a modem as I have, I will continue with what I have purchased as I believe I am able to afford it and therefore the Jump funding subsidy can be directed to those most in need of it.

I don't have the modem running the whole time, it gets turned on and off with the computer - and it comes on and connects itself up pretty quickly without any issues at all. It took very little effort to get it in a position where it gets 3 bars of signal (maximum strength) and the speeds it is giving me are really good, easily comparable with Jetstream (ADSL) and way better than the throttled speeds I have been getting on my free connection from the school. The Huawei B618 modem is not quite as full featured as the Nanobridge that connects to the school - the B618 can issue DHCP addresses for example but can't handle address reservations and some of the other bells and whistles that the Nanobridge manages. But it is quite satisfactory for home use. I have turned off its WLAN because I don't as yet need to be able to connect to it wirelessly and also to avoid interfering with the existing wireless network in the house from the Unifi AP Pro. 

At the moment Skinny have granted us extra free data because of the COVID-19 lockdown but I will still have to pay $39 at the end of the month whether I use the extra data or not but at least I don't have to worry about going over cap for the next month or two. But so far usage is going well and I don't have to worry about going over cap in any case because most days would average less than 2 GB.

Overall therefore Skinny 4G Wireless Broadband is highly recommended.

Wireless broadband [1]

For the past four years I have had the privilege of a free internet connection shared from a corporate site next door to me where I used to work. It has been and in fact still is good and I intend to keep using it. However in order to supplement it and work around some of its limitations I am now planning to sign up for Skinny Wireless 4G Broadband. This has only recently become available in my area and appears to be limited to the Spark network (not Vodafone).

Skinny have a few plans starting with $39 for 60 GB of data with a downlink speed around 36Mbps and uplink 10Mbps. This is slow by fibre standards but a lot faster than my throttled 1Mbps Wifi connection from next door. According to a current deal they will give the first month free but you have to pay for the modem which costs $99, however their overall rates are the cheapest - Vodafone where available starts at $53 (albeit for 120GB which Skinny charges $49 for) whilst Spark starts at $65 for 60 GB but they don't charge you for the modem. So the first month would cost $99 and each month thereafter $39.

Up until now when I have needed an independent connection I have used either a laptop or a Raspberry Pi tethered to my mobile phone. The problem is that these mobile data connections are throttled quite severely during busy times (i.e. daytime) and the Pi is too slow to manage very much without grinding to a halt. Under the new arrangement I will have a regular computer that gets its internet exclusively from the wireless modem but will still be connected to the internal network as well except that it will have a different default gateway from the rest so that it won't connect to the corporate internet. Its internal network connection will generally be used just for file transfers and remote access.

Of course now there is the question of do I need my relatively expensive mobile plan with its large amount of data, and in reality I don't. However incorporated into that plan is two free tablet shares that let me use my tablet and second phone on the same data connection without an additional cost. So that is the reason I will probably stick with my current mobile plan but it is tempting to think I could switch to a cheaper prepay plan if I was willing to drop the tablet shares. One day perhaps Vodafone will stop letting me use my current plan but they are probably doing alright on it for now as it is pretty rare for me to use anything like my full data allocation and it will be even less likely with the wireless broadband installed. This is a backup to the corporate connection which is still going to be used for the rest of my needs.

Rsync Backup System [5]: Ongoing Backup Journey

Since writing the first four articles on this topic, I have not yet figured out how to implement an incremental backup strategy, and so I am still doing only the full backup every few months. Hence right now I am doing my first full backup of mainpc for three months.

The first issue that has come up is zfs saying there are no pools available on the server that mounts the backup disks and runs the backup. This apparently is because zpool is not designed with removable disks in mind, even though they have been unmounted prior to removal. After a lot of head scratching trying to figure out how to get zfs to recognise the previously created pool on the removable disk I had inserted for the backup, I eventually stumbled across the zpool import command. Once that was issued the pool was said to be online and the zfs set mountpoint command could be issued:

zfs set mountpoint=/mnt/backup/fullbackup mpcbkup2

At that point I could go to the mount path and work with the contents of the disk.

One possibility for this issue could be that the disk is physically in a different path on the computer than the one it was created in. A key factor is that zfs appears to be designed primarily to work with /dev/sdx device paths. We already know that for regular disk mounting we can use UUIDs to get around the problem of the device path /dev/sdx changing at the whim of the operating system at boot time, which actually does happen. For some reason or other the backup disk in this instance was at /dev/sdh when it may have originally been in a zpool mounted at /dev/sdb according to the earlier articles I wrote. At any rate, using the import command can bring the zpool back to life. Perhaps there is a need to have a command that suspends a zpool when the disk is taken offline, since it appears to be necessary to do more than unmount the disk.

I looked into this further and after considering the options, the best command to use is
zpool export mpcbkup2

This command basically closes down the pool for it to be removed from the system. The system will then report that no pools are available. The next time we need to use it, it can be imported back into the system and then mounted as above. 

I also need to buy another backup disk for ensuring each major server has two full backup volumes. At the moment I don't have enough disks to assure this. It will only cost about $100 to get another 2 TB disk.

The lingering question of course is how to detect which files were backed up on which date and therefore which files need to be backed up incrementally. To make any progress with this, the very first step is to find out if rsync can log the files that have been successfully backed up, and find some way of automatically scanning the log for the names of files, and then store them somewhere (for example, an SQLite database). Another option is to find some way of having rsync only back up files that have been modified after the date of the last full backup. So far I actually haven't spent nearly any time at all thinking too much about these options, because the simplest solution by far would be to have some sort of command or script that handles this completely automatically. Linux lacks the file modification flag that is implemented in Windows (the archive bit). People explain this away by saying the FS has superior capabilities and that a file modification bit is a very crude capability, but it still isn't possible to get around the fact that being able to reset that flag after each backup, and then being able to scan for files where it has been set again, are very easy to implement. In Linux there is no easy way of being able to record that information unless you store a date somewhere for each file and then scan against those dates. I explored the possibility of writing an extended attribute for a file, the question there is where this data is stored as we do not want to modify the file itself. So I hope to spend more time over the next month or so exploring these issues further.

I am currently trialling having a log file produced and using this form of the rsync command:

rsync -arXvz --progress --delete backupuser@192.168.x.y:/home/patrick/ /mnt/backup/fullbackup/patrick --log-file=/home/patrick/rsync.log --log-file-format "mainpc|%f|%M|%l|%b|%o|%U"

The only issue to date with it is there is supposed to be a %a option but that is not being recognised so having the remote IP address logged is not available so the script has been customised to output the actual remote computer name as a literal.  The rest of the information is logged in the format and the pipe characters can be used as delimiters to separate the parameters. So I have made some progress on this issue and now the question is how to use the information to analyse what is needed for future backups.

With the backup of mainpc I wiped the disk first, but for serverpc I chose to send the data to the existing backup which means it can run a lot faster because it is not transferring every single piece of data to the disk. So that option should speed up the backup and using the --delete option will remove files on the disk that are no longer present in the source directory.

I have to set up mediapc to be able to back itself up, which will be implemented as a "backupuser" account that is logged on to in the terminal virtual console, and then runs the backup for mediapc locally with read permissions to the source files/folders.

An option for progressing the backups is simply to use the full backup disks to create the incrementals as well, but this is possibly going to need bigger disks. The advantage is that rsync can handle this by default with the incrementals pushed into a separate directory. I would want to have possibly three backup disks for each comptuer, which brings the need to buy more disks.

At any rate this will take some time to devise. Another alternative is separate incremental disks, with a second removable disk caddy installed in this computer, so that it can backup to a completely separate disk. This does have the advantage of not requiring new disks, and keeping the full and incremental backups separate.

Change Monitor Configurations Permanently in LXQt for Screen Recording

As we have discussed in the past, I use LXQt in two different configurations with my various computers. Lubuntu, which now uses LXQt as its desktop environment, is what most of my non-desktop computers use. For example a media playback computer and a laptop are normally set up with Lubuntu which is quick and easy to install as it has all the drivers included in the setup.

For my desktops I am using Debian, with LXQt on one computer in particular because this computer needs stuff like hibernation that is better supported in Debian, but also with LXQt it gets around some of the bugs in KDE which I use on other desktops, as well as ensuring the resources that KDE uses aren't wasted in its lower spec. So I am testing these settings on that desktop because it has two screens and it is in fact the main computer that I have at the moment that these settings are important for.

Because I use this computer to do screen recording (using SimpleScreenRecorder), the screens have to be nearly at diagonal placement to each other. That is, the left hand screen and the right hand screen are to the left and right of each other, but in a vertical sense there is only a tiny overlap at the top right corner of the left screen and the bottom left corner of the right screen. This means the mouse can only be moved from one screen to the other by passing through a very small area of overlap that is in the top right corner of the left screen and the bottom left corner of the right screen, and also windows needing to be dragged from one screen to the other can be passed through this small area.

The reason for this configuration is that for screen recording, we have the software that is doing the recording running on the left (control) screen, recording the right (target) screen. We want to ensure what we need to do on the left screen does not cause anything unwanted to be displayed on the right screen. Some examples of this are:

• accidentally moving the mouse from the control screen onto the target screen
• any action on the control screen which causes pop up notifications, such as operating keyboard volume up and down. Even if the popup is on the control screen, if it is adjacent to an area of the target screen it can cause unwanted display of information on the target screen.

We ensure this by making sure as described above, minimum overlap between the two screens, just enough to get the mouse from one to the other and drag windows from one to the other. We still have to be careful not to run the control screen applications maximised to ensure the mouse is kept out of the top right corner of the control screen at all times other than actually going from one screen to the other.

The LXQt monitor settings GUI doesn't actually position the screens exactly as we would like to. They can be aligned vertically and horizontally close to the suggested outcome, but with a small amount of unwanted superimposition of the two screens. Fortunately you aren't limited to using that GUI to set positions. After changing the positions in the GUI and closing applying or saving settings, you can further tweak these with manual editing in the file ~/.config/lxqt/lxqt-config-monitor.conf. These seem to be just the same sort of parameter specs as running xrandr will spit out for you. Under the [currentConfig] section are the ones presently being used for the computer and with a few adjustments we can eliminate that superimposition and ensure the overlap is as small as possible. For example, with the control screen being screen 1 on HDMI-1 (settings\1) at a resolution of 1440x900, xPos=0 yPos=745 are the settings and for the target screen being screen 2 on HDMI-2 (settings\2) resolution 1360x768, xPos=1440 ypos=0. 

The names of the displays xrandr generates for this system are interesting as it has one HDMI port, one DVI port and one VGA port, yet the DVI port appears as an HDMI port in the list of outputs. It seems this initial configuration of the displays happens just after logon. Also, the settings are saved in the user's profile (~/.config folder) which means when reinstalling, these settings should be reused into the new installation of LXQt.

Using a Raspberry Pi as a livestream player [1]

When I wrote the first post of this series I fully expected using my former Windows 10 PC as a home theater PC would work out. That didn't last long and the Windows 10 part was soon moved to another computer so the Mini-ITX system could be reinstalled with Lubuntu for the specific role that it was needed for. That hasn't worked out either, in both cases the older low-spec hardware is the issue, not specifically because it is low-spec but because it is older. The AMD E350, while good for desktop use, is not recent enough to be able to decode modern video codecs fast enough to be able to give good video playback on Youtube and other live streams.

So as the updates to that post attest, I had moved to using the Antec chassis system with Lubuntu, but that has not worked out after about a month of use. The only reasonably cheap solution, therefore, is to move to using a Raspberry Pi for this application. Although my one and only Pi is being used at my desk for a particular application where it gets its internet connection off my phone's mobile data, I can use an old laptop for that particular purpose in the meantime while I use the Pi for the bedside PC application (for night prayer/intercession use), until I can get another Pi.

The model 3 B Pi is just slightly too underspec to keep up with all livestreams. It actually still does work very well with Youtube on 240p or lower, but at higher rates tends to drop out a little. That is OK for now, and far better than the E350. The Pi 4 which has just been released is available with 2 GB of onboard RAM at $85 for the board ($105 with 4 GB of RAM) and is much faster than the 3 B and has many enhancements such as dual mini-HDMI ports, USB C power , etc. To that price you have to add a case and power supply, which can be done in stages as funds permit. There is no real hurry on that as I can get by for now with the 3 B, which is currently powered off a spare Ipad power adapter. I could continue a similar arrangement with the Pi 4 and save on the cost of a power adapter, but it will be a toss up whether to go with 2 GB or 4 GB of RAM, and also which distro to run on it if I want to use Firefox, which is apparently unavailable for Raspbian.

So the Antec chassis will go back to being my Windows 10 PC, which in turn can be moved out of a bulky spare desktop chassis again.

Rsync Backup System [4]: Linux File and Directory Security Using ACLs

As per our series on Rsync backups, we desire to use a different user from the one that owns the home directory in order to ensure they have only the permissions they need when logging in remotely. At this point whilst by default the backup user could access many things on the computer, there were some files that they received permission denied errors for.

Whilst there is a simple file and directory permission scheme built into Linux, the acl extensions are very worthwhile as they give a lot more control, especially with the all-important ability to have inheritable permissions on a parent folder. In this case we'd want to give the backup user read-only permissions on the home directory that are inherited by all subdirectories and files as they are created.

Getting ACL is pretty simple, firstly you need to install the acl package using apt. The second requirement is to see if a volume is mounted with acl support. Use a command called tune2fs -l /dev/xxx (part of e2fsprogs package) and look for "Default mount options". If it says acl in there then the volume will be mounted by default with ACLs enabled. If this is not a default then you'd want to go into the volume mount entry in /etc/fstab and you can add acl to the mount options there. In this case because in fstab I used "defaults" as the mount options, acl is automatically turned on at mount time. If I needed to specify acl then I could change the word "defaults" to "defaults,acl" in the fstab entry to enable ACLs.

ZFS also has its own (more extensive) ACL support which is not covered by this post as we are referring to the existing filesystems which are ext4 on the computers being backed up in this case. I am not planning to change the existing RAID arrays from MDADM to ZFS but it could be an interesting consideration for the future should a disk array be replaced in any system.

Now how to configure ACLs for our requirement? Commands used to set and get ACLs are setfacl and getfacl respectively. To set up read access for backupuser on /home/patrick we need to use a command that looks like this:

setfacl -dR -m u:backupuser:rx /home/patrick

If we then use a ls -l (or ls -al) command in /home/patrick we can see next to the standard permissions a + sign which tells us that ACLs are set. In this case I can see whilst all directories have ACLs set, the files that are in each directory are not set with ACLs. So I would need to run another command to set these:

setfacl -R -m u:backupuser:rx /home/patrick/* /home/patrick/.* /home/patrick/*.*

The two different commands are as follows:

The first one using 

-d setting the default ACL for each directory (the default ACL to be applied to any new file or directory)

-R operating recursively on all subdirectories

-m modifying the ACLs

u:backupuser:r is user backupuser granting read permission.

The second one is practically the same except there is no default applied because it is for existing files and the paths following it are the different filespecs to match. 

Whenever a new file or directory is created in future it will inherit the default ACLs that have been defined using the first command which means backupuser will get the permissions it needs. Multiple paths to apply the ACLs to can be specified but only one user specification can be put in at a time.

ACLs override the default permissions system when they come into effect and so you need to have good knowledge of how Linux permissions work. In particular you need to ensure your user has x (execute) as well as r (read) permissions specified in ACLs for directories in order to be able to traverse directories, otherwise you will keep getting permissions issues. It took me a bit of work to get the ACLs set up but once they are done properly, the advantage is that inheritable permissions are possible, so for my backups, the backup user will have automatic rights to new files and folders that are created in future. In fact for any computer the way to make it happen is to set a default ACL on /home that gives the backup user rx permissions on everything below in future.

With the ACLs properly set the test backup was able to access all the files it needed to and complete the backup. Since I knew there were roughly 114,000 items waiting for backup, it was interesting to watch rsync at work discovering new files (a big increase from the roughly 7000 it had access to the first time). 

The means of doing the incrementals is still being worked out as it is not as straightforward as I had hoped. The log file is not consistent in its format (at least not so far) that would make it easy to determine which files successfully transferred. The option I am tending towards would probably consist of getting a list of files that have changed since the last backup, then getting rsync to copy one at a time, looking at the result code that it returns, and if there is no error then writing an extended attribute of the source file to indicate a successful backup result. The extended attributes will probably be backup.full = <date> and backup.incr = <date> and these two extended attributes will be written by the appropriate backup script. A full backup simply updates the attribute at backup completion without checking it. Whereas for an incremental, we compare the last modified date in the file with the date stored (backup.incr if set, otherwise backup.full) and do the backup if the file has been modified since the last backup date. So I expect to start more testing reasonably soon with developing scripts to perform these tasks.

But so far as the full backups go, it's doing pretty good. I have just set up another full backup and it's working very well. The second time setting it all up was so much smoother. It also looks like ZFS disk compression works very well and has achieved some useful saving but I will check out exactly what sort of improvement I am getting on these disks.

Rsync Backup System [3]: Using Rsync For Full Backup

So last time we talked about how to set up a single disk with ZFS to use compression. Having got our backup disks sorted, the next step is to work out how to use rsync to do the actual backups.

rsync is written by the same people that devised samba and is a very powerful piece of software. What we need to do is to first of all set up a dedicated backup user on the system that is to be backed up, that has only read access to the filesystem. Having this is extremely important in case we make a mistake and accidentally overwrite the files on the source filesystem (which is quite possible with such a powerful system).

Using useradd we can set up backupuser in this case with a very simple password consisting of four consecutive digits and then by default it will have read only access to other users' home drives.  This means it can read my home directory. backupuser in this case is used with SSH to access the home drive over the network for creating the backup.

The next step is to look at the required form of the rsync command. There are certainly a lot of options for this. Assuming the backup destination zpool has been mounted to /mnt/backup/fullbackup and the source is on 192.168.x.y at /home/patrick then a good starting point for the backup would be along the following lines:

rsync -arXvz --progress --delete backupuser@192.168.x.y:/home/patrick/ /mnt/backup/fullbackup/patrick --log-file=/home/patrick/rsync.log

This looks to cover all bases needed. The options are -a which means copy in archive mode (preserving all source file attributes. -r which means recurse into source directories, -v which means verbose and -z which means compress during transfers (which can speed things up). --progress means during each file transfer you will see the bytes transferred, speed, percentage and ETA which is useful for large files. --delete will remove extraneous files from destination directories (which means files that aren't in the source). This option is obviously useful when a file gets deleted or moved in the source. -X means to preserve extended file attributes if any exist.

At the moment I am testing this with a full backup of serverpc with this running in a terminal window. There are some issues with some directories not having permission for backupuser which so far has only affected a few hidden folders but will have to be looked into further. Previous backups always used patrick as the user but it is pretty important to have a special backup user with restricted permissions which is really a best practice for any kind of professional computer setup. An example is because a mistake with the rsync command could wipe out the source directory if there was read-write access to it.

After looking at log file options there is this option that we can set the log file format for rsync using an extra parameter and this is what I came up with in deciding what format of information would be useful in each line of the rsync log:

rsync -arXvz --progress --delete backupuser@192.168.x.y:/home/patrick/ /mnt/backup/fullbackup/patrick --log-file=/home/patrick/rsync.log --log-file-format "%a|%f|%M|%l|%b|%o|%U"

Using in particular the log file format option, it seems to be usefully logging the information we need for each file. However the %a logfile option does not actually seem to be recognised by rsync. 

There is one thing that rsync does not do by itself and that is incremental backups to a separate disk. rsync is designed to be able to by default create incremental backups in a separate directory from the one where the full backup is stored, but the full backup directory must be online at the time when the incremental is done so that it knows which files have changed. This is a problem when, as I intend, you want to be able to use separate disks for the incrementals. Here you see the fundamental issue with the Linux file system (at least ext4) that does not have the separate archive flag for a file that NTFS has. The argument goes that it isn't necessary but that archive bit is the way you can tell that a file has been modified since the backup last ran, and increment it. 

There are several possible solutions to this and one of them is to create an extended attribute for each source file. This is possible using the setfattr command. So we could create this on each of the source files following the full backup (it would have to be a separate script process following the execution of rsync). Maybe the script would be part of a verify process that we run after the backup is carried out, to verify that the source and destination files both exist, and then write the extended attribute to the source file. The issue is that it may well prove difficult to be sure the source file was backed up unless we can have a look at a log file and prove that it lists the source files and then feed that into a script that produces the extended attribute. Anyway, the point of this is to write an extended attribute to each source file that says when that file was last backed up. This could be a part of an incremental script that uses find to get all files that were last backed up since a certain date. Or we can just use a range of dates for the find command to get the incremental file list using the last modified file time. This will all be looked at in more detail when I start working on incrementals, because for now I am just doing a full backup like I did before with rdiff-backup.

HOWTO: Set font size in virtual consoles

In Debian you can have virtual terminal consoles, which are different from a terminal emulator window. Before Buster came along, a lot of commands could be run in a terminal emulator, but these days, more often than not, commands have to be run in a virtual console instead. The difference is that a virtual console is fullscreen, not a window, and doesn't run under the GUI, which goes into the background at that point.

You can have six separate consoles that you switch between by pressing Ctrl-Alt-Fx where x is 1 to 6, i.e. the F1 to F6 keys. To switch back to the GUI you press Ctrl-Alt-F7.

The next issue is the font size which I tend to find rather too small. Fortunately this is easy to fix. Use nano to edit /etc/default/console-setup and change the following lines to read as follows:

CODESET="Lat15"

FONTFACE="TerminusBold"

FONTSIZE="16x32"

This gives us quite a large easy to read font. 

Run the command setupcon to apply it immediately, it should become the permanent setting.

The fonts are all stored in /usr/share/consolefonts directory so you can look up what is available if you want something different. What I haven't done yet is work out if I can change the colour from white.

Rsync Backup System [2]: Using ZFS For Compressed Backup Disks

So last time I talked briefly about ideas for using Rsync to do my backups. Over time this will gel into a whole lot of stuff like specific scripts and so on. Right now there will be a few different setup steps to go through. The first stage is to come up with a filesystem for backup disks, which are removable, and this time around I am having a look at ZFS.

Basing on some stuff I found on the web, the first step is to install zfs on the computer that does the backups. Firstly the contrib repository needs to be enabled in Debian, and then after that we need to install the following packages:

apt install dpkg-dev linux-headers-$(uname -r) linux-image-amd64

apt-get install zfs-dkms zfsutils-linux

After completing these steps I had to run modprobe zfs as well to get it running for systemd (this was flagged by an error message from systemd). This only works on the current running session. To ensure there are no "ZFS modules are not loaded" errors after rebooting we need to edit /etc/modules and add zfs to the list of modules to be loaded at startup.

Since Debian Buster we have to run a lot of commands in a virtual console by pressing Ctrl-Alt-F1, that previously could be run in a terminal window. This is certainly the case for the zfsutils commands so the following steps need to be run in such a console.

The next step after installation is to look at the steps needed to create a filesystem on a disk and set up compression on it. In this case the existing disk is called spcbkup1 and its ext4 partition currently set up on it appears as /dev/sdb1 which means the actual disk is /dev/sdb.

After deleting the existing filesystem from /dev/sdb the following command creates the storage pool:
zpool create -m /mnt/backup/spcbkup1 spcbkup1 /dev/sdb

whereby the new storage "pool" (just a single disk which is not really what ZFS is made for but it can be done) is set up to mount to /mnt/backup/spcbkup1, is called spcbkup1 and is using the physical disk /dev/sdb.

Issuing the blkid command at this point tells me I have two new partitions. Apart from /dev/sdb1 which is of type "zfs_member" there is also another partition called /dev/sdb9 for some other purpose known to zfs. /dev/sdb1 is mounted up at this point and can be used as a normal disk.

To turn on compression we then use this command:
zfs set compression=lz4 spcbkup1

There is one more question and that is automounting. When you put an entry into /etc/fstab then the default for mounting is automatic. For a removable disk we don't want this, so the entries in /etc/fstab end up looking like this:
UUID=....     /mnt/backup/spcbkup1 ext4 noauto 0 2
whereby noauto as the options means it will not automount. Instead you have to manually mount it before use. Obviously you do this after inserting the disk. This requires me to hibernate or turn off the computer before inserting or removing the disk.

ZFS is a little different. We use zfs set mountpoint=none spcbkup1 to remove the mount point before removing the disk. Then later on when we want to put the disk back in we would use zfs set  mountpoint=/mnt/backup/spcbkup1 spcbkup1 to reset the mountpoint. As we can change mountpoints on the fly without a config file or needing to know a long UUID, this immediately and obviously lends itself to being able to mount more than one disk to the same mountpoint. In other words, I can dispense with a different mountpoint for each of the disks and mount them all to the same path. So for this backup scheme I will have two mountpoints for spcbkup. One mountpoint is for the backup disk(s) for the full backup, and the other is for the backup disk(s) for the incremental backup. These will look like:
/mnt/backup/spcbkup-full
/mnt/backup/spcbkup-incr

where the suffix is self explanatory hopefully
and as long as I swap the incremental disks in and out then the backups proceed automagically.

For this scheme we just have one backup disk for the full backup and we are sharing one backup disk for incrementals for all sources so the paths will look like
/mnt/backup/fullbackup
/mnt/backup/incrbackup

and the two incremental disk pools will be called incrbackup-odd and incrbackup-even which refers to the week number. The backup script will work out the week number itself and work accordingly. The plan is each incremental disk will contain two generations of backups and with two disks, there will accordingly be four incremental generations at any one time. In practice we take the week number and get the modulus after dividing by 4, which will give a number from 0 to 3, then the backup for that week is done into a folder on the disk called mainpc-x or serverpc-x or mediapc-x where x is the modulus. rsync will be set to ensure that any redundant files are deleted from the target at each sync and the week number is also used to calculate the date range for the command so that it knows how to handle file modification times to identify the files that have changed within that date range. The script automatically runs the same day and same time each week for the incrementals. At about every three months a full backup is done on a different day from the scripted day so the script is not disturbed, this amounts with three computers to one full backup each month.

It is important to unmount using the above commands before removing the disk otherwise the system will behave badly at next startup (although the Raidon disk caddies are theoretically hotswappable, in practice I am using them as non-hotswap for various reasons, so the system is shutdown or hibernated before changing disks).

So having worked out how to set up disks, the next step is to work out how to use rsync and I will be running a full backup of serverpc first of all.

How to create a desktop menu entry manually in Linux

This is kind of an odd thing to be posting about nowadays with Linux desktops mostly including a menu editor or installable third party packages like menulibre and alacarte being able to achieve the same outcome. However, LXQt does not include a menu editor that is comparable to the above options, and whilst both of the above can be installed in Lubuntu or Debian/LXQt, the results of running them are confusing. It is possible with PCManFM-Qt file manager to edit applications shortcuts directly in a specific area of the standard user interface but that is confusing and difficult to understand. Fortunately the majority of desktop environments support a standard method of creating shortcuts as .desktop files stored in a standard location.

The .desktop files are created in either of the following locations: /usr/share/applications or ~/.local/share/applications. The difference between these is that the former location is computer based and the latter is user based. For this example we have downloaded Firefox Developer and extracted it from the download file. This does not give us that menu shortcut so we have to create it. Typically we will copy the FFDE files to ~/Applications folder and create a local shortcut in ~/.local/share/applications which has a double advantage at reinstallation time that both the application and its shortcut are located in the user profile and therefore do not need to be reinstalled when the operating system is.

The typical format of the .desktop file looks like the following (in this case the actual parameters used to start Firefox Developer from where we installed it):

[Desktop Entry]
Categories=Internet
Comment=Firefox Aurora with Developer tools
Exec=/home/patrick/Applications/firefox-dev/firefox %u
Icon=/home/patrick/Applications/firefox-dev/browser/chrome/icons/default/default128.png
Name=Firefox Developer Edition
NoDisplay=false
StartupNotify=true
Terminal=0
TerminalOptions=
Type=Application
Version=1.0
 

Some of the above might not be needed.

The key issue, however, is that LXQt (at least on Lubuntu, but probably also on Debian) does not seem to recognise user specific icons as part of its menu. When one was created like the above, it ended up being put into the Other submenu rather than the Internet submenu. Likewise one could be created with menulibre and would be created in the same location by default (~/.local/share/applications) but ran into the same issue that LXQt would push it into the Other section of the menu. I am still experimenting to see how this can be overcome but for now, either method is possible but if both are creating the same outcome then using menulibre is obviously preferred as a GUI.

Using a Windows 10 computer as a media player

UPDATE 19-10-7: Due to cost the Win10PC was put into an old obsolete chassis and parked in a corner of the room so it can be used when needed but it will not sit on my desk and will hardly ever be used. This was accomplished using an identical spare G-E350 WIN8 mainboard and the boot disk removed from the Mini-ITX chassis and it works fine. The Mini-ITX chassis was then set up with Lubuntu using the 500 GB HDD that was in it for a data disk for Win10PC.  x11vnc has been installed to enable remote control for maintenance purposes and with Kodi it works a lot better than when it was running Windows 10.

UPDATE 19-10-6: Whilst this computer will do for now I am looking at replacing it when I have the resources with a new Raspberry Pi (the new RPI 4B has a significant performance improvement over the 3B that I already have) or NUC running Linux just because it is getting a bit long in the tooth with performance issues, but in reality most of these issues are related to Windows 10's excessive resource demands on it. The other option being to put the Windows 10 computer into another chassis and release this chassis for the spare board of the same type and run that on Linux so I will consider the options for that.

UPDATE 19-10-3: After the heap finally managed to update itself to Windows 10 release 1903 (a whole drama and story in itself, which has taken months of failing to install the update, and me deciding I just could not be bothered troubleshooting, then finally it managed to install itself successfully last night), Kodi started having playback problems only with videos, every few seconds the audio would drop out momentarily. The solution I found that worked was to go into Settings, Player Settings, Videos, change the rendering method to DXVA and turn off DXVA hardware acceleration.

Back in December 2014 I got an Antec Mini-ITX chassis to use with a Gigabyte GA-E350 WIN8 board that I had purchased. I actually had two of these boards (the other one is stored as a spare). Given it's almost five years I'll elaborate that I originally got these boards, which were very cheap since they were designed for low power compact computers needing a power supply of less than 50 watts and have an integrated AMD E350 CPU. These photos were taken at the time I built the system and are missing from the article that is on the blog and I need to put them back in it sometime.

The bare chassis looking through the grille on one side which is for cooling, through the I/O plate on the back.

 The partly assembled system. The board is installed and screwed in place and the internal cables (mainly front panel USB / switch / LED) are just loose waiting to be plugged into the board. You certainly need the low profile cooler on this board because a full size one would have trouble fitting inside.

 The system assembled and running, with the display showing a setup screen. The extra cables that weren't in the previous picture are for the hard drives.

 On the back of the chassis under a blank panel is the mounting plate for the 2.5" hard disks (there is room for two of them). You can see one is installed here with the power cable hanging loose in the 2nd mounting position. Behind the HDD is the bottom of the motherboard. The mounting plate is detachable because the disks sit on the inside of it and there are insulating pieces supplied to prevent the HDDs from shorting out on the metal mount plate.

Looking inside the complete system installing Windows. The cables have been tied back in place ready for the lid to be put on. The power input cable, which is for 19VDC from an external power supply that is supplied with the system, can be seen upper right. There is a small power supply board inside the chassis that generates the usual voltages (+/-12V +/-5V +3.3V etc) at up to 60 watts for the system with the usual ATX power connector.

Anyway if you followed this blog you'd know that this system has ended up as my one and only Windows computer and now has 10 Home running on it, and until recently was hardly ever used. But now I want to get some use out of it so I've been setting it up as a media player computer for bedside use, which means mostly playing music or playing videos with the screen turned off. The main issue with an older system like this one is the CPU won't have the codecs built in for playing a lot of video which means it can't handle the higher resolution or high bit rate videos well, since video playback on most operating system these days depends on the CPU doing hardware decoding, a key reason why I upgraded a Sandy Bridge system last year because it had trouble playing back WEBM videos. In this case for the use this system is being put to, that isn't a problem, and Youtube videos that stutter can be changed to a lower bit rate in the browser.

So what software do we want to use compared to what we can use in Linux? Kodi is available on Windows and it worked fine this time. I had trouble with it on a previous Windows installation where there was no sound. Windows Media Player has been out of Windows for so long that I have almost forgotten what it was like, and the alternative app built into 10 is nowhere near as good as Kodi. The other main application needed is a sync tool to sync the video and music libraries from another computer. Since I use NFS for my day to day networking stuff in Linux and MS doesn't make their NFS networking stuff available to Home Editions I had to set up a Samba share on the mediapc and network using the built in SMB/CIFS support for Windows. Karen's Replicator is the ideal program for syncing. I used to use this to back up my Windows computers back in the day and it does everything needed to get a full mirror that also replicates deletions on the source.

How to PERMANENTLY disable an input device in Linux (Xorg Display Server)

About 2 months ago I posted on how to use the xinput command (on Ubuntu and derivatives) to disable an input device. This, however, only works until the next time the computer is restarted. In addition, not all distros provide the xinput command; Debian doesn't, and I haven't done any research to determine whether this command needs to be installed or if there is a Debian specific alternative. For the time being, the device in which I need this function to work is my Dell E6410 laptop with an Alps pointing stick which is drifting. By default this is enabled alongside the touchpad, so you can use either of these devices, but there is no simple way to disable the touch stick, even if you plug in a USB mouse as I often do.

The answer for when the Xorg display server is used (generally this is the default display server based on X11 for many years, that is now gradually being superseded by Wayland) is to change the configuration settings in a file that is stored in /usr/share/X11/xorg.conf.d/ and in this case the file name to be edited is 40-libinput.conf . In order to effect the disablement, the xinput list command was first used to get the name of the touchpad device, and then this section making use of that name was added to the bottom of the aforementioned file:

Section "InputClass"
        Identifier "disable touch stick"
        MatchProduct "AlpsPS/2 ALPS DualPoint Stick"
        Option "ignore" "on"
EndSection

So that section in the file is pretty straightforward. The MatchProduct entry contains the exact device name string that we got from xinput list and the Option specified "ignore" "on" tells the Xorg server to ignore the device and not enable it.

After doing a reboot the touch stick was found to be out of operation and the xinput list command no longer lists this device.

As I noted in my previous post, the flaky touch stick on this laptop has been a significant issue since it will cause the mouse pointer to drift across the screen quite randomly even when an external mouse is in use and possible options to disable the touch stick at a hardware level (there is for example a Bios setting that in theory would disable it with an external mouse connected) have not worked so far. I did not investigate any possibility however of physically disconnecting the device inside the laptop, but I believe this could be quite difficult as the touch stick is physically located between the keyboard keys and thus it is probably at a hardware level integrated into the keyboard itself and therefore could not be unplugged.

Having this option to disable the touch stick is proving extremely useful since the laptop is otherwise in very good condition for its age and with the installation lately of a 160 GB SSD in place of the regular hard drive and the replacement a few years ago of the original battery, with the limited amount of use since then meaning the new battery should still have at least 90% of its life left, if carefully looked after it will keep going for another decade probably.

I am guessing this xorg.conf.d file entry will work with most distros (including Debian) even if xinput command doesn't work on some.

Setting up a second swap partition for Linux

In my last post I waxed lyrical about the merits of the Linux file system and how it is possible to have two swap partitions. Today I am setting up just that in my computer. It has an existing SSD with 100 GB available for swap, and due to an upgrade on the other computer, another 120 GB SSD has become available to install into it to add a second swap partition thereby more than doubling the swap space. This means the system should be able to edit very large Gimp files up to 50 GB whereas at the moment it can only handle ones up to about 20 GB because of all the other stuff I usually have running on it.

The first step to achieve this, having installed the disk and found it is in the system as /dev/sdc, is to make life easier and install GPartEd, the graphical partition editor (preferable to using command line in this instance). We can then use GPartEd to remove any existing partitions on the disk, and then allocate the entire space on it as a swap partition.

The next step is to run the mkswap command on our new partition (although it looks like GPartEd in fact already did this step)

mkswap /dev/sdc1

And then we can use the swapon command to activate it:

swapon /dev/sdc1

after which running swapon -s will list the swaps currently in use which we find lists both of them and also conveniently the KDE system load monitor widget tells me the swap is now 198 GiB in size (it was 87 GiB and the new SSD added 111 GiB).

To permanently add the second swap partition it needs to be put into /etc/fstab:

UUID=83c0e622-bcdc-41fd-a1ce-ec3f6c88bfcc none            swap    sw              0       0which is basically copying the previous swap file entry except for the UUID which we get from running blkid

Then I tested by two Gimp projects at the same time that are around 40 GB in total. Originally the first one would not even load in the system, while the second one would use the entire swap space. Loading both together only used less than half of the new swap space, making it possible to work on them without any concerns about the system running out of swap and crashing Gimp.

 

How to disable an input device in Lubuntu

I have a Dell Latitude E6410 laptop and it has one of those stick pointing devices built into the keyboard. They are just a little joystick that you can use to move the mouse except it is just a tiny little thing that fits between keys and is actually quite hard to use.

Anyway this week the stick thing started pushing the mouse to the top right hand corner all of the time. So I have looked up how to disable it from being in operation, since it was doing this with everything, even in the BIOS or the Linux installer.

So the command to use for (L)ubuntu is xinput so the first thing is to use its list parameter to get a list of devices. Each device will have an id number next to it, in this case 18.

Then the next thing is simply type in xinput disable 18 and that disables the device. Which means using the touchpad and an external USB mouse just got a whole lot easier. I'm not sure there is any hardware way like a key sequence you could press that would achieve this, so this was a great outcome, as I doubt there was anything else I could have done, although I was able to use the stick to move the mouse around. Without being able to resolve that the laptop would have been nearly unusable, which would be a concern as it is otherwise in extremely good condition.

Networking / Wireless with Debian LXQt

There are some issues possible with LXQt if you have Wifi because of limitations in the current Buster install. Some of these may be resolved by the time Buster is released...but not all, for reasons explained below.

First issue for Wifi is if the drivers are non-free, as is fairly likely. You'll notice this at setup if you get asked for a driver file to allow the wireless networking connection to be accessed (more likely if using the network based installation). This occurs because Debian is configured by default to have non-free stuff in separate repositories and access to those has to be specifically enabled in the sources.list file. Basically you have to change the first set of repository statements in the file that point to main, to point to main contrib non-free. This is due to the Debian licensing model and philosophy.

After that you can, for example, install firmware-iwlwifi which is the package for Intel wireless. 

The next issue is that there is no network connection manager installed by default in LXQt with Buster. Installing network-manager-gnome is the way to get this, along with nm-applet. Then you have to edit /etc/NetworkManager/NetworkManager.conf and change the following:

[ifupdown]
managed=false

change the false into true. This is necessary to ensure that you can manage the wired interface as by default you will not be able to manage this.

After that you should be able to get the wireless interface working on your computer. If you don't have wireless, having Network Manager installed is still useful if you need to set up anything non default with the wired networking such as a static IP address. 

UPDATE: I had a lot of trouble on this computer getting Network Manager to be able to set the IP address and parameters for the Ethernet cable port properly. The settings in the /etc/network/interfaces file read

iface enp3s0 inet dhcp

whereas in Network Manager I had changed to manual configuration which should have been shown in the file. So there is an ability to directly edit the parameters in that file so in our case we write something like

iface enp3s0 inet static
    address 192.168.1.222
    netmask 255.255.255.0

with no gateway or DNS servers or anything because this particular interface is only being used on the intranet and not to access the internet. But the question remains why Network Manager is not setting those parameters in the file. In other words it seems Network Manager was only able to configure the current session when it was used and not the configuration settings for next time. Network Manager was apparently able to handle the wireless connection. Possibly the issue of what is being managed in the Network Manager settings mentioned above.