MediaWiki API result

{
    "batchcomplete": "",
    "continue": {
        "gapcontinue": "UNE",
        "continue": "gapcontinue||"
    },
    "warnings": {
        "main": {
            "*": "Subscribe to the mediawiki-api-announce mailing list at <https://lists.wikimedia.org/postorius/lists/mediawiki-api-announce.lists.wikimedia.org/> for notice of API deprecations and breaking changes."
        },
        "revisions": {
            "*": "Because \"rvslots\" was not specified, a legacy format has been used for the output. This format is deprecated, and in the future the new format will always be used."
        }
    },
    "query": {
        "pages": {
            "4": {
                "pageid": 4,
                "ns": 0,
                "title": "Roy's notes",
                "revisions": [
                    {
                        "contentformat": "text/x-wiki",
                        "contentmodel": "wikitext",
                        "*": "= Page move =\n\nAfter borrowing space on [https://wiki.malinux.no/ Malin's wiki] for some years, further edits will be done here, so that next time the wiki goes down after something strange happened, she won't get a ton of mesages from me, but I'll have to fix it myself - oh well ;)\n\n= LVM, md and friends =\n\n== mdadm ==\n\n'''mdadm''' is Linux' software raid management tool, controlling this through the underlying kernel '''md''' driver. Through use of various RAID levels, you can achive a stable and fast system where you may be able to lose a drive or two without losing data.\n\n''Please keep in mind that RAID is '''not''' backup. If something goes wrong (if you do something stupid, if more than x drives die, if the house burns down etc), you'll still need an external backup to get your data back.''\n\n=== RAID levels ===\n\nRAID is a way to combine several disks of the same size to a single blockdevice, that the OS understands as a disk. If you use disks of varying size, the array will only use the amount per drive equal to the smallest drive, per disk, in the RAIDset.\n\nThis is better explained on [https://en.wikipedia.org/wiki/Standard_RAID_levels wikipedia], but I'll summerise as best I can below.\n\n==== RAID-0 ====\n\nA RAID-0 or stripeset is a way to add two or more drives to a single md device without any redundancy. The pros are it is simple and fast and you get to use all the disk space available. The con is, if you lose a single drive, you lose the whole lot. Please only use this for testing, caching or other unimportant.\n\n==== RAID-1 ====\n\nRAID-1 or a mirror is a way to combine two or more drives to a single md device with a total space as a single drive in the mirror. Typically, two drives are used in a mirror, but some use more. With RAID-1, you get the same write speed as a single drive and a bit more for reads, as Linux will attempt to read from both (or all) sides of the mirror in parallel. For real world data, this is not linear, though. You don't get double the read speed with two drives in a mirror and three with a three-way mirror, but still, it'll help, especially for random I/O.\n\n==== RAID-2 and 3 ====\n\nAncient stuff, not supported by md.\n\n==== RAID-4 ====\n\nSame principle as RAID-0, only here we have an additional disk for storing parity data. With this setup, you need two or more drives and can lose any of them. If the parity data goes away, you still have the data and if one of the data drives dies, its data be reconstructed from parity.\n\n==== RAID-5 ====\n\nRAID-5 is mostly the same as RIAD-4 except that here, we have rotating parity. Since RAID-4 uses a dedicated parity disk, this gets far more writes than the data disks and becomes a bottleneck. RAID-5 solves this.\n\n==== RAID-6 ====\n\nIn RAID-6, we take RAID-5 to a new dimension, literally, adding another set of parity.\n\n==== RAID-10 ====\n\nRAID-10 is a non-standard RAID level some call RAID-1e or similar. It is like a mirror, but can grow over an odd number of drives and 'mirror' across them (see wikipedia link above for details).\n\n=== Creating a mirror ===\n\n''To be continued''\n\n== Linux' Logical Volume Manager (LVM) ==\n\n=== LVM in general ===\n\nLVM is designed to be an abstraction layer on top of physical drives or RAID, typically [https://raid.wiki.kernel.org/index.php/RAID_setup mdraid] or [https://help.ubuntu.com/community/FakeRaidHowto fakeraid]. Keep in mind that fakeraid should be avoided unless you really need it, like in conjunction with dual-booting linux and windows on fakeraid. LVM broadly consists of three elements, the \"physical\" devices (PV), the volume group (VG) and the logical volume (LV). There can be multiple PVs, VGs and LVs, depending on requirement. More about this below. All commands are given as examples, and all of them can be fine-tuned using extra flags in need of so. In my experience, the defaults work well for most usecases.\n\nI'm mentioning filesystem below too. Where I write ext4, the samme applies to ext2 and ext3.\n\n==== Create a PV ====\n\nA PV is the \"physical\" parts. This does not need to be a physical disk, but can also be another RAID, be it an mdraid, fakeraid, hardware raid, a virtual disk on a SAN or otherwisee or a partition on one of those. \n\n<syntaxhighlight lang=\"bash\">\n#  These add three PVs, one on a drive (or hardware raid) and another two on mdraids.\npvcreate /dev/sdb\npvcreate /dev/md1 /dev/md2\n</syntaxhighlight>\n\nFor more information about pvcreate, see [https://linux.die.net/man/8/pvcreate the manual].\n\n==== Create a VG ====\n\nThe volume group consists of one or more PVs grouped together on which LVs can be placed. If several PVs are grouped in a VG, it's generally a good idea to make sure these PVs have some sort of redundancy, as in mdraid/fakeraid or hwraid. Otherwise it will be like using a [https://en.wikipedia.org/wiki/RAID RAID-0] with a single point of failure on each of the independant drives. LVM has [https://unix.stackexchange.com/questions/150644/raiding-with-lvm-vs-mdraid-pros-and-cons  RAID code in it] as well, so you can use that. I haven't done so myself, as I generally stick to mraid. The reason is mdraid is, in my opinion older and more stable and has more users (meaning bugs are reported and fixed faster whenever they are found). That said, I beleive the actual RAID code used in LVM RAID are the same function calls as for mdraid, so it may not be much of a difference. I still stick with mdraid. To create a VG, run\n\n<syntaxhighlight lang=\"bash\">\n# Create volume group my_vg\nvgcreate my_vg /dev/md1\n</syntaxhighlight>\n\nNote that if vgcreate is run with a PV (as /dev/md1 above) that is not defined as a PV (like above), this is done implicitly, so if you don't need any special flags to pvcreate, you can simply skip it and let vgcreate do that for you.\n\n==== Create an LV ====\n\nLVs can be compared to partitions, somehow, since they are bounderies of a fraction or all of that of a VG. The difference between them and a partition, however, is that they can be grown or shrunk easily and also moved around between PVs without downtime. This flexibility makes them superiour to partitions as your system can be changed without users noticing it. By default, an LV is alloated \"thickly\", meaning all the data given to it, is allocated from the VG and thus the PV. The following makes a 100GB LV named \"thicklv\". When making an LV, I usually allocate what's needed plus some more, but not everything, just to make sure it's space available for growth on any of the LVs on the VG, or new LVs.\n\n<syntaxhighlight lang=\"bash\">\n# Create a thick provisioned LV named thicklv on the VG my_vg\nlvcreate -n thicklv -L 100G my_vg\n</syntaxhighlight>\n\nAfter the LV is created, a filesystem can be placed on it unless it is meant to be used directly. The application for direct use include swap space, VM storage and certain database systems. Most of these will, however, work on filesystems too, although my testing has shown that on swap space, there is a significant performance gain for using dedicated storage without a filesystem. As for filesystems, most Linux users use either ext4 or xfs. Personally, I generally use XFS these days. See my notes below on filesystem choice.\n\n<syntaxhighlight lang=\"bash\">\n# Create a filesystem on the LV - this could have been mkfs -t ext4 or whatever filesystem you choose\nmkfs -t xfs /dev/my_vg/thicklv\n</syntaxhighlight>\n\nThen just edit /etc/fstab with correct data and run mount -a, and you should be all set.\n\n==== Create LVM cache ====\nLVMcache is LVM's solution to caching a slowish filesystem on spinning rust with the help of much faster SSDs. For this to work, use a separate SSD or a mirror or two (just in case) an add the new disk/md dev to the same VG. In this case, we use a small mirror, md1. LVM is not able to cache to a disk or partition outside the volume group.\n\n<syntaxhighlight lang=\"bash\">\nvgextend data /dev/md1\n  Physical volume \"/dev/md1\" successfully created.\n  Volume group \"data\" successfully extended\n</syntaxhighlight>\n\nThen create two LVs, one for the data (contents of files) and one for the metadata (filenames, directories, attributes etc). Typically, the metadata part won't need to be very large. 100M should usually do unless you have ekstremely many small files. I guess there are ways to calculate it, but again, 1GB should do for everyone (or was that 640k?). As for the data cache, I chose to use 40G here for a 15TB RAID. It will only cache what actively use anyway, so no need for overkill.\n\n<syntaxhighlight lang=\"bash\">\nlvcreate -L 1G -n _cache_meta data /dev/md1\nlvcreate -L 100G -n _cache data /dev/md1\n\n# If you want to use the rest of the cache device, here /dev/md1, replace -L 100G with -l +100%PVS, *not*, -l +100%FREE, as that will use all the available space in the VG regardless of there being a PV mentioned in the config.\n\n# Some would want --cachemode writeback, but if paranoid, I wouldn't recommend it. Metadata or data can be easily\n# corrupted in case of failure. On the other hand, most filesystems will recover from this these days since they use\n# journaling and writeback caching ''really'' speeds up things.\nlvconvert --type cache-pool --cachemode writethrough --poolmetadata data/_cache_meta data/_cache\nlvconvert --type cache --cachepool data/_cache data/data\n</syntaxhighlight>\n\nThat should be it. If you benchmarked the disk before this, you should try again and you'll probably get a nice surprise. If not, well, see below how to turn this off again :)\n\n==== Disabling LVM cache ====\n\nIf you want to change the cached LV, such as growing og shrinking or otherwise, you'll have to disable caching first and then re-enable it. There's no quick-and-easy fix, but you just do as you did when enabling in the first place. \n\nForst, disable caching for the LV. This will do it cleanly and remove the LVs earlier created for caching the main device. \n\n<syntaxhighlight lang=\"bash\">\nlvconvert --uncache data/data\n</syntaxhighlight>\n\nUncaching an LV can sometimes lead to lvconvert hanging, printing \"uncaching block xxx\" repeatedly without the number increasing. If this happens, the solution I found was the \"windows solution\". Interrupt the uncaching with ctrl+c (it's made to do that nicely) and give the machine a reboot.\n\nNow, since you now have a VG with an empty PV in it, earlier used for caching, I'd '''really''' recommend removing this for now to avoid expanding onto that as well. Since they're in the same VG, this might happen and if you get so far as to extend the lv and the filesystem on it, and this filesystem is XFS or similar filesystems without possibilities of reducing the size, you have a wee problem.\n\nRemove that PV from the VG\n\n<syntaxhighlight lang=\"bash\">\nvgreduce data /dev/md1\n</syntaxhighlight>\n\nThe PV is still there, but pvs should now show it not connected to a VG\n\n''PS: I once made this error after first extending a 2x16TB RAID10 to 3x16TB, removed the cache to allow lvextend to work, extended -l +100%FREE and xfs_growfs /dev/that/dev and then - oops - I had a filesystem on the raid *plus* a separate SSD. Since XFS can't be shrunk, and since there was a ton of data there and a full restore would take a week or three, I ended up getting yet another 16TB drive before doing it all the right way. So don't do this at home\u2122 (or any other place).''\n\n==== Growing devices ====\n\nLVM objects can be grown and shrunk. If a PV resides on a RAID where a new drive has been added or otherwise grown, or on a partition or virtual disk that has been extended, the PV must be updated to reflect these changes. The following command will grow the PV to the maximum available on the underlying storage. \n\n<syntaxhighlight lang=\"bash\">\n# Resize the PV /dev/md (not the RAID, only the PV on the RAID) to its full size\npvresize /dev/md1\n</syntaxhighlight>\n\nIf a new PV is added, the VG can be grown to add the space on that in addition to what's there already. \n\n<syntaxhighlight lang=\"bash\">\n# Extend my_vg to include md2 and its space\nvgextend my_vg /dev/md2\n</syntaxhighlight>\n\nWith more space available in the VG, the LV can now be extended. Let's add another 50GB to it.\n\n<syntaxhighlight lang=\"bash\">\n# Extend thicklv - add 50GB. If you know you won't need the space anywhere else, you may want to\n# lvresize -l+100%FREE instead. Keep in mind the difference between -l (extents) and -L (bytes)\nlvresize -L +50G my_vg/thicklv\n</syntaxhighlight>\n\n''PS: Cached LVs can't be shrunk directly, you'll have to uncache them first. Please see above''\n\n==== Growing filesystems ====\nAfter the LV has grown, run ''xfs_growfs'' (xfs) or ''resize2fs'' (ext4) to make use of the new data. To grow the filesystem to all available space on ext4, run the below command.  To partly grow the filesystem or to shrink it or otherwise, please see the manuals.\n\nThe filesystem can be mounted when this is run. If you for some reason get an '''access denied''' error when running this as root or with sudo, it's likely caused by a filesystem error. To fix this, unmount the filesystem first and run '''fsck -f /dev/my_vg/thicklv''' and remount it before retrying to extend it.\n\n<syntaxhighlight lang=\"bash\">\nresize2fs /dev/my_vg/thicklv\n</syntaxhighlight>\n\nSimilarly, with XFS, but note that xfs_growfs doesn't take the device name, but the filesystem's mount point. In the case of XFS, also note that the filesystem '''''must''''' be mounted to be grown. This, in contrast to how it was in the old days when filesystems had to be unmounted to be grown.\n\n<syntaxhighlight lang=\"bash\">\nxfs_growfs /my_mountpoint\n</syntaxhighlight>\n\n==== Shrinking filesystems ====\nSome filesystems, like ext4, support shrinking, meaning you can reduce the size of the filesystem and then shrink the size of the LV. XFS does not support this and neither do any other current Linux filesystems, so I'll stick to ext4 here.\n\nThere are probably a lot of ways to do this, but my advice is:\n\n* Shrink filesystem to its minimum\n* Shrink the lv to the wanted size\n* Grow the filesystem to maximum available size (in the lv)\n\nTo do this, you'll need to unmount the filesystem first, since this operation can't be done while its mounted, unlike the other operations I have discussed above. If the filesystem is the root filesystem or you somehow can't unmount it because its got open files, you'll need to boot into single user mode (aka rescue mode). If it isn't the rootfs, check first with '''lsof''' to see if you can find open files on it and stop the processes holding those. Try to umount again and if that doesn't work, you'll need to boot into single. After that, it's quite straight forward. I'll use '''vg''' as the VG name here and '''lv''' as the LV name for simplicity.\n\n<syntaxhighlight lang=\"bash\">\numount /dev/vg/lv # if this succeeds, continue\nfsck -f /dev/vg/lv # if this fails, try again. If things fail badly, you'll need to sort those out before going on\nresize2fs -M /dev/vg/lv # resize to minimum.\nlvresize -L <new size> /dev/vg/lv # keep in mind that if the new size is less than your current filesystem size, as reported by resize2fs, you *will* see data corruption, so make sure it's at least the current size or perhaps a few megs larger, just in case.\nresize2fs /dev/vg/lv # grow the filesystem to fill the lv\nfsck -f /dev/vg/lv # just doublecheck\nmount /dev/vg/lv # mount it according to /etc/fstab and check if it looks right\n</syntaxhighlight>\n\n==== Rename system VG ====\n\nSome distros create VG names like those-from-a-sysadmin-with-a-well-developed-paranoia-not-to-hit-a-duplicate. Debian, for instance, uses names like \"my-full-hostname-vg\". Personally, I don't like these, since if I were to move a disk or vdisk around to somewhere else, I'd make sure not to add a disk named 'sys' to an existing system. If you do, most distros will refuse to use the VGs with duplicate names, resulting in the OS not booting until either one is specified by its UUID or just one removed. As I'm aware of this, I stick to the super-risky thing of all system VGs named 'sys' and so on.\n\nIf you do this, keep in mind that it may blow up your computer and take your girl- or boyfriend with it or even make either of them pregnant, allowing Trump to rule your life and generally make things suck. You're on your own!\n\n<syntaxhighlight lang=\"bash\">\n# Rename the VG\nvgrename my-full-hostname-vg sys\n</syntaxhighlight>\n\nNow, in /boot/grub/grub.cfg, change the old lv path from something like '/dev/mapper/debian--stretch--machine--vg-root' to your new and fancy '/dev/sys/root. Likewise, in /etc/fstab, change occurences of '/dev/mapper/debian--stretch--machine--vg-' (or similar) with '/dev/sys/'. Here, this was like this - the old ones commented out.\n\n<syntaxhighlight lang=\"fstab\">\n#/dev/mapper/debian--stretch--box--vg-root      /               ext4            errors=remount-ro,discard       0       1\n/dev/sys/root                                   /               ext4            errors=remount-ro,discard       0       1\n# /boot was on /dev/vda1 during installation\nUUID=myverylonguuidwithatonofgoodinfoinit       /boot           ext2            defaults                        0       2\n#/dev/mapper/debian--stretch--box--vg-swap_1    none            swap            sw                              0       0\n/dev/sys/swap_1                                 none            swap            sw                              0       0\n</syntaxhighlight>\n\nUpdate /etc/initramfs-tools/conf.d/resume with similar data for swap.\n\nYou might want to run 'update-initramfs -u -k all' after this, but I'm not sure if it's needed.\n\nNow, pray to the nearest god(s) and give it a reboot and it should (probably) work.\n\nAfter reboot, run '''swapoff -a''', then '''mkswap /dev/sys/swap_1''' (or whatever it's called on your system) and '''swapon -a''' again. You may want to give it another reboot or two for kicks, but it should work well even without that.\n\n=== Migration tools ===\n\nAt times, storage regimes change, new storage is added and sometimes it's not easy to migrate with the current hardware or its support systems. Once I had to migrate a 45TiB fileserver from one storage system to another, preferably without downtime. The server originally had three 15TiB PVs of which two were full and the third half full. I resorted to using [https://linux.die.net/man/8/pvmove pvmove] to just move the data on the PVs in use to a new PV. We started out with creating a new 50TiB PV, sde, and then to pvmove.\n\n<syntaxhighlight lang=\"bash\">\n# Attach /dev/sde to the VG my_vg\nvgextend my_vg /dev/sde\n\n# Move the contents from /dev/sdb over to /dev/sde block by block. If a target is not given in pvmove,\n# the contents of the source (sdb here) will be put somewhere else in the pool.\npvmove /dev/sdb /dev/sde\n</syntaxhighlight>\n\nThis took a while (as in a week or so) - the pvmove command uses old code and logic (or at least did that when I migrated this in November 2016), but it affected performance on the server very little, so users didn't notice. After the first PV was migrated, I continued with the other two, one after the other, and after a month or so, it was migrated. We used this on a number of large fileservers, and it worked flawlessly. Before doing the production servers I also tried interrupting pvmove in various ways, including hard resets, and it just kept on after the reboot.\n\n'''''NOTE:''''' ''Even though it worked well for us, '''always''' keep a good backup before doing this. Things may go wrong, and without a good backup, you may be looking for a hard-to-find new job the next morning''.\n\n=== Thin provisioned LVs ===\n\nThin provisioning on LVM is a method used for not allocating all the space given to an LV. For instance, if you have a 1TB VG and want to give an LV 500GB, although it currently only uses a fraction of that, a thin lv can be a good alternative. This will allow for adding a limit to how much it can use, but also to let it grow dynamically without manual work by the sysadmin. Thin provisioning adds another layer of abstaction by creating a special LV as a ''thin pool'' from which data is allocated to the ''thin volume(s)''\n\n==== Create a thin pool ====\n\nAllocate 1TB to the thin pool to be used for thinly provisioned LVs. Keep in mind that the space allocated to the thin pool is in fact thick provisioned. Only the volumes put on ''thinpool'' are thin provisioned. This will create two hidden LVs, one for data and one for metadata. Normally the defaults will do, but check the manual if the data doesn't match the usual patterns (such as billions of files, resulting in huge amounts of metadata). I ''beleive'' the metadata part should be resizable at a later time if needed, but I have not tested it.\n\n<syntaxhighlight lang=\"bash\">\n# Create a pool for thin LVs. Keep in mind that the pool itself is not thin provisioned, only the volumes residing on it\nlvcreate --size 1T --type thin-pool --thinpool thinpool my_vg\n</syntaxhighlight>\n\n==== Create a thin volume ====\n\nYou have the thinpool, now put a thin volume on it. It will allocate some space for metadata, but probably not much (a few megs, perhaps).\n\n<syntaxhighlight lang=\"bash\">\n# Now, create a volume with a virtual (-V) size of half a terabyte named thin_pool, but using the thinpool (-T) named thin_pool for storage\nlvcreate -V 500G -T my_vg/thin_pool --name thinvol\n</syntaxhighlight>\n\nThe new volume's device name will be /dev/thinvol. Now, create a filesystem on it, add to fstab and mount it. The '''df''' command will return available space according to the virtual size (-V), while lvs will show how much data is actually used on each of the thinly provisioned volumes.\n\n== Filesystem dilemmas ==\n\n=== XFS or ext4 or something else ===\nThe choice of filesystem varies for your use. Distros such as RHEL/CentOS has moved to XFS by default from v7. Debian/Ubuntu still sticks to ext4, like most other. I'll discuss my thoughts below on ext4 vs XFS and leave the other filesystems for later.\n\n==== bcachefs ====\n'''bcachefs''' is the latest newcomer, which started out as '''bcache''', a caching system for linux, using SSDs or NVMEs or similar to cache slower HDDs. It has later evolved to a full filesystem, supporting similar to mirroring, but across multiple drives of different sizes, as some may know the commercial '''unraid''' does. It supports full checksumming and healing of data when errors occur. It stripes data across available media, making sure there's at least two (or more, depending on configuration) drives holding the data to allow for failures. Bcachefs also supports similar to RAID levels similar to RAID5 or RAID6 with ''erasure coding''. The latter is not quite stable at the time of writing (2022-09-25), but should work well soon (fingers crossed).\n\n==== btrfs ====\n'''btrfs''', pronounced \"butterfs\" or \"betterfs\" or just spelled out, is a filesystem that mimics that of ZFS. It has been around for 10 years or so, and has proven stable as a dropin replacement for xfs and ext4 and so on, but its real magic in building a full stack with a full RAID subsystem, hasn't yet proven to be stable. In fact, after 15 years of development, it's hardly usable for that. So you might want to use it for its snapshots and compression, btrfs send/receive etc, but preferably with something below, like mdraid and lvm to help out with redundancy and perhaps SSD caching.\n\n==== ext4 ====\n'''ext4''' is probably the most used filesystem on linux as of writing. It's rock stable and has been extended by a lot of extensions since the original ext2. It still retains backward compatibility, being able to mount and fsck ext2 and ext3 with the ext4 driver. However, it doesn't handle large filesystems very well. If a filesystem is created for <16TiB, it cannot be grown to anything larger than 16TiB. This has changed around 2020, though, with mot filesystems created for 64bit use. You still can't change a 32bit filesystem to 64bit. One other issue is handling errors. If a large filesystem (say 10TiB) is damaged, an fsck may take several hours, leading to long downtime. When I refer to ext4 further in this text, the same applies to ext2 and ext3 unless otherwise specified.\nAs mentioned below, ext4 easily outperforms xfs on metadataoperations, so if you have a filesystem with millions of files, ext4 is your friend. However, ext4 still uses static inode allocation and if a filesystem's inode tables are full (one inode per file/directory/symlink etc), so is the filesystem. Read the manual and lookup the -N flag in mkfs.ext4.\n\n==== XFS ====\n'''XFS''', originally developed by SGI some time in the bronze age, but has been worked on heavily after that. RHEL and Centos version 7 and forward, uses XFS as the default filesystem. SuSE does the same. It works well, but for metadata operations, it's significantly slower than that of ext4, so if you need fast file access for millions of files on a filesystem, '''ext4''' is still your friend. Just read the notes in the above paragraphs. XFS still  scales well, but it lacks at least two things ext4 has. It cannot be shrunk (not that you normally need that, but nice if you have a typo or you need to change things). Also, it doesn't allow for automatic fsck on bootup. If something is messed up on the filesystem, you'll have to login as root on the console (not ssh), which might be an issue on some systems. The fsck equivalent, xfs_repair, is a *lot* faster than ext4's fsck, though.\n\n==== ZFS ====\nZFS is like a combination of RAID, LVM and a filesystem, supporting full checksumming, auto-healing (given sufficient redundancy), compression, encryption, replication, deduplication (if you're brave and have a ''ton'' of memory and can afford some risks) and a lot more. It's a separate chapter and needs a lot more talk than paragraph here.\n\n== Low level disk handling ==\n\nThis section is for low-level handling of disks, regardless of storage system elsewhere. These apply to mdraid, lvmraid and zfs and possibly other solutions, with the exception of individual drives on hwraid (and maybe fakeraid), since there, the drives are hidden from Linux (or whatever OS).\n\n=== S.M.A.R.T. and slow drives ===\nModern (that is, from about 1990 or later) have S.M.A.R.T. (or just smart, since it's easier to type) monitoring built in, meaning the disk's controller is monitoring itself. This is handy and will ideally tell you in advance that a drive is flakey or dying. Modern (2010+) smart monitoring is more or less the same. It can be trusted about 50% of the time. Usually, if smart detects an error, it's a real error. I don't think I've seen it reporting a false positive, but others may know better. \n\n==== smartmontools ====\nFirst, install smartmontool and run smartclt -H /dev/yourdisk (/dev/sda or something) to get a health report. Then perhaps run smartctl -a /dev/yourdisk and look for 'current pending sectors' This should be zero. Also check the temperature, which normally should be much above 50.\n\n==== single, slow drive ====\nWhat may happen, is smart not seeing anything and life goes on like before, only slower and less prouductive, without telling anyone. If you stubler over this issue, you'll probably see it during a large rsync/zfs send/receive or a resync/rebuild/resilver of the raid/zpool. During this, it feels like everything is slow and your RAID has turned into a RAIF (redundant array of inexpensive floppies). To check if you have a single drive messing up it all, check with '''iostat -x 2''', having 2 being the delay between each measurement. Interrupt it with ctrl+x (if you're like Neo) or ctrl+c if you're someone else. If there's a rougue drive there, it should stand out like sdg below\n\n avg-cpu:  %user   %nice %system %iowait  %steal   %idle\n            0.38    0.00    1.75    0.00    0.00   97.87\n\n Device            r/s     w/s     rkB/s     wkB/s   rrqm/s   wrqm/s  %rrqm  %wrqm r_await w_await aqu-sz rareq-sz wareq-sz  svctm  %util\n sda              0.00    0.00      0.00      0.00     0.00     0.00   0.00   0.00    0.00    0.00   0.00     0.00     0.00   0.00   0.00\n sdb              0.00    0.00      0.00      0.00     0.00     0.00   0.00   0.00    0.00    0.00   0.00     0.00     0.00   0.00   0.00\n sdc              0.00    0.00      0.00      0.00     0.00     0.00   0.00   0.00    0.00    0.00   0.00     0.00     0.00   0.00   0.00\n sdd              0.00    0.00      0.00      0.00     0.00     0.00   0.00   0.00    0.00    0.00   0.00     0.00     0.00   0.00   0.00\n sde              0.00    0.00      0.00      0.00     0.00     0.00   0.00   0.00    0.00    0.00   0.00     0.00     0.00   0.00   0.00\n sdf              0.00    0.00      0.00      0.00     0.00     0.00   0.00   0.00    0.00    0.00   0.00     0.00     0.00   0.00   0.00\n sdg              3.50    0.00   2352.00      0.00     0.00     0.00   0.00   0.00 14306.29    0.00  13.85   672.00     0.00 285.71 100.00\n sdh              0.00    0.00      0.00      0.00     0.00     0.00   0.00   0.00    0.00    0.00   0.00     0.00     0.00   0.00   0.00\n\nIn ZFS land, you should be able to get similar data with '''zpool iostat -v <pool>'''\n\nNow you know the bad drive (sdg), pull it from the raid with '''mdadm --fail /dev/mdX /dev/sdX'''. Now test the drive's performance manually, just simply:\n\n # hdparm -t /dev/sdg\n \n /dev/sdg:\n  Timing buffered disk reads:   2 MB in  5.37 seconds = 381.46 kB/sec\n\nBingo - nothing you'd want to keep. For a good drive, it should be something like 100-200MB/s\n\nPS: Keep in mind that you have a degraded RAID by now in case you had a spare waiting for things to happen.\n\nTry to replace the cable and/or try the disk on another port/controller. If the result from hdparm persists, it's probably a bad cable\n\nSo, then, just psycially locate the drive, pull it out, take out the discs and magnets and recycle the rest.\n\n=== \"Unplug\" drive from system ===\n<syntaxhighlight lang=\"bash\">\n# Find the device unit's number with something like\nfind /sys/bus/scsi/devices/*/block/ -name sdd\n# returning /sys/bus/scsi/devices/3:0:0:0/block/sdd here, \n# meaning 3:0:0:0 is the SCSI device we're looking for.\n\n# Given this is the correct device, and you want to 'unplug' it, do so by\necho 1 > /sys/bus/scsi/devices/3:0:0:0/delete\n</syntaxhighlight>\n\n=== Rescan disk controllers ===\nIf a drive is added and for some reason doesn't get detected, or is removed by the command above, it can be rediscovered with a sysfs command to the scsi host to which it is connected. Since there may be quite a few of these, an easy way is to just scan them all and check the output of '''dmesg -T''' after running it.\n\n<syntaxhighlight lang=\"bash\">\n# Make a list of controllers and send a rescan message to each of them. It won't do anything \n# for those where nothing has changed, but it will show new drives where they have been added.\n\nfor host_scan in /sys/class/scsi_host/host*/scan\ndo\n    echo '- - -' > $host_scan\ndone\n</syntaxhighlight>\n\n=== Rescan disks ===\nJust like disk controllers, if something changes, it isn't always visible without a reboot. This may be something like extending a vmware drive. If you add one, the linux guest will recognize it immediately, but if you resize it, this may not be the case. To force a rescan, try this.\n\n<syntaxhighlight lang=\"bash\">\nfor disk_rescan in /sys/class/block/sd*/device/rescan\ndo \n    echo 1 > $disk_rescan\ndone\n</syntaxhighlight>\n\nFor both this and the controller rescans, just run '''lsblk''' to check for updates. You should also find the changes in '''dmesg -T'''\n\n=== Fail injection with debugfs ===\n[https://lxadm.com/Using_fault_injection https://lxadm.com/Using_fault_injection]\n\n== mdadm stuff ==\n=== Resync ===\nTo resync a raid, that is, check, run\n\n echo check > /sys/block/$dev/md/sync_action\n\nwhere $dev is something like md0. If you have several raids, something like this should work\n\n for dev in md{0,1}\n do\n   echo repair > /sys/block/$dev/md/sync_action\n done\n\nAlso useful in a one-liner like\n\n for dev in md{0,1} ; do echo repair > /sys/block/$dev/md/sync_action ; done\n\nDifferent raids on different drives will do this in parallel. If the drives are shared somehow with partitions or similar, the check or repair will wait for the other to finish before going on.\n\nAs for repair vs check, [https://raid.wiki.kernel.org/index.php/RAID_Administration this article] describes it well. I stick to using repair unless it's something rare where I don't want to touch the data.\n\nIf you need to stop the check or repair, just\n\n echo idle > /sys/block/$dev/md/sync_action\n\n=== Spare pools ===\nIn the old itmes, mdadm supported only a dedicated spare per md device, so if working with a large set of drives (20? 40?), where you'd want to setup more raid sets to increase redundancy, you'd dedicate a spare to each of the raid sets. This has changed (some time ago?), so in these modern, heathen days, you can change mdadm.conf, usually placed under /etc/mdadm, and add 'spare-group=somegroup' where 'somegroup' is a string identifying the spare group. After doing this, run '''update-initramfs -u''' and reboot and add a spare to one of the raid sets in the spare group, and md will use that or those spares for all the raidsets in that group.\n\nAs some pointed out on #linux-raid @ irc.freenode.net, this feature is very badly documented, but as far as I can see, it works well.\n\n==== Example config ====\n\n<syntaxhighlight lang=\"bash\">\n# Create two raidsets\n\nmdadm --create /dev/md1 --level=6 --raid-devices=6 /dev/vd[efghij]\nmdadm --create /dev/md2 --level=6 --raid-devices=6 /dev/vd[klmnop]\n\n# get their UUID etc\n\nmdadm --detail --scan\nARRAY /dev/md/1 metadata=1.2 name=raidtest:1 UUID=1a8cbdcb:f4092350:348b6b80:c054d74c\nARRAY /dev/md/2 metadata=1.2 name=raidtest:2 UUID=894b1b7c:cb7eba70:917d6033:ea5afd2b\n\n# Put those lines into /etc/mdadm/mdadm.conf and and add the spare-group\n\nARRAY /dev/md/1 metadata=1.2 name=raidtest:1 spare-group=raidtest UUID=1a8cbdcb:f4092350:348b6b80:c054d74c\nARRAY /dev/md/2 metadata=1.2 name=raidtest:2 spare-group=raidtest UUID=894b1b7c:cb7eba70:917d6033:ea5afd2b\n\n# update the initramfs and reboot\n\nupdate-initramfs -u\nreboot\n\n# add a spare drive to one of the raids\n\nmdadm --add /dev/md1 /dev/vdq\n\n# fail a drive on the other raid\n\nmdadm --fail /dev/md2 /dev/vdn\n\n# check /proc/mdstat to see md2 rebuilding with the spare from md1\n\n</syntaxhighlight>\n\n=== Recovery ===\n\nThe other day, I came across a problem a user had on #linux-raid@irc.freenode.net. He had an old Qnap NAS that had died and tried plugging the drives in to his Linux machine and got various errors. The two-drive RAID-1 did not come up as it should, '''dmesg''' was full of errors from one of the drives (both connected on USB) and so forth.\n\nWe went on and started by taking down the machine and just connecting one of the drives. I had him check what was assembled with\n\n cat /proc/mdstat\n\nThat returned /proc/md127 assembled, but LVM didn't find anything. So running a manual scan\n\n pvscan --cache /dev/md127\n\nThis made linux find the PV, VG and a couple of LVs, but probably because the mdraid was degraded, the LVs were deactivated, according to '''lvscan'''. Activating the one with a filesystem manually\n\n lvchange -a y /dev/<vg>/<lv>\n\nSubstitute <vg> and <lv> with the names listed by vgs and lvs.\n\nThis worked, and the filesystem on /dev/<vg>/<lv> could be mounted correctly.\n\n=== Renaming a RAID ===\n\n''Not finished''\n<!--\n\nBack around 2001, I asked on the linux-raid list for disk tagging, not default back then, but supported, and thought of the possibility of disk #1 getting replaced with disk #1 from another raid because of a messup. After this, the hostname was added to the raid name in the superblock. So when a RAID is created, it is named \"hostname:raidname\", where the raidname is either a number (0,1...) or a name. \n\nFor instance, running this on my VM named \"raidtest\":\n\n mdadm --create --level=1 --raid-devices=2 /dev/md/stuff /dev/sd[st]\n\nmdadm --detail --scan now shows this\n\n ARRAY /dev/md/stuff metadata=1.2 name=raidtest:stuff UUID=c4cdf69f:393a28b1:47648ed3:726613de\n\nNow, you might want to change this name for some reason, and it's not supported under \"grow\", so what to do\u2026 The name is, after all, local to the disk, since it's there to avoid messing up a raid (or two) after a disk mixup.\n\n mdadm --assemble /dev/md/stuff --name 'someotherhost:stuff' --update=homehost /dev/sd[st]\n-->\n\n=== The badblock list ===\n'''md''' has an internal badblock list (BBL), added around 2010, to list the bad blocks on the disk to avoid using those. While this might sound like a good idea, the badblock list is updated in case of a read error, which can be caused by several things. To make things worse, the BBL is replicated to other drives when a drive fails or the array is grown. So if you start out with sda and sdb and sdb suffers some errors, nothing wrong, md will read from sda, but flag those sectors on sdb as bad. So after a while, perhaps sda dies and gets replaced, with the same content, including the BBL. Then, at some point you might to replace sdb and it, too gets the BBL. So it sticks - forever. Also, there's no real point of keeping a BBL list, since the drive has its own and even if a sector error occurs, the array will have sufficient redundancy to recover from that (unless you use RAID-0 which you shouldn't). If a drive finds a bad sector, it'll be reallocated by the drive itself without the user's/admin's/system's knowledge. If a sector can't be reallocated, it means the drive is bad and should be replaced.\n\nTo check if you have a badblock list on a drive, run\n\n<syntaxhighlight lang=\"bash\">\nmdadm --examine-badblocks /dev/sdX # replace X with something reasonable\n</syntaxhighlight>\n\nThere's no official way to remove it and although you can assemble it with the no-bbl option, this will only work if there's an empty BBL. Also, this will require downtime. As far as I can see, the only way to do this currently, is with a wee hack that should work. Just replace the names of mddev/diskdev with your own.\n\n<syntaxhighlight lang=\"bash\">\n#!/bin/bash\n\nmddev=\"/dev/md0\"\ndiskdev=\"/dev/sda\"\nmdadm $mddev --fail $diskdev --remove $diskdev --re-add $diskdev --update=no-bbl\n\n</syntaxhighlight>\n\nIf there's a BBL on the drive already, replace the --update part to 'force-no-bbl'. This is not documented anywhere as per se, but it works.\n \nI haven't found a way to turn this off permanently, but there may be something coming around when the debate on the linux-raid mailing list closes up on this. There's a bbl=no in mdadm.conf, but that's only for creation. I will continue extensive testing for this.\n\nFor more info about this topic, take a look at [https://raid.wiki.kernel.org/index.php/The_Badblocks_controversy this article]\n\n=== mdadm tweaks ===\n\n==== Migrate from a mirror (raid-1) to raid-10 ====\n\nCreate a mirror\n<syntaxhighlight lang=\"bash\">\nmdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sdb /dev/sdc\n</syntaxhighlight>\n\nLVM (pv/vg/lv) on md0 (see above), put a filesystem on the lv and fill it with some data.\n\nNow, some months later, you want to change this to raid-10 to allow for the same amount of redundancy, but to allow more disks into the system.\n\n<pre>\nmdadm --grow /dev/md0 --level=10\nmdadm: Impossibly level change request for RAID1\n</pre>\n\nSo - no - doesn't work. But - we're on \n\nSince we're on lvm already, this'll be easy. If you're using filesystems directly on partitions, you can do this the same way, but without the pvmove part, using rsync or whateever you like instead. I'd recommend using lvm for for the new raid, which should be rather obvious from this article. Now plug in a new drive and create a new raid10 on that one. If you two new drives, install both. \n\n<syntaxhighlight lang=\"bash\">\n# change \"missing\" to the other device name if you installed two new drives\nmdadm --create /dev/md1 --level=10 --raid-devices=2 /dev/vdd missing\n</syntaxhighlight>\n\nNow, as described above, just vgextend the vg, adding the new raid and run pvmove to move the data from the old pv (residing on the old raid1) to the new pv (on raid10). Afte rpvmove is finished (which may take awile, see above), just\n\n<syntaxhighlight lang=\"bash\">\nvgreduce raidtest /dev/md0\npvremove /dev/md0\nmdadm --stop /dev/md0\n</syntaxhighlight>\n\n\u2026and your disk is free to be added to the new array. If the new raid is running in degraded mode (if you created it with just one drive), better don't wait too long, since you don't have redundancy. Just mdadm --add the devs.\n\nIf you now have /dev/mdstat telling you your raid10 is active and have three drives, of which one is a spare, it should look something like this\n\n<pre>\nPersonalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]\nmd1 : active raid10 vdc[3](S) vdb[2] vdd[0]\n      8380416 blocks super 1.2 2 near-copies [2/2] [UU]\n</pre>\n\nJust mdadm --grow --raid-devices=3 /dev/md1\n\n''RAID section is not complete. I'll write more on migraing from raid10 to raid6 later. It's not straight forward, but easier than this one :)''\n\n==== Interrupting an mdadm --grow ====\nWhile working on a friend's machine, we started growing a raid-10 with two 18TB drives, adding a third. While this worked fine-ish some time, it was slow, possibly beause of a slow USB dock (although everything was USB3 according to lsusb etc). At the worst, we saw speeds of 200kB/s for resync, meaning it'd take 18 months or so to finish, which isn't very enjoying So, we'll need to see how md handles interruption of this resync.\n\nOn a similar setup of hers on a test vm, I added a disk to a 5-disk raid-10 and issued '''mdadm --grow --raid-devices 6''' and let it run for some seconds. I then rebooted the machine, '''reboot''', and waited for it to come up. It did, but the filesystem on the raid-10 wasn't mounted. Since it was flagged as nofail in fstab, it didn't bother the boot process, but it was nonetheless unavailable. The status in '''/proc/mdstat''' showed '''auto-read-only''' and '''pvs''' just showed is as '''(unknown)'''. I tried to '''echo check > /sys/block/md1/md/sync_action''', and this activated the RAID and it continued from where it was interrupted. Apparently, even sending it an '''idle''' command is sufficient to wake it up. After this, I could mount the filesystems s usual. I also tried '''reboot -f''' and '''echo b > /proc/sysrq-trigger''', the latter being the best analogue to using a hard reset switch. This [[wikipedia:Magic_SysRq_key|wikipedia article]] has more on sysrq. It all worked the same way and after checking sha512sum, the data stored on the original filesystem, was untouched.\n\n== Tuning ==\n\nWhile trying to compare a 12-disk RAID (8TB disks) to a hwraid, mdraid was slower, so we did a few things to speed things up:\n\nWhile testing with [https://linux.die.net/man/1/fio fio], monitor /sys/block/md0/md/stripe_cache_active, and see if it's close to /sys/block/md0/md/stripe_cache_size. If it is, double the size of the latter\n\n<syntaxhighlight lang=\"bash\">\necho 4096 > /sys/block/md0/md/stripe_cache_size\n</syntaxhighlight>\n\nContinue until stripe_cache_active stabilises, and then you've found the limit you'll need. You may want to double that for good measure. \n\n(to be continued)\n\n== Copying partition layout ==\n\nWhile it's strictly not related to md, it's mostly used there. If you have a, say, set of two 500GB drive in a mirror and one of them dies, you plugin a new 1TB drive and want to use partitions (typically for the boot raid), you may want to restrict the space used on the disk so that the leftovers can be used for other things. The best is then to just use '''sfdisk''' to copy the old drive's partition table to the new one. Later, you can just add more partitions on the free space on the larger one. We'll call the 500GB disk /dev/sdg and the 1TB disk /dev/sdh here, for reference.\n\n<pre>\n# sfdisk -d /dev/sdg | sfdisk /dev/sdh\n</pre>\n\nThat's it - just add /dev/sdh1 to the md raid.\n\n== Links ==\n\nBelow are a few links with useful info\n\n* [https://raid.wiki.kernel.org/index.php/Irreversible_mdadm_failure_recovery Irreversible mdadm failure recovery]\n\n= ZFS =\n\nZFS can do a lot of things most filesystems or volume managers can't. It checksums all data and metadata and so long that the system uses ECC enabled memory (RAM), it will have complete control over the whole data set, and when (not if) an error occurs, it'll fix the issue without even throwing a warning. To fix the issue, you'll obviously need sufficient redundancy (mirrors or RAIDzN). \n\n== ZFS send/receive between machines ==\n\nZFS has a send/receive mechanism that allows for snapshots to be sent over the wire to a receiving end. This can be a full filesystem, or an incremental change since last send/reveive. In a WAN scenario, you'll probably want to use VPN for this. On a controlled network, sending in cleartext is also possible. You may as well use ssh, but keep in mind that ssh was never designed to be used as a fullblown vpn solution and is just too slow or the task with large amounts of data. You may, though, use mbuffer, if on a loal LAN.\n\n<pre>\n# Allow traffic from the sending IP in whatever firewall interface you're using (if you're using a firewall at all, that is)\nufw allow from x.x.x.x\n# \n# Start the receiver first. This listens on port 9090, has a 1GB buffer,\n    and uses 128kb chunks (same as zfs):\n\nmbuffer -s 128k -m 1G -I 9090 | zfs receive data/filesystem\n</pre>\n\nTo be continued one day\u2026 See [https://everycity.co.uk/alasdair/2010/07/using-mbuffer-to-speed-up-slow-zfs-send-zfs-receive/ this] for some more. I'll get back with details on it.\n\n= Ceph =\n\nmanuell variant her:\nhttps://docs.ceph.com/en/latest/install/manual-deployment/\n\nGj\u00f8re ceph-deploy mulig:\n* lage brukere for ceph\n* gi nevnte brukere sudo\n* lage et keypair p\u00e5 noden som skal kj\u00f8re ceph-deploy\n* Legge inn dette i authorized_keys for brukeren p\u00e5 samtlige noder\n\nWorkarounds s\u00e5 langt:\n* Unsupported: Endre /etc/debian_version til 10 midlertidig\n* Resolve error: Kommenter vekk 127.0.1.1 i /etc/hosts\n\n<pre>\nceph-deploy install --no-adjust-repos raidtest.karlsbakk.net\n# eventuelt flere mon-instanser (quorum, dvs oddetall er best. 3 er et godt tall)\n# i s\u00e5 fall, endre ceph.conf i hjemmemappen til ceph-deploy sin bruker og legg til i initial_nodes\nceph-deploy mon create-initial\n\n# noder som skal kunne kj\u00f8re ceph-kommandoen (cluster admins)\nceph-deploy admin raidtest.karlsbakk.net\n\n# noder som skal kj\u00f8re ceph mds (metadata)\nceph-deploy mds raidtest.karlsbakk.net\n\n# noder som skal kj\u00f8re ceph mgr (mer metadata)\nceph-deploy mgr raidtest.karlsbakk.net\n\n# noder som skal tilby http-gateway inn mot objektlagringen (s3, swift)\nceph-deploy rgw raidtest.karlsbakk.net\n\n</pre>\n\nVise tilgjengelige disker \n\n<pre>\neph-deploy disk list raidtest.karlsbakk.net # list opp disker via ceph-deploy\n</pre>\n\nnuke partisjonstabell/lvm/osv p\u00e5 en eksisterende disk\n<pre>\nceph-deploy disk zap HOST DISK\n</pre>\n\n<pre>\nceph-deploy osd create --data /dev/sdh raidtest.karlsbakk.net\nceph-deploy osd create --data /dev/sdi raidtest.karlsbakk.net\n</pre>\n\n<pre>\nceph config set global osd_pool_default_size 2\n</pre>\n\n\nKonvertere crushmap til enkeltnodebruk:\nhttps://docs.ceph.com/en/latest/rados/operations/crush-map-edits/\nceph osd getcrushmap -o crushmap.bin\ncrushtool -d crushmap.bin -o crushmap.txt\n\n= General Linux system control =\n\n== Add more CPU cores (hotplug) ==\n\nIf working with virtual machines, adding a new core can be useful if the VM is slow and the load is multithreaded or multiprocess. To do this, add a new CPU in the hypervisor (be it KVM or VMware or Hyper-V or whatever). Some hypervisors, like VMware, will activate it automatically if open-vm-tools is installed. Please note that open-vm-tools is for VMware only. I don't know of any such thing for KVM or other hypervisors (although I beleive VirtualBox has its own set of tools, but not as a package). If this doesn't work, run this one to add all available cores.\n\n<pre>\nfor cpu in /sys/devices/system/cpu/cpu[0-9]*/online\ndo\n    state=$( cat $cpu )\n    if [ \"$state\" -eq 0 ]\n    then\n        echo 1 > $cpu\n    fi\ndone\n</pre>\n\nWhere X is the CPU number you want to add. You can 'cat /sys/devices/system/cpu/cpuX/online' to check if it's online.\n\n== Add more memory (hotplug) ==\n\nAs with cores above, the same applies to memory. If the guest can't be informed, you can add all available memory diretly with this\n\n<pre>\nfor f in /sys/devices/system/memory/memory[0-9]*/state\ndo\n    state=$( cat $f )\n    if [ \"$state\" = \"offline\" ]\n    then\n        echo online > $f\n    fi\ndone\n</pre>\n\n= Mount partitions on a disk image =\n\nSometimes you'll have a disk image, either from recovering a bad drive after hours (or days or weeks) of ddrescue, and sometimes you just have a disk image from a virtual machine where you want to mount the partitions inside the image. There are three main methods of doing this, which are more or less synonmous, but some easier than the other.\n\n== Basics ==\n\nA disk image is usually like a disk, consisting of either a large filesystem, a PV or a partition table with one or partitions onto which resides a filesystem, a pv, swap or something else. If it's partitioned, and you want your operating system to mount a filesystem or allow it to see whatever LVM stuff lies on it, you need to isolate the partitions. \n\n== Manual mapping ==\n\nIn the oldern days, this was done manually, something like this\n\n<pre>\nroot@mymachine:~# fdisk -l /dev/vda\nDisk /dev/vda: 25 GiB, 26843545600 bytes, 52428800 sectors\nUnits: sectors of 1 * 512 = 512 bytes\nSector size (logical/physical): 512 bytes / 512 bytes\nI/O size (minimum/optimal): 512 bytes / 512 bytes\nDisklabel type: dos\nDisk identifier: 0xc37b7ea5\n\nDevice     Boot    Start      End  Sectors  Size Id Type\n/dev/vda1  *        2048 50333311 50331264   24G 83 Linux\n/dev/vda2       50333312 52426369  2093058 1022M  5 Extended\n/dev/vda5       50333314 52426369  2093056 1022M 82 Linux swap / Solaris\nroot@mymachine:~#\n</pre>\n\nTo calculate the start and end of each partition, you just take the start sector and multiply it by the sector size (512 bytes here) and you have the offset in bytes. After this, it's merely an losetup -o <offset> /dev/loopX <nameofimagefile> and you have the partition mapped to your /dev/loopX (having X being something typically 0-255. After this, just mount it or run pvscan/vgscan/lvscan to probe whatever lvm config is there, and you should be able to mount it like any other filesystem.\n\n== kpartx ==\n\n'''kpartx''' does the same as above, more or less, just without so much hassle. Most of it should be automatic and easy to deal with, except it may fail to disconnect the loopback devices sometimes, so you may have to '''losetup -d''' them manually. See the manual, '''kpartx (8)'''. Please note that '''partx''' works similarly and I'd guess the authors of the two tools are arguing a lot of which one's the best.\n\n== guestmount ==\n\n'''guestmount''' is something similar again, but also supports file formats like '''qcow2''' and possibly other, proprietary filesystems like '''vmdk''' and '''vdi''', but don't take my word for it - I haven't tested. Again, see the manual or just read up on the description [http://ask.xmodulo.com/mount-qcow2-disk-image-linux.html?fbclid=IwAR3snTeZvGzp9PLdX11EQhCfOpux6I93CiJ3A2pUomkkRLly9Xo4851mkTY here]. It seems rather trivial.\n\n= Linux on laptops =\n\n== Wifi with HP EliteBook 725/745 on Ubuntu ==\n\nThe HP EliteBook 725/745 and similar are equipped with a BCM4352 wifi chip. As with a lot of other stuff from Broadcom, this lacks an open hardware description, so thus no open driver exist. The proper fix, is to replace the NIC with something with an open driver, but again, this isn't always possible, so a binary driver exists. In ubuntu, run, somehow connect the machine to the internet and run\n\n<pre>\nsudo apt-get update\nsudo apt-get install bcmwl-kernel-source\nsudo modprobe wl\n</pre>\n\nIf you can't connect the machine to the internet, download the needed packages on another machine and put it on some usb storage. These packages should suffice:\n\n<pre>\n# apt-cache depends bcmwl-kernel-source\nbcmwl-kernel-source\n  Depends: dkms\n  Depends: linux-libc-dev\n  Depends: libc6-dev\n  Conflicts: <bcmwl-modaliases>\n  Replaces: <bcmwl-modaliases>\n</pre>\n\nThen install manually with '''dpkg -i file1.deb file2.deb''' etc\n\nAfter this, ip/ifconfig/iwconfig shouldd see the new wifi nic, probably '''wlan0''', but due to a [https://bugs.launchpad.net/ubuntu/+source/broadcom-sta/+bug/1343151 old, ignored bug], you won't find any wifi networks. This is because the driver from Broadcom apparently does not support interrupt remapping, commonly used on x64 machines. To turn this off, change '''/etc/default/grub''' and change the line '''GRUB_CMDLINE_LINUX_DEFAULT=\"quiet splash\"''', adding intremap=off to the end before the quote: '''GRUB_CMDLINE_LINUX_DEFAULT=\"quiet splash intremap=off\"'''. Save the file and run '''sudo update-grub''' and reboot. After this, wifi should work well.\n\n== Wifi with HP Compaq Presario CQ57 on Ubuntu ==\n\nThe HP Compaq Presario CQ57 uses the RT5390 wifi chipset. This has been supported for quite some time now, and I was quite surprised to find it not working on a laptop a friend was having. The driver loaded correctly, but Ubuntu insisted of the laptop being in '''flight mode'''. Checking '''rfkill''', it showed me two NICs.\n\n<pre>\n# rfkill list all\n0: phy0: Wireless LAN\n\tSoft blocked: no\n\tHard blocked: yes\n1: hp-wifi: Wireless LAN\n\tSoft blocked: yes\n\tHard blocked: no\n</pre>\n\nTurning rfkill on and off resulted in nothing much, except some error messages in the kernel log (dmesg)\n\n<pre>\n[fr. jan. 24 15:10:20 2020] ACPI Error: Field [B128] at bit offset/length 128/1024 exceeds size of target Buffer (160 bits) (20170831/dsopcode-235)\n[fr. jan. 24 15:10:20 2020]\n                            Initialized Local Variables for Method [HWCD]:\n[fr. jan. 24 15:10:20 2020]   Local0: 00000000a34b7928 <Obj>           Integer 0000000000000000\n[fr. jan. 24 15:10:20 2020]   Local1: 00000000b0aa6865 <Obj>           Buffer(8) 46 41 49 4C 04 00 00 00\n[fr. jan. 24 15:10:20 2020]   Local5: 00000000a9811efd <Obj>           Integer 0000000000000004\n[fr. jan. 24 15:10:20 2020] Initialized Arguments for Method [HWCD]:  (2 arguments defined for method invocation)\n[fr. jan. 24 15:10:20 2020]   Arg0:   0000000078957d1b <Obj>           Integer 0000000000000001\n[fr. jan. 24 15:10:20 2020]   Arg1:   00000000725c9c6e <Obj>           Buffer(20) 53 45 43 55 02 00 00 00\n[fr. jan. 24 15:10:20 2020] ACPI Error: Method parse/execution failed \\_SB.WMID.HWCD, AE_AML_BUFFER_LIMIT (20170831/psparse-550)\n[fr. jan. 24 15:10:20 2020] ACPI Error: Method parse/execution failed \\_SB.WMID.WMAD, AE_AML_BUFFER_LIMIT (20170831/psparse-550)\n</pre>\n\nAfter upgrading BIOS and testing different kernels, I was asked to try to unload the '''hp_wmi''' driver. I did, and things changed a bit, so I tried blacklisting it, creating the file '''/etc/modprobe.d/blacklist-hp_wmi.conf''' with the following content\n\n<pre>\n# Blacklist this - it doesn't work!\nblacklist hp_wmi\n</pre>\n\nI gave the machine a reboot and afte that, the '''hp-wifi''' entry was gone in the rfkill output, and things works.\n\n= Raspberry pi =\n\nI couldn't quite decide if the raspberry pi should be a separate section or part of Linux, but hell, it can run more than Linux, although 99,lots% of people just uses Linux on them. This is a small list of things to know about them; things not on the front page of the manual or perhaps hidden even better.\n\n== Which pi? ==\n\nI just setup three raspberry pi machines and although some are quite different, like v1 to vEverythingelse or the Pi zero versions to the normal ones, not all of us remember how to distinguish between them, and sometimes they're in a nice 3d printed (or otherwise) chassis or otherwise hidden. So, how to check three different ones:\n\n<pre>\nroot@home-assistant:~ $ cat /proc/device-tree/model ; echo\nRaspberry Pi 2 Model B Rev 1.1\n\nroot@green-pi:~ $ cat /proc/device-tree/model ; echo\nRaspberry Pi 3 Model B Rev 1.2\n\nroy@yellow-pi:~ $ cat /proc/device-tree/model ; echo\nRaspberry Pi 4 Model B Rev 1.1\n</pre>\n\nWhile this works well, for the latter, the pi4, it doesn't tell how much memory I have. It comes in variants of 1, 2 and 4GB, and this is the latter.\n\nFor a more detailed list, the command '''awk '/^Revision/ {sub(\"^1000\", \"\", $3); print $3}' /proc/cpuinfo''' will give you a revision number, detailing the type of pi, who produced it etc. Please see [https://elinux.org/RPi_HardwareHistory https://elinux.org/RPi_HardwareHistory] for a full table.\n\n== Growing or shrinking the rootfs ==\n\n=== Inhibit growing the rootfs ===\n\n[https://www.raspberrypi.org/software/ Raspberry Pi OS] (former Raspbian) grows its partition table and rootfs to the media size (usually an SD card) at first boot. While this can be a good idea for most work, sometimes it's not. If you want to change this behaviour, mount the SD card's boot partition, typically a small partition at the start of the medium and change the file '''cmdline.txt'''. It should look something like this\n\n console=serial0,115200 console=tty1 root=PARTUUID=f4481065-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait quiet init=/usr/lib/raspi-config/init_resize.sh splash plymouth.ignore-serial-consoles\n\nTo \n\n console=serial0,115200 console=tty1 root=PARTUUID=f4481065-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait\n\n=== Post-install growing of the fs ===\n\nIf you have installed the system and not yet grewn the fs, you'll need to grow the partition table first. The most basic way to do this is to just remove the large partition holding the current OS and creating a new, larger one, starting at the ''exact'' same sector and then growing the filesystem in the partition.\n\n<pre>\nroot@raspberrypi:~# fdisk /dev/mmcblk0\n\nWelcome to fdisk (util-linux 2.33.1).\nChanges will remain in memory only, until you decide to write them.\nBe careful before using the write command.\n\n\nCommand (m for help): p\nDisk /dev/mmcblk0: 14.9 GiB, 15931539456 bytes, 31116288 sectors\nUnits: sectors of 1 * 512 = 512 bytes\nSector size (logical/physical): 512 bytes / 512 bytes\nI/O size (minimum/optimal): 512 bytes / 512 bytes\nDisklabel type: dos\nDisk identifier: 0xf4481065\n\nDevice         Boot  Start     End Sectors  Size Id Type\n/dev/mmcblk0p1        8192  532479  524288  256M  c W95 FAT32 (LBA)\n/dev/mmcblk0p2      532480 7774207 7241728  3.5G 83 Linux\n\nCommand (m for help): d\nPartition number (1,2, default 2): 2\n\nPartition 2 has been deleted.\n\nCommand (m for help): n\nPartition type\n   p   primary (1 primary, 0 extended, 3 free)\n   e   extended (container for logical partitions)\nSelect (default p):\n\nUsing default response p.\nPartition number (2-4, default 2):\nFirst sector (2048-31116287, default 2048): 532480\nLast sector, +/-sectors or +/-size{K,M,G,T,P} (532480-31116287, default 31116287): +4G\n\nCreated a new partition 2 of type 'Linux' and of size 4 GiB.\nPartition #2 contains a ext4 signature.\n\nDo you want to remove the signature? [Y]es/[N]o: n\n\nCommand (m for help): w\n\nThe partition table has been altered.\nSyncing disks.\n\nroot@raspberrypi:~# reboot\n</pre>\n\nAfter the machine is back up, open a terminal and you should see everything as before, except '''lsblk''' should show a larger '''mmcblk0p2''' partition. To resize the filesystem, just run this as root\n\n<pre>\nresize2fs /dev/mmcblk0p2\nresize2fs 1.44.5 (15-Dec-2018)\nFilesystem at /dev/mmcblk0p2 is mounted on /; on-line resizing required\nold_desc_blocks = 1, new_desc_blocks = 1\nThe filesystem on /dev/mmcblk0p2 is now 1048576 (4k) blocks long.\n\nroot@infoskjerm:~# df -h /\nFilesystem      Size  Used Avail Use% Mounted on\n/dev/root       3.9G  3.0G  739M  81% /\n</pre>\n\n== Monitoring ==\n\n'''vcgencmd''' from '''/opt/vc/bin''', but symlinked to '''/usr/bin''' so it's available in path, is useful for fetching hardware info about the pi. For example, measuring the temperature\n\n<pre>\nroot@yellow-pi:~# vcgencmd measure_temp\ntemp=63.0'C\n</pre>\n\nOr read directly from the sensor - result is given in millidegrees (Celsius)\n\n<pre>\nroot@lagus:~# cat /sys/class/thermal/thermal_zone0/temp\n50464\n</pre>\n\nThe command is generally badly documented and parts of it seems outdated for newer hardware. Here's the output from a Raspberry pi 4 with 4GB RAM\n\n<pre>\nroy@yellow-pi:~ $ vcgencmd get_mem arm\narm=948M\nroy@yellow-pi:~ $ vcgencmd get_mem gpu\ngpu=76M\n</pre>\n\n== Netbooting a pi ==\n\nOn Raspberry pi 2 (a somewhat late revision), 3 (preferably 3+) and 4, you can boot directly off the net. To do this, you need to prepare the pi for netboot, to setup a boot server in the dhcp scope on the dhcp server and setup an NFS server from which to boot. I have a router running OpenWRT, which uses dnsmasq to serve DHCP and I've setup a separate linux vm to serve the boot files over NFS, using LVM with read-write snapshots for each clients, as this serves the role of thin provisioning the system. Details follow below. \n\n=== Configuring the pi for netboot ===\nThe command '''rpi-eeprom-config''' is used to setup the pi's eeprom (somewhat similar to that of the BIOS in a PC). We're mostly interested in the boot order here, and several modes may (and should) be given, telling the pi to start trying the first one and then go on further. The default value on a pi4 is 0xf41, so decoded from right, it says \"Try SD card, then USB mass storage before falling back to a reboot, restarting the cycle. I chose 0xf241 to allow sd card booting if present, but then falling back to networking. To change this, run\n\n<pre>\n$ sudo rpi-eeprom-config -e\n</pre>\n\nand edit the fields you want to change.\n\n'''''NOTE:''''' ''I have not tried setting the boot order to 0xf or something like that and I don't know if this will render the pi unsable or not. Better not try it yourself unless you're knee deep in old pi's.\n\nAnd then, give the pi a reboot.\n\nI'll start by going through pi4, since that's what I have right now. This comes with booting supported out of the box, but not enabled. \n{| class=\"wikitable sortable\"\n|+ Raspberry pi 4 boot modes\n|-\n! Value !! Mode !! Description\n|-\n| 0x1 || SD CARD || SD card (or eMMC on Compute Module 4)\n|-\n| 0x2 || NETWORK || Network boot\n|-\n| 0x3 || RPIBOOT || RPIBOOT - See usbboot (since 2020-09-03)\n|-\n| 0x4 || USB-MSD || USB mass storage boot (since 2020-09-03)\n|-\n| 0x5 || BCM-USB-MSD || USB 2.0 boot from USB Type-C socket or USB Type-A socket on CM4 IO board. (since 2020-12-14)\n|-\n| 0xe || STOP || Stop and display error pattern (since 2020-09-03). A power cycle is required to exit this state.\n|-\n| 0xf || RESTART || Start again with the first boot order field. (since 2020-09-03)\n|}\n=== DHCP ===\n\n==== dnsmasqd ====\n\nTo configure the dhcp server, login to the router using ssh, since these can't be set in Lucy, the webui, you just edit the dhcp config in /etc/config/dhcp to something like this\n\n<pre>\nconfig dhcp 'lan'\n; blablabla\n        list dhcp_option '66,192.168.10.56'\n        option filename 'bootcode.bin'\n</pre>\n\n''Note dhcp option 66 - that's 'server name', meaning the server address to the dhcp server.''\n\nNow just restart dnsmasq\n\n<pre>\n# /etc/init.d/dnsmasq restart\n</pre>\n\nThe pi will contact the tftp server given and request the files there with a prefix of its serialnumber followed by slash. You can find the serialnumber in /proc/cpuinfo thus\n\n<syntaxhighlight language=\"bash\">\ncat /proc/cpuinfo |awk '/^Serial/ { print substr($3,9) }'\n</syntaxhighlight>\n\n==== ISC DHCP Server ====\nIf using the isc-dhcpd, the following cofig should work well\n\n<syntaxhighlight language=\"bash\">\n# dhcp-options\nsubnet 10.x.x.0 netmask 255.255.255.0 {\n    range 10.x.x.10 10.x.x.199;\n    option routers 10.x.x.1;\n    default-lease-time 3600;\n    max-lease-time 7200;\n    filename \"bootcode.bin\";\n    server-name \"x.y.z.199\";\n    next-server x.y.z.199;\n    option vendor-class-identifier \"PXEClient\";\n# The one below is \"Raspberry Pi Boot\" hex (I think).\n#   option vendor-encapsulated-options 6:1:3:a:4:0:50:58:45:9:14:0:0:11:52:61:73:70:62:65:72:72:79:20:50:69:20:42:6f:6f:74:ff;\n# Apparently this should work like the above, only better, since the three spaces are necessary for older pi revisions.\n    option vendor-encapsulated-options \"Raspberry Pi Boot   \"\n}\n</syntaxhighlight>\n\n=== Thin-provisioning clients ===\nIn one setup, I installed a new version of Raspbian on a pi2 with a 64GB SD card. The installer makes sure the partitioning and fs size is changed after the initial bootup, so we'll need to fix that. This can be disabled somewhere (google it), but I forgot that, so I installed the SD card in another machine and checked with '''lsblk''' where it was located (sdb, with root on sdb2), so I ran a mandatory fsck -f first, then resized the filesystem to its minimum, removed the partition on which it was located, create a new one with the same starting point, but 10GB in size, added another to fill the gap, grew the current rootfs to the partition's full size and setup LVM.\n\nBelow from another card for documentation purposes, this time on sda:\n<pre>\n# fsck -f /dev/sda2\nfsck from util-linux 2.33.1\n...\n# resize2fs -M /dev/sda2\nResizing the filesystem on /dev/sda2 to 1239086 (4k) blocks.\nThe filesystem on /dev/sda2 is now 1239086 (4k) blocks long.\n# printf \"Filesystem size is %d MB\\n\" $(( 1239086 / 256 ))\nFilesystem size is 4840 MB\n# fdisk /dev/sda\n\nWelcome to fdisk (util-linux 2.33.1).\nChanges will remain in memory only, until you decide to write them.\nBe careful before using the write command.\n\n\nCommand (m for help): p\nDisk /dev/sda: 14.9 GiB, 15931539456 bytes, 31116288 sectors\nDisk model: STORAGE DEVICE\nUnits: sectors of 1 * 512 = 512 bytes\nSector size (logical/physical): 512 bytes / 512 bytes\nI/O size (minimum/optimal): 512 bytes / 512 bytes\nDisklabel type: dos\nDisk identifier: 0xea7d04d6\n\nDevice     Boot  Start      End  Sectors  Size Id Type\n/dev/sda1         8192   532479   524288  256M  c W95 FAT32 (LBA)\n/dev/sda2       532480 31116287 30583808 14.6G 83 Linux\n\nCommand (m for help): d\nPartition number (1,2, default 2): 2\n\nPartition 2 has been deleted.\n\nCommand (m for help): n\nPartition type\n   p   primary (1 primary, 0 extended, 3 free)\n   e   extended (container for logical partitions)\nSelect (default p): p\nPartition number (2-4, default 2): 2\nFirst sector (2048-31116287, default 2048): 532480\nLast sector, +/-sectors or +/-size{K,M,G,T,P} (532480-31116287, default 31116287): +8g\n\nCreated a new partition 2 of type 'Linux' and of size 8 GiB.\nPartition #2 contains a ext4 signature.\n\nDo you want to remove the signature? [Y]es/[N]o: n\n\nCommand (m for help): p\nDisk /dev/sda: 14.9 GiB, 15931539456 bytes, 31116288 sectors\nDisk model: STORAGE DEVICE\nUnits: sectors of 1 * 512 = 512 bytes\nSector size (logical/physical): 512 bytes / 512 bytes\nI/O size (minimum/optimal): 512 bytes / 512 bytes\nDisklabel type: dos\nDisk identifier: 0xea7d04d6\n\nDevice     Boot  Start      End  Sectors  Size Id Type\n/dev/sda1         8192   532479   524288  256M  c W95 FAT32 (LBA)\n/dev/sda2       532480 17309695 16777216    8G 83 Linux\n\nCommand (m for help): n\nPartition type\n   p   primary (2 primary, 0 extended, 2 free)\n   e   extended (container for logical partitions)\nSelect (default p): p\nPartition number (3,4, default 3):\nFirst sector (2048-31116287, default 2048): 17309696\nLast sector, +/-sectors or +/-size{K,M,G,T,P} (17309696-31116287, default 31116287):\n\nCreated a new partition 3 of type 'Linux' and of size 6.6 GiB.\n\nCommand (m for help): t\nPartition number (1-3, default 3):\nHex code (type L to list all codes): 8e\n\nChanged type of partition 'Linux' to 'Linux LVM'.\n\nCommand (m for help): p\nDisk /dev/sda: 14.9 GiB, 15931539456 bytes, 31116288 sectors\nDisk model: STORAGE DEVICE\nUnits: sectors of 1 * 512 = 512 bytes\nSector size (logical/physical): 512 bytes / 512 bytes\nI/O size (minimum/optimal): 512 bytes / 512 bytes\nDisklabel type: dos\nDisk identifier: 0xea7d04d6\n\nDevice     Boot    Start      End  Sectors  Size Id Type\n/dev/sda1           8192   532479   524288  256M  c W95 FAT32 (LBA)\n/dev/sda2         532480 17309695 16777216    8G 83 Linux\n/dev/sda3       17309696 31116287 13806592  6.6G 8e Linux LVM\n\nCommand (m for help): w\nThe partition table has been altered.\nCalling ioctl() to re-read partition table.\nSyncing disks.\n</pre>\n\nRepartitioning done, now grow the shrunken filesystem to fill up the new, smaller partition\n\n<pre>\n# resize2fs /dev/sda2\nresize2fs 1.44.5 (15-Dec-2018)\nResizing the filesystem on /dev/sda2 to 2097152 (4k) blocks.\nThe filesystem on /dev/sda2 is now 2097152 (4k) blocks long.\n</pre>\n\nThis leaves us with sda3, the new one, so we'll create a new volume group on it along with a logical volume to be used as the base later\n<pre>\n# vgcreate data /dev/sda3\n# lvcreate -n _client_ -L 10g data\n# mkfs -t ext4 /dev/data/_client_\n</pre>\n\n= BIOS upgrade from Linux =\n\nGenerally, BIOS upgrades have been moved to the BIOS itself these days. This saves a lot of work and quite possibly lives after those who earier rammed their heads through walls, now can upgrade directly from the internet instead of using obscure operating systems best avoided. Sometimes, however, one must use these nevertheless. This is a quick run-through on your options.\n\n== DOS ==\n\nMost non-automatic BIOS upgrades are installed from DOS. Doing a BIOS upgrade this way, is generally non-problematic. Just install a USB thingie with [https://www.freedos.org/ FreeDOS], copy your file(s) onto the thing and boot on it. The commandline is the same as old MS/DOS, except a wee bit more userfriendly, but not a lot.\n\n== Windows ==\n\nSome macahines, like laptops from '''HP''', may have BIOS upgrades that are made to be installed from Windows and won't run in FreeDOS or MS/DOS, instead throwing an error, saying '''This program cannot be run in DOS mode'''. This means you'll have to boot on a Windows rescue disc and do the job from there. Googling this, tells you it's quite easy, you just choose 'make rescue disc' from Windows, and it's all good, except if you don't run Windows, that is. To remedy this problem, do as follows:\n\n=== Download Windows ===\n\nSince you don't run Windows and possibly don't have a spouse or friend around with such unhealthy habits either, you need to get it. It's quite easy - just google 'download windows' and it'll send you to [https://www.microsoft.com/en-us/software-download/windows10ISO somewhere like this].\n\n=== Burn it! ===\n\nNow it's time to burn the installer on a DVD ROM. You'll need a double-layered one, since the OS is > 4.7GiB, but I'm sure you have a ton of those lying around. Now, just place your optical medium in your optical drive and burn, baby, burn!\n\n=== Or make a USB thing? ===\n\nNot a lot of machines have optical drives these days, so you may want to use an USB pen drive or something instead. This is easy, just make the USB bootable with the Windows application Microsoft has made for this. Unfortunately, this only runs on Windows, and unlike stuff like Debian, where the '''iso''' file can be dd'ed directly onto a USB drive to make it bootable, the Windows '''iso'''s don't come with this luxury. There are lots of methods to make bootable USB things from iso files out there, but the only thing most of them have in common, is that they generally don't work.\n\n==== WoeUSB ====\n\nAfter hours of swearing, it's nice to come across software like [https://github.com/slacka/WoeUSB WoeUSB]. It may not be perfect, it relies on a bunch of GUI stuff you'll never need and so on, but it '''''works''''', and that's the important part! Just clone the repo and RTFM and it's all nice. It's slow, but hell, getting a windows machine and/or an optical drive might be slower.\n\nWoeUSB does nothing fancy, but it '''does''' work. It will create a filesystem, FAT32 or NTFS (better use NTFS, FAT32 has its limits). It creates normal filesystems and so on. Just make sure to copy in the BIOS update file, usually an exe file, to run when Windows is up and running.\n\n==== Install BIOS ====\n\nBoot on the Windows USB thing and choose 'repair computer' and then 'command prompt'. Find the install file, and if you're lucky, you'll find it needs a 32bit OS, just like I did. Since the 64bit version doesn't include 32bit compatibility in the installer, revert to downloading the 32bit version of windows and start over. Pray to your favourite god(s) not to get across such machines again - it might help.\n\n= 3D printer stuff =\n\nBelow are a number of not very ordered paragraphs about 3D printer related stuff, mostly based on my experience. Their content may be interesting or perhaps practical knowledge, but then, that depends on the reader.\n\n== 3D printer related introductions on youtube or elsewhere ==\n\nFirst off, I'd like to mention some youtube videos that are all (or mostly) a nice start if you're new to 3D printers.\n\n=== [https://www.youtube.com/watch?v=nb-Bzf4nQdE&list=PLDJMid0lOOYnkcFhz6rfQ6Uj8x7meNJJx Thomas Salanderer's 10-video series on 3D printing basics] ===\n\nThomas Salanderer is a an experienced 3D printer user/admin/fixer/etc and he has a lot of interesting videos. Some accuse him for being a [Prusa https://www.prusa3d.com/] fanboy, which he quite obviously is, but that doesn't stop him from making interesting and somewhat neutral videos. The series walks you thought the first steps of how things work and so on and hopefully opens more doors than anything else I've seen.\n\n=== [https://www.youtube.com/watch?v=rp3r921DBGI Teaching Tech's \"3D printer tuning reborn\"] ===\n\nTeaching Tech is, like Salanderer or even more, good pedagogically and explains a lot. They (or he) do a bunch of testing and describing pros and cons with different things, just like other youtubers in the sme business. The mentioned video shows how to tune a 3D printer after you have first learned the basics, and leans on a webpage made to help you out without too much manual work.\n\nThis was a short list, but I guess it'll grow over time.\n\n== Hydrophilic filament ==\n\nMost popular filament types, including PLA, PETG, PP or PA (Polyamide, normally known as Nylon) and virtualy any filament type named [https://www.matterhackers.com/news/filament-and-water poly-something] (and thus with an acronym of P-something) are hydrophilic (also (somewhat incorrectly) called hydroscopic), meaning so long they're dryer than the atmosphere around them, they'll do their best to suck out the atmosphere's humidity. This is why most filament are shrink-wrapped with a small bag of silica gel in it. If such filament is exposed for normal humid air for some time, it'll become very hard to use. Most of my experience is with PLA, which can handle a bit (depending on make), but still not a lot. With PLA, if you end up with prints where the filament seems not to stick, it may help with a higher temperature. Otherwise, the filament must be [https://all3dp.com/2/how-to-dry-filament-pla-abs-and-nylon/ baked]. If you don't want to use your oven, try something like a [https://www.jula.no/catalog/hjem-og-husholdning/kjokkenmaskiner/mattilberedningsapparater/frukt-sopptorker/frukt-sopptorker-001641/#tab02 fruit and mushroom dryer]. Then get a good, sealble plastic box and add something like half a kilogram of [https://www.ebay.com/sch/sis.html?_nkw=1KG+Orange+Replacement+Desiccant+Indicating+Silica+Gel+Beads+for+Drawers%2C+Camera&_id=131938742628&&_trksid=p2057872.m2749.l2658 silica gel] to an old sock, tie it and put it in the your new dry box.\n\nPlease note that ABS is hydrophobic, so you won't have to worry too much there. Also, remember that PA/Nylon is extremely hydrophilic. Even a couple of days in normal air humidity can ruin it completely and will require baking.\n\n== Upgrading the firmware on an Ender 3 ==\n\nThis is about upgrading the firmware, and thus also flashing a bootloader to the Creality Ender 3 3D printer. Most of this is well documented on several place, like o [https://www.youtube.com/watch?v=fIl5X2ffdyo the video from Teaching tech] and elsewhere, but I'll just summarise some issues I had.\n\n=== Using an arduino Nano as a programmer ===\n\nQuick note before you read on. If you have an Ender 3 controller version 1.1.5 or newer (or perhaps even a bit older), a CR-10S or some other more or less modern printers or controllers, they come with a bootloader installed already, so don't bother flashing one. The one that's in there already, should work well. So if you have one of these, just skip this and jump to the [[Roy's_notes#Flashing_the_printer|next section]].\n\nHowever, the normal CR-10 (without the S), Ender 3 and a few other printers from Creality come without a bootloader, so you'll need to flash one before upgrading the firmware. Well, strictly speakning, you don't need one, you can save those 8kB or so for other stuff and use a programmer to write the whole firmware, but then you'll need to wire up everything every time you want a new firmware, so in my world, you '''do''' need the bootloader, since it's easier.\n\nTo install a bootloader, you'll need a programmer, and I didn't have one available. Having some el-cheapo china-copies of Arduinos around, I read I could use one and tried so. Although the docs mostly mention the Uno etc, it's just as simple with the Nano copy.\n\nSo, grab an arduino, plug it to your machine with a USB cable, and, using the Arduino IDE, use the ArduinoISP sketch there (found under Examples) to burn the software needed for the arduino to work as a programmer. When this is done, connect a 10\u00b5F capacitor between RST and GND to stop it from resetting when used as a programmer. Connect the MOSI, MISO and SCK pins from the ICSP block (that little isolated 6-pin block on top of the ardu). See [https://www.arduino.cc/en/Tutorial/ArduinoISP here] for a pinout. Then find Vcc and GND either on the ICSP block or elsewhere. Some sites say that on some boards you shouldn't use the pins on the ICSP block for this. I doubt it matters, though. Then connect another jumper wire from pin 10 on the ardu to work as the reset pin outwards to the chip being programmed (that is, on the printer). The ICSP jumper block pin layout is the same on the printer and on the ardu, and probably elsewhere. Sometimes [https://xkcd.com/927/ standardisation] actually works\u2026\n\nWhen it's all wired up, change the setup in the Arduino IDE to use the Sanguino board (which may need to be installed there), the Atmega 1284 16MHz board with the programmer set to     \"Arduino as ISP\". Also make sure to choose the right serial port. When all is done, choose Tools | Burn programmer and it should take a few seconds to finish.\n\n[[File:Ardu-nano-as-programmer.jpg|400px]]\n\n== Flashing the printer ==\n\nWhen the boot loader is in place, possibly after some wierd errors from during the process, the screen on the 3d printer will go blank and it'll behave like being bricked. This is ok. Now disconnect the wires and connect the USB cable between computer and printer. If you haven't installed support for the Sanguino board, that is, the variant of the 1284-chip and surrounding electronics that is the core of the, you need to install it first. In the Arduino IDE, choose the Tools / Boards / Boards manager boot option and search for Sanguino. After this, you should find the Sanguino or Sanguino 1284p in the boards menu. After this, you should be able to communicate with the printer's controller. Download the [https://www.th3dstudio.com/knowledgebase/th3d-unified-firmware-package/ TH3D unified firmware], making sure it's the last version. Uncompress the zip and with Finder/Explorer/whateversomething'scalledonlinux, browse to '''TH3DUF_R2/Firmware/TH3DUF_R2.ino''' and open it. It should open directly in the Android IDE. Keep in mind that the names TH3DUF_R2 and TH3DUF_R2.ino will vary between versions, but you get the point. Now configure it for your particular needs. You'll find most of this in the Configuration.h tab (that's really a file). Most of the other files won't be necessary unless you're doing some more advanced stuff, like replacing th controller with something non-standard or similar, but then again, that's another chapter (or book, even). The video mentioned above describes this well. After flashing the new firmware, you'll probably get some timeouts and errors, since apparently it resets the printer after giving it the new firmware, but without notifying the flashing software of it doing so. If this happens, calm down and reboot the printer and check its tiny monitor - it should work. If it doesn't work, and if you flashed the bootload yourself, try to flash it again, and if that doesn't work, try flashing the programmer again yet another time, just for kicks. Again, if you have a newer controller like the v1.1.5, don't touch the bootloader, as the one already installed, should work well as it is.\n\nPS: I tried enabling '''MANUAL_MESH_LEVELING''', which in the code says that ''If used with a 1284P board the bootscreen will be disabled to save space''. This effectively disabled the whole printer, and apparently may be making the firmware image a wee bit too large for it to fit the 1284P on the ender. It works well without it, though, and it may be fixed by now, since I tested this back in early 2019.\n\n== And then\u2026 ==\n\nWith the new firmware in place, the printer should work as before, although with thermal runaway protection to better avoid fires in case the shit hits the fan, and to allow for things like autolevelling. With any luck, [https://www.precisionpiezo.co.uk/ this thing] may be ready for use by the time you read this - it surely looks nice!\n\n= Octoprint/Octopi =\n\n''This could have been under some other section, but again, I just branch out even though most of it is about sections mentioned elsewhere.''\n\nWhen setting up a 3d printer for the first time, you usually get an SD card for storage and transfer data to the printer using [https://en.wikipedia.org/wiki/Sneakernet Sneakernet]. Some printers use micro SD cards, while other fullsize SD cards, which imho is better, since they don't get lost that easily, but otherwise do the same thing. This works, but is somewhat tedious. If you want to control the printer from a PC or phone, there's a simple trick for that as well - octoprint.\n\n[https://octoprint.org/ Octoprint], written by Gina H\u00e4u\u00dfge, runs on most platforms including linux, windows, BSD*, macos etc. It supports controlling a camera to some extent out of the box and there's a large number of plugins availablee to do a lot of things you possibly haven't thought of (and quite often, you'd ever need). The easiest way to install it, is to just grab a Raspberry pi - preferably some of the newer models (will get to that later) and download [https://octoprint.org/download/ octopi] and follow the instructions on that page.\n\n== Performance issues ==\n\nSome report (and I have seen it myself) issues with the pi's performance with regards to both handling the octoprint processes (which should be as close to realtime as possible) and handling other stuff like I/O, networking etc. The point is that at pi3 or older, has all peripherals connected to an internal USB hub. This might be practical, but performance-wise it's '''''not'''''. If you in addition connect a webcam to USB, you're asking for trouble, since USB will be your bottleneck. With a raspberry pi camera, the ones connected to the pi directly with a ribbon cable, this should not use USB, so it's better. Still, again, for older pi's, there might be some shortage in therms of cpu or i/o. The octopi runs something like raspbian which is a fork of debian which is a linux distro, so it all boils down to knowing linux.\n\nYour typical octopi will run octoprint, a streamer, usually '''mjpg-streamer''', a simple, lightweight videostreamer that outputs a stream of jpeg-images (about the same format as cinemas use - that is - not MPEG). This may be a bit tough on the cpu and potentially i/o. Add inn a dash of USB overload and you're may see trouble.\n\nStill - a pi3 should be able to handle the load, but it might help to prioritise corretly. The octoprint process is the important part here, so we could give that full priority both in CPU and I/O and. Unfortunately, it doesn't seem like the current octopi has been migrated to using systemd for the startup. We'll deal with this later. To fix this in the old SysV init style file present there, the following patch should work against the file /etc/init.d/octoprint\n\n<syntaxhighlight lang=\"diff\">\n--- octoprint.old\t2020-07-20 18:46:10.166651965 +0200\n+++ octoprint\t2020-07-20 18:53:21.724803211 +0200\n@@ -22,6 +22,9 @@\n PIDFILE=/var/run/$NAME.pid\n PKGNAME=octoprint\n SCRIPTNAME=/etc/init.d/$PKGNAME\n+NICELEVEL=-19                        # Top CPU priority\n+IONICE_CLASS=1                       # Realtime I/O class\n+IONICE_PRIORITY=0                    # Realtime I/O priority (probably not needed with class=realtime)\n\n # Read configuration variable file if it is present\n [ -r /etc/default/$PKGNAME ] && . /etc/default/$PKGNAME\n@@ -70,10 +73,11 @@\n\n    is_alive $PIDFILE\n    RETVAL=\"$?\"\n-\n    if [ $RETVAL != 0 ]; then\n        start-stop-daemon --start --background --quiet --pidfile $PIDFILE --make-pidfile \\\n-       --exec $DAEMON --chuid $OCTOPRINT_USER --user $OCTOPRINT_USER --umask $UMASK -- serve $DAEMON_ARGS\n+       --exec $DAEMON --chuid $OCTOPRINT_USER --user $OCTOPRINT_USER --umask $UMASK \\\n+       --nicelevel $NICELEVEL --ioched $IONICE_CLASS:$IONICE_PRIORITY \\\n+       --serve $DAEMON_ARGS\n        RETVAL=\"$?\"\n    fi\n }\n</syntaxhighlight>\n\nThis ismply upgrades the process to be allowed to use most of the resources on the pi, regardless of what other processes are whining about.\n\nPS: Code not tested - I don't have a pi right here. Will test later, but I'm pretty sure it'll work. After all, it's just a few new arguments to start-stop-daemon\n\n= Samba =\n\n== Apple Time Machine ==\nTime machine is a nice backup tool from Apple, running on their macOS operating system. It was introduced with Mac OS X 10.5 Leopard in October 2007. Apple's own documentation tells us to get an Apple Time Capsule (or Airport Time Capsule), that is, a small, network-attached external drive. Either that, or a normal USB drive of sorts. Some of us have home servers, so apparently, it's not hard to setup that with Samba (Windows file server emulator - ish).\n\nFor each mac computer, create a Samba account and for each account, create a share. This will be setup with a few extra flags, as described below. Note that it must be a separate share per backed up computer - a subfolder per user will not do. Apple Time Machine must be able to beleive it ows that 'disk'.\n\nAdd this to your '''/etc/samba/smb.conf''':\n<syntaxhighlight lang=\"conf\">\n[time_machine_someuser]\n    comment = Time Machine backup for Mac\n    path = /data/time_machine/someuser\n    browseable = yes\n    read only = no\n    valid users = someuser\n    vfs objects = catia fruit streams_xattr\n    fruit:time machine = yes\n</syntaxhighlight>\n\nReload samba with '''systemctl reload smb''' and connect to it from your mac (CMD+k in Finder, write smb://ip.addr.of.server/, press enter, browse to '''time_machine_someuser''' and connect) and open your Time Machine settings under System Settings and click Add Drive\n\nroy"
                    }
                ]
            },
            "2": {
                "pageid": 2,
                "ns": 0,
                "title": "The Linux MD Bad Block List",
                "revisions": [
                    {
                        "contentformat": "text/x-wiki",
                        "contentmodel": "wikitext",
                        "*": "== Introduction ==\n\nIn the Linux Software RAID driver, '''md''' (as in Multiple Devices) the Bad Block List (BBL) was [https://neil.brown.name/blog/20100519043730 introduced around 2010] to allow for creating a map of bad sectors on member drives in a RAID. The intention was then to stay away from these sectors to help save your data. Some five years later, a text, [https://raid.wiki.kernel.org/index.php/The_Badblocks_controversy The Badblocks controversy], was posted, listing up a number of reasons the BBL is a bad idea in the first place.\n\nFirstly, the BBL attempts to keep a list of sectors that were noted to be unreadable as to avoid these sectors. While this may seem like a good idea, drives have their own microcontrollers, flash memory and so on to keep such lists themselves. This has been around on consumer equipment since the first IDE drives hit the market in the end of the 1980s. The common drives earlier than this, typically MFM drives, didn't have any such drive intelligence and left that to the operating system. A modern drive uses \nReed\u2013Solomon errorcorrecion or similar and sorts out bad sectors without bothering the OS about it. At the time you see a bad sector from the OS, the number of bad sectors on the drive is usually a lot higher, since a lot has been relocated by the controller already, and the drive should be replaced. To start mapping out bad sectors at this point, is like wetting yourself to keep warm - not a good idea.\n\n''(to be continued\u2026)''"
                    }
                ]
            }
        }
    }
}