Roy's notes - Revision history

Roy: /* Octoprint/Octopi */

2024-02-16T12:17:44Z

Octoprint/Octopi

← Older revision		Revision as of 12:17, 16 February 2024
Line 1,284:		Line 1,284:

	PS: Code not tested - I don't have a pi right here. Will test later, but I'm pretty sure it'll work. After all, it's just a few new arguments to start-stop-daemon		PS: Code not tested - I don't have a pi right here. Will test later, but I'm pretty sure it'll work. After all, it's just a few new arguments to start-stop-daemon

			= Samba =

			== Apple Time Machine ==
			Time machine is a nice backup tool from Apple, running on their macOS operating system. It was introduced with Mac OS X 10.5 Leopard in October 2007. Apple's own documentation tells us to get an Apple Time Capsule (or Airport Time Capsule), that is, a small, network-attached external drive. Either that, or a normal USB drive of sorts. Some of us have home servers, so apparently, it's not hard to setup that with Samba (Windows file server emulator - ish).

			For each mac computer, create a Samba account and for each account, create a share. This will be setup with a few extra flags, as described below. Note that it must be a separate share per backed up computer - a subfolder per user will not do. Apple Time Machine must be able to beleive it ows that 'disk'.

			Add this to your '''/etc/samba/smb.conf''':
			<syntaxhighlight lang="conf">
			[time_machine_someuser]
			comment = Time Machine backup for Mac
			path = /data/time_machine/someuser
			browseable = yes
			read only = no
			valid users = someuser
			vfs objects = catia fruit streams_xattr
			fruit:time machine = yes
			</syntaxhighlight>

			Reload samba with '''systemctl reload smb''' and connect to it from your mac (CMD+k in Finder, write smb://ip.addr.of.server/, press enter, browse to '''time_machine_someuser''' and connect) and open your Time Machine settings under System Settings and click Add Drive

	roy		roy

Roy: Interrupting an mdadm --grow

2023-04-04T22:57:12Z

Interrupting an mdadm --grow

← Older revision		Revision as of 22:57, 4 April 2023
Line 516:		Line 516:
	For more info about this topic, take a look at [https://raid.wiki.kernel.org/index.php/The_Badblocks_controversy this article]		For more info about this topic, take a look at [https://raid.wiki.kernel.org/index.php/The_Badblocks_controversy this article]

	=== mdadm ~~workarounds~~ ===		=== mdadm tweaks ===

	==== Migrate from a mirror (raid-1) to raid-10 ====		==== Migrate from a mirror (raid-1) to raid-10 ====
Line 564:		Line 564:

	''RAID section is not complete. I'll write more on migraing from raid10 to raid6 later. It's not straight forward, but easier than this one :)''		''RAID section is not complete. I'll write more on migraing from raid10 to raid6 later. It's not straight forward, but easier than this one :)''

			==== Interrupting an mdadm --grow ====
			While working on a friend's machine, we started growing a raid-10 with two 18TB drives, adding a third. While this worked fine-ish some time, it was slow, possibly beause of a slow USB dock (although everything was USB3 according to lsusb etc). At the worst, we saw speeds of 200kB/s for resync, meaning it'd take 18 months or so to finish, which isn't very enjoying So, we'll need to see how md handles interruption of this resync.

			On a similar setup of hers on a test vm, I added a disk to a 5-disk raid-10 and issued '''mdadm --grow --raid-devices 6''' and let it run for some seconds. I then rebooted the machine, '''reboot''', and waited for it to come up. It did, but the filesystem on the raid-10 wasn't mounted. Since it was flagged as nofail in fstab, it didn't bother the boot process, but it was nonetheless unavailable. The status in '''/proc/mdstat''' showed '''auto-read-only''' and '''pvs''' just showed is as '''(unknown)'''. I tried to '''echo check > /sys/block/md1/md/sync_action''', and this activated the RAID and it continued from where it was interrupted. Apparently, even sending it an '''idle''' command is sufficient to wake it up. After this, I could mount the filesystems s usual. I also tried '''reboot -f''' and '''echo b > /proc/sysrq-trigger''', the latter being the best analogue to using a hard reset switch. This [[wikipedia:Magic_SysRq_key\|wikipedia article]] has more on sysrq. It all worked the same way and after checking sha512sum, the data stored on the original filesystem, was untouched.

	== Tuning ==		== Tuning ==

Roy: /* Create LVM cache */

2022-10-16T13:17:22Z

Create LVM cache

← Older revision		Revision as of 13:17, 16 October 2022
Line 107:		Line 107:
	</syntaxhighlight>		</syntaxhighlight>

	~~Thn~~ create two LVs, one for the data (contents of files) and one for the metadata (filenames, directories, attributes etc). Typically, the metadata part won't need to be very large. 100M should usually do unless you have ekstremely many small files. I guess there are ways to calculate it, but again, 1GB should do for everyone (or was that 640k?). As for the data cache, I chose to use 40G here for a 15TB RAID. It will only cache what actively use anyway, so no need for overkill.		Then create two LVs, one for the data (contents of files) and one for the metadata (filenames, directories, attributes etc). Typically, the metadata part won't need to be very large. 100M should usually do unless you have ekstremely many small files. I guess there are ways to calculate it, but again, 1GB should do for everyone (or was that 640k?). As for the data cache, I chose to use 40G here for a 15TB RAID. It will only cache what actively use anyway, so no need for overkill.

	<syntaxhighlight lang="bash">		<syntaxhighlight lang="bash">

Roy: /* Create LVM cache */

2022-09-25T01:29:47Z

Create LVM cache

← Older revision		Revision as of 01:29, 25 September 2022
Line 112:		Line 112:
	lvcreate -L 1G -n _cache_meta data /dev/md1		lvcreate -L 1G -n _cache_meta data /dev/md1
	lvcreate -L 100G -n _cache data /dev/md1		lvcreate -L 100G -n _cache data /dev/md1

			# If you want to use the rest of the cache device, here /dev/md1, replace -L 100G with -l +100%PVS, not, -l +100%FREE, as that will use all the available space in the VG regardless of there being a PV mentioned in the config.

	# Some would want --cachemode writeback, but if paranoid, I wouldn't recommend it. Metadata or data can be easily		# Some would want --cachemode writeback, but if paranoid, I wouldn't recommend it. Metadata or data can be easily

Roy: /* single, slow drive */

2022-09-25T01:19:05Z

single, slow drive

← Older revision		Revision as of 01:19, 25 September 2022
Line 311:		Line 311:

	==== single, slow drive ====		==== single, slow drive ====
	What may happen, is smart not seeing anything and life goes on like before, only slower and less prouductive, without telling anyone. If you stubler over this issue, you'll probably see it during a large rsync/zfs send/receive or a resync/rebuild/resilver of the raid/zpool. During this, it feels like everything is slow and your RAID has turned into a RAIF (redundant array of inexpensive floppies). To check if you have a single drive messing up it all, check with '''iostat -x 2''', having 2 being the delay between each measurement. Interrupt it with ctrl+x. If there's a rougue drive there, it should stand out like sdg below		What may happen, is smart not seeing anything and life goes on like before, only slower and less prouductive, without telling anyone. If you stubler over this issue, you'll probably see it during a large rsync/zfs send/receive or a resync/rebuild/resilver of the raid/zpool. During this, it feels like everything is slow and your RAID has turned into a RAIF (redundant array of inexpensive floppies). To check if you have a single drive messing up it all, check with '''iostat -x 2''', having 2 being the delay between each measurement. Interrupt it with ctrl+x (if you're like Neo) or ctrl+c if you're someone else. If there's a rougue drive there, it should stand out like sdg below

	avg-cpu: %user %nice %system %iowait %steal %idle		avg-cpu: %user %nice %system %iowait %steal %idle

Roy: /* Filesystem dilemmas */

2022-09-24T23:30:50Z

Filesystem dilemmas

← Older revision		Revision as of 23:30, 24 September 2022
Line 284:		Line 284:
	The choice of filesystem varies for your use. Distros such as RHEL/CentOS has moved to XFS by default from v7. Debian/Ubuntu still sticks to ext4, like most other. I'll discuss my thoughts below on ext4 vs XFS and leave the other filesystems for later.		The choice of filesystem varies for your use. Distros such as RHEL/CentOS has moved to XFS by default from v7. Debian/Ubuntu still sticks to ext4, like most other. I'll discuss my thoughts below on ext4 vs XFS and leave the other filesystems for later.

	=== bcachefs ===		==== bcachefs ====
	'''bcachefs''' is the latest newcomer, which started out as '''bcache''', a caching system for linux, using SSDs or NVMEs or similar to cache slower HDDs. It has later evolved to a full filesystem, supporting similar to mirroring, but across multiple drives of different sizes, as some may know the commercial '''unraid''' does. It supports full checksumming and healing of data when errors occur. It stripes data across available media, making sure there's at least two (or more, depending on configuration) drives holding the data to allow for failures. Bcachefs also supports similar to RAID levels similar to RAID5 or RAID6 with ''erasure coding''. The latter is not quite stable at the time of writing (2022-09-25), but should work well soon (fingers crossed).		'''bcachefs''' is the latest newcomer, which started out as '''bcache''', a caching system for linux, using SSDs or NVMEs or similar to cache slower HDDs. It has later evolved to a full filesystem, supporting similar to mirroring, but across multiple drives of different sizes, as some may know the commercial '''unraid''' does. It supports full checksumming and healing of data when errors occur. It stripes data across available media, making sure there's at least two (or more, depending on configuration) drives holding the data to allow for failures. Bcachefs also supports similar to RAID levels similar to RAID5 or RAID6 with ''erasure coding''. The latter is not quite stable at the time of writing (2022-09-25), but should work well soon (fingers crossed).

	=== btrfs ===		==== btrfs ====
	'''btrfs''', pronounced "butterfs" or "betterfs" or just spelled out, is a filesystem that mimics that of ZFS. It has been around for 10 years or so, and has proven stable as a dropin replacement for xfs and ext4 and so on, but its real magic in building a full stack with a full RAID subsystem, hasn't yet proven to be stable. In fact, after 15 years of development, it's hardly usable for that. So you might want to use it for its snapshots and compression, btrfs send/receive etc, but preferably with something below, like mdraid and lvm to help out with redundancy and perhaps SSD caching.		'''btrfs''', pronounced "butterfs" or "betterfs" or just spelled out, is a filesystem that mimics that of ZFS. It has been around for 10 years or so, and has proven stable as a dropin replacement for xfs and ext4 and so on, but its real magic in building a full stack with a full RAID subsystem, hasn't yet proven to be stable. In fact, after 15 years of development, it's hardly usable for that. So you might want to use it for its snapshots and compression, btrfs send/receive etc, but preferably with something below, like mdraid and lvm to help out with redundancy and perhaps SSD caching.

	=== ext4 ===		==== ext4 ====
	'''ext4''' is probably the most used filesystem on linux as of writing. It's rock stable and has been extended by a lot of extensions since the original ext2. It still retains backward compatibility, being able to mount and fsck ext2 and ext3 with the ext4 driver. However, it doesn't handle large filesystems very well. If a filesystem is created for <16TiB, it cannot be grown to anything larger than 16TiB. This has changed around 2020, though, with mot filesystems created for 64bit use. You still can't change a 32bit filesystem to 64bit. One other issue is handling errors. If a large filesystem (say 10TiB) is damaged, an fsck may take several hours, leading to long downtime. When I refer to ext4 further in this text, the same applies to ext2 and ext3 unless otherwise specified.		'''ext4''' is probably the most used filesystem on linux as of writing. It's rock stable and has been extended by a lot of extensions since the original ext2. It still retains backward compatibility, being able to mount and fsck ext2 and ext3 with the ext4 driver. However, it doesn't handle large filesystems very well. If a filesystem is created for <16TiB, it cannot be grown to anything larger than 16TiB. This has changed around 2020, though, with mot filesystems created for 64bit use. You still can't change a 32bit filesystem to 64bit. One other issue is handling errors. If a large filesystem (say 10TiB) is damaged, an fsck may take several hours, leading to long downtime. When I refer to ext4 further in this text, the same applies to ext2 and ext3 unless otherwise specified.
	As mentioned below, ext4 easily outperforms xfs on metadataoperations, so if you have a filesystem with millions of files, ext4 is your friend. However, ext4 still uses static inode allocation and if a filesystem's inode tables are full (one inode per file/directory/symlink etc), so is the filesystem. Read the manual and lookup the -N flag in mkfs.ext4.		As mentioned below, ext4 easily outperforms xfs on metadataoperations, so if you have a filesystem with millions of files, ext4 is your friend. However, ext4 still uses static inode allocation and if a filesystem's inode tables are full (one inode per file/directory/symlink etc), so is the filesystem. Read the manual and lookup the -N flag in mkfs.ext4.

	=== XFS ===		==== XFS ====
	'''XFS''', originally developed by SGI some time in the bronze age, but has been worked on heavily after that. RHEL and Centos version 7 and forward, uses XFS as the default filesystem. SuSE does the same. It works well, but for metadata operations, it's significantly slower than that of ext4, so if you need fast file access for millions of files on a filesystem, '''ext4''' is still your friend. Just read the notes in the above paragraphs. XFS still scales well, but it lacks at least two things ext4 has. It cannot be shrunk (not that you normally need that, but nice if you have a typo or you need to change things). Also, it doesn't allow for automatic fsck on bootup. If something is messed up on the filesystem, you'll have to login as root on the console (not ssh), which might be an issue on some systems. The fsck equivalent, xfs_repair, is a lot faster than ext4's fsck, though.		'''XFS''', originally developed by SGI some time in the bronze age, but has been worked on heavily after that. RHEL and Centos version 7 and forward, uses XFS as the default filesystem. SuSE does the same. It works well, but for metadata operations, it's significantly slower than that of ext4, so if you need fast file access for millions of files on a filesystem, '''ext4''' is still your friend. Just read the notes in the above paragraphs. XFS still scales well, but it lacks at least two things ext4 has. It cannot be shrunk (not that you normally need that, but nice if you have a typo or you need to change things). Also, it doesn't allow for automatic fsck on bootup. If something is messed up on the filesystem, you'll have to login as root on the console (not ssh), which might be an issue on some systems. The fsck equivalent, xfs_repair, is a lot faster than ext4's fsck, though.

	=== ZFS ===		==== ZFS ====
	ZFS is like a combination of RAID, LVM and a filesystem, supporting full checksumming, auto-healing (given sufficient redundancy), compression, encryption, replication, deduplication (if you're brave and have a ''ton'' of memory and can afford some risks) and a lot more. It's a separate chapter and needs a lot more talk than paragraph here.		ZFS is like a combination of RAID, LVM and a filesystem, supporting full checksumming, auto-healing (given sufficient redundancy), compression, encryption, replication, deduplication (if you're brave and have a ''ton'' of memory and can afford some risks) and a lot more. It's a separate chapter and needs a lot more talk than paragraph here.

Roy: /* Filesystem dilemmas */

2022-09-24T23:30:15Z

Filesystem dilemmas

← Older revision		Revision as of 23:30, 24 September 2022
Line 287:		Line 287:
	'''bcachefs''' is the latest newcomer, which started out as '''bcache''', a caching system for linux, using SSDs or NVMEs or similar to cache slower HDDs. It has later evolved to a full filesystem, supporting similar to mirroring, but across multiple drives of different sizes, as some may know the commercial '''unraid''' does. It supports full checksumming and healing of data when errors occur. It stripes data across available media, making sure there's at least two (or more, depending on configuration) drives holding the data to allow for failures. Bcachefs also supports similar to RAID levels similar to RAID5 or RAID6 with ''erasure coding''. The latter is not quite stable at the time of writing (2022-09-25), but should work well soon (fingers crossed).		'''bcachefs''' is the latest newcomer, which started out as '''bcache''', a caching system for linux, using SSDs or NVMEs or similar to cache slower HDDs. It has later evolved to a full filesystem, supporting similar to mirroring, but across multiple drives of different sizes, as some may know the commercial '''unraid''' does. It supports full checksumming and healing of data when errors occur. It stripes data across available media, making sure there's at least two (or more, depending on configuration) drives holding the data to allow for failures. Bcachefs also supports similar to RAID levels similar to RAID5 or RAID6 with ''erasure coding''. The latter is not quite stable at the time of writing (2022-09-25), but should work well soon (fingers crossed).

	==== btrfs ====		=== btrfs ===
	'''btrfs''', pronounced "butterfs" or "betterfs" or just spelled out, is a filesystem that mimics that of ZFS. It has been around for 10 years or so, and has proven stable as a dropin replacement for xfs and ext4 and so on, but its real magic in building a full stack with a full RAID subsystem, hasn't yet proven to be stable. In fact, after 15 years of development, it's hardly usable for that. So you might want to use it for its snapshots and compression, btrfs send/receive etc, but preferably with something below, like mdraid and lvm to help out with redundancy and perhaps SSD caching.		'''btrfs''', pronounced "butterfs" or "betterfs" or just spelled out, is a filesystem that mimics that of ZFS. It has been around for 10 years or so, and has proven stable as a dropin replacement for xfs and ext4 and so on, but its real magic in building a full stack with a full RAID subsystem, hasn't yet proven to be stable. In fact, after 15 years of development, it's hardly usable for that. So you might want to use it for its snapshots and compression, btrfs send/receive etc, but preferably with something below, like mdraid and lvm to help out with redundancy and perhaps SSD caching.

	==== ext4 ====		=== ext4 ===
	'''ext4''' is probably the most used filesystem on linux as of writing. It's rock stable and has been extended by a lot of extensions since the original ext2. It still retains backward compatibility, being able to mount and fsck ext2 and ext3 with the ext4 driver. However, it doesn't handle large filesystems very well. If a filesystem is created for <16TiB, it cannot be grown to anything larger than 16TiB. This has changed around 2020, though, with mot filesystems created for 64bit use. You still can't change a 32bit filesystem to 64bit. One other issue is handling errors. If a large filesystem (say 10TiB) is damaged, an fsck may take several hours, leading to long downtime. When I refer to ext4 further in this text, the same applies to ext2 and ext3 unless otherwise specified.		'''ext4''' is probably the most used filesystem on linux as of writing. It's rock stable and has been extended by a lot of extensions since the original ext2. It still retains backward compatibility, being able to mount and fsck ext2 and ext3 with the ext4 driver. However, it doesn't handle large filesystems very well. If a filesystem is created for <16TiB, it cannot be grown to anything larger than 16TiB. This has changed around 2020, though, with mot filesystems created for 64bit use. You still can't change a 32bit filesystem to 64bit. One other issue is handling errors. If a large filesystem (say 10TiB) is damaged, an fsck may take several hours, leading to long downtime. When I refer to ext4 further in this text, the same applies to ext2 and ext3 unless otherwise specified.
	As mentioned below, ext4 easily outperforms xfs on metadataoperations, so if you have a filesystem with millions of files, ext4 is your friend. However, ext4 still uses static inode allocation and if a filesystem's inode tables are full (one inode per file/directory/symlink etc), so is the filesystem. Read the manual and lookup the -N flag in mkfs.ext4.		As mentioned below, ext4 easily outperforms xfs on metadataoperations, so if you have a filesystem with millions of files, ext4 is your friend. However, ext4 still uses static inode allocation and if a filesystem's inode tables are full (one inode per file/directory/symlink etc), so is the filesystem. Read the manual and lookup the -N flag in mkfs.ext4.

	==== XFS ====		=== XFS ===
	'''XFS''', originally developed by SGI some time in the bronze age, but has been worked on heavily after that. RHEL and Centos version 7 and forward, uses XFS as the default filesystem. SuSE does the same. It works well, but for metadata operations, it's significantly slower than that of ext4, so if you need fast file access for millions of files on a filesystem, '''ext4''' is still your friend. Just read the notes in the above paragraphs. XFS still scales well, but it lacks at least two things ext4 has. It cannot be shrunk (not that you normally need that, but nice if you have a typo or you need to change things). Also, it doesn't allow for automatic fsck on bootup. If something is messed up on the filesystem, you'll have to login as root on the console (not ssh), which might be an issue on some systems. The fsck equivalent, xfs_repair, is a lot faster than ext4's fsck, though.		'''XFS''', originally developed by SGI some time in the bronze age, but has been worked on heavily after that. RHEL and Centos version 7 and forward, uses XFS as the default filesystem. SuSE does the same. It works well, but for metadata operations, it's significantly slower than that of ext4, so if you need fast file access for millions of files on a filesystem, '''ext4''' is still your friend. Just read the notes in the above paragraphs. XFS still scales well, but it lacks at least two things ext4 has. It cannot be shrunk (not that you normally need that, but nice if you have a typo or you need to change things). Also, it doesn't allow for automatic fsck on bootup. If something is messed up on the filesystem, you'll have to login as root on the console (not ssh), which might be an issue on some systems. The fsck equivalent, xfs_repair, is a lot faster than ext4's fsck, though.

	==== ZFS ====		=== ZFS ===
	ZFS is like a combination of RAID, LVM and a filesystem, supporting full checksumming, auto-healing (given sufficient redundancy), compression, encryption, replication, deduplication (if you're brave and have a ''ton'' of memory and can afford some risks) and a lot more. It's a separate chapter and needs a lot more talk than paragraph here.		ZFS is like a combination of RAID, LVM and a filesystem, supporting full checksumming, auto-healing (given sufficient redundancy), compression, encryption, replication, deduplication (if you're brave and have a ''ton'' of memory and can afford some risks) and a lot more. It's a separate chapter and needs a lot more talk than paragraph here.

Roy: /* Filesystem dilemmas */

2022-09-24T23:29:27Z

Filesystem dilemmas

← Older revision		Revision as of 23:29, 24 September 2022
Line 283:		Line 283:
	=== XFS or ext4 or something else ===		=== XFS or ext4 or something else ===
	The choice of filesystem varies for your use. Distros such as RHEL/CentOS has moved to XFS by default from v7. Debian/Ubuntu still sticks to ext4, like most other. I'll discuss my thoughts below on ext4 vs XFS and leave the other filesystems for later.		The choice of filesystem varies for your use. Distros such as RHEL/CentOS has moved to XFS by default from v7. Debian/Ubuntu still sticks to ext4, like most other. I'll discuss my thoughts below on ext4 vs XFS and leave the other filesystems for later.

			=== bcachefs ===
			'''bcachefs''' is the latest newcomer, which started out as '''bcache''', a caching system for linux, using SSDs or NVMEs or similar to cache slower HDDs. It has later evolved to a full filesystem, supporting similar to mirroring, but across multiple drives of different sizes, as some may know the commercial '''unraid''' does. It supports full checksumming and healing of data when errors occur. It stripes data across available media, making sure there's at least two (or more, depending on configuration) drives holding the data to allow for failures. Bcachefs also supports similar to RAID levels similar to RAID5 or RAID6 with ''erasure coding''. The latter is not quite stable at the time of writing (2022-09-25), but should work well soon (fingers crossed).

			==== btrfs ====
			'''btrfs''', pronounced "butterfs" or "betterfs" or just spelled out, is a filesystem that mimics that of ZFS. It has been around for 10 years or so, and has proven stable as a dropin replacement for xfs and ext4 and so on, but its real magic in building a full stack with a full RAID subsystem, hasn't yet proven to be stable. In fact, after 15 years of development, it's hardly usable for that. So you might want to use it for its snapshots and compression, btrfs send/receive etc, but preferably with something below, like mdraid and lvm to help out with redundancy and perhaps SSD caching.

	==== ext4 ====		==== ext4 ====
Line 293:		Line 299:
	==== ZFS ====		==== ZFS ====
	ZFS is like a combination of RAID, LVM and a filesystem, supporting full checksumming, auto-healing (given sufficient redundancy), compression, encryption, replication, deduplication (if you're brave and have a ''ton'' of memory and can afford some risks) and a lot more. It's a separate chapter and needs a lot more talk than paragraph here.		ZFS is like a combination of RAID, LVM and a filesystem, supporting full checksumming, auto-healing (given sufficient redundancy), compression, encryption, replication, deduplication (if you're brave and have a ''ton'' of memory and can afford some risks) and a lot more. It's a separate chapter and needs a lot more talk than paragraph here.

	~~==== btrfs ====~~
	'''btrfs''', pronounced "butterfs" or "betterfs" or just spelled out, is a filesystem that mimics that of ZFS. It has been around for 10 years or so, and has proven stable as a dropin replacement for xfs and ext4 and so on, but its real magic in building a full stack with a full RAID subsystem, hasn't yet proven to be stable. In fact, after 15 years of development, it's hardly usable for that. So you might want to use it for its snapshots and compression, btrfs send/receive etc, but preferably with something below, like mdraid and lvm to help out with redundancy and perhaps SSD caching.

	~~==== bcacahefs ====~~
	'''bcachefs''' is the latest newcomer, which started out as '''bcache''', a caching system for linux, using SSDs or NVMEs or similar to cache slower HDDs. It has later evolved to a full filesystem, supporting similar to mirroring, but across multiple drives of different sizes, as some may know the commercial '''unraid'' does. It supports full checksumming and healing of data when errors occur. It stripes data across available media, making sure there's at least two (or more, depending on configuration) drives holding the data to allow for failures. Bcachefs also supports similar to RAID levels similar to RAID5 or RAID6 with ''erasure coding''. The latter is not quite stable at the time of writing (2022-09-25), but should work well soon (fingers crossed).

	== Low level disk handling ==		== Low level disk handling ==

Roy: /* btrfs */

2022-09-24T23:25:17Z

btrfs

← Older revision		Revision as of 23:25, 24 September 2022
Line 295:		Line 295:

	==== btrfs ====		==== btrfs ====
	btrfs, pronounced "butterfs" or "betterfs" or just spelled out, is a filesystem that mimics that of ZFS. It has been around for 10 years or so, and has proven stable as a dropin replacement for xfs and ext4 and so on, but its real magic in building a full stack with a full RAID subsystem, hasn't yet proven to be stable. In fact, after 15 years of development, it's hardly usable for that. So you might want to use it for its snapshots and compression, btrfs send/receive etc, but preferably with something below, like mdraid and lvm to help out with redundancy and perhaps SSD caching.		'''btrfs''', pronounced "butterfs" or "betterfs" or just spelled out, is a filesystem that mimics that of ZFS. It has been around for 10 years or so, and has proven stable as a dropin replacement for xfs and ext4 and so on, but its real magic in building a full stack with a full RAID subsystem, hasn't yet proven to be stable. In fact, after 15 years of development, it's hardly usable for that. So you might want to use it for its snapshots and compression, btrfs send/receive etc, but preferably with something below, like mdraid and lvm to help out with redundancy and perhaps SSD caching.

			==== bcacahefs ====
			'''bcachefs''' is the latest newcomer, which started out as '''bcache''', a caching system for linux, using SSDs or NVMEs or similar to cache slower HDDs. It has later evolved to a full filesystem, supporting similar to mirroring, but across multiple drives of different sizes, as some may know the commercial '''unraid'' does. It supports full checksumming and healing of data when errors occur. It stripes data across available media, making sure there's at least two (or more, depending on configuration) drives holding the data to allow for failures. Bcachefs also supports similar to RAID levels similar to RAID5 or RAID6 with ''erasure coding''. The latter is not quite stable at the time of writing (2022-09-25), but should work well soon (fingers crossed).

	== Low level disk handling ==		== Low level disk handling ==

Roy: /* ZFS */

2022-09-24T23:13:22Z

ZFS

← Older revision		Revision as of 23:13, 24 September 2022
Line 292:		Line 292:

	==== ZFS ====		==== ZFS ====
	ZFS is like a combination of RAID, LVM and a filesystem, supporting full checksumming, auto-healing (given sufficient redundancy), compression, encryption, replication, deduplication (if you're brave and have a ''ton'' of memory) and a lot more. It's a separate chapter and needs a lot more talk than paragraph here.		ZFS is like a combination of RAID, LVM and a filesystem, supporting full checksumming, auto-healing (given sufficient redundancy), compression, encryption, replication, deduplication (if you're brave and have a ''ton'' of memory and can afford some risks) and a lot more. It's a separate chapter and needs a lot more talk than paragraph here.

	==== btrfs ====		==== btrfs ====