rmauer/netboot.nix
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

add diskco pkg

Browse Source
This commit is contained in:
Richard Mauer
2024-12-29 21:35:58 -05:00
parent dca840c3f1
commit da11e4ce56
142 changed files with 10235 additions and 59 deletions
Download Patch File Download Diff File Expand all files Collapse all files

178
pkgs/disko/docs/HowTo.md Normal file
View File

@@ -0,0 +1,178 @@
# How-to Guide: Disko
## How to use Disko without NixOS
TODO: Still to be documented
## Upgrading From Older disko versions
TODO: Include documentation here.
For now, see the
[upgrade guide](https://github.com/JillThornhill/disko/blob/master/docs/upgrade-guide.md)
## Installing NixOS module
You can use the NixOS module in one of the following ways:
<details>
<summary>Flakes (Current recommendation)</summary>
If you use nix flakes support:
```nix
{
inputs.disko.url = "github:nix-community/disko/latest";
inputs.disko.inputs.nixpkgs.follows = "nixpkgs";
outputs = { self, nixpkgs, disko }: {
# change `yourhostname` to your actual hostname
nixosConfigurations.yourhostname = nixpkgs.lib.nixosSystem {
# change to your system:
system = "x86_64-linux";
modules = [
./configuration.nix
disko.nixosModules.disko
];
};
};
}
```
</details>
<details>
<summary>niv</summary>
First add it to [niv](https://github.com/nmattia/niv):
```console
niv add nix-community/disko
```
Then add the following to your configuration.nix in the `imports` list:
```nix
{
imports = [ "${(import ./nix/sources.nix).disko}/module.nix" ];
}
```
</details>
<details>
<summary>npins</summary>
First add it to [npins](https://github.com/andir/npins):
```console
npins add github nix-community disko
```
Then add the following to your configuration.nix in the `imports` list:
```nix
let
sources = import ./npins;
disko = import sources.disko {};
in
{
imports = [ "${disko}/module.nix" ];
…
}
```
</details>
<details>
<summary>nix-channel</summary>
As root run:
```console
nix-channel --add https://github.com/nix-community/disko/archive/master.tar.gz disko
nix-channel --update
```
Then add the following to your configuration.nix in the `imports` list:
```nix
{
imports = [ <disko/module.nix> ];
}
```
</details>
<details>
<summary>fetchTarball</summary>
Add the following to your configuration.nix:
```nix
{
imports = [ "${builtins.fetchTarball "https://github.com/nix-community/disko/archive/master.tar.gz"}/module.nix" ];
}
```
or with pinning:
```nix
{
imports = let
# replace this with an actual commit id or tag
commit = "f2783a8ef91624b375a3cf665c3af4ac60b7c278";
in [
"${builtins.fetchTarball {
url = "https://github.com/nix-community/disko/archive/${commit}.tar.gz";
# replace this with an actual hash
sha256 = "0000000000000000000000000000000000000000000000000000";
}}/module.nix"
];
}
```
</details>
## Using the NixOS module
```nix
{
# checkout the example folder for how to configure different disko layouts
disko.devices = {
disk = {
vdb = {
device = "/dev/disk/by-id/some-disk-id";
type = "disk";
content = {
type = "gpt";
partitions = {
ESP = {
type = "EF00";
size = "100M";
content = {
type = "filesystem";
format = "vfat";
mountpoint = "/boot";
mountOptions = [ "umask=0077" ];
};
};
root = {
size = "100%";
content = {
type = "filesystem";
format = "ext4";
mountpoint = "/";
};
};
};
};
};
};
};
}
```
this will configure `fileSystems` and other required NixOS options to boot the
specified configuration.
If you are on an installer, you probably want to disable `enableConfig`.
disko will create the scripts `disko-create` and `disko-mount` which can be used
to create/mount the configured disk layout.

22
pkgs/disko/docs/INDEX.md Normal file
View File

@@ -0,0 +1,22 @@
# disko - Declarative disk partitioning
<img title="" src="./logo.png" alt="" width="220">
## Table of Contents
### For users
- [README](../README.md)
- [Quickstart](./quickstart.md)
- [System Requirements](./requirements.md)
- [How to Guide](./HowTo.md)
- [Disko-Install](./disko-install.md)
- [Disko-Images](./disko-images.md)
- [Support Matrix](./supportmatrix.md)
- [Reference](./reference.md)
- [Upgrade Guide](./upgrade-guide.md)
- [Migrating to the new GPT layout](./table-to-gpt.md)
### For contributors
- [Running and debugging tests](./testing.md)

122
pkgs/disko/docs/disko-images.md Normal file
View File

@@ -0,0 +1,122 @@
# Generating Disk Images with Secrets Included using Disko
Using Disko on NixOS allows you to efficiently create `.raw` VM images from a
system configuration. The generated image can be used as a VM or directly
written to a physical drive to create a bootable disk. Follow the steps below to
generate disk images:
## Generating the `.raw` VM Image
1. **Create a NixOS configuration that includes the disko and the disk
configuration of your choice**
In the this example we create a flake containing a nixos configuration for
`myhost`.
```nix
# save this as flake.nix
{
description = "A disko images example";
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
disko.url = "github:nix-community/disko/latest";
disko.inputs.nixpkgs.follows = "nixpkgs";
};
outputs = { self, disko, nixpkgs }: {
nixosConfigurations.myhost = nixpkgs.lib.nixosSystem {
system = "x86_64-linux";
modules = [
# You can get this file from here: https://github.com/nix-community/disko/blob/master/example/simple-efi.nix
./simple-efi.nix
disko.nixosModules.disko
({ config, ... }: {
# shut up state version warning
system.stateVersion = config.system.nixos.version;
# Adjust this to your liking.
# WARNING: if you set a too low value the image might be not big enough to contain the nixos installation
disko.devices.disk.main.imageSize = "10G";
})
];
};
};
}
```
2. **Build the disko image script:** Replace `myhost` in the command below with
your specific system configuration name:
```console
nix build .#nixosConfigurations.myhost.config.system.build.diskoImagesScript
```
3. **Execute the disko image script:** Execute the generated disko image script.
Running `./result --help` will output the available options:
```console
./result --help
Usage: $script [options]
Options:
* --pre-format-files <src> <dst>
copies the src to the dst on the VM, before disko is run
This is useful to provide secrets like LUKS keys, or other files you need for formatting
* --post-format-files <src> <dst>
copies the src to the dst on the finished image
These end up in the images later and is useful if you want to add some extra stateful files
They will have the same permissions but will be owned by root:root
* --build-memory <amt>
specify the amount of memory in MiB that gets allocated to the build VM
This can be useful if you want to build images with a more involed NixOS config
The default is 1024 MiB
```
An example run may look like this:
```
sudo ./result --build-memory 2048
```
The script will generate the actual image outside of the nix store in the
current working directory. The create image names depend on the names used in
`disko.devices.disk` attrset in the NixOS configuration. In our code example
it will produce the following image:
```
$ ls -la main.raw
.rw-r--r-- root root 10 GB 2 minutes ago main.raw
```
## Additional Configuration
- For custom image name output, define the image name in your Disko configuration:
```console
disko.devices.disk.<drive>.imageName = "nixos-x86_64-linux-generic-btrfs"; # Set your preferred name
```
The image scirpt will produce `nixos-x86_64-linux-generic-btrfs.raw` instead of `<drive>.raw`.
- For virtual drive use, define the image size in your Disko configuration:
```console
disko.devices.disk.<drive>.imageSize = "32G"; # Set your preferred size
```
## Understanding the Image Generation Process
1. Files specified in `--pre-format-files` and `--post-format-files` are
temporarily copied to `/tmp`.
2. Files are then moved to their respective locations in the VM both before and
after the Disko partitioning script runs.
3. The NixOS installer is executed, having access only to `--post-format-files`.
4. Upon installer completion, the VM is shutdown, and the `.raw` disk files are
moved to the local directory.
> **Note**: The auto-resizing feature is currently not available in Disko.
> Contributions for this feature are welcomed. Adjust the `imageSize`
> configuration to prevent issues related to file size and padding.
By following these instructions and understanding the process, you can smoothly
generate disk images with Disko for your NixOS system configurations.

233
pkgs/disko/docs/disko-install.md Normal file
View File

@@ -0,0 +1,233 @@
# disko-install
**disko-install** enhances the normal `nixos-install` with disko's partitioning
feature. It can be started from the NixOS installer but it can also be used to
create bootable USB-Sticks from your normal workstation. Furthermore
`disko-install` has a mount mode that will only mount but not destroy existing
partitions. The mount mode can be used to mount and repair existing NixOS
installations. This document provides a comprehensive guide on how to use
**disko-install**, including examples for typical usage scenarios.
## Requirements
- a Linux system with Nix installed.
- a target disk or partition for the NixOS installation.
- a Nix flake that defines your desired NixOS configuration.
## Usage
### Fresh Installation
For a fresh installation, where **disko-install** will handle partitioning and
setting up the disk, use the following syntax:
```console
sudo nix run 'github:nix-community/disko/latest#disko-install' -- --flake <flake-url>#<flake-attr> --disk <disk-name> <disk-device>
```
Example:
First run `nixos-generate-config --root /tmp/config --no-filesystems` and edit
`configuration.nix` to your liking.
Then add the following `flake.nix` inside `/tmp/config/etc/nixos`. In this
example we assume a system that has been booted with EFI:
```nix
{
inputs.nixpkgs.url = "github:nixos/nixpkgs?ref=nixos-unstable";
inputs.disko.url = "github:nix-community/disko/latest";
inputs.disko.inputs.nixpkgs.follows = "nixpkgs";
outputs = { self, disko, nixpkgs }: {
nixosConfigurations.mymachine = nixpkgs.legacyPackages.x86_64-linux.nixos [
./configuration.nix
disko.nixosModules.disko
{
disko.devices = {
disk = {
main = {
# When using disko-install, we will overwrite this value from the commandline
device = "/dev/disk/by-id/some-disk-id";
type = "disk";
content = {
type = "gpt";
partitions = {
MBR = {
type = "EF02"; # for grub MBR
size = "1M";
priority = 1; # Needs to be first partition
};
ESP = {
type = "EF00";
size = "500M";
content = {
type = "filesystem";
format = "vfat";
mountpoint = "/boot";
mountOptions = [ "umask=0077" ];
};
};
root = {
size = "100%";
content = {
type = "filesystem";
format = "ext4";
mountpoint = "/";
};
};
};
};
};
};
};
}
];
};
}
```
Identify the device name that you want to install NixOS to:
```console
$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
sda 8:0 1 14.9G 0 disk
└─sda1 8:1 1 14.9G 0 part
zd0 230:0 0 10G 0 disk
├─zd0p1 230:1 0 500M 0 part
└─zd0p2 230:2 0 9.5G 0 part /mnt
nvme0n1 259:0 0 1.8T 0 disk
├─nvme0n1p1 259:1 0 1G 0 part /boot
├─nvme0n1p2 259:2 0 16M 0 part
├─nvme0n1p3 259:3 0 250G 0 part
└─nvme0n1p4 259:4 0 1.6T 0 part
```
In our example, we want to install to a USB-stick (/dev/sda):
```console
$ sudo nix run 'github:nix-community/disko/latest#disko-install' -- --flake '/tmp/config/etc/nixos#mymachine' --disk main /dev/sda
```
Afterwards you can test your USB-stick by either selecting during the boot or
attaching it to a qemu-vm:
```
$ sudo qemu-kvm -enable-kvm -hda /dev/sda
```
### Persisting boot entries to EFI vars flash
**disko-install** is designed for NixOS installations on portable storage or
disks that may be transferred between computers. As such, it does not modify the
host's NVRAM by default. To ensure your NixOS installation boots seamlessly on
new hardware or to prioritize it in your current machine's boot order, use the
--write-efi-boot-entries option:
```console
$ sudo nix run 'github:nix-community/disko/latest#disko-install' -- --write-efi-boot-entries --flake '/tmp/config/etc/nixos#mymachine' --disk main /dev/sda
```
This command installs NixOS with **disko-install** and sets the newly installed
system as the default boot option, without affecting the flexibility to boot
from other devices if needed.
### Using disko-install in an offline installer
If you want to use **disko-install** from a custom installer without internet,
you need to make sure that in addition to the toplevel of your NixOS closure
that you plan to install, it also needs to contain **diskoScript** and all the
flake inputs for evaluation.
#### Example configuration to install
Add this to your flake.nix output:
```nix
{
nixosConfigurations.your-machine = inputs.nixpkgs.lib.nixosSystem {
system = "x86_64-linux";
# to pass this flake into your configuration (see the example below)
specialArgs = {inherit self;};
modules = [
{
# TODO: add your NixOS configuration here, don't forget your hardware-configuration.nix as well!
boot.loader.systemd-boot.enable = true;
imports = [ self.inputs.disko.nixosModules.disko ];
disko.devices = {
disk = {
vdb = {
device = "/dev/disk/by-id/some-disk-id";
type = "disk";
content = {
type = "gpt";
partitions = {
ESP = {
type = "EF00";
size = "500M";
content = {
type = "filesystem";
format = "vfat";
mountpoint = "/boot";
mountOptions = [ "umask=0077" ];
};
};
root = {
size = "100%";
content = {
type = "filesystem";
format = "ext4";
mountpoint = "/";
};
};
};
};
};
};
};
}
];
};
}
```
#### Example for a NixOS installer
```nix
# `self` here is referring to the flake `self`, you may need to pass it using `specialArgs` or define your NixOS installer configuration
# in the flake.nix itself to get direct access to the `self` flake variable.
{ pkgs, self, ... }:
let
dependencies = [
self.nixosConfigurations.your-machine.config.system.build.toplevel
self.nixosConfigurations.your-machine.config.system.build.diskoScript
self.nixosConfigurations.your-machine.config.system.build.diskoScript.drvPath
self.nixosConfigurations.your-machine.pkgs.stdenv.drvPath
# https://github.com/NixOS/nixpkgs/blob/f2fd33a198a58c4f3d53213f01432e4d88474956/nixos/modules/system/activation/top-level.nix#L342
self.nixosConfigurations.your-machine.pkgs.perlPackages.ConfigIniFiles
self.nixosConfigurations.your-machine.pkgs.perlPackages.FileSlurp
(self.nixosConfigurations.your-machine.pkgs.closureInfo { rootPaths = [ ]; }).drvPath
] ++ builtins.map (i: i.outPath) (builtins.attrValues self.inputs);
closureInfo = pkgs.closureInfo { rootPaths = dependencies; };
in
# Now add `closureInfo` to your NixOS installer
{
environment.etc."install-closure".source = "${closureInfo}/store-paths";
environment.systemPackages = [
(pkgs.writeShellScriptBin "install-nixos-unattended" ''
set -eux
# Replace "/dev/disk/by-id/some-disk-id" with your actual disk ID
exec ${pkgs.disko}/bin/disko-install --flake "${self}#your-machine" --disk vdb "/dev/disk/by-id/some-disk-id"
'')
];
}
```
Also see the
[NixOS test of disko-install](https://github.com/nix-community/disko/blob/master/tests/disko-install/default.nix)
that also runs without internet.

26
pkgs/disko/docs/interactive-vm.md Normal file
View File

@@ -0,0 +1,26 @@
# Running Interactive VMs with disko
disko now exports its own flavor of interactive VMs (similiar to
config.system.build.vm). Simply import the disko module and build the vm runner
with:
```
nix run -L '.#nixosConfigurations.mymachine.config.system.build.vmWithDisko'
```
You can configure the VM using the `virtualisation.vmVariantWithDisko` NixOS
option:
```nix
{
virtualisation.vmVariantWithDisko = {
virtualisation.fileSystems."/persist".neededForBoot = true;
# For running VM on macos: https://www.tweag.io/blog/2023-02-09-nixos-vm-on-macos/
# virtualisation.host.pkgs = inputs.nixpkgs.legacyPackages.aarch64-darwin;
};
}
```
extraConfig that is set in disko.tests.extraConfig is also applied to the
interactive VMs. imageSize of the VMs will be determined by the imageSize in the
disk type in your disko config. memorySize is set by disko.memSize

BIN
pkgs/disko/docs/logo.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 66 KiB

225
pkgs/disko/docs/quickstart.md Normal file
View File

@@ -0,0 +1,225 @@
# disko - Declarative disk partitioning
<img src="./logo.png" title="" alt="Project logo" width="247">
[Documentation Index](./INDEX.md)
## Quickstart Guide
This tutorial describes how to install NixOS on a single disk system using
`disko`. You will also need to refer to the NixOS manual, which is available
[here.](https://nixos.org/manual/nixos/stable/index.html#ex-config)
Please note that `disko` will reformat the entire disk and overwrite any
existing partitions. Dual booting with other operating systems is not supported.
### Step 1: Choose a Disk Configuration
Real-world templates are provided in this
[repository](https://github.com/nix-community/disko-templates).
More disk layouts for all filesystems can be also found in the
[example](https://github.com/nix-community/disko/tree/master/example) directory
of disko. However these examples are also used for regression tests in disko and
may have uncommon options in them to fully exercise all features of disko, that
you may need to change or remove.
Decide which of these layouts best suits your requirements. If you're not sure
which layout to pick, use the
[single-disk-ext4](https://github.com/nix-community/disko-templates/blob/main/single-disk-ext4/disko-config.nix)
configuration. This layout is compatible with both BIOS and EFI systems.
Refer to the [reference manual](./reference.md) for more information about the
sample layouts and how to build your own configuration.
To copy a template use this command in your nixos configuration directory:
```
nix flake init --template github:nix-community/disko-templates#single-disk-ext4
```
This will write a file called `disko-config.nix` into the current directory.
Import this file in your NixOS configuration:
```nix
{
imports = [ ./disko-config.nix ];
}
```
If you want to choose a layout from the disko example directory instead, you'll
need to make a note of the URL to the raw file. To do this, open the file in
Github. Immediately below the list of contributors, you will see a button
labelled 'RAW' near the right hand side. Click this. The URL of the raw file
will appear in the search bar of your browser. It will look something like this:
```
https://raw.githubusercontent.com/nix-community/disko/master/example/hybrid.nix
```
### Step 2: Boot the installer
Download the NixOS ISO image from the NixOS
[download page](https://nixos.org/download.html#nixos-iso), and create a
bootable USB drive following the instructions
in [Section 2.4.1 "Booting from a USB flash drive"](https://nixos.org/manual/nixos/stable/index.html#sec-booting-from-usb) of
the NixOS manual. Boot the machine from this USB drive.
### Step 3: Retrieve the disk name
Identify the name of your system disk by using the `lsblk` command as follows:
```console
lsblk
```
The output from this command will look something like this:
```
NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
nvme0n1     259:0    0   1,8T  0 disk
```
In this example, an empty NVME SSD with 2TB space is shown with the disk name
"nvme0n1". Make a note of the disk name as you will need it later.
### Step 4: Copy the disk configuration to your machine
In Step 1, you chose a disk layout configuration from the
[examples directory](https://github.com/nix-community/disko/tree/master/example),
and made a note of its URL.
Your configuration needs to be saved on the new machine for example
as /tmp/disk-config.nix. You can do this using the `curl` command to download
from the url you noted above, using the `-o` option to save the file as
disk-config.nix. Your commands would look like this if you had chosen the hybrid
layout:
```console
cd /tmp
curl https://raw.githubusercontent.com/nix-community/disko/master/example/hybrid.nix -o /tmp/disk-config.nix
```
### Step 5: Adjust the device in the disk configuration
Inside the disk-config.nix the device needs to point to the correct disk name.
Open the configuration in your favorite editor i.e.:
```console
nano /tmp/disk-config.nix
```
Replace `<disk-name>` with the name of your disk obtained in Step 1.
```nix
# ...
main = {
type = "disk";
device = "<disk-name>";
content = {
type = "gpt";
# ...
```
### Step 6: Run disko to partition, format and mount your disks
The following step will partition and format your disk, and mount it to `/mnt`.
**Please note: This will erase any existing data on your disk.**
```console
sudo nix --experimental-features "nix-command flakes" run github:nix-community/disko/latest -- --mode destroy,format,mount /tmp/disk-config.nix
```
After the command has run, your file system should have been formatted and
mounted. You can verify this by running the following command:
```console
mount | grep /mnt
```
The output should look like this if your disk name is `nvme0n1`.
```
/dev/nvme0n1p1 on /mnt type ext4 (rw,relatime,stripe=2)
/dev/nvme0n1p2 on /mnt/boot type vfat (rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,errors=remount-ro)
```
### Step 7: Complete the NixOS installation.
Your disks have now been formatted and mounted, and you are ready to complete
the NixOS installation as described in the
[NixOS manual](https://nixos.org/manual/nixos/stable/index.html#sec-installation) -
see the section headed "**Installing**", Steps 3 onwards. However, you will need
to include the partitioning and formatting configurations that you copied into
`/tmp/disk-config.nix` in your configuration, rather than allowing NixOS to
generate information about your file systems. When you are configuring the
system as per Step 4 of the manual, you should:
a) Include the `no-filesystems` switch when using the `nixos-generate-config`
command to generate an initial `configuration.nix`. You will be supplying the
file system configuration details from `disk-config.nix`. Your CLI command to
generate the configuration will be:
```console
nixos-generate-config --no-filesystems --root /mnt
```
This will create the file `configuration.nix` in `/mnt/etc/nixos`.
b) Move the `disko` configuration to /etc/nixos
```console
mv /tmp/disk-config.nix /mnt/etc/nixos
```
c) You can now edit `configuration.nix` as per your requirements. This is
described in Step 4 of the manual. For more information about configuring your
system, refer to the NixOS manual.
[Chapter 6, Configuration Syntax](https://nixos.org/manual/nixos/stable/index.html#sec-configuration-syntax)
describes the NixOS configuration syntax, and
[Appendix A, Configuration Options](https://nixos.org/manual/nixos/stable/options.html)
gives a list of available options. You can find also find a minimal example of a
NixOS configuration in the manual:
[Example: NixOS Configuration](https://nixos.org/manual/nixos/stable/index.html#ex-config).
d) When editing `configuration.nix`, you will need to add the `disko` NixOS
module and `disk-config.nix` to the imports section. This section will already
include the file `./hardware-configuration.nix`, and you can add the new entries
just below this. This section will now include:
```nix
imports =
[ # Include the results of the hardware scan.
./hardware-configuration.nix
"${builtins.fetchTarball "https://github.com/nix-community/disko/archive/master.tar.gz"}/module.nix"
./disk-config.nix
];
```
e) If you chose the hybrid-partition scheme, then choose `grub` as a bootloader,
otherwise follow the recommendations in Step 4 of the **Installation** section
of the NixOS manual. The following configuration for `grub` works for both EFI
and BIOS systems. Add this to your configuration.nix, commenting out the
existing lines that configure `systemd-boot`. The entries will look like this:
**Note:** Its not necessary to set `boot.loader.grub.device` here, since Disko
will take care of that automatically.
```nix
# ...
#boot.loader.systemd-boot.enable = true;
#boot.loader.efi.canTouchEfiVariables = true;
boot.loader.grub.enable = true;
boot.loader.grub.efiSupport = true;
boot.loader.grub.efiInstallAsRemovable = true;
# ...
```
f) Finish the installation and reboot your machine,
```console
nixos-install
reboot
```

47
pkgs/disko/docs/reference.md Normal file
View File

@@ -0,0 +1,47 @@
# Reference Manual: disko
## Module Options
We are currently having issues being able to generate proper module option
documentation for our recursive disko types. However you can read the available
options [here](https://github.com/nix-community/disko/tree/master/lib/types).
Combined with the
[examples](https://github.com/nix-community/disko/tree/master/example) this
hopefully gives you an overview.
## Command Line Options
```
Usage: ./disko [options] disk-config.nix
or ./disko [options] --flake github:somebody/somewhere#disk-config
With flakes, disk-config is discovered first under the .diskoConfigurations top level attribute
or else from the disko module of a NixOS configuration of that name under .nixosConfigurations.
Options:
* -m, --mode mode
set the mode, either distroy, format, mount, format,mount or destroy,format,mount
destroy: unmount filesystems and destroy partition tables of the selected disks
format: create partition tables, zpools, lvms, raids and filesystems if they don't exist yet
mount: mount the partitions at the specified root-mountpoint
format,mount: run format and mount in sequence
destroy,format,mount: run all three modes in sequence. Previously known as --mode disko
* -f, --flake uri
fetch the disko config relative to this flake's root
* --arg name value
pass value to nix-build. can be used to set disk-names for example
* --argstr name value
pass value to nix-build as string
* --root-mountpoint /some/other/mnt
where to mount the device tree (default: /mnt)
* --dry-run
just show the path to the script instead of running it
* --no-deps
avoid adding another dependency closure to an in-memory installer
requires all necessary dependencies to be available in the environment
* --debug
run with set -x
* --yes-wipe-all-disks
skip the safety check for destroying partitions, useful for automation
```

9
pkgs/disko/docs/requirements.md Normal file
View File

@@ -0,0 +1,9 @@
# disko - Declarative disk partitioning
<img title="" src="./logo.jpeg" alt="" width="220">
[Documentation Index](./INDEX.md)
## System Requirements
TODO: Populate this

9
pkgs/disko/docs/supportmatrix.md Normal file
View File

@@ -0,0 +1,9 @@
# disko - Declarative disk partitioning
<img title="" src="./logo.jpeg" alt="" width="220">
[Documentation Index](./INDEX.md)
## Support Matrix
TODO: Populate this

137
pkgs/disko/docs/table-to-gpt.md Normal file
View File

@@ -0,0 +1,137 @@
# Migrating to the new GPT layout
## Situation
When evaluating your NixOS system closure the following trace appears:
```
trace: warning: The legacy table is outdated and should not be used. We recommend using the gpt type instead.
Please note that certain features, such as the test framework, may not function properly with the legacy table type.
If you encounter errors similar to:
"error: The option `disko.devices.disk.disk1.content.partitions."[definition 1-entry 1]".content._config` is read-only, but it's set multiple times,"
this is likely due to the use of the legacy table type.
```
The solution is to migrate to the new `gpt` layout type.
## Precondition
Disko was set up with
- `type = "table"` and
- `format = "gpt"`,
for example like this:
```nix
{
disko.devices.disk.example = {
type = "disk";
device = "/dev/nvme0n1";
content = {
type = "table";
format = "gpt";
partitions = [
{
name = "ESP";
start = "0";
end = "512MiB";
fs-type = "fat32";
bootable = true;
content = {
type = "filesystem";
format = "vfat";
mountpoint = "/boot";
};
}
{
name = "root";
start = "512MiB";
end = "100%";
content.format = "ext4";
}
];
};
};
}
```
## Remediation
The new GPT layout (`type = "gpt"`) uses partlabels to realize the partiton
numbering. For this reason you have to manually set up partition labels, if you
want to resolve this issue.
### Create GPT partition labels
For each partition involved, create the partition label from these components:
- The partition number (e.g. /dev/nvme0n**1**, or /dev/sda**1**)
- The parent type in your disko config (value of
`disko.device.disk.example.type = "disk";`)
- The parent name in your disko config (attribute name of
`disko.devices.disk.example`, so `example` in this example)
- The partition name in your disko config (attribute name of
`disko.devices.disk.content.partitions.*.name`)
```bash
# sgdisk -c 1:disk-example-ESP /dev/nvme0n1
# sgdisk -c 2:disk-example-zfs /dev/nvme0n1
Warning: The kernel is still using the old partition table.
The new table will be used at the next reboot or after you
run partprobe(8) or kpartx(8)
The operation has completed successfully.
```
### Update disko configuration
Make the following changes to your disko configuration:
1. Set `disko.devices.disk.example.content.type = "gpt"`
1. Remove `disko.devices.disk.example.content.format`
1. Convert `disko.devices.disk.example.content.partitions` to an attribute set and
promote the `name` field to the key for its partition
1. Add a `priority` field to each partition, to reflect the intended partition
number
Then rebuild your system and reboot.
### Recovering from mistake
If you made a mistake here, your system will be waiting for devices to appear,
and then run into timeouts. You can easily recover from this, since rebooting
into an old generation will still use the legacy way of numbering of partitions.
## Result
The fixed disko configuration would look like this:
```nix
{
disko.devices.disk.example = {
type = "disk";
device = "/dev/nvme0n1";
content = {
type = "gpt";
partitions = {
ESP = {
size = "512MiB";
type = "EF00";
priority = 1;
content = {
type = "filesystem";
format = "vfat";
mountpoint = "/boot";
mountOptions = [ "umask=0077" ];
};
};
root = {
size = "100%";
priority = 2;
content.format = "ext4";
};
};
};
};
}
```

160
pkgs/disko/docs/testing.md Normal file
View File

@@ -0,0 +1,160 @@
# Running and debugging tests
Disko makes extensive use of VM tests. All examples you can find in
[the example directory](../example) have a respective test suite that verifies
the example is working in [the tests directory](../tests/). They utilize the
[NixOS test functionality](https://nixos.org/manual/nixos/stable/#sec-nixos-tests).
We use a wrapper around this called `makeDiskoTest`. There is currently (as of
2024-10-16) no documentation for all its arguments, but you can have a look at
[its current code](https://github.com/nix-community/disko/blob/master/lib/tests.nix#L44C5-L58C10),
that should already be helpful.
However, you don't need to know about all of the inner workings to interact with
the tests effectively. For some of the most common operations, see the sections
below.
## Run just one of the tests
```sh
nix build --no-link .#checks.x86_64-linux.simple-efi
```
This will run the test in [`tests/simple-efi.nix`](../tests/simple-efi.nix),
which builds a VM with all disks specified in the
[`example/simple-efi.nix`](../example/simple-efi.nix) config connected as
virtual devices, run disko to format them, reboot, verify the VM boots properly,
and then run the code specified in `extraTestScript` to validate that the
partitions have been created and were mounted as expected.
### How `extraTestScript` works
This is written in Python. The most common lines you'll see look something like
this:
```python
machine.succeed("test -b /dev/md/raid1");
machine.succeed("mountpoint /");
```
The `machine` in these is a machine object, which defines
[a multitude of functions to interact with and test](https://nixos.org/manual/nixos/stable/#ssec-machine-objects),
assumptions about the state of the VM after formatting and rebooting.
Disko currently (as of 2024-10-16) doesn't have any tests that utilize multiple
VMs at once, so the only machine available in these scripts is always just the
default `machine`.
## Debugging tests
If you make changes to disko, you might break a test, or you may want to modify
a test to prevent regressions. In these cases, running the full test with
`nix build` every time is time-consuming and tedious.
Instead, you can build and then run the VM for a test in interactive mode. This
will create the VM and all virtual disks as required by the test's config, but
allow you to interact with the machine on a terminal afterwards.
First, build the interactive test driver and run it:
```
nix build .#checks.x86_64-linux.simple-efi.driverInteractive
result/bin/nixos-test-driver --keep-vm-state
```
This will open an IPython prompt in which you can use th same objects and
functions as in `extraTestScript`. In there, you can run
```
machine.shell_interact()
```
to start the VM and attach the terminal to it. This will also open a QEMU
window, in which you can log in as `root` with no password, but that makes it
more difficult to paste input and output. Instead, wait for the systemd messages
to settle down, and then **simply start typing**. This should make a `$` prompt
appear, indicating that the machine is ready to take commands. The NixOS manual
calls out a few special messages to look for, but these are buried underneath
the systemd logs.
Once you are in this terminal, you're running commands on the VM. The only thing
that doesn't work here is the `exit` command. Instead, you need to press Ctrl+D
and wait for a second to return to the IPython prompt.
In summary, a full session looks something like this:
```
# nix build .#checks.x86_64-linux.simple-efi.driverInteractive
# result/bin/nixos-test-driver --keep-vm-state
start all VLans
start vlan
running vlan (pid 146244; ctl /tmp/vde1.ctl)
(finished: start all VLans, in 0.00 seconds)
additionally exposed symbols:
machine,
vlan1,
start_all, test_script, machines, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, Machine
>>> machine.shell_interact()
machine: waiting for the VM to finish booting
machine: starting vm
machine: QEMU running (pid 146286)
machine # [ 0.000000] Linux version 6.6.48 (nixbld@localhost) (gcc (GCC) 13.3.0, GNU ld (GNU Binutils) 2.42) #1-NixOS SMP PREEMPT_DYNAMIC Thu Aug 29 15:33:59 UTC 2024
machine # [ 0.000000] Command line: console=ttyS0 panic=1 boot.panic_on_fail clocksource=acpi_pm loglevel=7 net.ifnames=0 init=/nix/store/0a52bbvxr5p7xijbbk17qqlk8xm4790y-nixos-system-machine-test/init regInfo=/nix/store/3sh5nl75bnj1jg87p5gcrdzs0lk154ma-closure-info/registration console=ttyS0
machine # [ 0.000000] BIOS-provided physical RAM map:
...
... more systemd messages
...
machine # [ 6.135577] dhcpcd[679]: DUID 00:01:00:01:2e:a2:74:e6:52:54:00:12:34:56
machine # [ 6.142785] systemd[1]: Finished Kernel Auditing.
machine: Guest shell says: b'Spawning backdoor root shell...\n'
machine: connected to guest root shell
machine: (connecting took 6.61 seconds)
(finished: waiting for the VM to finish booting, in 6.99 seconds)
machine: Terminal is ready (there is no initial prompt):
machine # [ 6.265451] 8021q: 802.1Q VLAN Support v1.8
machine # [ 6.186797] nsncd[669]: Oct 16 13:11:55.010 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 3s }, path: "/var/run/nscd/socket"
...
... more systemd messages
...
machine # [ 12.376900] systemd[1]: Reached target Host and Network Name Lookups.
machine # [ 12.379265] systemd[1]: Reached target User and Group Name Lookups.
$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
fd0 2:0 1 4K 0 disk
sr0 11:0 1 1024M 0 rom
vda 253:0 0 1G 0 disk /
vdb 253:16 0 4G 0 disk
├─vdb1 253:17 0 500M 0 part
└─vdb2 253:18 0 3.5G 0 part
```
You can find some additional details in
[the NixOS manual's section on interactive testing](https://nixos.org/manual/nixos/stable/#sec-running-nixos-tests-interactively).
## Running all tests at once
If you have a bit of experience, you might be inclined to run `nix flake check`
to run all tests at once. However, we instead recommend using
[nix-fast-build](https://github.com/Mic92/nix-fast-build). The reason for this
is that each individual test takes a while to run, but only uses <=4GiB of RAM
and a limited amount of CPU resources. This means they can easily be evaluated
and run in parallel to save time, but `nix` doesn't to that, so a full test run
takes >40 minutes on a mid-range system. With `nix-fast-build` you can scale up
the number of workers depending on your system's capabilities. It also utilizes
[`nix-output-monitor`](https://github.com/maralorn/nix-output-monitor) to give
you a progress indicator during the build process as well. For example, on a
machine with 16GB of RAM, this gives you a 2x speed up without clogging your
system:
```sh
nix shell nixpkgs#nix-fast-build
nix-fast-build --no-link -j 2 --eval-workers 2 --flake .#checks
```
You can try higher numbers if you want to. Be careful with scaling up
`--eval-workers`, each of these will use 100% of a CPU core and they don't leave
any time for hyperthreading, so 4 workers will max out a a CPU with 4 cores and
8 threads, potentially rendering your system unresponsive! `-j` is less
dangerous to scale up, but you probably don't want to go higher than
`(<ram in your system> - 4GB)/4GB` to prevent excessive swap usage, which will
would slow down the test VMs to a crawl.

173
pkgs/disko/docs/upgrade-guide.md Normal file
View File

@@ -0,0 +1,173 @@
# 2023-07-09 121df48
Changes:
- BTRFS subvolumes are mounted if and only their `mountpoint` is set.
Especially, if you have specific mount options for a subvolume (through
`mountOptions`), make sure to set `mountpoint` otherwise the subvolume will not
be mounted.
This change allows more flexibility when using BTRFS, giving access to its
volume management functionality.
It allows layouts as the following:
```nix
content = {
type = "btrfs";
# BTRFS partition is not mounted as it doesn't set a mountpoint explicitly
subvolumes = {
# This subvolume will not be mounted
"SYSTEM" = { };
# mounted as "/"
"SYSTEM/rootfs" = {
mountpoint = "/";
};
# mounted as "/nix"
"SYSTEM/nix" = {
mountOptions = [ "compress=zstd" "noatime" ];
mountpoint = "/nix";
};
# This subvolume will not be mounted
"DATA" = { };
# mounted as "/home"
"DATA/home" = {
mountOptions = [ "compress=zstd" ];
mountpoint = "/home";
};
# mounted as "/var/www"
"DATA/www" = {
mountpoint = "/var/www";
};
};
};
```
corresponding to the following BTRFS layout:
```
BTRFS partition # not mounted
|
|-SYSTEM # not mounted
| |-rootfs # mounted as "/"
| |-nix # mounted as "/nix"
|
|-DATA # not mounted
|-home # mounted as "/home"
|-www # mounted as "/var/www"
```
# 2023-04-07 7d70009
Changes:
- ZFS datasets have been split into two types: `zfs_fs` and `zfs_volume`.
- The `zfs_type` attribute has been removed.
- The size attribute is now only available for `zfs_volume`.
Updated example/zfs.nix file:
```nix
{
datasets = {
zfs_fs = {
type = "zfs_fs";
mountpoint = "/zfs_fs";
options."com.sun:auto-snapshot" = "true";
};
zfs_unmounted_fs = {
type = "zfs_fs";
options.mountpoint = "none";
};
zfs_legacy_fs = {
type = "zfs_fs";
options.mountpoint = "legacy";
mountpoint = "/zfs_legacy_fs";
};
zfs_testvolume = {
type = "zfs_volume";
size = "10M";
content = {
type = "filesystem";
# ...
}
```
Note: The `zfs_type` attribute has been replaced with a type attribute for each
dataset, and the `size` attribute is only available for `zfs_volume`. These
changes have been reflected in the `example/zfs.nix` file.
# 2023-04-07 654ecb3
The `lvm_lv` type is always part of an `lvm_vg` and it is no longer necessary to
specify the type.
This means that if you were using the `lvm_lv` type in your code, you should
remove it. For example, if you were defining an `lvm_lv` type like this:
```nix
{
type = "lvm_lv";
size = "10G";
# ...
}
```
You should now define it like this:
```nix
{
size = "10G";
# ...
}
```
Note that the `type` field is no longer necessary and should be removed from
your code.
# 2023-04-07 d6f062e
Partition types are now always part of a table and cannot be specified
individually anymore. This change makes the library more consistent and easier
to use.
Example of how to change code:
Before:
```nix
{
type = "partition";
name = "ESP";
start = "1MiB";
end = "100MiB";
part-type = "primary";
}
```
After:
```nix
{
name = "ESP";
start = "1MiB";
end = "100MiB";
part-type = "primary";
}
```
Note that the `type` field is no longer necessary and should be removed from
your code.
# 2023-03-22 2624af6
disk config now needs to be inside a disko.devices attrset always
# 2023-03-22 0577409
the extraArgs option in the luks type was renamed to extraFormatArgs
# 2023-02-14 6d630b8
btrfs, `btrfs_subvol` filesystem and `lvm_lv` extraArgs are now lists