Julian Tölle
fdfb284533
The base image used requires ~0.42Gi. Even if the uploaded image is smaller, those bytes are currently not overwritten and still part of the stored snapshot. By zeroing the root disk first, those unwanted bytes are removed and not stored with the snapshot. This has two benefits: 1. Snapshots are billed by their compressed (shown) size, so small images are now a bit cheaper. 2. The time it takes to create a server from the snapshot scales with the snapshot size, so smaller snapshots means the server can start more quickly. This reduces the size of an example Talos x86 image from 0.42Gi before, to 0.2Gi afterwards. An example Flatcar image was 0.47Gi before, and still has that size with this patch. There are two ways to zero out the disk: - `dd if=/dev/zero of=/dev/sda` actually writes zeroes to every block on the device. This takes around a minute to do. - `blkdiscard /dev/sda` talks to the disk direclty and instructs it to discard all blocks. This only takes around 5 seconds. As both have the same effect on image size, but `blkdiscard` is SO MUCH faster, I have decided to use it. Even though only small images benefit from this, this is now enabled by default as the downside (5 second slower upload) does not justify additional flags or options to enable/disable this. Closes #96 |
||
|---|---|---|
| .github | ||
| cmd | ||
| docs | ||
| hcloudimages | ||
| internal | ||
| scripts | ||
| .gitignore | ||
| .goreleaser.yaml | ||
| CHANGELOG.md | ||
| go.mod | ||
| go.sum | ||
| go.work | ||
| go.work.sum | ||
| LICENSE | ||
| main.go | ||
| README.md | ||
| renovate.json | ||
hcloud-upload-image
Quickly upload any raw disk images into your Hetzner Cloud projects!
About
The Hetzner Cloud API does not support uploading disk images directly, and it only provides a limited set of default images. The only option for custom disk images that users have is by taking a "snapshot" of an existing servers root disk. These can then be used to create new servers.
To create a completely custom disk image, users have to follow these steps:
- Create server with the correct server type
- Enable rescue system for the server
- Boot the server
- Download the disk image from within the rescue system
- Write disk image to servers root disk
- Shut down the server
- Take a snapshot of the servers root disk
- Delete the server
This is an annoyingly long process. Many users have automated this with Packer &
packer-plugin-hcloud before, but Packer offers a lot of
additional complexity to wrap your head around.
This repository provides a simple CLI tool & Go library to do the above.
Getting Started
CLI
Binary
We provide pre-built deb, rpm and apk packages. Alternatively we also provide the binaries directly.
Check out the GitHub release artifacts for all of these files and archives.
Arch Linux
You can get hcloud-upload-image-bin from the AUR.
Use your preferred wrapper to install:
yay -S hcloud-upload-image-bin
go install
If you already have a recent Go toolchain installed, you can build & install the binary from source:
go install github.com/apricote/hcloud-upload-image@latest
Docker
There is a docker image published at ghcr.io/apricote/hcloud-upload-image.
docker run --rm -e HCLOUD_TOKEN="<your token>" ghcr.io/apricote/hcloud-upload-image:latest <command>
Usage
export HCLOUD_TOKEN="<your token>"
hcloud-upload-image upload \
--image-url "https://example.com/disk-image-x86.raw.bz2" \
--architecture x86 \
--compression bz2
To learn more, you can use the embedded help output or check out the CLI help pages in this repository.:
hcloud-upload-image --help
hcloud-upload-image upload --help
hcloud-upload-image cleanup --help
Go Library
The functionality to upload images is also exposed in the library hcloudimages! Check out the reference documentation for more details.
Install
go get github.com/apricote/hcloud-upload-image/hcloudimages
Usages
package main
import (
"context"
"fmt"
"net/url"
"github.com/hetznercloud/hcloud-go/v2/hcloud"
"github.com/apricote/hcloud-upload-image/hcloudimages"
)
func main() {
client := hcloudimages.NewClient(
hcloud.NewClient(hcloud.WithToken("<your token>")),
)
imageURL, err := url.Parse("https://example.com/disk-image-x86.raw.bz2")
if err != nil {
panic(err)
}
image, err := client.Upload(context.TODO(), hcloudimages.UploadOptions{
ImageURL: imageURL,
ImageCompression: hcloudimages.CompressionBZ2,
Architecture: hcloud.ArchitectureX86,
})
if err != nil {
panic(err)
}
fmt.Printf("Uploaded Image: %d", image.ID)
}
Contributing
If you have any questions, feedback or ideas, feel free to open an issue or pull request.
License
This project is licensed under the MIT license, unless the file explicitly specifies another license.
Support Disclaimer
This is not an official Hetzner Cloud product in any way and Hetzner Cloud does not provide support for this.