Online Fedora Upgrade

I have installed Fedora on my computer 3 or 4 years ago. Since this installation, I have installed many packages and upgraded it every 6 months. Unfortunately, Gnome Software has never worked for me, and I never took the time to debug it. Like many people, I am using the old method with dnf-system-upgrade, which worked pretty well. But this time, I had a segfault on dnf during the offline upgrade. The only information I got was the following lines:

kernel: dnf[846]: segfault at 8 ip 00007f39b860f724 sp 00007ffd00588520 error 4 in libc-2.25.so[7f39b8586000+1cb000]
systemd[1]: dnf-system-upgrade.service: Main process exited, code=dumped, status=11/SEGV
systemd[1]: Failed to start System Upgrade using DNF.

Just after the segfault, my computer rebooted on Fedora 26.

How dnf-system-upgrade works

Some information presented here could be not totally accurate. If you have a doubt, please follow the links.

dnf-system-upgrade uses a feature of systemd described here: Implementing Offline System Updates. By creating a symlink named /system-update, at the next reboot systemd will boot to a specific target named system-update.target. That’s what we name “an offline upgrade” contrary to an upgrade run in the default.target which I call “an online upgrade”.

But when will this symlink created ? When you run the following command:

dnf system-upgrade reboot

We can see in the source of the plugin that a symlink pointing to a dnf folder is created:

DEFAULT_DATADIR = '/var/lib/dnf/system-upgrade'
MAGIC_SYMLINK = '/system-update'

# ...
os.symlink(DEFAULT_DATADIR, MAGIC_SYMLINK)
# ...
reboot()

We can now investigate which services are triggered by this target:

$ ls /usr/lib/systemd/system/system-update.target.wants/
dnf-system-upgrade.service  fwupd-offline-update.service  packagekit-offline-update.service

We are specifically interested by dnf-system-upgrade.service which basically run the following command:

dnf --releasever=27 system-upgrade upgrade

At the end of this script, the symlink /system-update is destroyed and the computer rebooted.

Upgrading online to bypass the bug

I didn’t have any idea to debug this segfault in the offline mode. So I searched a way to run this command on an online system. If the command segfaults, I’ll have some tools to investigate it. If the command works, I’ll have an upgraded system. In my case, once run online, everything went well.

Before typing any command, you must know that this tool is not intended to be run this way. You may break your Fedora installation or kill your family. You really should run all your commands in a virtual terminal as it will not kill dnf if Gnome crashes during the install.

First, you must have downloaded the update:

dnf upgrade --refresh
dnf system-upgrade download --releasever=27

For more information, you can refer to the article Upgrading Fedora 26 to Fedora 27

Then, you will need to update dnf-system-upgrade configuration stored in /var/lib/dnf/system-upgrade.json and set “upgrade_status” to “ready”:

{
  "download_status": null,
  "upgrade_status": "ready", <--- this line
  "exclude": [],
  "allow_erasing": true,
  "datadir": "/var/lib/dnf/system-upgrade",
  "distro_sync": true,
  "best": false,
  "target_releasever": "27",
  "system_releasever": "26",
  "releasever": "23",
  "install_packages": {},
  "enable_disable_repos": []
}

You must create the /system-update symlink, otherwise the upgrade command will fail:

ln -s /var/lib/dnf/system-upgrade /system-update

And finally, you can run the upgrade command:

dnf --releasever=27 system-upgrade upgrade

Your computer will automatically reboot at the end. Now you’ll have either a working Fedora 27 or a broken Linux distribution. Good luck and have fun !