latest kernel broke my system

[rob]

My hardware is about 9/10m years old now , and I have no wish to update
it. I have been using linux for about 6/7 yeas with very few issues.

Yesterday I let mint upgrade my system and there was kernel upgrade
included. I had tom reboot afterwards, but my system wouldn't load back
to the desktop. Instead there was some message saying that my system is
in Emergency mode. In among the text there was something about
journalctl etc. FSCK wouldn't run, not recognised, as was the same with
gparted.
All I could do was press enter which took me to a login option? or I
could press ctl D or something.

Anyway as nothing I did or knew worked, I booted up a live LM21.2 USB
stick and managed to copy my data to another USB stick. Then I did a
reinstall of LM, and once I had a working system, I restored 'Personal
data. Then copied my files from the USB stick to the relevant locations.

For the backup and retore, I am assuming it was deja vu backup tool,
is there a better one.

[Big Al]

My hardware is about 9/10m years old now , and I have no wish to update it. I have been using linux
for about 6/7 yeas with very few issues.
Yesterday I let mint upgrade my system  and there was kernel upgrade included. I had tom reboot
afterwards, but my system wouldn't load back to the desktop. Instead there was some message saying
that my system is in Emergency mode. In among the text there was something about journalctl etc.
FSCK wouldn't run, not recognised, as was the same with gparted.
All I could do was press enter which took me to a login option? or I could press ctl D or something.
Anyway as nothing I did or knew worked, I booted up a live LM21.2 USB stick and managed to copy my
data to another USB stick. Then I did a reinstall of LM, and once I had a working system, I restored
'Personal data. Then copied my files from the USB stick to the relevant locations.
For the backup and retore, I am assuming it was deja vu backup tool,
is there a better one.

You don't say what kernel # you upgraded to but I'm on 5.15.0-105.
Not crashing my system but the past few kernels -100 thru -105 have had a serious effect on my system.
Going to sleep and restore leaves my keyboard dead in the water.
Shutdown displayed many warning/error messages.

The latter has been fixed with the -105 kernel but the keyboard still has to be fixed by modifying
the kernel load commands in grub. Even that much has cost me the caps lock light. Can't seem to
get anyone to acknowledge it in the kernel.org site.

So I feel your pain.

[RobH]

Post by Big Al

Post by rob
My hardware is about 9/10m years old now , and I have no wish to
update it. I have been using linux for about 6/7 yeas with very few
issues.
Yesterday I let mint upgrade my system  and there was kernel upgrade
included. I had tom reboot afterwards, but my system wouldn't load
back to the desktop. Instead there was some message saying that my
system is in Emergency mode. In among the text there was something
about journalctl etc. FSCK wouldn't run, not recognised, as was the
same with gparted.
All I could do was press enter which took me to a login option? or I
could press ctl D or something.
Anyway as nothing I did or knew worked, I booted up a live LM21.2 USB
stick and managed to copy my data to another USB stick. Then I did a
reinstall of LM, and once I had a working system, I restored 'Personal
data. Then copied my files from the USB stick to the relevant locations.
For the backup and retore, I am assuming it was deja vu backup tool,
is there a better one.

You don't say what kernel # you upgraded to but I'm on 5.15.0-105.
Not crashing my system but the past few kernels -100 thru -105 have had
a serious effect on my system.
Going to sleep and restore leaves my keyboard dead in the water.
Shutdown displayed many warning/error messages.
The latter has been fixed with the -105 kernel but the keyboard still
has to be fixed by modifying the kernel load commands in grub.   Even
that much has cost me the caps lock light.   Can't seem to get anyone to
acknowledge it in the kernel.org site.
So I feel your pain.

I think it was 5.15.0-105, but on reboot I did try other /older kernels
either in normal boot or the other boot mode?
Neither made any difference.
Currently I'm using the 5.15.0-76-generic kernel

I also had problems some earlier kernels last year with both my keyboard
and mouse not working.

[Jeff Layman]

Post by rob
My hardware is about 9/10m years old now , and I have no wish to update
it. I have been using linux for about 6/7 yeas with very few issues.
Yesterday I let mint upgrade my system and there was kernel upgrade
included. I had tom reboot afterwards, but my system wouldn't load back
to the desktop. Instead there was some message saying that my system is
in Emergency mode. In among the text there was something about
journalctl etc. FSCK wouldn't run, not recognised, as was the same with
gparted.
All I could do was press enter which took me to a login option? or I
could press ctl D or something.
Anyway as nothing I did or knew worked, I booted up a live LM21.2 USB
stick and managed to copy my data to another USB stick. Then I did a
reinstall of LM, and once I had a working system, I restored 'Personal
data. Then copied my files from the USB stick to the relevant locations.
For the backup and retore, I am assuming it was deja vu backup tool,
is there a better one.

You can boot to a previous kernel by holding down the right shift key
after switching on. See
<https://forums.linuxmint.com/viewtopic.php?p=2189554#p2189554>. You
might find the whole thread of interest.

I use "Backups" (Deja Dup) for my Home files. When they changed the GUI
a couple of years ago I thought it less intuitive than the old one, but
it still works.

[Nic]

Post by Jeff Layman

Post by rob
My hardware is about 9/10m years old now , and I have no wish to update
it. I have been using linux for about 6/7 yeas with very few issues.
Yesterday I let mint upgrade my system  and there was kernel upgrade
included. I had tom reboot afterwards, but my system wouldn't load back
to the desktop. Instead there was some message saying that my system is
in Emergency mode. In among the text there was something about
journalctl etc. FSCK wouldn't run, not recognised, as was the same with
gparted.
All I could do was press enter which took me to a login option? or I
could press ctl D or something.
Anyway as nothing I did or knew worked, I booted up a live LM21.2 USB
stick and managed to copy my data to another USB stick. Then I did a
reinstall of LM, and once I had a working system, I restored 'Personal
data. Then copied my files from the USB stick to the relevant locations.
For the backup and retore, I am assuming it was deja vu backup tool,
is there a better one.

You can boot to a previous kernel by holding down the right shift key
after switching on. See
<https://forums.linuxmint.com/viewtopic.php?p=2189554#p2189554>. You
might find the whole thread of interest.
I use "Backups" (Deja Dup) for my Home files. When they changed the
GUI a couple of years ago I thought it less intuitive than the old
one, but it still works.

Will the kernel choice be persistent or on next boot it reverts to the
broken kernel?

[Jeff Layman]

Post by Nic

Post by Jeff Layman

Post by rob
My hardware is about 9/10m years old now , and I have no wish to update
it. I have been using linux for about 6/7 yeas with very few issues.
Yesterday I let mint upgrade my system  and there was kernel upgrade
included. I had tom reboot afterwards, but my system wouldn't load back
to the desktop. Instead there was some message saying that my system is
in Emergency mode. In among the text there was something about
journalctl etc. FSCK wouldn't run, not recognised, as was the same with
gparted.
All I could do was press enter which took me to a login option? or I
could press ctl D or something.
Anyway as nothing I did or knew worked, I booted up a live LM21.2 USB
stick and managed to copy my data to another USB stick. Then I did a
reinstall of LM, and once I had a working system, I restored 'Personal
data. Then copied my files from the USB stick to the relevant locations.
For the backup and retore, I am assuming it was deja vu backup tool,
is there a better one.

You can boot to a previous kernel by holding down the right shift key
after switching on. See
<https://forums.linuxmint.com/viewtopic.php?p=2189554#p2189554>. You
might find the whole thread of interest.
I use "Backups" (Deja Dup) for my Home files. When they changed the
GUI a couple of years ago I thought it less intuitive than the old
one, but it still works.

Will the kernel choice be persistent or on next boot it reverts to the
broken kernel?

I don't think so. Once you've successfully booted, you have to open the
Update Manager and select View | Linux kernels. Note the warning which
appears, then click on "Continue", highlight the kernel you don't want,
and click on "remove kernels".

[Nic]

Post by Jeff Layman

Post by Nic

Post by Jeff Layman

Post by rob
My hardware is about 9/10m years old now , and I have no wish to update
it. I have been using linux for about 6/7 yeas with very few issues.
Yesterday I let mint upgrade my system  and there was kernel upgrade
included. I had tom reboot afterwards, but my system wouldn't load back
to the desktop. Instead there was some message saying that my system is
in Emergency mode. In among the text there was something about
journalctl etc. FSCK wouldn't run, not recognised, as was the same with
gparted.
All I could do was press enter which took me to a login option? or I
could press ctl D or something.
Anyway as nothing I did or knew worked, I booted up a live LM21.2 USB
stick and managed to copy my data to another USB stick. Then I did a
reinstall of LM, and once I had a working system, I restored 'Personal
data. Then copied my files from the USB stick to the relevant locations.
For the backup and retore, I am assuming it was deja vu backup tool,
is there a better one.

You can boot to a previous kernel by holding down the right shift key
after switching on. See
<https://forums.linuxmint.com/viewtopic.php?p=2189554#p2189554>. You
might find the whole thread of interest.
I use "Backups" (Deja Dup) for my Home files. When they changed the
GUI a couple of years ago I thought it less intuitive than the old
one, but it still works.

Will the kernel choice be persistent or on next boot it reverts to the
broken kernel?

I don't think so. Once you've successfully booted, you have to open
the Update Manager and select View | Linux kernels. Note the warning
which appears, then click on "Continue",  highlight the kernel you
don't want, and click on "remove kernels".

Thank you, that is good information, remove the bad kernel.

[RobH]

Post by Jeff Layman

Post by rob
My hardware is about 9/10m years old now , and I have no wish to update
it. I have been using linux for about 6/7 yeas with very few issues.
Yesterday I let mint upgrade my system  and there was kernel upgrade
included. I had tom reboot afterwards, but my system wouldn't load back
to the desktop. Instead there was some message saying that my system is
in Emergency mode. In among the text there was something about
journalctl etc. FSCK wouldn't run, not recognised, as was the same with
gparted.
All I could do was press enter which took me to a login option? or I
could press ctl D or something.
Anyway as nothing I did or knew worked, I booted up a live LM21.2 USB
stick and managed to copy my data to another USB stick. Then I did a
reinstall of LM, and once I had a working system, I restored 'Personal
data. Then copied my files from the USB stick to the relevant locations.
For the backup and retore, I am assuming it was deja vu backup tool,
is there a better one.

You can boot to a previous kernel by holding down the right shift key
after switching on. See
<https://forums.linuxmint.com/viewtopic.php?p=2189554#p2189554>. You
might find the whole thread of interest.
I use "Backups" (Deja Dup) for my Home files. When they changed the GUI
a couple of years ago I thought it less intuitive than the old one, but
it still works.

I did try alternative older kernels but it made no difference . Still no
boot up , with the same text at the top of the screen.

[stepore]

Post by RobH
I did try alternative older kernels but it made no difference . Still no
boot up , with the same text at the top of the screen.

If you tried older kernel, then obviously it wasn't the kernel that
_broke_ your system; otherwise the old kernel would fix your issue. It
was probably video drivers.

[RobH]

Post by stepore

Post by RobH
I did try alternative older kernels but it made no difference . Still no
boot up , with the same text at the top of the screen.

If you tried older kernel, then obviously it wasn't the kernel that
_broke_ your system; otherwise the old kernel would fix your issue. It
was probably video drivers.

Yes it well could have been, but how and why after my system was running
problem free for over a year

[Nic]

Post by RobH

Post by stepore

Post by RobH
I did try alternative older kernels but it made no difference . Still no
boot up , with the same text at the top of the screen.

If you tried older kernel, then obviously it wasn't the kernel that
_broke_ your system; otherwise the old kernel would fix your issue.
It was probably video drivers.

Yes it well could have been, but how and why after my system was
running problem free for over a year

can you retrieve the list of the most recent updates, there may be a
clue to this mystery, and how to avoid a particular update.

[Nic]

Post by rob
My hardware is about 9/10m years old now , and I have no wish to
update it. I have been using linux for about 6/7 yeas with very few
issues.
Yesterday I let mint upgrade my system  and there was kernel upgrade
included. I had tom reboot afterwards, but my system wouldn't load
back to the desktop. Instead there was some message saying that my
system is in Emergency mode. In among the text there was something
about journalctl etc. FSCK wouldn't run, not recognised, as was the
same with gparted.
All I could do was press enter which took me to a login option? or I
could press ctl D or something.
Anyway as nothing I did or knew worked, I booted up a live LM21.2 USB
stick and managed to copy my data to another USB stick. Then I did a
reinstall of LM, and once I had a working system, I restored 'Personal
data. Then copied my files from the USB stick to the relevant locations.
For the backup and retore, I am assuming it was deja vu backup tool,
is there a better one.

Were you able to run GParted from a live distro, and then run FSCK from
within GParted on the broken drive?

[RobH]

Post by Nic

Post by rob
My hardware is about 9/10m years old now , and I have no wish to
update it. I have been using linux for about 6/7 yeas with very few
issues.
Yesterday I let mint upgrade my system  and there was kernel upgrade
included. I had tom reboot afterwards, but my system wouldn't load
back to the desktop. Instead there was some message saying that my
system is in Emergency mode. In among the text there was something
about journalctl etc. FSCK wouldn't run, not recognised, as was the
same with gparted.
All I could do was press enter which took me to a login option? or I
could press ctl D or something.
Anyway as nothing I did or knew worked, I booted up a live LM21.2 USB
stick and managed to copy my data to another USB stick. Then I did a
reinstall of LM, and once I had a working system, I restored 'Personal
data. Then copied my files from the USB stick to the relevant locations.
For the backup and retore, I am assuming it was deja vu backup tool,
is there a better one.

Were you able to run GParted from a live distro, and then run FSCK from
within GParted on the broken drive?

No,to be honest I just wanted to get my data files from the broken
system so I could restore them on the new install.

Something to think about next time it happens.

[Paul]

My hardware is about 9/10m years old now , and I have no wish to update it. I have been using linux for about 6/7 yeas with very few issues.
Yesterday I let mint upgrade my system  and there was kernel upgrade included. I had tom reboot afterwards, but my system wouldn't load back to the desktop. Instead there was some message saying that my system is in Emergency mode. In among the text there was something about journalctl etc. FSCK wouldn't run, not recognised, as was the same with gparted.
All I could do was press enter which took me to a login option? or I could press ctl D or something.
Anyway as nothing I did or knew worked, I booted up a live LM21.2 USB stick and managed to copy my data to another USB stick. Then I did a reinstall of LM, and once I had a working system, I restored 'Personal data. Then copied my files from the USB stick to the relevant locations.
For the backup and retore, I am assuming it was deja vu backup tool,
is there a better one.

We know your /etc/fstab looks different than other audience members.

https://serverfault.com/questions/1052945/whats-the-most-failsafe-way-to-mount-remote-filesystems-at-boot

You probably put your nfs/smb/cifs mount at the very beginning
of the /etc/fstab file, and if the system gets hung up trying
to execute the first line, it might not get to the other lines
in /etc/fstab and bring up your slash, your slash home, your swap
partition or /swapfile .

By adding a "nofail" or a timeout to the mount, then that stops
the boot from "depending" on the NAS.

A test case for whether your NAS mount is done properly:

1) Shut down Linux Mint.
2) Unplug network cable from NAS. You can leave
the NAS running if you want.
3) Try to boot Linux Mint.
4( If it does not finish boot, plug in the NAS network
cable and try again.

If you set up your mount line properly, the NAS being absent from
the LAN, should not stop Linux Mint from booting.

Paul

[RobH]

Post by Paul

My hardware is about 9/10m years old now , and I have no wish to update it. I have been using linux for about 6/7 yeas with very few issues.
Yesterday I let mint upgrade my system  and there was kernel upgrade included. I had tom reboot afterwards, but my system wouldn't load back to the desktop. Instead there was some message saying that my system is in Emergency mode. In among the text there was something about journalctl etc. FSCK wouldn't run, not recognised, as was the same with gparted.
All I could do was press enter which took me to a login option? or I could press ctl D or something.
Anyway as nothing I did or knew worked, I booted up a live LM21.2 USB stick and managed to copy my data to another USB stick. Then I did a reinstall of LM, and once I had a working system, I restored 'Personal data. Then copied my files from the USB stick to the relevant locations.
For the backup and retore, I am assuming it was deja vu backup tool,
is there a better one.

We know your /etc/fstab looks different than other audience members.
https://serverfault.com/questions/1052945/whats-the-most-failsafe-way-to-mount-remote-filesystems-at-boot
You probably put your nfs/smb/cifs mount at the very beginning
of the /etc/fstab file, and if the system gets hung up trying
to execute the first line, it might not get to the other lines
in /etc/fstab and bring up your slash, your slash home, your swap
partition or /swapfile .
By adding a "nofail" or a timeout to the mount, then that stops
the boot from "depending" on the NAS.
1) Shut down Linux Mint.
2) Unplug network cable from NAS. You can leave
the NAS running if you want.
3) Try to boot Linux Mint.
4( If it does not finish boot, plug in the NAS network
cable and try again.
If you set up your mount line properly, the NAS being absent from
the LAN, should not stop Linux Mint from booting.
Paul

Actually it was not the cifs share, because that was or is on another
computer.
Because no other kernel would give a successful boot, I'm now thinking
it may have been a video driver??

Thanks anyway

[Paul]

Actually it was not the cifs share, because that was or is on another computer.
Because no other kernel would give a successful boot, I'm now thinking it may have been a video driver??
Thanks anyway

Could be. I've had my share of brickage.

For a period of 18 months, the kernel had a bug in paravirtualization,
and it mixed up Windows Virtual PC and Hyper-V and would try to use the
wrong drivers. It meant that VMs I was trying to run, would not work
until that was fixed.

Another reason, might be a DKMS step has gone "out of range". They
have release numbers (of validity) and once the software gets too old,
something has to be done to freshen DKMS stuff.

You name it, there is a way to break it.

It keeps you on your toes, can't relax exactly.

And yes, X11 occasionally has cards removed from support, because
"this card would be too crusty to rewrite the driver". Every day is a
cloudy day, in the video card canning department :-)

Paul