Dev Boots Linux 292,612 Times to Find Kernel Bug (tomshardware.com) 32

Posted by EditorDavid on Saturday June 17, 2023 @04:57PM from the hardcore-boots dept.

Long-time Slashdot reader waspleg shared this story from Hot Hardware: Red Hat Linux developer Richard WM Jones has shared an eyebrow raising tale of Linux bug hunting. Jones noticed that Linux 6.4 has a bug which means it will hang on boot about 1 in 1,000 times. Jones set out to pinpoint the bug, and prove he had caught it red handed. However, his headlining travail, involving booting Linux 292,612 times (and another 1,000 times to confirm the bug) apparently "only took 21 hours." It also seems that the bug is less common with Intel hardware than AMD based machines.

Dev Boots Linux 292,612 Times to Find Kernel Bug

This discussion has been archived. No new comments can be posted.

Load All Comments

Search 32 Comments Log In/Create an Account

Comments Filter:

Dedication (Score:5, Insightful)

by saloomy ( 2817221 ) writes: on Saturday June 17, 2023 @05:04PM (#63611280)

To your craft is awesome. The joy you feel when you find it is also amazing.

- Re:Dedication (Score:5, Funny)
  
  by arglebargle_xiv ( 2212710 ) writes: on Sunday June 18, 2023 @05:03AM (#63612272)
  
  This bug would never have occurred with Windows. The average user reboots Windows 292,612 times each month so it'd have been detected long ago.
  
- Re: (Score:2)
  
  by flyingfsck ( 986395 ) writes:
  
  Damn, I can relate to that. In my younger days, I spent a week rebooting a military fire control computer to find a start-up bug.
- Re: (Score:2)
  
  by NobleNobbler ( 9626406 ) writes:
  
  Thank you for this rare dose of /. positivity!
- Credit (Score:2)
  
  by stooo ( 2202012 ) writes:
  
  A lot of credit go to the people who made the Linux system faster to boot :)
Oblig. (Score:5, Funny)

by SeaFox ( 739806 ) writes: on Saturday June 17, 2023 @05:13PM (#63611292)

You could say he gave that bug the boot.

- Re: (Score:1)
  
  by Motleypuss ( 10291831 ) writes:
  
  Many, many times. Just to make sure. :D
Truly the hero we needed and did not deserve. (Score:3)

by PseudoThink ( 576121 ) writes: on Saturday June 17, 2023 @05:20PM (#63611304)

Nt

Digital isn't always precise and definitive (Score:2)

by innocent_white_lamb ( 151825 ) writes:

Digital stuff isn't always as precise and definitive as one would expect.
One plus one should equal two but due to the complexity of the machine and its interactions with both external and internal forces, sometimes one plus one equals 3, maybe 0, or 65535.
- Re: (Score:2)
  
  by tlhIngan ( 30335 ) writes:
  
  Digital stuff isn't always as precise and definitive as one would expect.
  One plus one should equal two but due to the complexity of the machine and its interactions with both external and internal forces, sometimes one plus one equals 3, maybe 0, or 65535.
  Oh it's precise, but there's enough variations in hardware to have race conditions. We know hardware race conditions as setup/hold violations where violating the timing may do the right thing, but it can also capture the wrong bit. And there are still anal
Found the repro; bug is still unclear (Score:4, Insightful)

by walbourn ( 749165 ) writes: on Saturday June 17, 2023 @05:27PM (#63611318)

After reading the post, it's clear that Richard did a great job getting a rare bug down to a single commit. Hugely helpful. Still, the actual BUG here isn't clear so the fix isn't obvious.

- Re:Found the repro; bug is still unclear (Score:5, Informative)
  
  by Some nick or other ( 4033849 ) writes: on Saturday June 17, 2023 @05:29PM (#63611322)
  
  It appears that the kernel devs have determined what the bug is: https://lore.kernel.org/all/be... [kernel.org]
  
Either side of bisect (Score:2)

by bill_mcgonigle ( 4333 ) * writes:

involving booting *pre-commit* Linux 292,612 times *successfully* (and another 1,000 times *post-commit* to confirm the *hanging* bug)
2023 (Score:4, Funny)

by jovius ( 974690 ) writes: on Saturday June 17, 2023 @05:33PM (#63611330)

The year of the Linux sweatshop

Lucky guy (Score:5, Interesting)

by fahrbot-bot ( 874524 ) writes: on Saturday June 17, 2023 @05:42PM (#63611344)

booting Linux 292,612 times (and another 1,000 times to confirm the bug) apparently "only took 21 hours."
Way back in 1999, I had to test the real time clocks and OS timekeeping for Y2K preparedness on several HP 9000 [ALNT]-Class production systems (three of each) systems running HP-UX 11 which involved resetting the clocks to various dates/times and cold rebooting the systems about 20 times. I wrote an Init script to handle everything except toggling the power switches. The T-600 systems took 20 minutes to cold boot to single-user and a few more to run the script and shutdown, while the other systems were a bit faster but still long enough for it to get old. It was a very long, lonely night in the machine room.
Kudos to Jones.

- Y2K (Score:3)
  
  by nuckfuts ( 690967 ) writes:
  
  This is the reason why Y2K wasn't a widespread disaster as people feared it might be. It's not because the the danger had been overblown (as many people said afterward). It's because people like this spent hours in testing and remediating beforehand.
  I personally tested a Data General mini-computer running DG-UX (Unix) by shutting it down, rolling the date forward in the BIOS, and restarting the machine. Never mind obscure date bugs in the application software - the server was unable to even boot! We wound u
- First Debian release (Score:2)
  
  by hawk ( 1151 ) writes:
  
  The intial stable release of Debian (1.1, iirc) was delayed a couple of days when I tripped over and reported one of these.
  On the Gateway (?) machines in the student lab I was using for a test install, *alternate* boots from floppy would fail. I tested on another couple of machines, and found it consistent.
Regular day. (Score:4, Funny)

by derplord ( 7203610 ) writes: on Saturday June 17, 2023 @05:46PM (#63611350)

So it's essentially just another day trying to get nVidia drivers to work properly.

Linux developers go above and beyond once again. (Score:2)

by VoodooCryptologist ( 7614904 ) writes:

I've always been impressed by how determined and dedicated Linux developers are. It's really inspiring to see them commit so much time and energy into a project like this.
- - Re: (Score:2)
    
    by VoodooCryptologist ( 7614904 ) writes:
    
    Microsoft would have probably done nothing about it because it happens so infrequently. It seems like they don't act unless it's a really serious bug.
- Re: (Score:2)
  
  by ThosLives ( 686517 ) writes:
  
  The sleuthing is impressive - but wouldn't it have been better to simply design the function before writing it the first time? A well defined domain and range for function inputs, along with some basic what-if analysis of fault conditions, probably would have uncovered the potential issue before initial release.
  This is what disciplined engineering gives you - avoiding many problems before they occur. Sure it doesn't get all of them, but it gets most of them. And it's always[1] less expensive to find design
A man with two clocks (Score:4, Interesting)

by dsgrntlxmply ( 610492 ) writes: on Saturday June 17, 2023 @08:55PM (#63611728)

A man with two clocks never knows what time it is. The days that I spent finding that a very large vendor supplied SoC support package was not using the monotonic clock...

It happened to me -- twice (Score:2)

by tonymercmobily ( 658708 ) writes:

This has happened to me twice in the last 2 years. Unexplained hang -- I looked at it and went, why?
This has to be it
Speedrunning the Linux kernel? (Score:2)

by Briareos ( 21163 ) writes:

Seems Linux needs a "100% glitchless" category at speedruns.com...
Impressive (Score:1)

by ipkpjersi ( 10437470 ) writes:

That is honestly seriously impressive dedication. I don't know how someone would have the patience to do that lol
what kind of setup is he using? (Score:1)

by shyeahright ( 9589250 ) writes:

a simple calculator told he thats more than 3 boots a second for 21 straight hours. yikes!
Well (Score:2)

by DarkRookie2 ( 5551422 ) writes:

That blog post was uneventful and boring to read.

There may be more comments in this discussion. Without JavaScript enabled, you might want to turn on Classic Discussion System in your preferences instead.

Dev Boots Linux 292,612 Times to Find Kernel Bug (tomshardware.com) 32

Dev Boots Linux 292,612 Times to Find Kernel Bug More Login

Dev Boots Linux 292,612 Times to Find Kernel Bug

Dedication (Score:5, Insightful)

Re:Dedication (Score:5, Funny)

Re: (Score:2)

Re: (Score:2)

Credit (Score:2)

Oblig. (Score:5, Funny)

Re: (Score:1)

Truly the hero we needed and did not deserve. (Score:3)

Digital isn't always precise and definitive (Score:2)

Re: (Score:2)

Found the repro; bug is still unclear (Score:4, Insightful)

Re:Found the repro; bug is still unclear (Score:5, Informative)

Either side of bisect (Score:2)

2023 (Score:4, Funny)

Lucky guy (Score:5, Interesting)

Y2K (Score:3)

First Debian release (Score:2)

Regular day. (Score:4, Funny)

Linux developers go above and beyond once again. (Score:2)

Re: (Score:2)

Re: (Score:2)

A man with two clocks (Score:4, Interesting)

It happened to me -- twice (Score:2)

Speedrunning the Linux kernel? (Score:2)

Impressive (Score:1)

what kind of setup is he using? (Score:1)

Well (Score:2)

Related Links Top of the: day, week, month.

Slashdot Top Deals

Slashdot