Wednesday, November 2, 2011

A Kernel for All Seasons

I have received quite a bit of feedback recently on the pursuit of a -lowlatency kernel for Ubuntu Studio and it appears that my efforts to concisely explain and document the requirements for such a kernel and available methodology of achieving it are deficient.  I will attempt to correct this state.

I will start with the reasons Ubuntu Studio desires a different kernel and the paths that lead to the current state.

Into My Own Thing
I would posit that most Ubuntu Studio users are mostly concerned with audio recording and as such they desire a stable, low latency.  Let's break "stable, low latency" down for easy consumption.
  • Latency can be simply defined as the time between when a sound is generated and it is heard
  • Low latency is desired because higher latencies (i.e. delay) can be confusing and throw off any sense of rhythm if the sound is heard later than you expect it
  • Lastly, we want a stable latency because we want to avoid xruns (underruns or overruns in the buffer) which can introduce pops, clicks, digital distortion, and other unwanted noise into your music
Trying to record music in time with other music is unachievable if the latencies are too high.  An unstable latencies could introduce an unwanted and unpleasant artifact as a result of an xrun into that solo you spent weeks trying to get just right!

Additionally, numerous laptop users using Firewire audio interfaces have suffered from irq conflicts.

Explained more accessibly, the Firewire interfaces might share a bus with and compete against other devices for attention from the motherboard.  These Interrupt Requests (irq's) are prioritized and sometimes other items are more highly prioritized than the audio interface.  Not a completely correct analogy, but musicians would prefer that the audio was properly recorded over the mouse updating its position on the monitor.

Historically only the -rt (or -realtime) kernel would provide this functionality via Ingo's patch.  However, code from the -rt patch has been integrated into the main kernel tree and this particular functionality is now available in the => 2.6.39 -generic kernel.

You Can Make It If You Try
Sometimes you can define your path by defining where you can't go, so let's explore our constraints to see where our true direction lies. 

The largest constraint is that the process for building the Ubuntu Studio image is an automated process (or daemon called buildd, I believe) which can only use packages that are inside the official Ubuntu repository (also called archive). Again, using exclusion; we cannot use packages from PPA's (Personal Package Archives), or the kernel that guy made over there at another multimedia distro, or even a kernel from Debian. And even if Debian has a -rt kernel, Ubuntu will NOT sync that kernel into the repositories as Ubuntu rolls their own.

Therefore, our options are to use a kernel package from either the Main or Universe repository to build our image.

Next, the Ubuntu Kernel Team is NOT going to build and maintain an additional kernel.  I don't blame them, this is a lot of additional work and responsibility, especially for a rather small niche of highly expectant users, and the kernel team does enough already.  And since this kernel will be maintained by non-core-dev individuals, this package falls under the purview of MOTU and will reside in the Universe repository.

Therefore, this new kernel will be a community (i.e. Ubuntu Studio developers) supported kernel... in an Inception-like abstract, a community within a community.

Our path, in this context, is that the Ubuntu Studio developers are to build a kernel that will be archived in the Universe repository.

Dance to the Music
So the last movement of this piece is defining which kernel to built.

The -rt time kernel has some significant aspects, both good and bad.

GOOD - it can provide exceedingly good low latencies

BAD - it requires an invasive patch that isn't always available to align with Ubuntu's chosen kernel version, requires compiling some video drivers again

NEITHER - irq threading to prevent irq conflicts isn't an issue anymore as the -generic kernel now provides this functionality.

In my opinion, the -lowlatency kernel has exceedingly more positives than negatives.

GOOD - can be based on the Ubuntu kernel which keeps versions aligned, only requires compile time flag changes to build, doesn't require additional video driver building, provides good latencies

BAD - doesn't provide as good latencies as -rt

While we found that the -lowlatency kernel generally didn't perform quite as well as the -rt kernel we did establish that most testers found that latencies were more than acceptable.  Some hardware sets will be the exception, but we believe we will be able to adequately support the majority of users with this kernel.

Therefore, it seems the most practical decision is to move towards the -lowlatency for its performance, availability, and longevity

And this leaves us with Professor Plum with the candlestick in the library.  Kidding.

Hopefully, this post explains Ubuntu Studio's kernel considerations, the possible and tenable vectors for acquiring a kernel, and the decisions for choosing the -lowlatency kernel.


lsd said...

I think that's a good summary of the issues. If you need some good examples of times when the latency is directly observable and completely unavoidable (since I've sometimes seen non-audio people ask why low-latency is really required):

* using MIDI softsynths -- here, the latency translates in to a delay between when you hit a key on your MIDI keyboard and when the note sounds
* using real-time effects, such as when playing a guitar through an effects app like Rakarrack -- here, the latency translates in to a delay between when you play something and when the processed audio plays back

FWIW, I've been using the abogani 3.0.0 lowlatency kernel on my system, and it's been fantastic -- the latency with the -generic kernel on my Firewire interface was unacceptable, but with the -lowlatency kernel and the rtirq script, JACK is very solid at 8ms latency, and it's pretty solid at 4ms, too.

Anonymous said...

Maybe you should consider moving to different build infrastructure if it is so inflexible as to not support secondary repositories such as Debian or PPAs?

Scott Lavender said...

@lsd, thank you for your comments and i'll make sure alessio knows about your comments as well

@anonymous, thank you for the comment.

canonical handles many tasks and responsibilities for ubuntu studio, and the build is only one facet.

canonical provides an entire infrastructure to test and maintain the building process alone, never mind the additional infrastructure required to test the images, post the images, host the images, and bandwidth for downloads.

additionally there is also the human infrastructure that supports the all these processes...most are paid canonical employees.

i think the most practical path is still pursuing the -lowlatency kernel in the repository.

Unknown said...

I agree that a -lowlatency kernel is probably the best choice. If combined with preventing all low-priority background tasks from running at the wrong time, it should perform well enough for most. Even hard-core performance oriented audio distributions like AVLINUX tend to go with -lowlatency over -rt as a default these days, mostly to get the most up-to-date firewire drivers. Most non-audio desktop / laptop users would prefer a -lowlatency kernel over the current Ubuntu default, they just don't know it. I used Ubuntu Studio 8.04 and liked it, but subsequent releases drove me away with completely unacceptable latency on my hardware.