We're pleased to announce the release of RSAS 1.0.5. Rocket Streaming Audio Server (RSAS) is a high-performance webserver for distributing live streaming audio through the web, optimized for low latency and high numbers of simultaneous listeners.
This new release is a recommended update for all users.
We've redesigned the Looped Audio Files feature
(aka fallback files) to solve some long-standing issues, allowing them to now be much more useful as fallbacks. Fallback files
no longer rely on client-side buffering to throttle playback, which improves compatibility with players and prevents
delay from accumulating. Looped Audio File sources now start automatically too, which makes them more usable
as fallbacks in practice.
When connected to a Looped Audio File mount, listeners will now all hear the same audio from a common playback position.
Previously, each listener would hear the audio starting from the beginning of the file. We acknowledge this change in
behaviour may not be ideal for all broadcasters, but in order to implement the other usability improvements mentioned above,
we had to make a tradeoff, and this was a result of that.
HLS Improvements
HLS playlists now include the EXT-X-PROGRAM-DATE-TIME, which helps players do timeshifting over long periods of itme.
Dependencies and Build Overhaul
We've performed a major upgrade of our build system as part of ongoing maintenance necessary to keep RSAS production-ready, secure,
and supported on as many platforms as possible. RSAS has been upgraded to use OpenSSL 3.1 and is now statically linked
against it, to allow us to continue supporting older Linux distributions for longer. These build system changes also recently
allowed us to respond quickly to security concerns over liblzma and audit our use of it.
On March 30th, 2024, the discovery of an backdoor in XZ Utils / liblzma specifically targeting the SSH process was announced and assigned CVE-2024-3094.
RSAS is unaffected by the scope of the security issue known to date. We investigated this vulnerability because some
versions of RSAS depended on liblzma, but not on any version known to be backdoored.
We will update this post and notify our mailing list if the status changes.
We're pleased to announce the release of RSAS 1.0.4. Rocket Streaming Audio Server (RSAS) is a high-performance webserver for distributing live streaming audio through the web, optimized for low latency and high numbers of simultaneous listeners.
This release is primarily a bugfix release, but there are a few minor new features as well. This version is a recommended update for all HLS users.
RSAS now detects the codec used by each stream and includes this information in the /health API. Having
this information available at a glance is useful for troubleshooting and validation.
HLS Customization
You can now customize the HLS segment size and number of segments in a playlist, which allows broadcasters to make
their streams rewindable further into the past. For more information, see HLS Settings.
Custom HTTP Headers
Custom HTTP headers can now be added to all responses, or scoped by port, VHost, or mount. This feature makes RSAS
more flexible and eliminates the need to put a reverse proxy in front of it in many enterprise environments.
We are still finalizing our roadmap for the next major release and are not yet ready to announce details on features.
What we can share is that we will be overhauling our build system and are hoping to see some performance improvements and support for
new platforms (Raspberry Pi). We will also be upgrading to OpenSSL 3, as OpenSSL 1.1.1 is EOL.
In order to continue supporting RSAS on older platforms like CentOS 7, which do not have OpenSSL 3, we will be switching
to statically linking all dependencies, including OpenSSL. As a consequence, it will be even more important for you to stay appraised
of RSAS releases in case there's another Heartbleed-style situation, so please make sure to join our low-traffic rsas-announce mailing list if you
are running RSAS on the public internet.
We're pleased to announce the release of RSAS 1.0.3 and Rocket Broadcaster 1.3.37! Rocket Streaming Audio Server (RSAS) is a high-performance webserver for distributing live streaming audio through the web, with low latency and high listener capacity. Rocket Broadcaster is a streaming audio encoder for capturing live audio and sending it to a high capacity server, such as RSAS, for distribution to listeners.
RSAS 1.0.3
RSAS 1.0.3 is primarily a bugfix release that includes a fix for HLS on Windows and a rare issue that could cause source connections
to get stuck on Linux.
After more than a year of development, we're pleased to announce RSAS 1.0 is now available! First released 3 years ago, Rocket Streaming Audio Server (RSAS) is a high-performance webserver designed for delivering live streaming audio through the web, with low latency and high listener capacity.
This major milestone has something for all our users and delivers on our vision of creating an Icecast-compatible streaming server that brings live audio broadcasting into the 21st century. RSAS 1.0 offers exciting new features and higher performance than ever before. Each new feature has been designed with usability in mind, while continuing to offer the unmatched high performance we strive for.
Today, RSAS powers live streaming audio for thousands of radio stations, major sports teams, governments, national public broadcasters, emergency services, telecom operators, and our Radio Mast CDN.
We owe our amazing user community a big thank you for your feedback, feature suggestions, and bug reports that have helped
shape RSAS. Special thanks to our partners at G&L Geißendörfer & Leschinsky and Andy Steven at North Broadcast for going the extra mile with bug reporting and testing.
What's new in RSAS 1.0?
HLS Audio
HLS is a modern streaming protocol designed for better reliability and lower battery usage on mobile
devices. Until now, HLS for live streaming audio faced limited adoption by radio broadcasters due to a lack of support
from encoders and streaming servers, as it remains unsupported by Icecast and SHOUTcast. HLS audio stacks in the wild
often consist of ad-hoc shell scripts, obscure FFMPEG commands, and various webservers glued together which lack the robustness and observability expected by broadcast engineers.
We are excited to announce that RSAS now supports HLS for live audio, and provides
broadcasters with an easy migration path to adopt HLS for more reliable streaming to mobile devices.
RSAS transmuxes your existing MP3 and AAC streams into HLS (without re-encoding), so no changes to your existing encoder setup are needed and effectively making HLS audio encoders obsolete. Additionally, HLS listeners are included in the listener statistics provided by /health and HLS listener sessions are reported in the access.log just like traditional HTTP streaming listeners, for compatibility with your existing listener statistics processing setup. This design allows broadcasters to easily adopt HLS without changes to your encoder or listener statistics setup. RSAS provides a tightly integrated HLS live audio solution that provides the benefits of HLS without the costs of rearchitecting your live streaming.
Some of the HLS features included in this release are:
Converting existing MP3 and HLS streams to HLS - just enable HLS and then add /hls.m3u8 to the end of your stream URL
ID3 metadata for HLS
Integration with /health listener counts and access.log listener session logging.
Relaying HLS streams from other RSAS instances (with intelligent caching)
RSAS now has the ability to play a preroll audio clip to for listeners when they connect, before playing your live audio. For convenience, your preroll can be specified as a URL to a remote audio file, which is downloaded and cached by RSAS. This powerful feature allows your preroll to be updated dynamically and saves you the hassle of distributing preroll audio files to multiple servers.
Midroll support allows you to insert audio clips (ads, jingles, etc.) in the middle of your stream. Ad breaks are triggerable via metadata from your encoder or via the Manage API. Midroll support is programmable and requires integration with your own midroll webhook handler, where your handler returns a list of audio files to insert into a stream.
Ad insertion is not yet supported with HLS, but is on the roadmap for a future release.
HTTP/1.1 and File Serving Performance Optimizations
RSAS is now an extremely fast webserver for serving static files too.
Since HLS involves serving audio in file-based segments, in order to optimize HLS performance on both the server and client side, we implemented HTTP/1.1 and revamped our static file serving code. RSAS now supports HTTP/1.1 "Keep-Alive" and range requests for seeking in static files such as podcasts or video files. (RSAS can serve MP4 video files with seeking out-of-the-box now!)
Static file serving is now fully asynchronous and is blazing fast.
How fast? On Windows, RSAS beats both nginx and Apache in static file requests per second. On Linux, RSAS beats Apache and ties with nginx in single threaded performance.
Single-threaded benchmark serving an 88 KB file with Apache2 2.4.54, nginx 1.22.1, and RSAS 1.0.0.
Linux tests performed on Ubuntu 22.04 on a Ryzen 2700X. Windows tests ran on Windows 11 and a Ryzen 5700G.
Overhauled Linux Packages
On Debian, Ubuntu, and CentOS, our RSAS packages have been upgraded to install RSAS as a systemd service and run as a separate rsas user. Our Linux static binary and FreeBSD tarballs also now include an installer script.
By popular demand, the /health endpoint can now be password protected to prevent enumeration of your streams. We've also introduced mount-specific /<mount>/health endpoints, which can also be password protected.
More Icecast APIs
For better integration with third party Icecast statistics platforms, RSAS now implements several missing Icecast APIs, including:
We've also introduced a new low-traffic mailing list, rsas-announce, for release announcements and security advisories. If you're running RSAS in production, we recommend joining.
We expect a number of small bugfix point releases to roll out over the coming weeks as we receive feedback on the 1.0 release.
The next major features under consideration are ad insertion support for HLS (preroll, midroll, postroll), support for kernel TLS to improve HTTPS performance on modern kernels, and support for other streaming protocols. We'd
also like to have RPMs for Alma Linux and Rocky Linux (which may still work with our existing CentOS 8 packages).
📧 Feedback and Feature Suggestions
RSAS 1.0 is the culmination of years of feedback from broadcasters. Your feedback is essential in steering the future of live streaming audio. If you have feedback or feature suggestions, we'd love to hear from you.
Three years ago, we benchmarked RSAS, Icecast-KH, and SHOUTcast to
see how each performed with 10,000 listeners connected. RSAS was shown to be the fastest streaming audio server, with the lowest CPU usage, which is the main bottleneck. With the release of RSAS 1.0, we
decided it was time for a new round of benchmarks.
This year, we're going to try ramping up each server to 240,000 listeners and see what happens.
We'll also be trying to improve our benchmarking technique and answer two questions:
1. Which streaming audio server can handle the most listeners?
This question is important for broadcasters because more efficient streaming server software means
being able to serve more listeners on the same (or even slower) hardware, leading to cost savings. It is also
important to test the maximum capacity of streaming server to check scaling behaviour:
Does the streaming server effectively use all available cores?
Are there any internal bottlenecks in the streaming server that prevent it from scaling?
This year, we're also testing vanilla Xiph Icecast. Although the "Icecast-KH" fork of Icecast is regarded as having better performance,
we wanted to see how Xiph Icecast actually compares.
2. How consistent is the performance of each at high loads?
We've heard from numerous users that Icecast has a listener limit of around 24,000 listeners, after which playback goes choppy. Today, we're going to investigate this and see if we can reproduce it. We're going to push each streaming server to
its limit and see what happens.
Test Setup
We've upgraded our test systems since our last benchmarks, so we can try higher listeners counts:
Both PCs were directly connected using a pair of Mellanox ConnectX-3 Ethernet cards and a 40 Gbps direct attach copper cable.
On the client PC, multiple instances of ab were ramped up and connected via HTTP to the server PC, each simulating 1000 concurrent listeners,
until we reached 240,000 or the server's limit. Standard tuning was also applied to each PC to allow them to handle high numbers of network connections (increased socket descriptor limits, etc.).
Listener counts were recorded by polling the status page of each server.
The server PC was configured with the following software:
RSAS 1.0.0 pre-release
Icecast 2.4.4
Icecast-KH 2.4.0-kh15
SHOUTcast 2.6.0.753
These are all the latest versions as of August 30th, 2021. A single 64 kbps MP3 relay was configured on each Icecast-compatible server.
For Shoutcast, a single 64 kbps MP3 stream was encoded by Rocket Broadcaster using the Shoutcast 1 protocol.
Benchmark Results
CPU Usage vs. Listeners
The average CPU usage of RSAS, Icecast, and SHOUTcast, ramping up to 240,000 listeners connected, sampled over 300 seconds.
Xiph Icecast
Surprisingly, vanilla Xiph Icecast didn't even get off the ground and failed our test immediately. Spinning up 3000 listeners immediately overloaded Icecast, causing it to become
unresponsive and freeze. We even slowed down our test to only spin up 100 listeners every 3 seconds, and this too caused it to become overloaded.
We were unable to collect data because when our benchmark suite polled the Icecast status page to retrieve the listener count, Icecast was unresponsive.
Since it couldn't run the test, Xiph Icecast is omitted from our graphs and the rest of our analysis.
Icecast-KH
Icecast-KH reaches saturation after about 30K listeners
Icecast-KH seems to fair well for the first 50 seconds of the test, as it ramps up to about 30,000 listeners. However,
beyond this point, we see the CPU usage going flat. Icecast-KH is accepting new connections, yet somehow the CPU usage
is not increasing. What's going on?
When manually connecting with a player, we can hear the problem - We're able to connect, but Icecast-KH doesn't send
us any audio. Despite low CPU usage, once the Icecast reaches this saturation point, listeners are able to connect, but are no longer being served streaming audio. This is problematic because it means your production Icecast server may
be failing to serve listeners and hitting an internal bottleneck without you realizing.
We'll see that this theory is confirmed in the bandwidth graph below.
Shoutcast
Shoutcast reaches saturation after about 20K listeners
Shoutcast fairs similarly to Icecast-KH in this test, but exhibits slightly different behaviour. After about 20,000 listeners,
Shoutcast's CPU usage stops increasing indicating the server has reached saturation and the rate at which it can accept new listeners drops. Once it reaches saturation, it is no longer able to keep up with adding new listeners, so by the end of the test, it only reached
60,000 listeners. At saturation, response times increased majorly and new listeners were not able to receive audio.
RSAS
RSAS reaches 240,000 listeners!
RSAS is the only streaming server that manages to run this test successfully, accepting and serving audio to 240,000
listeners simultaneously in our test setup. The graph shows what we want to see: CPU usage increasing linearly as
listener counts increasing. The spikes are showing the extra work RSAS is doing to accept a batch of 1000 new listeners
and are normal behaviour. Above 200,000 we see that the rate at which it accepts new listeners slows down, but it
manages to keep up and successfully reaches 240,000 listeners and serves live audio to every listener.
How do we know RSAS was serving audio successfully to each listener?
To help verify this, we measured the bandwidth usage while running these tests, in addition to doing manual testing. Before running each test, we connected as a real listener to ensure the audio was not choppy as the load test ran. We also connected as a new listener at the end of each test, to verify new listeners were still receiving audio.
Bandwidth vs. Listeners
To help interpret the odd CPU usage scaling behaviour and reports of choppy audio with Icecast at saturation, we measured the bandwidth used during each test:
The bandwidth graphs above are consistent with the CPU usage behaviour we noticed earlier and explains why
listeners receive choppy audio or no audio with Icecast-KH and Shoutcast after saturation - Those servers simply stop sending
audio!
RSAS is the only server that
actually manages to serve live streaming audio to 240,000 listeners. We can see that the bandwidth scales up
linearly with the listener count, as it should, and that it reaches the expected bandwidth usage for this number
of listeners (64 kbps x 240,000 = 15.4 Gbps).
RSAS is able to accomplish this by efficient use of multithreading and optimizations for modern hardware. We optimize
RSAS not just for bragging rights, but so that it scales efficiently on your hardware and is easy to monitor in
production. Even at 95% CPU usage, RSAS is still able to serve smooth audio to your listeners, whereas Icecast and Shoutcast start having problems at much lower load levels, which make them difficult to monitor and troubleshoot in
production.
"Why is my stream choppy?" - Here we showed that the answer can be that your Icecast or Shoutcast server is saturated, despite having low CPU usage.
The Bottom Line
RSAS comes out on top again as the highest performing streaming audio server, allowing broadcasters to serve more listeners than ever before with the same hardware.
Questions about RSAS or planning high capacity streaming audio? Get in touch!
We're pleased to announce the release of Rocket Broadcaster 1.3.27. Over the last year, we've quietly released several
incremental updates with small bugfixes and support for new metadata formats. The cumulative number of changes
is now substantial and worth an upgrade for most users.
The changes in Rocket Broadcaster 1.3.27 are:
Improved WASAPI compatibility with some soundcards with limited sample formats. This should fix the dreaded "Audio Device Error"
with some finicky soundcards.
Fixed compatibility issue causing distorted output with some ASIO devices
Fixed ASIO input volume control and master monitoring
Other recent features and fixes include:
Allow disabling Audio Output and using Input Device as the clock source. (A valid output device is no longer required.)
Multiple application instances will now use separate log files.
Support for Seamless Takeover with RSAS. Broadcasters can now takeover a stream and boot an existing source.
Implemented restarting of System Audio Capture if it fails (useful with Microsoft Remote Desktop)
Pro Edition - If you're a Pro Edition user, click the download link in your purchase confirmation email again to get the latest update,
or visit My Products in the Oscillicious Shop.
Feedback
Lastly, your feedback is what helps drive Rocket Broadcaster development. If you're looking for a missing feature,
or have ideas for how we can make Rocket Broadcaster even better, please let us know!
RSAS Status Update
RSAS is still under heavy development and we've quietly released many alphas
of the 0.1.21 series, which is now likely to evolve into a proper "1.0" release. This next release will see improved HTTP/1.1
support for serving static files and podcasts, support for preroll and postroll jingles/ads, the option to auto-stop relays,
a significant number of bugfixes, and with any luck, some form of HLS support. All of those feature are already complete, but
HLS still needs some polish before we can ship it. Stay tuned for RSAS 1.0 news!
We have been monitoring the recently discovered Log4j vulnerability (CVE-2021-44228)
and have taken action to audit and monitor our infrastructure for potential risks. As creators of streaming audio software and
a global streaming audio platform, we wanted to reassure broadcasters about the status of our products:
Rocket Broadcaster - Not vulnerable to the Log4j security vulnerability. Rocket Broadcaster and its third party components do not use Java nor Log4j.
Rocket Streaming Audio Server (RSAS) - Not vulnerable to the Log4j security vulnerability. Does not use Java.
Radio Mast - Not vulnerable to the Log4j security vulnerability. More details here.
For more information about the scope of this Log4j vulnerability and the steps we've taken to assess our infrastructure and keep broadcasters safe,
please see our Log4J Vulnerability Update on the Radio Mast blog.
We're pleased to announce the release of RSAS 0.1.20! Rocket Streaming Audio Server (RSAS) is a high-performance webserver designed for broadcasting live streaming audio through the web, with low latency and high listener capacity.
This release contains major performance improvements, a new management API, tons of small improvements to existing features, and bugfixes.
We're skipping it. Late in the development cycle of 0.1.19, we started pursuing a new line of optimizations to boost performance on very large servers with hundreds of thousands of listeners.
We started working on these optimizations in a separate branch for 0.1.20 and continued final preparations for the 0.1.19 release.
However, our benchmark results for 0.1.20 were so good that we put it through our QA process and quickly started using it in production
at Radio Mast. We've excited to share 0.1.20 with you as an even better
version of RSAS than we initially planned.
Without further ado, here's what's new in RSAS 0.1.20:
Performance Boost
In this update, our new optimizations have achieved a roughly 2x reduction in CPU usage at high workloads, enabling broadcasters to reach more listeners
than ever before on a single server. We've also further reduced the CPU usage for relays. No changes or config tweaks are
required, simply upgrade and enjoy.
On our server test rig with a Ryzen 2700 processor (8 cores, 16 threads), RSAS is now able to serve 240,000 concurrent
listeners on a 64 kbps MP3 stream. This is on a consumer-class CPU that is already 2 generations old. Importantly, RSAS fully utilizes all available
cores and is limited by CPU speed alone.
RSAS /health screenshot, htop showing CPU utilization, and nload showing network traffic with 240,000 listeners.
In the nload screenshot, RSAS is sending almost 15 Gbps of traffic!
We'll be publishing an updated set of benchmarks with more details soon, so be sure to check back.
Manage API
The new Manage API allows you to perform management actions on individual mounts. Two actions have been implemented thus far, allowing you to
force a source to disconnect (kick) or move listeners to a different mount.
Compatible encoders can now seamless kick and takeover an existing stream when they connect.
When connecting to RSAS with Rocket Broadcaster 1.3.22 and up, if a stream mount is already in use, users will be
prompted if they want to kick the existing source. This allows broadcasters to seamlessly takeover a stream or switch DJs without any dead air.
Other Small Features:
Relay URLs can now be changed dynamically via config reloads. Listeners will get moved to the new URL.
Added admin-password config setting from Icecast (currently only used for the Manage API).
Added optional <config-version> field to config file format and is exposed in /health. This can help you track whether your new config has been
loaded.
Added last_config_reload field to /health (UNIX timestamp)
Added --test/-t flag for checking config file syntax
Added warning if default passwords used (hackme/adminhackme)
Prevent blank passwords from being used.
Improved Icecast compatibility:
Implemented <hidden> mount flag from Icecast, which hides a stream from the emulated Icecast status page and status-json.xsl page.
Added <alias> directive, allowing you to remap URL paths.
Added the "stream started" timestamp to Icecast status page and to the status-json.xsl.
HTTP referer now shows up in the access log.
The "referer" header is not supported by all browsers and players, which can make traffic attribution tricky. To make accounting for your traffic sources easier, you can now override this header by setting the "referer" querystring parameter. (eg. /mystream?referer=myapp)
Fixed serving static files on Windows, which prevented the LetsEncrypt wizard from working for some users.
Fixed FLAC compatibility with BUTT encoder.
Fixed rare relay race condition that could lead to a crash.
Fixed HTTP "referer" always being blank in the access log.
Fixed a rare crash related to TLS.
What's Next?
RSAS 0.1.21 is planned to come shortly, and will include improved HTTP/1.1 support for serving static audio files on-demand, such as podcasts.
It will allow seekable playback of large audio and video (!) files out-of-the-box.
These features will also help round out our HLS implementation, which we're hoping to release as an experimental feature soon.
Audio preroll (aka intro files) is high on our list of priorities and we are planning this for Fall 2021.
We're hoping to throw in some experimental features with this too, like the ability to trigger audio insertion mid-stream (mid-roll).
Lastly, we would like to thank our dedicated users for all your feedback on our previous releases. Your feedback has helped
us make RSAS better for the global broadcasting community. Thank you!
We're pleased to announce the release of RSAS 0.1.18! Rocket Streaming Audio Server (RSAS) is a high-performance webserver designed for broadcasting live streaming audio through the web, with low latency and high listener capacity.
This release adds a couple new features including persistent relays, SNI support for multiple SSL/TLS certificates, as well as
major performance optimizations, bugfixes, and more. The update is the culmination of 6 months of development work which
included over 18 minor alpha releases that were tested across the globe.
Persistent relays are now the default type of relay.
In previous releases, all relays were "on-demand", which meant they only started relaying after the first listener
attempted to connect. Persistent relays automatically connect at startup and will continually try to reconnect if the
stream ever drops. Since a relay must be online before it can be used as a fallback, persistent relays are now recommended for fallbacks.
A single RSAS instance can now use multiple TLS certificates. This is needed when you have multiple domains pointing at a single server
and you wish to use HTTPS on all of them. RSAS can now load a different TLS certificate for each configured hostname, via a mechanism called SNI.
To configure multiple TLS certificates, a new <vhosts> section needs to be declared in your config file.
Various parts of RSAS have seen major performance improvements, including a large reduction in CPU usage and
reduced disk access. We're working on new benchmarks, but so far we've seen that RSAS can handle over 10X the listeners compared
to the other streaming server software we tested.
Upgrading?
Please see the "Behaviour change" notes below. Most users should be able to upgrade without issue. Users with large numbers
of relays should test that persistent relays operate as intended with their setup. If you need the old on-demand
relay behaviour instead of persistent relays, add <on-demand>1</on-demand> to your <relay> config section.
Bugfix: Improved compatibility with Unity game engine - Only send in-band ICY metadata if it changed.
Bugfix: Various other bugfixes
What's next?
We'll be continuing to refine RSAS and improve the features that our users are utilizing the most. Over the last few
months, much of our effort was focused on optimization and refinement of existing features. We are still experimenting with HLS
and working on rounding out the HLS feature set. We've also laid the groundwork for some other features we're not yet
ready to announce, so stay tuned.
