Content is certainly king, and today video is the king of kings. Users love to stream whether they’re binging on House of Cards or watching March Madness games. Today, users can watch in more places and on more devices than ever. At POSSIBLE Mobile we build a lot of streaming apps, supporting Roku, Apple TV, Android TV, Fire TV, Chromecast, in addition to iOS and Android devices. On all of these devices, YouTube and Netflix have set a high bar in terms of speed and quality that users have learned to expect even though they have no concept of the complexity hidden behind the scenes. Slow load times, constant buffering, or jittery frames are worthy of poor ratings.
For Android video players, a lot of projects use a 3rd party library or Android’s built in MediaPlayer. However, there’s a better solution. ExoPlayer is Google’s open source video player built on low level media APIs. A couple of our projects have made the switch. Those projects are not switching back and I strongly believe that any Android app with a video player should use ExoPlayer – Youtube, Periscope, Starz, and FXNow already use it. It is fast. Shockingly fast. It’s hosted on GitHub with a great community of developers, a great support team who have closed 930 issues to date, and a team from Google actively working to improve it. It’s 100% Java so there’s no need to make the mental switch from one language to another as you read through its code. Often 3rd party libraries are Java wrappers around native code, which causes a lot of headaches when debugging. ExoPlayer can be updated to the latest version in normal app updates, unlike MediaPlayer which requires the user to update their OS to update. Unlike 3rd party libraries, it isn’t a black box with obfuscated, and often native, code.
When announced, ExoPlayer made a slight ripple at 2014’s I/O event. At the time it only supported DASH and SmoothStreaming, but now it supports traditional formats like MP3, MP4, and WebM, as well as HLS (HTTP Live Streaming) – the de facto standard in streaming content since it works for both iOS and Android. It also supports DRM. All of the streaming apps we build at POSSIBLE Mobile use HLS. Hopefully, projects will switch to DASH soon since it is much faster and the accepted international standard, but first the engineers in Cupertino need to learn that standards exist for a reason. When the switch to DASH finally happens, projects using ExoPlayer will be better positioned to make the switch since they will only have to swap a single piece of the implementation, the RendererBuilder.
With context out of the way, let’s dive into details. First add the following to your gradle dependencies:
Note 1: Latest version as of writing is 1.5.6
Note 2: Requires jCenter. Go ahead and replace mavenCentral() with jcenter(). jCenter is a superset of mavenCentral.
Note 3: Min SDK is 16. If you’re supporting devices older than Jelly Bean, I’m sorry but this isn’t for you.
Noted. Next we need a handful of key ingredients and a googol callbacks. Yes, the options and listeners can be overwhelming at first glance, but don’t over think the callbacks, enjoy the fine grained control and adaptability they present. For starters, most of the callbacks can be ignored or merely logged out.
The overall ExoPlayer implementation requires a Surface, a RendererBuilder, an ExoPlayer object, and a PlayerControl. The Surface is the drawing canvas for your video frames. The RendererBuilder asynchronously retrieves renderers, which is a long word for makers or providers. A renderer provides a stream of audio, video, captions, or metadata. The ExoPlayer class glues all of the components together by preparing renderers, attaching the video renderer to the surface, and building the controller; but mostly it delegates responsibility to those other objects. The PlayerControl handles play, pause and seeking operations. It’s very straightforward to use.
All of those components can be grouped inside a fragment, layout, or activity. I personally like implementing the video player logic in it’s own custom layout and including that layout within an activity. This avoids the odd life cycle of a fragment, while still separating the video player logic from the activity. Since the activity can be crowded with authentication, ads, analytics, lifecycle, controls, etc, for cleaner code it helps to keep the video player code in a separate place.
Another way to keep your player code clean is to use inheritance when you build the video player’s custom layout. For example, a controller class can handle the interaction with controls and extend a class that solely deals with analytics. That analytics class can listen to callbacks to notify an analytics manager of events and extend an abstract class. That abstract class can handle all of the lifecycle and initialization that the player requires to work. It could extend a class that logs out the unnecessary callbacks so those don’t clutter up the other classes. And that logging class can extend a FrameLayout. I’ve implemented a player like this and it has helped immensely to keep the code organized. When something in the player inevitably breaks, it’s much faster to track down the part of the player that is breaking and there’s very little clutter obscuring the bug so it has nowhere to hide.
One of the most complicated parts of an ExoPlayer implementation is building the renderers. This part is very HLS specific and requires a little HLS context. HLS requires a url to load a manifest, also called a playlist. This master playlist includes urls to other manifests for audio, captions, and video segments. The short segments of video can be encoded at different bit rates for the same part of the content. Based on performance, the player can shift up or down between bit rates to provide the most continuous playback at the best bit rate. The RendererBuilder’s job is to start with the first url and output the renderers by loading the manifest and initializing components necessary for the renderers.
Because this is network based, it must be handled in the background thread. Note: I don’t think it’s necessary to detail all the parameters involved in each method mentioned. For one, this description would balloon tremendously. Secondly, ExoPlayer frequently changes the necessary parameters in minor updates so this information would quickly be outdated. For an up to date and more in depth look at HlsRendererBuilder, check the demo app’s example.
The HlsRendererBuilder needs an interface that the video player will implement. To start building, it needs to construct an AsyncRendererBuilder, which will go into the background and start the process. The constructor and necessary instance variables are not shown.
This AsyncRendererBuilder is a private class inside HlsRendererBuilder. Calling single load on the ManifestFetcher starts the asynchronous loading of the HlsPlaylist. The AsyncRendererBuilder must implement ManifestFetcher.ManifestCallback to listen for when loading is complete.
After a successful load of a valid HlsPlaylist, I like to think of the next steps involved like Russian nesting dolls with each one going inside the next one until finally reaching the outermost doll, aka the renderers. A DataSource builds a ChunkSource, which builds a SampleSource. The SampleSource builds the renderers for video, audio, text, etc.
There’s a whole lot of other options, objects, and listeners necessary to build each components. I recommend following the demo linked above to understand how to build the extra objects and what options to use for defaults. For the listeners use a single logging class as mentioned above and only implement the callbacks necessary for your analytics and controls. For a basic player, it can just override onPlayerStateChanged to get updates when buffering starts or finishes and when the content completes. Also the player should handle the error callbacks.
Once the renderers have been built, they need to be passed back to the main video player implementation. The ExoPlayer object will then prepare the renderers and attach the VideoRenderer to the surface.
exoplayer.sendMessage(videoRenderer, MediaCodecVideoTrackRenderer.MSG_SET_SURFACE, surface);
This is also an opportunity to store the renderers for future calls. For examples of why you should store the renderers, the audio volume can be adjusted by holding onto the AudioRenderer. Also when the surface is created or destroyed, the Exoplayer object needs to attach or detach the VideoRenderer to or from the surface. To get those surface callbacks, you must attach a SurfaceHolder.Callback to your SurfaceHolder.
Finally, add setPlayWhenReady(true) so that playback will start as soon as it has loaded, which will be pleasantly fast. There’s a lot more that has to be done for a polished, complete video player from adding controls to adjusting the buffering times and sizes for different devices. However, following the steps above should get HLS content playing within your app.
For some inspiration, I’ll mention some ways that the extensibility of ExoPlayer helped in my project. My project extended the default VideoRenderer and added time checks at the front of its processOutputBuffer method, which allows us to know exactly when the video has reached an ad break or the 50% complete mark. Another nifty extension involved overriding the captions classes to add a captions fix that we found in a pending pull request, which hadn’t been merged into the latest version. By now, a fix has been merged in so our custom captions classes have been deleted. Additionally, we use an OkHttpDataSource instead of the default DataSource, which allows our app to use a single source for networking, reducing complexity and potential errors. None of these extensions or modifications would be possible with any other video player, which really speaks to how amazing ExoPlayer is.
Feel free to grab my project here and play with it. Try tweaking some of the values to see how performance is affected. You can adjust the minimum buffering necessary for the initial load and subsequent buffers. Note: RENDERER_COUNT is the number of renderers you’re adding to ExoPlayer and shouldn’t change unless you add a new renderer for captions or metadata. ExoPlayer’s GitHub page also has a pretty good demo and the developer guide is another great source of info.
To be honest, ExoPlayer is not a single knockout punch then drop the mic solution. Implementing ExoPlayer will be a considerable spike in effort and switching from the known to the unknown replaces known bugs with unknown bugs. Even after mastering the player’s lifecycle, all your analytics, player controls, and ads will have to be re-added and thoroughly tested.
Before you shake your head and wander away to do something easier like beating AlphaGo in a Go match, remember the benefits of ExoPlayer – open-source, performance, and highly customizable. Also don’t forget the costs of using black box video libraries.
FATAL SIGNAL 11
That was an actual error from a 3rd party library with no context or hints as to why it happened. When building Android video players, let’s work with an open-source, extensible, efficient video library instead of 3rd party libraries or the built in MediaPlayer. ExoPlayer has a lot of perks and will continue to improve. Make the switch for performance. Make the switch for your sanity. Make the switch today – or as soon as your project has time for some extra scope.