To learn more about Picture-in-Picture I created a playground project to have a play with the new features. You can check this out for yourself here!
When speaking about Android TV, I previously mentioned how it would be great to allow users to continue exploring content whilst still consuming media, just like we can do on current set-top boxes. However, at DroidCon Bucharest over the weekend I was excited to be able to add some slides and talk about the new Picture-in-Picture feature that allows us to do this!
Before my flight back from the conference I had a couple of hours to kill, so I created a quick & minimal playground application to try out the Picture-in-Picture functionality. Let’s take a look at what I learnt from my experience.
Hold up! What is Picture-In-Picture?
Picture-in-Picture (PiP) allows you to display a pinned window in the corner of the screen, allowing the user to continue consuming media content from the original playback window instance whilst continuing to navigate around your application. Using this pinned window we can persist a viewing activity whilst another activity is browsed, allowing the user to remain immersed in the TV experience.
The PiP window is displayed at 240 x 135 dp in one of the screens corners and is always on top of all other content, the corner in which it is displayed is automatically decided by the system. There are several different cases where we could make use of this window:
- The user may want to just browse around your application whilst they’re currently consuming media. For example, when the user navigates back from watching something we can continue to display the media being consumed whilst they continue using the app as normal.
- The user may be coming to the end of the current content they’re watching, so we’d want to display some relevant information whilst the content finishes. For example, during the credits of the current TV episode being watched we could use the main screen to display information about the next episode in the series whilst the finishing episode continues to play in the PiP window.
- The user may wish to queue up additional content, so that they continue consuming media seamlessly once the current item has finished. Here, we could use the Main Window to display cards of additional content that the user can add to their ‘Up Next’ queue to be consumed after the current item finishes.
I think these are situations where the PiP window acts as a great addition to the leanback library, allowing us to provide a more productive, focused and frictionless experience to the user.
Enough talking! How can I implement it?
Well, thanks to the leanback library, implementing Picture-in-Picture into our applications is a fairly simple process – there’s only several steps we need to take to allow us to display PiP-windows to our users!
Declare Picture-in-Picture support for the activity
To begin with, we must add some new attributes to our activity declarations that we wish to allow support for picture-in-picture. We can do this like so:
- resizeableActivity – This attribute allows us to enable/disable multi-window support for our activities. When true, our activity can be launched in split-screen / freeform modes. When false (default), the activity will be launched in full screen.
- supportsPictureInPicture – Declares whether or not the activity supports Picture-in-Picture mode.
- configChanges – We use this attribute to let the system know we’re taking care of these changes ourselves. If we don’t do this for Picture-in-Picture windows then our activity will be recreated when PiP transitions take place.
And it’s as simple as that, we can easily add these attributes to enable Picture-in-Picture for our existing activities. So now we’ve declared our activity supports the PiP feature, we need to add the functionality to our app.
Adding the Picture-in-Picture Action
As of Android N, the PlaybackControlsRow now comes included with the new action we need to add the Picture-in-Picture capability to our controls. This new action is known as the PictureInPictureAction and can be created like so:
PlaybackControlsRow.PictureInPictureAction action = new PlaybackControlsRow.PictureInPictureAction(activity);
We can then then simply add our new instance of the action to our playback controls adapter:
And just like the other actions that come with the leanback library, the PiP actions and drawable states are all handled for us. Producing a result like below:
Doesn’t it look great? So next we just need to listen for and react to click events on this new button of ours!
Switch your Activity to Picture-In-Picture mode
Just like we listen for the other actions, we can listen for click events on the action using the reference to our action and its ID, like so:
Note: If you don’t want to use the action reference, you can simply use the ID of the action which is R.id.lb_control_picture_in_picture
When we detect that the PictureInPictureAction was clicked, we simply make a call to the activity enterPictureInPicture() method. This essentially tells our activity to transition to Picture-in-Picture mode and on devices that are not running Android N, this method call will simply be ignored.
Taking care of the UI whilst in Picture-in-Picture mode
Our activity / fragment can also listen for when it enters Picture-in-Picture mode. This is done by overriding the onPictureInPicture method, which receives a boolean value stating whether it’s currently in Picture-in-Picture mode or not.
This can be a good time to hide any User Interface components that are currently being displayed on the screen. This is recommended to do as when we display the Picture-in-Picture window, we don’t want to be showing many components in such a small window – the only element displayed should be the currently playing media. When our user returns to a full screen activity, we can then bring these UI components back into view.
Continuing playback whilst in Picture-in-Picture mode
When our activity enters PiP mode, we need to use the onPause in our activity lifecycle to continue the playback of our media:
When our activity enters PiP-mode, the onPause method is called as the system treats our activity as if it’s in a paused state. However, we don’t want our media to act in this way so we must handle the playback of our media from this method by continuing the playback of our media.
Note: When we exit PiP-mode to return to a full screen activity, the onResume method of our activity is called.
What about any Gotchas I should know about?
It’s still early days so I haven’t had too much of a chance to find any oddities that you should be aware of. However, there is one thing I cam across….
In my sample app, when you launch Picture-in-Picture mode for a playback activity the window transforms into a PiP window and the ContentFragment comes into view for you to browse content. Now, at this point if you then launch another PlaybackActivity then the new media begins to play inside of a new instance of a PlaybackActivity whilst the previous PiP remains in view. To me this is very odd, there shouldn’t be media playing inside of a PiP window whilst another video is playing in a full-screen activity (unless I’m really trying to catch up on a TV show and watch two episodes at once 😉 ).
As far as I know, there doesn’t seem to be an easy way to retrieve a current instance of a PiP window – we can only check if an activity is in Picture-in-Picture mode by using the inPictureInPicture() method from the activity itself. Because of this, there currently two things we could do here:
Kill the previous PlaybackActivity
In my playground app, as a quick workaround (I had a flight to catch!) I register to an event bus in my playback activity and listen for any events where a new PlaybackActivity is launched. If this event is received, then we simply call finish() in the PlaybackActivity – this means that the activity will be killed and before the new PlaybackActivity is shown, meaning only one PlaybackActivity will only be active at any one time.
Re-use the PlaybackActivity
However, ideally we should make use of the existing PlaybackActivity rather than killing and recreating an instance when we already have an instance that exists. What we could do here is just use the newly selected media and pass it’s content to the instance of the PlaybackActivity that is currently in PiP-mode, transitioning out of PiP-mode at the same time to return to a full-screen viewing activity. This could be a nice way of overcoming this and it’d feel more natural returning to a single Playback window instance when the user is wanting to consume new media.
Are there any Best Practices I should follow?
Now we’ve got this new functionality in our app, we should be sure that we’re not making things difficult for the user. Whilst this seems like a pretty simple component, there are several things to bear in mind when implementing the PiP window into your application to ensure a frictionless experience for the user.
Note: These are currently my opinions and likely to change. I’d love to hear of any situations you think may be harmful to the users experience!
Don’t obscure content
We’re being given the ability to display content on-top of other content on screen, so because of this we should be careful not to obscure any important information from the user. For example, if we’re displaying information about the next episode on-screen then we should make sure that the PiP window does not make this information unreadable for the user. Because we currently don’t have control over which corner the PiP window is displayed, you should bear this in mind and ensure critical information is not displayed in any corner of the screen.
Use PiP windows to display media content only
Only windows that are displaying media content should be able to enter PiP-mode. For example, the user would not benefit from a settings or detail screens entering PiP mode as this content would be both non-interactable and indistinguishable. Showing consumable content in this window allows us to make the most of the PiP component and provides the user with a way to browse our application whilst still consuming content.
Don’t show Pip windows on-top of consumable content
We should only show PiP windows on-top of content that is not consumable, avoid showing a PiP window on-top of another piece of consumable media. For example, we shouldn’t show a PiP window on-top of another Playback Activity as this would make it extremely difficult for the user to focus on what they’re watching. To avoid this, only use PiP mode when the ‘base’ activity is a browsable, not consumable.
Hide UI components when in PiP mode
When we enter PiP-mode we should hide any UI components that are currently being displayed in that activity, such as playback controls. This means that when the window transition occurs, the consumable content will be easily distinguishable without hiding these elements. These controls aren’t usable when in PiP mode anyway, and such a small area can become easily crowded with unnecessary components. When the PiP window returns to a full-screen activity we should return these UI components to their original visible state.
And that’s it!
So we’ve taken a look at this new Picture-in-Picture window and how to implement it – so now it’s time to build this into your existing / new applications! I think it’s an awesome addition to the Android TV platform and I’m interested to see real-world uses! If you have any questions, please feel free to tweet me or leave a response below!
p.s don’t forget to hit the recommend button if you enjoyed this article 🙂
Check out my other projects at hitherejoe.com