Android TV aims to build on this existing experience by bringing an immersive and engaging experience to households. In order to make this possible, Google has developed the LeanBack library to enable developers to create their own media experiences to deploy to Android TV devices.
Vineyard
To really understand how the Leanback library works, I decided to create Vineyard – a Vine Client for Android TV. This app currently allows the user to:
- Browse feeds from a range of Vine Categories
- Watch Vine video posts
- Search for Vine Users and Hashtags
- View video grids for Users and Hashtags
- Toggle the Auto-Loop option via Settings
There’s not many screens to the application, but it can get a little complex to follow. The structure of the activities/fragment of the application consists of:
Doesn’t look too bad, right?
We’ll be using the screens and components of this app throughout this article as examples of how to create the different parts that represent the application.
I really enjoyed creating this app and I feel that Android TV is a great platform to build for. Whilst I only skim the surface of how I created the application, I decided to write this article in the hope that others will want to create for the platform and be able to use what I have created to help them along the way.
Understanding TV Navigation
To begin with, let’s take a look at how navigation works on Android TV, as the navigation model differs greatly from both phones and tablets.
The standard Android TV remote control has a set of buttons in the form of a 5 button D-Pad, allowing users to navigate in four directions and select a desired item. The controller also has a back button to navigate back through the flow of an application, along with a home button to return to the home screen of the Android TV system.
Android TVs do not use an on-screen pointer, as unlike phones and tablets there is no touch screen support for TVs. Whilst touch screens allow for users to randomly access different components on the screen in whatever order they like, android TV uses a focus based navigation model which only allows users to move between components one-by-one in either a vertical or horizontal manner.
Note: There is no menu button used in the Android TV UI, meaning that the overflow menu we are used to on mobile devices is not present.
Leanback Library
The Leanback library offers a great hand when it comes to building an Android TV application, it comes with plenty of out-of-the-box classes that make it that bit easier for you to create something that is both consistent to the platform and easy to use. Some parts of the framework we use in Vineyard are:
- BrowseFragment — Allows you to easily show rows of browsable content cards
- VerticalGridFragment — Allows you to easily show a vertical grid of browsable content cards
- SearchFragment — Provides an interface to search for content provided by your application
- ImageCardView — A simple content card, extending BaseCardView, that displays an Image with Title and Description beneath it when focused.
- Presenter — used to generate views and bind objects to them, we use instances of ListRowPresenter, RowPresenter and VerticalGridPresenter.
- PlaybackOverlayFragment — A fragment that can be used to display playback controls over the top of currently playing media
Setting up your manifest
There are a few attributes that we need to declare in our manifest before we get started:
- We need to declare that the use of the microphone for voice input used in the search feature is not required to use the app. This is because not all input devices will have a microphone.
- We need to declare that the use of a touchscreen is not required to use the app. Most TVs aren’t touchscreen (currently…), so we need to say that this hardware isn’t required to use the app.
- The last attribute is stating that the device must support Leanback, meaning that the app can only be installable from the Google Play Store on Android TVs. If your app is for more platforms than TV then you won’t need to add this declaration.
https://gist.github.com/hitherejoe/7b23588654eaa4dc3fa6
Finally, you’ll just need to add the Leanback dependancy to your build.gradle file and you’re all good to go!
compile 'com.android.support:leanback-v17:23.1.1'
Browse Fragment
The browse fragment is what we use to display rows of content to the user, these rows allow for easy interaction with the application content. You’ll see below in the example that we display rows of square cards alongside a list of categories. As the user navigates vertically through this list of categories, the row of cards will also transition to the currently focused category.
Our fragment extends the BrowseFragment class from the leanback library, which provides us with most of the layout complexities out of the box – at the minimum for basic applications, all you need to do is provide it with the data to display.
However, it’s important to understand how this data management works. There are several components that are required to make up the display of content in our fragment.
- Header Item Presenter — This presenter is used to display the list of categories on the left-hand side of the screen
- Array Object Adapter — This adapter is used to hold instances of the List Row, each item in the adapter makes up a row of content in the fragment
- List Row — A list row contains both the Header of the row and the adapter used to display data for it. In this case, the PostAdapter is used to manage the content we wish to display on the screen
But what about making it pretty? Well, we can give our fragment a basic makeover using several properties that can be easily set programatically:
- setBadgeDrawable() — Set the badge drawable to be displayed in the top-right hand corner of the screen
- setBrandColor() — Set the brand colour for the fragment, this is the background color of the category pane on the left-hand side of the screen
- setSearchAffordanceColor() — Set the colour of the Search icon in the top-left hand corner of the screen
- setHeadersState() — When we enter a category to browse videos, this setting decides if the category heading is to be displayed
- setHeadersTransitionOnBackEnabled() — When we’re browsing a row of cards and press the back button, if we want to transition back to the list of Categories then we use this method to enabled the feature
https://gist.github.com/hitherejoe/c434b46b78b116d1cc29
Categories
In the screenshot you’d have noticed the list of Vine categories in the pane on the left-hand side. As previously mentioned, this is managed by the Header Item Presenter that we set during the fragment setup:
https://gist.github.com/hitherejoe/e06f430fb846ac189c83
This makes our IconHeaderItemPresenter handle the presentation of our Header items. Using the Icon Header Item layout file, the presenter will:
- Retrieve the list of category names from the Strings file
- Set the icon as the corresponding drawable for the item
- Handle the selected state (alpha) of the categories
Content Rows
The BrowseFragment uses an ArrayObjectAdapter to store the data that is displayed as rows of content to the user. You can add your content to this by adding an instance of an adapter, with a HeaderItem, as part of a new ListRow instance.
We use a PresenterSelector to allow the display of multiple card types in our row of content.
Once we have the adapter setup, it’s pretty simple to add new data to it:
https://gist.github.com/hitherejoe/b64fa2c92f0bb084d057
The HeaderItem referenced in the code above is the title to be displayed above each row of content. It remains hidden until the user begins browsing a row of content.
We’ll get into the card views in a bit, but in a nutshell:
- Users can browse through the rows of content. When a card becomes focused, it will enlarge and the video for the Post will automatically start looping in the card view. Moving to another card will stop the video for the previous card and start the video for the current one.
- When the user reaches the last item in the row, a paginated request for the next set of items will automatically be made. If items are returned then this will be added to the list and the ‘next page’ value is added to the adapter. If we reach the last page then this is flagged and a request will not be made when the user next reaches the last item in the row.
- If there are no cards currently loaded and the request returns an error, then a “Try Again” card will be displayed so the user can re-attempt the request.
- If there are already some cards loaded and the user makes a paginated request which returns an error, then the user will simply be shown a toast error message. The request will then be reattempted when the user focuses on the last item of the list again.
- If there are no cards currently loaded and there are no results returned, then a “Check Again” card will be displayed so the user can make the request to check again.
Fragment Background
As the user navigates through the posts, we change the background of the fragment to a the thumbnail used for the focused Post. This helps to create a more dynamic experience and builds engagement. The Vine thumbnails aren’t the best quality, so it helps to have high quality images for this.
To do this, we use Glide to fetch the image and display it:
https://gist.github.com/hitherejoe/af248540ef6ff421c661
Custom Pagination Adapter
I wanted to be able to load small chunks of posts at a time, yet allow the user to load more if they reached the last item in the row – sounds like pagination to me! For this, I created the PaginationAdapter which handles several different aspects:
- It holds the tag which we’re using in our query, the next page to be fetched from the server and the anchor string used in the paginated requests.
- It uses a PresenterSelector to determine which item is to be displayed in the adapter. This allows us to display instance of Post Cards, Loading Cards and Icon Cards. It also contains methods to handle the display of these cards.
Having this custom adapter allows us to encapsulate the pagination data from outside of other classes and allows a lot of the logic to be reused for different adapter across the application.
Custom Views
Whilst the leanback library is great, it doesn’t provide a vast variety of card types for the display of content. In this case, the application uses several custom views that I created to enhance the user experience.
Video Card
When a card view displaying a Post item becomes focused by the user when they’re browsing content, the card will automatically start looping the video as a preview for that Post.
When the video is no longer focused, we call the finish method for the view and the card is returned to its normal ImageView state. It adds a nice touch to the application and helps to make the user feel more engaged then only seeing static content – a mimic of real TV behaviour.
To begin with, I created a custom LoopingVideoView which extends VideoView to handle the looping and audio properties of the video — this view is then used in the custom PreviewCardView layout I created.
As seen in the layout file for this widget, we show a ProgressBar along with a transparent overlay on top of the ImageView whilst the video is being loaded. Once the VideoReady listener is called we remove this progress bar and overlay, this is when the video starts playing.
P.s. Thanks Ben Breckler and Matt Oakes for this suggestion!
Loading Card
When the PaginationAdapter is waiting for data, we want the user to be aware that something is happening in the background. For this reason, I decided to create a simple view that contains a ProgressBar inside of it.
Now, whenever network requests are taking place this view is shown to notify the user that content is being loaded.
To implement this, we use a simple layout alongside the LoadingCardView which handles the actual display of the card contents. The LoadingPresenter is used by the content adapters to handle the initialisation and display of the card itself.
Icon Card
The IconCardView was created so that I could display an Icon along with a Title/Value combination. Whilst the adapter for the rows in our content fragments will detect an Option model, the IconItemPresenter will handle the actual display of our Icon Cards.
The structure of this view is also quite simple. We again extend the BaseCardView to create our IconCardView class.
Our Icon Card inflates the layout for the class which consists of an ImageView to display the icon, one TextView for the card title and another for either a subtitle or a value.
Creating these custom views helped to show that you really can do a lot with the content cards that are displayed on the screen. I didn’t originally have the VideoCardView, but when I implemented it it really made a difference to the look and feel of the application.
Whilst the BaseCardView is fairly flexible, it’s important to remember that these should be kept as visual as possible – you’ll notice I’ve kept the text content to a minimum. Being a TV app, it’s important to use visual approaches where possible.
Playback Activity
The PlaybackActivity is the class that handles the playing of our video content (you guessed it…). This is one of the classes that doesn’t come from the Leanback library, so is something you’ll have to build for yourself.
The activity contains two components in its layout:
- The VideoView that is used to play the video
- The PlaybackOverlayFragment provided by the leanback library that is used to display the controls and related content (we’ll look at that next)
To understand the way media playback is handled, i’d recommend checking out the code of the activity and the documentation for the MediaPlayer.
MediaSession actions are retrieved by the activity from the Playback Overlay Fragment to trigger events on the VideoView. We make use of a custom action to allow the auto-loop feature to be toggled from this activity. Our activity listens for this action from the playback controls fragment and acts accordingly:
https://gist.github.com/hitherejoe/851eb5b8a0dd33b18267
Playback Overlay Fragment
The PlaybackOverlayFragment comes from the Leanback library and is used to display playback controls and additional content over the top of any currently playing media. The display of these controls is made up of:
- PlaybackControlsRowPresenter — The presenter used to display playback controls. We can set an onActionClickedListener to respond to click events on these controls.
- ControlButtonPresenterSelector — This presenter holds the adapters used to display rows of controls in the fragment. In the controls you’ll notice a row of primary buttons, with a loop action underneath. A first adapter is used to hold the primary buttons, whilst a secondary adapter is used to add the loop button to the fragment.
As well as the display of playback controls, the fragment also displays a list of ‘Related Posts’. In the case of this application, we’re currently just using the list of posts that the user was previously browsing. This stream of content means the user can watch videos one after the other.
Post Grid Fragment
The PostGridFragment extends the VerticalGridFragment class so that we can display an extended grid of videos to the user. This works in a similar way to the BrowseFragment, except we setup the fragment using a VerticalGridPresenter so that our objects are rendered in a VerticalGridView.
The fragment is launched when you select a Hashtag or Username in the SearchFragment, this is so that you can view a complete set of Posts for the desired result. When selected, the chosen User or Tag object is passed to the PostGridFragment and the Posts are requested and displayed in the grid. If an error occurs when this takes place, then an ErrorFragment is displayed to the user and the activity is finished.
This fragment also uses our PaginationAdapter, the only difference is that the next page of items is request when the last row of the grid is reached, rather than the last item.
Search Fragment
The Search Fragment extends the SearchFragment provided by the Leanback library and allows the user to search for Vine posts via a keyword query.
When the user types text into the search field or uses the microphone to search, then a network request is made to the Vine API. We make two requests here:
- Search for Tags
- Search for Users
The two responses of these are combined and ordered by Post count before being displayed to the user.
Search Results
If search results are returned, then the items are added to our TagAdapter. This adapter uses the TagPresenter to present the search results in the form of a Tag Card.
We use an OnItemViewSelectedListener on the TagAdapter to detect when a Tag Card becomes focused. When this occurs, a network request is automatically triggered to retrieve the latest Posts for the currently focused Username or Hashtag. When the result is returned, the Post cards are displayed beneath the Tag cards.
From here the user can:
- Click on a Tag Card to launch a PostGridFragment to view an endless grid of Posts results for the selected Username or Hashtag.
- Click on a Post to launch the PlaybackActivity to begin watching a stream of videos.
No Search Results
When there are no search results we show a simple message to the user to let them know that their query didn’t return anything. When we receive the results from the search query, if there are none then we clear the adapter and add a new HeaderItem to display the ‘No results’ message:
https://gist.github.com/hitherejoe/ff2ef7eb0c4aa840741b
Custom Views
Again, we use a custom card view in the Search Fragment displayed above.
Tag Card
The Tag Card view is used to display a simple Icon and Title combination, in this use case it is used to display either a Username or Hashtag result — the layout is defined in the view_tag_card.xml layout file.
Using a card view for this component helps to create a consistent look and feel across the application.
Auto-loop Guided Step Fragment
In the Vine app, videos automatically loop when they’re being watched, so I thought it to be important for the TV version to mimic this behaviour. However, some users may just want the list of videos to play through to the end – so I decided to add this as a changeable setting using a GuidedStepFragment. This class from the Leanback library allows you to present the user with a single or series of decisions, this could be to either setup an application or change a single setting.
When our Auto-Loop card is selected in the Browse Fragment, we launch the GuidedStepActivity and in its onCreate method we use the addAsRoot() method to add a new instance of the AutoLoopStepFragment. The addAsRoot() method adds our fragment as the base fragment of the activity, meaning that there is no backstack entry added – causing the activity to finish when the back button is pressed.
Our AutoLoopStepFragment consists of several things:
Guidance
The guidance is the container on the left hand side of the screen, this is used to display the details such as:
- Title — The title to be shown for the guided step
- Description — The description to help guide the user
- Icon — The icon used to add visual representation to the guided step
These can be set by simply over-riding the onCreateGuidance method in your GuidedStepFragment:
https://gist.github.com/hitherejoe/cf97917df84e91ffe3ce
Actions
An action is an item which a user can select as choice for the guided step. Each action requires a:
- Id — An Id used to identify the selected action
- Title — The title to be displayed on the action
- Description — The description to be displayed on the action
- Check Set Id — The Id to identify a group of checkable items
In the screenshot above, you can see I have given my actions a checked state – this is to show the user their currently chosen auto-loop setting. To set an item as checked, we must call the setChecked() method on our GuidedAction once it has been built.
https://gist.github.com/hitherejoe/c206cb3836f6f9e59d17
Recommendations
The app uses a service to provide recommendations to the home screen of the Android TV system.
To do this we use two classes:
- RecommendationReceiver — Used to schedule a check for new recommendations
- UpdateRecommendationsService — Used to fetch and build new recommendations from the vine API
As you can see in the class above, we can use the recommendation builder to customise the look of the recommendation card and priority in which is should be displayed:
https://gist.github.com/hitherejoe/84442e103e4fd30a9dde
Displaying Errors
Other than displaying a toast to the user, we can show an ErrorFragment when an error occurs in our application. These allow us to show a simple message to the user and react accordingly when they dismiss the message.
It’s simple to create an error fragment:
https://gist.github.com/hitherejoe/9d95807b37a73c91866c
Testing the Application
Alongside Unit Tests, I wrote some Espresso tests to test the functionality of the application. Whilst this isn’t as simple as testing the functionality of an application on a mobile device, it is possible – so there is no excuse for skipping UI tests on your Android TV applications!
Let’s take a quick look at how you can test some of the components used in the Leanback library:
Main Activity Test
- The headers are contained in a VerticalGridView, which is a RecyclerView. The id of this view is browse_headers, so we can easily traverse this list using the RecyclerViewActions class. We can then check the content of the current item using a custom matcher.
Note: When the fragment is created, the first header item is already in focus so we don’t want to click on it.
https://gist.github.com/hitherejoe/fe629ad5acb2869a8cee
- We can open the Search Activity by using the Search Orb id to click on it, this way we can check the Search Activity launches and operates as expected.
https://gist.github.com/hitherejoe/89b922837b82e303669d
- Testing the Post card views wasn’t as nice as testing the categories. The cards aren’t in a RecyclerView, so we can’t use the RecyclerViewActions to help us out. Instead, we used our custom matcher again to click on each of the items.
Note: The id for the card view container is browse_container_dock, which you’ll need to use when writing your own tests.
https://gist.github.com/hitherejoe/8fc9d595ad45e06c1829
Post Grid Activity Test
Luckily for us the VerticalGridFragment class extends the BaseGridView class, which in turn extends the RecyclerView – which means we can use the RecyclerViewActions for our tests on the PostGridActivity!
- To check the Posts in our Grid, we just need to check each of the items is displayed. Using the id of the container (browse_grid) and the RecyclerViewActions class, we can click through each of the Post cards and check their content is displayed as expected.
https://gist.github.com/hitherejoe/06c24362ae432054fb78
- However, traversing the grid itself wasn’t as simple. We need to check through each of the items, this means an item must be in focus and then checked before we can move onto the next one. To do this I wrote a method that will start working it’s way through a row from left-to-right, when it detects it’s at the end it will go to the next row and work it’s way from right-to-left, and vice versa. Each time we reach checkItemAtPosition(), the code snippet above this is triggered.
https://gist.github.com/hitherejoe/e9696af7c95caca977ac
Search Activity Test
- We used a similar approach when testing the SearchActivity. However, this time we must reference the lb_results_frame container that holds the search results that we need to check for.
https://gist.github.com/hitherejoe/e9e026a70f5e8f0598c1
- We can input text into the Leanback search field by simply referencing the view by Id lb_search_text_editor
https://gist.github.com/hitherejoe/2b8e3121a14508d63360
Conclusion
Whilst I’ve only skimmed the surface of the approaches taken to create an application for Android TV, I hope it has provided some useful details that you can take away from reading this article.
To really understand how the Leanback library works and how you can create custom components to use in your applications, you really need to dig into the source code and read the documentation for the classes. There isn’t much available online in terms of how to build for Android TV, so the source code and documentation are going to be your best friends for this.
I’d love to hear about the applications you’re creating. If you wish to share anything or have any questions, feel free to drop me a tweet!
If you enjoyed this article, please hit the recommend button 🙂
Check out more of my projects at www.hitherejoe.com