Chords, Conversations and the Kotlin Client Library

I recently launched a side-project I had been working on for the last several months, Chord Assist. Whilst this guitar is made up of many different moving parts, at the centre is the brain of the conversational tool. Whilst I made use of dialogflow to build this functionality, I needed to make use of some server-side component to handle the logic of the conversational tool that could not be handled inside of dialogflow itself (known as fulfilment). For this I decided to use the Kotlin Client Library for Actions on Google! Note: This project is completely open-source and can be…

Chord Assist: Building an accessible smart guitar for the mute, deaf and blind

In 2017 I created an Android Things project called BrailleBox — a Braille News Reader for the partially sighted. I’m really passionate about helping others and using the knowledge I have to do so, even if it means learning new things to help achieve that goal. This was the main driver for creating Braille Box, but I felt that I could go further and create something that could help others engage in areas that I myself find joy, that may not currently be so accessible (whether accessible at all, or just not quite affordable right now). The result of this project was…

Exploring Actions on Google Responses: Basic Card

When sending responses from our Actions on Google conversational tools, there are a number of different ways in which we can present content to our users. In this post we’re going to look at adding responses to our conversations using the Basic Card. When building responses for our conversation tools we have the ability to send what is known as a Basic Card — this a response that is supported only for surfaces that support visual (actions.capability.SCREEN_OUTPUT) output. A Basic Card will look something like this on screen: The important thing to note when using the Basic Card is that there must…

Exploring Actions on Google Responses: Simple Response

When sending responses from our Actions on Google conversational tools, there are a number of different ways in which we can present content to our users. In this post we’re going to look at adding responses to our conversations using the Simple Response. When building responses for our conversation tools we have the ability to send what is known as a Simple Response — this a response that is supported for both visual (actions.capability.SCREEN_OUTPUT) and audio (actions.capability.AUDIO_OUTPUT) output. Consisting solely of textual content, a visual response will look something like this on screen: Screenshot taken from the emulator The textual content which is…

Localising Actions on Google conversations

Actions on Google is a pretty fascinating platform when it comes to the creation of conversational tools. So far myself I’ve been able to create a couple of small tools that could have the potential to help users out in one way or another. But there’s one thing that’s always been present on my mind — they’ve been pretty restrictive when it comes to creating a personalised experience for users outside of my spoken language and/or region. However, if we were to localise our conversation tool it would not only help us to create a more personalised experience for our users based…

Authenticating users with Actions on Google

Actions on Google is a pretty fascinating platform when it comes to the creation of conversational tools. So far myself I’ve been able to create a couple of small tools that could have the potential to help users out in one way or another. But they always been missing that one important thing, personalisation. Having access to some form of identification for the user not only allows us to create a more personalised experience, but also opens up numerous possibilities for the kind of conversational tools that we can create. in this article I want to dive into Account Linking…

Exploring Dialogflow: Understanding Agent Interaction

Dialogflow is a powerful tool that allows us to create conversational tools without the complications of needing to handle natural language processing. But before we dive into the platform, it’s important to understand all of the different concepts that tie together to create the conversational agents that we can create. When I started exploring the platform I jumped in without knowing what was what — so in this article I want to quickly run through each of the concepts to help provide some foundational understanding for the platform. Just as you would say Hello to your friend before conversing with them, invoking…