re you looking for a new way to look up programming concepts? If so, you could easily create an Amazon Alexa skill that’s equipped with the programming knowledge of your choice. This tutorial will teach you how to create one using a Twilio Autopilot Assistant and the Alexa channel.
Language: Python 3
APIs/Product(s): Twilio Autopilot, Amazon Alexa
Before we get started, you’ll need to understand the flow of the conversation between the Twilio Autopilot Assistant and the User. The User will start the Alexa skill and will be greeted by the Assistant. The User will then ask a question and in return will receive a response from the Assistant.
Since we’ll be using my AI Bot Monty for this tutorial, the prompts presented in the project files are dedicated to the Python language.
DISCLAIMER: In the event that tutorial files are not working properly for you, please refer to the Twilio Autopilot Python Quickstart documentation.
Sign Up for Twilio and Get a Phone Number
In order to work with Twilio’s products, you’ll need to sign up and purchase a voice enabled phone number. If you’re new to Twilio, you can start with a free trial. For further assistance on signing up and purchasing a phone number, visit Twilio.
Sign Up for Amazon Developer
To link the Twilio Autopilot Assistant with Alexa, you’ll need an Amazon Developer account.
Before getting started, you’ll want to ensure that you’ve downloaded Python 3 and the Twilio Python Helper Library. Follow the download and installation instructions available on each website to ensure that you’re all set and ready to go!
Download Project Files
Visit my GitHub repository to download the project files. In this tutorial, I will refer to each file based on how it is named in the GitHub repository. If you prefer to rename the project files, then I would suggest doing so only after the files are modified in the later steps.
Plan the Conversation
We’ll want to ensure that we’re able to ask the Assistant a variety of Python questions with their own unique responses. To help keep things organized, I would suggest grouping questions by topic and later adding to each topic once you’re satisfied with the Assistant’s output.
Within the GitHub repository download, open the assistant_tasks.xlsx file. We’ll use this template to plan out the conversation. This template is designed to help you organize each Python question by topic and plan out the creation of each task from start to finish.
To help you get started, I completed some of the template and provided an example of both a Greeting and Python Question task. You’ll need to fill in the yellow cells in the template. If you’re hoping to use this template for other topics, then consider brainstorming how you could break up your topics into different categories. Once you’ve figured that out, create tasks for each category.
Tasks consist of the action that the Assistant will take to communicate with the User. For the sake of this tutorial, I’ve named each task that you will create so that it can be referenced as we walk through the tutorial. However, you’re free to modify the names to better fit your project.
The Unique Name is the name of the task. This value should be unique for each task and should not be used more than once. Be sure to replace all spaces with an underscore.
Task Action Variable
Each task that we create for the Assistant will need a variable name. For ease of coding, I’ve kept the variable names in this column here pretty simple and similar to the name of the Task itself. However, you may change the variable names here if you would like.
File Name (Tasks)
To keep your project organized, you’ll want to create a naming convention that’s easy to remember and can be used consistently regardless of the amount of tasks that will be created. Think of a naming convention that works best for you and enter a file name for each of your tasks.
The Assistant decides which Task should be executed based on the training data that is used to comprehend the User’s response. The training data are known as Training Samples and consist of words or phrases that the User will say. With the exception of the Greeting task, you’ll want to create a minimum of 10 Training Samples that consist of ways in which the User could ask the Assistant a question.
Ex: Monty, Ask Monty, Start Monty, use Monty, Hey Monty
File Name (Samples)
Create a naming convention for your Training Samples and enter a file name for each training sample that’s respective of its task.
When we later open the code file for create_assistant.py, you’ll notice the action ‘Say’. Anything that is placed there is an indication of what the Assistant will say to the User. For this column, think of what you would like for the Assistant to say if the task is executed.
Note: The Assistant will recite the text verbatim. Therefore, the phrase ‘123 Sesame Street’ would be spoken as ‘One Hundred Twenty Three Sesame Street’. If you would like for the Assistant to say ‘One Two Three Sesame Street’, then you will need to type the Say action as ‘1 2 3 Sesame Street’ or ‘One Two Three Sesame Street’.
The Greeting is the first thing that the Assistant will say when the User starts the Alexa skill. You can say whatever you like here, however, I would suggest ending the greeting with a message that asks the User how the Assistant could be of assistance.
Ex: Hi, I’m Monty and I’m learning Python. My skills improve as I continue to learn. Ask me your Python questions to see if I can help you out.
In the event that the User provides a response and the Assistant is unable to comprehend or provide a response in return, the Assistant will use its Fallback response.
Ex: Sorry, I haven’t learned that yet. But maybe I couldn’t hear you clearly. Could you say that again?
Build Model Version
After we create Training Samples, we’ll need to build the training model for the Assistant. To keep track of each build, we’ll use a versioning system of ‘v.<version number>’
File Name (Build Model Version)
The file names for the build models should be consistent with the only difference being the version number appended to the end of the file name.
Once you complete the template, review to ensure that everything has been filled out properly. You’ll soon discover that completing the prep work for building the Assistant will come in handy once we start coding!
Create the Assistant
Before we begin creating Tasks, we’ll need to create the Assistant. In the create_assistant.py file, you'll find the template for creating the assistant. The code is also below. You'll need to replace the following values with your own:
- Replace account_sid placeholder with the Account Sid from your Twilio dashboard.
- Replace auth_token placeholder with the Auth Token from your Twilio dashboard.
- Think of a name for your Assistant. Replace the friendly_name placeholder with the name of your assistant - spaces are okay here! Once you’ve done that, you’ll need to replace the unique_name placeholder with the name of your assistant as well. However, if there are any spaces in your Assistant’s name, replace the space with an underscore.
Save the file and execute the Python code. If all works out well, the terminal should print the assistant_sid. The assistant_sid is unique to your newly created Assistant.
On the last line of the create_assistant.py file, you’ll find a commented line for assistant_sid. The assistant_sid will be used in all of the remaining files from the GitHub repository. To help keep track of the assistant_sid, copy the assistant_sid from the terminal and paste onto the commented line.
Set Default Tasks
We’ll first want to set the Default Tasks for the Assistant. The Greeting task and Fallback tasks will be the defaults for the Assistant. In the default_tasks.py file, you'll find the template for creating the assistant. Replace the placeholders for account_sid, auth_token, assistants with your personal values.
Note: assistants is your assisant_sid
Refer to the assistants_task.xls template to find the name Unique Name of the Greeting task and Fallback task. Within the defaults dictionary, the value for the assistant_initiation key will be the Unique Name for the Greeting task. The value for the fallback key will be the Unique Name for the Fallback task. Replace the placeholders for each respective value.
Execute the Python file. If the task is created successfully, the terminal will output the default_assistant_sid.
Create Assistant Tasks
Creating the Tasks for the Assistant becomes repetitive and happens relatively quickly once you get the hang of it. You’ll follow the same workflow below for creating each task:
Create Task > Create Training Samples > Build Model
We’ll first create the Greeting task and test out the Assistant to ensure that all functions properly.
In the create_task.py file, you'll find the template for creating a task. Replace the placeholders for account_sid, auth_token, and client.autopilot.assistants (your assistant_sid) with your personal values. You'll then append the Task Action Variable for the task to the task_actions variable.
Within actions, you’ll find a list of actions for the Assistant to execute. Starting with the say action, replace the placeholder text with the Say prompt from the template. Be sure to use an escape character if you’re using an apostrophe in your prompt.
Since we want our Assistant to listen for a response from the User, we’ll leave the listen action set to True. If we did not want the Assistant to listen for a response and instead end the conversation, we would set the listen action to False.
Within the task variable, you will find variables for unique_name and actions. Replace the unique_name placeholder with the Unique Name of your task. Afterwards, replace the actions placeholder with the name of the task_actions variable from the prior step.
In the code editor, select Save As and name the file based on what you entered in the template. Run the Python file. If the task is created successfully, the terminal will output “The task has been created!” in addition to the task_sid.
Create Training Samples
In the create_training_samples.py file, you'll find the template for creating training samples. The code is also available below. Replace the placeholders for account_sid, auth_token, assistants with your personal values.
In the phrases list, you’ll want to replace the placeholder samples with the ones you entered into the template for the task. Be sure to use an escape key if any of your phrases contain an apostrophe (i.e “I’m” would be “I\’m”). Also, Replace the .tasks placeholder with the Unique Name of the task.
In the code editor, select Save As and name the file based on what you entered in the template. Run the Python file. If the task is created successfully, the terminal will output the sample_sid for each sample that was created.
In the build_training_model.py file, you'll find the template for building the model. The code is also below. Replace the placeholders for account_sid, auth_token, assistants with your personal values. You'll also need to replace the unique-name placeholder with the Build Model Version in the template.
In the code editor, select Save As and name the file based on what you entered in the template. Run the Python file. If the task is created successfully, the terminal will output the model_build_sid.
Connect Assistant to Twilio Phone Number
Although we’ll be linking the Assistant to Alexa, I’ve found it easiest to test the initial task setup by linking the Assistant to your Twilio phone number. In order to link the Assistant to your Twilio phone number, you’ll need to do a bit of configuration in your Twilio console.
1. Go to your Twilio dashboard.
2. Select the phone number that you’d like to use for your Assistant.
3. Scroll down to the Voice & Fax Configuration section.
4. Next to A Call Comes In select Webhook and enter the following into the field:
5. Replace the placeholders for <ACCOUNT_SID> and <ASSISTANT_SID> with your personal values.
6. Save the updated settings.
Now that the Assistant is equipped to execute the Greeting task, we’ll want to call the Twilio number to test. When you dial your Twilio phone number, you should be greeted by the Assistant with the prompt for the Greeting task.
If the test is a success, continue the same process above for adding the rest of the tasks. I highly suggest testing out the Assistant each time you finish creating a task. If you find that your task prompts or samples need to be modified, you can do so from within the Twilio Console.
If you’d rather skip out on the phone call and test in the browser, head over to the Twilio Console and click on your Assistant. Use the Simulator to test your Assistant with both text and voice.
Test Assistant with a Query
To ensure that your Assistant routes a phrase or word to the appropriate task, you can leverage using Twilio Autopilot’s Query API resource endpoint. You can learn more about using queries by reading the documentation here.
If you’d rather create queries through the Twilio console, then you can get started here.
Link Assistant to Alexa
In order to link the Assistant to Alexa, you’ll need to open both the Amazon Developer Console and Twilio Autopilot Console. Since Twilio Autopilot is still in Beta, please refer to the documentation available on Twilio’s website for instruction on how to link the two.
Test with Alexa
Once you’ve linked the Assistant with Alexa, you can navigate to the Test tab in the Amazon Developer console and interact with the Assistant by typing or saying “ask <your skill name> <something something>”.
And that’s it! If you happen to get stuck, feel free to take a look at the files within the Sample_Files folder from the GitHub repository download.