How to build a virtual assistant? – Part 2

In this post We will be going to set up the system for voice recognition and voice generation.

Before delving into the details, lets structure our work space. Lets create a folder structure that will hold our project, but first we need to give a name to our assistant. In my case I will call her Ayda ans I will give her a female voice and personality,

I have the following folder structure on my computer for this project:

 

Choosing an STT engine

An STT engine is basically a piece of software that transforms speech text.  We will use an existing engine into our application. There exists a large variety of STT engines:

• Pocketsphinx:  open-source speech decoder by the CMU Sphinx project.  Unfortunately, the recognition rate is not the best but  will be performed offline.
• Google STT: the speech-to-text system by Google. It is the same software a powers OK, Google. There is a limited number of request per day and needs an active internet connection.
• AT&T STT similar ro Google Speech, it also performs decoding online and thus needs an active internet connection.
• IBM STT : part of Watson and similar to google, you’ll also need an active internet connection.

 

I opted for Google STT as a backend for my speech to text component.

It turned out that getting this functionality is fairly simple once the mentioned libraries are installed.

create a file called stt.py. We create a function listen(). The role of this function is to listen to the speech, send it to google and return the result to the calling function.

 

import speech_recognition as sr


def listen():
    # obtain audio from the microphone
    r = sr.Recognizer()
    with sr.Microphone() as source:
        print("Say something!")
        audio = r.listen(source)
    text=None

    # recognize speech using Google Speech Recognition
    try:
        # for testing purposes, we're just using the default API key
        # to use another API key, use `r.recognize_google(audio, key="GOOGLE_SPEECH_RECOGNITION_API_KEY")`
        # instead of `r.recognize_google(audio)`

        text=  r.recognize_google(audio)
    except sr.UnknownValueError:
        print("Google Speech Recognition could not understand audio")
    except sr.RequestError as e:
        print("Could not request results from Google Speech Recognition service; {0}".format(e))

    return text

Text to speech

The second component  for our virtual assistant is the ability to talk. for this purpose we will use a TTS engine. There are different engine available for all the platforms. Fortunately in my case OSX comes with  this component already installed.

To test the say command, open the command line and enter the following command:

As speech is a way to affect the external world, this function will go inside Affectors folder. create a file named tts.py.

The following code will accomplish this task.

 

import os
import sys
def say(message):
 """
 This function takes a message as an argument and converts it to speech
 depending on the OS.
 """
 if sys.platform == 'darwin':
 return os.system('say ' + message)

 

 

 

Now lets modify the main function to integrate this new function.

 

from Sensors import stt
from Affectors import tts

def main():
 speech_text=stt.listen()
 print(speech_text)
 tts.tts(speech_text)

main()

This simple programme will simply repeat what you you say.

We surely need to improve our programme, but for now we have all the required functionalities to interact by voice with our assistant.