Any time we want to work with a library we have to actually integrate it into our project. For p5.speech this will mean either downloading it or using a Content Distribution Network (CDN) link, and then writing enough code to check if it’s working.
We’re going to actually work with the library now, so we’ll need a p5 project to make that happen. Luckily we have the good old template p5 project ready to go for basic p5-ing.
p5.speech-experimentsexperiments subfolder?)p5.speech-experiments folder in your text editor (presumably VS Code)Now we’re ready to go.
When we start working on a new project we don’t magically have the speech library installed already, so we have to get used to an extra step where actually get hold of the library. This will almost always be via the library’s website so go to the p5.speech website.
There are two options available to us here. We can download the library or we can use a CDN. Both are fine, but let’s discuss briefly.
We can download the library as a file called p5.speech.js. The advantage here is that we will have the library on our computer as part of the project, so we don’t need to be connected to the internet to use it. Useful if you’re totally offline at the cottage working on JavaScript for some reason.
You might as well download p5.speech just for practice, so do that. You may end up with the library’s code displayed in your browser window. If that happens then save the file from your browser by going to File > Save.
A CDN is a Content Distribution Network. It’s a way of storing popular (or even unpopular!) libraries online so that you don’t have to download them to use them. Instead of downloading a file, you’ll usually just be given a script tag that you can include directly in your project. Something like
<script src="https://cdn.jsdelivr.net/gh/IDMNYU/p5.js-speech@0.0.3/lib/p5.speech.js"></script>
You can’t run a project that uses a CDN unless you’re online because it’s going to use the internet to get the library when the project runs. On the other hand, it means you don’t have to download a file and (if you care) it can take advantage of browser caching to avoid users downloading the same library multiple times across different projects. (You don’t have to care.)
Find the script tag for the CDN version of p5.speech and copy it.
Now we can make our choice. File or CDN? Totally up to you.
First, copy the p5.speech.js file you downloaded into your project’s js/libraries folder. That’s a sensible place for it!
Second, add a script tag to your index.html that tells your project where to find the library, like this:
<script src="js/libraries/p5.speech.js"></script>
It’s just like the existing script tags for p5.min.js and p5.sound.js. Put your new script tag after those two, since it’s kind of a sublibrary of p5.js.
Remember that CDN script tag you copied? Just paste it into your index.html after your p5.min.js and p5.sound.js script tags. That’s it.
Now that we’ve included the library, we have to make sure it actually works. That means we should try to write the simplest project that would prove it. We can leverage the examples we were looking at to figure this out for both synthesis and recognition.
For voice synthesis, there are only two important things, as discussed:
let voice = new p5.Speech()
voice.speak(`She sells seashells by the seashore.`);
So, all we really need is a program like this:
// Create the speech synthesis object and store it in "voice"
let voice = new p5.Speech();
function setup() {
}
function mousePressed() {
// Tell the speech synthesis object to say something when the mouse is pressed
voice.speak(`She sells sea shells on the seashore.`);
}
Note: I’ve put the speak() part into mousePressed() in part because it has the same “problem” as any audio: you can’t just play audio as soon as a webpage loads, there has to be some user input first (like a click).
If you run your program now you should be able to click and hear your computer say that sentence. Did it work?
Move on to testing the speech recognition.
That sucks, I’m sorry. There are a couple of major things to look at here.
If you see an error like
Uncaught TypeError: p5.Speech is not a constructor
It means that your project doesn’t see the library. So it thinks there’s no such thing as p5.Speech. If that happened, then make sure you look at things like typoes in your script tags (including capitalization and folder paths), or whether you forgot to put the p5.speech.js file in the right place if you were using that.
If you see an error like
Uncaught TypeError: voice.say is not a function
It means the program doesn’t think the function you’re using exists. In this case it’s because I used say instead of speak by mistake. But you’d get this with a typo too. Make sure that the way you’re writing the function name is exactly the same as in the documentation and/or the examples.
Ask for help.
Now it’s time to test out the voice recognition part of things. We’ll work from the “simple speech recognition” just because that’s a bit… simpler.
In this case, the simple example is much closer to the absolute basics which is nice. We’ll whittle away a few of the graphical flourishes to include only the crucial steps in voice recognition, which are:
// 1. Create a speech recognition object
let speechRecognizer = new p5.SpeechRec();
// Called at the start of the program
function setup() {
// 2. Tell the program the function to call if the recognizer hears something
speechRecognizer.onResult = handleResult;
// 3. Tell the recognizer to start listening
speechRecognizer.start();
}
// Called when the recognizer has heard something
function handleResult() {
// 4. Check if there is definitely a result
if (speechRecognizer.resultValue === true) {
// 5. If there is, print it out in the console
console.log(speechRecognizer.resultString);
// (e.g. speechRecognizer.resultString has what was said inside it)
}
}
(I’ve changed some variable names and function names to what I feel are better and more sensible names.)
You can see there are more steps here that we have to think about, including the idea that handleResult() is only called when speech has been heard. It’s the same idea as mousePressed() only being called when the mouse is pressed down.
We have to create a speech recognition object, tell it what function to call when it hears something, explicitly start it, and then do something when it hears speech.
If you run this program, you should be able to say something (once you see that it’s listening) and see your words displayed in the console. If you don’t have the console open while you are programming, then shame on you! Open it.
Did it work?
Great, time to move on to what’s next
Sorry to hear that (get it?!). Thankfully, because our program is so simple, it’s likely to be one of the exact same problems as before, so:
One special typo you could make that won’t cause an explicit error is a mistake in the conditional, e.g.:
// resaltValue ???
if (speechRecognizer.resaltValue === true) {
console.log(speechRecognizer.resultString);
}
This is a good reminder that even if there’s no error, it’s a good idea to proofread your code sometimes to make sure there aren’t secret typoes.
If you cannot find the problem, guess what… ask for help!
The library works. It says words and it hears words. We’ve seen the bare essentials of what it does, but we haven’t yet tried out any of the extras and we haven’t played with the possibilities. It’s time for that.
So let’s go ahead and get into some voice output experiments!
}