Wednesday, October 2, 2013

Step by step guide - Beagle Bone Black voice recognition on an LED matrix

So after I made a BeagleBone black output voice recognition to an LED Matrix, I thought I will post a "How-to" guide to make one yourself.

NOTE: If at any point you find yourself confused with a technical term, try to google it, if you are still having trouble, or you would like to make things more clear about this post, add a comment and I will follow up on it :)

First, Materials, Make sure you have all of these when you start:
1. BeagleBone Black
2. USB Audio card  - I narrowed it down for you (1$-3$)
3. Microphone - the smaller the better
4. MicroSD card 8GB
5. AdaFruit I2C LED Backpack - I used green.
6. Male - Female Jumper wire - You will need at least 4 wires
7. Internet connection - I used a LAN cable to keep it simple.

Step 1: Setting up the Angstrom distro on the Beagle
Use this tutorial to create an Angstrom image on the MicroSD card http://learn.adafruit.com/beaglebone-black-installing-operating-systems/windows
Next, plug the card into the Beaglebone and Boot to the new OS, make sure you have the USB cable connected.
Once all this is done, open up your com port and login as root. once you get a shell proceed to the next step.

Step 2: Setting up pocket sphinx 
Plug in the LAN cable to your BeagleBone, if you have WiFi good for you, just use a hub to have the USB audio enabled later.
Follow these steps:

  1. run opkg update
  2. opkg install python-distutils
  3. From here (https://groups.google.com/forum/#!topic/beagleboard/aBznzq_bNuU) follow these steps:
    1. Install libasound2, alsa-dev, alsa-lib-dev (http://www.angstrom-distribution.org/repo/) before compiling sphinxbase and pocketsphinx.
    2. Get sphinxbase-0.8, and pocketsphinx-0.8 from http://cmusphinx.sourceforge.net/ and extract the tar files.           http://downloads.sourceforge.net/project/cmusphinx/sphinxbase/0.8/sphinxbase-0.8.tar.gz       http://downloads.sourceforge.net/project/cmusphinx/pocketsphinx/0.8/pocketsphinx-0.8.tar.gz
    3. Change directory to the sphinxbase directory.
    4. Run "./autogen.sh" to generate the configure file.
    5. If needed run "./configure" 
    6. Run "make"
    7. Run "make install"
    8. Change directory to pocketsphinx directory and redo steps 4, 5, 6, 7.
  4. To check if the pocket sphinx installation worked power off the BeagleBone, make sure the USB audio card is plugged in and power on.
  5. run this command to do a quick test:
    "pocketsphinx_continuous -adcdev hw:1,0 -nfft 2048 -samprate 48000 2>/dev/null"
            note: hw:1,0 is the hw node of your usb audio.
Step 3: Setting up the environment for the AdaFruit LED Backpack
To setup the adaFruit libs and files
1. run "opkg install python-pip python-setuptools python-smbus"
2. run "pip install Adafruit_BBIO"

Step 4: Connect the LED Backpack to the BeagleBone
Switch P8 and P9, the person who created this image mixed them up. 
We will only use the left side header (P8 in this photo)

Connect the LED Backpack like this:
LED Backpack VCC --> PIN 7 SYS_5V
LED Backpack GND --> PIN 1 DGND
LED Backpack SDA --> PIN 20 I2C2_SDA
LED Backpack SCL  --> PIN 19 I2C2_SCL















Step 5: Setting up the code
Copy these files over to your BeagleBone, I put them on my ~/Desktop/ but you can choose wherever.
  1. Adafruit_LEDBackpack.py - From AdaFruit RaspberryPi example code
    https://docs.google.com/file/d/0B4cQqo_E3D-iS19rcnR1ZnpYVDg/edit?usp=sharing
  2. Adafruit_I2C.py - From AdaFruit RaspberryPi example code, Modified to use I2C 1 on the BeagleBone.
    https://docs.google.com/file/d/0B4cQqo_E3D-iUTh0VFZidDBXSVk/edit?usp=sharing
  3. Adafruit_8x8.py - From AdaFruit RaspberryPi example code.
    https://docs.google.com/file/d/0B4cQqo_E3D-idnRDY2ZybU5mdUU/edit?usp=sharing
  4. runVisibleVoice.py - My code to run PocketSphinx and to display the output on the LED Backpack
    https://docs.google.com/file/d/0B4cQqo_E3D-iVFdwUlVNa0lLTzQ/edit?usp=sharing
Once copied, simply run "python runVisibleVoice.py" and watch the magic :)

Feel free to modify my code in any way you want, but do not use it to make money. 
Share it, Don't sell it.

Here is a picture of my final version:


















29 comments:

  1. Hi, this is a very interesting proyect and im thinking of reproducing some of it (the voice recognition part at least), the thing is that i have a beagleboard-xM do you think it would work on this card??

    ReplyDelete
    Replies
    1. Try to replicate my environment (Angstrom, USB Audio, etc) as much as you can and I think you might actually have fun trying to get it to work :)

      Delete
    2. Currently I have Angstrom running on my beagleboard, so I just have to search for an USB Audio adapter, but I think it might actually work with the "stereo audio out/in" that the xM has incorporated.
      I'll let you know if it works!

      Thank you :]

      Delete
    3. In that case remember to check your audio HW node,
      "hw:1,0" is the HW.
      "pocketsphinx_continuous -adcdev hw:1,0 -nfft 2048 -samprate 48000 2>/dev/null"

      Delete
    4. Hello Ale,
      did you have any success with voice recognition on the beagleboard-xM? I am interested to know if you did as I have one also. I also have a BBB but think the audio capability would nake it easier to use that board.

      Thanks

      Mark

      Delete
  2. Hi, I am doing a similar project on my beaglebone black with voice recognition and I followed your instructions. But, while building sphinxbase0.8 I get the following error:

    gcc: error: -I/usr/include/python2.7: No such file or directory

    Apparently the include path to python that is passed to gcc is passed inside double quotes like the following:

    gcc "-I/usr/include/python2.7"

    I don't know if that is the cause, but that seems to be one of the candidates. So, I ended up building sphinxbase using --without-python. Can you please help me out on this or suggest some fix? I would really appreciate it.

    Thanks

    ReplyDelete
    Replies
    1. I was not successful in getting pocket sphinx python part to work either, i simply used a python script to wrap around the output from the pocket sphinx executable.
      I suggest you forget about the python for the moment, get the thing running and once you are sure that it works, you can make it better by working on solving the gcc problem.
      Sorry I couldn't be much help on this... good luck though! :)

      Delete
    2. No problem. Thanks a lot. Really appreciate it. The problem that I am facing right now is that when I run pocketsphinx_continuous it throws the following error:
      Error opening audio device default for capture: No such file or directory

      I have used a USB sound adapter and connected a microphone to it, but don't know why it's not detecting that. The output from arecord -L is

      arecord -L
      null
      Discard all samples (playback) or generate zero samples (capture)
      default:CARD=Device
      USB PnP Sound Device, USB Audio
      Default Audio Device
      sysdefault:CARD=Device
      USB PnP Sound Device, USB Audio
      Default Audio Device


      So, I tried running pocketsphinx_continuous -adcdev Device. But, no success.
      Planning to buy the beaglebone audio cape. Do you think that could solve this problem?

      Thanks

      Delete
    3. Make sure you have the USB audio card plugged in before you boot up the device, after that do an lsusb and see if it is recognised.

      I also commented before to someone else:

      remember to check your audio HW node,
      "hw:1,0" is the HW.
      "pocketsphinx_continuous -adcdev hw:1,0 -nfft 2048 -samprate 48000 2>/dev/null"

      try playing with the hw:1,0 flag, it might help

      Delete
  3. I've been having some problems following this blog.
    After following the listed instructions, I ran #lsusb, found audio card:
    Bus 001 Device 005: ID 0d8c:013c C-Media Electronics, Inc. CM108 Audio Controller
    Next, I ran ~# pocketsphinx_continuous -adcdev hw:1,5 -nfft 2048 -samprate 48000
    The tail of the output is as follows:
    INFO: ngram_search_fwdtree.c(186): Creating search tree
    INFO: ngram_search_fwdtree.c(191): before: 0 root, 0 non-root channels, 60 single-phone words
    INFO: ngram_search_fwdtree.c(326): after: max nonroot chan increased to 13428
    INFO: ngram_search_fwdtree.c(338): after: 457 root, 13300 non-root channels, 26 single-phone words
    INFO: ngram_search_fwdflat.c(156): fwdflat: min_ef_width = 4, max_sf_win = 25
    INFO: continuous.c(371): pocketsphinx_continuous COMPILED ON: Mar 10 2014, AT: 14:49:03

    Error opening audio device hw:1,5 for capture: No such file or directory
    FATAL_ERROR: "continuous.c", line 246: Failed to open audio device
    root@BBB_1813_20130725 Sun Mar 16 10:26:01 ~#
    Any suggestions

    ReplyDelete
    Replies
    1. bus 1 device 5? is that the only USB device plugged in?
      if not, try with only the USB audio dongle plugged in and reboot.
      if it is the only device plugged in, try to power up with the usb audio plugged in.
      if you have already done all of the above, try a different usb audio dongle.

      other than that, I am sorry but I have no ideas why it won't work...
      But do let me know if anything i suggested applies or not.
      good luck!

      Delete
    2. I did it again today with the same result. I unplugged video camera and typed:
      ~# lsusb
      Bus 001 Device 002: ID 1a40:0101 Terminus Technology Inc. 4-Port HUB
      Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
      Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
      Bus 001 Device 003: ID 1a40:0101 Terminus Technology Inc. 4-Port HUB
      Bus 001 Device 004: ID 1a40:0101 Terminus Technology Inc. 4-Port HUB
      Bus 001 Device 006: ID 0d8c:013c C-Media Electronics, Inc. CM108 Audio Controller
      Note: bus 001 device 006 today is audio device.
      Then I ran
      # pocketsphinx_continuous -adcdev hw:1,6 -nfft 2048 -samprate 48000
      The last 6 lines of output are as follows:
      INFO: ngram_search_fwdtree.c(338): after: 457 root, 13300 non-root channels, 26 single-phone words
      INFO: ngram_search_fwdflat.c(156): fwdflat: min_ef_width = 4, max_sf_win = 25
      INFO: continuous.c(371): pocketsphinx_continuous COMPILED ON: Mar 10 2014, AT: 14:49:03

      ALSA lib pcm_hw.c:1401:(_snd_pcm_hw_open) Invalid value for card
      Error opening audio device hw:1,6 for capture: No such file or directory
      FATAL_ERROR: "continuous.c", line 246: Failed to open audio device

      I tried another audio soundcard, but it wasn't recognized. I ordered one of your suggested soundcards today. It will arrive in about a week. In the meantime, if you can think of anything else to try, I would appreciate it. I tried it today without vidcam. Same result.


      Delete
    3. Honestly I have no idea...
      the only diff I can see is that you are using a usb camera. I guess we will have to wait the week and see then.

      Delete
  4. Hi,
    For some reason, when I try to run autogen for sphinxbase, it can't seem to find python. It's definitely there, and it's in the default location.
    The error from autogen:
    (Crucial bit): gcc: error: -I/usr/include/python2.7: No such file or directory

    When running opkg install python-distutils: Package python-distutils (2.7.3-r0.3.7) installed in root is up to date.

    I'd really appreciate your thoughts, if you have a minute. Thanks :)

    ReplyDelete
    Replies
    1. I am sorry, I have no idea, I did this for a week and went back to doing my regular things. My guess is that your problem is more generic with the image you use on the beagle, try a fresh one on a new SD card.
      If you do happen to figure it out, please post the how-to here, someone else might have the same issue someday.

      Delete
    2. Run "./authgen.sh --without-python" and it should work

      Delete
  5. I'm going to try and do this with my Octoscroller cape / LEDscape 5 LED matrices setup I have. I'll be running it with Debian also. I'd love some help/advice if you're available.

    ReplyDelete
    Replies
    1. I think because I am far far away and don't have the same HW as you, I might be able to only help with ideas or suggest fixes for problems, if you can use that kind of help, I'd be happy to do my best and help :)

      Delete
  6. Hi,is this beaglebone alone is capable of recognizing voice or do I need a speech synthesizing softwre installed on pc to process speech and send back instruction to beaglebone?

    ReplyDelete
    Replies
    1. When you say "standalone hack", you mean after installing pocketsphinx to the beaglebone, it will not need a personal computer to operate? its recognition will still work without the help of a personal computer? Sorry for noob question.

      Delete
    2. When i say "standalone" i mean, no internet connection is required, it will perform the voice recog offline. alos i will run off a battery pack so no wall or computer is needed.
      thats a good question :)

      Delete
  7. Is it possible to follow same procedure of pocketsphinx onto the image provided on the Beaglebone Black board, without use of Micro SD Card

    ReplyDelete
  8. Hi i have tried everything here and it worked for me, but i have a problem i programmed an application to activate a digital output under some command but at the first run it recognize the command but only activates the digital output only for one cycle, i do not know why, i think is something with the subprocess, do you have any clue?

    ReplyDelete
  9. Thank you so much for sharing! But could i ask you a question? how i can apply this source for my country language( not english)? Is this relate to Pocket-shpinx and where i can edit to do it?

    Regards!

    ReplyDelete
  10. Nice post, Thanks for the post. Speech Recognition In Windows are likely used in system which used to convert voice to text which is really helpful.

    ReplyDelete
  11. Thank you very much for this useful article. I like it. video voice to text

    ReplyDelete
  12. You make so many great points here that I read your article a couple of times. Your views are in accordance with my own for the most part. This is great content for your readers. voice over

    ReplyDelete