IOT Voice Recognition System

 
Screen Shot 2019-05-14 at 11.11.55 PM-mi

Overview

This was a graduate project for our Real-Time Embedded System Lab. It utilized a TM4C microcontroller with an RTOS to obtain live sounds and transmit them to a server via WiFi. A python server will then interpret the audio file and use voice recognition to convert the audio file into characters which were sent back to the microcontroller to show on it's LCD display.

 

Experience

We have developed a system that allows low cost embedded systems to use the Internet of Things (IOT) to offload intensive tasks to another processor. Because microcontrollers such as the TM4C excel at collecting data and changing given outputs based on certain algorithms or calculations, we aim to capitalize on the strengths of the TM4C by using a real-time operating system (RTOS) to capture input data from the microphone while running other tasks at the same time. The use of IOT allows us to accomplish speech recognition on our embedded system within a reasonable amount of time.

We are using the Max4466 microphone which collects data through periodic ADC reads. We are using Texas Instruments TM4C microcontroller running the RTOS developed in EE 380-12. The TM4C is communicating via SPI to the Sparkfun ESP32 Thing, an IOT capable device running FreeRTOS. The audio data collected by the TM4C from the microphone is piped to the ESP32 and is stored in RAM buffers. The ESP32 uses its built-in WiFi adaptor to establish a connection with the Apple Airport Express Router. It then attempts to connect with our Python server using a UDP connection via an Ad-Hoc network. Once a successful connection has been established, the ESP32 will send the data buffers sequentially to the Python server. The Python server listens for multiple incoming connections. Once the Python server receives a certain number of packets from the ESP32, it stitches together the streams of data into a WAV file and analyzes it using a speech recognition API. The analyzed text is then sent back to the ESP32 via UDP and then to the TM4C through UART. This text is then displayed on the screen and can be used to interpret commands.

 

Demo of System

 
Screen Shot 2019-05-14 at 8.41.23 PM-min
Screen Shot 2019-05-14 at 8.04.38 PM-min
Screen Shot 2019-05-14 at 7.59.53 PM-min
Screen Shot 2019-05-14 at 1.24.33 PM-min
Screen Shot 2019-05-14 at 1.23.39 PM-min
Screen Shot 2019-05-14 at 1.23.29 PM-min
IMG_1087-min.jpg
photo_2019-05-14 21.39.03-min.jpeg
 
 

Meet the Team

 

Keeshan Patel

Electrical/Computer Engineering Student

22829156_1833939989968310_3550833692198947116_o.jpg

Rita Kambil

Electrical/Computer Engineering Student

19621284_1490445210999192_51576007799181