IOT Voice Recognition System

Experience

​

We have developed a system that allows low cost embedded systems to use the Internet of Things (IOT) to offload intensive tasks to another processor. Because microcontrollers such as the TM4C excel at collecting data and changing given outputs based on certain algorithms or calculations, we aim to capitalize on the strengths of the TM4C by using a real-time operating system (RTOS) to capture input data from the microphone while running other tasks at the same time. The use of IOT allows us to accomplish speech recognition on our embedded system within a reasonable amount of time.

​

We are using the Max4466 microphone which collects data through periodic ADC reads. We are using Texas Instruments TM4C microcontroller running the RTOS developed in EE 380-12. The TM4C is communicating via SPI to the Sparkfun ESP32 Thing, an IOT capable device running FreeRTOS. The audio data collected by the TM4C from the microphone is piped to the ESP32 and is stored in RAM buffers. The ESP32 uses its built-in WiFi adaptor to establish a connection with the Apple Airport Express Router. It then attempts to connect with our Python server using a UDP connection via an Ad-Hoc network. Once a successful connection has been established, the ESP32 will send the data buffers sequentially to the Python server. The Python server listens for multiple incoming connections. Once the Python server receives a certain number of packets from the ESP32, it stitches together the streams of data into a WAV file and analyzes it using a speech recognition API. The analyzed text is then sent back to the ESP32 via UDP and then to the TM4C through UART. This text is then displayed on the screen and can be used to interpret commands.

​