EMG-BASED NEURAL NETWORK CLASSIFICATION OF MUSCLAR ACTIVITY AND VISUAL FEEDBACK

Performed by  :     Irene Sorianini, Federico Gala and Davide Massella

Supervised by :     Nicola Covre

From the development and thesis research of three master’s students we are creating a wearable device to monitor the muscular activity of the human trunk and upper limbs.

The device consists of three components:

  • the senrosized shirt;
  • the data acquisition and processing system;
  • the visual feedback system for the therapist.

1) SENSORIZED T-SHIRT AND SLEEVE

Thanks to the support of an experienced seamstress, the sensorized T-shirt features a reconfigurable allocation of dry electrodes via Velcro.

Sewing machine processing for the realization of the wearable interface
Velcro arrangement to allow for a reconfigurable electrode pattern

2) DATA ACQUISITION AND PROCESSING

The data acquisition and processing system is composed of the electrodes, the analog amplification system, the Delta Sigma analog digital conversion system and a raspberry Pi 4 unit.

 

THE ELECTRODES AND CONNECTION CABLES

Several types of simple dry electrodes were proposed and compared with state-of-the-art disposable electrodes.

Standard Adhesive Electrodes
Simple conductive stainless steel plate
Pair of conductive plates dipped in insulating silicone

Each electrode has been connected with the amplification system by means of shilded cables in order to reduce the 50 Hz noise perturbation.  Eight couples of electrodes has been considered and the analog amplification sistem provides to prepare the signal to the ADC conversion.

THE ANALOG AMPLIFICATION AND FILTERING

The Analog device amplifies with different reconfigurable stages. A first module with a variable gain of 20 and 50, then it follows a second, third and fourth amplification modules with a variable gain of 1, 2 and 5. Embedded in between the second the third amplification modules a sequence of analog filters is installed. The signal passes through a low pass filter  and a high pass filter with recofigurable cut off frequency, respectively  500Hz, 1KHz, 2KHz  and 5Hz, 10Hz, 20Hz.

Reconfigurable system for analog signal amplification and filtering

THE ACQUISITION SYSTEM

The Acquisition system consists of an ADS1298 RevB connected via SPI communication system to a Raspberry Pi 4.
The ADS1298 is an analog-to-digital conversion system from the Delta Sigma family and allows the acquisition of 8 channels up to 32KSPS.

Raspberry Pi 4
Texas Instruments ADS1298 RevB – DeltaSigma ADC 8channels, 32KSPS

Raspberry defines the ADC registers and the acquisition is managed by an interrapt on the GPIO port connected to the DRDY of the conversion board . Whenever a packet of 8 samples is ready it is transferred from ADS1298 and saved in raspberry memory.

DATA PROCESSING

The signal acquired and saved on Raspberry is processed to be used in parallel as a sample for classification by Machine Learnign and sent to the visual feedback system. The acquired signal is first interpolated to obtain a sequence with constant step in time. Given the presence of undesired spikes in the system, each channel is cleaned of outliers. Then, the signal is normalized with respect to the other channels by calculating mean and standard deviation.

Interpolation and windowing tests of the acquired signal – sequence of three biceps contractions with increasing effort
Monitoring with 4 channels (ch1, ch2, ch4) of a sequence of forearm contractions

DEEP NEURAL NETWORK AND LSTM FOR GESSTURE CLASSIFICATION

The signal thus arranged is ready to be classified by an LSTM neural network. The network was previously trained on a generated and tested dataset. As a first test, a limited group of classes was chosen to allow a training dataset to be quickly obtained and electrodes were placed right in the sleeve to monitor the arm muscles. A total of five actions were chosen:

  • ch_  Closed Hand
  • op_   Open Hand
  • i_  Index up
  • t_ Thumb up
  • 3_  Index, thumb and middle finger raised to dial the number 3
Confusion Matrix of the achieved results

THE VISUAL FEEDBACK SYSTEM FOR THE THERAPIST  (WORK IN PROGRESS)

In parallel to the classification system with neural networks, the signal will be sent via bluethoot to a processing unit that will reproduce a virtual model of the muscles activated by the patient. The goal is to repropose the information in augmented reality to the therapist via Hololens. Through RGB image processing techniques such as Open Pose or Open Vino, it will be possible to move the avatar following the actions of the subject.

Subject Avatar created with Blender
HoloLens 2