Neural Networks: Body Movement Classification by CNN Abstraction of LSTM Architecture
By (Justin) Hyobin You
Overview
Surrogate robotics requires an interface between human body movement and robotic interpretation with high accuracy and speed. In search of a highly responsive interface, I have developed a method to improve existing neural networks. Not yet tested to completion, the abstraction layer by convolutional neural network superimposed on the layered, Long Short-Term Memory (LSTM) classifier network may provide a method for accuracy improvement without a significant loss in reactivity (during classification, as well as training). The abstraction allows the same generic movements (i.e. walking up the stairs) to become quickly learned and classified with accuracy, the network learning to adapt to a specific user and subject. Preliminary results with elementary data have shown improved accuracy in body movement classification.
Using primitive data measurements via two instruments on a phone (accelerometer and gyroscope), six distinct body movements were classified with high accuracy. Beginning with two LSTM neural network layers, the accuracy was 90% following 300 iterations over the data set (70% training, 30% validation). With the addition of a small Convolutional Neural Network (CNN) above these layers, the accuracy was increased to 94% following the exact same training parameters (epoch-limited, with the same training optimization techniques). There was no noticeable increase in time with a single four-core CPU.
Future applications of this research include applying abstraction of neural networks in Body Machine Interfaces (BMI), thereby providing a mechanism for quick adaptation and specialized training and learning for a specific individual. This is especially significant, given the brain is polymorphic: electroencephalogram (EEG) data or more specifically for medical application, deep electrode recordings, collected for two individuals completing the same physical task is significantly different even over just the standardized motor cortex. This is especially true for those born without limbs or those who have lost limbs. Due to the brain's neuroplasticity, motor cortex areas form and reform, respectively, such that body movement and the corresponding subsection of the cortex cannot be easily predicted or generalized. To overcome the issue in the medical context, such a neural network with applied abstraction can shorten the difficult training period and arduous physical therapy, decreasing costs for the life-enhancing procedure while delivering a more accurate physical exhibition of mental intent.
Problem Statement
Develop a method of increasing accuracy of BMIs. Investigate the effect of abstraction and applying deep learning to classifier networks.
Experimental Design and Approach
There are constantly new methods of classification using neural networks. In the busy atmosphere of robotics and classifications of body movements, there are constant improvements to be made. To begin my study, I adapted to the use of a double-layered LSTM neural network for classifying six different activities.
To attend to the matter of the abstraction layer, I decided to approach with a convolutional neural network. Very applicable in terms of the of body movements (the final goal of these accuracy optimizations), CNN networks are essential in grammatical, syntactical evaluations. Furthermore, it the architecture itself, computer vision has been incredibly succesful with the use of CNN, most evident with the well known LeNet.
Results and Conclusions
The original LSTM neural network was run with 300 epochs over the UCI dataset. It was able to arrive at about 90% accuracy consistently.
To study the addition of the CNN layer, the whole network was initially set up as independent trainings. The adapted CNN-LSTM neural network was run by first training the base layer at LSTM over the data set. After somewhat satisfactory neural trajectories had been established, the complete network was run to train the CNN layer by freezing the weights in th LSTM network. This gave the possibility for the CNN layer to act as an adaptive pre-processing step, as well as the abstraction. However, due to the nature of the minimal data set, and perhaps the class of activities that were to be differentiated, there were high amounts of overlap between data that implicitly results in low accuracies. Due to this, the neural network often diverged. Prior to being entrapped by some inseparable data, however, the training was smoother. However, the network consistently diverged, and training could not be completed.
In resolution, the adapted CNN-LSTM neural network was run with identical parameters where synaptic strength were allowed to vary for every neuron-to-neuron connection. The whole network was trained synchronously. With this, the network consistently arrived 94% accuracy. The CNN neural network is relatively small, in comparison with the base network as well as neural network in general. With the slight adjustment, there was a significant increase in accuracy with very little timing delay.
Due to the time limitations, I was unable to attempt abstraction to other classification neural networks. I wish to show a similar improvements through the addition of the CNN abstraction layer for high accuracy multiclass classifier networks that have recently been published. As with my neuroscience and medical interests, my original trajectory was working with EEG data. Due to finding difficulties in obtaining the data set as necessary and the length of time to preprocess the individual channel signals led me to pursue this more immediate movement classification. However, I hope to show that similar results can be observed with EEG data input.
Visuals (training improvements and confusion matrices) in the original presentation are linked at the bottom of this page.
Code Directory
All of my code for this project can be found at the following GitHub repository: https://github.com/justinhyou/movement-classification-via-CNN-LSTM
Presentation
This is the link to my project presentation: https://goo.gl/6GVjas
For questions, please contact (Justin) Hyobin You at hy002014@mymail.pomona.edu.
Special thank you to Harvey Mudd College and Pomona College, as well as Professor Robert M. Keller.