Question Need to do the peer review of wiki article attached in 300-450 words Instructions: Answer all the questions following the instructions. Strictly do not use AI for solving questions. The solution should be free of Plagiarism. Solution to be formatted in APA and use appropriate references with in-text citations in APA./n1/23/24, 9:18 AM Application of Kalman Filter in Neural Signal Decoding . 180D-FW-2023/Knowledge-Base-Wiki Wiki 180D-FW-2023 / Knowledge-Base-Wiki <> Code Issues !* ) Pull requests Actions Projects Wiki Security Application of Kalman Filter in Neural Signal Decoding Edit New page Hongchang Kuang edited this page 2 weeks ago · 6 revisions Jump to bottom Application of Kalman Filter in Neural Signal Decoding Hongchang Kuang Word count: 2610 (including Latex formula and HTML code) Introduction Have you imagined a world that we could control things without moving our fingers? Will you fancy playing video games or driving your cars just with your brain? I have to tell you that this fantasy is no longer impossile with the help of Brain Machine Interface (BMI). In recent decades, as technology advancing and social ethics system maturing, a heated debate has been triggered over the usage of BMI. Some argues that current innovations are still not applicable for widely marketing products. However, most engineers and neuroscientists are excited by technological progress that facilitates brain mapping, the most sanguine of them comparing their growing ability to tremendous advances that led to the unimaginable success of the BMI projects. Companies like NeuralLink and Paradromics, as well as government agency have invested on the projects to test the possibilities and work on the real world applications [1]. This articles briefly introduces one of the prevailing technology in BMI: the application of Kalman filter in neural signal decoding, discussing its advantages and potential improvements, and what makes it robust and powerful that could possibly allows BMI to enhance our daily life. https://github.com/180D-FW-2023/Knowledge-Base-Wiki/wiki/Application-of-Kalman-Filter-in-Neural-Signal-Decoding 1/15 Q 1/23/24, 9:18 AM Application of Kalman Filter in Neural Signal Decoding . 180D-FW-2023/Knowledge-Base-Wiki Wiki Backgrounds in Neuroscience Before we get to the real technique, a simple background in neuroscience is neccessary to unravel you from the potential confusion in the world of jargons, since the mind, with our brain in general, is our main subject of research. The main study theme of neuroscience is to link molecules to mind. Human brain is a highly integrated network of approximately 100 billion individual nerve cells. Therefore, to fully understand the mechanism behind the BMI devices, a brief understanding of how neurons are organized into signaling pathways and how they communicated is necessary. We will introduce how brain functions from three different levels: brain, neuron, and neural signal (action potential). Further readings in related areas are also encouraged. The Brain and Behavior What does our brain look like? Surprisingly, it is complex and ordered at the same time. The central nervous system (CNS) is bilateral and symmetrical. Studies on brain with modern imaging techniques discovers that different regions on brain are specialized for different functions. As you may have gussed, for BMI to decode human cognitive information, the most important part that we would like to focus on is the cerebral cortex, where brain operations that are responsible for human cognitive abilities occur. As shown in the graph, It consists of four anatomically distinct lobes with different functionalities, including planning future actions, hearing, learning, vision, etc. With the concept of functional engineering prevailing in neuroengineering, it is natural to focus on a specific area on cerebral cortex to gather information of neuron activity. Therefore, to achieve fancy operations like controlling the cursor with our mind, for example, we will mainly focus on motor cortex in frontal lobe, which is responsible for planning future actions and the control of movement. https://github.com/180D-FW-2023/Knowledge-Base-Wiki/wiki/Application-of-Kalman-Filter-in-Neural-Signal-Decoding 2/15 1/23/24, 9:18 AM Application of Kalman Filter in Neural Signal Decoding . 180D-FW-2023/Knowledge-Base-Wiki Wiki A Motor cortex (Precentral gyrus) Somatic sensory cortex (Postcentral gyrus) Central sulcus Parietal lobe Frontal lobe Occipital lobe Temporal lobe Lateral sulcus B Arcuate fasciculus Angular gyrus Broca's area - Visual cortex Vocalization region of motor area Wernicke's area Primary auditory cortex Figure 1-4 The major areas of the cerebral cortex are shown in this lateral view of the of the left hemisphere. A. Outline of the left hemisphere. B. Areas involved in language. Wernicke's area processes the auditory input for language and is important to the understand- ing of speech. It lies near the primary auditory cortex and the angular gyrus, which combines auditory input with information from other senses. Broca's area controls the production of in- telligible speech. It lies near the region of the motor area that controls the mouth and tongue movements that form words. Wernicke's area communicates with Broca's area by a bidirec- tional pathway, part of which is made up of the arcuate fasci- culus. (Adapted from Geschwind 1979.) Source: Principles of Neural Science, Fourth Edition. Eric R. Kandel, James H. Schwartz, Thomas M. Jessell [2] Neurons https://github.com/180D-FW-2023/Knowledge-Base-Wiki/wiki/Application-of-Kalman-Filter-in-Neural-Signal-Decoding 3/15 1/23/24, 9:18 AM Application of Kalman Filter in Neural Signal Decoding . 180D-FW-2023/Knowledge-Base-Wiki Wiki What is the basic unit of our brain and what makes it complex? We have to give credit to these simple but powerful biological creatures: nerve cells (neurons). Just like VLSI transistors, even though they have relatively basic morphology and architecture, approximately 10^11 neurons in the brain can support long-ranged and intricate anatomical circuits, where the "complexity" arises from. Neuron have four regions: cell body (soma), dendrites, axon, and presynaptic terminals. The main component axon is what we should pay special attention to: it conveys signals (action potential) to other neurons in long distance (0.1mm - 3m), while ensure the signals propagate without distortion or failure and preserving its shape at very high speed (1 - 100m/s). Compared to transmission line in our real life, the efficiency and accuracy of neural signal transmission are mind-blowing. Two neurons can also communicate at the synapse chemically using neurotransmitters. Neurons, whose structure diagram is attached below, are the most powerful building blocks of the most complicared circuits in the world. https://github.com/180D-FW-2023/Knowledge-Base-Wiki/wiki/Application-of-Kalman-Filter-in-Neural-Signal-Decoding 4/15 1/23/24, 9:18 AM Application of Kalman Filter in Neural Signal Decoding . 180D-FW-2023/Knowledge-Base-Wiki Wiki Act the brai These nervous great va on our Apical dendrites odorant vey inf carry ir other k convey the forı travels patterns Inhibitory terminal creates fiber of an axon smell, a To i Cell body conduc Excitatory terminal fiber ing she of an axon lar inte: sulated - Nucleus become nation i ter 9. Basal dendrites Axon Ne (initial segment) hillock Axon branch rons. T. known Node of Ranvier called t -Presynaptic cell- Myelin sheath Axon Presynaptic terminal Synaptic cleft Synapse ..... Postsynaptic cells -> Postsynaptic dendrite Source: Principles of Neural Science, Fourth Edition. Eric R. Kandel, James H. Schwartz, Thomas M. Jessell [2] Neural Signal (Action Potential) https://github.com/180D-FW-2023/Knowledge-Base-Wiki/wiki/Application-of-Kalman-Filter-in-Neural-Signal-Decoding 5/15 1/23/24, 9:18 AM Application of Kalman Filter in Neural Signal Decoding . 180D-FW-2023/Knowledge-Base-Wiki Wiki Is the neural signal similar to the digital signal used for communication in the real life? Actually, they are nearly identical. Action potentials are the signals by which brain receives, analyzes and conveys information. Even though they are highly stereotyped just like digital signal (0 and 1), they are able to convey all the information that the brain may need, including vision, audition, emotion, etc. However, unlike the 0s and 1s in digital signals, since they do not have distinguishable shapes, all the information is preserved in the path that the signal travels and pattern of action potentials. The maximum magnitude of action potential can be up to 40 mV, and their frequency is about 500 Hz. On the other hand, it is also their highly stereotyped shape makes decoding them feasible. Just like displacement and velocity, in neural signal decoding, we are more interested in the "velocity" of action potential: firing rate, which is the number of spikes of action potentials per unit time. With this concept, we are able to have a criterion of how active a neuron is. -+40 3 mV E -70 Figure 2-3 This historic tracing is the first published intra- cellular recording of an action potential. It was obtained in 1939 by Hodgkin and Huxley from the squid giant axon, using glass capillary electrodes filled with sea water. Time marker is 500 Hz. The vertical scale indicates the potential of the internal electrode in millivolts, the sea water outside being taken as zero potential. (From Hodgkin and Huxley 1939.) Source: Principles of Neural Science, Fourth Edition. Eric R. Kandel, James H. Schwartz, Thomas M. Jessell [2] Methods With all the knowledge of neuroscience infused in your barin, I think your neurons should be ready for the real methods of decoding the neural data. Sit back, and I will walk you through this wonderful and inspiring process. Motor Prothesis Concept https://github.com/180D-FW-2023/Knowledge-Base-Wiki/wiki/Application-of-Kalman-Filter-in-Neural-Signal-Decoding 6/15 1/23/24, 9:18 AM Application of Kalman Filter in Neural Signal Decoding . 180D-FW-2023/Knowledge-Base-Wiki Wiki This architecture of our decoder is simple but elegant. As shown in the graph, the whole BMI system consists of multiple components, including signal detection, decode algorithm, and feedback. In this section, we will focus on continuous decoding of neural signal in hand movement (motor prosthesis). Image that you would like to grab the water bottle on the table. Your brain will go over two stages: planning and movement. In the planning stage, it need to plan the according trajectories precisely. In the following movement activity, it also needs to control the acceleration and speed of your arm. In general, the movement activity follows the plan activity, and is specially tuned for direction and speed of arm movement. As a result, we will be mainly concerned with the neural movement activity and the corresponding kinematics like hand position, velocity, acceleration, etc. Cortex Neural signals Electrode Spike Detection array Neuron number ..... Visual feedback Time 1 Sensory feedback 1 + Decode algorithm Motor prosthesis Control signals ABCDEFG HIJKLMN OPQRSTU VWXYZ Communication prosthesis Source: Slides from EC ENGR C143A "Neural Signal Processing" Lecture 14 by professor Jonathan Kao [3] Decoding Data and Notation We need to know what are we going to decode at first and what the experiment looks like. Let us set up the following experimental context: a monkey is asked to perform a task. It needs to move its hand to the cursor on the screen. When the cursor appears, it cannot move. As soon as it disappear, the monkey should move its hand to the destination. We call this whole process a reach. Apparently, for the reaching task, we have two sources of data to consider: 1. Observed kinematics (dimensionality M): https://github.com/180D-FW-2023/Knowledge-Base-Wiki/wiki/Application-of-Kalman-Filter-in-Neural-Signal-Decoding 7/15 1/23/24, 9:18 AM Application of Kalman Filter in Neural Signal Decoding . 180D-FW-2023/Knowledge-Base-Wiki Wiki p x p y k Ck = v x k v y k ⎣ 1 pr and pl represent the position of hand at time k (for x and y coordinates). Respectively, vg and represent the x and y velocity of hand at time k. We ignore the acceleration and add a 1 to the end for calculation simplicity and matching the dimension. 2. Neural activity (dimensionality N) ,1 ⎤ ⎦ ⎡ yk ⎤ 2 y k yk = ⋮ „N yk 1 ⎦ y" represents the neuron n's firing rate at time k. In our case N = 96, which means that we decode neural signals from 96 neurons at the same time. We add a 1 to the end for calculation simplicity and matching the dimension. We will further denote the decoded kinematics as Xx. For convenience, for the total period of time of these activities, we can also define: X = [x1 x2 ... CK] Y = [y1 y2 ... yK] where K is the total number of data points. If the time interval between two points is At, thus the total period of time of these activity will be T = KAt. General Ideas in Decoding The general idea of the decoding operation is straightforward: decode th kinetics information from neural activities, which means to infer X from Y. You might think this sounds easy. One simple method is that, if we could assume that their relation is linear, we can simply fit a line to them using least square since we have both observed data for X and Y: X = LY Basically is to minimize: 1 |2Ck - Lykl12 k=1 K https://github.com/180D-FW-2023/Knowledge-Base-Wiki/wiki/Application-of-Kalman-Filter-in-Neural-Signal-Decoding 8/15 1/23/24, 9:18 AM Application of Kalman Filter in Neural Signal Decoding . 180D-FW-2023/Knowledge-Base-Wiki Wiki With our knowledge in linear algebra, we get: L= XYT(YYT)-1 Now we have this L. Concretely, say that we have a new neural signal data k, then we can decode the kinetics &k using: ֏k = Lýk Problem solved! This is what we called Optimal Linear Estimator (OLE). Is this the final method that we should concern? Of course not! Unfortunately, as you may guess, its accuracy and robustness are not very ideal. Obviously, we have ignore something in this case. First, for the adjacent positions, say &n and &n+1, there must be some relation between them; second, even though we introduce the concept of velocity, we did not apply any physics law to exploit them. Well, the first problem seems easy to solve: we can simply take history into account by doing something like: ∑ P p=0 Rewrite it in matrix form: Xw =LwYw We can find Lw using least square as well: Lw =XUYT (YaYT)-1 Seems more advanced, doesn't it? This technique is called Wiener Filter. We enhance the OPE by considering data from the history. But what about the second problem? it's finally time to welcome Kalman Filter to the stage. Decoding with Kalman Filter The main concern for now is, we have additional information we have not used in our decoding process: the law of physics, since we understand the the positions of the arm are the integration of its velocity (we will be in real trouble if they are not). The natural way to incorporate this is to use Kalman Filter. If we add one more equation to the original OLE, we can get: @k+1 = Axk yk = Cxk https://github.com/180D-FW-2023/Knowledge-Base-Wiki/wiki/Application-of-Kalman-Filter-in-Neural-Signal-Decoding 9/15 1/23/24, 9:18 AM Application of Kalman Filter in Neural Signal Decoding . 180D-FW-2023/Knowledge-Base-Wiki Wiki The first equation is called state update process or dynamic process, where xk, the kinematics of the hand, is the state. A is the dynamic matrix where we can encode the law of physics. The second equation is observation process. yk is just the observation of neural signal. This inverse OLE equation helps us link the neural signal yk and kinematics xk. If you recall your signals and systems class, these two equations together is called Linear Dynamical System. In real life, we have to add some random noise to them: Xk+1 = Axk + Wk yk = Cxk + qk We can simply assume that they are gaussian noise. Zero mean and covariance matrix W and Q will be good in our case: Wk ~ N (0, W) qk ~N (0,Q) So how do we get the optimal estimator of ck from those two equations? Under certain assumption (linearity and gaussian noise), there is a recursive solution to this called Kalman Filter The spoiler is, we will reach to some kind of equation like this: ֏k = M1ºk-1 + M2yk Apparently, we take all factors into account: history, law of physics, and the neural signal. All of them have some information of what the newly decoded data should be. So the goal is to find two matrices M1 and M2. They are the function of A, C, W, Q. What is Kalman Filter actually doing? In one dimension, we need to find an estimator kl, .,k = E(xk |31, ... , yk) In other words, this means "we want to find the expected value of hand position at time k given all my observations of neural signal from time 1 to k". We also need to know the variance of it to estimate how much we trust this value (in multi-dimension this will be 2): "kl1 .. .. ,k = Var (kly1, ... , yk) In general, we would write the distribution of xkly1, ... , yk and then just take its expected value. This seems to be difficult, but we have a recursive solution provided by Kalman Filter. We will ignore the tedious derivation of it here. The process will be: 1. Initialize: 2010 = 0 2. Estimation updated process: Recursively calculate, until convergence of Kk: 2klk-1 = AZk-1|k-1AT + W https://github.com/180D-FW-2023/Knowledge-Base-Wiki/wiki/Application-of-Kalman-Filter-in-Neural-Signal-Decoding 10/15 1/23/24, 9:18 AM Application of Kalman Filter in Neural Signal Decoding . 180D-FW-2023/Knowledge-Base-Wiki Wiki Dkk = 2k-1|k-1 - 2k-1|k-1CT(CZk-1|k-1CT)-1CZk-1|k-1 T T ) We called the converged value . and Ko, and Kk is called the Kalman Gain After some more magic, we can calculate M1 and M2 by M1 = A -KCA M2 = Koo What about A, C, W, Q themselves? We need to calculate them using maximum likelihood method in machine learning. What we need to do is to maximize the following likelihood with parameters set 0: L = P(k> Uk) k=1 0 = A, W, C, Q) This means that, with the parameters set 0, we need those obaservations data to appear at the highest possibility. We can do this by taking the derivative of logarithm of L with respective to A, C, W, Q. Then simply set it to zero. What we will finally get is: A= X:,2:endX+ :,1:end-1 C = YX+ W = K-1 1 (X:,2:end - AX:,1:end-1)(X:,2:end - AX;,1:end-1)" Q = = (Y-CX)(Y -CX)T We also need to make sure that A obeys law of physics. With all these parameters, we should be able to decode xk using the formula we first propose. The final results of the decoding is shown in the graph on the left. On its right is the actually movement of all the reach in the experiment marked by different colors, where x and y axis represent the coordinates of hand positions. Although they don't seem to be close, the analysis on loss function of squared error proved the results to be reliable, and in real life they will work with high accuracy if we are not very serious about fidelity. https://github.com/180D-FW-2023/Knowledge-Base-Wiki/wiki/Application-of-Kalman-Filter-in-Neural-Signal-Decoding 11/15 1/23/24, 9:18 AM Application of Kalman Filter in Neural Signal Decoding . 180D-FW-2023/Knowledge-Base-Wiki Wiki decoded positions 200 100 0 -100 -200 - -200 -150 -100 -50 0 50 100 150 200 hand positions 150 100 - 50 0 -50 -100 - -150 -100 -50 0 50 100 Source: EC ENGR C143A Spring 2023 HW6Q4 by Hongchang Kuang [4] https://github.com/180D-FW-2023/Knowledge-Base-Wiki/wiki/Application-of-Kalman-Filter-in-Neural-Signal-Decoding 12/15 1/23/24, 9:18 AM Application of Kalman Filter in Neural Signal Decoding . 180D-FW-2023/Knowledge-Base-Wiki Wiki Conclusion I hope this method in neural signal processing is inspiring and fantastic. This article introduces the motivation of choosing Kalman Filter in decoding neural signal, and the basic mathematical recursion in arriving at the final answer. In general, when it comes to decode continous data, Kalman Filter is a powerful technique which incorporates factors like history, law of physics, and the characteristics of decoded data into the decoding process. It presents ideal accuracy and robustness. In our project, for example, we can choose to use Kalman Filter in smoothing the data collected from the IMU. Nevertheless, I cannot guarantee that Kalman Filter will be a panacea to all your problems: it also has some drawbacks. Kalman Filter can easily experience overshoot since it may weight too heavily on the history [3]. Some potential improvements in neural signal processing include rotating the velocity toward the destination (take angles into consideration) or use Closed Loop Decoder Adaptation (CLDA). In modern decoder with more advanced features, Deep Neural Network is also widely used. In the near future, we can expect that neural signal decoder would provide more advanced functionality like speech decoding, wating for you to explore further [5]. References [1] AI's Next Frontier: Are Brain-Computer Interfaces The Future Of Communication? Bernard Marr, Forbes. https://www.forbes.com/sites/bernardmarr/2023/08/11/ais-next-frontier-are-brain- computer-interfaces-the-future-of-communication/?sh=61d3d2b851d9 [2] Principles of Neural Science, Fourth Edition. Eric R. Kandel, James H. Schwartz, Thomas M. Jessell [3] Slides from EC ENGR C143A "Neural Signal Processing" by professor Jonathan Kao [4] Homework answers from EC ENGR C143A "Neural Signal Processing" by Hongchang Kuang [5] SCIENTISTS SAY NEW BRAIN-COMPUTER INTERFACE LETS USERS TRANSMIT 62 WORDS PER MINUTE. Victor Tangermann, Futurism. https://futurism.com/neoscope/scientists-new-brain- computer-interface-type-62-words-per-minute + Add a custom footer - Pages 36 Find a page ... Home https://github.com/180D-FW-2023/Knowledge-Base-Wiki/wiki/Application-of-Kalman-Filter-in-Neural-Signal-Decoding 13/15