Compared to traditional vanilla RNNs (recurrent neural networks), there are two advanced types of neurons: LSTM (long short-term memory neural network) and GRU (gated recurrent unit). In this blog, we will give a introduction to the mechanism, performance and effectiveness of the two neuron networks.

In standard RNNs, sigmoid or hyperbolic tangent activation function is generally used as an activation function. There are large areas of each function where the derivative is very close to 0, which means the weight updates are small, and RNNs get saturated. When the values of gradients are are extremely low or high, it is…

Recurrent Neural Networks (RNNs) are a special type of neural networks designed for sequence problems. RNNs add the explicit handling of order between observations when approximating a mapping function from input variables to output variables, which is capable to predict time series for neural networks. Traditional time series forecasting methods like ARIMA focus on univariate data with linear relationships. However, RNNs add the capability to learn possibly noisy and nonlinear relationships and provide direct support for multivariate and multi-step forecasting. This blog gives a brief introduction of the mechanism of RNN.

The trait of RNN is to evaluate sequences of…

There are many prediction problems that involve a time component such as forecasting some yield each year, forecasting some price each day, forecasting some rate each hour etc., which makes the problems more difficult to handle. This blog will introduce machine learning techniques to better analyze and predict time series.

A time series can be decomposed into four constituent components: level (baseline value), trend (linear behavior), seasonality (the periodic behavior) and noise. According to the number of observations recorded at each time, the dataset can be marked as univariate time series and multivariate time series. …

Options are financial derivatives based on the value of underlying securities. They give the buyer the right to buy (call options) or sell (put options) the underlying asset at a pre-determined price within a specific timeframe. There are also two basic styles of options: American and European. American options can be exercised any time before the expiration date of the option, whereas European options can only be exercised on the expiration date. This blog digged into an option-pricing model to understand the evaluation of European options.

The Black-Scholes model or Black-Scholes-Merton model is a mathematical model for pricing an options…

Monte Carlo simulation is a computerized mathematical technique that relys on repeated random sampling to obtain numerical results. It is used to model the probability of different outcomes in a process which is impractical or impossible to solve analytically. The modern version of this technique was first used to work on nuclear weapons projects. It is named after the Monte Carlo Casino in Monaco. The technique is used by professionals to tackle a wide range of fields such as finance, insurance, manufacturing, engineering, transportation, and science. …

Linear regression is an important predictive analytical tool in the data scientist’s toolbox. In this blog, we implement least squares to approximate solutions of over-determined systems of linear equations by minimizing the sum of the squares of the errors in the equations. An introduction of how to use linear algebra to solve regression problems into machine learning and predictive analysis is reported.

Linear equation has no solution when the matrix has more rows than columns, which means there are more equations than unknowns. Therefore, we cannot always get the error down to zero. However, a least squares solution can be…

Fourier transform is an efficient and powerful computational tool for data manipulations and data analysis. Although Fourier methods are commonplace in research, many computer users have relatively little understanding of its mathematical fundamentals. Therefore, the general rubic of the method and numerical algorithms are introduced in this blog. One important application, data denosing using FFT (Fast Fourier Transform) is discussed.

A physical process can be described either in the *time domain *or* frequency domain, *which can be represented as a function of time *t*, i.e., *h(t)* and a function of *frequency, f *or* angular frequency,ω *(*ω*=2𝜋*f*), i.e., *H(ω)*, respectively. …

What happened couldn’t have happened any other way…