This is the abstract of our recent paper at IJCNN 2017. This method addresses the problem of time series classification on low-dimensional and heterogeneous feature space with the use of recurrent neural networks.
We present Partition-wise Gated Recurrent Units (pGRUs) for point-based trajectory classification to detect real-world trawler fishing activities in the ocean. We propose partition-wise activation functions integrated with Gated Recurrent Units that partitions each feature and uses independent parameters to model distinct regions of the feature space. This approach enables us to leverage the benefits of deep learning on a low-dimensional and heterogeneous feature space by mapping the low-dimensional features into another space that can be better separated with piece-wise learnable parameters. We show that the proposed partition-wise activation functions can approximate a wide array of functions, including conventional activations such as sigmoid and hyperbolic tangent. Our experimental results demonstrate that pGRU learns the probability distribution from data and achieves substantial improvements over the state-of-the-art systems on the task of trawler fishing activity detection.
The slides are available at here.
Sometimes we may need to install Tensorflow from source without root access to the server, such as a cluster environment. This is even more true when you need to configure Tensorflow with your own environment: e.g., different Cuda and CuDNN versions. This post explains how to compile Bazel, Swig from source, and use them to compile Tensorflow.
Continue reading “Build Tensorflow from Source with Swig and Bazel without Root”
As a developer, if you do not want to restrict yourself with paid or limited numbers of private repos in services like Github and Bitbucket, it would be desirable to set up your own private git server. This post aims to provide a step-by-step instruction to setting up a private git server.
Continue reading “Setup Private Git Server in Ubuntu”
Restricted Boltzmann Machines (RBMs) is an unsupervised machine learning method that significantly contributes the early rise of deep learning in 2006: A Fast Learning Algorithm for Deep Belief Networks. Since then it becomes an important element in the Deep Learning family. Interestingly, RBMs manifest the idea of both probabilistic graphical models and neural networks. This post aims to provide an introduction to RBMs. Most of the formulas here are from .
Boltzmann Machines are energy-based models  where the joint probability distribution is characterized by a scalar energy to each configuration of the variables. In energy-based models, inference consists of clamping the values of a set of variables, and finding configurations of the remaining variables that minimize the energy function; learning consists in finding an energy function that minimizes the energy of observed variables. Boltzmann machines are also probabilistic graphical models using graph-based representation to factorize the probability distribution. Restricted Boltzmann Machines is a type of Boltzmann machine with special constraints – only a certain type of connection is allowed.
This post starts by introducing energy-based models, including the graphical representation of the model and its learning with gradient descent of log-likelihood. This post then discusses Boltzmann machines by placing a specific energy function in energy-based models. Restricted Boltzmann Machines are further discussed with the introduction of restrictions in Boltzmann Machines connections.
Continue reading “A Tutorial on Restricted Boltzmann Machines”
Tensorflow 1.0 is awesome, many nice features, including GridLSTM, LayerNormalization, CRF, etc. There is even a transitioning script that helps automatically converts your file from 0.x to 1.x. However, the official transitioning script does not include many changes that need to be made in my previous LSTM code. This post will list the changes that I discovered, and I hope it will make your life easier. If you have other problems, search the function name in the new API, you should find the corresponding new version. Note that sometimes the order of parameters may differ.
tf.nn.rnn_cell -> tf.contrib.rnn
tf.nn.seq2seq -> tf.contrib.legacy_seq2seq
There are at least five different approaches to derive the normal equation. This post aims to provide a living document of the normal equation as well as its interpretations. Continue reading “Five Ways to Derive the Normal Equation”
Inferential statistics are distinguished from descriptive statistics. It infers underlying distributions from and beyond observed data, as opposed to summarization of data. Continue reading “Inferential vs. Descriptive Statistics”
My Tensorflow mysteriously failed at the server. Without root access, I have to compile it from the source, to configure the GPU device. The error is module not found. Tried to pip install the module multiple times, but it still does not work. I can see the package in my site-packages directory, changed access to executable, but still not working.
It turns out that, I have set the PYTHONPATH variable in my ~/.bashrc, and pip installed packages to that specified directory. When I was trying to import the package, it could not find from the virtualenv directories. After removed the PYTHONPATH from my ~/.bashrc, everything works like a charm.
In this post, I am going to talk about Beta Distribution and some intuitive interpretations behind it.
Suppose we have two coins (A and B), and we are making a statistical experiment to identify whether these coins are biased or not. For coin A, we tossed 5 times and the results are: 1,0,0,0,0. (1 indicates Head and 0 indicates Tail). For coin B, we tossed 10 times and the results are: 1,1,0,0,0,0,0,0,0,0. The probability for theses two coins to be Tail are identical: 0.2. Is it safe to say, both coins equally favour the Tail?
Continue reading “Beta Distribution”
Google has provided its infrastructure via Google Cloud Platform to developers for building applications, as an alternative to Amazon Web Service. The setup of a new instance is simple and intuitive, however, it is frustrating not being able to connect to the instance via ssh. After some research, I found out how to resolve the problem, and I hope it can help others who are having the same issue.
Connection Failed: We are not able to connect to the VM on port 22.
I could not even telnet IP 22
1. Activate Google Cloud Shell
2. Add a rule to the firewall to allow traffic from port 22:
gcloud compute firewall-rules create allowssh –allow tcp:22 –source-ranges 0.0.0.0/2