Probability of Selecting Matched Pairs and the Hypergeometric Distribution
The problem Consider a case where we have a bag of marbles of size . The bag consists of black […]
PCA vs Autoencoders for Dimensionality Reduction
There are a few ways to reduce the dimensions of large data sets to ensure computational efficiency such as backwards […]
Confidentialise Your Data with the randomNames Package
Sensitive data has it’s restrictions for good reason. Personal data such as names and other identifiable information should be protected. […]
Improve Your Training Set with Unsupervised Learning
On my previous post Advanced Survey Design and Application to Big Data I mentioned unsupervised learning can be used to […]
Advanced Survey Design and Application to Big Data
I like to describe Official statistics as the All Bran of statistics, it’s bland and a bit boring but it […]
Deep Neural Network from Scratch in R
Neural networks evolved in the computer science domain are often the first thing people think of when they hear machine […]
Unsupervised Random Forest Example
A need for unsupervised learning or clustering procedures crop up regularly for problems such as customer behavior segmentation, clustering of […]
Design Matrix for Regression Explained
Recently I was asked about the design matrix (or model matrix) for a regression model and why it is important. […]
Introduction to Outlier Detection
Outlier detection and treatment is an important part of data analysis and one of the first things you should do […]
UTA100 2018: Race report
This was my first UTA100. A race I’ve long wanted to do. It’s an amazing event, the atmosphere is exhilarating […]
Agile Methodology for Data Science Projects
Agile project management methodology is a powerful framework when businesses are faced with uncertainty, rigid time frames and budget constraints. […]
Singular Value Decomposition: Image Compression
Singular value decomposition (SVD) is a matrix factorisation technique. It is a method to factorise any given matrix into 3 […]
What is the Bayes Factor anyway?
The Bayes Factor is a quantitative measure of much evidence there is for hypothesis A relative to hypothesis B given […]
Forecasting Air Pollution Levels with State Space Models
Recently I was looking at a Kaggle dataset on air quality and the relationships between the concentrations of contaminants and […]
State Space Models for Time Series Analysis and the dlm package
State space modelling is a popular technique for forecasting and smoothing time series data. There are two main components which […]
