Speech & Language Blog

Introduction Differentiation calculus is a core part of machine learning optimisation. Differentiation is often taught at school as finding a slope in geometry. While a theoretical concept of differentiation is important, I feel that connection between differentiation at school and differentiation for machine learning optimisation is not linked well. This post aims to bridge the gap between school math and machine learning optimisation, focusing on how the mathematical concept (differentiation) is essential for the practical algorithm (gradient descent) to support machine learning. The post begins with a very basic concept of slope finding. Finding a slope between two points The general formula to find a slope \(a\) of a function \( y = ax \) is: \[ a = \frac{y_2 - y_1}{x_2-x_1} \] The figure below illustrates rise (vertical) and run (horizontal) changes between two points, Point 1 \((x = 0, y=0)\)  and Point 2 \((x=2, y=4)\).  The slope of this function is \( a = \frac{4...

Introduction The fifth post of my probability series is about continuous probability distributions. Basic Probability Two Random Variables Chain Rule of Probability Theory Probability Distribution1 A quick recap: a probability distribution concerns probabilities of  all possible outcomes  of a sample space. The previous post introduced four different discrete probability distributions.  The current post introduces the most popular continuous probability distribution: normal (Gaussian) distribution. The normal distribution has been an essential tool for Automatic Speech Recognition a long time before deep learning. Normal distribution The normal distribution is a bit like the Eiffel Tower. It is a distribution known for its "bell curve". The distribution is pointy in the middle and wider towards the edges of the distribution. The name "Gaussian" distribution is from the mathematician, Gauss . The properties of the normal distribution are as follows: Number of trials:...

Introduction The fourth post of my probability series is about  probability distributions . Basic Probability Two Random Variables Chain Rule of Probability Theory In the previous posts, I discussed probabilities involving a single and multiple random variables. Briefly, a random variable in an experiment or a trial maps a specific outcome from a sample space to a numeric value.  The example of a probabilistic event which I kept using was whether I to go to Paris next year \(X=1\) or not \(X=0\). A probability of a specific event can be expressed like this: \[ P(X=1) = 0.7 \] The generalised probability is then like this: \[ P(X) \] where \(X\) can be anything in a defined sample space. A probability distribution concerns probabilities of all possible outcomes of a sample space. So, \( P(X=1)=0.7 \) is a part of a probability distribution, but the whole representation of the probability distribution is \( P(X) \). More concrete examples are: we think about "going to Paris...

Introduction  This is my third post on the probability theory: Basic Probability Two Random Variables The main focus of the series is on easy introduction to probability theory. A bit of recap : in the last post of Two Random Variables, I discussed probabilities of two random events including joint, conditional and marginal probabilities. That post was about probabilities of two dependent and independent events. The example was whether Emily and I go to Paris next year. If Emily and I did not know each other, Emily visiting Paris next year would have no impact on my chance of going to Paris (two events were independent). On the other hand, if I knew Emily and she text me about her trip to Paris, that could affect my chance of visiting Paris next year (two events were dependent). This post focuses on generalisation of multiple events and random variables. So, I am introducing a new event. An alien is visiting Paris. We need to think about how this event will impact chances of my and...

Introduction In the previous post of  Basic Probability , I discussed my chance of visiting Paris next year. I had a sample space like this: \( S = \{Meet, No\_more\_holiday, No\_money, Paris\_gone\_from\_Earth, ...\} \). The random variable of \( X \) was all about me going to Paris next year. What I am going to write in this post is when we have two sample spaces, two outcomes and two random variables. Having two random variables means that we need to consider the followings: two events happen simultaneously ( joint probability ), only one event happens regardless of the other event ( marginal probability ) and one event happens because of the other event ( conditional probability ). Let's define a simple sample space of me visiting Paris next year is \( S_X = \{me\_in\_paris, me\_not\_in\_paris\} \).  Let's define another sample space to have two events at the same time. The second event is whether Emily goes to Paris next year. We now have anot...