Speech & Language Blog

Introduction The fourth post of my probability series is about  probability distributions . Basic Probability Two Random Variables Chain Rule of Probability Theory In the previous posts, I discussed probabilities involving a single and multiple random variables. Briefly, a random variable in an experiment or a trial maps a specific outcome from a sample space to a numeric value.  The example of a probabilistic event which I kept using was whether I to go to Paris next year \(X=1\) or not \(X=0\). A probability of a specific event can be expressed like this: \[ P(X=1) = 0.7 \] The generalised probability is then like this: \[ P(X) \] where \(X\) can be anything in a defined sample space. A probability distribution concerns probabilities of all possible outcomes of a sample space. So, \( P(X=1)=0.7 \) is a part of a probability distribution, but the whole representation of the probability distribution is \( P(X) \). More concrete examples are: we think about "going to Paris...

Introduction  This is my third post on the probability theory: Basic Probability Two Random Variables The main focus of the series is on easy introduction to probability theory. A bit of recap : in the last post of Two Random Variables, I discussed probabilities of two random events including joint, conditional and marginal probabilities. That post was about probabilities of two dependent and independent events. The example was whether Emily and I go to Paris next year. If Emily and I did not know each other, Emily visiting Paris next year would have no impact on my chance of going to Paris (two events were independent). On the other hand, if I knew Emily and she text me about her trip to Paris, that could affect my chance of visiting Paris next year (two events were dependent). This post focuses on generalisation of multiple events and random variables. So, I am introducing a new event. An alien is visiting Paris. We need to think about how this event will impact chances of my and...

Introduction In the previous post of  Basic Probability , I discussed my chance of visiting Paris next year. I had a sample space like this: \( S = \{Meet, No\_more\_holiday, No\_money, Paris\_gone\_from\_Earth, ...\} \). The random variable of \( X \) was all about me going to Paris next year. What I am going to write in this post is when we have two sample spaces, two outcomes and two random variables. Having two random variables means that we need to consider the followings: two events happen simultaneously ( joint probability ), only one event happens regardless of the other event ( marginal probability ) and one event happens because of the other event ( conditional probability ). Let's define a simple sample space of me visiting Paris next year is \( S_X = \{me\_in\_paris, me\_not\_in\_paris\} \).  Let's define another sample space to have two events at the same time. The second event is whether Emily goes to Paris next year. We now have anot...

Introduction Probability theory is the foundation of machine learning. Knowledge of machine learning is a requirement for working on a speech and language processing project today. So, probability theory is essential for speech and language processing projects! The objective of this post is to refresh my knowledge of probability theory. I am keen to connect probability theory with real world examples, and to avoid throwing a bunch of theoretical definitions. Feel free to leave comments if my writing is incorrect.   Probability The probability is a chance of an occurrence of an event. The probability is a value between 0 and 1. In contrast, human words are not mathematical. Even if I say "I'll go to Paris next year, 100%", I might not go to Paris100% next year. When I was in Paris last time. The theoretical and mathematical probability has to be precise unlike human words. A ...

Introduction This post summarises acoustic features for various tasks of speech processing. Automatic speech recognition (ASR) is one of the most studied speech processing tasks. Acoustic features for ASR include Mel-frequency cepstrum coefficients (MFCCs) and spectogram-based features including Mel-spectrograms and Mel-filter banks. The choice of acoustic features depends on a choice of ASR model: Traditional machine learning (ML) models such as Gaussian mixture models (GMMs) have difficulties of handling correlated features and MFCCs are favourite for de-correlated coeffcients. More recent deep learning based models e.g., Conformer use acoustic feature vectors with correlation between neighbour dimensions: filterbanks. A popular model from OpenAI, Whisper , directly takes as input a log-Mel spectrogram which is technically the same representation as filterbanks (will be explained later). ...