Speech & Language Blog

Introduction The decibel is a unit to express loudness of sounds, and an important measurement in sound processing. The decibel is the logarithmic scale of sound intensity. The reason to use the logarithm is that human hearing is logarithmic rather than linear. The decibel is also not an absolute metric but a relative ratio of intensity of one sound compared to another. This part is very confusing because we think that familiar measurements like lengths "cm, m, km" and weight "g, kg..." are absolute units. There are many online resources that explain the logarithms or the decibel. I don't see resources that explain both the logarithms and the decibel. This is my motivation to create this post: keeping information about the logarithms and the decibel in one page.  Logarithms The logarithms are inverse operation of an exponent (power). \[ 2^3 = 8 \] \[ \log_28 = 3 \] The log of 8 to the base 2 is 3 , and 2 to...

Discrete Fourier Transform The last part of the previous post mentions the method to find frequencies in a signal: the Discrete Fourier Transform (DFT). This post dives deeper into the DFT. The main idea of the DFT is to find out which frequency component correlates with the given input signal . This mathematical formula looks scary. \[ X[k] = \sum_{n=0}^{N-1} x[n] \, e^{-j2\pi k n / N} \] k = current frequency to check correlation n = current sample N = number of samples x[n] = the value of the current sample of the given signal My goal in this post is to demonstrate what the DFT performs is simple. Input signal and correlation signal Let's say the input signal is a sine wave of 2 Hz. There are 30 samples to represent this signal. The input signal of 2 Hz should show the highest correlation at 2 Hz. Let's also have 5 different correlation signals varying from 0 Hz to 5 Hz. The first correlation signal has i...

Introduction The signal frequency is the "pitch" of the sound. Some facts about sound frequencies you might encounter in a pub quiz... Typically, male voices range from 85 to 180 Hz and female voices from 165 to 255 Hz .  Humans can easily hear the sound frequency up to 8,000 Hz and lose abilities to hear sounds beyond that frequency through age. The music note C is 261.63 Hz and E 329.63 Hz. The Python notebook is a convenient playground to generate sounds of those frequencies and listen to the sounds. https://github.com/yasumori/blog/blob/main/2025/2025_12_21_signal2.ipynb . An example code snipet to generate a 2,000 Hz sound is also below: import numpy as np from IPython import display def gen_audio(frequency, duration, sample_rate): t = np.linspace(0, duration, duration * sample_rate) return np.sin(2 * np.pi * frequency * t) hz_2000 = gen_audio(2000, 3, 44100) disp...

Acoustic Signal I had a hard time understanding basics of digital signal processing. I think that the reason was the fact that sounds were not visible. There is a way to "see" the sound, though. This YouTube video for example demonstrates that special cameras make the sound visible.   The sound is the virbation of particles in the air. Something invisible surrounding us rapidly moves back and forth. And our ears can hear this movement. I am writing this post to cover very basics of digital signal processing that I was very slow to understand. I will write concepts of Analogue-to-Digital conversion, sampling and Nyquist frequency. Analogue vs Digital Apart from the sound not usually visible to us, I think that I initially mixed up the concept of the digital signal with the analogue signal. I was still new to the idea that computers represent everything in discrete numbers. Two important things: An analogue s...

Introduction Building an automatic speech recognition (ASR) system has become easier than ever thanks to the availability of deep learning and ASR toolkits. These toolkits magically pre-process an audio file and a neural model can seamlessly take processed audio features, letting users needing not to know much details. This is my motivation to write a summary of how audio files are processed for a neural model. In this post, I will focus on  wav2vec2.0  proposed by Baevski et al. which learns speech representations in a self-supervised manner. This  post  from Hugging Face is an excellent tutorial showing how to fine-tune wav2vec2.0 for the ASR task. I would like to provide information focusing only on input structure of wav2vec2.0 to compliment the tutorial. Fig1: Audio input processing of wav2vec2.0 discussed in this post is the part circled by a red line. This figure is taken ...

Attending an Offline Conference Again  I was fortunate to have an opportunity to attend the Spoken Language Technology workshop 2022, held from 9th to 12th January in Doha, Qatar. I presented my final PhD work at the conference. Going back to an (almost) in-person conference was so refreshing and I learned so many things. In this post, I'll summarise my takeaways from the SLT 2022 conference. The photo below is the conference venue, Marsa Malaz Kempinski.  Human Speech Processing and Neural Networks As a linguistics student (back in 2012), I've been always keen to learn studies on relevance of human language processing to computer speech processing. On this topic, I remember one of the keynote presentations and one poster. The first keynote talk by Nima Mesgarani was interesting, mentioning that the shallow layers of an RNN-T encoder learns acoustic and phonetic information occurring in short time frames while the later layers capture lexical and semantic information which ca...