Phoneme Frequency Tables

We've identified two ways to rank the possible spelling of any given phoneme for our data set of over 26,000 words. In the first, we count the number of times a phoneme-grapheme correspondence occurs in the list of words. In the second, we give a weight to each word in the list depending on its frequency in connected text, then multiply each correspodence by that weight.

Why Adding a Frequency Weight Matters

Let's consider the sound /v/, which can be represented by the letter 'v' or the letter 'f'. In our data, there are about 1,700 words that use 'v' and only one word ('of') that uses 'f' (technically two words use this, thereof, being the other). Let's look at an excerpt from "Wizard of Oz" by L. Frank Baum to see why this matters.

When Dorothy stood in the doorway and looked around, she could see nothing but the great gray prairie on every side. Not a tree nor a house broke the broad sweep of flat country that reached to the edge of the sky in all directions. The sun had baked the plowed land into a gray mass, with little cracks running through it. Even the grass was not green, for the sun had burned the tops of the long blades until they were the same gray color to be seen everywhere.

As we can see above, the word 'of' occurs as often as all the other words that include the letter 'v'. It is one of the most common words in English!

So, if you're reading connected text and come across a spelling of the sound /v/, it's just as likely to be spelled with the letter 'f' as it is with the letter 'v'. This is a rather extreme example. Many of the other sounds have similar results regardless of how we rank them, but we can use this data to inform our teaching approaches.

Below you can see the breakdown of each correspondence for the sound /v/. We have a table for the direct frequencies and one for the weighted frequencies.

Frequency Tables by Phoneme

We've run rankings for our entire data set. Below you can find the frequency and weighted frequency for the spellings of each phoneme. You can click on each grapheme to see the words with that correspondence in our Word List Builder.

See our Methodology at the bottom for more details.

Methodology

In our data we break each word into its phoneme-grapheme correspondences. We use two rankings to score their frequency: direct and weighted.

For direct ranking:
Frequency (total) - We count the number of times a correspondence occurs in each word in our list. If it is used more than once in a word, then it is counted for each occurrence.

Frequency (%) - The percentage of time that a phoneme is represented by a grapheme.

For weighted ranking:
We use two different word frequency lists to determine a word's frequency. First, we use a Project Gutenberg word frequency list from 2006 to get a baseline in written text. Since Project Gutenberg (PG) is limited to public domain books it does not have recent words like 'aliens' and 'radiation'. We used Open Subtitles, open source movie subtitles, as our second list since it includes recent movies and hence, modern words.

We multiply the Open Subtitles frequency by 4 to bring it in line with values from the Project Gutenberg frequencies and then average the two frequencies. We wanted common words like 'it' and 'that' to be roughly equivalent for both data sets.

Weighted Frequency (total) - We look at each word, then multiply the number of times a particular correspondence occurs in the word by the word's frequency, then we add all these values together for that correspondence.

Here's an example. This is the calculation for 'f' = /v/.
PG frequency for 'of': 33,950,064.00
Subtitle frequency for 'of': 1,847,884.00
Average frequency: (33,950,064.00 + (1,847,884.00 x 4)) / 2 = 20,670,800.00
Number of times 'f' = /v/ in the word 'of': 1
Weighted Frequency for 'f' = /v/ in the word 'of': 20,670,800.00 x 1 = 20,670,800.00

We repeat this calculation for the far less common word 'thereof' to get the weighted frequency of 15,143.25

Finally, we sum the weighted frequencies and divide by 5, the lowest frequency for all correspondences, to make the numbers more manageble.
Weighted Frequency (total): (20,670,800.00 + 15,143.25) / 5 = 4,137,188.65

Weighted Frequency (%): The percentage of time that when you come across that phoneme in text that it will be represented by that particular grapheme.

* These calculations are based on the count of phoneme-grapheme correspondences across 26,755 words as of August 2023.

Other References

For previous work in this area see Paul Hanna's paper Phoneme-Grapheme Correspondences as Cues to Spelling Improvement