Introduction

The need to convey letters of an alphabet verbally is a common occurrence. You are on the phone and need to spell a name, car registration number or email address. If you are an English speaker, you will probably be struggling with phases like “‘d’ for ‘dog'” or “‘f’ for ‘Fred'”.

You may just choose to sound the letters directly as ‘de’, ‘ef’, etc. and hope they will be received correctly. With today’s modern, and generally high quality, voice networks this strategy is often successful but occasionally there are misheard letters and you are back to finding a word that represents them or repeating them more slowing and distinctly.

The problem occurs in all languages but this blog will focus on English. However, the principles should be easily transferred to other languages.

How to reliably transmit data over a noisy channel is a well known and much studied problem. The general approach is to include additional information with the original data. This additional or redundant information is used to recover the original data should it be affected during transmission. The trade-off is this additional information makes the transmission longer. So the challenge is to formulate this additional information in a ways that keep the transmission as short as possible but provides the best chance to recover the data should it become corrupted. In the digital domain this comes under the study of Coding Theory.

For our purposes, the additional data is in the form of a word which is used to represent a letter. A word is longer and has more sounds which makes it much easier to identify and less likely to be confused over a noisy background. The words chosen need to be:

1. Known by the speaker and recipient
2. Clearly associated with a letter either by agreement or implication
3. Sufficiently distinct from each another to improve differentiation and recognition
4. No longer than necessary for expediency

The first for these is accommodated by using well known common words, the second by treating the first letter of the word as the representative letter and the last two are the main design challenge.

Obviously using say, “pat” for “p” and “bat” for “b” is not particularly effective as “pat” and “bat” might themselves be confused. Whereas, for example, “pen” and “bat” seem to be more distinct.

Using long words would make the distinction even greater, for example, “banana” and “pendulum”, however these take longer to say and hence slow down the transmission process. So there is a trade-off between speed and accuracy.

We can summarise the goals of an effective International Spelling Alphabet (or word replacement scheme for letters) as:

1. Use common words easily recognised by both parties
2. Use short words to speed communications
3. Use words that are phonetically distinct to minimise confusion

Some Existing Examples

International Spelling Alphabet

International Spelling Alphabet (ISA), also known as the NATO Phonetic Alphabet, is widely used in aviation, military and other areas where accurate letter transmission is critical. Most people are familiar with it, even if they don’t remember all the words. It was developed in the 1950’s and hasn’t changed since.

 a - alfa            n - november
 b - bravo           o - oscar
 c - charlie         p - papa
 d - delta           q - quebec
 e - echo            r - romeo
 f - foxtrot         s - sierra
 g - golf            t - tango
 h - hotel           u - uniform
 i - india           v - victor
 j - juliett         w - whiskey
 k - kilo            x - x-ray
 l - lima            y - yankee
 m - mike            z - zulu

Note the deliberate misspelling of alfa (alpha) and juliett (juliet/juliette) to help clarify pronunciation for non-native English speakers.

Hexadecimal

When I started in IT, many years ago, I was taught to use the following words to represent the six hexadecimal letters:

 a - able
 b - baker
 c - charlie
 d - dog
 e - easy
 f - fox

We used these to read out hexadecimal character strings, which are conventionally represented by the sixteen characters 0-9, A-F. This worked well as they are quite short, easy to say and fairly distinct.

Later I discovered they came from the earlier US Phonetic Alphabet (USPA) which was replaced by the ISA (above) in 1956. As can be seen from the reference, there were several variations and names of these early alphabets. In this article we will use USPA to refer to the following words:

 a - able            n - nan
 b - baker           o - oboe
 c - charlie         p - peter
 d - dog             q - queen
 e - easy            r - roger
 f - fox             s - sugar
 g - george          t - tare
 h - how             u - uncle
 i - item            v - victor
 j - jig             w - william
 k - king            x - x-ray
 l - love            y - yoke
 m - mike            z - zebra

Alternative Approach

Some reasons to consider an alternative to the ISA, especially for more casual use, include:

1. Replacing the somewhat dated words and names with more contemporary and ‘politically correct’ alternatives
2. Using shorter words with fewer syllables to improve communication speed (especially given the higher quality of modern voice channels)
3. Selecting words with a verifiable minimum phonetic distance to optimise clarity

The approach used here is basically as follows:

1. Select a list of candidate words (at least one for each letter)
2. Convert them to a phonetic representation
3. Select one word for each letter from the candidate words that minimises word length and maximises phonetic distance

As will be seen each of these steps involve some unique challenges. Just selecting a pool of suitable words for the first step is more involved than you might expect.

The second step requires a dictionary with phonetic representation of the words. As mentioned, this blog is primarily focussed on English. But as any English speaker knows there are many dialects of English each with their particular pronunciation. So the phonetic dictionary used will influence the outcome, but in most cases this shouldn’t be a major issue. Of course the same approach could be used for any language which has a suitable phonetic dictionary.

The last step has three separate problems:

1. How to define the phonetic distance between two words
2. How to define a effective ‘cost/benefit’, or quality, function that captures the trade-off between word length and phonetic distance
3. How to select words to maximise this function

Disclaimer

Linguistics and its sub-field, phonology, are very mature subjects that have been extensively research and studied. The whole area has been re-examined and revised by the development of speech recognition technology that has introduce powerful mathematical tools and computing techniques that have further enhanced the field. So what follows should be seen, at best, as a simplification and generalisation of a much deeper and more complex discipline. Having said that, the use of convolution, a signal processing technique, to determine the phonetic distance between words (which accommodates phoneme duration) could be of some interest to the reader.

Finally, please note I’m not a professional linguist so don’t take the following as definitive.

Phonetic Dictionary

Carnegie Mellon University (CMU) has developed a large open source Pronouncing Dictionary. It has over 130,000 words presented in a format that is easy to process. It uses the ARPAbet phonetics set, which is based on US English, rather than the more general International Phonetic Alphabet but there are conversion tables between the two systems.

This post will use the CMU dictionary as its phonetic base. It is biased to US English pronunciation (not ideal for the Queen’s English purists:) but as we are only considering relative phonetic distances it shouldn’t be a significant issue.

Below is a small extract from the dictionary to show its structure:

 ...
 DOG  D AO1 G
 DOGS  D AO1 G Z
 DOGAN  D OW1 G AH0 N
 DOGBANE  D AO1 G B EY2 N
 DOGBERRY  D AO1 G B EH2 R IY0
 DOGE  D OW1 JH
 DOGEAR  D AA1 G IY0 R
 DOGEARED  D AA1 G IY0 R D
 DOGEARING  D AA1 G IY0 R IH0 NG
 DOGEARS  D AA1 G IY0 R Z
 DOGFIGHT  D AA1 G F AY2 T
 DOGFIGHTS  D AO1 G F AY2 T S
 DOGFISH  D AO1 G F IH2 SH
 DOGGED  D AO1 G D
 DOGGEDLY  D AO1 G AH0 D L IY0
 ...

The English language uses around 40 separate sounds or phonemes. They are divided into two main categories – consonants and vowels.

The CMU dictionary represents consonants as one or two character codes (starting with a consonants). Vowels are represented by three character codes (starting with a vowel) with the last character being a number indicating the degree of stress:

 0 - No stress
 1 - Primary stress
 2 - Secondary stress

Each word is on a separate line and comment lines begin with a semicolon (;).

To get a feel for the number and distribution of phonemes, Table 1 below shows phoneme counts and percentages, split by phoneme type–consonant (con) or vowel (vow)–and ordered by frequency (high to low) for the entire CMU dictionary. Note the vowel stress number has been ignored, that is, the vowel counts are the sum of the three different stress values for each vowel.

Table 1 CMU phoneme frequency

We can see there are 39 phonemes in total – 24 consonants and 15 vowels. The most commonly used consonant is N as in “night” and the most common vowel is AH as in “duck”.

The full table of CMU phonemes and examples of their use is shown in Table 2 below.

Table 2 CMU phoneme description

Phonetic Distance between Words

Let’s consider two words w1 and w2 which have an associated list of phonemes (as determine for the phonetic dictionary) of length m and n respectively:

w1 = p1, p2, ..., pm
w2 = q1, q2, ..., qn

There are various ways to define a distance between two sequences of elements (phonemes here). One of the best known is Levenshtein distance which counts the number of changes required convert one sequence to the other. However, in this case we would like a more nuanced approach that is capable of incorporating the following characteristics:

• differences between individual phonemes (i.e. some phonemes are more likely to be confused than others)
• consideration of the order the phonemes occur within the sequence
• ability to accommodate some notion of phoneme duration

This suggests treating the phoneme sequence as more of a signal. This coincides with the brain processes the sound of a word. Convolution is a common technique in signal processing and can be adapted here to provide an alternative way to define the distance between the two sequences. The basic idea is to ‘slide’ the two sequences over one another and to calculate a distance value based on the sum of the ‘distance’ between orthogonal pairs of phonemes. The minimum value become the distance for the sequences.

Consider an example of the two sequences (p1, p2, p3) and (q1, q2). These produce the four match-ups as the sequences slide across:

-- p1 p2 p3   |   p1 p2 p3   |   p1 p2 p3   |   p1 p2 p3 --
q1 q2 -- --   |   q1 q2 --   |   -- q1 q2   |   -- -- q1 q2

The distances for each of the four would be, in order:

where d(x,y) = distance between two individual phonemes and ‘–’ represents a null (or missing) phoneme.

The example above considers each phoneme as a single discrete unit with the same fixed length. More realistically, phonemes are of varying duration. The convolution approach can be extended to accommodate this by repeating the phonemes a number of times based on their duration. This is akin to sampling the word more frequently in a signal processing sense. Using the example above, with durations p1=3, p2=5, p3=4, q1=5 and q2=3, two consecutive example matches would be:

... |   p1 p1 p1 p2 p2 p2 p2 p2 p3 p3 p3 p3   |   p1 p1 p1 p2 p2 p2 p2 p2 p3 p3 p3 p3   | ...
... |   -- q1 q1 q1 q1 q1 q2 q2 q2 -- -- --   |   -- -- q1 q1 q1 q1 q1 q2 q2 q2 -- --   | ...

For brevity, the remaining convolution matches are not shown.

The next step is to define the distance between individual phonemes. There are three main types:

1. Consonants
2. Vowels
3. Null (‘–’ in the above examples)

Let’s assume all distances will fall in the range 0 to 1. That is:

As consonants and vowels are considered quite separate in the literature, we will make their distance the maximum 1.0. More precisely:

For null matches, the main consideration is that it should be set high enough to make a complete mismatch greater than a partial match so that there is always a minimum distance. Using the example above we want no overlap:

-- -- p1 p2 p3
q1 q2 -- -- --

to be a greater distance than at least one matching cases such as full overlap:

p1 p2 p3
q1 q2 --

If we generalise this to two phoneme sequences of length m and n with m ≥ n (without loss of generality) and let k = d(–,x) = d(x,–) we have (left hand side is no overlap and the right side is full overlap):

So we need k greater than or equal to 0.5 and less than 1.0. We can take the middle ground and define k = d(–,x) = d(x,–) = 0.75. Table 3 below summarises the distances for combinations of phoneme pairs where c₁ and c₂ are consonants, and v₁ and v₂ are vowels.

Table 3 Phoneme pair distances

For completeness, the distance between two nulls, d(–,–), is defined as zero, however, as this situation never arises in this analysis it is irrelevant.

Distance function

In mathematics, the distance function is a mapping of pairs for elements (x,y) to a real number with the following conditions:

It is clear from the distance table above that conditions 1, 2 and 3 apply provided d_c and d_v comply with these distance metrics. With this assumption condition 4 (triangle inequality) also holds as can be seen for the following table that enumerates all 27 combinations of a, b and c:

Table 4 Confirmation of the distance triangle rule for phoneme pairs

We now need to determine distances for pairs of consonants, d_c(c₁,c₂), and pairs of vowels, d_v(v₁,v₂). To maintain consistency with the distance criteria and conditions above d_c and d_v will need to:

• fall in the range 0 to 1 (inclusive)
• have a mean (average) of 0.5
• comply with the distance function conditions 1-4 above

To achieve this we will first develop a relative distance measure for pairs of consonants, and pairs of vowels, that comply with the distance function conditions. Their numeric values will then be uniformly adjusted so they have a range between 0 and 1, and have a mean of 0.5.

Phonetic Distance between Consonants (d_c) and Vowels (d_v)

Determining the relative distance between two consonants or two vowels is somewhat subjective. There is research that actual attempts to measure this directly through experimentation, for example Patterns of English phoneme confusions by native and non-native listeners – Cutler, Weber, Smits and Cooper 2004.

There are also many reference to metrics or features associated with consonants and vowels. For consonants it is common to associate them with various phonetic articulation features. For vowels it is usual to consider tongue position and lip roundness.

For this analysis we are going to use consonant features and vowel metrics to infer a numeric distance. For consonants, we assume the more features they have in common the closer they are, and conversely, the fewer features they share the further they are apart. For vowels, we will assume the more similar their tongue position and roundness, the closer they are.

Consonant Distance

Table 5 below is an example of the features associated with each consonant:

Table 5 Consonant features

There are various ways to turn this feature table into a distance measure but to maintain our distance analogy we will treat each feature as a dimension in a multi-dimensional space. A consonant’s coordinates in this space are set to one or zero depending on whether the consonant possesses that feature or not. We can scale each dimension by applying a weight or scaling factor. This provides a way to vary the relative importance of each feature and tune the distance values. Note scaling individual dimensions does not invalidate the distance conditions discussed above.

The distance between two consonants is calculated using the standard Euclidean distance function (in ℝⁿ space):

The weights (w_i) can be adjusted to normalise the resulting distances so that they fall within the range 0-1 and have a mean of 0.5. For those interested in the normalisation method see Appendix A. The result is the following consonant distance (d_c) table:

Table 6 Consonant Distance (d_c) Table

Table 6 can be downloaded in csv form here.

Vowel Distance

A similar approach can be applied to vowels only in this case the feature set is replaced with vowel metrics as discussed above. There are many metrics for vowels but we will focus on the three main ones:

• Openness (aka height)
• Backness (aka advance)
• Roundness

There is no exact value for these and they typically vary with different dialects. Here we will assign a value between zero and one for each metric for each vowel based on various vowel charts (see Table 7 below).

There is a further complication with vowels as they come in two main types:

• Pure vowels or monophthongs (EH, IH, AA, UW, AE, AH, ER, AO, IY, UH)
• Diphthongs (EY, AY, OW, AW, OY)

Pure vowels (monophthongs) are similar to consonants in that they have a single set of metric values. In contracts, diphthongs are a combination of two vowel sounds. They commence at one sound and end on another. Monophthongs can be thought of as points in the metric space, whereas diphthongs are line segments. Some of the diphthong starting and ending points are existing monophthongs but others are separate points not corresponding with any existing monophthong.

Table 7 below shows typical monophthongs vowel metrics derived from various sources. It includes three additional “virtual” monophthongs (DA, DE and DO) used by the diphthongs:

Table 7 Pure vowel (monophthong) metric data

As with consonants, the distance between two monophthong vowels is calculated using the standard Euclidean distance function:

The weights can be adjusted, in the same way as the consonants, to normalise the result (see Appendix A). Table 8 below shows the resulting monophthong vowel distances:

Table 8 Pure vowel (monophthong) distance table

To accommodate the diphthongs we begin with their starting and ending monophthongs as shown below:

Table 9 Diphthong starting and ending monophthongs

A natural way to handle diphthongs is to treat them as a straight line trajectory from the first vowel to the second vowel. This makes them a straight line segment in the metric space. To incorporate diphthongs into the vowel distance table we determine the average distance from a monophthong point to a diphthong line segment or the average distance from a diphthong line segment to another diphthong line segment.

These averages can be calculated either numerically, by assigning a (large) number of points over the line segment, or segments, calculating the distance to each point and averaging the result, or analytically, by defining the distance in terms of a variable and integrating (see Appendix B for details).

After adding the monophthong-to-diphthong and diphthong-to-diphthong distances, and removing the virtual diphthong vowels (DA, DE and DO) the final combined vowel table is:

Table 10 Combined monophthong and diphthong vowel distance table

Table 10 can be downloaded in csv form here.

Phoneme Duration

By their nature phonemes vary in duration. Their duration can be determined from experimental data. The duration times in Table 11 below are based data from Andras Kornai’s FORMAL PHONOLOGY.

For our purposes the duration times, measured in milliseconds, need to be converted into a number of repeated phonemes that reflect these times. Somewhat arbitrarily, a range from 5 to 15 repeats has been selected and the durations time have been “compressed” into this range to determine the repeats associated with each phoneme.

Vowels have three values for the three stress cases (p = Primary, s = Secondary and n = No stress). These were determined by making the the primary stress 20% greater, and no stress 20% less, than the secondary stress which took its base duration from the above reference. Repeats have been rounded to the nearest integer.

Table 11 Phoneme durations and repeats

One further refinement is to increase the duration of the final phoneme of a word if it is a vowel. There is a tendency to linger on the final vowel sound especially when the word is said in isolation. For example, consider the length of the “o” when saying “echo chamber” versus “echo” by itself. To accommodate this phenomenon the repeats of a final vowel are increased by 20%.

Word Distance

By combining the elements above, we are finally in a position to determine the phonetic distance between two arbitrary words using the following process:

1. Convert each word into a string of phonemes (using the CMU dictionary)
2. Repeat each phoneme in the string the number of times associated with it’s duration (table above)
3. Convolute the two strings and calculate the result using the vowel/consonant/null phoneme distance tables (see Appendix C for details on coding discrete convolution)
4. Take the minimum value of the convolution as the phonetic distance between the two words

This will produce a raw distance value. As a simple example consider the distance between “cat” and “mouse”.

1. Convert to phonemes:
cat   -> K AE1 T
mouse -> M AW1 S

2. Repeat phonemes based on their duration (vowel stress numbers removed for brevity)
cat -> K K K K K K K K K AE AE AE AE AE AE AE AE AE AE AE AE T T T T T T T T
mouse -> M M M M M M M AW AW AW AW AW AW AW AW AW AW AW AW AW AW AW S S S S S S S S S

3. Calculate the distance value for each case as the two strings "pass over" each other
... -  -  -  -  -  - - - - - - - - K K K K K K K K K AE AE AE AE ...
... AW AW AW AW AW S S S S S S S S S

... -  -  -  -  -  -  - - - - - - - K K K K K K K K K AE AE AE AE ...
... AW AW AW AW AW AW S S S S S S S S S

... -  -  -  -  -  -  -  - - - - - - K K K K K K K K K AE AE AE AE ...
... AW AW AW AW AW AW AW S S S S S S S S S
                                    :
                                    :
    - K K K K K K K  K  K  AE AE AE AE AE AE AE AE AE AE AE AE T T T T T T T T -
    M M M M M M M AW AW AW AW AW AW AW AW AW AW AW AW AW AW AW S S S S S S S S S

    K K K K K K K K  K  AE AE AE AE AE AE AE AE AE AE AE AE T  T T T T T T T - -
    M M M M M M M AW AW AW AW AW AW AW AW AW AW AW AW AW AW AW S S S S S S S S S

    K K K K K K K K K  AE AE AE AE AE AE AE AE AE AE AE AE T  T  T T T T T T - - -
    - M M M M M M M AW AW AW AW AW AW AW AW AW AW AW AW AW AW AW S S S S S S S S S
                                    :
                                    :
... AE AE AE AE AE AE T T T T T T T T - - - - - - -  -  -  -  -  -  -  -  -  - ...
... -  -  -  -  -  -  - - - - - - - M M M M M M M AW AW AW AW AW AW AW AW AW AW...

4. Select the smallest distance

The raw phonetic distance is not that useful as it is always possible to increase it by using longer words with more phonemes. The distance per phoneme is a more useful measure as it focuses on the distance between the phonemes themselves by discounting the length of the words.

However, for our purposes, we want to go one step further by imposing a cost on the length of words. That is, we want the greatest distance for the shortest words. So we can define a relative phonetic distance between two words (w₁ and w₂) as the raw phonetic distance divided by the average number of phonemes raise to some power λ (lambda for length sensitivity).

Note when λ = 0, rel-dist = raw-dist, and when λ = 1, rel-dist = raw-dist / avg-ph-cnt = raw-dist per phoneme. For high values of λ the relative distance is increasingly penalised (reduced) by the length of the words. This gives us a way to tune the relative distance to reflect how expensive word length is to the outcome.

To see how these measures work in practice, consider the following example comparing the relative phonetic distances between the word ‘cat’ and some other sample words. The number of phonemes in each word is shown in parentheses () after the word. The numbers in brackets [] after the distances are their order, or rank, (1=largest, 2=next largest, etc) within the column.

Table 12 Relative word distance for various λ values (with rank indicator)

This example confirms what we might expect from the distance measures considered so far:

1. Distance between “cat” and “cat” is zero (as would be expected)
2. Word pairs with more phonemes have larger raw distances (λ=0)
3. Relative distances place more emphasis on word lengths as λ increases. For example, “dog” goes from rank 4 to rank 1, and, “university” goes from rank 1 to rank 5 as λ increases from 0 to 3.

Another way to view this is to plot the relative distance of each word from “cat” (excluding “cat” itself) against λ. The plot in Figure 1 below uses a log scale to improve clarity. It shows how the longer words start with larger values but reduce more quickly as λ increases. Note also that words with the same phoneme count, such as “mat” and “dog” maintain their relative difference regardless of the value of λ.

Figure 1 Relative distance (log scale) from “cat” versus λ for some sample words

We can extend these ideas to groups of words instead of just pairs.

Word Groups

Before looking at ways to compare different groups of words, we should explicitly state two underlying assumptions we will use:

1. The greater the phonetic distance (as described above) between two words, the less likely they are to be confused
2. The number of phonemes in a word is a reasonable measure of the ‘cost’ of using (saying) the word

The first is the basis of the work above. We have implicitly assumed that the more differences in consonant features, vowel metrics and duration between words, the less likely the words will be confused. We can partly confirm this by our own experiences and partly by common sense. In reality, it is likely the degree of confusion between words is more nuanced, and shrouded in how our brains process and recognise sounds. However, for our purposes, the approach taken will have to suffice.

The second assumption is more straightforward. Although there are differences in the duration of individual phonemes, these tend to average out within a words. Further the transition between each phoneme also affects the duration of pronouncing the word. So the number of phonemes in a word is a reasonable measure of the cost associated with its reproduction.

Armed with these two assumptions, we can perform a cost/benefit analysis on groups of words. The cost is associated with the phoneme counts of the words, while the benefit is the phonetic separation achieved between them.

As a concrete example, let’s return to the six words used to represent the hexadecimal characters (A-F) discussed earlier.

Hex Words

The two tables below show the raw distance, and relative distance (λ=2), in parentheses, for each hexadecimal word pairings for the ISA and USPA abbreviations respectively.

Table 13 Raw and relative distance (λ=2) of the ISA hex words

Table 14 Raw and relative distance (λ=2) of the USPA hex words

The question is how best to compare these two schemes and, indeed, schemes in general?

As suggested above, we want to maximise the benefit/cost ratio (or minimise the cost/benefit ratio) where benefit is some function of distance and cost some function of the number of phonemes. We also may want to consider the frequency or probability of words, that is, how often a word (or its associated letter) occurs. For hexadecimals, and many other real life scenarios (e.g call signs), this is expected to be approximately the same for each letter/word. But in some contexts, particular letter/words may be used more frequently than others.

Cost

Let’s start with the cost side of the equation. As we are assuming the cost is proportional to the number of phonemes in the word, the cost of saying the entire set of words is the sum of the probability (p) multiplied by the phoneme count (c) of each word.

When the probability of each word is equally likely, as in the case of hexadecimal letters, p_i = 1/n and the cost equation becomes a simple arithmetic mean:

This is a simple and effective cost measure. It would be possible to emphasis, and penalise, more costly (longer) words by using the generalised or power mean. It raises each value to some power before summing them. This inflates larger values and increases their relative contribution to the sum (assuming the exponent is greater than one). To normalise the result, the root is taken to the same power as shown below:

However, for our purposes we will stick to the simple arithmetic mean of the phoneme counts.

Benefit

Now let’s consider the benefit side of the equation for a group of words.

If there are n words in the group, each word has n-1 neighbours and n-1 corresponding distances to them. So there is a total of n(n-1)/2 distances between the n words. To simplify matters, we will focus on the distance between a word and its nearest neighbour. This is the minimum distance associate with each word. It can be interpreted as a measure of the likelihood the word will be confused with another. It also means we only need to consider n distances for the benefit equation.

Figure 2 below shows graphs of the ISA and USPA words (nodes) with the links (edges) between them representing the raw distance from the table above. The minimum distance edges for each word are coloured along with their associated nodes (not to scale). It can be seen that the minimum minimum distance (red edge) is always shared between two nodes. While other nodes can either share or have their own minimum distance.

Figure 2 Minimum distance between ISA and USPA words

The result of this approach is that each letter/word is assigned a single distance value representing its worst case separation denoted d.

We can treat this minimum distance in a similar way to the cost phoneme count. Again the probability of the word being used can be similarly accommodated.

Of course there a many other ways the cost and benefit equations could be formulated. There is no “correct” approach, but each should provide similar general results–reducing the cost for shorter words and increasing the benefit for greater separation–with only the quantum varying.

Quality Measure

The ratio of benefit to cost provides a quality measure for a set of words. The better the separation the greater the quality. The shorter the words (few phonemes) the greater the quality. The sensitivity to word length can be tuned with the λ parameters as follows:

To simplify matters we will consider three “special” cases of the quality measure assuming equal likelihood of letters/words:

The first (qual₀) is just the average minimum distance. It takes no account of the length of the words. Some constraint on the word length can be externally imposed by limiting the length of the candidate words available for selection.

The second (qual₁) is the average minimum distance per phoneme. It takes basic account of the word length meaning longer words are not necessarily better than shorter ones but it places no particular penalty on longer words.

The last (qual₂) places more emphasis on word length by penalising longer words. This is the most useful for our purposes as it represents the most economic measure.

We are now in a position to assess the two hexadecimal scheme above using these quality measures:

Table 15 Hex word quality comparison

These results show that while the average minimum distance (qual₀) for ISA are greater than USPA, USPA improves as λ increases because its average phoneme count is significantly smaller. These figures suggest that for hexadecimal words, at least, USPA is a significantly more economic scheme (by about 44%).

We will look at some alternative hexadecimal schemes later that have even better economy. Before we can create new schemes, however, we need a source of candidate words.

Candidate Words

For each letter of the alphabet we need a set of suitable words to represent it. The words must:

1. Start with that letter (i.e. p for pot)
2. Sound the letter as expected (i.e. p for pot but not p for physics nor e for europe)

To achieve the second condition, an acceptable first phoneme (or phonemes) is assigned for each letter. The following table show the proposed phoneme mapping.

Table 16 Initial word phoneme(s) for each letter

Note:

1. There can be multiple phonemes to cover the case where a vowel, for example, sounds differently at the beginning of a word, for example, echo (EH) and easy (IY)
2. The letter C uses the CH sound (as in check) instead of the K sound (as in cat) to distinguish it from the letter K which uses the K sound (as in kite)
3. The letter X is a special case, as it invariably sounds like Z when used at the start of a word (e.g. xenon, xerox and xylophone). The one notable exception is x-ray which perforce will be used for the letter X.

Using the first letter and phonemes listed above we can assign words to letters. However, there are a few challenges in creating an suitable list of candidate words. Many of the words in the CMU are not suitable for various reasons and would make poor candidate words. For example, words that fall into any of the following categories may not be considered suitable:

• Uncommon, obscure, archaic or foreign
• Derogatory, racist, sexist, rude, indecent, swearwords, slang or intimate
• Negative, depressing, anti-social
• Technical or specialist in nature such as anatomical or botanical terms
• Warning or emergency (e.g. fire, help, danger)
• Numbers (e.g. one, two, …)
• Place names such as countries, cities, rivers, etc.
• Personal names (although these can be useful because they are often well known)
• Company names, trademarks or product names (with possible legal implications)
• Non root words (e.g plurals and other derivatives)
• Religious, sacred or iconic
• Abbreviations, acronyms or colloquial terms
• Homonyms and homophones
• Overly short or long words

This list is probably not exhaustive, in short, there are many and varied reasons to exclude words. Perhaps it is easier to define acceptable words as: common simple root words with a neutral or positive character that are unlikely to be offensive or symbolic to most people.

Consideration as to how the words will be used is also important. If they are for ad hoc use, in casual situations without a prior formal agreement, common and familiar words will be more effective. However, if the scheme is going to be used by a formal agreement between parties (e.g. ISA), it is more acceptable to use less common words as they will be familiar and expected by the parties involved. Indeed, somewhat obscure words may even be preferred as they are more readily recognisable as letter substitutes.

The CMU dictionary is full of words that would be caught in the unsuitable categories above. In particular, it has a large number of abbreviations, proper nouns and derivations. So as a more concise and manageable source of words we will use Google 10,000 most common English words as a base source. Of course the Google words also contain many words that fall into the undesirable categories above. As a result, a number of words have been excluded from the Google set to produce a subset for use in the examples below.

Alternative Hexadecimal Word Sets

Can we find alternative hexadecimal word sets to ISA and USPA that produce a higher quality result?

A simple approach would be to try every combination of the words representing the letters A-F and select the combination with the highest quality score. A little thought will reveal this has complexity scaling problems. If we assume about 10,000 / 26 (approx. 385) words per letter, there will be around 385⁶ or 3.25 x 10¹⁵ word combinations to analyse. Searching this entire solution space is problematic even with a fast computer and would be impossible with larger word groups like A-Z for the full alphabet.

Rather than looking for the optimal solution, we can find a sub-optimal, but hopefully still good solution, by reducing the size of the search. One simple approach is to:

1. Select a word at random for each letter from the word candidates.
2. Order the words from smallest to largest based on their minimum distance (i.e. distance to their nearest neighbour). When this is equal (which always occurs at least once) order by the number of candidate words available, least to most. So select the word with the closest neighbour and least number of alternative words first.
3. For the selected word, try replacing it with each of the available alternative words from the candidate words. Calculate the resulting quality measure for the new word set and select the best alternative word based on this measure, or leave the current word if no improved word can be found.
4. Repeat the above step for each word in turn.
5. If there are no improvement found for any word finish, otherwise loop back to Step 2.

This is a simple form of gradient accent where each dimension (letter/word) is examined in turn for “higher ground”. Once no further improvement can be found we have reached a local maximum. By repeating the process for different random starting points we can search for better local maximums.

Some results from applying this approach to candidate words restricted to 3-5 phonemes using our three quality measures above (qual₀, qual₁ and qual₂) are shown below in Tables 17, 18 and 19 respectively.

Local Maximum λ=0 (Hex35-λ0)

Table 17 Sample qual₀ local maximum for hex words using 3-5 phonemes

Local Maximum λ=1 (Hex35-λ1)

Table 18 Sample qual₁ local maximum for hex words using 3-5 phonemes

Local Maximum λ=2 (Hex35-λ2)

Table 19 Sample qual₂ local maximum for hex words using 3-5 phonemes

The Hex35-λ0 solution optimises for qual₀ where word length is ignored, so it only requires maximising the distance between words. Hence the result uses the longest words available – in this case five phonemes.

The Hex35-λ1 solution optimises for qual₁, where the distance per phoneme is important, so the word length is essentially irrelevant. As a result we see a variety of word lengths.

Finally for the Hex35-λ2 solution which optimises for qual₂, the word length penalises the result more and the solution favours words with the smallest phoneme count (while still trying to maximise separation distances).

Table 20 below compares these three local maximum solutions with the ISA and USPA schemes.

Table 20 Comparison of ISA, USPA and three local maximum word sets

We can see that the generated solutions deliver significantly better results in their quality category (highlighted cells) than either ISA or USPA. It also highlights that the solution produced using the local maximum procedure above is not necessarily the true maximum solution. The Hex35-λ2 word set is actually a better solution than the Hex35-λ1 word set for the qual₁ measure. So Hex35-λ1 is clearly sub-optimal.

These generated word solutions are, of course, only one example of many possible word combinations that also score highly. Listed below is Hex35-λ2 and some other results with high qual₂ scores in descending order:

• act, bar, choice, duo, easy, fly (2.38)
• arrow, bug, choice, duo, easy, fly (2.31)
• arrow, booth, choice, dry, easy, far (2.30)
• acre, beach, chuck, duo, end, fly (2.25)
• amy, beach, choice, dry, echo, far (2.24)
• ask, beach, choice, duo, era, fly (2.23)
• annie, blue, choice, dock, east, fill (2.21)
• arrow, bouquet, chase, door, easy, fly (2.15)
• alloy, blue, chief, doll, end, folk (2.12)

Finally, as a baseline of sorts, let’s include a random sample of words with 3-5 phoneme counts. Table 21 below includes the average results (rand-35) of 1000 words sets drawn randomly from the same 3-5 phoneme word pool used by the local maximum solutions.

Table 21 Includes average results from 1000 random samples taken from 3-5 phoneme words

It is interesting to note that for the six hexadecimal letters, ISA scores worse than a random sample on two of the three quality measures. USPA is marginally better on all three. All the constructed word sets score significantly better than the random samples for their optimised measure, and indeed better for the other measures as well.

Full English Alphabet

The methods used above can be extended to the entire 26 letters of the English alphabet. First lets compare the full ISA and USPA schemes with the average of 1000 random samples take from the 3-5 phoneme word pool in the same way we did for the hexadecimal case.

Table 22 Comparison of full ISA, USPA and average random sample from 3-5 phoneme words

The ISA scheme results are better here than the hexadecimal case for qual₀ and qual₁ against both USPA and the random samples. However for qual₂ it falls below both. This reflects the significantly longer words used in ISA, averaging 4.96 phonemes compared with USPA’s 3.81 and 4.02 for the random samples.

Strangely, ISA has a number of words with the same ending sound. It has six words ending in AH, five in OW and three in ER and IY:

• AH: alpha, delta, india, lima, papa, sierra
• OW: bravo, echo, kilo, romeo, tango
• ER: november, oscar, victor (noting ER is quite close to AH)
• IY: charlie, whiskey, yankee

It is not clear whether this was by design or a coincidence, but it reduces its quality results as defined above. In comparison, USPA has five words ending in ER with all the remaining only one or two words:

• ER: baker, peter, roger, sugar, victor

The full “histogram” of finals sounds (ordered high to low) for each is:

• ISA: 6 5 3 3 2 2 1 1 1 1 1
• USPA: 5 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1

Constructed Alternatives

The same local maximum approach used for hexadecimal words above can be applied to the entire alphabet (A-Z). It produces a wider variety of possible solutions. Most have a mixture of “good” and “not-so-good” words based the categories above. One of the main issues is whether to include personal names (e.g. John, Kate, Howard, etc.). The upside is that names provide a richer set of phoneme combinations and increase the available word pool. The down side is that names are not standard english (i.e. not generally found in dictionaries) and some are more familiar than others to the general population. I’ve chosen to include them in this case.

Table 23 below shows two examples of local maximums for each of our three quality measures (labelled -a and -b). All words are again drawn from the 3-5 phoneme word pool.

Table 23 Comparison of full ISA, USPA and two local maximums optimised for λ=0, 1 and 2

Clearly it is possible to find word sets that achieve better results than both ISA and USPA using the distance measures defined in this blog. In fact, the problem is choosing between the large number of good solutions available.

Appendix D lists some high scoring word sets using different criteria for candidate word pools and quality measures. I’ll leave it to the reader to choose their favourite.

Conclusion

The idea of using a quantitative measure to evaluate, and generate, spelling alphabets seems like a relatively straightforward exercise. However, as this post demonstrates, there are a number of complexities that make it more challenging than expected. These include:

• Mapping letters to words and their associated phonemes
• Defining a distance (i.e. difference) measure between phonemes
• Defining a distance measure between two words (i.e. two phoneme sequences)
• Defining a distance and quality measure between two groups of words
• Determining a pool of candidate words (for generation)
• Selecting words from the candidate pool to maximise a quality measure

An attempt to systematically address each of these is included in this blog.

Perhaps the most innovative element is the use of discrete convolution to measure the phonetic distance between two words. This allows different phoneme durations to be accommodated in a natural way. How well the word differences, determined by this convolution, compare with actual human perception is an open question. However it does provide a robust quantitative comparison method.

The analysis does suggest that ISA is far from optimal based on the presented measures. Even without this analysis its strange preference for words ending with AH (6 instances) and OW (5 instances) phonemes is a sign that improvement is possible. Of course its near universal adoption makes any change all but impossible. However for informal use, readers could select any of the options in Appendix D and be confident that there is at least some science behind their choice.

APPENDICES

Appendix A

Distance Normalisation and Scaling

Given a set of m points P ⊂ ℝⁿ and a set of weights {w₁, w₂, …, w_n} that scale each of the n dimensions, the Euclidean distance function for any two points x, y ∈ P is:

The requirement is to adjust the weights (w_i > 0) so that in general terms:

1. The contribution of each dimension is about equal that is:
   • inflate dimensions that are numerically small compared with others
   • deflate dimensions that are numerically large compared with others
   • don’t just make a w_i very small and convert the problem from ℝⁿ to ℝ^n-1
2. The maximum distance between any two points is 1.0
3. The average (mean) distance between points is 0.5

More formally:

Firstly note that the entire space can be scaled by a single factor, k, to expand or contract it. This factor can thus be used to achieve either condition 2, by scaling the maximum distance to 1.0, or, condition 3, by scaling the average to be 0.5, but not both. To achieve both we need to apply a non-linear monotonic function to the weights. Here we choose to raise them to a power r.

The first condition gives the following equation for the weights:

Substituting this back into the original distance equation gives:

Now for any given power r (eg. r = 1.0) it is easy (with a computer :) to determine k so that the average distance is 0.5 using:

Now, by inspection of the maximum distance, r can be adjusted to move it towards its target value of 1.0. Repeated iterations using a binary search (chop) approach allows r to be determined to any appropriate level of accuracy. Once r is determined, k is known and the final weights can be calculated.

This approach may not be the most efficient but as it only needs to be carried out once it is not that critical.

Appendix B

Mean Distance between a Point and a Line Segment

The distance between a point and a line segment can be calculated numerically or analytically. Numeric approach provides an approximation by dividing the line segments into a number of equally spaced points and taking the average of the distances from the point to the line segment sub-points. Increasing the number of sub-points improves the accuracy of the result.

The analytical solution proves to be a bit trickier than one might expect. One approach is given below although alternative solutions that are more straightforward may be possible.

Consider four points P, Q, R and S in ℝⁿ and their corresponding vectors p, q, r and s. R and S are the start and end of a line segment. Q is a point on the line segment and P is the reference point. The Euclidean distance between P and Q, or the magnitude of p–q (written p–q) is d as shown in the diagram below.

We want the mean (average) distance between P and the line segment RS.

Using the parametric definition of Q we have q = r + x(s – r) so that as x ranges from 0 to 1, q ranges from r to s.

Define the distance function, d(x), to be the magnitude of PQ (p–q) for a given x, then the mean distance is the integral of d(x) between 0 and 1. Below we use standard vector dot product algebra to expand d(x) in terms of r, s and p, and then group them in powers of x to form a quadratic equation.

The integral above is a bit messy but available from various mathematical applications or sites. The following is courtesy of WolframAlpha:

The mean distance can thus be calculate by plugging in the values of a, b and c and subtracting the integral’s value at x=0 from its value at x=1.

After coding up both approaches, there was good agreement (better than 10^-4) between them using 1000 sub-points for the numeric approximation.

Mean Distance between Two Line Segments

The approach taken above can be readily applied to the case of the mean distance between two line segments. Consider the following diagram.

Line segments TU and RS have parametric points P and Q on them respectively. Their corresponding parametric vectors (p and q) move from T to U and R to S as x varies from 0 to 1. As above, we define the distance d between P and Q in terms of x and integrate over the range 0 to 1 to get the average distance.

More formally:

We can plug these values into the integral above to calculate the result.

Appendix C

Coding Discrete Convolution

Coding the convolution of two strings (or any two sequences of discrete values) can appear a bit daunting. However it is actual straightforward if you:

• reverse the second string
• lay them out in a grid, with the first string at the top and the reversed second string on the left
• calculate the value of each cell using the corresponding horizontal and vertical character (or sequence value)
• sum each of the grid diagonals to get the final convolution value

It is even easier to program if you note that the diagonal cells all have the same index sum value (i.e. the sum of the two corresponding sequence indices). This index value can be used to group the cell results into diagonals for summing.

An example will make it clear. Consider the convolution of the two strings abcd and xyz. Using the above procedure we obtain a grid as follows:

The xyz string is reversed to zyx and the δ_ij value in the cells represent the distance between the i and j character (phoneme in our case). The small red numbers in the top right corner of the strings represents the index value of the phoneme in the string. The same numbers in the cells represent the sum of the indices of the two corresponding string phonemes. This sum can be used to programatically identify the diagonal associated with each cell. The diagonals have also been coloured for clarity. The convolution values are just the sum of the diagonals:

• convolution₀ = δ_az
• convolution₁ = δ_ay + δ_bz
• convolution₂ = δ_ax + δ_by + δ_cz
• convolution₃ = δ_bx + δ_cy + δ_dz
• convolution₄ = δ_cx + δ_dy
• convolution₅ = δ_dx

These correspond to the string positions shown below as the two string pass each other.

Example Clojure code might look like:

(defn convolution
  "Convolution of two sequences applying the function 'conv-fn'
  to adjacent values and summing the results."
  [seq1 seq2 conv-fn]
  (let [vec1 (vec seq1)
        vec2 (vec (reverse seq2))
        res (reduce
              (fn [acc [i1 i2]]
                (let [cell (conv-fn (vec1 i1) (vec2 i2))]
                  (update acc (+ i1 i2) (fnil + 0) cell)))
              {}
              (for [i1 (range (count vec1))
                    i2 (range (count vec2))]
                [i1 i2]))]
    (vals (sort res))))

Appendix D

Generated Example Word Sets

The following are some generated word sets using different criteria for the length of the candidate words (phoneme count) and the quality setting (λ value).

Sample 4-phoneme word sets maximising qual₀ (λ=0)

The following local maximum word sets use words with exactly four phonemes. Their raw distance (qual₀) is shown in brackets.

• alpha, boost, charm, diary, eric, flower, graph, hockey, icon, july, kilo, lounge, maria, neon, ozone, paper, queen, royal, score, today, under, virtue, wax, yoga, zinc (20.18)
• aqua, black, charge, diary, epic, flour, growth, hockey, image, july, kind, lewis, maria, neon, ozone, paper, queen, royal, spray, trio, under, virtue, willow, yoga, zinc (20.15)
• aqua, below, chance, diary, epic, flower, graph, hockey, image, joel, karl, launch, maria, neon, ozone, paper, queen, relay, scoop, trio, under, virtue, wealth, yoga, zinc (20.03)
• asset, broad, change, diary, eagle, flour, golf, hockey, image, july, kerry, lewis, maria, neon, ozone, paper, queen, royal, spice, trio, under, virtue, willow, yoga, zinc (19.88)
• aqua, bound, charge, deny, epic, flower, gamma, hobby, ivory, joel, kilo, lucy, mother, neon, ozone, paper, queen, relay, score, track, under, virtue, wealth, yoga, zinc (19.81)
• aqua, bother, chart, deny, eagle, flour, group, health, ivory, joel, kerry, launch, movie, neon, office, parade, queen, relay, score, trio, under, virtue, willow, yoga, zinc (19.80)
• aqua, breach, child, data, eric, flower, growth, howard, ivory, joel, kilo, launch, movie, neon, office, poker, quiz, rally, snap, today, under, virtue, wolf, yarn, zinc (19.73)
• alpha, bloom, change, depot, every, father, guru, hollow, icon, july, kerry, louise, march, neon, ozone, price, quit, royal, score, trio, under, virtue, wax, yoga, zinc (19.69)
• aqua, broad, change, diary, epic, flower, guru, hockey, image, joel, kept, luther, march, neon, ocean, paper, queen, relay, spice, trio, under, virtue, wolf, yellow, zinc (19.63)
• alpha, below, charge, drag, eric, flower, guru, highway, icon, july, kept, louise, money, nature, ozone, poem, quiz, round, spray, trio, under, visa, wales, yoga, zinc (19.58)
• alice, baker, change, dial, even, form, glad, health, ivory, july, kilo, louise, mother, neon, ozone, prior, quick, robbie, spouse, truth, under, virtue, wax, yoga, zinc (19.51)
• asset, baker, change, drag, every, flower, ghost, health, image, july, kilo, laura, maria, neon, ozone, papa, quiz, royal, speech, today, under, virtue, worthy, yarn, zinc (19.50)
• aqua, bloom, change, duty, eric, flower, graph, highway, image, july, kilo, louise, marsh, neon, often, poet, quiz, river, spray, trio, under, virtue, worthy, yoga, zinc (19.47)
• aqua, block, change, depot, eagle, flower, guru, health, ivory, july, karl, louise, mother, navy, office, past, quit, ryan, spray, truth, under, virtue, willow, yoga, zinc (19.39)
• asset, breath, change, diane, every, flower, guard, health, image, jewel, kilo, laura, movie, neighbour, ocean, poker, quote, rally, speech, today, under, virtue, wax, yeast, zinc (19.32)
• aqua, boost, charge, diary, epic, flag, guru, hello, image, july, kind, laura, marine, never, ocean, paper, quote, royal, spray, trio, under, virtue, worthy, yield, zinc (19.13)

Sample 3-phoneme word sets maximising qual₀ (λ=0)

The following local maximum word sets use words with exactly three phonemes. Their raw distance (qual₀) is shown in brackets.

• alloy, batch, cheap, door, easy, free, grow, height, ink, joke, ken, leo, myth, noise, oboe, pale, ross, spa, tower, view, wool, young, zoom (14.90)
• alloy, batch, choice, draw, easy, file, gate, hear, ink, judge, kim, leo, more, neck, oboe, play, ross, spa, tower, view, wool, yoke, zoom (14.74)
• alloy, batch, choice, duo, easy, fish, grey, hub, ink, jeep, ken, leo, mouth, nail, over, power, ross, spa, tall, view, wool, yoke, zoom (14.68)
• alloy, bang, cheap, door, east, full, gauge, house, ion, joke, king, leo, mug, noise, oboe, power, ruth, spa, tree, view, watch, year, zen (14.65)
• amy, batch, check, dog, era, free, grow, heath, ink, juice, keen, leo, mime, noise, oboe, power, ross, spa, tour, view, whale, young, zip (14.59)
• alloy, beach, choice, duo, easy, file, grey, hair, ion, jam, kate, leo, merge, nut, over, power, rock, spa, tall, view, wing, yoke, zoom (14.58)
• amy, beach, choice, draw, era, file, give, hall, ink, jar, kirk, leo, mouth, nan, oboe, play, race, spa, tower, view, watch, young, zoom (14.49)
• amy, blue, chief, dry, era, full, gauge, heel, ink, joan, kirk, leo, mouth, nan, oboe, power, rod, ski, touch, voice, walk, youth, zip (14.49)
• alloy, beach, choice, duo, easy, fame, grey, hood, ink, judge, kirk, long, mouth, near, oboe, poll, ruth, spa, tower, view, wife, yacht, zen (14.45)
• alloy, bush, chat, draw, east, free, gauge, hull, issue, joke, kick, leo, mayor, neil, over, paul, rock, sky, touch, voice, worth, year, zen (14.43)
• amy, bar, chief, dog, era, foam, grow, height, ink, jam, kirk, leo, match, noise, oboe, pale, rule, sky, tree, view, wire, young, zip (14.41)
• arrow, beach, chat, down, easy, few, game, hood, ink, june, kiss, leo, merge, near, over, play, rock, spa, tall, voice, wire, yoke, zen (14.38)
• alloy, batch, chase, dare, easy, firm, grey, hawk, ink, judge, keen, leo, moon, noise, oboe, power, ripe, spa, toll, view, whale, yacht, zip (14.35)
• amy, ball, choice, debt, era, flow, give, hire, ink, jar, kirk, leo, mouth, neil, oboe, page, rope, spa, tree, view, watch, young, zoom (14.30)
• alloy, bull, chase, deal, east, fare, grow, house, issue, joan, kirk, log, mayor, none, over, power, ram, spa, tree, void, watch, youth, zip (14.29)
• annie, beth, chase, duo, echo, few, goal, hood, ink, jar, keen, layer, mouth, night, over, patch, run, spa, tree, voice, walk, year, zoom (14.26)
• alloy, blue, cheap, dry, east, fog, guide, hull, ion, joke, king, leo, mouth, net, oboe, power, rush, ski, tall, voice, wage, year, zoom (14.13)
• alloy, batch, chase, dose, east, fare, grow, hire, issue, judge, keen, leo, mouth, nerve, owen, push, ross, spa, tree, void, walk, youth, zip (14.12)
• alloy, bush, chess, draw, east, few, gauge, hair, ion, june, kirk, lower, mouth, neil, oboe, play, ram, spa, tall, voice, watch, young, zip (14.10)
• alloy, blue, cheese, down, east, fell, goal, hawk, iron, jar, king, leo, mayor, none, oboe, patch, ride, spa, tree, voice, wake, youth, zip (14.08)

Sample 3-4 phoneme word sets maximising qual₂ (λ=2)

The following local maximum word sets use words with three or four phonemes. Their qual₂ (λ=2) is shown in brackets.

• alloy, beth, chip, draw, easy, free, girl, hire, ink, join, keen, leo, mouth, near, oboe, page, quote, run, sky, talk, under, view, wood, yacht, zoom (1.56)
• alloy, bell, choice, draw, easy, few, goal, hawk, ink, jack, kind, leo, mouth, neighbor, oboe, play, queen, rich, spa, teeth, under, virtue, wire, yacht, zoom (1.56)
• alloy, beth, chase, draw, easy, file, goal, hawk, ink, job, keep, leo, match, noise, oboe, play, quote, rear, spa, tower, under, view, wood, young, zoom (1.56)
• alloy, beef, chain, draw, easy, free, good, hire, ink, joke, kept, leo, mouth, nature, oboe, paul, quote, rock, sky, touch, under, view, worm, year, zoom (1.56)
• alloy, batch, choice, doug, easy, fall, grow, height, ink, jeep, kim, leo, mail, never, oboe, play, queen, ross, spa, town, under, view, wool, yoke, zoom (1.56)
• alloy, bench, cheat, draw, easy, feature, girl, house, ink, join, kilo, leo, myth, north, oboe, play, quick, rock, spa, tower, under, view, wage, young, zoom (1.55)
• alloy, blue, chair, doug, easy, feature, grey, hall, ion, joke, kim, leo, merge, nail, oboe, power, queen, rack, spa, tooth, under, voice, wood, yacht, zinc (1.55)
• amy, beth, choice, depot, era, full, grey, home, ink, july, keen, leo, more, none, oboe, power, quote, rack, spa, tooth, under, view, wage, yacht, zip (1.55)
• alloy, bath, choice, down, east, free, goal, hawk, icon, jake, keen, leo, moon, nature, oboe, push, quote, ross, spa, today, under, view, wire, young, zip (1.55)
• alloy, bull, choice, dog, easy, fiber, grow, hire, ink, jack, karl, leo, myth, noon, oboe, play, queen, rope, spa, town, under, view, wage, young, zen (1.54)
• alloy, beth, chat, draw, easy, free, gauge, hall, ink, joke, kerry, leo, mayor, near, oboe, peak, quiz, rock, sky, touch, under, voice, wine, yard, zoom (1.53)
• alloy, box, cheap, dash, east, file, grey, home, issue, jar, ken, leo, myth, nature, old, power, quote, roof, spa, tour, under, voice, wall, young, zinc (1.53)
• alloy, bull, chase, draw, easy, feature, grey, hawk, ink, job, keen, leo, match, noise, oboe, pair, quick, roof, spa, tower, under, view, won, yard, zip (1.52)
• alloy, ball, cheap, draw, east, few, gary, hood, iron, june, kind, leo, major, none, oboe, pitch, quick, rock, ski, tower, under, voice, worth, yoke, zinc (1.52)
• apt, book, chief, draw, easy, fall, guru, house, ivory, jack, ken, leo, mail, noise, old, play, queen, ridge, spa, touch, under, view, wire, yacht, zinc (1.52)
• alloy, bull, choose, dome, east, feature, grow, hear, issue, jake, kerry, leo, more, noise, oboe, play, queen, rug, spa, tower, under, van, watch, youth, zip (1.52)
• alloy, bull, choice, duo, east, free, gale, hawk, issue, jack, ken, logo, merge, north, owen, power, quote, ross, spa, teach, under, visa, wife, young, zip (1.51)
• alloy, bull, choose, duo, east, fairy, guard, hat, icon, join, keep, log, mouth, nature, oboe, poker, quote, rich, spa, tray, under, view, wire, young, zinc (1.50)
• amy, blue, chain, dock, east, fall, grey, hub, isaac, july, kick, leo, memo, neil, old, patch, quote, rare, spa, tiger, under, voice, wire, youth, zinc (1.50)
• act, bull, chat, door, emma, foam, grow, house, issue, judge, ken, leo, meat, noise, owen, play, quick, rock, spa, taylor, under, vary, wire, youth, zinc (1.49)

Sample 4-5 phoneme word sets maximising qual_1.5 (λ=1.5)

The following local maximum word sets use words with four or five phonemes. Their qual_1.5 (λ=1.5) is shown in brackets.

• alpha, bloom, child, dark, every, flour, graph, hockey, icon, july, kevin, launch, mother, neon, oscar, paper, quit, royal, score, trio, unique, virtue, willow, x-ray, yoga, zinc (2.50)
• amber, block, child, dual, eden, fridge, growth, holly, indoor, july, kilo, launch, marco, nylon, ozone, piano, query, ratio, spouse, taxi, unique, virtue, weather, x-ray, yoga, zinc (2.48)
• alex, bound, charge, diary, enter, flower, glass, hockey, image, july, kilo, leon, maria, nursery, ozone, prove, quiz, radar, score, tango, unique, virtue, wealth, x-ray, yoga, zinc (2.48)
• anchor, bother, cherry, diary, ethics, flower, gourmet, horse, image, july, karma, lounge, marine, nylon, ozone, poem, quit, ratio, school, track, unique, virtue, willow, x-ray, yoga, zebra (2.46)
• aqua, below, change, dairy, every, flower, grill, host, image, july, kitchen, louise, mother, neon, office, proper, quote, royal, speech, tank, under, virtue, width, x-ray, yarn, zebra (2.46)
• alpha, baby, change, dial, eric, flour, group, hammer, icon, july, kilo, louise, maria, north, ozone, poster, quiz, research, stock, tango, unique, virtue, want, x-ray, yoga, zinc (2.45)
• alpha, birthday, change, derek, emily, flower, ghost, highway, image, july, kilo, large, marine, nylon, ozone, patio, quiz, royal, spot, truth, unique, virtue, water, x-ray, yield, zinc (2.45)
• amateur, block, change, dual, epic, fully, growth, howard, ivory, jackie, kitchen, lucia, morris, neon, ozone, post, queen, recall, spice, trio, under, virtue, willow, x-ray, yard, zinc (2.45)
• alex, bother, chapter, daisy, every, flag, growth, heater, invoice, july, karma, louise, most, neon, often, powder, quit, royal, score, trio, unique, virtue, wild, x-ray, yellow, zinc (2.45)
• alex, beneath, charlie, decade, every, flower, greg, hockey, icon, july, kind, louise, major, neon, ozone, proper, query, royal, scoop, trio, under, virtue, warm, x-ray, yellow, zinc (2.44)
• alex, bound, charge, daily, eagle, father, group, hockey, ivory, july, kitchen, logo, matthew, neon, office, pressure, quiz, royal, stock, trio, under, viking, warm, x-ray, yellow, zinc (2.44)
• alpine, blake, charter, dual, every, forge, growth, howard, india, july, kitchen, lobby, month, nylon, office, prior, query, railway, scott, tank, uncle, virtue, willow, x-ray, yoga, zebra (2.43)
• aqua, brush, change, donor, eric, flyer, ground, howard, ivory, joel, kitchen, louise, manage, north, oscar, papa, query, ratio, spoke, today, unique, virtue, willow, x-ray, yeast, zinc (2.41)
• alice, bunch, chart, dating, emily, flower, growth, hockey, input, july, kilo, lewis, module, neon, order, parade, queen, rank, score, trio, unique, value, weather, x-ray, yoga, zebra (2.41)
• aqua, block, chance, daughter, enzyme, flyer, growth, howard, image, jewel, karl, larry, marine, nylon, oscar, piano, quit, railway, spouse, trio, unique, virtue, willow, x-ray, yeast, zinc (2.41)
• april, brad, change, daily, every, flower, gather, hockey, item, july, karl, laura, mold, neon, ozone, pillow, quote, royal, speech, tango, under, virtue, wealth, x-ray, yeast, zinc (2.41)
• andy, box, channel, dual, even, flyer, graph, horn, icon, july, kilo, lounge, major, notice, oscar, parade, quick, rubber, speech, tango, unique, virtue, worthy, x-ray, yield, zinc (2.41)
• alpha, body, charge, dual, emily, folder, guitar, highway, image, joint, kilo, luther, maria, nursery, ozone, piano, quiz, ratio, speech, trout, unique, virtue, warm, x-ray, yeast, zinc (2.41)
• alex, block, change, decor, enzyme, flyer, grill, hobby, image, jewel, karma, leon, module, nursery, oscar, parade, quote, rover, speech, tango, unique, viking, width, x-ray, yellow, zebra (2.40)
• after, broad, charge, delay, eric, future, gather, hockey, icon, jamie, kitchen, louise, month, neon, oval, parade, quiz, rainbow, solo, trio, unique, virtue, wolf, x-ray, yeast, zinc (2.38)

Alternative Warp Speed Scale

At the risk–hopefully not too great–of offending Star Trek enthusiasts and traditionalists, it seems that a simple update to the Warp speed definitions would allow for more realistic exploration of space without departing unduly from the original concept.

The update would use a simple base-10 log scale. Log scales are often used in science and technology with Decibel and Richter scales being common examples.

Under this revision, a Starship’s speed or velocity (v) would be defined as:

Equation 1 above uses w-1 (rather than just w) as the exponent to preserve Warp 1 as the speed of light (c), since 10⁰ = 1, which is in keeping with existing scales and the general understanding of Warp speeds. Warp 2 becomes 10 times the speed of light, Warp 3 is 100 times, etc.

Table 1 is a comparison with existing Warp speed factors (multiples of the speed of light):

Table 1 Comparison of Traditional and Proposed Warp Speed Factors

Series abbreviations:

• TOS = The Original Series
• TNG = The Next Generation
• DS9 = Deep Space Nine
• VOY = Voyager

The main advantages of the proposed approach are that it:

• aligns reasonably well, and in some cases better, with time/distance in the TV/film series
• makes the galaxy more navigable in human time frames
• makes intergalactic travel plausible
• would relax the somewhat artificial and tortured Warp 10 limit, however, this can be restored in a more mathematically rigorous way (see below)

While the proposed speeds are significantly greater than the standard scales, at least for higher Warp values, they fit more comfortably with the vast distances found in our Universe. For example, using the old scale (TNG say), to travel from Earth to the centre of our galaxy, about 25,000 light years, it would take over 16 years at Warp 9 and over 3 years at Warp 9.99.

Going to another (close) galaxy is simply implausible, taking 300 or more years at Warp 9.99. Given there are hundreds of billions of galaxies out there, it seems a bit limiting to restrict exploration to such a tiny fraction of the known universe.

By contrast, Table 2 below shows a distance and travel time chart for the proposed new scale.

Table 2 Distance and Travel Times for the Proposed New Warp Speeds

Remembering that a scale is just that, a scale. It doesn’t mean that starships can reach these speeds or sustain them for extended periods. Indeed, standard cruising speed may be significantly less than full speed because of engine stress and energy efficiency.

The next section builds on these ideas by placing limits on Warp speed based on various criteria.

Warp Speed Constraints

To provide limits on the speed of Warp capable ships, and to stay faithful to the idea of a universal maximum Warp speed, for example Warp 10, we can introduce some constraints. One approach is a blend of relativistic and newtonian mechanics.

First, let’s assume that:

1. The force required to move a ship though the Warp field is proportional to the square of its Warp speed, and,
2. There is an universal maximum Warp speed (μ) similar to the speed of light in standard physics

Based on these two assumptions, the force (F) required to “push” through the Warp field can be written mathematically as:

The numerator is the Warp squared factor from the first assumption above. The denominator is analogous to the Lorentz factor of special relativity which is usually written 1/√(c²-v²). It means the required force increases as the speed approaches the universal maximum Warp speed and becoming infinite at that maximum Warp speed. This meets the second assumption above and provides a robust mathematical underpinning of the limit.

Using the force definition from Equation 2 and the velocity from Equation 1, we can calculate the Power (P) required by a Warp engine to maintain a constant velocity using the standard relation:

• Power (P) = Force (F) by velocity (v)

This gives:

From Equation 3 the energy needed to travel a particular distance can be derived using the standard definitions:

• distance (d) = velocity (v) by time (t)
• Energy (E) = Power (P) by time (t)

This leads to the following energy equation:

In addition to power and energy calculations, we can derive an equation for efficient defined as the distance travelled per unit of energy consumed (similar to fuel efficiency for motor vehicles). The efficiency (eff) equation follows directly from Equation 4:

Setting a Universal Maximum Warp (μ)

We can choose any Warp speed as the universal maximum, but let’s stay with convention and make μ=10.

The constant a in the above equations depends on the “units” we use to measure the various quantities. For simplicity, let’s make P=1 when w=1. We can put these values into the Power equation (3) above and solve for a as follows:

Then substituting a back into Equations 3 gives:

Using the same approach for the Energy and Efficiency equations, namely:

• Energy = 1 per unit distance (say 1 lightyear) when w = 1
• Efficiency = 1 (100%) when w = 1

We get:

Table 3 below shows comparative results for Power, Energy and Efficiency for various Warp speeds based on Equations 6, 7 and 8. It also includes the time it would take to travel one light year at the given Warp speed as a reference.

Table 3 Comparative Power, Energy and Efficiency for Various Warp Speeds (μ=10)

These results show that:

• Power requirements rapidly (exponentially) increase as Warp speeds increase
• Energy (fuel) requirements modestly increase as Warp speeds increase
• Efficiency, which is essentially the inverse of energy consumption, reduces in line with energy increases

The massive power requirements speak to the difficulty of achieving higher Warp speeds. It confirms the understanding that significant technical advances are required to go just one step up the Warp speed ladder. It also supports the idea that even a fractional increase in Warp speed comes at a significant cost in terms of additional power requirements.

While the power demands are very high at higher Warp speeds, the energy consumption (i.e. fuel use) remains relatively modest (although increasing). This means the main technical challenge in Warp drive design is to convert the available energy at a high enough rate to provide the power demands needed to achieve a given Warp speed.

The energy efficiency figures can be interpreted, in starship terms, as light years per unit of fuel, for example antimatter. So you get, say, 100 light years per kilogram of antimatter at Warp 1, but only 24.6 at Warp 2, and this drops down to just over half a light year (0.54) at Warp 9, of course you cover that half light year in a fraction of a second!

Despite the rather hypothetical situation, the overall results are consistent with what we might expect from Starship pseudo physics or, perhaps, Warp drive mechanics.

Engine Power Levels

We can further expand on these results to show how the engine power output affects Warp speeds. Lets assume full speed – the rated top speed – is achieved when the engine is producing 100% of its rated power output. Table 4 below shows the corresponding Warp speeds associated with various percentages of this power output for different rated Warp speeds (leftmost column).

Table 4 Warp Speeds for Various Engine Power Percentages (μ=10)

Table 4 shows that running on lower power settings doesn’t significantly reduce the Warp speed compared with its rated full speed (100% power). And conversely, running the engine above its rated power (i.e. overloading it) produces only marginal gains in Warp speed.

Based on this observation, let’s assume Starfleet defines the following Warp speed/power designations:

Table 5 Starfleet Designations with Associated Rated Power Percentages

1. Flank is a US nautical term used to denote speeds in excess of “full” speed.

Using these Starfleet designations, the corresponding Warp speeds, and the time to travel one lightyear (ly) at those speeds, for different Warp rated ships are shown in Table 6 below.

Table 6 Actual Warp Speed and Time to Travel 1 ly for Starfleet Designated Speeds (μ=10)

This means a starship that is rated as Warp 5, for instance, has a standard cruising speed of Warp 4.6 and maximum speed of Warp 5.2. However, there are two main reasons why ships cannot simply run at their maximum Warp speed:

• Energy cost (fuel)
• Engine endurance

We have already looked at energy consumption and seen that energy efficiency does reduce significantly at higher Warp speeds, but an even stronger reason to reduce Warp speed is engine endurance. This is explored in more detail in the next section.

Engine Endurance

Engine endurance is the length of time the engine can operate at a particular power output level before it requires maintenance. Typically Warp engines must be taken offline for a period to allow the maintenance to be performed.

Without maintenance, the risk of a major engine failure or even a catastrophic engine breach significantly increases.

I’m givin’ it all she’s got, Captain! If I push it any farther, the whole thing’ll blow!

– Montgomery ‘Scotty’ Scott

The higher the output power, and hence Warp speed, the shorter the operating time. Conversely, the lower the power output the longer the engine can run before requiring maintenance.

The engine endurance is a function of the ratio of actual engine power compared to the rated (100%) engine power (ρ = P / P_r). This ratio is one (1) when the actual power equals the rated power. Less than one when the actual power is lower than the rated power, such as cruising, and greater than one in overload situations.

There are various ways to formulate an endurance equation but let’s assume that Starfleet Warp drives have an endurance (ε) characterised by an inverted sigmoid function.

As a refresher, the following graphs show a “standard” sigmoid function and some progressive variations to transform it into something more suitable for use as an endurance function.

These versions use the natural logarithm base (e), rather than 10, say, but as shown below the actual base value is irrelevant.

For those not interested in the mathematical details, please skip to the Summary Table at the end of the post where all the results are summarised in one combined table.

The general form of the endurance equation is:

The Lorentzian term √(μ²-w_r²) in the numerator is designed to limit the endurance as the rated Warp speed (w_r) increases. This changes the endurance characteristics for different rated Warp speeds. It makes the endurance approach zero as the rated Warp speed approaches the universal maximum Warp limit (μ).

By setting the exponent to ρ-0.5, the endurance curve is symmetric about ρ=0.5.

The constant a, in conjunction with √(μ²-w_r²), forms the upper limit (asymptote) of the sigmoid function and becomes a scaling factor that depends on the units used to measure endurance.

The constants b and c can be determined by making the following assumptions:

1. Endurance at half power (ρ=0.5) is half the maximum endurance (i.e. ε(0.5) = a√(μ²-w_r²)/2). This also makes the maths below simpler.
2. A Starfleet regulation states that for a Starship to be rated at a particular Warp speed (w_r), its engine must be able to sustain that Warp speed for a minimum period of time (ε_r).

For brevity, we will assign k = √(μ²-w_r²) in the follows calculations. Note k is a constant for each rated Warp speed.

Substituting Assumption 1, ε(0.5)=a√(μ²-w_r²)/2=ak/2, into Equation 9 allows the value of b to be determined as follows:

Putting this value of b back into Equation 9 gives:

Substituting Assumption 2, ε(1) = ε_r, into Equation 10 allows the value of c to be determined as follows:

Putting this value of c back into Equation 10 above gives:

Which, remembering, e^nlog(x) = xⁿ, simplifies to:

Finally, let’s provide some concrete values for μ, a and ε_r so we can calculate some actual results. We will stick with μ=10 for the universal Warp limit. Let’s use a (Earth) day as the endurance unit. Setting ε_r to 0.5 days (i.e. 12 hours) as the minimum period the engine must sustain the rated Warp speed.

Noting that k will range between 0 and 10 (mostly nearer 10), setting a to 10 will give a maximum endurance of 100 days. This seems about right. Substituting μ=10, a=10 and ε_r=0.5 into Equation 11 gives:

A plot of endurance at different rated Warp speeds is shown below:

The following points can be drawn from this plot:

• Overall engine endurance declines progressively as the rated Warp speeds increase. Put simply, high performance engines need more frequent maintenance.
• All the individual lines meet at (ρ=1, ε=0.5), although it is hard to see in the plot.
• For any given rated Warp speed, the endurance declines significantly as the power ratio (ρ) increases. In particular, running at standard (33%) power provides much better endurance than express (67%) or higher power ratios.

Table 7 below is a tabular version of the graph above. It shows engine endurance times for the designated power settings for different rated Warp speeds.

Table 7 Endurance Times for Different Rated Warp Speeds (μ=10)

Table 7 clearly shows that the endurance at full speed is the same (0.5 day = 12 hr) for each rated Warp speed in accordance with Assumption 2 above.

It also shows, which is not clear from the graph, that the engine endurance of overloaded power settings (i.e. flank and maximum) are dramatically less than full power, measured in only minutes or seconds. This plays into the idea that, in an emergency, it is possible to squeeze a bit more speed from the engine but only for comparatively short periods.

Summary

The various results above can be combined into a single summary table. Table 8 below shows the following information:

• Warp speed (at the designated power setting)
• Endurance (maximum time the engine can run before maintenance)
• Maximum distance before maintenance (Warp speed x endurance time)
• Time to travel 1 lightyear (at the specified Warp speed)
• Energy (fuel) to travel 1 lightyear in an arbitrary unit. In this case we have used kilograms of antimatter

for ships of different rated Warp speeds running at Starfleet designated power settings. The data is based on the equations developed earlier.

Table 8 Rated Warp Summary Table

For example, consider a Warp 5 rated ship. At Standard Warp, it will travel at Warp 4.6 which it can sustain for 2.4 months, or just over 800 lightyears, before maintenance is required. At this speed, it will take 2.2 hours and use 2.4 kg of antimatter for each lightyear travelled.

In an emergency, it can travel at Warp 5.2 but only for 45 seconds.

It is also worth noting that Standard Warp provides the best total range for a given rated Warp (i.e. the maximum distance travelled before engine maintenance is required). This agrees with commonsense that there is a “sweet spot” that balances engine endurance and raw speed.

In this post we have:

• Proposed an alternative log base-10 Warp scale that more realistically reflects the vastness of the Universe and makes it more navigable in human time frames
• Developed Warp Power, Energy and Efficiency pseudo-equations by combining relativistic and classical mechanics which provides a robust mathematical underpinning of a universal Warp speed limit
• Introduced the notion of engine endurance to restrict the distance Warp capable ships can travel at various Warp speeds before engine maintenance is required

STAR TREK and related marks are trademarks of CBS Studios Inc.

Introduction

The previous Bridge post looked at various aspects of a single hand including suit distributions, High Card Points and honour holdings. This post will look at similar aspects but will focus on the combined holdings of two hands. We will only consider the situation immediately after the deal and before any cards are played.

Although there are four players (and four hands) in a Bridge deal, they form two partnerships comprising of the North-South and East-West hands. Therefore the combined holdings of the 26 cards held between these hands is a key element of the game.

A quick reminder of some terminology:

The same utility functions used in the previous post are used here. They are repeated below for convenience.

(defn fact
  "Return the factorial of integer n >= 0"
  [n]
  (apply *' (range 1 (inc n))))

(defn comb
  "Return the combination - ways of selecting k from n (k<=n)"
  [n k]
  (/ (fact n) (*' (fact k) (fact (- n k)))))

Suit Distribution of Two Hands

We can ask equivalent questions about the probability of specific suit distributions between two hands as we did for one in the previous post.

How many partnership combinations are there?

There are two ways to view this. One is to only consider the combined 26 card holding without regard to the holdings of the individual hands, in this case the answer is simply the number of ways you can select 26 cards from 52:

The other way to view it, is that the individual holdings of each hand are important. In this case, the answer is the number of ways 13 cards can be selected from 52, multiplied by the number of ways 13 cards can be selected from the remaining 39 cards:

Needless to say both numbers are sufficiently large that you and your partner will not be holding the same cards very often :). However, these values are important as they form the denominators for some of the probability equations developed later.

Suit Distribution of Combined Hands (26-cards analysis)

Taking the first view–combined 26-cards–we can answer questions like:

What is the probability of suit distributions of two combined hands?

As we did with one hand, we can look at the probability of the combined distribution of two hands by iterating over all possible distributions and calculating the probability based on 26 cards instead of 13 cards.

This produces the specific distribution with each suit treated uniquely. For example, there will be separate results for 8-6-6-6, 6-8-6-6, 6-6-8-6 and 6-6-6-8 (i.e. where the 8 cards are Spades, Hearts, Diamonds and Clubs respectively).

The Clojure code is shown below:

"Probability of specific suit distribution for
two combined hands (ie 26 cards)"
(def distr-combined-specific
  (let [c5226 (comb 52 26)]
    (for [s (range 14)
          h (range 14)
          d (range 14)
          c (range 14)
          :when (= 26 (+ s h d c))]
      (let [cs (* (comb 13 s)
                  (comb 13 h)
                  (comb 13 d)
                  (comb 13 c))
            pp (/ (* 100.0 cs) c5226)]
        [[s h d c] pp]))))

(sort-by last > distr-combined-specific)
;=> ([[6 6 7 7] 1.7484724571169408]
;    [[6 7 6 7] 1.7484724571169408]
;    [[6 7 7 6] 1.7484724571169408]
;    [[7 6 6 7] 1.7484724571169408]
;    [[7 6 7 6] 1.7484724571169408]
;    [[7 7 6 6] 1.7484724571169408]
;    [[5 7 7 7] 1.3113543428377057]
;    [[6 6 6 8] 1.3113543428377057]
;    ...

(apply + (map last distr-combined-specific))
;=> 100.0

(count distr-combined-specific)
;=> 1834

From these results, we can see there are 1834 specific combined distributions whose probabilities total to 100.0% (as expected). The most common specific distribution (or shape) has two 7-cards suits and two 6-cards suits (e.g. 7-7-6-6, 7-6-7-6, etc).

The generic distribution combines specific distributions of the same general shape, for example, the four specific distributions mentioned earlier would be grouped under the single generic distribution 8 6 6 6.

The following code performs this grouping and also counts the number of specific distributions associated with each generic distribution.

"Probability of generic suit distribution for two combined hands"
(def distr-combined-generic
  (reduce
    (fn [acc [d p]]
      (let [g-distr (vec (sort > d))
            acc (update-in acc [g-distr :prob] (fnil + 0.0) p)
            acc (update-in acc [g-distr :cnt] (fnil inc 0))]
        acc))
    (sorted-map)
    distr-combined-specific))

;=> {[7 7 6 6] {:prob 10.490834742701646, :cnt 6},
;    [7 7 7 5] {:prob 5.245417371350823, :cnt 4},
;    [8 6 6 6] {:prob 5.245417371350823, :cnt 4},
;    [8 7 6 5] {:prob 23.604378171078707, :cnt 24},
;    [8 7 7 4] {:prob 6.556771714188528, :cnt 12},
;    [8 8 5 5] {:prob 3.3193656803079423, :cnt 6},
;    [8 8 6 4] {:prob 4.917578785641396, :cnt 12},
;    ...

(apply + (map :prob (vals distr-combined-generic)))
;=> 99.99999999999996

(count distr-combined-generic)
;=> 104

As expected, the number of generic distributions is significantly less at 104. The most common one by far is 8 7 6 5 (23.6%). Note that this is different from the most common specific distribution because there are many more (24) specific distributions that make this generic distribution than there are for 7 7 6 6 which has only six.

The full results are shown in Table 1 below and are downloadable here as a csv file.

The “Specific count” column is the number for specific distributions that make up each generic distribution. So the columns are related as follows:

Table 1 Percentage generic probability of combined hands

One immediate observation is that most distributions have at least one suit with eight or more cards. Only the first two distributions (7766 and 7775) don’t and these make up just 15.736% of all distributions.

In Bridge, the combined number of cards (or “fit”) in a suit (particularly a major suit) is important because it often determines whether the two hands should play in a suit contract or a No Trumps contract. With a combined suit holding of eight or more cards, the odds favour a suit contract over a No Trump contract. A fit in a second suit further favours a suit contract and often means a higher contract (or more tricks) is possible with less than the expected number of HCP. The hands are said to have a “good” fit.

We can summarise the results from Table 1 to show the likelihood of various primary and secondary suit fits as shown in Table 2 below.

Table 2 Percentage probability of primary (P) and secondary (S) suit ‘fits’ in combined hands

While this view of the combined 26-cards provides some useful insights it does not distinguish between the holdings in each hand separately. So an 8-card fit, for example, could be made up of any combination of 0-8, 1-7, 2-6, 3-5, 4-4, 5-3, 6-2, 7-1 or 8-0 between the two hands. In the next section we look at how the two hands separately contribute to the combined result.

Suit Distribution of Two Separate Hands (2 x 13-cards)

Using the second view–the distribution between two separate hands–provides a more comprehensive result. It captures the detail of the distribution associated with each hand but generates significantly more data that requires additional processing to exact useful information.

As we did previously, let’s consider an example, say one hand has the specific distribution 3-5-3-2 (i.e. 3♠︎, 5♥︎, 3♦︎ and 2♣︎) and the other has 2-3-4-4 (i.e. 2♠︎, 3♥︎, 4♦︎ and 4♣︎). We know the number of ways to select the first hand from the total ways for all hands is:

The second hand must be selected from the remaining cards which means only 10 of the remaining Spades, 8 of the remaining Hearts, 10 of the remaining Diamonds and 11 of the remaining Clubs, and the total ways it can be selected must come for the remaining 39 cards as follows:

The probability of these two specific distributions is the product of the ‘ways-to-select’ divided by the product of the ‘total-ways’:

Importantly, if we took the second hand first, and the first hand second, the numerator would be quite different. The new calculation would be:

Somewhat amazingly, although what we would expect intuitively, the result turns out to be the same. The mathematical magic behind this is that the factors for each suit are interchangeable:

This explains why the result is independent of the order of the hands.

This example can be easily generalised for two hands with specific distributions s1-h1-d1-c1 and s2-h2-d2-c2 as:

The following Clojure code calculates the percentage probability of all possible specific two hand distributions. As each of the eight possible suit counts can range from 0 to 13 there are potentially 14⁸ = 1,475,789,056 iterations. To reduce this somewhat epic number, the second suit upper range is limited to 13 less the first suit, which brings it down to 121,550,625 iterations. It is still a sizeable bit of number crunching which takes about 15 seconds on my MacBook Pro.

(def distr-two-hands-specific
  (let [c5213c3913 (* (comb 52 13) (comb 39 13))]
    (for [s1 (range 14) s2 (range (- 14 s1))
          h1 (range 14) h2 (range (- 14 h1))
          d1 (range 14) d2 (range (- 14 d1))
          c1 (range 14) c2 (range (- 14 c1))
          :when (and
                  (= 13 (+ s1 h1 d1 c1))
                  (= 13 (+ s2 h2 d2 c2)))]
      (let [cs (* (comb 13 s1) (comb (- 13 s1) s2)
                  (comb 13 h1) (comb (- 13 h1) h2)
                  (comb 13 d1) (comb (- 13 d1) d2)
                  (comb 13 c1) (comb (- 13 c1) c2))
            pp (/ (* 100.0 cs) c5213c3913)]
        [[s1 h1 d1 c1] [s2 h2 d2 c2] pp]))))

(sort-by last > distr-two-hands-specific)
;=> ([[3 3 3 4] [3 3 4 3] 0.08237519989109292]
;    [[3 3 4 3] [3 3 3 4] 0.08237519989109292]
;    [[3 3 3 4] [3 4 3 3] 0.08237519989109292]
;    [[3 3 4 3] [3 4 3 3] 0.08237519989109292]
;    [[3 4 3 3] [3 3 3 4] 0.08237519989109292]
;    [[3 4 3 3] [3 3 4 3] 0.08237519989109292]
;    [[3 3 3 4] [4 3 3 3] 0.08237519989109292]
;    [[3 3 4 3] [4 3 3 3] 0.08237519989109292]
;    [[3 4 3 3] [4 3 3 3] 0.08237519989109292]
;    [[4 3 3 3] [3 3 3 4] 0.08237519989109292]
;    [[4 3 3 3] [3 3 4 3] 0.08237519989109292]
;    [[4 3 3 3] [3 4 3 3] 0.08237519989109292]
;    [[3 3 3 4] [3 3 3 4] 0.07060731419236536]
;    [[3 3 4 3] [3 3 4 3] 0.07060731419236536]
;    [[3 4 3 3] [3 4 3 3] 0.07060731419236536]
;    [[4 3 3 3] [4 3 3 3] 0.07060731419236536]
;    [[2 3 4 4] [4 3 3 3] 0.0617813999183197]
;    [[2 4 3 4] [4 3 3 3] 0.0617813999183197]
;    [[2 4 4 3] [4 3 3 3] 0.0617813999183197]
;    [[3 2 4 4] [3 4 3 3] 0.0617813999183197]
;    ...)

(apply + (map last distr-two-hands-specific))
;=> 99.99999999998808

(count distr-two-hands-specific)
;=> 239344

The results show there are 239,344 specific two hand distributions whose total probability sums to 100% (as expected). This is a lot of data to digest but it allows us to answer a broad range of distributional questions like:

What is the probability of two hands having exactly the same shape?

The following code uses the result calculated above to sum the probabilities of each case where the two distributions are identical.

"Probability of identical distributions in each hand"
(reduce
  (fn [acc [d1 d2 p]]
    (if (= d1 d2)
      (update acc (vec (sort > d1)) (fnil + 0.0) p)
      acc))
  (sorted-map)
  distr-two-hands-specific)

;=> {[4 3 3 3] 0.28242925676946146,
;    [4 4 3 2] 0.27801629963243857,
;    [4 4 4 1] 0.008845973170123048,
;    [5 3 3 2] 0.08472877703083843,
;    [5 4 2 2] 0.027801629963243858,
;    [5 4 3 1] 0.016175493796796423,
;    ...}

The full result is shown in Table 3 below.

Table 3 Percentage probability of two hands with exactly the same shape

These hands can cause problems, particularly with a major suit fit, as they are often played in a suit contact where a No Trumps contract is frequently better because of the lack of ruffing opportunities.

The following sections take the above specific two hand distribution results and interpret them for:

• Single suit
• Two suits
• Full distribution (all four suits)

Single Suit Analysis

This section focuses on the details of any given single suit to answer questions such as:

How is any one suit distributed between the two hands?

If we pick one suit, Spades say, what is the probability of holding X cards in one hand and Y cards in the other. We can process the results above with the following code to build a table of X-Y ‘fit’ probabilities.

"Probability of holding a particular X-Y fit in any one suit"
(def one-suit-fit
  (reduce
    (fn [acc [d1 d2 p]]
      (update acc [(d1 0) (d2 0)] (fnil + 0.0) p))
    (sorted-map)
    distr-two-hands-specific))

;=> {[0 0] 0.0016378547895184352,
;    [0 1] 0.019771247102043903,
;    [0 2] 0.09490198608981093,
;    [0 3] 0.23923208993472986,
;    [0 4] 0.3518118969628408,
;   ...

(apply + (map val one-suit-fit))
;=> 100.0

The full results are in Table 4 below.

Table 4 Percentage Probability of X cards in one hand and Y cards in the other for a given suit

We can recast Table 4 data in various ways such as:

Given you have X cards in a suit what is the probability partner holds Y cards in the same suit?

In Table 5 below, each row in Table 4 has been ‘normalised’ by dividing by the row total so each row now sums to 100%. The number of cards held by you (X) is shown in the first column. The number held by partner (Y) is shown in the top row. The corresponding value is P(Y|X) or the probability of partner holding Y cards if you hold X card in a particular suit.