Introduction to channel capacity, symbol rate (baud) & message space. This is the first step towards more modern measures of information. Featuring the Baudot multiplex system & Thomas Edison's quadruplex telegraph.
Transcript
Transcript generated by YouTube auto-captions. May contain errors.
how is it we should measure information in a way that applies to any communication system you can think of human animal or alien well let's return to the late 19th century where at the time we were focused as we are today on speed and one goal to improve speed was to design a machine which allowed The Operators to input letters which we can think of as primary symbols and have the machine automate the lower level signaling events such as pulses of electricity what we can call secondary symbols and machines can be driven by some clock Source allowing it to generate a precise and Rapid pulse stream which presumably would run much faster than any human hand and one great example of this was the bedau multiplex system and the design was put into service in 1874 and it built off the same conceptual ideas we've seen in the shutter Telegraph it consisted of five Keys which could be played in any combination think of it as a chord each combination would represent a unique message with five notes each either on or off you can play 2 to the power of 5 or 32 different chords the code assigned the 32 different chords to each letter of the alphabet with the leftover used for Carriage returns new line and spaces so the operator would literally play letters and their machine would automatically output a pulse stream representing the letters like this for letter T or like this for letter r or like this for letter B so we have an output signal containing various combinations of DC impulses a signal that accurately represents the message typed on the teletypewriter behind the counter the mechanical nerves of the system change words to holes on tape and the holes on tape to electrical impulses speeding over the wires notice at the lowest level this system is exchanging either the present or absence of electrical current in a sequence divided using a clock so how fast can our internal clocks run well the the limiting speed was not the clock then and today the speed of transmission was physically limited by the minimum spaces between these impulses or the pulse rate and this problem plagued Engineers who were testing underground submarine cables using the existing moris Code system and it's similar to an echo or a sustained note if one sends dots too fast over a long under C circuit they will run together at the receiving end because the symbol we receive at the far end of the circuit will be a slightly longer smoothed out rise and fall not an exact replica and sending pulses too fast results in inter symol interference this occurs for example when the longer flow of a current bleeds into the next time Division and perhaps reverses a zero to a one so even if we are automating the detection of these current levels there is a fundamental limit to how far we can squeeze two pulses together and this is the same problem Alice and Bob ran into with their string communication system which we called the maximum pluck speed if they plucked any faster than two plucks per second they noticed they started to bleed together and they got confused so this is called the symbol rate remember a symbol can be broadly defined as the current state of some observable signal which persists for a fixed period of time whether you are using fire sound electrical current anything a signaling event is simply a change from one state to another so the symbol rate is the number of signaling events which can be squeezed together in one second it also became clear that there was one other way to increase the capacity of a communication system we can increase the number of different signaling events for example with Alice and Bob's string communication system they soon found that varying the type of PLU allowed them to send their messages faster for example hard medium versus soft PLU or high pitch versus low pitch PLU by tightening the cable different amounts and this was an idea implemented by Thomas Edison which he applied to the mors code system and it was based on the idea that you could use weak and strong batteries to produced signals of different strengths he also used two directions as gaus and Weber did forward versus reverse current and two intensities so he had plus 3 volts + 1 Vol minus 1 Vol and minus 3vt four different current values which could be exchanged it enabled Western Union to save money by greatly increasing the number of messages the company could send without building new lines and this is known as the quadruplex Telegraph and it continued to be used into the 20th century but again as we expanded the number of different signaling events we ran into another problem for example why not send a TH or a million different voltage levels per pulse well as you may expect fine grain differences lead to difficulties on the receiving end and with electrical systems the resolution of these differences is always limited by electrical noise if we attach a probe to any electrical [Music] line and zoom in closely enough we will always find Minute undesired currents and this is an unavoidable result of natural processes such as heat or geomagnetic storms and even latent effects of the Big Bang so the differences between between signaling events must be great enough that noise cannot randomly bump a signaling event from one type to another clearly now we can step back and begin to define the capacity of a communication system using these two very simple ideas first how many symbol transfers per second which we called symbol rate and today it's known simply as bod for ail B and we can Define this as n where it's n symbol transfers per second and number two how many differences per symbol which we can think of as the symbol space how many symbols can we select from at each point and we can call this s and as we've seen before these parameters can be thought of as a decision tree of possibilities because each symbol can be thought of as a decision where the number of branches depend on the number of differences and after n symbols we have a tree with s to the power of n leaves and since each path through this tree can represent a message we can think of the number of leaves as the size of the message space and this is easy to visualize the message space is simply the width of the base of one of these trees and it defines the total number of possible messages one could send given a sequence of end symbols for example let's say Alice sends Bob a message which consists of two plucks and they are using a hard versus soft pluck as their communication system this means she has the ability to Define one of four possible messages to Bob and if instead they were using a system of hard versus medium versus soft PLU then with two pluck she has the ability to Define one of 3^ 2 equals 9 messages and with three pluck this jumps to one of 27 messages now if instead Alice and Bob were exchanging written notes in class which contain only two letters on a piece of paper then a single note would contain one of 26 to the power of 2 or 676 possible messages it's important to realize now that we no longer care about the meaning applied to these chains of differences merely how many different messages are possible the resulting sequences could represent numbers names feelings music or perhaps even some alien alphabet we could never understand when we look at a communication system now we can begin to think about its capacity as how many different things you could say and we can then use the message space to Define exactly how many differences are possible in any situation and this simple yet elegant idea forms the basis for how information will be later defined and this is the final step that brings us to Modern information Theory and it emerges in the early 20th century for