May 6, 2015 03:22 PM
A brief introduction to bits and why they're not the same as 0s and 1s...
https://plus.maths.org/content/informati...-questions
EXCERPT: A bit is the amount of information you need to choose between two equally probable alternatives. Imagine you are standing at the fork in the road...
[...] Navigating a series of forks in the road is, in some respects, similar to the game of 20 questions. In this game your opponent chooses a word (usually a noun), and you (the astute questioner) are allowed to ask twenty questions in order to discover the identity of this word. Crucially, each question must have a yes/no (i.e. binary) answer, and therefore the answer provides you with at most one bit of information.
Why at most? By analogy with the navigation example, where each decision at a road fork halved the number of remaining destinations, each question should halve the number of remaining possible words. A question to which you already know the answer is a poor choice of question. For example, if your question is, "Is the word in the dictionary?", then the answer is almost certainly, "Yes!", an answer which is predictable, and which therefore provides you with no information.
Conversely, a well-chosen question is one to which you have no idea whether the answer will be yes or no — there is a 50:50 chance of it being either. In this case, the answer provides exactly one bit of information. The cut-down version of 20 questions in the figure below shows...
https://plus.maths.org/content/informati...-questions
EXCERPT: A bit is the amount of information you need to choose between two equally probable alternatives. Imagine you are standing at the fork in the road...
[...] Navigating a series of forks in the road is, in some respects, similar to the game of 20 questions. In this game your opponent chooses a word (usually a noun), and you (the astute questioner) are allowed to ask twenty questions in order to discover the identity of this word. Crucially, each question must have a yes/no (i.e. binary) answer, and therefore the answer provides you with at most one bit of information.
Why at most? By analogy with the navigation example, where each decision at a road fork halved the number of remaining destinations, each question should halve the number of remaining possible words. A question to which you already know the answer is a poor choice of question. For example, if your question is, "Is the word in the dictionary?", then the answer is almost certainly, "Yes!", an answer which is predictable, and which therefore provides you with no information.
Conversely, a well-chosen question is one to which you have no idea whether the answer will be yes or no — there is a 50:50 chance of it being either. In this case, the answer provides exactly one bit of information. The cut-down version of 20 questions in the figure below shows...