Das Miscellany2019-05-17T21:15:10+00:00http://alandavies.org/blogAlan Davieswebsite@alandavies.orgOn trees and paths2019-05-01T00:00:00+00:00http://alandavies.org/blog/2019/05/01/on-trees-and-paths<h1 id="the-question">The question</h1>
<p>This post is inspired by a question that I was asked in an interview once. Here it is …</p>
<p>Assume that you have an unordered binary tree where the values stored at the nodes are positive integers. Write code
to find the length of the longest path through the tree that consists of nodes whose values are in strict increasing or
decreasing order. A path proceeds from node to node via the obvious parent-child relationships and can go up and down.</p>
<p>An example is in order. In this tree …</p>
<p><img src="/images/binary_tree_question_example.png" alt="Example binary tree" class="center-image" /></p>
<p>… the longest path is of length four. The path is [10, 20, 30, 40] (starting from the lowest 10 node, proceeding up
to the 30 node and then down to the lowest 40 node). We could also list this path in descending order, the order of
traversal doesn’t matter.</p>
<h1 id="solution-strategies">Solution strategies</h1>
<p>So how would we go about solving this for a general binary tree? Well, most tree algorithms involve some sort of
recursive traversal while maintaining state. The traversals are pretty generic, it’s the state you track along the way
and how you use that state that will be key.</p>
<h1 id="binary-tree-traversals">Binary tree traversals</h1>
<p>There are various types of tree traversal. The first distinction is between breadth-first and depth-first. In the
former the tree is fully explored left to right at each level before proceeding to the next level, whereas in the
latter the tree is fully explored root to leaf by following the left child at each node before then jumping up and
following the right node.</p>
<p><img src="/images/binary_tree_traversal_breadth_vs_depth_first.png" alt="Breadth vs depth first" class="center-image" /></p>
<p>With a depth-first search there are three further types of traversal based on the order in which the root node and
children (sub-trees) of a given tree are processed. These are pre-order (the root is processed first then the
children), in-order (the left is processed first, then the root, then the right) and post-order (the children are
processed first and then the root).</p>
<p><img src="/images/binary_tree_traversal_pre_vs_in_vs_post_order.png" alt="Pre vs in vs post order" class="center-image" /></p>
<h1 id="generic-code-for-tree-traversal">Generic code for tree traversal</h1>
<p>To solve this problem we are going to need a depth-first, post-order traversal, i.e. one where we process all the
children before we process the root. This represents an exhaustive, bottom up search. Let’s write some code to do
that.</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">namespace</span> <span class="nn">GinjaSoft.TreeStuff</span>
<span class="p">{</span>
<span class="k">using</span> <span class="nn">System</span><span class="p">;</span>
<span class="k">public</span> <span class="k">class</span> <span class="nc">Node</span><span class="p"><</span><span class="n">T</span><span class="p">></span>
<span class="p">{</span>
<span class="k">public</span> <span class="nf">Node</span><span class="p">(</span><span class="n">T</span> <span class="k">value</span><span class="p">)</span> <span class="p">{</span> <span class="n">Value</span> <span class="p">=</span> <span class="k">value</span><span class="p">;</span> <span class="p">}</span>
<span class="k">public</span> <span class="n">T</span> <span class="n">Value</span> <span class="p">{</span> <span class="k">get</span><span class="p">;</span> <span class="k">set</span><span class="p">;</span> <span class="p">}</span>
<span class="k">public</span> <span class="n">Node</span><span class="p"><</span><span class="n">T</span><span class="p">></span> <span class="n">Left</span> <span class="p">{</span> <span class="k">get</span><span class="p">;</span> <span class="k">set</span><span class="p">;</span> <span class="p">}</span>
<span class="k">public</span> <span class="n">Node</span><span class="p"><</span><span class="n">T</span><span class="p">></span> <span class="n">Right</span> <span class="p">{</span> <span class="k">get</span><span class="p">;</span> <span class="k">set</span><span class="p">;</span> <span class="p">}</span>
<span class="p">}</span>
<span class="k">public</span> <span class="k">static</span> <span class="k">class</span> <span class="nc">BinaryTreeTraversals</span>
<span class="p">{</span>
<span class="k">public</span> <span class="k">static</span> <span class="k">void</span> <span class="nf">DepthFirstPostOrder</span><span class="p">(</span><span class="n">Node</span><span class="p"><</span><span class="n">T</span><span class="p">></span> <span class="n">root</span><span class="p">,</span> <span class="n">Func</span><span class="p"><</span><span class="n">Node</span><span class="p"><</span><span class="n">T</span><span class="p">>,</span> <span class="n">TResult</span><span class="p">,</span> <span class="n">TResult</span><span class="p">,</span> <span class="n">TResult</span><span class="p">></span> <span class="n">fn</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">TResult</span> <span class="n">leftResult</span> <span class="p">=</span> <span class="k">default</span><span class="p">;</span>
<span class="n">TResult</span> <span class="n">rightResult</span> <span class="p">=</span> <span class="k">default</span><span class="p">;</span>
<span class="k">if</span><span class="p">(</span><span class="n">root</span><span class="p">.</span><span class="n">Left</span> <span class="p">!=</span> <span class="k">null</span><span class="p">)</span> <span class="n">leftResult</span> <span class="p">=</span> <span class="nf">DepthFirstPostOrder</span><span class="p">(</span><span class="n">root</span><span class="p">.</span><span class="n">Left</span><span class="p">,</span> <span class="n">fn</span><span class="p">);</span>
<span class="k">if</span><span class="p">(</span><span class="n">root</span><span class="p">.</span><span class="n">Right</span> <span class="p">!=</span> <span class="k">null</span><span class="p">)</span> <span class="n">rightResult</span> <span class="p">=</span> <span class="nf">DepthFirstPostOrder</span><span class="p">(</span><span class="n">root</span><span class="p">.</span><span class="n">Right</span><span class="p">,</span> <span class="n">fn</span><span class="p">);</span>
<span class="k">return</span> <span class="nf">fn</span><span class="p">(</span><span class="n">root</span><span class="p">,</span> <span class="n">leftResult</span><span class="p">,</span> <span class="n">rightResult</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>This is a generic code. What we actually “do” at each node is factored out into a function that the client code will
supply. This function has to adhere to a given prototype, it needs to take three parameters (a reference to the node
to process and two instances of a result type - that will be generated from the processing of the left and right
children respectively) and it needs to return a result type that will represent the result of processing the node.</p>
<h1 id="specific-code-for-the-problem-at-hand">Specific code for the problem at hand</h1>
<p>Here’s how we might write some client code to use the generic traversal code. First we need to define the result type
(TResult) that will represent the result of processing a node. This type will be used to hold information about the
paths that exist in each sub-tree within a tree as we recursively traverse all the nodes in a depth-first post-order
manner.</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">namespace</span> <span class="nn">GinjaSoft.TreeStuff</span>
<span class="p">{</span>
<span class="k">public</span> <span class="k">partial</span> <span class="k">class</span> <span class="nc">LongestPath</span>
<span class="p">{</span>
<span class="c1">//</span>
<span class="c1">// This type contains information about the paths that exist within a given sub-tree</span>
<span class="c1">//</span>
<span class="k">public</span> <span class="k">class</span> <span class="nc">NodeInfo</span>
<span class="p">{</span>
<span class="c1">// The value of the tree's root node</span>
<span class="k">public</span> <span class="kt">uint</span> <span class="n">RootValue</span> <span class="p">{</span> <span class="k">get</span><span class="p">;</span> <span class="k">set</span><span class="p">;</span> <span class="p">}</span>
<span class="c1">// The length of the longest path of nodes (up through the root node) with consecutively increasing values </span>
<span class="k">public</span> <span class="kt">uint</span> <span class="n">MaxIncPathLenToRoot</span> <span class="p">{</span> <span class="k">get</span><span class="p">;</span> <span class="k">set</span><span class="p">;</span> <span class="p">}</span>
<span class="c1">// The length of the longest path of nodes (up through the root node) with consecutively decreasing values</span>
<span class="k">public</span> <span class="kt">uint</span> <span class="n">MaxDecPathLenToRoot</span> <span class="p">{</span> <span class="k">get</span><span class="p">;</span> <span class="k">set</span><span class="p">;</span> <span class="p">}</span>
<span class="c1">// The length of the longest path of nodes with either consecutively increasing or decreasing values. This path</span>
<span class="c1">// does not have to include the root node and also doesn't have to extend exclusively up the tree towards the</span>
<span class="c1">// root. The path can start from a given node, proceed upwards through other nodes and then downwards again</span>
<span class="c1">// through child nodes. This is the value of interest.</span>
<span class="k">public</span> <span class="kt">uint</span> <span class="n">MaxPathLen</span> <span class="p">{</span> <span class="k">get</span><span class="p">;</span> <span class="k">set</span><span class="p">;</span> <span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Here’s how we’ll use this type with the DepthFirstPostOrder method …</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">namespace</span> <span class="nn">GinjaSoft.TreeStuff</span>
<span class="p">{</span>
<span class="k">public</span> <span class="k">partial</span> <span class="k">class</span> <span class="nc">LongestPath</span>
<span class="p">{</span>
<span class="c1">//</span>
<span class="c1">// This function is the solution to the interview question</span>
<span class="c1">//</span>
<span class="k">public</span> <span class="k">static</span> <span class="kt">uint</span> <span class="nf">GetMaxPathLength</span><span class="p">(</span><span class="n">Node</span><span class="p"><</span><span class="kt">uint</span><span class="p">></span> <span class="n">tree</span><span class="p">)</span>
<span class="p">{</span>
<span class="kt">var</span> <span class="n">nodeInfo</span> <span class="p">=</span> <span class="nf">GetNodeInfo</span><span class="p">(</span><span class="n">tree</span><span class="p">);</span>
<span class="k">return</span> <span class="n">nodeInfo</span><span class="p">.</span><span class="n">MaxPathLen</span><span class="p">;</span>
<span class="p">}</span>
<span class="c1">//</span>
<span class="c1">// Implementation</span>
<span class="c1">//</span>
<span class="k">internal</span> <span class="k">static</span> <span class="n">NodeInfo</span> <span class="nf">GetNodeInfo</span><span class="p">(</span><span class="n">IBinaryNode</span><span class="p"><</span><span class="kt">uint</span><span class="p">></span> <span class="n">tree</span><span class="p">)</span>
<span class="p">{</span>
<span class="c1">// Unfortunately I can't pass ProcessNode directly to DepthFirstPostOrder. C# won't do the implicit cast from</span>
<span class="c1">// method group to Func and so a local variable is required. Sigh.</span>
<span class="n">Func</span><span class="p"><</span><span class="n">Node</span><span class="p"><</span><span class="kt">uint</span><span class="p">>,</span> <span class="n">NodeInfo</span><span class="p">,</span> <span class="n">NodeInfo</span><span class="p">,</span> <span class="n">NodeInfo</span><span class="p">></span> <span class="n">fn</span> <span class="p">=</span> <span class="n">ProcessNode</span><span class="p">;</span>
<span class="k">return</span> <span class="n">BinaryTreeTraversals</span><span class="p">.</span><span class="nf">DepthFirstPostOrder</span><span class="p">(</span><span class="n">tree</span><span class="p">,</span> <span class="n">fn</span><span class="p">);</span>
<span class="p">}</span>
<span class="k">private</span> <span class="k">static</span> <span class="n">NodeInfo</span> <span class="nf">ProcessNode</span><span class="p">(</span><span class="n">Node</span><span class="p"><</span><span class="kt">uint</span><span class="p">></span> <span class="n">root</span><span class="p">,</span> <span class="n">NodeInfo</span> <span class="n">left</span><span class="p">,</span> <span class="n">NodeInfo</span> <span class="n">right</span><span class="p">)</span>
<span class="p">{</span>
<span class="c1">// ...</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>The ProcessNode function is the meat of the solution. It will be called recursively for each node (sub-tree) in the
tree. Each call will contain the following parameters: the root node of the current sub-tree and two NodeInfo objects
representing the result of the calls to ProcessNode for the left and right child nodes respectively.</p>
<p>As with most recursive functions we need to identify the base case, for which we return the base result, and otherwise
generate a result based on the state passed on the call stack. The logic for the latter case is as follows …</p>
<p>If the root value is greater than the root value for a given child then MaxIncPathLenToRoot through the new root will be
one larger than that of the child. Conversely if the root value is less than the child root value then the new
MaxDecPathLenToRoot will be one larger than that of the child. The final Max(Inc/Dec)PathLenToRoot values will be the
larger of the results for the left and right children.</p>
<p>Then we need to update MaxPathLen. The new value will start out as the larger of MaxPathLen from the two children. Then
we will check the new Max(Inc/Dec)PathLenToRoot values and take the larger of the two if it is larger than the current
MaxPathLen. Finally we check for an up/down path through the current node, i.e. whether the current root value lies
between the child root values, in which case we can calculate a new candidate MaxPathLen as the sum of the
Max(Inc/Dec)PathLenToRoot values from the children plus one. If this value is greater than the current MaxPathLen then
we use it.</p>
<p>Here’s the actual code …</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">namespace</span> <span class="nn">GinjaSoft.TreeStuff</span>
<span class="p">{</span>
<span class="k">public</span> <span class="k">partial</span> <span class="k">class</span> <span class="nc">LongestPath</span>
<span class="p">{</span>
<span class="k">private</span> <span class="k">static</span> <span class="n">NodeInfo</span> <span class="nf">ProcessNode</span><span class="p">(</span><span class="n">Node</span><span class="p"><</span><span class="kt">uint</span><span class="p">></span> <span class="n">root</span><span class="p">,</span> <span class="n">NodeInfo</span> <span class="n">left</span><span class="p">,</span> <span class="n">NodeInfo</span> <span class="n">right</span><span class="p">)</span>
<span class="p">{</span>
<span class="c1">// Base result for a leaf node</span>
<span class="kt">var</span> <span class="n">returnValue</span> <span class="p">=</span> <span class="k">new</span> <span class="n">NodeInfo</span> <span class="p">{</span>
<span class="n">RootValue</span> <span class="p">=</span> <span class="n">root</span><span class="p">.</span><span class="n">Value</span><span class="p">,</span>
<span class="n">MaxIncPathLenToRoot</span> <span class="p">=</span> <span class="m">1</span><span class="p">,</span>
<span class="n">MaxDecPathLenToRoot</span> <span class="p">=</span> <span class="m">1</span><span class="p">,</span>
<span class="n">MaxPathLen</span> <span class="p">=</span> <span class="m">1</span>
<span class="p">};</span>
<span class="c1">// If there are no children to process then we are done</span>
<span class="k">if</span><span class="p">(</span><span class="n">left</span> <span class="p">==</span> <span class="k">null</span> <span class="p">&&</span> <span class="n">right</span> <span class="p">==</span> <span class="k">null</span><span class="p">)</span> <span class="k">return</span> <span class="n">returnValue</span><span class="p">;</span>
<span class="c1">// Process children ...</span>
<span class="k">if</span><span class="p">(</span><span class="n">left</span> <span class="p">!=</span> <span class="k">null</span><span class="p">)</span> <span class="p">{</span>
<span class="c1">// Update the min/max PathLenToRoot based on the current root value and left child root value</span>
<span class="k">if</span><span class="p">(</span><span class="n">root</span><span class="p">.</span><span class="n">Value</span> <span class="p">></span> <span class="n">left</span><span class="p">.</span><span class="n">RootValue</span><span class="p">)</span> <span class="n">returnValue</span><span class="p">.</span><span class="n">MaxIncPathLenToRoot</span> <span class="p">=</span> <span class="n">left</span><span class="p">.</span><span class="n">MaxIncPathLenToRoot</span> <span class="p">+</span> <span class="m">1</span><span class="p">;</span>
<span class="k">else</span> <span class="k">if</span><span class="p">(</span><span class="n">root</span><span class="p">.</span><span class="n">Value</span> <span class="p"><</span> <span class="n">left</span><span class="p">.</span><span class="n">RootValue</span><span class="p">)</span> <span class="n">returnValue</span><span class="p">.</span><span class="n">MaxDecPathLenToRoot</span> <span class="p">=</span> <span class="n">left</span><span class="p">.</span><span class="n">MaxDecPathLenToRoot</span> <span class="p">+</span> <span class="m">1</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">if</span><span class="p">(</span><span class="n">right</span> <span class="p">!=</span> <span class="k">null</span><span class="p">)</span> <span class="p">{</span>
<span class="c1">// Update the min/max PathLenToRoot based on the current root value and right child root value</span>
<span class="k">if</span><span class="p">(</span><span class="n">root</span><span class="p">.</span><span class="n">Value</span> <span class="p">></span> <span class="n">right</span><span class="p">.</span><span class="n">RootValue</span><span class="p">)</span> <span class="p">{</span>
<span class="kt">var</span> <span class="n">pathLen</span> <span class="p">=</span> <span class="n">right</span><span class="p">.</span><span class="n">MaxIncPathLenToRoot</span> <span class="p">+</span> <span class="m">1</span><span class="p">;</span>
<span class="n">returnValue</span><span class="p">.</span><span class="n">MaxIncPathLenToRoot</span> <span class="p">=</span> <span class="n">Math</span><span class="p">.</span><span class="nf">Max</span><span class="p">(</span><span class="n">returnValue</span><span class="p">.</span><span class="n">MaxIncPathLenToRoot</span><span class="p">,</span> <span class="n">pathLen</span><span class="p">);</span>
<span class="p">}</span>
<span class="k">else</span> <span class="k">if</span><span class="p">(</span><span class="n">root</span><span class="p">.</span><span class="n">Value</span> <span class="p"><</span> <span class="n">right</span><span class="p">.</span><span class="n">RootValue</span><span class="p">)</span> <span class="p">{</span>
<span class="kt">var</span> <span class="n">pathLen</span> <span class="p">=</span> <span class="n">right</span><span class="p">.</span><span class="n">MaxDecPathLenToRoot</span> <span class="p">+</span> <span class="m">1</span><span class="p">;</span>
<span class="n">returnValue</span><span class="p">.</span><span class="n">MaxDecPathLenToRoot</span> <span class="p">=</span> <span class="n">Math</span><span class="p">.</span><span class="nf">Max</span><span class="p">(</span><span class="n">returnValue</span><span class="p">.</span><span class="n">MaxDecPathLenToRoot</span><span class="p">,</span> <span class="n">pathLen</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="c1">// The initial new MaxPathLen is the max of the children ...</span>
<span class="kt">var</span> <span class="n">leftMaxPathLen</span> <span class="p">=</span> <span class="n">left</span> <span class="p">!=</span> <span class="k">null</span> <span class="p">?</span> <span class="n">left</span><span class="p">.</span><span class="n">MaxPathLen</span> <span class="p">:</span> <span class="m">0</span><span class="p">;</span>
<span class="kt">var</span> <span class="n">rightMaxPathLen</span> <span class="p">=</span> <span class="n">right</span> <span class="p">!=</span> <span class="k">null</span> <span class="p">?</span> <span class="n">right</span><span class="p">.</span><span class="n">MaxPathLen</span> <span class="p">:</span> <span class="m">0</span><span class="p">;</span>
<span class="n">returnValue</span><span class="p">.</span><span class="n">MaxPathLen</span> <span class="p">=</span> <span class="n">Math</span><span class="p">.</span><span class="nf">Max</span><span class="p">(</span><span class="n">leftMaxPathLen</span><span class="p">,</span> <span class="n">rightMaxPathLen</span><span class="p">);</span>
<span class="c1">// Take either of the two new max path lengths through the root if larger ...</span>
<span class="n">returnValue</span><span class="p">.</span><span class="n">MaxPathLen</span> <span class="p">=</span> <span class="n">Math</span><span class="p">.</span><span class="nf">Max</span><span class="p">(</span><span class="n">returnValue</span><span class="p">.</span><span class="n">MaxPathLen</span><span class="p">,</span> <span class="n">returnValue</span><span class="p">.</span><span class="n">MaxIncPathLenToRoot</span><span class="p">);</span>
<span class="n">returnValue</span><span class="p">.</span><span class="n">MaxPathLen</span> <span class="p">=</span> <span class="n">Math</span><span class="p">.</span><span class="nf">Max</span><span class="p">(</span><span class="n">returnValue</span><span class="p">.</span><span class="n">MaxPathLen</span><span class="p">,</span> <span class="n">returnValue</span><span class="p">.</span><span class="n">MaxDecPathLenToRoot</span><span class="p">);</span>
<span class="c1">// Now check for paths that go up and down through the root ...</span>
<span class="k">if</span><span class="p">(</span><span class="n">left</span> <span class="p">!=</span> <span class="k">null</span> <span class="p">&&</span> <span class="n">right</span> <span class="p">!=</span> <span class="k">null</span><span class="p">)</span> <span class="p">{</span>
<span class="k">if</span><span class="p">(</span><span class="n">root</span><span class="p">.</span><span class="n">Value</span> <span class="p">></span> <span class="n">left</span><span class="p">.</span><span class="n">RootValue</span> <span class="p">&&</span> <span class="n">root</span><span class="p">.</span><span class="n">Value</span> <span class="p"><</span> <span class="n">right</span><span class="p">.</span><span class="n">RootValue</span><span class="p">)</span> <span class="p">{</span>
<span class="kt">var</span> <span class="n">upDownPathLen</span> <span class="p">=</span> <span class="n">left</span><span class="p">.</span><span class="n">MaxIncPathLenToRoot</span> <span class="p">+</span> <span class="n">right</span><span class="p">.</span><span class="n">MaxDecPathLenToRoot</span> <span class="p">+</span> <span class="m">1</span><span class="p">;</span>
<span class="n">returnValue</span><span class="p">.</span><span class="n">MaxPathLen</span> <span class="p">=</span> <span class="n">Math</span><span class="p">.</span><span class="nf">Max</span><span class="p">(</span><span class="n">returnValue</span><span class="p">.</span><span class="n">MaxPathLen</span><span class="p">,</span> <span class="n">upDownPathLen</span><span class="p">);</span>
<span class="p">}</span>
<span class="k">else</span> <span class="k">if</span><span class="p">(</span><span class="n">root</span><span class="p">.</span><span class="n">Value</span> <span class="p"><</span> <span class="n">left</span><span class="p">.</span><span class="n">RootValue</span> <span class="p">&&</span> <span class="n">root</span><span class="p">.</span><span class="n">Value</span> <span class="p">></span> <span class="n">right</span><span class="p">.</span><span class="n">RootValue</span><span class="p">)</span> <span class="p">{</span>
<span class="kt">var</span> <span class="n">upDownPathLen</span> <span class="p">=</span> <span class="n">left</span><span class="p">.</span><span class="n">MaxDecPathLenToRoot</span> <span class="p">+</span> <span class="n">right</span><span class="p">.</span><span class="n">MaxIncPathLenToRoot</span> <span class="p">+</span> <span class="m">1</span><span class="p">;</span>
<span class="n">returnValue</span><span class="p">.</span><span class="n">MaxPathLen</span> <span class="p">=</span> <span class="n">Math</span><span class="p">.</span><span class="nf">Max</span><span class="p">(</span><span class="n">returnValue</span><span class="p">.</span><span class="n">MaxPathLen</span><span class="p">,</span> <span class="n">upDownPathLen</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="k">return</span> <span class="n">returnValue</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<h1 id="solution-validation">Solution validation</h1>
<p>Let’s verify that our code will solve the example that we provided before …</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">namespace</span> <span class="nn">GinjaSoft.TreeStuff.Tests.LongestPathTests</span>
<span class="p">{</span>
<span class="k">using</span> <span class="nn">Xunit</span><span class="p">;</span>
<span class="k">using</span> <span class="nn">Xunit.Abstractions</span><span class="p">;</span>
<span class="k">public</span> <span class="k">class</span> <span class="nc">GetNodeInfoTests</span>
<span class="p">{</span>
<span class="k">private</span> <span class="k">readonly</span> <span class="n">ITestOutputHelper</span> <span class="n">_output</span><span class="p">;</span>
<span class="k">public</span> <span class="nf">GetNodeInfoTests</span><span class="p">(</span><span class="n">ITestOutputHelper</span> <span class="n">output</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">_output</span> <span class="p">=</span> <span class="n">output</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">[</span><span class="n">Fact</span><span class="p">]</span>
<span class="k">public</span> <span class="k">void</span> <span class="nf">ThroughPathDoesNotIncludeRoot</span><span class="p">()</span>
<span class="p">{</span>
<span class="c1">// 10</span>
<span class="c1">// / \</span>
<span class="c1">// 30 40</span>
<span class="c1">// / \</span>
<span class="c1">// 40 20</span>
<span class="c1">// \</span>
<span class="c1">// 10</span>
<span class="kt">var</span> <span class="n">tree</span> <span class="p">=</span> <span class="k">new</span> <span class="n">Node</span><span class="p"><</span><span class="kt">uint</span><span class="p">>(</span><span class="m">10</span><span class="p">)</span> <span class="p">{</span>
<span class="n">Left</span> <span class="p">=</span> <span class="k">new</span> <span class="n">Node</span><span class="p"><</span><span class="kt">uint</span><span class="p">>(</span><span class="m">30</span><span class="p">)</span> <span class="p">{</span>
<span class="n">Left</span> <span class="p">=</span> <span class="k">new</span> <span class="n">Node</span><span class="p"><</span><span class="kt">uint</span><span class="p">>(</span><span class="m">40</span><span class="p">)</span> <span class="p">{</span> <span class="p">},</span>
<span class="n">Right</span> <span class="p">=</span> <span class="k">new</span> <span class="n">Node</span><span class="p"><</span><span class="kt">uint</span><span class="p">>(</span><span class="m">20</span><span class="p">)</span> <span class="p">{</span>
<span class="n">Right</span> <span class="p">=</span> <span class="k">new</span> <span class="n">Node</span><span class="p"><</span><span class="kt">uint</span><span class="p">>(</span><span class="m">10</span><span class="p">)</span> <span class="p">{</span> <span class="p">}</span>
<span class="p">}</span>
<span class="p">},</span>
<span class="n">Right</span> <span class="p">=</span> <span class="k">new</span> <span class="n">Node</span><span class="p"><</span><span class="kt">uint</span><span class="p">>(</span><span class="m">40</span><span class="p">)</span> <span class="p">{</span> <span class="p">}</span>
<span class="p">};</span>
<span class="kt">var</span> <span class="n">nodeInfo</span> <span class="p">=</span> <span class="nf">GetNodeInfo</span><span class="p">(</span><span class="n">tree</span><span class="p">);</span>
<span class="n">Assert</span><span class="p">.</span><span class="nf">Equal</span><span class="p">(</span><span class="m">10u</span><span class="p">,</span> <span class="n">nodeInfo</span><span class="p">.</span><span class="n">RootValue</span><span class="p">);</span>
<span class="n">Assert</span><span class="p">.</span><span class="nf">Equal</span><span class="p">(</span><span class="m">1u</span><span class="p">,</span> <span class="n">nodeInfo</span><span class="p">.</span><span class="n">MaxIncPathLenToRoot</span><span class="p">);</span>
<span class="n">Assert</span><span class="p">.</span><span class="nf">Equal</span><span class="p">(</span><span class="m">3u</span><span class="p">,</span> <span class="n">nodeInfo</span><span class="p">.</span><span class="n">MaxDecPathLenToRoot</span><span class="p">);</span>
<span class="n">Assert</span><span class="p">.</span><span class="nf">Equal</span><span class="p">(</span><span class="m">4u</span><span class="p">,</span> <span class="n">nodeInfo</span><span class="p">.</span><span class="n">MaxPathLen</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p><a href="https://www.urbandictionary.com/define.php?term=woot">Woot!</a> The test passes.</p>
<h1 id="resources">Resources</h1>
<ul>
<li>GitHub <a href="https://github.com/CaptainGinja/blog-trees-and-paths">repo</a> with code and full unit test suite</li>
</ul>
On an interesting 9 digit number2019-04-15T00:00:00+00:00http://alandavies.org/blog/2019/04/15/on-an-interesting-9-digit-number<h1 id="a-puzzle">A puzzle</h1>
<p>This post is inspired by a <a href="https://www.youtube.com/watch?v=3wv92-MAhss">puzzle</a> that was posed by Mr. James Grime on
his YouTube channel, singingbanana. The puzzle can be stated thus …</p>
<p>Find a nine digit number, using the digits 1 to 9 without repeats, such that the first two digits form a number
divisible by 2; the first three digits form a number divisible by 3; the first four digits form a number divisible by 4;
and so on up through 9. Trivially, the first digit will always be a number divisible by 1.</p>
<p>How many such numbers have this property?</p>
<h1 id="solution-strategies">Solution strategies</h1>
<p>Clearly we could write some code to implement a brute-force solution. We could generate all the possible 9 digit
numbers and test for the requisite properties of the sub-numbers. That would be inelegant though. Surely we can use
some clever mathematics to solve this, or at least to significantly pare down the space of possible solutions such that
we could solve it by hand. Let’s try.</p>
<h1 id="a-mathematical-approach">A mathematical approach</h1>
<p>Let’s use the letters \(a\) through \(i\) as labels for the digits in the number that we seek. So the number is
\(abcdefghi\) and we can describe its required properties using the following expressions …</p>
<p>A: \(10a + b \equiv 0 \bmod 2\)</p>
<p>B: \(100a + 10b + c \equiv 0 \bmod 3\)</p>
<p>C: \(1000a + 100b + 10c + d \equiv 0 \bmod 4\)</p>
<p>D: \(10000a + 1000b + 100c + 10d + e \equiv 0 \bmod 5\)</p>
<p>E: \(100000a + 10000b + 1000c + 100d + 10e + f \equiv 0 \bmod 6\)</p>
<p>F: \(1000000a + 100000b + 10000c + 1000d + 100e + 10f + g \equiv 0 \bmod 7\)</p>
<p>G: \(10000000a + 1000000b + 100000c + 10000d + 1000e + 100f + 10g + h \equiv 0 \bmod 8\)</p>
<p>H: \(100000000a + 10000000b + 1000000c + 100000d + 10000e + 1000f + 100g + 10h + i \equiv 0 \bmod 9\)</p>
<p>Now, since the coefficients of \(a, b, c, …\) can be reinterpretted as values modulo the appropriate number (2, 3,
4, …), we can simplify the above expressions. For example, \(10 \equiv 0 \bmod 2\) and so expression A becomes
\(b \equiv 0 \bmod 2\). Also \(100 \equiv 1 \bmod 3\) and \(10 \equiv 1 \bmod 3\), so expression B becomes
\(a + b + c \equiv 0 \bmod 3\). Overall …</p>
<p>A: \(b \equiv 0 \bmod 2\)</p>
<p>B: \(a + b + c \equiv 0 \bmod 3\)</p>
<p>C: \(2c + d \equiv 0 \bmod 4\)</p>
<p>D: \(e \equiv 0 \bmod 5\)</p>
<p>E: \(4a + 4b + 4c + 4d + 4e + f \equiv 0 \bmod 6\)</p>
<p>F: \(a + 5b + 4c + 6d + 2e + 3f + g \equiv 0 \bmod 7\)</p>
<p>G: \(4f + 2g + h \equiv 0 \bmod 8\)</p>
<p>H: \(a + b + c + d + e + f + g + h + i \equiv 0 \bmod 9\)</p>
<p>We also know that all of the values \(a, b, c, …\) have to be in \(\{1, 2, 3, 4, 5, 6, 7, 8, 9\}\) and must be
unique. So clearly \(e = 5\) and …</p>
<p>\(b\) is even (since \(b \equiv 0 \bmod 2\))</p>
<p>\(d\) is even (since \(2c + d\) is even and \(2c\) is even)</p>
<p>\(f\) is even (since \(4a + 4b + 4c + 4d + 4e + f\) is even and \(4 \times\) anything is even)</p>
<p>\(h\) is even (since \(4f + 2g + h\) is even and \(4f\) and \(2g\) are even)</p>
<p>And …</p>
<p>\(a\) is odd and not \(5\)</p>
<p>\(c\) is odd and not \(5\)</p>
<p>\(g\) is odd and not \(5\)</p>
<p>\(i\) is odd and not \(5\)</p>
<p>… since all the even numbers are taken by \(\{b, d, f, h\}\) and \(e = 5\).</p>
<p>Now, let’s look at expression C above, i.e. \(2c + d \equiv 0 \bmod 4\). We know that \(c \in \{1, 3, 7,
9\}\) so …</p>
<p>\(c = 1\): \(2 + d \equiv 0 \bmod 4 \implies d \equiv 2 \bmod 4\), or</p>
<p>\(c = 2\): \(6 + d \equiv 0 \bmod 4 \implies d \equiv 2 \bmod 4\), or</p>
<p>\(c = 3\): \(14 + d \equiv 0 \bmod 4 \implies d \equiv 2 \bmod 4\), or</p>
<p>\(c = 4\): \(18 + d \equiv 0 \bmod 4 \implies d \equiv 2 \bmod 4\)</p>
<p>We already know that \(d \in \{2, 4, 6, 8\}\). Of these, the only ones that satisfy \(d \equiv 2 \bmod 4\) are
\(\{2, 6\}\). So \(d \in \{2, 6\}\).</p>
<p>Now, let’s look at expression G above, i.e. \(4f + 2g + h \equiv 0 \bmod 8\). Since \(4f + 2g + h\) has to be a
muliple of 8 it also has to be a multiple of 4 too. So \(4f + 2g + h \equiv 0 \bmod 4\) which implies that
\(2g + h \equiv 0 \bmod 4\). Now, by the same logic as we applied above for \(2c + d \equiv 0 \bmod 4\) where
\(c \in \{1, 3, 7, 9\}\) we can take \(2g + h \equiv 0 \bmod 4\) where \(g \in \{1, 3, 7, 9\}\) and
deduce that \(h \in \{2, 6\}\).</p>
<p>The fact that \(d \in \{2, 6\}\) and \(h \in \{2, 6\}\) means that no other digit can be a 2 or a 6. So now
we know …</p>
<p>\(a \in \{1, 3, 7, 9\}\)</p>
<p>\(b \in \{4, 8\}\)</p>
<p>\(c \in \{1, 3, 7, 9\}\)</p>
<p>\(d \in \{2, 6\}\)</p>
<p>\(e = 5\)</p>
<p>\(f \in \{4, 8\}\)</p>
<p>\(g \in \{1, 3, 7, 9\}\)</p>
<p>\(h \in \{2, 6\}\)</p>
<p>\(i \in \{1, 3, 7, 9\}\)</p>
<p>Now, recall that expression E states that \(4a + 4b + 4c + 4d + 4e + f \equiv 0 \bmod 6\). But \(e = 5\) so this
becomes \(4a + 4b + 4c + 4d + 20 + f \equiv 0 \bmod 6\) which implies that
\(4a + 4b + 4c + 4d + (2 \bmod 6) + f \equiv 0 \bmod 6\) or \(4a + 4b + 4c + 4d + f \equiv 4 \bmod 6\).</p>
<p>Expression B says \(a + b + c \equiv 0 \bmod 3\) so \(4(a + b + c) + 4d + f \equiv 4 \bmod 6\) gives us
\(4(0 \bmod 3) + 4d + f \equiv 4 \bmod 6\) which imples that \((0 \bmod 12) + 4d + f \equiv 4 \bmod 6\) which
implies that \(4d + f \equiv 4 \bmod 6\).</p>
<p>We know that \(d \in \{2, 6\}\) …</p>
<p>\(d = 2 \implies 8 + f \equiv 4 \bmod 6 \implies 2 \bmod 6 + f \equiv 4 \bmod 6 \implies f \equiv 2 \bmod 6
\implies f = 8\)</p>
<p>\(d = 6 \implies 24 + f \equiv 4 \bmod 6 \implies 0 \bmod 6 + f \equiv 4 \bmod 6 \implies f \equiv 4 \bmod 6
\implies f = 4\)</p>
<p>So either \(def = 258\) or \(def = 654\)</p>
<p>Let’s assume \(def = 258\). So then \(b\) must be \(4\) and so \(abc\) could be \(143\), \(147\),
\(149\), \(341\), \(347\), \(349\), \(741\), \(743\), \(749\), \(941\), \(943\) or \(947\). But
\(abc\) must be divisible by \(3\) which eliminates several of these and leaves us with just \(147\) or \(741\).</p>
<p>If \(def = 654\) then \(b\) must be \(8\) and \(abc\) could be \(183\), \(187\), \(189\), \(381\),
\(387\), \(389\), \(781\), \(783\), \(789\), \(981\), \(983\) or \(987\). But, again, \(abc\) must be
divisible by \(3\) which leaves us with just \(183\) or \(189\), \(381\), \(387\), \(783\), \(789\),
\(981\) or \(987\).</p>
<p>So now we know that \(abcdef\) can be …</p>
<p>\(147258\)</p>
<p>\(741258\)</p>
<p>\(183654\)</p>
<p>\(189654\)</p>
<p>\(381654\)</p>
<p>\(387654\)</p>
<p>\(783654\)</p>
<p>\(789654\)</p>
<p>\(981654\)</p>
<p>\(987654\)</p>
<p>Enumerating the possible values of \(ghi\) gives us …</p>
<p>\(147258369\) - \(14725836\) is not divisible by \(8\)</p>
<p>\(147258963\) - \(1472589\) is not divisible by \(7\)</p>
<p>\(741258369\) - \(7412583\) is not divisible by \(7\)</p>
<p>\(741258963\) - \(7412589\) is not divisible by \(7\)</p>
<p>\(183654729\) - \(1836547\) is not divisible by \(7\)</p>
<p>\(183654927\) - \(1836549\) is not divisible by \(7\)</p>
<p>\(189654327\) - \(1896543\) is not divisible by \(7\)</p>
<p>\(189654723\) - \(1896547\) is not divisible by \(7\)</p>
<p>\(381654729\)</p>
<p>\(381654927\) - \(3816549\) is not divisible by \(7\)</p>
<p>\(387654129\) - \(3876541\) is not divisible by \(7\)</p>
<p>\(387654921\) - \(3876549\) is not divisible by \(7\)</p>
<p>\(783654129\) - \(7836541\) is not divisible by \(7\)</p>
<p>\(783654921\) - \(78365492\) is not divisible by \(8\)</p>
<p>\(789654123\) - \(7896541\) is not divisible by \(7\)</p>
<p>\(789654321\) - \(7896543\) is not divisible by \(7\)</p>
<p>\(981654327\) - \(9816543\) is not divisible by \(7\)</p>
<p>\(981654723\) - \(9816547\) is not divisible by \(7\)</p>
<p>\(987654123\) - \(9876541\) is not divisible by \(7\)</p>
<p>\(987654321\) - \(9876543\) is not divisible by \(7\)</p>
<p>So, only one number isn’t eliminated. The unqiue answer to the puzzle is \(381654729\).</p>
On parabolas and multiplication2019-01-14T00:00:00+00:00http://alandavies.org/blog/2019/01/14/on-parabolas-and-multiplication<p>I visited the National Museum of Mathematics in New York City with my family just recently. It’s a neat little museum
that can be experienced in a few hours and I highly recommend it. One of the exhibits there is called “String Product”.
It’s a large model of a paraboloid that sits in the middle of a spiral staircase between floors and illustrates an
interesting property of the simple parabola \(y = x^2\). If you take two positive numbers \(a\) and \(b\) and
draw two vertical lines, parallel to the y-axis, from \(x = -a\) and \(x = b\), and then draw a line through the
two points where these vertical lines cross the parabola, then that line will meet the y-axis at the value \(a * b\).
So this gives a nice geometric trick for multiplication.</p>
<p>Try it out …</p>
<div id="plot-1" style="margin:auto;width:fit-content"></div>
<p>
a: <input id="a-value" size="3" />
, b: <input id="b-value" size="3" />
, a * b = <input id="ab-value" size="3" disabled="true" />
<button id="button" onclick="buttonClick()">Plot</button>
</p>
<script>
function buttonClick() {
var a = document.getElementById("a-value").value;
var b = document.getElementById("b-value").value;
plot(a, b);
}
function checkNumber(x) {
return !(isNaN(x) || x < 0 || x > 10);
}
function checkNumbers(numbers) {
var ok = true;
numbers.forEach(function(x) {
if(!checkNumber(x)) {
alert('Inputs must be numbers between 0 and 10');
ok = false;
}
});
return ok;
}
function plot(a, b) {
if(!checkNumbers([a, b])) return;
// New line equation and annotation text labels
var lineEquation = (b - a) + 'x + ' + (a * b);
var aText = 'x = -' + a;
var bText = 'x = ' + b;
var productText = 'y = ' + (a * b);
// Update config
config = {
target: '#plot-1',
width: 700,
height: 500,
disableZoom: true,
xAxis: { domain: [-12, 12] },
yAxis: { domain: [0, 110] },
data: [ { fn: 'x^2', color: 'blue' } ],
annotations: []
};
// Reset the plot.
// It seems that I need to do this in order to get the new annotation text labels to show correctly.
functionPlot(config);
// New line plot and annotations
config.data[1] = { fn: lineEquation, color: 'green' };
config.annotations[0] = { x: -a, text: aText };
config.annotations[1] = { x: b, text: bText };
config.annotations[2] = { y: a * b, text: productText };
// Update the plot and the equation result value
functionPlot(config);
document.getElementById("ab-value").value = a * b;
}
// Initial values to plot
var aValue = 2;
var bValue = 4;
document.getElementById("a-value").value = aValue;
document.getElementById("b-value").value = bValue;
plot(aValue, bValue);
</script>
<h1 id="proof">Proof</h1>
<p>Why is this so? Let’s see.</p>
<p>Points on the parabola are parameterized by the coordinates \((x, x^2)\). So if we take the two positive integers
\(a\) and \(b\) and then look at the vertical lines \(x = -a\) and \(x = b\), the points where these cross the
parabola are \((-a, a^2)\) and \((b, b^2)\).</p>
<p>The general equation of a line is \(y = mx + c\) where \(m\) is the slope and \(c\) is the value at which the line
crosses the y-axis. To find \(m\) and \(c\) we can use the two points that we know are on the line, namely
\((-a, a^2)\) and \((b, b^2)\). So …</p>
<p>1) From point <script type="math/tex">(-a, a^2): a^2 = -ma + c</script></p>
<p>2) From point <script type="math/tex">(b, b^2): b^2 = mb + c</script></p>
<p>We can eliminate c from these two equations by subtracting them …</p>
<script type="math/tex; mode=display">b^2 - a^2 = mb + ma + c - c</script>
<script type="math/tex; mode=display">b^2 - a^2 = m(b + a)</script>
<p>Note that <script type="math/tex">b^2 - a^2 = (b + a)(b - a)</script> and so …</p>
<script type="math/tex; mode=display">(b + a)(b - a) = m(b + a)</script>
<p>By canceling, we get <script type="math/tex">m = (b - a)</script> and then substituting back in equation #2 above we get …</p>
<script type="math/tex; mode=display">b^2 = (b - a)b + c</script>
<script type="math/tex; mode=display">b^2 = b^2 - ab + c</script>
<script type="math/tex; mode=display">c = ab</script>
<p>So the value at which the line crosses the y-axis is equal to <script type="math/tex">ab</script>.</p>
<p>QED.</p>
On Primes2018-02-18T00:00:00+00:00http://alandavies.org/blog/2018/02/18/on-primes<p>A prime number is a positive integer (a natural number) that is only evenly divisible by itself and one. The number
one itself, by convention, is not considered a prime. A natural number, greater than one, that is not prime is said
to be composite.</p>
<p>Primes can be considered as the building blocks of all positive numbers. A statement that is more formally expressesd
by the <a href="https://en.wikipedia.org/wiki/Fundamental_theorem_of_arithmetic">fundametal theorem of arithmetic</a> which states
that every natural number, greater than one, is either a prime or can be factorized as a product of primes that is
unique except for their order.</p>
<p>Some interesting questions come to mind …</p>
<ol>
<li>How many primes are there?</li>
<li>How common are primes?</li>
<li>How do we determine whether a given number is prime?</li>
<li>What is the billionth prime?</li>
</ol>
<p>I already answered #1 in my post about <a href="/blog/2018/02/12/on-proof">proof</a>. There
are infinitely many.</p>
<p>The distribution of primes within the natural numbers can be statistically modelled. The
<a href="https://en.wikipedia.org/wiki/Prime_number_theorem">prime number theorem</a> formalizes the intuitive idea that primes
become less common as they become larger and introduces the
<a href="https://en.wikipedia.org/wiki/Prime-counting_function">prime counting function</a>, <script type="math/tex">\pi(N)</script>, defined as the number
of prime numbers less than or equal to N. The use of <script type="math/tex">\pi</script> as a function here is unrelated to the number
<script type="math/tex">\\pi</script>.</p>
<p>The prime number theorem states that …</p>
<script type="math/tex; mode=display">\pi(n) \sim \frac{n}{\log n}</script>
<p>Or more formally …</p>
<script type="math/tex; mode=display">\lim_{n\to\infty} \frac{\pi(n)}{\frac{n}{\log n}} = 1</script>
<p>This means that for large enough N, the probability that a random integer not greater than N is prime is very close to
<script type="math/tex">\frac{1}{\log N}</script>.</p>
<h1 id="how-to-determine-that-a-given-number-n-is-prime">How to determine that a given number, N, is prime</h1>
<p>Such a test is known as a primality test.</p>
<p>A simple brute force algorithm would be to enumerate all of the natural numbers less than N and see whether any of them
evenly divide N. If any of them do then N is composite, otherwise it is prime.</p>
<p>Actually, we only have to test potential divisors less than or equal to <script type="math/tex">\sqrt N</script>. This is because if N is
composite then at least one of its factors must be less than or equal to <script type="math/tex">\sqrt N</script>. To justify this assume that N
is composite and <script type="math/tex">N = a \times b</script>. If both <script type="math/tex">a</script> and <script type="math/tex">b</script> were greater than <script type="math/tex">\sqrt N</script> then <script type="math/tex">a \times
b</script> would be greater than N. This is clearly impossible and so at least one of the factors must be <script type="math/tex">% <![CDATA[
<= \sqrt N %]]></script>.</p>
<p>Ideally we would enumerate only the prime numbers less than N but this presupposes that we know all such primes. We
can limit the test divisors somewhat by noting that all primes > 3 can be written as <script type="math/tex">6n - 1</script> or <script type="math/tex">6n + 1</script> for
<script type="math/tex">n = 1, 2, ...</script> so so we only have to enumerate test factors of that form.</p>
<p>To see why this is so note that we can enumerate all natural numbers (> 5) as …</p>
<p><script type="math/tex">6n, 6n + 1, 6n + 2, 6n + 3, 6n + 4, 6n + 5</script> for <script type="math/tex">n = 1, 2, 3, ...</script></p>
<p>The numbers of the form <script type="math/tex">6n, 6n + 2</script> and <script type="math/tex">6n + 4</script> are all divisible by 2 and therefore are not prime. The numbers
of the form <script type="math/tex">6n</script> and <script type="math/tex">6n + 3</script> are all divisible by 3 and therefore are not prime either. This just leaves those of
the form <script type="math/tex">6n + 1</script> and <script type="math/tex">6n + 5</script> as candidate primes. The <script type="math/tex">6n+5</script> numbers are equivalent to <script type="math/tex">6n - 1</script>.</p>
<p>Here’s an initial implementation of the brute force primality test in C# …</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">namespace</span> <span class="nn">PrimeNumbers</span> <span class="p">{</span>
<span class="k">using</span> <span class="nn">System</span><span class="p">;</span>
<span class="k">public</span> <span class="k">static</span> <span class="k">class</span> <span class="nc">NumberExtentions</span> <span class="p">{</span>
<span class="k">public</span> <span class="k">static</span> <span class="kt">bool</span> <span class="nf">IsPrime</span><span class="p">(</span><span class="k">this</span> <span class="kt">ulong</span> <span class="n">number</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">if</span><span class="p">(</span><span class="n">number</span> <span class="p"><</span> <span class="m">2</span><span class="p">)</span> <span class="k">return</span> <span class="k">false</span><span class="p">;</span> <span class="c1">// 0 & 1 are not prime</span>
<span class="k">if</span><span class="p">(</span><span class="n">number</span> <span class="p"><</span> <span class="m">4</span><span class="p">)</span> <span class="k">return</span> <span class="k">true</span><span class="p">;</span> <span class="c1">// 2 & 3 are prime</span>
<span class="k">if</span><span class="p">(</span><span class="n">number</span> <span class="p">%</span> <span class="m">2</span> <span class="p">==</span> <span class="m">0</span><span class="p">)</span> <span class="k">return</span> <span class="k">false</span><span class="p">;</span> <span class="c1">// 4, 6, 8, 10, 12, 24, ... are composite</span>
<span class="k">if</span><span class="p">(</span><span class="n">number</span> <span class="p">%</span> <span class="m">3</span> <span class="p">==</span> <span class="m">0</span><span class="p">)</span> <span class="k">return</span> <span class="k">false</span><span class="p">;</span> <span class="c1">// 9, 15, 21, 27, 33, ... are composite</span>
<span class="c1">// Now test for factors of the form 6n - 1 and 6n + 1 for n = 1, 2, 3, ...</span>
<span class="c1">// 6n - 1 : 5, 11, 17, 23, 29, ...</span>
<span class="c1">// 6n + 1 : 7, 13, 19, 25, 31, ...</span>
<span class="c1">// ... up through floor(sqrt(number))</span>
<span class="kt">var</span> <span class="n">max</span> <span class="p">=</span> <span class="p">(</span><span class="kt">ulong</span><span class="p">)</span><span class="n">Math</span><span class="p">.</span><span class="nf">Sqrt</span><span class="p">(</span><span class="n">number</span><span class="p">);</span>
<span class="c1">// We will get here for number = 5, 7, 11, 13, 17, 19, 23, 25, 29, 31, 35, ...</span>
<span class="c1">// Those numbers with floor(sqrt(number)) < 5 will not go through this loop at all</span>
<span class="c1">// That's OK though since all of those numbers are prime: 5, 7, 11, 13, 17, 19, 23</span>
<span class="c1">// 6n - 1 and 6n + 1 for n = 1, 2, 3, ...</span>
<span class="c1">// is equivalent to n and n + 2 for n = 6m + 5 where m = 0, 1, 2, ...</span>
<span class="k">for</span><span class="p">(</span><span class="kt">ulong</span> <span class="n">n</span> <span class="p">=</span> <span class="m">5</span><span class="p">;</span> <span class="n">n</span> <span class="p"><=</span> <span class="n">max</span><span class="p">;</span> <span class="n">n</span> <span class="p">+=</span> <span class="m">6</span><span class="p">)</span> <span class="p">{</span>
<span class="k">if</span><span class="p">(</span><span class="n">number</span> <span class="p">%</span> <span class="n">n</span> <span class="p">==</span> <span class="m">0</span><span class="p">)</span> <span class="k">return</span> <span class="k">false</span><span class="p">;</span>
<span class="k">if</span><span class="p">(</span><span class="n">number</span> <span class="p">%</span> <span class="p">(</span><span class="n">n</span> <span class="p">+</span> <span class="m">2</span><span class="p">)</span> <span class="p">==</span> <span class="m">0</span><span class="p">)</span> <span class="k">return</span> <span class="k">false</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">return</span> <span class="k">true</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>We can also leverage known primes. If N is small then we can simply do a lookup into a list of all known primes up to
some value.</p>
<p>There are also probabilistic methods for testing primality, e.g. the
<a href="https://en.wikipedia.org/wiki/Miller%E2%80%93Rabin_primality_test">Miller-Rabin test</a>. This sort of method takes the
number to be tested along with a factor indicating the required accuracy. Methods like this are efficient and useful
when testing very large numbers for primality but for moderately sized numbers their performance lags significantly
behind the exhaustive test methods.</p>
<h1 id="the-nth-prime">The nth prime</h1>
<p>Being able to test a given number for primality is one thing but enumerating the primes in sequence is something else.
In order to determine what is the billionth prime we need to generate, and count, prime numbers from 2 up to one
billion. What’s an efficient way to do this?</p>
<p>Well, let’s start with brute force and then try to optimize from there. We’ll start by looking for the millionth
prime, a little less ambitious than the billionth to start with.</p>
<h2 id="method-1">Method 1</h2>
<p>Loop over all the natural numbers, test each for primality using the above IsPrime function, count the primes and stop
when we get to the millionth one. Do this all in a single thread.</p>
<p>Results …</p>
<p>With JIT compiler optimized C# code running on my 2017 MacBook Pro Core i7 @ 2.9GHz this takes, on average, 7.17
seconds to complete and then reports that the millionth prime is <script type="math/tex">15,485,863</script>. According to the
<a href="https://primes.utm.edu/nthprime/">Interwebz</a> that is the correct answer.</p>
<p>We can do better. We don’t need to test every natural number for primality. As in the IsPrime test function we can
iterate over the superset of the primes defined by <script type="math/tex">6n-1</script> and <script type="math/tex">6n+1</script>.</p>
<h2 id="method-2">Method 2</h2>
<p>As method 1 but looping over the smaller set of candidate primes.</p>
<p>Results …</p>
<p>A tiny bit better. Runtime is now 7.03 seconds on average. No significant improvement though.</p>
<p>We need a different approach.</p>
<h1 id="the-sieve-of-eratosthenes">The sieve of Eratosthenes</h1>
<p>We will look to a polymath from antiquity for inspiration, a man named
<a href="https://en.wikipedia.org/wiki/Eratosthenes">Eratosthenes of Cyrene</a>. He was a Greek mathematician, geographer, poet,
astronomer and music theorist who ultimately become the chief librarian of the Library at Alexandria. He is credited
with the invention of a simple simple algorithm for finding all the prime numbers up to any given limit. This
algorithm is now known as the Sieve of Eratosthenes.</p>
<p>The algorithm iteratively marks as composite the multiples of each number up to a given limit.</p>
<h2 id="example-to-find-all-the-primes-less-than-or-equal-to-30">Example: To find all the primes less than or equal to 30</h2>
<p>First list all the natural numbers from 2 to 30 …</p>
<script type="math/tex; mode=display">2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30</script>
<p>The first number in the list is <script type="math/tex">2</script>. Go through the list and cross out all the multiples of <script type="math/tex">2</script> other than <script type="math/tex">2</script>
itself..</p>
<script type="math/tex; mode=display">2, 3, \color{red}{4}, 5, \color{red}{6}, 7, \color{red}{8}, 9, \color{red}{10}, 11,
\color{red}{12}, 13, \color{red}{14}, 15, \color{red}{16}, 17, \color{red}{18}, 19,
\color{red}{20}, 21, \color{red}{22}, 23, \color{red}{24}, 25, \color{red}{26}, 27,
\color{red}{28}, 29, \color{red}{30}</script>
<p>The next uncrossed number in the list is <script type="math/tex">3</script>. Go through the list and cross out all the multiples of <script type="math/tex">3</script> other
than <script type="math/tex">3</script> itself …</p>
<script type="math/tex; mode=display">2, 3, \color{lightgrey}{4}, 5, \color{lightgrey}{6}, 7, \color{lightgrey}{8}, \color{red}{9},
\color{lightgrey}{10}, 11, \color{lightgrey}{12}, 13, \color{lightgrey}{14}, \color{red}{15},
\color{lightgrey}{16}, 17, \color{lightgrey}{18}, 19, \color{lightgrey}{20}, \color{red}{21},
\color{lightgrey}{22}, 23, \color{lightgrey}{24}, 25, \color{lightgrey}{26}, \color{red}{27},
\color{lightgrey}{28}, 29, \color{lightgrey}{30}</script>
<p>The next uncrossed number in the list is <script type="math/tex">5</script>. Go through the list and cross out all the multiples of <script type="math/tex">5</script> other
than <script type="math/tex">5</script> itself …</p>
<script type="math/tex; mode=display">2, 3, \color{lightgrey}{4}, 5, \color{lightgrey}{6}, 7, \color{lightgrey}{8}, \color{lightgrey}{9},
\color{lightgrey}{10}, 11, \color{lightgrey}{12}, 13, \color{lightgrey}{14}, \color{lightgrey}{15},
\color{lightgrey}{16}, 17, \color{lightgrey}{18}, 19, \color{lightgrey}{20}, \color{lightgrey}{21},
\color{lightgrey}{22}, 23, \color{lightgrey}{24}, \color{red}{25}, \color{lightgrey}{26}, \color{lightgrey}{27},
\color{lightgrey}{28}, 29, \color{lightgrey}{30}</script>
<p>The next uncrossed number in the list is <script type="math/tex">7</script>. Multiples of <script type="math/tex">7</script> will result in no more exclusions since all such
numbers have already been crossed out. We should note that this will be the case as soon as the first uncrossed out
number in a given pass through the array is greater than <script type="math/tex">\sqrt{N}</script> where N is the size of the array, <script type="math/tex">30</script> in our
example.</p>
<p>Note that all the even numbers will be crossed out as we walk through the array eliminating multiples of <script type="math/tex">2</script> so
there’s really no point writing them down in the first place. We can just write down the odd numbers and save half the
space. Of course we have to remember that there is one even prime though, namely <script type="math/tex">2</script>.</p>
<p>Another thing to note is that for each starting number, <script type="math/tex">n</script>, all the multiples of that number less than the square of
<script type="math/tex">n</script> will have already been crossed out, so we can start crossing out multiples of <script type="math/tex">n</script> from <script type="math/tex">n^2</script>.</p>
<p>At this point we are done. All of the remaining uncrossed numbers are the primes.</p>
<p>We can write an implementation of this algorithm in C# …</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">namespace</span> <span class="nn">PrimeNumbers</span> <span class="p">{</span>
<span class="k">using</span> <span class="nn">System</span><span class="p">;</span>
<span class="k">public</span> <span class="k">static</span> <span class="k">class</span> <span class="nc">Numbers</span> <span class="p">{</span>
<span class="k">public</span> <span class="k">static</span> <span class="n">IEnumerable</span><span class="p"><</span><span class="kt">uint</span><span class="p">></span> <span class="nf">PrimesLessThan</span><span class="p">(</span><span class="kt">uint</span> <span class="n">maxNumber</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">if</span><span class="p">(</span><span class="n">maxNumber</span> <span class="p"><</span> <span class="m">3</span><span class="p">)</span> <span class="k">yield</span> <span class="k">break</span><span class="p">;</span>
<span class="c1">// Allocate an array of flags</span>
<span class="c1">// We don't need to consider even numbers so we only need maxNumber / 2</span>
<span class="kt">var</span> <span class="n">numberIsComposite</span> <span class="p">=</span> <span class="k">new</span> <span class="kt">bool</span><span class="p">[</span><span class="n">maxNumber</span> <span class="p">/</span> <span class="m">2</span><span class="p">];</span>
<span class="k">yield</span> <span class="k">return</span> <span class="m">2</span><span class="p">;</span> <span class="c1">// 2 is the only even prime, output that one by default</span>
<span class="c1">// For odd n, 3 -> sqrt max, output primes and mark all prime multiples (from n^2) as composite</span>
<span class="kt">uint</span> <span class="n">n</span> <span class="p">=</span> <span class="m">3</span><span class="p">;</span>
<span class="kt">var</span> <span class="n">sqrtMaxNumber</span> <span class="p">=</span> <span class="p">(</span><span class="kt">uint</span><span class="p">)</span><span class="n">System</span><span class="p">.</span><span class="n">Math</span><span class="p">.</span><span class="nf">Sqrt</span><span class="p">(</span><span class="n">maxNumber</span><span class="p">);</span>
<span class="k">for</span><span class="p">(;</span> <span class="n">n</span> <span class="p"><=</span> <span class="n">sqrtMaxNumber</span><span class="p">;</span> <span class="n">n</span> <span class="p">+=</span> <span class="m">2</span><span class="p">)</span> <span class="p">{</span>
<span class="k">if</span><span class="p">(</span><span class="n">numberIsComposite</span><span class="p">[</span><span class="n">n</span> <span class="p">/</span> <span class="m">2</span><span class="p">])</span> <span class="k">continue</span><span class="p">;</span>
<span class="k">yield</span> <span class="k">return</span> <span class="n">n</span><span class="p">;</span>
<span class="k">for</span><span class="p">(</span><span class="kt">ulong</span> <span class="n">m</span> <span class="p">=</span> <span class="n">n</span> <span class="p">*</span> <span class="n">n</span><span class="p">;</span> <span class="n">m</span> <span class="p"><</span> <span class="n">maxNumber</span><span class="p">;</span> <span class="n">m</span> <span class="p">+=</span> <span class="m">2</span> <span class="p">*</span> <span class="n">n</span><span class="p">)</span> <span class="c1">// m is ulong to avoid overflow</span>
<span class="n">numberIsComposite</span><span class="p">[</span><span class="n">m</span> <span class="p">/</span> <span class="m">2</span><span class="p">]</span> <span class="p">=</span> <span class="k">true</span><span class="p">;</span>
<span class="p">}</span>
<span class="c1">// Continue to walk through the rest of the array and output each prime number</span>
<span class="k">for</span><span class="p">(;</span> <span class="n">n</span> <span class="p"><</span> <span class="n">maxNumber</span><span class="p">;</span> <span class="n">n</span> <span class="p">+=</span> <span class="m">2</span><span class="p">)</span>
<span class="k">if</span><span class="p">(!</span><span class="n">numberIsComposite</span><span class="p">[</span><span class="n">n</span> <span class="p">/</span> <span class="m">2</span><span class="p">])</span> <span class="k">yield</span> <span class="k">return</span> <span class="n">n</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>This will generate each prime up to and including maxNumber.</p>
<p>We can use this algorithm to find the millionth prime. The only tricky part is that we need to pass in a value for the
largest number to consider. In other words we need to come up with an estimate for how big the millionth prime will
be. We will address this limitation in due course but for now let’s just get a rough benchmark on the performance of
this strategy.</p>
<h2 id="method-3">Method 3</h2>
<p>Use the Sieve of Eratosthenes with an upper bound of <script type="math/tex">15,500,000</script> for the internal array of numbers. Count the
resulting primes and output the millionth one. Here’s the code …</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">namespace</span> <span class="nn">ScratchConsoleApp</span>
<span class="p">{</span>
<span class="k">using</span> <span class="nn">System</span><span class="p">;</span>
<span class="k">using</span> <span class="nn">System.Diagnostics</span><span class="p">;</span>
<span class="k">using</span> <span class="nn">PrimeNumbers</span><span class="p">;</span>
<span class="k">class</span> <span class="nc">Program</span>
<span class="p">{</span>
<span class="k">static</span> <span class="k">void</span> <span class="nf">Main</span><span class="p">(</span><span class="kt">string</span><span class="p">[]</span> <span class="n">args</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">const</span> <span class="kt">uint</span> <span class="n">maxNumber</span> <span class="p">=</span> <span class="m">15500000</span><span class="p">;</span> <span class="c1">// Magically chosen max value ...</span>
<span class="kt">uint</span> <span class="n">count</span> <span class="p">=</span> <span class="m">0</span><span class="p">;</span>
<span class="kt">uint</span> <span class="n">nthPrime</span> <span class="p">=</span> <span class="m">0</span><span class="p">;</span>
<span class="kt">var</span> <span class="n">sw</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">Stopwatch</span><span class="p">();</span>
<span class="n">sw</span><span class="p">.</span><span class="nf">Start</span><span class="p">();</span>
<span class="k">foreach</span><span class="p">(</span><span class="kt">var</span> <span class="n">prime</span> <span class="k">in</span> <span class="n">Numbers</span><span class="p">.</span><span class="nf">PrimesLessThan</span><span class="p">(</span><span class="n">maxNumber</span><span class="p">))</span> <span class="p">{</span>
<span class="p">++</span><span class="n">count</span><span class="p">;</span>
<span class="k">if</span><span class="p">(</span><span class="n">count</span> <span class="p">==</span> <span class="n">n</span><span class="p">)</span> <span class="n">nthPrime</span> <span class="p">=</span> <span class="n">prime</span><span class="p">;</span>
<span class="p">}</span>
<span class="n">sw</span><span class="p">.</span><span class="nf">Stop</span><span class="p">();</span>
<span class="n">Console</span><span class="p">.</span><span class="nf">WriteLine</span><span class="p">(</span><span class="s">$"The </span><span class="p">{</span><span class="n">n</span><span class="p">}</span><span class="s">th prime is </span><span class="p">{</span><span class="n">nthPrime</span><span class="p">}</span><span class="s">"</span><span class="p">);</span>
<span class="n">Console</span><span class="p">.</span><span class="nf">WriteLine</span><span class="p">(</span><span class="s">$"Elapsed = </span><span class="p">{</span><span class="n">sw</span><span class="p">.</span><span class="n">Elapsed</span><span class="p">}</span><span class="s">"</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Results …</p>
<p>With JIT compiler optimized C# code running on my 2017 MacBook Pro Core i7 @ 2.9GHz this takes, on average, 0.13
seconds to complete and then reports that the millionth prime is <script type="math/tex">15,485,863</script>, which we know to be correct.</p>
<p>Wow! That’s much faster than before.</p>
<p>This implementation has additional scope for improvement as well.</p>
<ul>
<li>Support finding primes larger than uint.MaxValue</li>
<li>Create a simpler interface to allow us to request the Nth prime and remove the need to specify an upper bound for the
value of the prime</li>
<li>Chunking, to achieve better locality of memory access (to improve CPU L1 and L2 cache hit rates)</li>
<li>Use threads to leverage multiple CPU cores in parallel</li>
</ul>
<p>There are some simple tweaks too. It turns out that the IEnumerable interface and the yield return construct have some
overhead. If we change the code to …</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">namespace</span> <span class="nn">PrimeNumbers</span> <span class="p">{</span>
<span class="k">using</span> <span class="nn">System</span><span class="p">;</span>
<span class="k">public</span> <span class="k">static</span> <span class="k">class</span> <span class="nc">Numbers</span> <span class="p">{</span>
<span class="k">public</span> <span class="k">static</span> <span class="k">void</span> <span class="nf">ForEachPrimeLessThan</span><span class="p">(</span><span class="kt">uint</span> <span class="n">maxNumber</span><span class="p">,</span> <span class="n">Action</span><span class="p"><</span><span class="kt">uint</span><span class="p">></span> <span class="n">primeFn</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">if</span><span class="p">(</span><span class="n">maxNumber</span> <span class="p"><</span> <span class="m">3</span><span class="p">)</span> <span class="k">return</span><span class="p">;</span>
<span class="c1">// Allocate an array of flags</span>
<span class="c1">// We don't need to consider even numbers so we only need maxNumber / 2</span>
<span class="kt">var</span> <span class="n">numberIsComposite</span> <span class="p">=</span> <span class="k">new</span> <span class="kt">bool</span><span class="p">[</span><span class="n">maxNumber</span> <span class="p">/</span> <span class="m">2</span><span class="p">];</span>
<span class="nf">primeFn</span><span class="p">(</span><span class="m">2</span><span class="p">);</span> <span class="c1">// 2 is the only even prime, output that one by default</span>
<span class="c1">// For odd n, 3 -> sqrt max, output primes and mark all prime multiples (from n^2) as composite</span>
<span class="kt">uint</span> <span class="n">n</span> <span class="p">=</span> <span class="m">3</span><span class="p">;</span>
<span class="kt">var</span> <span class="n">sqrtMaxNumber</span> <span class="p">=</span> <span class="p">(</span><span class="kt">uint</span><span class="p">)</span><span class="n">System</span><span class="p">.</span><span class="n">Math</span><span class="p">.</span><span class="nf">Sqrt</span><span class="p">(</span><span class="n">maxNumber</span><span class="p">);</span>
<span class="k">for</span><span class="p">(;</span> <span class="n">n</span> <span class="p"><=</span> <span class="n">sqrtMaxNumber</span><span class="p">;</span> <span class="n">n</span> <span class="p">+=</span> <span class="m">2</span><span class="p">)</span> <span class="p">{</span>
<span class="k">if</span><span class="p">(</span><span class="n">numberIsComposite</span><span class="p">[</span><span class="n">n</span> <span class="p">/</span> <span class="m">2</span><span class="p">])</span> <span class="k">continue</span><span class="p">;</span>
<span class="nf">primeFn</span><span class="p">(</span><span class="n">n</span><span class="p">);</span>
<span class="k">for</span><span class="p">(</span><span class="kt">ulong</span> <span class="n">m</span> <span class="p">=</span> <span class="n">n</span> <span class="p">*</span> <span class="n">n</span><span class="p">;</span> <span class="n">m</span> <span class="p"><</span> <span class="n">maxNumber</span><span class="p">;</span> <span class="n">m</span> <span class="p">+=</span> <span class="m">2</span> <span class="p">*</span> <span class="n">n</span><span class="p">)</span> <span class="c1">// m is ulong to avoid overflow</span>
<span class="n">numberIsComposite</span><span class="p">[</span><span class="n">m</span> <span class="p">/</span> <span class="m">2</span><span class="p">]</span> <span class="p">=</span> <span class="k">true</span><span class="p">;</span>
<span class="p">}</span>
<span class="c1">// Continue to walk through the rest of the array and output each prime number</span>
<span class="k">for</span><span class="p">(;</span> <span class="n">n</span> <span class="p"><</span> <span class="n">maxNumber</span><span class="p">;</span> <span class="n">n</span> <span class="p">+=</span> <span class="m">2</span><span class="p">)</span>
<span class="k">if</span><span class="p">(!</span><span class="n">numberIsComposite</span><span class="p">[</span><span class="n">n</span> <span class="p">/</span> <span class="m">2</span><span class="p">])</span> <span class="nf">primeFn</span><span class="p">(</span><span class="n">n</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>… and use it like this …</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">namespace</span> <span class="nn">ScratchConsoleApp</span>
<span class="p">{</span>
<span class="k">using</span> <span class="nn">System</span><span class="p">;</span>
<span class="k">using</span> <span class="nn">System.Diagnostics</span><span class="p">;</span>
<span class="k">using</span> <span class="nn">PrimeNumbers</span><span class="p">;</span>
<span class="k">class</span> <span class="nc">Program</span>
<span class="p">{</span>
<span class="k">static</span> <span class="k">void</span> <span class="nf">Main</span><span class="p">(</span><span class="kt">string</span><span class="p">[]</span> <span class="n">args</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">const</span> <span class="kt">uint</span> <span class="n">maxNumber</span> <span class="p">=</span> <span class="m">15</span><span class="n">_500_000</span><span class="p">;</span> <span class="c1">// Magically chosen max value ...</span>
<span class="kt">uint</span> <span class="n">count</span> <span class="p">=</span> <span class="m">0</span><span class="p">;</span>
<span class="kt">uint</span> <span class="n">nthPrime</span> <span class="p">=</span> <span class="m">0</span><span class="p">;</span>
<span class="kt">var</span> <span class="n">sw</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">Stopwatch</span><span class="p">();</span>
<span class="n">sw</span><span class="p">.</span><span class="nf">Start</span><span class="p">();</span>
<span class="n">Numbers</span><span class="p">.</span><span class="nf">ForEachPrimeLessThan</span><span class="p">(</span><span class="n">maxNumber</span><span class="p">,</span> <span class="n">prime</span> <span class="p">=></span> <span class="p">{</span>
<span class="p">++</span><span class="n">count</span><span class="p">;</span>
<span class="k">if</span><span class="p">(</span><span class="n">count</span> <span class="p">==</span> <span class="n">n</span><span class="p">)</span> <span class="n">nthPrime</span> <span class="p">=</span> <span class="n">prime</span><span class="p">;</span>
<span class="p">});</span>
<span class="n">sw</span><span class="p">.</span><span class="nf">Stop</span><span class="p">();</span>
<span class="n">Console</span><span class="p">.</span><span class="nf">WriteLine</span><span class="p">(</span><span class="s">$"The </span><span class="p">{</span><span class="n">n</span><span class="p">}</span><span class="s">th prime is </span><span class="p">{</span><span class="n">nthPrime</span><span class="p">}</span><span class="s">"</span><span class="p">);</span>
<span class="n">Console</span><span class="p">.</span><span class="nf">WriteLine</span><span class="p">(</span><span class="s">$"Elapsed = </span><span class="p">{</span><span class="n">sw</span><span class="p">.</span><span class="n">Elapsed</span><span class="p">}</span><span class="s">"</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>… then the results are slightly better.</p>
<h1 id="a-better-sieve">A Better sieve</h1>
<p>Let’s make our sieve implementation better. First we want to support finding larger primes which means that we need to
use 64 bit integers (ulong) as opposed to 32 bit (uint). For the smaller primes this will be a waste of space but we
will need the extra capacity in order to work with primes up to and including the billionth one.</p>
<p>Our existing sieve implementation uses a single working array of bool where the index of each element corresponds to a
number that we want to test for primality. This means that the maximum number that we can test is the array size less
one. Now, the .Net Framework imposes a
<a href="https://stackoverflow.com/questions/1391672/what-is-the-maximum-size-that-an-array-can-hold">limit</a> of Int32.MaxValue
(<script type="math/tex">2^{31}-1</script>) on the number of elements in an
<a href="https://msdn.microsoft.com/en-us/library/System.Array(v=vs.110).aspx">array</a> and therefore also imposes a limit on the
size of the maximum prime that we can find. We optimize our use of the working array by ignoring even numbers (> 2) and
associating index <script type="math/tex">n</script> with the number <script type="math/tex">2n + 1</script> but this still caps us out at 4 billion or so. To test larger
numbers we will need a different approach.</p>
<h2 id="sieving-in-blocks">Sieving in blocks</h2>
<p>We can search for primes in blocks and reuse our working array for each block. Let’s look at an example of how this
will work. Say we want to limit the size of our working array to 30 elements but we want to find all the primes up to
300.</p>
<p>Recall that we only ever need to eliminate multiples of primes from our working array where the primes are less than or
equal to the square root of the largest number in the array. Now, floor(sqrt(300)) = 17 and so first we need to find
all the primes up to 17. To do this we need a working array with 9 elements. We assume that we have a working array
with 30 elements so we have enough for this first task.</p>
<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Array index (n): 0 1 2 3 4 5 6 7 8 9 10 ... 29
Corresponding number (2n + 1): 1 3 5 7 9 11 13 15 17 19 21 ... 59
</code></pre></div></div>
<p>We run the basic sieve (as before) on this working array to generate the primes: 2, 3, 5, 7, 11, 13 & 17.</p>
<p>Now with this list of primes saved we will proceed to examine the numbers up to 300 in blocks of 60 (2 * working array
size). Here’s the first block …</p>
<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>block #1
startNumber (s) = 0
n: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
s+2n+1: 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59
</code></pre></div></div>
<p>We eliminate all multiples of the primes that we determined before. For each prime <script type="math/tex">p</script> we eliminate <script type="math/tex">p^2 + 2kp</script> for
<script type="math/tex">k = 0, 1, 2, ...</script> until we pass the end of the block.</p>
<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>block #1
startNumber (s) = 0
n: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
s+2n+1: 1 3 5 7 11 13 17 19 23 29 31 37 41 43 47 53 57
</code></pre></div></div>
<p>Now the next block …</p>
<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>block #2
startNumber (s) = 60
n: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 ... 29
s+2n+1: 61 63 65 67 69 71 73 75 77 79 81 83 85 87 89 91 93 95 97 99 101 103 105 107 109 111 ... 119
</code></pre></div></div>
<p>… which becomes the following after eliminating multiples of our primes …</p>
<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>block #2
startNumber (s) = 60
n: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 ... 29
s+2n+1: 61 67 71 73 79 83 89 97 101 103 107 109 ...
</code></pre></div></div>
<p>Let’s look at a later block …</p>
<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>block #5
startNumber (s) = 240
n: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 ... 29
s+2n+1: 241 243 245 247 249 251 253 255 257 259 261 263 265 267 269 271 273 275 277 279 281 ... 299
</code></pre></div></div>
<p>Here the first number in the block is 241. As we enumerate our primes we end up skipping over many multiples of them
before we get to a number that is in the block.</p>
<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>p p^2 +2p +2p +2p +2p +2p +2p +2p +2p +2p +2p +2p +2p +2p +2p +2p +2p +2p +2p +2p +2p +2p +2p ...
3 9 15 21 27 33 39 45 51 57 63 69 75 81 87 93 99 105 111 117 123 129 135 141 ...
5 25 35 45 55 65 75 85 95 105 115 125 135 145 155 165 175 185 195 205 215 225 235 245 ...
7 49 63 77 91 105 119 133 147 161 175 189 203 217 231 245 259 273 287 301 ...
11 121 143 165 187 209 231 253 275 297 319 ...
13 169 195 221 247 273 299 ...
17 289 ...
</code></pre></div></div>
<p>Rather than having to iterate over all of these redundant prime multiples we can skip ahead with some appropriate
arithmetic (see below).</p>
<p>Here’s some C# code to sieve a block …</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">private</span> <span class="k">static</span> <span class="k">void</span> <span class="nf">SieveBlock</span><span class="p">(</span><span class="kt">bool</span><span class="p">[]</span> <span class="n">isPrime</span><span class="p">,</span>
<span class="n">IReadOnlyList</span><span class="p"><</span><span class="kt">ulong</span><span class="p">></span> <span class="n">primes</span><span class="p">,</span>
<span class="kt">uint</span> <span class="n">startIndex</span><span class="p">,</span>
<span class="kt">ulong</span> <span class="n">startNumber</span><span class="p">,</span>
<span class="kt">ulong</span> <span class="n">endNumber</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">foreach</span><span class="p">(</span><span class="kt">var</span> <span class="n">p</span> <span class="k">in</span> <span class="n">primes</span><span class="p">)</span> <span class="p">{</span>
<span class="k">if</span><span class="p">(</span><span class="n">p</span> <span class="p">==</span> <span class="m">2</span><span class="p">)</span> <span class="k">continue</span><span class="p">;</span> <span class="c1">// 2 is a special case not covered by our working array</span>
<span class="c1">// Start eliminating elements of the working array from p^2</span>
<span class="kt">var</span> <span class="n">n</span> <span class="p">=</span> <span class="n">p</span> <span class="p">*</span> <span class="n">p</span><span class="p">;</span>
<span class="k">if</span><span class="p">(</span><span class="n">n</span> <span class="p">></span> <span class="n">endNumber</span><span class="p">)</span> <span class="k">break</span><span class="p">;</span>
<span class="c1">// In the for loop below, n = p*p + 2*k*p for +ve integers k = 0, 1, 2, ...</span>
<span class="c1">// We don't need to do anything until n is >= startNum so fast forward until that is true</span>
<span class="k">if</span><span class="p">(</span><span class="n">n</span> <span class="p"><</span> <span class="n">startNumber</span><span class="p">)</span> <span class="p">{</span>
<span class="c1">// Find the smallest k such that n + 2*k*p >= startNum</span>
<span class="c1">// k >= (startNum - n) / (2 * p)</span>
<span class="c1">// k >= a / b for a = startNum - n and b = 2 * p</span>
<span class="c1">// k = a / b if a % b == 0 otherwise k = (a / b) + 1</span>
<span class="kt">var</span> <span class="n">a</span> <span class="p">=</span> <span class="n">startNumber</span> <span class="p">-</span> <span class="n">n</span><span class="p">;</span>
<span class="kt">var</span> <span class="n">b</span> <span class="p">=</span> <span class="m">2</span> <span class="p">*</span> <span class="n">p</span><span class="p">;</span>
<span class="kt">var</span> <span class="n">k</span> <span class="p">=</span> <span class="n">a</span> <span class="p">%</span> <span class="n">b</span> <span class="p">==</span> <span class="m">0</span> <span class="p">?</span> <span class="n">a</span> <span class="p">/</span> <span class="n">b</span> <span class="p">:</span> <span class="p">(</span><span class="n">a</span> <span class="p">/</span> <span class="n">b</span><span class="p">)</span> <span class="p">+</span> <span class="m">1</span><span class="p">;</span>
<span class="n">n</span> <span class="p">=</span> <span class="n">n</span> <span class="p">+</span> <span class="p">(</span><span class="m">2</span> <span class="p">*</span> <span class="n">k</span> <span class="p">*</span> <span class="n">p</span><span class="p">);</span>
<span class="p">}</span>
<span class="c1">// Mark all multiples of this prime as composite in the current range of the working array</span>
<span class="k">for</span><span class="p">(;</span> <span class="n">n</span> <span class="p"><</span> <span class="n">endNumber</span><span class="p">;</span> <span class="n">n</span> <span class="p">+=</span> <span class="m">2</span> <span class="p">*</span> <span class="n">p</span><span class="p">)</span> <span class="p">{</span>
<span class="kt">var</span> <span class="n">index</span> <span class="p">=</span> <span class="n">startIndex</span> <span class="p">+</span> <span class="p">(</span><span class="n">n</span> <span class="p">-</span> <span class="n">startNumber</span><span class="p">)</span> <span class="p">/</span> <span class="m">2</span><span class="p">;</span>
<span class="n">isPrime</span><span class="p">[</span><span class="n">index</span><span class="p">]</span> <span class="p">=</span> <span class="k">false</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Using block processing like this we can come up with a sieve that will work for much larger numbers.</p>
<h2 id="method-4">Method 4</h2>
<p>Use a block processing Sieve of Eratosthenes with a block size of 500,000 and an upper bound of 15,500,000. Count the
resulting primes and output the millionth one.</p>
<p>Results …</p>
<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>c:\src\eratosthenes\BlockSieve.exe -blockSize=500000 -maxNumber=15500000 -nthPrime=1000000
The 1,000,000th prime is 15,485,863
Elapsed = 00:00:00.0622126
</code></pre></div></div>
<p>What about the 100 millionth prime? Well, our implementation can do that now.</p>
<p>Results …</p>
<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>c:\src\eratosthenes\BlockSieve.exe -blockSize=500000 -maxNumber=2100000000 -nthPrime=100000000
The 100,000,000th prime is 2,038,074,743
Elapsed = 00:00:09.6867032
</code></pre></div></div>
<p>What about the 500 millionth prime?</p>
<p>Results …</p>
<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>c:\src\eratosthenes\BlockSieve.exe -blockSize=500000 -maxNumber=11100000000 -nthPrime=500000000
The 500,000,000th prime is 11,037,271,757
Elapsed = 00:00:57.2004362
</code></pre></div></div>
<p>But what about that billionth prime?</p>
<p>Results …</p>
<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>c:\src\eratosthenes\BlockSieve.exe -blockSize=500000 -maxNumber=23000000000 -nthPrime=1000000000
The 1,000,000,000th prime is 22,801,763,489
Elapsed = 00:02:05.9492586
</code></pre></div></div>
<p>According to the <a href="https://primes.utm.edu/nthprime/">Interwebz</a> that is indeed the correct answer for the billionth
prime.</p>
<p>But wait, we’re not done …</p>
<h2 id="parallel-block-processing">Parallel block processing</h2>
<p>Once we have determined our intial list of primes then we can eliminate composite numbers from blocks in parallel.</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">private</span> <span class="k">static</span> <span class="k">void</span> <span class="nf">ParallelSieveBlocks</span><span class="p">(</span><span class="kt">ulong</span> <span class="n">endNumber</span><span class="p">,</span>
<span class="kt">uint</span> <span class="n">blockSize</span><span class="p">,</span>
<span class="n">Func</span><span class="p"><</span><span class="kt">ulong</span><span class="p">,</span> <span class="kt">bool</span><span class="p">></span> <span class="n">primeFn</span><span class="p">,</span>
<span class="kt">bool</span><span class="p">[]</span> <span class="n">isPrime</span><span class="p">,</span>
<span class="n">IReadOnlyList</span><span class="p"><</span><span class="kt">ulong</span><span class="p">></span> <span class="n">primes</span><span class="p">,</span>
<span class="kt">uint</span> <span class="n">numTasks</span><span class="p">,</span>
<span class="kt">ulong</span> <span class="n">startNum</span><span class="p">)</span>
<span class="p">{</span>
<span class="kt">var</span> <span class="n">tasks</span> <span class="p">=</span> <span class="k">new</span> <span class="n">List</span><span class="p"><</span><span class="n">Task</span><span class="p">>((</span><span class="kt">int</span><span class="p">)</span><span class="n">numTasks</span><span class="p">);</span>
<span class="k">for</span><span class="p">(</span><span class="kt">uint</span> <span class="n">m</span> <span class="p">=</span> <span class="m">0</span><span class="p">;</span> <span class="n">m</span> <span class="p"><</span> <span class="n">numTasks</span><span class="p">;</span> <span class="p">++</span><span class="n">m</span><span class="p">)</span> <span class="p">{</span>
<span class="c1">// Init the start index of the section of the working array that this task will use and the</span>
<span class="c1">// bounds of the numbers that it will process</span>
<span class="kt">var</span> <span class="n">taskStartIndex</span> <span class="p">=</span> <span class="n">m</span> <span class="p">*</span> <span class="n">blockSize</span><span class="p">;</span>
<span class="kt">var</span> <span class="n">taskStartNum</span> <span class="p">=</span> <span class="n">startNum</span> <span class="p">+</span> <span class="p">(</span><span class="n">m</span> <span class="p">*</span> <span class="m">2</span> <span class="p">*</span> <span class="n">blockSize</span><span class="p">);</span>
<span class="kt">var</span> <span class="n">taskEndNum</span> <span class="p">=</span> <span class="n">taskStartNum</span> <span class="p">+</span> <span class="p">(</span><span class="m">2</span> <span class="p">*</span> <span class="n">blockSize</span><span class="p">);</span>
<span class="k">if</span><span class="p">(</span><span class="n">taskStartNum</span> <span class="p">>=</span> <span class="n">endNumber</span><span class="p">)</span> <span class="k">break</span><span class="p">;</span> <span class="c1">// Short circuit if there's no more work to</span>
<span class="k">if</span><span class="p">(</span><span class="n">taskEndNum</span> <span class="p">></span> <span class="n">endNumber</span><span class="p">)</span> <span class="n">taskEndNum</span> <span class="p">=</span> <span class="n">endNumber</span><span class="p">;</span> <span class="c1">// Adjust endNum if we are the last block</span>
<span class="n">tasks</span><span class="p">.</span><span class="nf">Add</span><span class="p">(</span><span class="n">Task</span><span class="p">.</span><span class="nf">Run</span><span class="p">(()</span> <span class="p">=></span> <span class="nf">SieveBlock</span><span class="p">(</span><span class="n">isPrime</span><span class="p">,</span> <span class="n">primes</span><span class="p">,</span> <span class="n">taskStartIndex</span><span class="p">,</span> <span class="n">taskStartNum</span><span class="p">,</span> <span class="n">taskEndNum</span><span class="p">)));</span>
<span class="p">}</span>
<span class="n">Task</span><span class="p">.</span><span class="nf">WaitAll</span><span class="p">(</span><span class="n">tasks</span><span class="p">.</span><span class="nf">ToArray</span><span class="p">());</span>
<span class="p">}</span>
</code></pre></div></div>
<p>How many parallel tasks can we spawn though? There’s really no point in using more than the number of cores on the
local system. Thus we will end up with a partially parallel but still somewhat serial algorithm.</p>
<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">public</span> <span class="k">static</span> <span class="k">void</span> <span class="nf">ParallelSieve</span><span class="p">(</span><span class="kt">ulong</span> <span class="n">endNumber</span><span class="p">,</span>
<span class="n">Func</span><span class="p"><</span><span class="kt">ulong</span><span class="p">,</span> <span class="kt">bool</span><span class="p">></span> <span class="n">primeFn</span><span class="p">,</span>
<span class="kt">uint</span> <span class="n">blockSize</span> <span class="p">=</span> <span class="m">600000</span><span class="p">,</span>
<span class="kt">uint</span> <span class="n">numTasks</span> <span class="p">=</span> <span class="m">0</span><span class="p">)</span>
<span class="p">{</span>
<span class="c1">// Save the value of floor(sqrt(endNumber)) and use it as a cap on the initial set of primes</span>
<span class="kt">var</span> <span class="n">sqrt</span> <span class="p">=</span> <span class="p">(</span><span class="kt">ulong</span><span class="p">)</span><span class="n">Math</span><span class="p">.</span><span class="nf">Sqrt</span><span class="p">(</span><span class="n">endNumber</span><span class="p">);</span>
<span class="k">if</span><span class="p">(</span><span class="n">numTasks</span> <span class="p">==</span> <span class="m">0</span><span class="p">)</span> <span class="n">numTasks</span> <span class="p">=</span> <span class="p">(</span><span class="kt">uint</span><span class="p">)</span><span class="n">Environment</span><span class="p">.</span><span class="n">ProcessorCount</span><span class="p">;</span>
<span class="c1">// We allocate a working array of bool with blockSize elements for each task</span>
<span class="kt">var</span> <span class="n">arraySize</span> <span class="p">=</span> <span class="n">blockSize</span> <span class="p">*</span> <span class="n">numTasks</span><span class="p">;</span>
<span class="c1">// We will use this same array for the initial serial sieve to find the primes up to 'sqrt' so</span>
<span class="c1">// it needs to be at least big enough to hold all the numbers up to that value. Remember that</span>
<span class="c1">// we are only allocating half the values though (the / 2 optimization) so we need to account</span>
<span class="c1">// for that.</span>
<span class="kt">var</span> <span class="n">serialSieveSize</span> <span class="p">=</span> <span class="p">(</span><span class="n">sqrt</span> <span class="p">+</span> <span class="m">1</span><span class="p">)</span> <span class="p">/</span> <span class="m">2</span><span class="p">;</span>
<span class="k">if</span><span class="p">(</span><span class="n">arraySize</span> <span class="p"><</span> <span class="n">serialSieveSize</span><span class="p">)</span> <span class="n">arraySize</span> <span class="p">=</span> <span class="p">(</span><span class="kt">uint</span><span class="p">)</span><span class="n">serialSieveSize</span><span class="p">;</span>
<span class="c1">// Allocate and init the Working array. This will be reused by the tasks during each loop.</span>
<span class="kt">var</span> <span class="n">isPrime</span> <span class="p">=</span> <span class="k">new</span> <span class="kt">bool</span><span class="p">[</span><span class="n">arraySize</span><span class="p">];</span>
<span class="nf">InitArray</span><span class="p">(</span><span class="n">isPrime</span><span class="p">,</span> <span class="m">0</span><span class="p">);</span>
<span class="c1">// Allocate enough space to store 'sqrt' primes</span>
<span class="kt">var</span> <span class="n">primes</span> <span class="p">=</span> <span class="k">new</span> <span class="n">List</span><span class="p"><</span><span class="kt">ulong</span><span class="p">>((</span><span class="kt">int</span><span class="p">)</span><span class="n">sqrt</span><span class="p">);</span>
<span class="c1">// First serially determine all the primes up to and including 'sqrt'</span>
<span class="nf">Sieve</span><span class="p">(</span><span class="n">isPrime</span><span class="p">,</span> <span class="n">sqrt</span> <span class="p">+</span> <span class="m">1</span><span class="p">,</span> <span class="n">p</span> <span class="p">=></span> <span class="n">primes</span><span class="p">.</span><span class="nf">Add</span><span class="p">(</span><span class="n">p</span><span class="p">));</span>
<span class="c1">// We will now proceed in parallel blocks of size 'blockSize'. Each block will be for a subset</span>
<span class="c1">// of the numbers up to endNumber.</span>
<span class="c1">// We have to process all the numbers up to endNumber. How many blocks is this? Don't forget</span>
<span class="c1">// remainders.</span>
<span class="kt">var</span> <span class="n">numElements</span> <span class="p">=</span> <span class="n">endNumber</span> <span class="p">/</span> <span class="m">2</span><span class="p">;</span>
<span class="kt">var</span> <span class="n">numBlocks</span> <span class="p">=</span> <span class="p">(</span><span class="kt">uint</span><span class="p">)(</span><span class="n">numElements</span> <span class="p">%</span> <span class="n">blockSize</span> <span class="p">==</span> <span class="m">0</span> <span class="p">?</span>
<span class="n">numElements</span> <span class="p">/</span> <span class="n">blockSize</span> <span class="p">:</span>
<span class="p">(</span><span class="n">numElements</span> <span class="p">/</span> <span class="n">blockSize</span><span class="p">)</span> <span class="p">+</span> <span class="m">1</span><span class="p">);</span>
<span class="c1">// We will process the blocks numTasks at a time. How many loops will we require? Don't</span>
<span class="c1">// forget remainders.</span>
<span class="kt">var</span> <span class="n">numLoops</span> <span class="p">=</span> <span class="n">numBlocks</span> <span class="p">%</span> <span class="n">numTasks</span> <span class="p">==</span> <span class="m">0</span> <span class="p">?</span> <span class="n">numBlocks</span> <span class="p">/</span> <span class="n">numTasks</span> <span class="p">:</span> <span class="p">(</span><span class="n">numBlocks</span> <span class="p">/</span> <span class="n">numTasks</span><span class="p">)</span> <span class="p">+</span> <span class="m">1</span><span class="p">;</span>
<span class="c1">// Main loop</span>
<span class="k">for</span><span class="p">(</span><span class="kt">uint</span> <span class="n">n</span> <span class="p">=</span> <span class="m">0</span><span class="p">;</span> <span class="n">n</span> <span class="p"><</span> <span class="n">numLoops</span><span class="p">;</span> <span class="p">++</span><span class="n">n</span><span class="p">)</span> <span class="p">{</span>
<span class="nf">InitArray</span><span class="p">(</span><span class="n">isPrime</span><span class="p">,</span> <span class="n">n</span><span class="p">);</span> <span class="c1">// Re-initialize the array for reuse</span>
<span class="c1">// Init the bounds of the numbers that this loop will be working with</span>
<span class="kt">var</span> <span class="n">startNum</span> <span class="p">=</span> <span class="p">(</span><span class="kt">ulong</span><span class="p">)</span><span class="n">n</span> <span class="p">*</span> <span class="m">2</span> <span class="p">*</span> <span class="n">blockSize</span> <span class="p">*</span> <span class="n">numTasks</span><span class="p">;</span>
<span class="kt">var</span> <span class="n">endNum</span> <span class="p">=</span> <span class="n">startNum</span> <span class="p">+</span> <span class="p">(</span><span class="m">2</span> <span class="p">*</span> <span class="n">blockSize</span> <span class="p">*</span> <span class="n">numTasks</span><span class="p">);</span>
<span class="k">if</span><span class="p">(</span><span class="n">endNum</span> <span class="p">></span> <span class="n">endNumber</span><span class="p">)</span> <span class="n">endNum</span> <span class="p">=</span> <span class="n">endNumber</span><span class="p">;</span>
<span class="c1">// Launch tasks to process each block of the working array in parallel and wait for them to</span>
<span class="c1">// complete</span>
<span class="nf">ParallelSieveBlocks</span><span class="p">(</span><span class="n">endNumber</span><span class="p">,</span> <span class="n">blockSize</span><span class="p">,</span> <span class="n">isPrime</span><span class="p">,</span> <span class="n">primes</span><span class="p">,</span> <span class="n">numTasks</span><span class="p">,</span> <span class="n">startNum</span><span class="p">);</span>
<span class="c1">// All tasks done, enumerate over the working array and output primes</span>
<span class="k">if</span><span class="p">(</span><span class="n">n</span> <span class="p">==</span> <span class="m">0</span><span class="p">)</span> <span class="nf">primeFn</span><span class="p">(</span><span class="m">2</span><span class="p">);</span> <span class="c1">// 2 is a special case</span>
<span class="kt">var</span> <span class="n">totalBlockSize</span> <span class="p">=</span> <span class="n">numTasks</span> <span class="p">*</span> <span class="n">blockSize</span><span class="p">;</span>
<span class="k">for</span><span class="p">(</span><span class="kt">uint</span> <span class="n">m</span> <span class="p">=</span> <span class="m">0</span><span class="p">;</span> <span class="n">m</span> <span class="p"><</span> <span class="n">totalBlockSize</span><span class="p">;</span> <span class="p">++</span><span class="n">m</span><span class="p">)</span> <span class="p">{</span>
<span class="k">if</span><span class="p">(!</span><span class="n">isPrime</span><span class="p">[</span><span class="n">m</span><span class="p">])</span> <span class="k">continue</span><span class="p">;</span>
<span class="kt">var</span> <span class="n">prime</span> <span class="p">=</span> <span class="n">startNum</span> <span class="p">+</span> <span class="p">(</span><span class="m">2</span> <span class="p">*</span> <span class="n">m</span><span class="p">)</span> <span class="p">+</span> <span class="m">1</span><span class="p">;</span>
<span class="k">if</span><span class="p">(</span><span class="nf">primeFn</span><span class="p">(</span><span class="n">prime</span><span class="p">))</span> <span class="k">return</span><span class="p">;</span> <span class="c1">// Return if the client code indicates they are done</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<h2 id="method-5">Method 5</h2>
<p>Use a parallel block processing Sieve of Eratosthenes with a block size of 500,000 and various upper bounds, count the
resulting primes and output the millionth, 100 millionth, 500 millionth and the billionth ones.</p>
<p>Results …</p>
<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>c:\src\eratosthenes\ParallelBlockSieve.exe -blockSize=500000 -maxNumber=15500000 -nthPrime=1000000
The 1,000,000th prime is 15,485,863
Elapsed = 00:00:00.0558394
c:\src\eratosthenes\ParallelBlockSieve.exe -blockSize=500000 -maxNumber=2100000000 -nthPrime=100000000
The 100,000,000th prime is 2,038,074,743
Elapsed = 00:00:07.4914955
c:\src\eratosthenes\ParallelBlockSieve.exe -blockSize=500000 -maxNumber=11100000000 -nthPrime=500000000
The 500,000,000th prime is 11,037,271,757
Elapsed = 00:00:41:1140395
c:\src\eratosthenes\ParallelBlockSieve.exe -blockSize=500000 -maxNumber=23000000000 -nthPrime=1000000000
The 1,000,000,000th prime is 22,801,763,489
Elapsed = 00:01:23.4179141
</code></pre></div></div>
<p>Not bad.</p>
<p>I experimented with different block sizes but ultimately found that results were best when the block size was chosen to
ensure that each block of working array (in use by the threadpool thread running each task) was small enough that it
would stay in L2/3 cache for a core. For larger blocks we may be able to do more work in parallel, and thus ultimately
execute fewer loops, but we pay more in memory access latency than we gain in saved serial calculations.</p>
<h1 id="results-comparison">Results comparison</h1>
<p>Here’s a summary of the runtimes of our different methods (All times are minutes, seconds and milliseconds).</p>
<table class="center">
<thead>
<tr>
<th style="text-align: center">nth Prime</th>
<th style="text-align: center">Sequential IsPrime</th>
<th style="text-align: center">Sieve</th>
<th style="text-align: center">Block Sieve</th>
<th style="text-align: center">Parallel Block Sieve</th>
</tr>
</thead>
<tbody>
<tr>
<td style="text-align: center">1,000,000</td>
<td style="text-align: center">07:03.000</td>
<td style="text-align: center">00:00.130</td>
<td style="text-align: center">00:00.062</td>
<td style="text-align: center">00:00.056</td>
</tr>
<tr>
<td style="text-align: center">100,000,000</td>
<td style="text-align: center">n/a</td>
<td style="text-align: center">n/a</td>
<td style="text-align: center">00:09.687</td>
<td style="text-align: center">00:07.491</td>
</tr>
<tr>
<td style="text-align: center">500,000,000</td>
<td style="text-align: center">n/a</td>
<td style="text-align: center">n/a</td>
<td style="text-align: center">00:57.200</td>
<td style="text-align: center">00:41.114</td>
</tr>
<tr>
<td style="text-align: center">1,000,000,000</td>
<td style="text-align: center">n/a</td>
<td style="text-align: center">n/a</td>
<td style="text-align: center">02:05.949</td>
<td style="text-align: center">01:23.418</td>
</tr>
</tbody>
</table>
On Proof2018-02-12T00:00:00+00:00http://alandavies.org/blog/2018/02/12/on-proof<p>Proof. An interesting word with several meanings …</p>
<p>noun</p>
<ul>
<li>Evidence or argument establishing or helping to establish a fact or the truth of a statement</li>
<li>A trial print of something, in particular</li>
<li>A <a href="https://en.wikipedia.org/wiki/Alcohol_proof">measure</a> of the content of ethanol in an alcoholic beverage</li>
</ul>
<p>adjective</p>
<ul>
<li>Able to withstand something damaging; resistant</li>
<li>Denoting a trial impression of a page or printed work</li>
</ul>
<p>verb</p>
<ul>
<li>Make (fabric) waterproof</li>
<li>Make a proof of (a printed work, engraving, etc.)</li>
</ul>
<p>Although I very much like the third noun usage, today we are going to be discussing proof in the context of mathematics
and in particular, the various forms in which it can be achieved.</p>
<h1 id="the-mathematical-edifice">The Mathematical Edifice</h1>
<p>I like to think of mathematics as a structure and a tall and impressive one at that, which only ever gets taller over
time. Each level of the structure builds on the layers below and at the very bottom is a foundation upon which
everything depends. These foundations are known as <a href="https://en.wikipedia.org/wiki/Axiom">axioms</a> things which are
assumed to be true and require no proof of themselves. Above the axioms are theorems, statements which are shown to be
true based on the logical application of the axioms and theorems that lie below them. The truth of each level of the
structure depends on the truth of the layers below. In any lower level theorem is proven to be false then the truth of
all the theorems above is thrown into doubt.</p>
<p>Geometry is a classic example of an edifice built on a set of axioms where the type of structure that you end up with
depends critically on which axioms are assumed and not assumed, the so called
<a href="https://en.wikipedia.org/wiki/Parallel_postulate">parallel postulate</a> being a ready example.
<a href="https://en.wikipedia.org/wiki/Euclidean_geometry">Eucliden Geometry</a> is the study of geometry that satisfies all of
<a href="https://en.wikipedia.org/wiki/Euclidean_geometry#Axioms">Euclid’s axioms</a>, including the parallel postulate. A
geometry where the parallel postulate does not hold is termed
<a href="https://en.wikipedia.org/wiki/Non-Euclidean_geometry">non-Euclidean</a> and there are several types thereof (hyperbolic,
elliptic, etc.)</p>
<p>The process of building the edifice depends wholy on the nature of mathematical proof.</p>
<h1 id="mathematical-proof">Mathematical Proof</h1>
<p>Formally, in mathematics, a proof is an inferential argument for a mathematical statement. In the argument, other
previously established statements, such as theorems, can be used. In principle, a proof can be traced back to
self-evident or assumed statements, known as axioms, along with accepted rules of inference. Axioms may be treated as
conditions that must be met before the statement applies. Proofs are examples of exhaustive deductive reasoning or
inductive reasoning and are distinguished from empirical arguments or non-exhaustive inductive reasoning (or “reasonable
expectation”). A proof must demonstrate that a statement is always true (occasionally by listing all possible cases and
showing that it holds in each), rather than enumerate many confirmatory cases. An unproved proposition that is believed
to be true is known as a conjecture. Proofs employ logic but inevitably include some amount of natural language which
can admit some ambiguity. In fact, the vast majority of proofs in written mathematics can be considered as applications
of rigorous informal logic.</p>
<p>Pause. Breathe.</p>
<p>Informally, proofs are things that are required in order to pass examinations in mathematics but which many students
consider not that important in the grand scheme of things and sometimes even a
<a href="http://hitchhikers.wikia.com/wiki/Babel_Fish">load of dingo’s kidneys</a>.</p>
<p>The presentation of a mathematical proof generally follows a format. The proposition to be proved is first stated along
with any related assumptions about the elements involved in the proposition. What follows is a logical progression of
statements, the truth of each of which is inferred from the statements that came before it coupled with other things
(definitions, axioms and theorems) that we already know to be true. This progression of true statements proceeds until
the original proposition is stated, at which point the proof is complete. Finally, and traditionally, the term QED
appears which is an acronym for the Latin phrase quod erat demonstrandum meaning “what was to be demonstrated” or “what
was to be shown”. An alternative, although slightly inaccurate, translation is “thus it has been demonstrated”.</p>
<h1 id="types-of-proof">Types of proof</h1>
<p>Mathematical proofs fall into various categories. Each of these can be considered an approach to a proof, or a tool
that can be employed in pursuit of a proof. Many theorems can be prooved in different ways using different techniques.
For example, the classic <a href="https://en.wikipedia.org/wiki/Pythagorean_theorem">theorm of Pythagoras</a> is said to have at
least 370 known distinct proofs.</p>
<h2 id="proof-by-deduction">Proof by Deduction</h2>
<p>A proof by deduction is a technique in which the succession of true statements is achieved via
<a href="https://en.wikipedia.org/wiki/Deductive_reasoning">deductive reasoning</a>. We proceed from statement A to statement B
via an implication inferred from the assumed truth of A conjoined with other known truths (definitions, axioms and
theorems).</p>
<p>The direction of the implication of truth between two statements must be carefully considered. For two related
statements A and B we can have.</p>
<p>A if B</p>
<ul>
<li>“B is true” implies that “A is true” (B => A)</li>
<li>“A is false” implies that “B is false” (!A => !B)</li>
<li>This is the same as B only if A</li>
</ul>
<p>A only if B</p>
<ul>
<li>“B is false” implies that “A is false” (!B => !A)</li>
<li>“A is true” implies that “B is true” (A => B)</li>
<li>This is the same as B if A</li>
</ul>
<p>A if and only if B</p>
<ul>
<li>“A is true” implies that “B is true” (A => B)</li>
<li>“A is false” implies that “B is false” (!A => !B)</li>
<li>“B is true” implies that “A is true” (B => A)</li>
<li>“B is false” implies that “A is false” (!B => !A)</li>
<li>
<p>The two statements are logically equivalent (A <=> B)</p>
</li>
<li>
<p>Example 1:*</p>
</li>
<li>Statement A: x > 10</li>
<li>Statement B: x > 5</li>
</ul>
<p>B if A (equivalently A => B) but not A if B. B does not imply A. E.g. x = 6 satisfies B but not A.</p>
<p><em>Example 2:</em></p>
<ul>
<li>Statement A: n is an even number</li>
<li>Statement B: n = 2k for some integer value k</li>
</ul>
<p>A if and only if B (equivalently A <=> B)</p>
<h2 id="proof-by-induction">Proof by induction</h2>
<p>A proof by induction is just like an ordinary proof in which every step must be justified. However, it employs a neat
trick which allows you to prove a proposition about an arbitrary positive integer n by first proving it is true when n =
1 and then assuming it is true for n = k and showing (via other deductive reasoning) that it is also true for n = k + 1.
By doing this we can then say that since the proposition is true for n = 1 then it must also be true for n = 2 (since
true for k implies true for k + 1); and since it’s true for n = 2 then it must also be true for n = 3 and so on
and so forth ad infinitum. Thus the proposition must be true for all positive integer values of n.</p>
<h2 id="proof-by-contradiction">Proof by contradiction</h2>
<p>Proof by contradiction starts out by assuming that the opposite of the proposition is true, and then shows that such an
assumption leads to a contradiction. Thus the original assumption (that the proposition is false) cannot be true and so
we can deduce that the proposition must be true. This style of proof is a particular kind of the more general form of
argument known as <a href="https://en.wikipedia.org/wiki/Reductio_ad_absurdum">reductio ad absurdum</a>.</p>
<p><a href="https://en.wikipedia.org/wiki/G._H._Hardy">G. H. Hardy</a> described proof by contradiction as “One of a mathematician’s
finest weapons”, saying “It is a far finer gambit than any chess gambit: a chess player may offer the sacrifice of a
pawn or even a piece, but a mathematician offers the game.”</p>
<h2 id="other">Other</h2>
<p>Other styles of proof exist, such as proof by construction and proof by exhaustion but the main tools in a
mathematician’s toolbox are those outlined above. Another style, although not formal, is the visual proof or
<a href="https://en.wikipedia.org/wiki/Proof_without_words">proof without words</a>. A great example of this would be the
following visual representation of Pythagoras’ Theorem.</p>
<p><img src="/images/pythagoras_visual_proof.png" alt="pythagoras" /></p>
<h1 id="example-proofs">Example Proofs</h1>
<p>Let’s consider some interesting mathematical propositions and then use the different types of proof to turn those
propositions into theorems.</p>
<h2 id="1089">1089</h2>
<p>I’m not sure if this one counts as a theorem but it’s a fun mathematical trick that we can demonstrate to be true via
deductive reasoning based on the initial conditions.</p>
<p><em>Proposition …</em></p>
<p>Pick any three digit positive decimal integer where the first digit is greater than the last. Reverse the digits to
form another positive three digit integer and subtract this new integer from the first. The result will be another
three digit positive integer. Reverse the digits of this new number and then add the resulting number to the result
from the first operation. The answer will be 1089 and will always be 1089 regardless of the initial choice of number.</p>
<p><em>Examples …</em></p>
<ul>
<li><script type="math/tex">N = 321</script>; <script type="math/tex">N_{rev} = 123</script>; <script type="math/tex">N - N_{rev} = M = 198</script>; <script type="math/tex">M_{rev} = 891</script>; <script type="math/tex">M + M_{rev} = 1089</script></li>
<li><script type="math/tex">N = 780</script>; <script type="math/tex">N_{rev} = 087</script>; <script type="math/tex">N - N_{rev} = M = 693</script>; <script type="math/tex">M_{rev} = 396</script>; <script type="math/tex">M + M_{rev} = 1089</script></li>
<li><script type="math/tex">N = 500</script>; <script type="math/tex">N_{rev} = 005</script>; <script type="math/tex">N - N_{rev} = M = 495</script>; <script type="math/tex">M_{rev} = 594</script>; <script type="math/tex">M + M_{rev} = 1089</script></li>
</ul>
<p><em>Proof (by deduction) …</em></p>
<p>Let’s assume that the digits of our initial number, <script type="math/tex">N</script>, are <script type="math/tex">a</script>, <script type="math/tex">b</script> & <script type="math/tex">c</script>.</p>
<p>We know: <script type="math/tex">a \in [1, 9]</script>, <script type="math/tex">b \in [0, 9]</script>, <script type="math/tex">c \in [0, 9]</script>, <script type="math/tex">a > c</script> and <script type="math/tex">N = 100a + 10b + c</script>.</p>
<p>We know that <script type="math/tex">a</script> cannot be zero because <script type="math/tex">a > c</script>.</p>
<p>Now, <script type="math/tex">N_{rev} = 100c + 10b + a</script> and therefore <script type="math/tex">N - N_{rev} = M = (100a + 10b + c) - (100c + 10b + a)</script>.</p>
<p>Simplifying, we have <script type="math/tex">M = 100a - a + 10b - 10b - 100c + c = 99a - 99c = 99(a - c)</script>.</p>
<p>We know from our initial assumptions that <script type="math/tex">a - c > 0</script> and <script type="math/tex">a - c \in [1, 9]</script> so <script type="math/tex">M</script> is a positive multiple
of <script type="math/tex">99</script> and further, <script type="math/tex">M \in \{ 099, 198, 297, 396, 495, 594, 693, 792, 891, 990 \}</script>.</p>
<p>For each of these possible values of <script type="math/tex">M</script>, the middle digit is <script type="math/tex">9</script> and the first and last digits sum to give
<script type="math/tex">9</script>. If we write the digits of M as <script type="math/tex">a</script>, <script type="math/tex">9</script> & <script type="math/tex">c</script> then …</p>
<script type="math/tex; mode=display">M + M_{rev} = (100a + 90 + c) + (100c + 90 + a) = 100(a + c) + 180 + a + c</script>
<p>But we know that <script type="math/tex">a + c = 9</script> and therefore <script type="math/tex">M + M_{rev} = 900 + 180 + 9 = 1089</script>.</p>
<p>QED</p>
<h2 id="a-formula-for-the-sum-of-the-first-n-positive-integers">A Formula for the Sum of the First N Positive Integers</h2>
<p><em>Proposition …</em></p>
<p><script type="math/tex">1 + 2 + 3 + 4 + ... + n = \frac{n(n + 1)}{2}</script> for all positive integers <script type="math/tex">n</script>.</p>
<p><em>Proof (by induction) …</em></p>
<p>Assume that the proposition is true for <script type="math/tex">n = N</script>. So we can take this as true …</p>
<script type="math/tex; mode=display">1 + 2 + 3 + 4 + ... + N = \frac{N(N + 1)}{2}</script>
<p>Add <script type="math/tex">N + 1</script> to both sides to give …</p>
<script type="math/tex; mode=display">1 + 2 + 3 + 4 + ... + N + (N + 1) = \frac{N(N + 1)}{2} + (N + 1)</script>
<p>We can rearange the right hand side to give …</p>
<script type="math/tex; mode=display">\frac{N(N + 1)}{2} + (N + 1) = \frac{N(N + 1) + 2(N + 1)}{2} = \frac{N^2 + 3N + 2}{2} = \frac{(N + 1)(N + 2)}{2}</script>
<p>and so …</p>
<script type="math/tex; mode=display">1 + 2 + 3 + 4 + ... + N + (N + 1) = \frac{(N + 1)(N + 2)}{2}</script>
<p>which is the same as the original proposition but with <script type="math/tex">n = N + 1</script>.</p>
<p>So the proposition being true for <script type="math/tex">n = N</script> logically implies it is also true for <script type="math/tex">n = N + 1</script>.</p>
<p>It is trivially true for <script type="math/tex">n = 1</script> since <script type="math/tex">1 = \frac{1(2)}{2}</script> is true.</p>
<p>Therefore it must also be true for <script type="math/tex">n = 2, 3, 4, ...</script>.</p>
<p>Therefore it is true for all positive integers.</p>
<p>QED</p>
<h2 id="the-infinitude-of-the-primes">The Infinitude of the Primes</h2>
<p>Definition: A prime number is a positive integer that is only evenly divisible by itself and one.</p>
<p><em>Proposition …</em></p>
<p>There are infinitely many prime numbers.</p>
<p><em>Proof (by construction/induction) …</em></p>
<p>Let’s take any finite set of prime numbers <script type="math/tex">\{p_1, p_2, p_3, ..., p_n\}</script> and then let’s form the number
<script type="math/tex">N = (p_1 \times p_2 \times p_3 \times ... \times p_n) + 1</script>.</p>
<p><script type="math/tex">N</script> must have a prime factor but that factor can’t be any of <script type="math/tex">\{p_1, p_2, p_3, ..., p_n\}</script> because, by
construction, there will always be a remainder of <script type="math/tex">1</script> when <script type="math/tex">N</script> is divided by any of the primes in our set.</p>
<p>Therefore there must be another prime that is not in our set.</p>
<p>But we chose our initial finite set of primes arbitrarily. Thus for any finite set of primes there must be another that
is not in the set.</p>
<p>Therefore there are infinitely many prime numbers.</p>
<p>QED</p>
<h2 id="the-irrationality-of-the-square-root-of-two">The Irrationality of the Square Root of Two</h2>
<p>Definition: A rational number is one that can be expressed as the quotient <script type="math/tex">\frac{p}{q}</script> of two integers, a
numerator <script type="math/tex">p</script> and a non-zero denominator <script type="math/tex">q</script>. An irrational number is a real number that is not a rational
number.</p>
<p><em>Proposition …</em></p>
<p>The square root of two is an irrational number.</p>
<p><em>Proof (by contradiction) …</em></p>
<p>Assumption: <script type="math/tex">\sqrt{2}</script> is rational</p>
<p>Thus <script type="math/tex">\sqrt{2}</script> can be expressed as <script type="math/tex">\frac{p}{q}</script> where <script type="math/tex">p</script> and <script type="math/tex">q</script> are integers with no common
factors (other than <script type="math/tex">1</script>) and where <script type="math/tex">q</script> is non-zero.</p>
<p><script type="math/tex">\sqrt{2} = \frac{p}{q}</script> therefore <script type="math/tex">2 = \frac{p^2}{q^2}</script> and so <script type="math/tex">2q^2 = p^2</script></p>
<p>Thus <script type="math/tex">p^2</script> must be an even number since it is an integer multiple of <script type="math/tex">2</script>.</p>
<p>Now, if <script type="math/tex">p</script> was odd, i.e. <script type="math/tex">p = 2k + 1</script> for some integer <script type="math/tex">k</script> then <script type="math/tex">p^2 = (2k + 1)^2 = 4k^2 + 4k + 1</script>
which is also odd. If <script type="math/tex">p</script> was even, i.e. <script type="math/tex">p = 2k</script> for some integer <script type="math/tex">k</script> then <script type="math/tex">p^2 = (2k)^2 = 4k^2</script> which
is also even. So <script type="math/tex">p^2</script> odd <=> <script type="math/tex">p</script> odd and <script type="math/tex">p^2</script> even <=> <script type="math/tex">p</script> even.</p>
<p>So, <script type="math/tex">p</script> is even, i.e. <script type="math/tex">p = 2k</script> for some integer k.</p>
<p>Substituting this back into <script type="math/tex">2q^2 = p^2</script> gives <script type="math/tex">2q^2 = (2k)^2 = 4k^2</script> and thus <script type="math/tex">q^2 = 2k^2</script> which means that
<script type="math/tex">q^2</script> is even as well (since it’s a multiple of <script type="math/tex">2</script>) and so, by the same argument as above, <script type="math/tex">q</script> must be even
too.</p>
<p>So, <script type="math/tex">p</script> and <script type="math/tex">q</script> are both even.</p>
<p>But this contradicts our initial assumption that <script type="math/tex">p</script> and <script type="math/tex">q</script> have no common factors other than <script type="math/tex">1</script>. Thus
our initial assumption must be false and so <script type="math/tex">\sqrt{2}</script> can’t be rational. Thus it is irrational.</p>
<p>QED</p>
On Mathematics and its Teaching2017-12-01T00:00:00+00:00http://alandavies.org/blog/2017/12/01/on-mathematics<p>I love mathematics. It’s logical, universal and beautiful. I loved mathematics as a child and was naturally good at it
in school. In fact it was one of the things that defined me as a kid, I was good at math and science. I wasn’t into
sports or any other significant activities back then. Yes, I was one of those kids. Mathematics just came naturally to
me and I wanted to learn more and more. I wanted to know everything.</p>
<p>I grew up in a nondescript family in a nondescript town and went to (I realize now, looking back) nondescript, mediocre
schools. I wasn’t driven to achieve; there was no expectation that I should aspire to attend an elite university that
would act as a stepping stone for me to then move on to a “career”. I just went to school and learnt what I was
presented with.</p>
<p>I did have the benefit of having some great, fun math teachers though; in middle school (what, in the UK, we called high
school - Redmoor High School in <a href="https://www.urbandictionary.com/define.php?term=shit-hole">Hinckley</a> to be precise)
and then in high school (sixth form college in the UK, specifically the John Cleveland College, also
located in the same Leicestershire idyll).</p>
<p>Now I described these teachers as “great” and at the time that’s what I thought they were. Looking back now though,
with the benefit of hindsight, I can say that they didn’t do that great a job of preparing me to study mathematics at a
higher level (more on this below). However, they were encouraging and they made the subject fun, and that is an
essential aspect of mathematical pedagogy; and something that was noticeably absent from my experience of
university-level teaching.</p>
<p>I was streamed ahead in middle school and then again in high school; and for the last two years there I was in a tiny
class (with only five kids in total if I recall correctly) where everyone was motivated to learn and so the teachers
could just focus on teaching without having to do general crowd control as well. There was also a competitive dynamic
between three of us in the class that really drove us on. I recall it fondly.</p>
<p>One day my teachers suggested that my fellow math geeks and I should apply to Oxbridge (a
<a href="https://en.wikipedia.org/wiki/Portmanteau">portmanteau</a> used to collectively refer to the two ancient elite
universities in the UK, Oxford and Cambridge). This was an idea that would never have occurred to me or my family
ourselves. As I said before, I was not raised to aspire to such things. The high school didn’t have a grand tradition
of sending students to Oxbridge either but they clearly had an eye for it and we were considered worthy. From that
point (towards the end of my penultimate year of school) our math class shifted to focus on the relevant entrance exams
and preparing us for interview. We were successful too. Ultimately I and two of my fellow classmates were accepted,
two of us to Oxford and one to Cambridge. And lo, I got to
<a href="https://academia.stackexchange.com/questions/9524/when-referring-to-study-at-oxford-why-is-the-word-read-used">read</a>
mathematics at <a href="https://www.sjc.ox.ac.uk/">St. John’s College</a>, Oxford. And boy was it different.</p>
<p>A few things changed when I got to Oxford. First, I went from being one of the brightest kids in school (without really
having to try) to being middle of the pack at best. This forced me to reevaluate my sense of who I was and what defined
me. Also, many of my fellow students studying mathematics, as well as the lecturers and tutors, were - how can I say
this - really nerdy and not cool. Now God knows I was never cool myself in high school but I aspired to be, and I think
I had the core ability to be social and funny, I just had to grow into it and that process really started once I was at
university. I remember having to spend time together with some of my fellow mathematicians and tutors (one in
particular) and feeling really awkward in their company. The tutor hardly spoke. I absolutely felt that these were
“not my people” and thus began the process where I started to drift away from mathematics. I couldn’t love the subject
anymore because to do maths at this level was to be like these people.</p>
<p>I gravitated more to some of the students who were studying engineering, physics and chemistry. They were still smart
and geeky (geeky is NOT the same as nerdy) but more fun to be around. I also very much enjoyed the company of those who
were studying a whole variety of completely different things (history, english, geography, psychology, philosophy, law,
…) as I was exposed to these people on a daily basis. More on this below.</p>
<p>Now one of the great things about being at Oxford is the fact that you are a member of a college as well as a member of
the university. There are 38 colleges that collectively make up the University of Oxford with each college being an
independent institution with its own history. My college
(<a href="https://en.wikipedia.org/wiki/St_John%27s_College,_Oxford">St. John’s</a>) was founded in 1555 and housed about 400
undergraduate students, 250 postgraduate students and 100 academic staff. The students and staff are drawn from all
academic disciplines and so life in college is a wonderful mix of personalities, ideas and intellectual points of view.
The college bar, when evaluated purely as a bar, was
<a href="http://cherwell.org/2015/03/07/bar-review-st-johns/">bloody aweful</a> but it was the place where this wonderful mix of
people hung out and where you could find people discussing and debating all aspects of intellectual pursuit as well as a
variety of trivia and stupidity (which was just as awesome). It also served the absolute best cheese and ham toasties,
and <a href="/images/baked_potato_with_cheese_and_beans.jpg">baked potatoes with cheese and beans</a>. Then there was the
<a href="https://en.wikipedia.org/wiki/Common_Room_(university)">Junior Common Room</a> (or JCR) which would hold regular meetings
and debates, well lubricated by booze provided by the JCR Pratt (an official position on the JCR Committee who’s
responsibilities included being a pratt and getting the beer in for JCR meetings). I recall one debate in particular
where great energy was expended in resolving the matter of whether Mr Nathan Byers (the JCR Pratt at the time) should
change his name (by <a href="https://en.wikipedia.org/wiki/Deed_poll">deed poll</a>) to Nathan-Madonna Byers in honor of the
“Queen of Pop”. We supported the motion and he did actually go ahead and change his name. What a stand-up
<a href="https://us.linkedin.com/in/nabyers">fellow</a>.</p>
<p>Anyway, back to the main point of this post. Just being at St. John’s was much more interesting than studying
mathematics. Not to say that I didn’t want to study and learn, I did. Which brings me to my next point.</p>
<p>Undergraduate teaching at Oxford is done in two parts. I’ve already described the college system and the fact that when
you are at Oxford you are a member of a college as well as a member of the university. The university as a whole still
exists of course and there are university departments for each subject area. All tutors (professors) are a member of a
college but also a member of a university department. The departments organize and teach the curriculum for a subject
area and set exams. It’s your performance in these final exams that ultimately determine your
<a href="https://en.wikipedia.org/wiki/British_undergraduate_degree_classification">class of degree</a>.
Teaching consits of group lectures at the department level augmented by small group
<a href="https://www.greenes.org.uk/our-history/the-history-of-the-tutorial/">tutorials</a> (in my case it was two students to one
tutor) with your local college professors within college. This all sounds great and it would be if all of the lecturers
and tutors were good teachers. Unfortunately, in my experience and especially in Mathematics, often they were not.</p>
<p>An aspect of the teaching of mathematics at the undergraduate level is that it is presented in a fundamentally different
way than in high school. A degree of rigor is introduced that represents a step change in the way things are taught.
There is much less of a focus on the student developing an intuitive understanding of the concepts and instead things
are presented in terms of a series of “definition, theorem, proof” cycles. This was something that I had not seen
before and was quite hard for me to come to terms with. This is what I meant when I said that my high school teachers
didn’t prepare me to study mathematics at a higher level. I learnt from several of my fellow students at Oxford that
they had been introduced to some degree of rigor, and this style of presentation, in school and it really helped them.
I guess that’s the difference between a state school in a nondescript provincial town and a fee-paying private school.
Sigh.</p>
<p>I stuggled to grasp the new presentation of the subject. Often the rigor seemed pointless (“But that’s just obvious!”)
and othertimes it obfuscated and hid the concept that was being introduced.</p>
<p>Given time it became more familiar and looking back now I absolutely understand and appreciate the focus on rigor and
the logical construction of things. At the time though I was frustrated and didn’t feel like I could ask for help in
getting up the learning curve since my tutors felt unapproachable. It was yet another factor that tainted my orignal
love of the subject.</p>
<p>I ultimately graduated from Oxford and earned degree in mathematics with a good enough grade (a 2:1). The competitive
high school kid I used to be would never have been satisfied with anything less than a first but I wasn’t that kid
anymore. I just wanted to get on with the next phase of my life and that I did.</p>
<p>Some years later I read a <a href="https://www.amazon.com/dp/014014739X">book</a> on mathematics written for the layman and I once
again saw something in it that was beautiful. I read more, and especially read a lot more about the history of
mathematical discovery and the iconic figures who developed much of what we take for granted today. The historical
context was fascinating and the presentation, geared to the non-academic, emphasised the concepts and the consequences
first and foremost. I ultimately came to the conclusion that you can’t study math in isolation, it’s too sterile and
without meaning. By taking a leisurely tour through its history and landscape I rediscovered my love for it. I’ve
since been motivated to try to re-learn much of what I was first exposed to back at university and also I now feel a
personal mission to help others to learn math the “right way”, so that they can see it and
<a href="https://en.wikipedia.org/wiki/Grok">grok</a> it while they also learn it properly.</p>
<p>I commit to writing posts about math as I continue my journey of rediscovery through its gentle rolling hills.</p>
On Concurrent UPSERTS2017-11-30T00:00:00+00:00http://alandavies.org/blog/2017/11/30/on-concurrent-upserts<h1 id="introduction">Introduction</h1>
<p>It’s a very common usecase to have to either <em>INSERT</em> a new row into a table or <em>UPDATE</em> an existing row depending on
whether the row already exists. This logic is commonly referred to as an UPSERT. Let’s see how we can handle this in
T-SQL.</p>
<p>We assume that we have this table …</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">CREATE</span> <span class="k">TABLE</span> <span class="n">dbo</span><span class="p">.</span><span class="n">Foo</span> <span class="p">(</span>
<span class="n">fooId</span> <span class="n">int</span> <span class="k">NOT</span> <span class="k">NULL</span><span class="p">,</span>
<span class="n">stuff</span> <span class="n">varchar</span><span class="p">(</span><span class="mi">256</span><span class="p">)</span> <span class="k">NOT</span> <span class="k">NULL</span>
<span class="k">CONSTRAINT</span> <span class="n">Foo__PK</span> <span class="n">CLUSTERED</span> <span class="p">(</span><span class="n">fooId</span><span class="p">)</span>
<span class="p">);</span>
</code></pre></div></div>
<p>Some naive T-SQL code to handle the UPSERT might be …</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">IF</span> <span class="k">EXISTS</span> <span class="p">(</span> <span class="k">SELECT</span> <span class="o">*</span> <span class="k">FROM</span> <span class="n">dbo</span><span class="p">.</span><span class="n">Foo</span> <span class="k">WHERE</span> <span class="n">fooId</span> <span class="o">=</span> <span class="o">@</span><span class="n">fooId</span> <span class="p">)</span>
<span class="k">UPDATE</span> <span class="n">dbo</span><span class="p">.</span><span class="n">Foo</span>
<span class="k">SET</span> <span class="n">stuff</span> <span class="o">=</span> <span class="o">@</span><span class="n">stuff</span>
<span class="k">WHERE</span> <span class="n">fooId</span> <span class="o">=</span> <span class="o">@</span><span class="n">fooId</span><span class="p">;</span>
<span class="k">ELSE</span>
<span class="k">INSERT</span> <span class="n">dbo</span><span class="p">.</span><span class="n">Foo</span> <span class="p">(</span> <span class="n">fooId</span><span class="p">,</span> <span class="n">stuff</span> <span class="p">)</span>
<span class="k">VALUES</span> <span class="p">(</span> <span class="o">@</span><span class="n">fooId</span><span class="p">,</span> <span class="o">@</span><span class="n">stuff</span> <span class="p">);</span>
</code></pre></div></div>
<p>This will work fine for a single connection but with multiple connections and high concurrency this will start to fail
frequently with primary key violations.</p>
<p>There are various ways to handle this badly. If you are interested in these then I encourage you to read several of the
articles listed in the further reading section below.
<a href="http://michaeljswart.com/2017/07/sql-server-upsert-patterns-and-antipatterns/">This one</a> in particular summarizes
things very well. However, if all you care about is how to do it well then I offer the following solution (from
<a href="http://weblogs.sqlteam.com/dang/archive/2009/01/31/UPSERT-Race-Condition-With-MERGE.aspx">here</a>).</p>
<h1 id="solution">Solution</h1>
<p>I recommend using a MERGE statement With HOLDLOCK.</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">MERGE</span> <span class="n">dbo</span><span class="p">.</span><span class="n">Foo</span> <span class="k">WITH</span> <span class="p">(</span><span class="n">HOLDLOCK</span><span class="p">)</span> <span class="k">AS</span> <span class="n">target</span>
<span class="k">USING</span> <span class="p">(</span><span class="k">SELECT</span> <span class="o">@</span><span class="n">fooId</span> <span class="k">AS</span> <span class="n">fooId</span><span class="p">,</span> <span class="o">@</span><span class="n">stuff</span> <span class="k">AS</span> <span class="n">stuff</span><span class="p">)</span> <span class="k">AS</span> <span class="k">source</span>
<span class="k">ON</span> <span class="k">source</span><span class="p">.</span><span class="n">fooId</span> <span class="o">=</span> <span class="n">target</span><span class="p">.</span><span class="n">fooId</span>
<span class="k">WHEN</span> <span class="n">MATCHED</span> <span class="k">THEN</span> <span class="k">UPDATE</span>
<span class="k">SET</span> <span class="n">target</span><span class="p">.</span><span class="n">stuff</span> <span class="o">=</span> <span class="k">source</span><span class="p">.</span><span class="n">stuff</span>
<span class="k">WHEN</span> <span class="k">NOT</span> <span class="n">MATCHED</span> <span class="k">THEN</span>
<span class="k">INSERT</span> <span class="p">(</span> <span class="n">fooId</span><span class="p">,</span> <span class="n">stuff</span> <span class="p">)</span>
<span class="k">VALUES</span> <span class="p">(</span> <span class="o">@</span><span class="n">fooId</span><span class="p">,</span> <span class="o">@</span><span class="n">stuff</span> <span class="p">);</span>
</code></pre></div></div>
<p>No transaction is required here since <em>MERGE</em> is an atomic statement. <em>MERGE</em> takes out a key update lock by default so
we don’t need to use an <em>UPDLOCK</em> hint (as is the case with some other possible solutions). We do need a <em>HOLDLOCK</em>
hint though in order to ensure that SQL Server doesn’t release the key update lock before the <em>INSERT</em>.</p>
<h1 id="references-and-further-reading">References and Further Reading</h1>
<ul>
<li><a href="http://michaeljswart.com/2017/07/sql-server-upsert-patterns-and-antipatterns/">http://michaeljswart.com/2017/07/sql-server-upsert-patterns-and-antipatterns/</a></li>
<li><a href="http://michaeljswart.com/2011/09/mythbusting-concurrent-updateinsert-solutions/">http://michaeljswart.com/2011/09/mythbusting-concurrent-updateinsert-solutions/</a></li>
<li><a href="https://samsaffron.com/blog/archive/2007/04/04/14.aspx">https://samsaffron.com/blog/archive/2007/04/04/14.aspx</a></li>
<li><a href="http://weblogs.sqlteam.com/dang/archive/2009/01/31/UPSERT-Race-Condition-With-MERGE.aspx">http://weblogs.sqlteam.com/dang/archive/2009/01/31/UPSERT-Race-Condition-With-MERGE.aspx</a></li>
<li><a href="http://weblogs.sqlteam.com/dang/archive/2007/10/28/Conditional-INSERTUPDATE-Race-Condition.aspx">http://weblogs.sqlteam.com/dang/archive/2007/10/28/Conditional-INSERTUPDATE-Race-Condition.aspx</a></li>
</ul>
On Things I Hate2017-11-29T00:00:00+00:00http://alandavies.org/blog/2017/11/29/on-things-i-hate<p>“Not that I condone vandalism, or any -ism for that matter. -Isms in my opinion are not good. A person should not
believe in an -ism, he should believe in himself. I quote John Lennon, ‘I don’t believe in Beatles, I just believe in
me.’ Good point there. After all, he was the walrus. I could be the walrus. I’d still have to bum rides off people.”</p>
<p><a href="https://www.youtube.com/watch?v=0KFVLWX7eEY">Anonymous</a></p>
On Floating Point 2.02017-11-29T00:00:00+00:00http://alandavies.org/blog/2017/11/29/on-floating-point-2<p>I recently got hold of a copy of a very interesting <a href="https://www.amazon.com/dp/1539692876/">book</a> all about the
development of the early PC game Wolfenstein 3D. Rather than being a history of ID Sofware, and the team that went on
to develop the seminal games Doom and Quake (for which see <a href="https://www.amazon.com/dp/0812972155">here</a>), it’s more of a
review of the Wolfenstein 3D codebase (which is open source and available on
<a href="https://github.com/id-Software/wolf3d">GitHub</a>) and a presentation of the challenges that were overcome in developing
a groundbreaking 3D game for the PC with typical specs as of the time:
<a href="https://en.wikipedia.org/wiki/Intel_80386">i386</a>, <a href="https://en.wikipedia.org/wiki/Video_Graphics_Array">VGA</a> graphics,
<a href="https://en.wikipedia.org/wiki/Sound_Blaster">Soundblaster</a> sound card, running
<a href="https://en.wikipedia.org/wiki/MS-DOS">MS-DOS</a> with perhaps 2MB RAM.</p>
<p>A PC of that era had significantly more CPU horsepower (in terms of
<a href="https://en.wikipedia.org/wiki/Instructions_per_second">MIPS</a>) than any of the contemporary games consoles and that was
what made it appealing as a target platform for a 3D game. The nature of the segmented memory model that DOS imposed
(as a consequence of being tied to <a href="https://en.wikipedia.org/wiki/Real_mode">Real Mode</a> on the 386), the architecture of
the VGA graphics card and the lack of hardware floating point on the 386 presented lots of challenges to the project,
and the strategies that Carmack and co. came up with to make it all work are ingenius and very interesting. It’s a good
read … for those of a more geeky persuasion anyway. People like me, and - I suppose - you too dear reader.</p>
<p>All this is very interesting but not really the subject of this blog post. This post is a follow up to a
<a href="/blog/2014/09/24/on-floating-point">previous one</a> I wrote about the floating point
representation of real numbers and was inspired by a description of floating point from this book that was unlike any
that I have seen before. It represented a different way to think about that particular numeric format and, I think, is
a very helpful way to try to visualize (and understand) how it was designed. I also realized that I’d left out
descriptions of the “special” floating point values in my previous article and I needed to remedy that. So here we go.
<a href="https://www.youtube.com/watch?v=J1MVUes6bVw&t=20">Onward</a>!</p>
<h1 id="the-traditional-view">The Traditional View</h1>
<p>The traditional description of floating point representation involves the presentation of the following pattern of the
32 bits in a regular float in memory …</p>
<p><img src="/images/ieee_754_float_example.png" alt="sfsdf" class="center-image" /></p>
<p>… and the following expression involving a sum of powers of 2 …</p>
<script type="math/tex; mode=display">value = (-1)^{s} \times (1 + m) \times 2^e</script>
<p>where \( s = b_{31} \), \( e = e_{biased} - 127 \), \( e_{biased} = \sum_{i=23}^{30}{b_i2^{i-23}} \) and
\(m = \sum_{i=22}^{0}{b_i}2^{i-23}\).</p>
<p>It then proceeds to inform you that one bit of memory is used to indicate the sign of the number (the value \(s\) in
the expression, where 0 indicates positive and 1 indicates negative), eight are used to represent the biased exponent
(the value \(e_{biased}\)) and 23 are used to store the digits of what is known as the mantissa where there is an
assumed leading 1 bit and the stored bits are assumed to be the part to the right of the binary point (the equivalent of
the decimal point in the decimal representation of a real number). These are bits \(b_i\) for \(i\) in
\([22, 0]\).</p>
<h1 id="the-alternative-view">The Alternative View</h1>
<p>An alternative spin on things is to observe that the floating-point representation of a number is an approximation to a
given real number \(R\) as follows …</p>
<ul>
<li>First we note whether \(R\) is positive or negative and save that in a value (which we will call \(s\)) where
positive will be stored as \(0\) and negative will be stored as \(1\)</li>
<li>Then we determine which two consecutive powers of \(2\) bound \(|R|\) (the absolute value of \(R\)) above and
below. Let’s denote these two values as \(2^e\) and \(2^{e+1}\).</li>
<li>Next we divide the difference between \(2^e\) and \(2^{e+1}\) into \(N\) equal parts and find the number \(n\)
such that \(2^e + \frac{n}{N} <= |R| < 2^e + \frac{n + 1}{N}\)</li>
<li>The floating point approximation to \(R\) is parameterized by the values \(s\), \(e\), \(N\) and \(n\)</li>
</ul>
<p>So we basically map out a comb of \(N\) discrete values between each two consecutive powers of \(2\) (within a range
of such powers, \(2^{e_{min}}\) and \(2^{e_{max}}\)) and approximate \(|R|\) as the closest tooth of the comb
equal to or less than \(|R|\).</p>
<p>I hope you can see that the accuracy of this approximation will depend on \(N\). The larger the value, the finer
grain the divisions between the bounding powers of two will be and the closer we can get to the actual value of
\(|R|\). I also hope you can see that the range of possible values we can approximate (from smallest to largest) will
depend on \(e_{min}\) and \(e_{max}\).</p>
<h1 id="examples">Examples</h1>
<p>Let’s look at some examples to illustrate things. We’ll assume that \(N\) is 8 for now and look at how we would
represent the real numbers \(0.05\), \(6\) and \(50\).</p>
<p>First, \(0.05\). This number falls between \(2^{-5} = \frac{1}{32} = 0.03125\) and
\(2^{-4} = \frac{1}{16} = 0.0625\) and the gap between the teeth of our comb will be
\(\frac{2^{-5}}{8} = \frac{1}{256} = 0.00390625\).</p>
<p><img src="/images/best_representation_of_point_0_5.png" alt="" /></p>
<p>We can see that we can’t represent \(0.05\) perfectly and end up using \(\frac{3}{64} = 0.046875\) as an
approximation.</p>
<p>Next, 6. This number falls between \(2^2 = 4\) and \(2^3 = 8\) and the gap between the teeth of our comb will be
\(\frac{2^2}{8} = \frac{1}{2} = 0.5\).</p>
<p><img src="/images/best_representation_of_6.png" alt="" /></p>
<p>In this case we can represent \(6\) perfectly and so we do.</p>
<p>Finally, 50. This number falls between \(2^5 = 32\) and \(2^6 = 64\) and the gap between the teeth of our comb
will be \(\frac{2^5}{8} = 4\).</p>
<p><img src="/images/best_representation_of_50.png" alt="" /></p>
<p>Again, we can’t represent \(50\) perfectly. The best we can do, within the bounds of the parameters we have chosen,
is to use \(48\) as an approximation.</p>
<h1 id="mapping-between-the-two-views">Mapping Between the Two Views</h1>
<p>How does this view map to the traditional view of floating point and to the bit representation that we saw before?</p>
<p>Well, the sign flag \(s\) should be obvious; it maps to the sign bit.</p>
<p>The value \(N\) from the new view is equal to the maximum value that can be stored in the mantissa bits of the bit
representation. In a regular 32 bit float there are 23 bits to store the mantissa. The maximum (unsigned) integer
value that you can store in 23 bits is \(2^{23} = 8{,}388{,}608 \) and so, in a 32 bit float, the range between
consecutive powers of two is quantized into \(8{,}388{,}608\) parts.</p>
<p>Finally we need to think about the range of possible values for the first power in the pair of consecutive powers of
\(2\). We referred to this before as \([2^{e_{min}}\), \(2^{e_{max}}]\). In a 32 bit float, 8 bits are used to
store a biased exponent which can take any value from the range \([0, 255]\). This is a biased exponent and the
actual exponent (in the sum interpretation of float) is equal to \(\texttt{stored_exponent} - 127\). Thus the actual
exponent can take any value in the range \([-127, 128]\). Therefore our pair of consecutive powers of \(2\) can be
anything from \([2^{-127}, 2^{-126}]\) to \([2^{128}, 2^{129}]\).</p>
<p>Remember the traditional view …</p>
<p><img src="/images/ieee_754_float_example.png" alt="sfsdf" class="center-image" /></p>
<script type="math/tex; mode=display">value = (-1)^{s} \times (1 + m) \times 2^e</script>
<p>where
<script type="math/tex">e = e_{biased} - 127</script>,
<script type="math/tex">e_{biased} = \sum_{i=23}^{30}{b_i2^{i-23}}</script> and
<script type="math/tex">m = \sum_{i=22}^{0}{b_i}2^{i-23}</script></p>
<p>Here’s the alternate view …</p>
<script type="math/tex; mode=display">value = (-1)^{s} \times (2^e + \frac{n}{N})</script>
<p>where
<script type="math/tex">n = \sum_{i=0}^{22}{b_i2^i}</script> and
<script type="math/tex">N = 2^{23}</script></p>
<h1 id="some-observations">Some Observations</h1>
<p>The choice of \(N\) is critical and represents the granularity of the quantization that we impose on the ranges
between consecutive powers of \(2\). The quantum is equal to \(\frac{1}{N}\).</p>
<p>The mantissa \(m\) represents how many quanta we add to the first power of \(2\).</p>
<p>The choice of storage size for \(e\) is critical and dictates the extremities of the range of values that we can
represent.</p>
<p>We always use the same granularity of quantization (the same number of bins) regardless of the power of \(2\) at the
start of the range. This means that the absolute size of each bin (our quantum of value) grows the larger the number
that we are representing. It also means that the relative size of each bin, relative to the starting value of the range
remains the same for all ranges though. We can see this in the following image …</p>
<p><img src="/images/floating_point_range_quantization.png" alt="" /></p>
<p>Here we are breaking each range into \(16\) bins. Each range is progressively wider but the number of bins remains the
same. Each bin between \(1\) and \(2\) is wider than each bin between \(\frac{1}{8}\) and \(\frac{1}{4}\).</p>
<h1 id="special-values">Special Values</h1>
<p>Floating point uses a few special values. These are \(0\), \(\texttt{Infinity}\), \(\texttt{NaN}\) (short for Not
a Number) and Denormalized Numbers; and there are actually two kinds of \(\texttt{NaN}\) value: Signalling and Quiet.
More on the difference between Signalling and Quiet in a moment. For now let’s look at the representation of these
values.</p>
<p>\(0\) is indicated by \(e_{biased} = 0\) and \(m = 0\). This leaves two possibilities for the sign bit and so
there are actually two values \(+0\) and \(-0\).</p>
<p>\(\texttt{Infinity}\) is indicated by \(e_{biased} = 255\) and \(m = 0\). Again the sign can be \(0\) or
\(1\) and so there are actually two values \(+\texttt{Infinity}\) and \(-\texttt{Infinity}\).</p>
<p>There are lots of \(\texttt{NaN}\) values. Any value with \(e_{biased} = 255\) and \(m != 0\) is considered
\(\texttt{NaN}\). Signalling \(\texttt{NaN}\) values have the most significant bit of \(m\) (i.e. \(b_{22}\))
set to \(0\) and Quiet \(\texttt{NaN}\) values have it set to \(1\).</p>
<p>Signalling vs. Quiet …</p>
<p>Various floating point operations can generate an invalid result (i.e. dividing by zero) and such operations will return
a value of \(\texttt{NaN}\). Whether the operation results in a Signalling \(\texttt{NaN}\) or a Quiet
\(\texttt{NaN}\) gets into the arcana of floating point unit (FPU) implementation and is beyond my current knowledge.
I can say though that if an operation results in a Quiet \(\texttt{NaN}\) then there is no indication that anything is
unusual until the program checks the result and sees the \(\texttt{NaN}\). That is, computation continues without any
signal from the FPU (or library if floating-point is implemented in software). However a Signalling \(\texttt{NaN}\)
will immediately produce a signal, usually in the form of exception from the FPU. Whether the exception is thrown
depends on the state of the FPU.</p>
<p>Denormalized Numbers …</p>
<p>These are represented by \(e_{biased} = 0\). This state indicates a slightly different interpretation of the rest of
the bits in the value. <a href="https://en.wikipedia.org/wiki/Denormal_number">Denormalized numbers</a> are a special case used to
smooth out the progression of represented numbers into and through \(0\). When \(e_{biased} = 0\) we use this
formula for the value …</p>
<script type="math/tex; mode=display">value = (-1)^{s} \times (m) \times 2^{e + 1}</script>
<p>… as opposed to the regular …</p>
<script type="math/tex; mode=display">value = (-1)^{s} \times (1 + m) \times 2^e</script>
<p>Or with our alternate view …</p>
<script type="math/tex; mode=display">value = (-1)^{s} \times (\frac{n}{N})</script>
<p>… as opposed to …</p>
<script type="math/tex; mode=display">value = (-1)^{s} \times (2^e + \frac{n}{N})</script>
<h1 id="one-final-picture-or-two-or-three-">One Final Picture or two, or three …</h1>
<p>To close let’s imagine a hypothetical one byte floating point format with 1 bit for the sign, 3 bits for the biased
exponent and 4 bits for the mantissa. In this format, \(e_{biased}\) can range from \(0\) to \(7\) and \(m\)
can range from \(0\) to \(15\). We can show all the possible values in a grid, as the rational values …</p>
<p><img src="/images/floating_point_grid_rational.png" alt="" /></p>
<p>… and as the corresponding decimal values …</p>
<p><img src="/images/floating_point_grid_decimal.png" alt="" /></p>
<p>The values above include the denormalized numbers. To see the smoothing effect that these have let’s look at a picture
showing the values of our representation near \(0\) with and without denormalization. First without …</p>
<p><img src="/images/floating_point_chart_no_denorm.png" alt="" /></p>
<p>… and now with …</p>
<p><img src="/images/floating_point_chart.png" alt="" /></p>
<p>Much <a href="https://www.youtube.com/watch?v=5MvVot5RKHI&t=44">smoother</a>.</p>
<h1 id="other-resources">Other Resources</h1>
<p>Lots of people have written about floating point online. I have no illusions that my witterings are especially
insightful or worthy. I have cribbed much of my knowledge from other places on the web actually. Here are some links
to such resources. Go read more on the subject.</p>
<ul>
<li><a href="https://randomascii.wordpress.com/category/floating-point/">https://randomascii.wordpress.com/category/floating-point/</a></li>
<li><a href="http://floating-point-gui.de/errors/comparison/">http://floating-point-gui.de/errors/comparison/</a></li>
<li><a href="http://realtimecollisiondetection.net/blog/?p=89">http://realtimecollisiondetection.net/blog/?p=89</a></li>
</ul>
On Tech Conferences2017-11-26T00:00:00+00:00http://alandavies.org/blog/2017/11/26/on-tech-conferences<p>For some time now I’ve wanted to attend a big tech conference. Part of the motivation was to attend a large event but
the main part was to be exposed to a concentrated collection of information and learning opportunities. I finally
managed to remember to bring a proposal to my employer and I got approval to attend the 2017
<a href="http://www.pass.org/AttendanEvent/Conferences/PASSSummit.aspx">PASS Summit</a> in Seattle in November.</p>
<p>PASS is the Professional Association for SQL Server. It’s a community organization for users of the Microsoft Data
Platform (SQL Server and related technologies) that “facilitates member networking and the exchange of information
through local groups, online events, local/regional events and international conferences”, or so their blurb says.
They hold an international summit each year, typically in Seattle, which is sponsored by Microsoft and other vendors of
tools and technologies related to the Microsoft Data Platform. This was the event that I was going to attend.</p>
<p>The main conference was held over three days (Wed-Fri) but there were two days of pre-conference events to be held on
the Monday and Tuesday as well. On these days one could attend a full day focused seminar on various topics. Since I
was going to be flying all the way from New York to Seattle for this thing I thought that I may as well make the most of
it and booked myself into the pre-conference sessions as well.</p>
<p>PASS has historically been a conference for <a href="https://en.wikipedia.org/wiki/Database_administrator">DBAs</a>,
<a href="https://en.wikipedia.org/wiki/Data_architect">Data Architects</a> and
<a href="https://en.wikipedia.org/wiki/Business_intelligence">BI</a> (<a href="https://twitter.com/managerspeak">an oxymoron?</a>)
professionals. More recently though they have been trying to include developers in their ranks and the tracks on offer
at the summit (and pre-conference) reflected that. I chose to attend a full day on “Modern Web API Design” on Monday
and another full day on “Entity Framework” (the Microsoft full-fat
<a href="https://en.wikipedia.org/wiki/Object-relational_mapping">ORM framework</a>) on the Tuesday. The Tuesday session was
ultimately cancelled due to the fact that the presenter’s inbound flight was cancelled due to bad weather on the East
Coast and I ended up attending a full day workshop on “Expert Performance Tuning” instead.</p>
<p>These pre-conference sessions were interesting and well attended but come the first day of the main conference (on
Wednesday) I realized just how many people were actually going to attend this thing; literally thousands. The event was
held at the Washington State Conference Center in downtown Seattle, a huge facility that we took over for the week.
Catering for this many people was a military exercise and the food was about as good as you would expect. I suffered
the breakfast buffet for a few days before I finally gave it up and just grabbed some good coffee (this was Seattle
after all) and pastries in the hotel each morning.</p>
<p>One other colleague from work was attending the conference with me but we were exploring different tracks and although
we did reconnect regularly our paths didn’t cross too much. Every session, meal and coffee break was an opportunity to
meet other SQL Server professionals and users and meet them I did, many of whom were deeply knowledgeable. It was a
truly international event with a big European contingent and I met people from the UK, Denmark, Sweden, France & Germany
as well as all across the US. As expected, the attendees were predominently DBAs but I did meet a fair share of fellow
developers as well.</p>
<p>I must say though that sterotypes aside the PASS <a href="https://www.urbandictionary.com/define.php?term=massive">Massive</a> was
an eclectic mix of people with <a href="/images/tech_conference_restrooms.png">impressive diversity</a>, overwhelmingly of
<a href="/images/buff_geek.png">rude health</a> and <a href="/images/brent_ozar_mullet_wig.jpg">sartorial good taste</a>. The
<a href="/images/dba_beard.jpg">beards</a> though … what can I say? Hirsuite is not a sufficiently descriptive word.</p>
<p>During the main conference there was a “keynote” presentation each morning followed by a series of roughly one hour long
presentations running in sixteen parallel tracks. At any time you had a choice of sixteen things to see; a lot of
content to choose from. The presentations were all delivered by members of the PASS community and on the whole were not
marketing pieces for related products and tools, which can be the case at some conferences. Most of the sessions were
interesting and useful to me and only a few were disappointing. Clearly I only managed to experience a sixteenth of the
material that was on offer but I should be able to access slides and recordings of all of the presentations in due
course. It was definitely worth my time to attend, although I’m glad that I wasn’t the one paying for the flights,
hotel and conference fee …</p>
<p>Of course, Microsoft are the largest sponsor of this event and I would say that it is the biggest single conference for
SQL Server that there is, although there are bigger conferences for Microsoft technologies in general, and for Microsoft
platform developers in particular (Ignite, Build, etc.). As such there were some significant new releases for the
Microsoft data platform that were showcased during the week, SQL Server 2017 in particular. In addition to several new
features, the 2017 release of SQL Server brings full(ish) support for Linux as a platform and there were some
fascinating presentations on the details of how that port was achieved. Psst: It wasn’t a port. More on that in
another blog post.</p>
<p>Here are some of my takeaways from the summit experience.</p>
<h1 id="notable-quotes">Notable Quotes</h1>
<p>“If you want the data model to be simple then go out and make the world simple, then come back to me”</p>
<p>“SQL Server does key-value, it’s just a table with two columns”</p>
<h1 id="notable-presentations">Notable presentations</h1>
<p>Hacking SQL Server</p>
<ul>
<li>Of course you should protect against SQL Injection attacks</li>
<li>Encrypt your connections and use certificates to authenticate servers in order to prevent man in the middle attacks
and TDS inspection/injection</li>
<li>Divide by zero exceptions can facilitate some clever brute force attacks against encrypted data. If you know the
types of the fields in a table (and if I recall correctly the system tables are not encrypted so you can query for
them) then you can create successive queries of the form <code class="highlighter-rouge">SELECT 1 / (testValue - fieldName) FROM tableName</code> that
explore the space of possible values of each field and wait until you get a divide by zero exception. Once you do
then you know that the field value is equal to <code class="highlighter-rouge">testValue</code>.</li>
</ul>
<p>Data Architecture and NoSQL</p>
<ul>
<li>Data models still matter in the NoSQL world</li>
<li>There is no schemaless data</li>
<li>Data patterns have existed for a long time and are just as relevant today, it’s just that now we have more database
systems within which to express them, especially true graph databases</li>
</ul>
<p>Lightweight ORMs</p>
<ul>
<li>Entity Framework is not the only game in town when it comes to ORMs for C#. Dapper is simple (based on POCOs),
lightweight and fast. I will be giving it a try.</li>
</ul>
<p>SQL Server Service Broker</p>
<ul>
<li>It has a bad rep and is not “cool”</li>
<li>However it is very powerful and has a legitimate tool in many workflows</li>
<li>We should all keep it in our toolboxes</li>
</ul>
<p>SQL Server Client Tools</p>
<ul>
<li>There’s a new Electron-based (i.e. like VS Code) cross-platform client app for SQL Server called
<a href="https://docs.microsoft.com/en-us/sql/sql-operations-studio/what-is">SQL Server Operations Studio</a>. It’s not fully
featured but is featured enough, lightweight and fast. It’s a real alternative to SSMS for general querying and
database management.</li>
</ul>
<p>SQL Server on Linux</p>
<ul>
<li>Works and works well. It’s the same binaries that run on Windows.</li>
<li>When SQL Server runs on Linux it thinks it’s running on Windows</li>
<li>It installs simply and quickly. Far quicker than the install on Windows …</li>
<li>Performance is basically the same</li>
<li>You can run it in Docker and that makes for a very compelling DevOps story for dev/testing</li>
<li>I wouldn’t consider running it in prod right now (HADR is not so rubust yet) but the Docker use case is fascinating
and I will definitely be looking at that for dev/test</li>
</ul>
<h1 id="notable-unplanned-events">Notable Unplanned Events</h1>
<p>Bob Ward (Principal SQL Server Architect at Microsoft) was in the middle of his presentation on how they conceieved of,
and delivered, a version of SQL Server that runs on Linux when his laptop started installing automatic updates for
Windows which couldn’t be cancelled. It then proceeded to reboot (several times) while it proceeded with the update.
His presentation was completely hosed and morphed into a (still very interesting) Q&A session. I must say that he
handled the whole affair with some grace.</p>
<p>Later I found this Haiku which I profer in his honor …</p>
<p><img src="/images/windows_update_haiku.jpg" alt="Windows Update Haiku" /></p>
<h1 id="random-observations">Random Observations</h1>
<p>On the PASS Summit …</p>
<ul>
<li>I am not old relative to the DBA community but there are a bunch of super smart and really knowledgeable people out
there who are a lot younger than me. So … I still feel old.</li>
<li>Brent Ozar is a funny dude; and he is the <a href="/images/brent_ozar_stig.jpg">Stig</a>.</li>
<li><a href="http://www.sommarskog.se/">Erland Sommarskog</a> may write very informative articles but, in person, he’s pretty dull.
Sorry Erland.</li>
<li>All .Net/ C# references I saw were to .Net Core and all demos were using VS Code and .Net Core. It’s the future.</li>
<li>I didn’t win any of the vendor raffles. Life is so unfair.</li>
<li>Why does a conference of this size only choose to lay on coffee stations in the morning and at lunch?!? Don’t they
understand that tech workers need regular cafination in order to function?</li>
</ul>
<p>On Seattle …</p>
<ul>
<li>It has cool out of the way jazz restaurants</li>
<li>It has great dive bars that play great music</li>
<li>It has great coffee</li>
<li>And I thought London had a lot of <a href="/images/hippies_use_side_door.jpg">Hipsters</a></li>
<li>It has great <a href="/images/seattle_beer.jpg">craft beer</a></li>
<li>It’s very <a href="/images/gender_neutral_restrooms.jpg">progressive</a></li>
</ul>
On Mac vs PC2017-11-18T00:00:00+00:00http://alandavies.org/blog/2017/11/18/on-mac-vs-pc<p>I’ve been a PC person for years. At work I only ever used (and use) PCs, although I worked with remote VMS (yes,
<a href="https://en.wikipedia.org/wiki/OpenVMS">that VMS</a>) and Linux systems all the time. A few years back though, I bought a
MacBook Pro 13 in order to get access to Xcode and try my hand at IOS development. Thus was my hands on intro to the
world of Mac …</p>
<p>I have come to love that MacBook. It was a long and windy courtship full of frustration as I had to unlearn years of
muscle memory based on Windows norms. I still struggle with that today but I’m better. The one thing that has led me to
become a fully paid up member of the cult of Mac though is that trackpad.</p>
<p>The trackpad is sublime. It’s like glass, like silk. The mouse pointer glides around the screen and the two finger
scrolling is a pure joy. No PC laptop has come close to matching the feel of it and believe me I’ve looked and tried a
bunch. There’s something truly magical about the fact that Apple controls the whole package: hardware, drivers, OS,
apps. They can, and do, focus on the tiniest of details and I’ve decided that those details matter. I can’t stand to
use any other laptop now.</p>
<p>Desktop computers are different. With a good external mouse the experience of using Windows on a PC can be as good.
I have yet to find a Windows laptop with a trackpad to match though.</p>
<p>And so, now that I find myself wanting to get a new power laptop for home use (for dev but also for general stuff) I
have to go with a new MacBook Pro 15. I must say that the keyboard doesn’t feel great as yet. I hope I’ll get used to
it. Thinkpads are still the gold standard for laptop keyboards I’d say. But that trackpad is still like butter, and
it’s sooo much bigger now. And the new TouchBar is intriguing.</p>
<p>I’m done. I’m a fanboy. I don’t care that they are the most expensive laptops. They are the best laptops. By far.</p>
On Getting Back on the Horse2017-11-18T00:00:00+00:00http://alandavies.org/blog/2017/11/18/on-getting-back-on-the-horse<p>If you don’t keep doing something regularly then you will not do it regularly and soon that new habit will add up to
weeks, months and even years. I was so motivated to start this blog back in late 2014, and I kept it up for a while,
but I didn’t force myself to stick at it and soon it became an afterthought. Then boom … two years pass since my
last post. Life is a succession of habits. It’s hard to develop good habits and it’s so easy to fall into bad habits.</p>
<p>When you recognize a bad habit though - or the absence of a good habit that you should have - that’s the first step to
changing it. And so … let’s try to change. Let’s give this blog a new lease of life. I am recommitting to this
and will endeavor to write posts regularly starting today.</p>
<p>A good way to get started with something is to change your environment, and so with this site. I was hosting this on a
WordPress instance with Go Daddy but I was never happy with that experience and the site’s presentation. I don’t know
exactly how the WordPress instance was hosted but the site was slow to load on the first request and wasn’t especially
responsive thereafter. That suggests some kind of virtualization with minimal resources.</p>
<p>An old colleague of mine keeps a blog and it was a post there that introduced me to a tool called Jekyll which allows
you to statically build a blog site from a bunch of markdown files. I checked that out, liked what I saw and also
learnt that Jekyll is natively supported by GitHub via a facility they offer called GitHub Pages. You can maintain a
repo at GitHub that contains a Jekyll project and GitHub will automatically build and publish the site as you check in
changes to your repo. This works pretty well and is the new home of this little corner of the Internet.</p>
<p>As part of the move I’ve broken away from WordPress and its stock themes and I’ve taken it upon myself to own all the
HTML, CSS and site design myself. This is something that I have dabbled with before but never used in earnest. I’ve
learnt a lot as a result and those learnings will become a post at some point soon. The site design will evolve I’m
sure but the great thing about Jekyll is that the source for the site builds to static files that can then be served up
with minimal overhead from the web server environment. As such, the site is very snappy to load and use.</p>
<p>So here we are, back on the horse. Now I just need to learn how to ride this damn thing. I pull on the tail and it
goes right?</p>
On pythagorean cups2015-12-29T00:00:00+00:00http://alandavies.org/blog/2015/12/29/on-greedy-cups<h1 id="is-it-magic-or-is-it-science">Is it magic or is it science?</h1>
<p>Don’t be greedy now. <a href="https://www.youtube.com/watch?v=AnAPWR2OS0w">Fill your cup</a> but not too much.</p>
On LEGO guns2015-09-12T00:00:00+00:00http://alandavies.org/blog/2015/09/12/on-lego-guns<p>More <a href="https://www.youtube.com/watch?v=t23tq-5Vb8Y">fun</a> with LEGO<sup>®</sup>. This time we make a gun, and a
<a href="https://www.amazon.com/dp/1593272847">badass</a> one too.</p>
On How To Tie Your Shoes2015-07-08T00:00:00+00:00http://alandavies.org/blog/2015/07/08/on-how-to-tie-your-shoes<p>A <a href="https://www.youtube.com/watch?v=8zdZD0E9o24">video post</a> this time. How do you tie your shoes? Have you actually
even given this any thought since you learnt when you were a little kid? There’s always room to improve.</p>
On How Legos is Wrong2015-07-08T00:00:00+00:00http://alandavies.org/blog/2015/07/08/on-how-legos-is-wrong<p>For the last time, it’s LEGO<sup>®</sup> damnit! A LEGO<sup>®</sup> set generally contains many
LEGO<sup>®</sup> bricks although most people, as I know from personal experience, just end up with a big old box
of LEGO<sup>®</sup> that gets tipped out on to the floor for play. And then when you step on a
LEGO<sup>®</sup> brick in your bare feet it hurts like hell and you hop around for a while desperately trying not
to land on any more LEGO<sup>®</sup>.</p>
On Group Theory and LEGO2015-07-08T00:00:00+00:00http://alandavies.org/blog/2015/07/08/on-group-theory-and-lego<p>Another <a href="https://www.youtube.com/watch?v=OmMuUVCLtkg">video post</a>! This time involving a Rubik’s Cube and a bunch of
LEGO<sup>®</sup>. By the way, anyone who says
<a href="/blog/2015/07/08/on-how-legos-is-wrong">Legos</a> should be taken out back and shot. Only
Americans say Legos and they are wrong, about many things. Although they are right about some things too; but that,
dear reader, is a post for another day.</p>
On useless stuff2015-05-16T00:00:00+00:00http://alandavies.org/blog/2015/05/16/on-useless-stuff<h1 id="what-do-you-do-when-you-get-an-arduino-starter-kit-as-a-gift">What do you do when you get an Arduino starter kit as a gift?</h1>
<p>You make a <a href="https://www.youtube.com/watch?v=CcecDjoiCIE">useless box</a>! Yes, I know you can make one of these with just
a servo motor and two switches but you can also program an Arduino to do it. Why do it the simple way when you can add
complexity?! Enjoy!</p>
On Computers2015-03-20T00:00:00+00:00http://alandavies.org/blog/2015/03/20/on-computers<h1 id="my-first-computer">My First Computer</h1>
<p>At the risk of giving away my approximate age I would like to present my very first computer, the Sinclair ZX81; and
what a beast it was with its Z80A 8-bit 3.5 MHz CPU, 1kB of on board RAM, 8kB of ROM containing a BASIC interpreter,
monochrome UHF video output (supporting up to 64 x 48 pixels of graphics mode via 2x2 block characters) and the
ubiquitous (at the time) external audio tape data storage. Ah, those were the days. How could one resist such
<a href="/images/zx81_advert.jpg">awesomeness?</a></p>
<p>My Dad bought this for me around 1981/82 I think. This is where I first started to code, in BASIC, although my programs
did nothing more sophisticated than print text to the screen. You can’t do much with 1kB of RAM so, in short order, we
added a RAM expansion pack to bring us up to a whopping 16kB. You had to be careful not to jostle things though since
the edge connector was a bit delicate and too much movement could crash the machine. That was a bit hard given the
nature of the keyboard …</p>
<h1 id="a-step-up">A Step Up</h1>
<p>My next computer was a huge step up in power and sophistication: an Acorn BBC Micro model B. I’ll have you know that
these were the mutt’s nuts back in the day.</p>
<p><img src="/images/bbc_micro.jpg" alt="BBC Micro" /></p>
<p>This was an expensive device as I recall; and I chose to get it, at age 13 or so, as opposed to go on a foreign exchange
trip to the USA with school. And that, dear reader, tells you everything you need to know about the sort of child that
I was. It had a 2 MHz 6502 CPU, 32kB of RAM and multiple graphics modes including colour (yes, that U is supposed to be
there - this was a computer made at the behest of the BRITISH Broadcasting Corporation remember). I wrote more BASIC
code here, mostly just drawing colored geometrical graphics on the screen as I recall. I also remember my parents
buying me a magazine called <a href="http://en.wikipedia.org/wiki/Input_Magazine">INPUT</a> which I collected and studied
(somewhat) for a year. More significant though were the games, including such gems as Chuckie Egg and the seminal
Elite. I never did make it to Elite as I recall, just Deadly. Mostly via trading narcotics and slaves … a great
education for a young boy.</p>
<p>Chuckie Egg. Fiendishly addictive.</p>
<p><img src="/images/chuckie_egg.gif" alt="Chuckie Egg" /></p>
<p>Elite! 3D wire-frame graphics with hidden line removal. Groundbreaking at the time.</p>
<p><img src="/images/elite.jpg" alt="Elite" /></p>
<h1 id="school-stuff">School Stuff</h1>
<p>I remember that in the school library at Redmoor High School (this would be Junior High for the Americans out there) we
had a few computers. We had a BBC Micro of course (on which we would sneak in a few games of Elite over lunch) but we
also had one of these bad boys (a Research Machines 380Z) that I don’t recall doing much with to be honest …</p>
<p>RM 380Z. I used to observe this thing from afar, in awe of its raw computing power. It had floppy disks!</p>
<p><img src="/images/rm380z.jpg" alt="RM 380Z" /></p>
<p>And I think we had a Commodore PET too. I had a cat at home myself.</p>
<p><img src="/images/commodore_pet.jpg" alt="Commodore Pet" /></p>
<p>It was around this time that I started going to a computer club in the evenings at the school where my Mum used to work.
I didn’t go for long though as I recall. I wanted to learn to program in “machine code” but the chap who was running the
club didn’t really know much more than COBOL so I was a bit disappointed. I do remember playing with some Logo robots
though; and spending an hour typing in a <a href="http://en.wikipedia.org/wiki/Type-in_program">printed program listing</a> from a
magazine only to then hear a tinny computer speaker squeak out the theme from Close Encounters of the Third Kind. Oh
the wonder of it all.</p>
<h1 id="the-games-machine">The Games Machine</h1>
<p>Then I got one of these, an
<a href="http://www.theregister.co.uk/2014/02/12/archaeologic_amstrad_cpc_464/?page=1">Amstrad CPC 464</a>. This was primarily
used to play games on, including as I recall a poor knock off of Konami’s
<a href="/images/konami_track_and_field.jpg">Track and Field</a> arcade game. This was the era when the arcade was the place where
the “real” computer games lived and kids used to hang out for hours on end watching each other try to “beat the
machine”. Oh, them were the days.</p>
<p>Alan Michael Sugar Trading FTW! Back then noone knew that he’d be on the telly so much in the future.</p>
<p><img src="/images/amstrad_cpc464.jpg" alt="Amstrad CPC464" /></p>
<p>I had me one of these bad boys too. You could customize all the buttons and it had a rapid fire switch too. I recall
hacking said Track and Field game by setting the left leg run button to ‘x’ and the right leg run button to ‘X’ in the
game and then programming the joystick rapid-fire function to press ‘shift’. Ah, such youthful sport.</p>
<p><img src="/images/quickshot_joystick.jpg" alt="Quickshot!" /></p>
<p>Then there were the magazines. One publishing company seemed to have a lock on the cool kids with versions of basically
the same format of mag for each of the popular games computers of the time (the Sinclair Spectrum, Commodore 64 and
Amstrad). Here’s the Amstrad one.</p>
<p><img src="/images/amtix.jpg" alt="Amtix" /></p>
<p>It was basically just full of games reviews.</p>
<h1 id="games">Games</h1>
<p>Some games that I spent way too long playing back in the day …</p>
<p>The Hobbit - Where’s Thorin?</p>
<p><img src="/images/the_hobbit.jpg" alt="The Hobbit" /></p>
<p>Manic Miner - In the Halls of the Mountain King …</p>
<p><img src="/images/manic_miner.jpg" alt="Manic Miner" /></p>
<p>Way of the Exploding Fist - Don’t try to kick me in the head, I’ll just crouch and punch you in the balls if you do …</p>
<p><img src="/images/way_of_the_exploding_fist.jpg" alt="Way of the Exploding Fist" /></p>
<p>Knight Lore - Ultimate, Play the Game!</p>
<p><img src="/images/knight_lore.png" alt="Knight Lore" /></p>
<h1 id="time-out">Time Out</h1>
<p>After a few years I lost interest in computer games and became embroiled in the world of
<a href="/images/a_d_and_d_players_handbook.jpg">Dungeons and Dragons</a>. This was a great chapter in my childhood and led to me
making some very good friends. There was nothing quite like some old school role playing with your mates around your
parents’ dinner table. There’s another blog post that I’ll write one day.</p>
<h1 id="getting-back-on-the-horse">Getting Back on the Horse</h1>
<p>I didn’t really return to computers until my first job after university. I did use some of the machines in the college
computer lab to write up my CV for job applications during my final year but the main thing I recall from that is having
to learn how to use MS World on a Mac and it taking me ages. So much for the intuitiveness of Macs. I was thrust into
the world of MS DOS with Lotus 123 (which seemed so natural and powerful) and Word Perfect (which I hated, I used to
write letters in Lotus 123 … honestly). I got myself a Dell PC for home, because my neighbor worked for Dell and
could get me a deal. I think it was a 486 20MHz with a 40MB hard drive. It came with Windows 3.1 which seemed so cool
at the time.</p>
<p>After about a year of work I admitted that I hated my first job (we all make mistakes) and I realized that I needed to
do something more cerebral where my smarts would get me somewhere. I ended up getting a lucky break and finding a job
with a very young but very successful American company focused on delivering financial data and analytics to the
institutional investment community. Clearly computers were an integral part of that and so I started to learn more and
more.</p>
<p>I never studied Computer Science at university. Everything I now know I learnt on the job based on my own motivations.
That was over 20 years ago. It’s been a fun ride.</p>
<h1 id="some-final-perspective">Some Final Perspective</h1>
<p>This has been a nostalgic walk down memory lane but also a graphic reminder of the power of
<a href="http://en.wikipedia.org/wiki/Moore%27s_law">Moore’s Law</a> in that today (March 2015 as of this writing), for US$35, you
get get a <a href="http://en.wikipedia.org/wiki/Raspberry_Pi">fully integrated computer</a> the size of a credit card that is
orders of magnitude more powerful than any of these old things. That’s progress for you.</p>
<p>What sort of computers will we have, and take for granted, in another 20 years time? I guess we’ll see in due course.</p>
On Turning on RCS2015-03-19T00:00:00+00:00http://alandavies.org/blog/2015/03/19/on-turning-on-rcs<p>This is the second in a series of posts on Microsoft SQL Server. If you are the sort of person who doesn’t care about
context and the logical flow of information then please, feel free to read on. However, I do suggest that you start
your stroll through my mumblings on this subject at the
<a href="/blog/2015/03/19/on-rdbms">beginning</a>. It’s your choice though.</p>
<h1 id="read-committed-snapshot">Read Committed Snapshot</h1>
<p>Microsoft SQL Server offers a database-level setting called <em>READ_COMMITTED_SNAPSHOT</em> that controls whether data
snapshots are used for transactions that run under the Read Committed isolation level. For a primer on the whole
notion of transactions, isolation levels and the nature of this setting I direct your attention to the
<a href="/blog/2015/03/19/on-rdbms">first</a> in this series of posts. As mentioned in that previous
article, turning on this setting can, in some edge cases, lead to a change in the behavior of transactions running
under the <em>READ COMMITTED</em> isolation level. This is an edge case but it’s an instructive one to explore since it will
give us a greater understanding of the nuance of row-versioning and snapshotting in the process.</p>
<p>Before we proceed I want to emphasise that running a database with the <em>READ_COMMITTED_SNAPSHOT</em> setting on is not a bad
thing by any stretch of the imagination. In fact it’s a great feature to enable and will minimize contention in a
database application. There’s no risk of inconsistent behavior among running transactions when it is on, but there is a
risk of a behavior change from when it is on to when it is off, or vice versa. The risk is in having a live database
that has been running with the setting off for a while and then turning it on. Some use-cases will see a behavior
change when you do this. However if you are creating a new database then I would suggest that you enable it from the
start.</p>
<h2 id="an-example-of-a-behavior-change">An Example of a Behavior Change</h2>
<p>Let’s look at the behavior of a theoretical situation. Open a connection to an MS SQL instance (we will call this
connection #1) and run this initial query:</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">-- Query 1 ---------------------------------------------------------------------</span>
<span class="n">USE</span> <span class="n">master</span><span class="p">;</span>
<span class="k">GO</span>
<span class="n">IF</span> <span class="k">EXISTS</span> <span class="p">(</span><span class="k">SELECT</span> <span class="o">*</span> <span class="k">FROM</span> <span class="n">sys</span><span class="p">.</span><span class="n">databases</span> <span class="k">WHERE</span> <span class="n">name</span> <span class="o">=</span> <span class="s1">'MarblesTest'</span><span class="p">)</span>
<span class="k">BEGIN</span>
<span class="k">ALTER</span> <span class="k">DATABASE</span> <span class="n">MarblesTest</span> <span class="k">SET</span> <span class="n">SINGLE_USER</span> <span class="k">WITH</span> <span class="k">ROLLBACK</span> <span class="k">IMMEDIATE</span><span class="p">;</span>
<span class="k">DROP</span> <span class="k">DATABASE</span> <span class="n">MarblesTest</span><span class="p">;</span>
<span class="k">END</span>
<span class="k">GO</span>
<span class="k">CREATE</span> <span class="k">DATABASE</span> <span class="n">MarblesTest</span><span class="p">;</span>
<span class="c1">--ALTER DATABASE MarblesTest SET READ_COMMITTED_SNAPSHOT ON;</span>
<span class="k">GO</span>
<span class="n">USE</span> <span class="n">MarblesTest</span><span class="p">;</span>
<span class="k">GO</span>
<span class="k">CREATE</span> <span class="k">TABLE</span> <span class="n">dbo</span><span class="p">.</span><span class="n">Marbles</span> <span class="p">(</span>
<span class="n">id</span> <span class="n">INT</span> <span class="k">PRIMARY</span> <span class="k">KEY</span><span class="p">,</span>
<span class="n">color</span> <span class="n">CHAR</span><span class="p">(</span><span class="mi">5</span><span class="p">)</span>
<span class="p">);</span>
<span class="k">GO</span>
<span class="k">INSERT</span> <span class="k">INTO</span> <span class="n">dbo</span><span class="p">.</span><span class="n">Marbles</span> <span class="k">VALUES</span> <span class="p">(</span> <span class="mi">1</span><span class="p">,</span> <span class="s1">'Black'</span> <span class="p">),</span> <span class="p">(</span> <span class="mi">2</span><span class="p">,</span> <span class="s1">'White'</span> <span class="p">);</span>
<span class="k">GO</span>
</code></pre></div></div>
<p>Note that the ALTER DATABASE statement to turn on <em>READ_COMMITTED_SNAPSHOT</em> is commented out and so the database will be
created with that setting off (the default).</p>
<p>Now execute this query on the same connection:</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">-- Query 2 ---------------------------------------------------------------------</span>
<span class="n">USE</span> <span class="n">MarblesTest</span><span class="p">;</span>
<span class="k">GO</span>
<span class="k">DECLARE</span> <span class="o">@</span><span class="n">id</span> <span class="n">INT</span><span class="p">;</span>
<span class="k">BEGIN</span> <span class="n">TRAN</span>
<span class="k">SELECT</span> <span class="o">@</span><span class="n">id</span> <span class="o">=</span> <span class="k">MIN</span><span class="p">(</span><span class="n">id</span><span class="p">)</span>
<span class="k">FROM</span> <span class="n">dbo</span><span class="p">.</span><span class="n">Marbles</span>
<span class="k">WHERE</span> <span class="n">color</span> <span class="o">=</span> <span class="s1">'Black'</span><span class="p">;</span>
<span class="k">UPDATE</span> <span class="n">dbo</span><span class="p">.</span><span class="n">Marbles</span>
<span class="k">SET</span> <span class="n">color</span> <span class="o">=</span> <span class="s1">'White'</span>
<span class="k">WHERE</span> <span class="n">id</span> <span class="o">=</span> <span class="o">@</span><span class="n">id</span><span class="p">;</span>
</code></pre></div></div>
<p>The query will complete immediately. Now open up another connection to your MS SQL instance (we will call this
connection #2) and run this query:</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">-- Query 3 ---------------------------------------------------------------------</span>
<span class="n">USE</span> <span class="n">MarblesTest</span><span class="p">;</span>
<span class="k">GO</span>
<span class="k">DECLARE</span> <span class="o">@</span><span class="n">id</span> <span class="n">INT</span><span class="p">;</span>
<span class="k">BEGIN</span> <span class="n">TRAN</span>
<span class="k">SELECT</span> <span class="o">@</span><span class="n">id</span> <span class="o">=</span> <span class="k">MIN</span><span class="p">(</span><span class="n">id</span><span class="p">)</span>
<span class="k">FROM</span> <span class="n">dbo</span><span class="p">.</span><span class="n">Marbles</span>
<span class="k">WHERE</span> <span class="n">color</span> <span class="o">=</span> <span class="s1">'Black'</span><span class="p">;</span>
<span class="k">UPDATE</span> <span class="n">dbo</span><span class="p">.</span><span class="n">Marbles</span>
<span class="k">SET</span> <span class="n">color</span> <span class="o">=</span> <span class="s1">'Red'</span>
<span class="k">WHERE</span> <span class="n">id</span> <span class="o">=</span> <span class="o">@</span><span class="n">id</span><span class="p">;</span>
<span class="k">COMMIT</span>
<span class="k">GO</span>
</code></pre></div></div>
<p>This query will block and sit executing until you take some further action. Now go back to connection #1 and excute this
query:</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">-- Query 4 ---------------------------------------------------------------------</span>
<span class="k">COMMIT</span>
<span class="k">GO</span>
</code></pre></div></div>
<p>This query will complete immediately, and once it has completed the other query (running on connection #2) will complete
too.</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">-- Query 5 ---------------------------------------------------------------------</span>
<span class="k">SELECT</span> <span class="o">*</span> <span class="k">FROM</span> <span class="n">dbo</span><span class="p">.</span><span class="n">Marbles</span><span class="p">;</span>
</code></pre></div></div>
<p>Now, in either connection run this query:</p>
<p>You will see this result:</p>
<table class="sql-result-table">
<thead>
<tr>
<th>id</th>
<th>color</th>
</tr>
</thead>
<tbody>
<tr>
<td>1</td>
<td>White</td>
</tr>
<tr>
<td>2</td>
<td>White</td>
</tr>
</tbody>
</table>
<p>Now, go back to connection #1, uncomment the <em>ALTER DATABASE</em> … <em>SET READ_COMMITTED_SNAPSHOT</em> … line from within
Query 1 and run it again. This will drop and recreate the database, but this time with the setting on.</p>
<p>Now rerun the other queries on the different connections exactly as before. This time the final result will be:</p>
<table class="sql-result-table">
<thead>
<tr>
<th>id</th>
<th>color</th>
</tr>
</thead>
<tbody>
<tr>
<td>1</td>
<td>Red</td>
</tr>
<tr>
<td>2</td>
<td>White</td>
</tr>
</tbody>
</table>
<p>“Err, what!?” I hear you say. “How can that be?”</p>
<p>Let’s take a closer look at the queries from above, starting with query 2 (running on connection #1) …</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">-- Query 2 ---------------------------------------------------------------------</span>
<span class="n">USE</span> <span class="n">MarblesTest</span><span class="p">;</span>
<span class="k">GO</span>
<span class="k">DECLARE</span> <span class="o">@</span><span class="n">id</span> <span class="n">INT</span><span class="p">;</span>
<span class="k">BEGIN</span> <span class="n">TRAN</span>
<span class="k">SELECT</span> <span class="o">@</span><span class="n">id</span> <span class="o">=</span> <span class="k">MIN</span><span class="p">(</span><span class="n">id</span><span class="p">)</span>
<span class="k">FROM</span> <span class="n">dbo</span><span class="p">.</span><span class="n">Marbles</span>
<span class="k">WHERE</span> <span class="n">color</span> <span class="o">=</span> <span class="s1">'Black'</span><span class="p">;</span>
<span class="k">UPDATE</span> <span class="n">dbo</span><span class="p">.</span><span class="n">Marbles</span>
<span class="k">SET</span> <span class="n">color</span> <span class="o">=</span> <span class="s1">'White'</span>
<span class="k">WHERE</span> <span class="n">id</span> <span class="o">=</span> <span class="o">@</span><span class="n">id</span><span class="p">;</span>
</code></pre></div></div>
<p>First we’ll look at what happens when the <em>READ_COMMITTED_SNAPSHOT</em> setting is off. This query starts a transaction and
then proceeds to issue a <em>SELECT</em> statement to determine the minimum id across all of the rows in the Marbles table that
have a color of ‘Black’. Since the transaction is running with an isolation level of <em>READ COMMITTED</em> (the default) and
the <em>READ_COMMITTED_SNAPSHOT</em> setting is off, then this tries to, and does, take out a shared lock on all of the rows
that match the predicate, all one of them. The <em>SELECT</em> statement’s predicate selects just that one row, the row with
id = 1, and then calculates the minimum id across that one row, which is obviously 1; and so we set @id to 1. The
transaction then releases its shared lock as soon as the statement completes. Next it issues an <em>UPDATE</em> statement to
set the color of the row (with id = 1) to ‘White’. This tries to, and does, take out an exclusive lock on that row and
the <em>UPDATE</em> completes. The lock is not released yet however. It will be held until the transaction is committed, and
this query does not commit the transaction. That comes later.</p>
<p>With the <em>READ_COMMITTED_SNAPSHOT</em> setting on nothing materially different happens (at least in terms of why we see this
strange behavior). The first statement in the transaction (the <em>SELECT</em>) will not issue a shared lock in this case,
instead it will read from a snapshot of the transactionally consistent row data as of the start of the statement. The
second statement (the <em>UPDATE</em>) will still take out an exclusive lock as before and that lock will again be held until
the transaction commits, which will happen at some point later.</p>
<p>Now let’s take another look at query 3 (running on connection #2) …</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">-- Query 3 - Annotated ---------------------------------------------------------</span>
<span class="n">USE</span> <span class="n">MarblesTest</span><span class="p">;</span>
<span class="k">GO</span>
<span class="k">DECLARE</span> <span class="o">@</span><span class="n">id</span> <span class="n">INT</span><span class="p">;</span>
<span class="k">BEGIN</span> <span class="n">TRAN</span>
<span class="c1">-- With the READ_COMMITTED_SNAPSHOT setting off, this query will block here</span>
<span class="k">SELECT</span> <span class="o">@</span><span class="n">id</span> <span class="o">=</span> <span class="k">MIN</span><span class="p">(</span><span class="n">id</span><span class="p">)</span>
<span class="k">FROM</span> <span class="n">dbo</span><span class="p">.</span><span class="n">Marbles</span>
<span class="k">WHERE</span> <span class="n">color</span> <span class="o">=</span> <span class="s1">'Black'</span><span class="p">;</span>
<span class="c1">-- With the READ_COMMITTED_SNAPSHOT setting on, this query will block here</span>
<span class="k">UPDATE</span> <span class="n">dbo</span><span class="p">.</span><span class="n">Marbles</span>
<span class="k">SET</span> <span class="n">color</span> <span class="o">=</span> <span class="s1">'Red'</span>
<span class="k">WHERE</span> <span class="n">id</span> <span class="o">=</span> <span class="o">@</span><span class="n">id</span><span class="p">;</span>
<span class="k">COMMIT</span>
<span class="k">GO</span>
</code></pre></div></div>
<p>Again, we’ll first consider what happens when the <em>READ_COMMITTED_SNAPSHOT</em> setting is off. The query starts a
transaction and then proceeds to issue a <em>SELECT</em> statement to determine the minimum id across all of the rows in the
Marbles table that have a color of ‘Black’. Since the transaction is running with an isolation level of <em>READ
COMMITTED</em> (the default) and the <em>READ_COMMITTED_SNAPSHOT</em> setting is off then this tries to take out a shared lock on
all of the rows that match the predicate. It can’t take out all of those locks though since the other transaction
(running on connection #1) has an exclusive lock on one of the rows that this transaction wants a shared lock on. So,
this transaction (on connection #2) blocks here and the first statement (the <em>SELECT</em>) will not run yet. Once we
execute the commit statement back on connection #1 then that transaction releases its exclusive lock on the row it
updated and the transaction (running on connection #2) can now take the shared lock on that row and proceed with its
<em>SELECT</em> statement. Because this transaction is running as <em>READ COMMITTED</em> (meaning that it will see transactionally
consistent data as of the start of each statement) then it will read the updated data written by the other transaction
and thus will now see that both rows have a color of ‘White’. The minimum id value across the rows with a color of
‘Black’ is thus now NULL (there are no rows with a color of ‘Black’) and so @id is set to NULL. The subsequent <em>UPDATE</em>
statement has no effect since there are no rows that match the predicate id = NULL. The transaction is committed and
this query completes. The end result is that we have both rows with color = ‘White’.</p>
<p>With the <em>READ_COMMITTED_SNAPSHOT</em> setting on we see different behavior. The query starts a transaction and then
proceeds to issue a <em>SELECT</em> statement to determine the minimum id across all of the rows in the Marbles table that have
a color of ‘Black’. Since the transaction is running with an isolation level of <em>READ COMMITTED</em> (the default) and the
<em>READ_COMMITTED_SNAPSHOT</em> setting is on then this statement does not require a shared lock and instead reads from a
snapshot copy of the transactionally consistent data as of the start of the statement. This snapshot will contain the
row data as it was before the other transaction (on connection #1) started, i.e. rowId 1 with a color of ‘Black’ and
rowId 2 with a color of ‘White’. So the <em>SELECT</em> query’s predicate will select the one row with a color of ‘Black’
(rowId 1) and that will also be the minimum id of course. Thus @id will end up being set to 1. The subsequent <em>UPDATE</em>
statement will try to take out an exclusive lock on the row with rowId 1 but will be unable to get it because the other
transaction (on connection #1) is holding an exclusive lock on the same row. Once we execute the commit statement back
on connection #1 then that transaction releases its exclusive lock on the row and the transaction (running on
connection #2) can now take the exclusive lock and proceed with its <em>UPDATE</em>. Note that at this time the modification
to rowId 1 (color now set to ‘White’) is committed. Because the transaction (running on connection #2) is running as
<em>READ COMMITTED</em> then this statement will see a transactionally consistent view of the data as of the start of the
statement (i.e. it will see rowId 1 with the modified color of ‘White’) but that doesn’t really matter since this
statement is just going to go ahead and update the color of that row to ‘Red’. This it does. The transaction is then
committed and the query completes. The end result is that we have rowId 1 with a color of ‘Red’ and rowId 2 with a
color of ‘White’.</p>
<p>So there you go. Changing the <em>READ_COMMITTED_SNAPSHOT</em> setting can change the behavior of queries. Be wary. Having
your databases run with <em>READ_COMMITTED_SNAPSHOT</em> on can provide real benefits to the concurrency of database queries
and it is worth doing. Just make sure that you turn it on early in the life of your database (ideally at the start) so
that clients do not become accustomed to the shared-lock based behavior.</p>
On RDBMS2015-03-19T00:00:00+00:00http://alandavies.org/blog/2015/03/19/on-rdbms<p>I’ve been working with relational database technologies on and off for a big part of my tech career but over the last
few years I’ve had the fortune (some may say the misfortune …) to be able to use a few of them in some depth. I’m
currently involved in a heavy evaluation of RDBMS products (including Oracle, DB2, SQL Server, PostgreSQL and
MySQL/MariaDB) as part of a standardization initiative, and I’m getting to learn a lot about all of them (their
differences, commonalities and nuances). However, as of this writing, the one that I know best is Microsoft SQL Server
(MS SQL).</p>
<h1 id="cognitive-friction">Cognitive Friction</h1>
<p>One thing that I’ve noticed as I’ve got up the learning curve with MS SQL, and helped others get up that same curve, is
that there are some cognitive frustrations that experienced developers inevitably suffer when they first start working
with SQL and relational engines. Developers naturally want to control the algorithm that is used to access data and
perform a calculation, but that’s precisely what you DO NOT do when you write a SQL query. SQL is a declarative
language, as opposed to an imperative one, and it’s the query optimizer in the RDBMS that actually “writes” the code to
really execute the query. I have spent many an hour staring at execution plans and thinking to myself “Why is the
fracking optimizer choosing to do it that way as opposed to the way that I want it to be done?”. There comes a point
where you just have to let go and trust that the optimizer knows what it’s doing. This is when you also start to
develop an appreciation of the difference between a good cost-based optimizer and a poor one. You also appreciate the
critical importance of creating the appropriate constructs in your database (e.g. indexes, foreign keys, up-to-date
statistics) to give the optimizer as much information as you can on which to base its decisions.</p>
<h1 id="transactions-and-the-acid-properties">Transactions and the ACID Properties</h1>
<p>Another cognitive hurdle is that of concurrency. An RDBMS is generally a multi-user system and has to manage and
coordinate access to data across multiple client connection at varying granularities. It’s all part and parcel of
ensuring that the database engine can support its ACID promise. Remember that one? <strong>A</strong>tomicity, <strong>C</strong>onsistency,
<strong>I</strong>solation and <strong>D</strong>urability.</p>
<p>At the heart of this is the concept of a <a href="http://en.wikipedia.org/wiki/Database_transaction">transaction</a>. As
Wikipedia says so well, a transaction symbolizes a unit of work performed within a database management system (DBMS)
against a database, and treated in a coherent and reliable way independent of other transactions. They provide an
“all-or-nothing” proposition, stating that each work-unit must either complete in its entirety or have no effect
whatsoever. Further, the system must isolate each transaction from other transactions, results must conform to existing
constraints in the database and transactions that complete successfully must get written to durable storage.
Transactions are one of the key mechanisms through which a DBMS ensures that it meets its ACID contract. And speaking
of that contract …</p>
<h2 id="atomicity">Atomicity</h2>
<p>The atomicity property requires that each transaction be “all-or-nothing”; if one part of the transaction fails, the
entire transaction fails, and the database state is left unchanged. An atomic system must guarantee atomicity in each
and every situation including power failures, errors and crashes. To the outside world, a committed transaction appears
(by its effects on the database) to be indivisible (“atomic”) and an aborted transaction does not happen.</p>
<h2 id="consistency">Consistency</h2>
<p>The consistency property ensures that any transaction will bring the database from one valid state to another. Any data
written to the database must be valid according to all defined rules, including constraints, cascades, triggers and any
combination thereof. This does not guarantee correctness of the transaction in all ways the application programmer
might have wanted (that is the responsibility of application-level code) but merely that any programming errors cannot
result in the violation of any defined rules in the database.</p>
<h2 id="isolation">Isolation</h2>
<p>The isolation property ensures that the concurrent execution of transactions results in a system state that would be
obtained if transactions were executed serially, i.e. one after the other. Providing isolation is the main goal of
concurrency control. More on this below.</p>
<h2 id="durability">Durability</h2>
<p>The durability property requires that once a transaction has been committed, it will remain so, even in the event of
power loss, crashes or errors. To defend against power loss, transactions (or their effects) must be recorded in
non-volatile memory.</p>
<h1 id="cognitive-friction-revisited">Cognitive Friction Revisited</h1>
<p><em>“So that’s all fine and dandy Alan, thanks for the background, but what has that got to do with this other cognitive
hurdle that you say developers have to get over? What’s the thing that they need to let go of?” - Ed.</em></p>
<p>Why, I’m glad you asked. It’s locking, or rather their control (or lack of control) over it. Any developer with
experience of multi-threaded programming will look at an RDBMS and automatically think that data will need to be locked
to ensure transaction isolation. However they will quickly realize that they never explicitly issue locks on things
themselves. As with writing the actual query execution plan, this is the job of the database engine as opposed to the
job of the database developer. This lack of direct control can freak out many an individual, especially when they see
queries blocking each other and potentially even getting into deadlocks. Just recently I saw one, very experienced,
C++ developer throw his hands up and claim that a database was “broken” because it was experiencing frequent query
deadlocks. He wanted to “take control” of its locking but could not. It was not until I helped him to debug why his
queries were deadlocking, and educated him about transaction isolation levels (see below), that he finally calmed down.</p>
<p>This is another example of the need to “let go” and trust that the database engine will ultimately do the right thing,
as long as you embrace declarative programming and specify the sort of isolation level that you require for a given
query. That’s the real trick; knowing that you can control the required isolation level for a transaction, or set of
transactions, and understanding what those isolation levels mean. It’s more declarative programming again.</p>
<h1 id="transaction-isolation-levels">Transaction Isolation Levels</h1>
<p>MS SQL supports four transaction isolation levels by default with a fifth one (Snapshot) being available if a
database-level setting is on. There’s also another database-level setting that controls the behavior of the Read
Committed transaction isolation level. More on this in due course. Full details of the available levels can be found
on <a href="https://msdn.microsoft.com/en-us/library/ms173763(v=sql.110).aspx">MSDN</a> but a quick description follows.</p>
<h2 id="read-uncommitted">Read Uncommitted</h2>
<p>Transactions running at this level can read rows that have been modified by other transactions but not yet committed.
They also do not issue shared locks to prevent other transactions from modifying data read by the current transaction.
Such transactions are also not blocked by exclusive locks that would prevent the current transaction from reading rows
that have been modified but not committed by other transactions. When this option is set, it is possible to read
uncommitted modifications (you can get “dirty reads”). Values in the data can be changed and rows can appear or
disappear in the data set before the end of the transaction.</p>
<p>This is the least restrictive of the isolation levels and is not typically used unless application developers
specifically do not need consistency in their data model.</p>
<h2 id="read-committed">Read Committed</h2>
<p>Transactions running at this level cannot read data that has been modified but not committed by other transactions, thus
preventing “dirty reads”. However, data can be changed by other transactions between individual statements within the
current transaction, resulting in non-repeatable reads (a row returned from a previous query has a different value when
queried again) or phantom data (different rows come back from a repeat evaluation of the same query).</p>
<p>This is the default transaction isolation level. Unless a different level is specified (via the <em>SET TRANSACTION
ISOLATION LEVEL</em> command) then this is the level that will be used. The mechanism by which MS SQL achieves the semantics
of this isolation level depends on the value of the database-level setting <em>READ_COMMITTED_SNAPSHOT</em>. If that setting is
off (the default) then the semantics are implemented via locking (reads will take out shared locks on objects and will
block writers but not other readers). However, if the setting is on then MS SQL will use a row versioning scheme in
order to allow transactions to see a snapshot of the state of committed data as of the start of each statement. Shared
locks will no longer be taken by read statements in the transaction and so other transactions, that need exclusive
locks, will not be blocked and their modifications will be accommodated via the row versioning scheme.</p>
<p>Note that the life of the data snapshot is just for the statement. The next statement in a multi-statement transaction
will see a new snapshot (as of the start of that statement), and so data committed by other transactions will be visible
within the current transaction. Non-repeatable reads and phantom data are still possible.</p>
<h2 id="repeatable-read">Repeatable Read</h2>
<p>Transactions running at this level cannot read data that has been modified but not yet committed by other transactions;
and also, no other transactions can modify data that has been read by the current transaction until the current
transaction completes. This is achieved via locking. Shared locks are placed on all data read by each statement in the
transaction and are held until the transaction completes. This prevents other transactions from modifying any rows that
have been read by the current transaction.</p>
<p>Note that although other transactions can’t modify (update or delete) rows that match the search conditions of
statements issued by the current transaction (i.e. the non-repeatable reads problem is avoided), they can insert new
rows that would match those search conditions. This means that phantom data is still possible if the current
transaction were to re-run statements.</p>
<p>Because shared locks are held to the end of a transaction instead of being released at the end of each statement,
concurrency is lower than the default <em>READ COMMITTED</em> isolation level. It’s a trade off. If you need the data
isolation guarantees of this level then you have to use it, but there will be more blocking of other transactions.</p>
<h2 id="serializable">Serializable</h2>
<p>In transactions running at this level: statements cannot read data that has been modified but not yet committed by other
transactions, no other transactions can modify data that has been read by the current transaction until the current
transaction completes, and other transactions cannot insert new rows with key values that would fall in the range of
keys read by any statements in the current transaction; until the current transaction completes.</p>
<p>This is achieved via yet more aggressive locking. Range locks are placed on the range of key values that match the
search conditions of each statement executed in the transaction. This blocks other transactions from updating or
inserting any rows that would qualify for any of the statements executed by the current transaction. Thus if any of the
statements in a transaction are executed a second time, they will read the same set of rows.</p>
<p>This is the most restrictive of the isolation levels because it locks entire ranges of keys and holds the locks until
the transaction completes. Again, it’s a trade off. Transaction concurrency is lowest when this level is in use but
sometimes you just need the isolation guarantees.</p>
<h2 id="snapshot">Snapshot</h2>
<p>As mentioned above, this level is not available by default but is enabled via a database-level setting
(<em>ALLOW_SNAPSHOT_ISOLATION</em>). Similar to the implementation strategy for the <em>READ COMMITTED</em> isolation level (when the
database-level <em>READ_COMMITTED_SNAPSHOT</em> setting is on), MS SQL uses row versioning to facilitate snapshots of committed
data and avoid shared locks. Different semantics to <em>READ COMMITTED</em> are provided though.</p>
<p>Any statement in a transaction running at this level will read the state of committed data that existed at the start of
the transaction. Data modifications made by other transactions after the start of the current transaction are not
visible to any statements executing in the current transaction. So effectively, the data snapshot lives for the life of
the transaction as opposed to just one statement. Note however that the transaction can view changes made by itself.
For example, if the transaction performs an UPDATE on a table and then issues a <em>SELECT</em> statement against the same
table, the modified data will be visible.</p>
<p>Reads will not take out shared locks (as with the <em>READ COMMITTED</em> level when the <em>READ_COMMITTED_SNAPSHOT</em> setting is
on) and so other transactions that want to write data will not be blocked. Also though, transactions writing data do
not block other <em>SNAPSHOT</em> level transactions from reading data.</p>
<h1 id="snapshot-disambiguation">Snapshot Disambiguation</h1>
<p>One of the things that I see people confusing a lot when they start using MS SQL is the difference between the
<em>READ_COMMITTED_SNAPSHOT</em> setting and the <em>SNAPSHOT</em> transaction isolation level. People tend to think that
<em>READ_COMMITTED_SNAPSHOT</em> is, itself, another transaction isolation level just like <em>SNAPSHOT</em>. It is not. It is a
database setting that “changes the implementation strategy” of the existing <em>READ COMMITTED</em> transaction isolation
level. If someone says that they are running a transaction as <em>READ_COMMITTED_SNAPSHOT</em> or using the
<em>READ_COMMITTED_SNAPSHOT</em> isolation level then that is wrong, and the speaker should be corrected.</p>
<p>Executing a transaction under the <em>READ COMMITTED</em> isolation level is a statement (this is declarative programming
remember) of the sort of isolation that you require for your transaction. The semantics of that isolation are as
described above. By default MS SQL will achieve these semantics via the use of shared locks on the objects that each
statement in your transaction is reading; and those shared locks will cause other transactions that want to write to the
same object, to block since they will request exclusive locks on those objects. However, if the database-level
<em>READ_COMMITTED_SNAPSHOT</em> setting is on, then MS SQL will achieve the required semantics via row-versioning and no
shared locks will be taken. This will result in no contention with transactions that need to write data and is
generally considered a GoodThing™.</p>
<p>Turning on the <em>READ_COMMITTED_SNAPSHOT</em> database-level setting is a significant step. It will cause MS SQL to
physically change the way that it stores data pages on disk, since it will now store “versions” of rows. Additional
information has to be added to every row that is written to a data page under a row-versioned scheme and the old
versions of the rows will be stored in tempdb as opposed to the regular data pages. Your database will grow in size and
will be more reliant on tempdb. The degree to which this is the case will depend on the pattern of access from your
clients.</p>
<p>Turning on this setting can, in some edge cases, lead to a change in the behavior of transactions running under the
<em>READ COMMITTED</em> isolation level. An example of such an edge case can be found
<a href="/blog/2015/03/19/on-turning-on-rcs">here</a>. Now, there’s
nothing wrong with turning on <em>READ_COMMITTED_SNAPSHOT</em>, in fact I think that it should typically be used. There’s no
risk of inconsistent behavior among running transactions when it is on, but there is a risk of a behavior change from
when it is on to when it is off, or vice versa. If you determine that your database would benefit from row-versioning
for the <em>READ COMMITTED</em> isolation level then turn it on early in the life of the database and then leave it on. If
your database has been in place for years, and many client applications may have queries in place that use
<em>READ COMMITTED</em> (which they will since it’s the default) then be careful since there is a risk, however small, that
those clients will see a behavior change.</p>
<h1 id="optimism-in-locking">Optimism in Locking</h1>
<p>We have been talking about row-versioning for a bit but let’s come back to locking. Locking is essential in order for
an RDBMS to support a transaction’s requested isolation semantics when writes are involved, but that locking can either
be done in a pessimistic or an optimistic way.</p>
<p>Under a “Pessimistic Locking” scheme an object (row, page, table, …) is locked immediately when a write lock is
requested, while in an “Optimistic Locking” scheme it is only locked when the changes made to that record are updated.
With pessimistic locking a write transaction will always succeed but under optimistic locking there’s a chance that
another transaction might have written to the record first. When the first transaction then tries to commit it’s change
MS SQL will detect the data inconsistency and cause the transaction to fail.</p>
<p>So, optimistic locking leads to the potential for write transactions to fail, and client applications will need to catch
such failures and re-try, where it’s likely that the write will then succeed. Optimistic locking will provide for less
blocking between transactions though and thus will allow client load against the database to scale better. It’s a trade
off. I like to think that it’s one that is well worth it though, as long as you can coordinate the necessary logic
changes with client applications. As with the <em>READ_COMMITTED_SNAPSHOT</em> option above, if you plan to use <em>SNAPSHOT</em>
isolation early in the life of a database, before client applications become accustomed to the behavior, then it’s
easier to deploy.</p>
<p>You can read up on all the nuances of how to enable the <em>READ_COMMITTED_SNAPSHOT</em> and <em>ALLOW_SNAPSHOT_ISOLATION</em>
settings <a href="https://technet.microsoft.com/en-us/library/ms175095%28v=sql.105%29.aspx">here</a>.</p>
<p><img src="/images/sql_injection.png" alt="SQL Injection" /></p>
On Who You Want to Be2015-02-27T00:00:00+00:00http://alandavies.org/blog/2015/02/27/on-who-you-want-to-be<p>I recently learned that an old colleague of mine just passed away. This was sad news and a surprise, even though I knew
that he had been diagnosed with cancer some time ago and that his time was limited. He had not been working for a while
and so the status of his health had moved beyond my day to day visibility. It’s always a shock when someone you know
finally dies, even though we know that everyone will die one day. He was relatively young (early 50s) and left a loving
family and many friends and colleagues behind.</p>
<p>I have not experienced too much death as an adult. My grandparents mostly passed away when I was a child, and at that
age you don’t tend to think much about the ramifications of death other than the fact that you feel sad for a while.
You certainly don’t then tend to evaluate your own life and what you are doing. Recently, a few people that I know have
passed and it’s made me think about who we are and the way we live our lives.</p>
<p>My old work colleague, Mike D, was one such person. He was one of the early founding fathers of the company where I
myself have worked for 20 years, FactSet Research Systems. Now, the very phenomenon of working for the same firm for
20 years in today’s day and age is rare but in my case it’s been a journey that has followed the arc of a great tiny
company growing up to be a great big company (there’s another blog post to write about that …). Mike was there from
when FactSet was basically a startup and was closer to 30 years in. He didn’t hire me, I was actually hired in another
continent by another founding father (who never seems to get the historical credit that he deserves), but he was always
a significant figure in the background in my career. He was one of FactSet’s earliest engineers and did a lot of the
foundational work that made the FactSet system what it is today. He did hire many of my contemporaries, a lot of whom
are still around too, and to hear the tributes that they have offered after his death, it’s clear that he was very
influential to them; he was a mentor, a role model and just a good guy. They commented on his contribution to the
company but, more than that, they talked about him as a person, what he was like and what he did over and above his work
as an engineer, engineering director and ultimately President and COO of the company.</p>
<p>It’s this last aspect that resonates with me and the reason that I felt compelled to write this blog post. He was a
multi-faceted man with diverse interests and a good sense of humor. As I said, he didn’t hire me and I didn’t work for
him directly. He was never a close, personal mentor figure to me or even an engineering role model. I only ever knew
him once he was already quite senior in the company, as Director of the entire Software Engineering department. It
would be easy for someone that senior to feel aloof and remote but he was never like that.</p>
<p>I remember taking trips with him to visit some of our clients and talking about work but also about books and art. He
was the person who persuaded me that I had to read Zen and the Art of Motorcycle Maintenance. He was the one who
persuaded me that I had to visit the Musée d’Orsay in Paris.</p>
<p>I didn’t have a formal education in Computer Science and I had joined FactSet as a customer support rep. However, once
I saw what software engineering was all about it was clear to me that that was what I wanted to do. I took it upon
myself to learn the necessary skills but it was Mike who ultimately said yes and allowed me to join the software
department.</p>
<p>I remember working with him in Greenwich on FactSet’s first, and ultimately aborted, attempts to build a web product
back in 98/99. I remember working on refinements to our =FDS() Excel functions and then strategizing with Mike on ways
to “sell” this approach to one of FactSet’s other engineering founding fathers, who was most definitely aloof and not
always of a mind to agree to do things that were not his ideas in the first place. I remember him visiting our remote
offices in London and Tokyo when I worked there and enjoying some great dinners.</p>
<p>Others talk about his prowess on the sports field, and indeed he was a big guy and a varsity athlete in several sports.
They also talk about how he was an avid fan who always had something to say about your team and would often make a
cutting joke out of it. That didn’t mean much to me, being a non-athlete myself and also a Brit, unfamiliar with the
American sports fan dynamic. I just remember him looking like Clark Kent and being able to sit very low in his office
chair for such a tall man. There were some great images of his fandom though (see above).</p>
<p>He was not the greatest individual engineer, although his work ethic was amazing; he was not the greatest COO, being
unwilling to shake things up when they really needed it sometimes, but he was a leader that people respected. He was
someone who epitomized the idea of there being a FactSet family. He wasn’t a personal mentor to me but it’s clear that
he was one to many others and he set the tone for much of the style and approach of FactSet’s engineering for years to
come.</p>
<p>He was someone who represented the idea that you need to be more than just the role you serve in at a company. You need
to influence the people around you as a person just as much as you do as an individual contributor (salesperson,
marketer, engineer) or manager. Share your interests and love of many things and try to instill in others a similar
love. Be a person who people will remember for more than the job you do day to day.</p>
<p>I’ve worked with, and for, many people over the years who were nothing much more than the job they did. I never felt a
personal connection to those people and I will not remember them fondly. Don’t be that guy, be like Mike. I believe I
have tried to do this and I will continue to do so all the more.</p>
<p>Goodbye sir. You were a good man, I wish I had known you better.</p>
<p>So dear reader … Who are you? Who do you want to be? How do you want to be remembered?</p>
On Puns2014-10-08T00:00:00+00:00http://alandavies.org/blog/2014/10/08/on-puns<p>I tried to catch some fog. I mist.</p>
<hr />
<p>When chemists die they barium.</p>
<hr />
<p>Jokes about German sausage are the wurst.</p>
<hr />
<p>A soldier who survived mustard gas and pepper spray is now a seasoned veteran.</p>
<hr />
<p>I know a guy who’s addicted to brake fluid. He says he can stop anytime.</p>
<hr />
<p>How does Moses make his tea? Hebrews it.</p>
<hr />
<p>I stayed up all night to see where the sun went. Then it dawned on me.</p>
<hr />
<p>This girl said she recognized me from the vegetarian club, but I’d never met herbivore.</p>
<hr />
<p>I’m reading a book about anti-gravity. I can’t put it down.</p>
<hr />
<p>I did a theatrical performance about puns. It was a play on words.</p>
<hr />
<p>They told me I had type A blood, but it was a Type O.</p>
<hr />
<p>A dyslexic man walks into a bra.</p>
<hr />
<p>PMS jokes aren’t funny. Period.</p>
<hr />
<p>Why were the Indians here first? They had reservations.</p>
<hr />
<p>Class trip to ihe Coca-Cola factory. I hope there’s no pop quiz.</p>
<hr />
<p>Energizer Bunny arrested: Charged with battery.</p>
<hr />
<p>I didn’t like my beard at first. Then it grew on me.</p>
<hr />
<p>How do you make holy water? Boil the hell out of it!</p>
<hr />
<p>What do you call a dinosaur with an extensive vocabulary? A thesaurus.</p>
<hr />
<p>When you get a bladder infection, urine trouble.</p>
<hr />
<p>What does a clock do when it’s hungry? It goes back four seconds.</p>
<hr />
<p>I wondered why the baseball was getting bigger. Then it hit me!</p>
<hr />
<p>Broken pencils are pointless.</p>
On Character Encodings2014-10-08T00:00:00+00:00http://alandavies.org/blog/2014/10/08/on-character-encodings<p>In the beginning was the telegraph. It started with human operators manually sending messages in Morse code but in
time, technological evolution led to automatic teleprinters using codings such as
<a href="http://en.wikipedia.org/wiki/Baudot_code">Baudot code</a> and ultimately <a href="http://en.wikipedia.org/wiki/ASCII">ASCII</a>, the
latter growing out of the desire to support upper and lower case letters as well as numerals and punctuation.</p>
<p>ASCII coding is actually very clever. It uses 7 bits to encode a total of 128 ‘characters’ including control characters
(for the teleprinter or end display device) as well as actual characters for display. The structure of the encoding is
what’s interesting though. All the control characters (with the exception of DEL) start with the bit sequence \(00\).
This means that it’s easy to check whether a character is printable or not, you just have to check bits 6 and 7 for 0.
Also, the letters of the alphabet are all encoded in sequence starting with ‘A’ at 65 (\(1000001\)). Then ‘B’ is
\(1000010\), ‘C’ is \(1000010\), etc. So if you mask bit 7 then you get direct access to the position of the
character in the alphabet. Then the lowercase letters start with ‘a’ at 97 (\(1100001\)) and you can see the same
sequence but with bit 6 set to 1 as well. To shift from uppercase to lowercase is a simple bit flip (the equivalent of
adding or subtracting 32). Finally, the numerical digits start with ‘0’ at 48 (\(0110000\)) and then run
consecutively through ‘9’ at \(0111001\), and again we see that by masking bits 5 and 6 we just get the actual binary
values 0 - 9. DEL is an interesting character. It’s not a control character meant to indicate that the user typed the
key to delete the previous character, that’s Backspace (with code 8). Rather DEL is a way to void out an existing
character that may have been stored somewhere. A good example of this is when punched cards would have been used to
store data to be used in mainframe batches. To void out a character (indicated by a pattern of holes punched in a
column on a card) you just had to punch out all the holes, effectively making that character deleted, i.e. ‘DEL’
(\(1111111\)).</p>
<p>Here’s the table:</p>
<p><a href="/images/ascii.png"><img src="/images/ascii.png" alt="ASCII Table" /></a></p>
<p>So far, so good … for the English speaking world at least. But what about countries that used additional characters
in their alphabets? They developed their own encoding systems of course. The arrival of the computer age, and the fact
that systems processed data in 8 bit chunks (bytes), helped in that now a character would be represented as an 8 bit
pattern and so 256 different values could be stored. ASCII was naturally extended and the values 128 - 255 were used to
store additional characters. Different regions of the world chose to use this extra space differently though and thus
was born the idea of codepage where a byte was interpreted relative to any one of a number of different lookup tables in
order to determine what character it actually represented. Most codepages used exactly the same bit patterns to encode
the standard ASCII character set but they all used the patterns with a leading 1 bit (values above 127) differently.
The Latin-1 codepage (aka ISO/IEC 8859-1) was used widely in Western Europe since it contained all the characters needed
for those languages. Other codepages were created to accommodate the cyrillic and other alphabets. In countries like
Japan, China and Korea - whose languages contained way more glyphs than could be encoded in a single byte - multibyte
\encoding schemes were invented. Often these were still compatible with ASCII but the byte values above 127 were used
to indicate a shift into a different page where the following byte value was interpreted. This allowed them to encode
pretty much all the characters that they needed. Standards proliferated and incompatibilities abounded. It wasn’t so
much of a problem in the early days of computing but with the advent of the Internet, and the fact that data generated
on one system (e.g. in Japan) could be shared with another system (e.g. in France), the incompatibilities were brought
into stark relief.</p>
<p>Things could be made to work via clever switching of codepages but it wasn’t pretty. A new standard was needed. An
industry working group was formed called the Unicode Consortium and, via a minor miracle, they managed to create a new,
all encompassing standard called, unsurprisingly, Unicode. In very simple terms Unicode is a big table that assigns a
unique number (a Unicode code point) to all characters. The Unicode standard actually includes a lot more than that
though: rules for character collation and other essential matters such as support for right to left text; but for our
purposes we can think of it as character = unique number. The code point is just a number now, how it is stored in
computer memory is another matter.</p>
<p>Initially computer vendors implemented Unicode by storing all characters using either two or four bytes. These two
schemes were known as UCS-2 and UCS-4 respectively (UCS = Universal Character Set). There were a few problems with this
approach though. Firstly, the vast majority of the textual data stored on computers around the world was English and
using two (and especially using four) bytes per character, to store data that was mostly just ASCII, was incredibly
wasteful. UCS-4 encoded files were four times the size of the equivalent ASCII files. Secondly, the C programming
language had introduced the concept of the null terminated string (commonly called C strings) whereby string data was
stored as an array of bytes (characters) with the end of the string marked by a null byte. This assumed string
structure was baked in to a “lot” of code, and that code was now incompatible with strings stored as arrays of UCS-2/4
characters because those character arrays contained null bytes in the middle of the strings. Code written to expect C
strings would misinterpret the leading null byte in the UCS-2 encoding of an uppercase letter ‘A’ (ASCII code 65) as an
end of string marker.</p>
<p>UCS-2 and UCS-4 were fine for new applications that stored, and interpreted, all strings as arrays of multiple bytes but
they were no good for the efficient transmission of character data (because of the bloat) and interaction with legacy
APIs (because they broke the C string paradigm). A new encoding was needed.</p>
<p>So along came UTF-8, a variable length Unicode encoding scheme, supposedly designed in an evening on a
<a href="https://www.cl.cam.ac.uk/~mgk25/ucs/utf-8-history.txt">placemat</a> in a diner. UTF = Unicode Transformation Format.
UTF-8 is very clever too.</p>
<ul>
<li>The UTF-8 encoding of ASCII is ASCII. Nice. For all the character data out there that fits in that space it just
stays the same.</li>
<li>UTF-8 does not have embedded nulls and so UTF-8 strings can still be considered as null-terminated byte arrays, and
thus can be consumed by legacy C APIs.</li>
<li>UTF-8 supports Unicode codepoints above 127 via a variable length byte encoding with the following scheme:</li>
</ul>
<p><a href="/images/utf8_encoding.png"><img src="/images/utf8_encoding.png" alt="UTF-8 Encoding" /></a></p>
<ul>
<li>Notice that the following bytes in a multibyte sequence all start with \(10\). This means that those bytes will
never be misinterpreted as ASCII characters.</li>
<li>Also it’s very easy for code to move forwards and backwards by characters in a UTF-8 string. Simply scan bytes
(forwards or backwards) until you find the next (previous) one that starts with something other than \(10\).</li>
</ul>
<p>Let’s look at an example: é, the lower case acute accented e character, whose Unicode codepoint is 233. This is
encoded as follows:</p>
<p><a href="/images/e_acute_encodings.png"><img src="/images/e_acute_encodings.png" alt="e acute encodings" /></a></p>
<p>UTF-8 became incredibly popular and I think it’s fair to say that it is now considered the de-facto way for character
data to be encoded and exchanged across the Internet. In fact I saw a statistic the other day that stated that now
there is more UTF-8 encoded data stored across computer hosts in the world than old codepage encoded data.</p>
<p>For strings stored in memory inside of a given modern application it’s likely that a fixed width character encoding will
still be used, for the efficient (offset based) random access to characters that it gives. However as soon as that
character data leaves the application domain and is exchanged with another application it will almost certainly be
serialized as UTF-8.</p>
On Pi2014-10-07T00:00:00+00:00http://alandavies.org/blog/2014/10/07/on-pi<p>Given that my last post was about floating point I thought this comic strip was very apropos, and it also introduces
the topic of \(\pi\) which is worth a post of its own.</p>
<p>So much has been written about \(\pi\) already, and it’s probably the most broadly known mathematical constant, but for
those (hopefully few) who don’t know what it is I refer you to this button …</p>
<p><img src="/images/pi_button1.jpg" alt="Pi button" title="Whatever you do, don't click this button!" onclick="javascripts:alert('I told you not to click it!')" /></p>
<p>The ratio of the circumference of a circle to it’s diameter is a constant that is the same for any size circle. This
constant has been known since antiquity and is represented by the greek letter \(\pi\) (pronounced pie). It is an
irrational, and transcendental, number whose decimal representation is \(3.141592…\).</p>
<p>Now, why does the definition involve the diameter of the circle as opposed to the radius? This is a question around
which there is some controversy and there are people who are firm believers that \(\pi\)
<a href="http://www.tauday.com/">is wrong</a> and instead we should use the number \(6.283185…\) that they represent by the greek
letter \(\tau\) (pronounced tau).</p>
<p>Most people can quote you the first few digits of \(\pi\) (\(3.141592…\)), and to many that “is” \(\pi\), but of
course we know (as we learnt in my post about <a href="/blog/2014/09/24/on-floating-point">floating point</a> it is but one
possible representation, namely the base 10 positional notation representation. We can represent \(\pi\) in different
bases as we can any number. Here are a few alternative representations:</p>
<p><img src="/images/digits_of_pi_in_various_bases.png" alt="digits_of_pi_in_various_bases" /></p>
<h1 id="resources-about-pi">Resources about \(\pi\)</h1>
<ul>
<li><a href="http://en.wikipedia.org/wiki/Pi">What Wikipedia has to say on the subject</a></li>
<li><a href="http://www.joyofpi.com/">The Joy of Pi</a> (the website that accompanied the
<a href="http://www.amazon.com/dp/0802775624">book</a>)</li>
<li><a href="http://www.geom.uiuc.edu/~huberty/math5337/groupe/welcome.html">A slice of Pi</a></li>
<li><a href="http://www.exploratorium.edu/pi/pi_archive/Pi10-6.html">A million digits of Pi</a></li>
<li><a href="https://www.youtube.com/watch?v=5iUh_CSjaSw">Vi Hart on Pi</a></li>
<li><a href="https://www.youtube.com/watch?v=jG7vhMMXagQ">More Vi Hart on Pi</a></li>
</ul>
On Floating Point2014-09-24T00:00:00+00:00http://alandavies.org/blog/2014/09/24/on-floating-point<p>I work for a software company that develops productivity applications for financial professionals (bankers, traders,
portfolio managers, etc.). Our core activity is the distribution and presentation of numbers. Again and again over the
years I have seen questions coming from clients, or from our client support staff, about the accuracy of numbers. Lay
people (i.e. non software developers) understand the basic notion that computers store numbers to a limited degree of
precision but their conceptual understanding stops there. If I had a dollar for every time I’ve heard someone say (or
more accurately, write) that “floating point” numbers (they know the term at least) are accurate to only seven
significant figures, then I would be a rich man. It’s not that straight forward. I’ve tried, and failed, on several
occasions to educate people as to how floating point representation actually works. However, despite that track record,
I am going to try one more time.</p>
<h1 id="base">Base</h1>
<p>The first thing we need to understand is the concept of base in the representation of numbers. Actually, even before
that we need to recognize that “123” is just a symbolic representation of the abstract concept of the number one hundred
and twenty three. And (and this is the important part) it’s only one of a whole range of possible symbolic
representations. “123” is the representation of the number in base 10 positional notation. Now that representational
system happens to be a very natural and convenient one that is rooted in the history of western culture. In fact, it’s
how we “say” numbers in English (and many other languages): “one hundred and twenty three”. Well, roughly speaking that
is, one might argue that perhaps it should be “one hundred, two tens and three”, but we’re not here to talk about the
history of the evolution of the English language; we’re here to talk about floating point, and first about base.</p>
<p>Let’s think about what a set of digits in base 10 positional notation actually means. It’s a description of a sum of
multiples of powers of ten.</p>
<script type="math/tex; mode=display">123_{base10} = (1 \times 10^2) + (2 \times 10^1) + (3 \times 10^0)</script>
<p>More generally, for an integer N:</p>
<script type="math/tex; mode=display">N_{base10} = \sum_{i=+\infty}^{0}{d_i10^i}</script>
<p>Where \(d_i\) are the digits of N in base 10.</p>
<p>We can extend this beyond integers of course.</p>
<script type="math/tex; mode=display">123.45_{base10} = (1 \times 10^2) + (2 \times 10^1) + (3 \times 10^0) + (4 \times 10^{-1}) + (5 \times 10^{-2})</script>
<p>Or more generally, for a real number R:</p>
<script type="math/tex; mode=display">R_{base10} = \sum_{i=+\infty}^{-\infty}{d_i10^i}</script>
<p>Of course we don’t have to use base 10; any base will do. The representation of R in base B is:</p>
<script type="math/tex; mode=display">R_{baseB} = \sum_{i=+\infty}^{-\infty}{d_iB^i}</script>
<p>For example, using base 8:</p>
<script type="math/tex; mode=display">123.45_{base10} = (1 \times 8^2) + (7 \times 8^1) + (3 \times 8^0) + (3 \times 8^{-1}) + (4 \times 8^{-2}) +
(6 \times 8^{-3}) + (3 \times 8^{-4}) + (1 \times 8^{-5}) + ...</script>
<script type="math/tex; mode=display">123.45_{base10} = 173.34631..._{base8}</script>
<p>Note the use of the ellipsis there to indicate that the digits continue. They continue forever actually with the four
digit pattern \(4631\) repeating again and again. The thing to note here is that a fraction with a relatively compact
representation in one base (\(.45_{base10}\)) can have a much more verbose representation in another base. And the
reverse can be true too.</p>
<script type="math/tex; mode=display">0.001953125_{base10} = (1 \times 8^{-3}) = 0.001_{base8}</script>
<p>The fractional part of the base N positional notation representation for a rational number will either terminate or fall
into a repeating sequence of a fixed number of digits. Whether it’s one or the other will depend on the base. An
irrational number, however, will have a non-terminating, non-repeating fractional part (e.g. \(\pi = 3.14159265 …\)).</p>
<p>Consider \(1 \over 10\). This is \(0.1\) in base 10 but \(0.000110011001100…\) in base 2, where the pattern of
repeated digits (bits in this case) is \(0011\). And \(1 \over 3\) is \(0.1\) in base 3 but is \(0.3333333…\) in
base 10 with \(3\) repeated forever.</p>
<h1 id="scientific-notation">Scientific Notation</h1>
<p>The second thing we need to understand is the concept of scientific notation, a standard way of writing numbers that are
too big or too small to be conveniently written in decimal form. In this notation any Real number R is represented as:</p>
<script type="math/tex; mode=display">R = m \times B^e</script>
<p>Where m is the normalized value of R (called the significand or mantissa) and e is an integer (called the exponent). m
is chosen such that \(B^0 \leq |m| < B^1\) and e is chosen accordingly. For example:</p>
<script type="math/tex; mode=display">123.456_{base10} = 1.23456 \times 10^2</script>
<script type="math/tex; mode=display">0.00123456_{base10} = 1.23456 \times 10^{-3}</script>
<h1 id="significant-figures">Significant Figures</h1>
<p>The final thing we need to understand is that for obvious reasons we can’t store and manipulate numbers in a
representation that requires an infinite number of digits. We are always going to have to limit the amount of space
(computer memory or literal space on a page) required, and trade precision for storage size. So, we have the concept of
the maximum number of contiguous digits (known as the number of significant figures) that can be part of a given
representation.</p>
<p>Let’s look at a base 10 example using scientific notation and assume that we use a fixed 6 significant figures for the
mantissa and 2 for the exponent.</p>
<script type="math/tex; mode=display">123.456789012_{base10} = 1.23457 \times 10^{02}</script>
<script type="math/tex; mode=display">123,456,789,012_{base10} = 1.23457 \times 10^{11}</script>
<script type="math/tex; mode=display">0.000000000123456789012_{base10} = 1.23457 \times 10^{-10}</script>
<p>In the first case we actually store \(123.457\) which is off by \(0.000210988\); in the second case we actually store
\(123,457,000,000\) which is off by \(210,988\); and in the third case we actually store \(0.000000000123457\) which is
off by \(0.000000000000000210988\). In all cases the delta in percent terms is the same: \(0.000171\%\).</p>
<h1 id="floating-point">Floating Point</h1>
<p>And so finally we come to floating point representation as codified in the IEEE 754 standard, Commonly encountered
formats are 32 bit (“single precision” or “float”) and 64 bit (“double precision” or “double”). Both use the same
structure to represent a number in base 2 scientific notation but allocate a different number of bits to each part. A
single precision value is structured like this:</p>
<p><img src="/images/ieee_754_float_example.png" alt="sfsdf" class="center-image" /></p>
<p>There’s an initial sign bit, followed by 8 bits in which the exponent value (e) is stored as an unsigned integer (with a
bias of 127), followed by 23 bits in which the digits of the normalized mantissa are represented with an implicit
leading 1 digit (i.e. \(1.b_{22}b_{21}b_{20}…b_0\)). The represented value is given by:</p>
<script type="math/tex; mode=display">value = (-1)^{s} \times (1 + m) \times 2^e</script>
<p>where \( e = e_{biased} - 127 \), \( e_{biased} = \sum_{i=23}^{30}{b_i2^{i-23}} \) and
\(m = \sum_{i=22}^{0}{b_i}2^{i-23}\).</p>
<p>In the above example:</p>
<script type="math/tex; mode=display">sign = 0</script>
<script type="math/tex; mode=display">m = \sum_{i=22}^{0}{b_i2^{i-23}} = 2^{-2} = 0.25</script>
<script type="math/tex; mode=display">e_{biased} = \sum_{i=23}^{30}{b_i2^{i-23}} = 2^2 + 2^3 + 2^4 + 2^5 + 2^6 = 124</script>
<script type="math/tex; mode=display">e = -3</script>
<script type="math/tex; mode=display">2^e = 2^{-3}</script>
<script type="math/tex; mode=display">value = (-1)^0 \times (1 + 0.25) \times 2^{-3} = 0.15625</script>
<p>Double precision values use 11 bits to store the exponent and 52 bits to store the mantissa.</p>
<p>So, what about the misconception that we mentioned at the top of this post, that a float is accurate to 7 significant
figures? Well, I hope you now realize that it depends on the number and how that number can be represented as a sum of
powers of 2. The number \(1 \over 65536\) is \(0.0000152587890625\) in base 10 but that can be represented with 100%
accuracy in a float since it’s just \(1 \times 2^{-16}\). However the number \(1 \over 10\), \(0.1\) in base 10, cannot
be represented accurately in a float since it cannot be represented as a finite sum of powers of 2.</p>
<script type="math/tex; mode=display">0.1 = 2^{-4} + 2^{-5} + 2^{-8} + 2^{-9} + 2^{-12} + 2^{-13} + 2^{-16} + 2^{-17} + ...</script>
<p>With only 24 significant binary digits the infinite summation terminates and is equal to \(3355443 \over 33554432\)
which is \(0.0999999940395355\).</p>
<p>So, the next time someone says that a float has 7 significant figures, the correct answer is “well, that depends …”.</p>
On Intellectual Jokes2014-09-19T00:00:00+00:00http://alandavies.org/blog/2014/09/19/on-intellectual-jokes<p>I saw this old joke again today …</p>
<p>“What’s the difference between a joke and a rhetorical question?”</p>
<p>… and it reminded me of the beauty of the “Intellectual Joke” - a joke that is only funny to those “in the know”,
those with a base of knowledge or understanding about certain topics. I love these jokes. So I thought I’d compile a
list of a few of my favorites.</p>
<hr />
<p>Entropy isn’t what it used to be.</p>
<hr />
<p>Q: What’ s the difference between an entomologist and an etymologist?</p>
<p>A: An etymologist knows the difference.</p>
<p>Ed: Something about that joke bugs me a little.</p>
<hr />
<p>Some scientists want to cool my body down to -273.15 degrees Celsius. My wife thinks it’ll kill me, but I think I’ll be
\(0\)k.</p>
<hr />
<p>Is it solipsistic in here or is it just me?</p>
<hr />
<p>Werner Heisenberg, Kurt Gödel, and Noam Chomsky walk into a bar. Heisenberg turns to the other two and says, “Clearly
this is a joke, but how can we figure out if it’s funny or not?” Gödel replies, “We can’t know that because we’re
inside the joke.” Chomsky says, “Of course it’s funny. You’re just telling it wrong.”</p>
<hr />
<p>Werner Heisenberg and Erwin Schrödinger are driving together and get pulled over for speeding. The cop asks Heisenberg
“Do you know how fast you were going?” Heisenberg replies, “No, but we know exactly where we are!” The officer looks
at him confused and says “you were going 108 miles per hour!” Heisenberg throws his arms up and cries, “Great! Now
we’re lost!”</p>
<p>The officer looks over the car and asks Schrödinger if the two men have anything in the trunk. ”A cat,” Schrödinger
replies. The cop opens the trunk and yells “Hey! This cat is dead.” Schrödinger angrily replies, “Well he is now.”</p>
<hr />
<p>Q: Why do engineers confuse Halloween and Christmas?</p>
<p>A: Because Oct 31 = Dec 25</p>
<hr />
<p>There are 10 kinds of people in the world, those who understand binary and those who don’t.</p>
<hr />
<p>There are two kinds of people in the world, those who think that there are two kinds of people in the world and those
who don’t.</p>
<hr />
<p>There are two kinds of people in the world, those who can extrapolate from incomplete data and …</p>
<hr />
<p>Three logicians walk into a bar. The bartender asks “Do all of you want a drink?” The first logician says “I don’t
know.” The second also says “I don’t know.” The third says “Yes!”</p>
<hr />
<p>A Helium atom walks into a bar and orders a beer. The bartender says, “Sorry, we don’t serve noble gases here.” He
doesn’t react.</p>
<hr />
<p>The French existentialist Jean-Paul Sartre was sitting in a cafe when a waitress approached him: “Can I get you
something to drink, Monsieur Sartre?” Sartre replied, “Yes, I’d like a cup of coffee with sugar, but no cream.”
Nodding agreement, the waitress walked off to fill the order and Sartre returned to working. A few minutes later,
however, the waitress returned and said, “I’m sorry, Monsieur Sartre, we are all out of cream - how about with no milk?”</p>
<hr />
<p>Q: What does the “B” stand for in Benoît B. Mandelbrot?</p>
<p>A : Benoît B. Mandelbrot.</p>
<hr />
<p>I’m so meta even this acronym.</p>
<hr />
<p>xkcd - Some of my favorites</p>
<ul>
<li><a href="http://imgs.xkcd.com/comics/sandwich.png">sudo</a></li>
<li><a href="http://imgs.xkcd.com/comics/tabletop_roleplaying.png">Hey, no recursion!</a></li>
<li><a href="http://imgs.xkcd.com/comics/the_general_problem.png">The general problem</a></li>
<li><a href="https://imgs.xkcd.com/comics/honor_societies.png">The first rule of tautology club</a></li>
<li><a href="https://imgs.xkcd.com/comics/centrifugal_force.png">You spin me right round baby right round …</a></li>
</ul>
On Writing2014-09-15T00:00:00+00:00http://alandavies.org/blog/2014/09/15/on-writing<p>Writing … a skill that is vastly underappreciated.</p>
<p>Good writing is rare these days, especially in the modern business arena where throw away communication - email and,
increasingly, instant messaging - tends to dominate. A well constructed, and well edited, document is a pleasant breath
of fresh air among the usual collection of odors that masquerade as typical business communication. So, it seems
appropriate that my first real blog post should be on the subject of writing. It is afterall one of the main reasons
that I decided to finally start a blog, long after blogs have become old hat, to be replaced with the modern world of
ocial media, throw away “posts” and 140 character tweets.</p>
<p>Now, my views are, inevitably, colored by the arena of my experience, that of software development in the service of
financial services. Perhaps other fields - the law or journalism - are still bastions of well constructed rhetoric, I
don’t know. However, in the world of software, good code is seen far more often than good prose.</p>
<p>I have always valued communication skills alongside technical skills and I’d like to think that I have a good collection
of both. My career to date has certainly been defined by the blend of communication and technology, and often by the
task of communicating “about” technology, whether that is as part of the sales or client support function, internal
project management, or training and documentation. The moniker “Communication Skills” is ascribed to a category of
competences including oral presentation, coaching, selling, consulting, writing and more. We all get opportunities to
exercise our verbal skills every day, and I can wax extemporaneous with the best of them, but how often do we go out of
our way to exercise our skill in writing?</p>
<p>I am starting this blog for various reasons: to create more of a public face for myself (if I’m honest) but also to
give me a reason to write. I want to take the time to craft articles about topics that interest me and publish them for
posterity. Perhaps some people will get some value out of the things I write, I sincerely hope so, but it doesn’t
really matter if they do or they don’t. I’m doing this so that I can exercise some discipline in order to research and
craft a collection of articles, each of which will represent a whole lot more effort than a Facebook post or a tweet.
Now this is not to say that I don’t see Facebook and Twitter as valuable communication channels, I do. I use Facebook
all the time (perhaps too much actually) but it’s a very different medium, all (for me at least) about staying in
contact with friends (old and new, near and far). I’m a big fan of throw away posts and sharing random witticisms and
aphorisms but that’s not writing. This blog will be my writing.</p>
<p>At least I’m going to limit myself to the “typed” word as opposed to the written word; I’m not that much of a masochist.
I wrote way too many essays by hand back in school (far too many years ago) and I don’t think my hand would last for
more than a page these days before it would collapse into a withered claw of cramp. No diary for me, just some online
articles about software development, mathematics, guitars, culture and other miscellaneous crap.</p>
<p>Of course this could all be hubris and bravado and I might end up writing a bunch of cheesy fanboy pieces on nostalgic
topics from my youth.</p>
<p>So you have been warned. Abandon all hope ye who continue beyond this post …</p>
On Getting Started2014-09-12T00:00:00+00:00http://alandavies.org/blog/2014/09/12/on-getting-started<p>I have been meaning to create a personal website and associated blog for some time now and finally, and self-evidently,
I have. Every journey has a first step and every blog has a first post, so this is it. It will be brief and will end
here. Until next time.</p>