Bayes’ Theorem by intuition

There are¬†numerous articles all over the internet explaining Bayes’ theorem.¬†It is thought to be the¬†basis for logical reasoning and is important for us all¬†to understand. Yet¬†most of us find mathematical symbols confronting and examples used in the explanations¬†hard to follow.¬†So, this post is my attempt to explain the theorem intuitively and plainly as possible.

Mathematically, it is defined as:

Bayes' Theorem

where P(X) is the probability of event X occurring while P(X|Y) implies probability of event X happening given that event Y has happened

To put things into perspective, let’s discuss¬†two¬†simple examples:

Example in usage with percentage data

In a fictional suburb called Livingstone, the census says that an average of 70% of the¬†population¬†suffers from¬†a drinking problem. Now imagine you and I are driving around this region. I point to a random person walking on the street and ask you what is the likelihood that this person has a drinking problem. It is very likely that you’ll say 70% or that the probability is 0.7 (get from percentage to probability by dividing with 100).

Well if the person was a man from say 18 to 30 yrs of age, it would most likely be right and you’d¬†be much more confident on your answer. But¬†what if¬†I told you it¬†was a 5 yr old child instead? Saying that a 5 yr old child¬†has the¬†likelihood of having a drinking problem¬†is 70% is obviously absurd. Now we see the problem. Yes, I deliberately gave you¬†incomplete information so that you would trick yourself by making false assumptions¬†when you read the words ‘average’ and ‘person’ in the description above .

So, what should you have done to be ‘closer’ to correct estimation? For one, not assume that one statistic – here it was percentage – is enough to¬†tell the full truth. This is why there is a whole branch of mathematics called Statistics. Secondly, when someone quotes average measure of a dataset it is usually cumulative of different distribution and so we must correct it for our target distribution – which in the latter case was children. This second requirement¬†is exactly what Bayes’ theorem is.

Example in usage with using evidence to support hypothesis

Bayes’ theorem is usually used for reasoning how well a theorem is supported by evidence.¬†So, let’s now explore the implication of this theorem for supporting a¬†hypothesis using evidence.

You have a cat and a dog as pets. When you return home from work and see that the milk box is toppled and leaking (observation). You know that the cats¬†like milk more (prior information before looking at the evidence). But then you see some milk¬†on the dog’s face¬†(evidence).

Before we continue,¬†let’s us evaluate this. Notice that if we don’t consider¬†the evidence, the cat seems like the likely suspect but when we do, the dog seems like the suspect. The similarity with the previous example¬†can also be evident. In¬†it, before we knew that the ‘person’ was a child, we were confident that the answer was 0.7 but after we did know that we¬†had doubts that¬†it was not the right answer. This is Bayes’ theorem in summation.

This whole scenario can be shown in probability diagram as follows:

bayestheorem3

From the diagram above, we can see that we want to¬†take the evidence agreeing with hypothesis¬†part (represented in the diagram with P(Evidence is true) ‚ą© P(Hypothesis is true) intersection)¬†from the probability space of¬†evidence being true¬†to probability space of hypothesis being true given that evidence is true.

How? First we should know that the intersection part has contribution from both P(Evidence is true | Hypothesis is true) and P(Hypothesis is true) probability spaces. We need is contribution only from P(Evidence is true | Hypothesis is true). For that, we divide the intersection probability with P(Hypothesis is true) as:

bayestheorem5

Now we will want to rescale this from P(Evidence is true) to P(Hypothesis is true), in the next section.

Rewriting the equation

We can rewrite the original mathematical¬†Bayes’ theorem equation as:

bayestheorem3

Or

Bayes' Theorem

If you are not familiar, this shows the associative property of multiplication.

In the rewrite above,¬†separating the terms with ‘*’ (multiply operation)¬†is important to make a point as we will see. Multiplication operation¬†has an associative property (which¬†means ‘(A*B)/C’ is equal to ‘A*(B/C)’). This property ‘obscures’ the implied meaning when the formula is¬†written in the original form. This is unfortunate because mathematical statements are supposed to show truth statement simply.

First, a minor detour to hit the point home. Remember how we calculate percentage? If you get 7 out of 10 in a test your percentage is calculated as:

bayestheorem4Now, notice the similarity¬†with the rewritten equation¬†above. This similarity is not coincidental. When calculating percentage we are¬†moving our scale from ‘out of 10’ to ‘out of hundred’.

Similarly, in the first example involving percentage, we are moving from average drinking problem from all age groups to average of no. of drinkers given selection from a specific age group. And in the second example we are moving from the prior belief that cats prefer milk to evidence to how evidence supports its likelihood to be correct and moved away from it as evidence against it was strong.

For the first example, if thinking in terms of probability is difficult, the following might make it easier to understand:

bayestheorem6

Summation

We start with initial belief called prior. As we collect evidence, its likelihood of being true allows us to change our belief either towards or away from prior.

 

UWP Custom control: Expandable Row ListView

I was working on a Universal Windows Platform app where I needed a ListView whose rows can be expanded to reveal further data underneath it. I think these controls are called DataRowTables in Web UI programming. So, I decided to write one.

ExpandableRowListView usage animation

To use it, you can either bind its ‘ItemsSource’ to an ‘IEnumerable<ExpandableRowListViewItem>’ or add static items in XAML or add static items in code using ‘ExpandableRowListView.Items.Add(…)’ method. These usages are shown in the project containing the source code.

Source code

Implementing custom captions renderer for Android VideoView

From Android KitKat (API level 16), programmers can use ‘addSubtitleSource()’ on VideoView to add a WebVTT format subtitle track. That’s it. Nothing else to do. The native subtitle handler even automatically listens and changes drawing styles according to system accessibility caption settings. (FYI – System captions settings are under Settings->Accessibility->Captions) Awesome!

But only that it isn’t.¬†Most of the application workflows have a captions button for the user to turn captions on/off. Unfortunately, there are no public member functions on VideoView control to allow the developer, in proxy of app user, to control captions visibility from within the app. All the developer can do is redirect the user to system accessibility page using ‘Intent’. Even then, the captions settings are global. Sure, if the user has permanent hearing disability, this is no problem. He/she would want to turn on caption for all the apps with one switch. Unfortunately, this is not always the case. Even a normal user may want to mute sound and watch the video with captions in a quiet environment. Then, the app will need a button within it for the user to toggle captions. Another, though minor, quirk is that ‘addSubtitleSource()’ works with WebVTT format. The ubiquitous captions format is SRT. This will require you to wire an intermediate converter (A simple SRT to WebVTT converter that I wrote for testing can be downloaded here). Hence, there is atleast two strong cases to implement a custom captions handler.

Android Custom Captions Renderer

1. Get an instance of internal MediaPlayer

The VideoView control has a private member object instance of MediaPlayer control and implements ‘addSubtitleSource()’ on top of it. So, the first plan of attack is getting an instance of this¬†MediaPlayer object instance. One way of doing this is using Java’s reflection capabilities to call hidden member functions in VideoView class – i.e. ‘VideoView.class.getDeclaredMethod(“<Method name>”).invoke(<Params>)’. This, of course, is very ‘hacky’ and we want to stay away from this as much as we can in production code. Fortunately, there’s another way. If we create a new derived class from ‘VideoView’, we can override the ‘onPrepared()’ event and get an instance of its internal MediaPlayer object in the event listener’s function parameter. When this event fires, the video player has loaded the video and is ready to accept captions track. Using MediaPlayer’s ‘addTimedTextSource()’, we can add an external captions track. This can be SRT format.

2. Handle special condition where caption file is on the network

If your captions file is on the network though, we have another hurdle to jump. The function accepts only local file path or a local file descriptor so we have to download the captions to a local file in a temp directory and then add code to handle file clean¬†up when it is no longer needed. If your code fails to properly cleanup, you will be cluttering user’s precious storage space. Alternatively, we could go another path by using ‘MemoryFile’. MemoryFile is a file mapped into memory used for inter-process communication. Nice thing about this is that the system takes care of the file maintenance part while we just read/write to memory. Using Java’s reflection capabilities we can call hidden member ‘getFileDescriptor()’ on MemoryFile to get a ‘FileDescriptor’ to use with ‘addTimedTextSource()’. I would argue that though ‘getFileDescriptor()’ is hidden, it is safe to call it. It ties directly with underlying Linux API and there isn’t any reason for Android ¬†framework to change or remove this function in the future.

3. Rendering caption on specific time

When you set a listener using ‘MediaPlayer.setOnTimedTextListener()’, we have only asked MediaPlayer to notify us using ‘onTimedText’ event when caption text start/end timing is reached on playback. We still have to put a TextView on top of VideoView to show the captions. This can be done easily in the video activity. When the listener handler is invoked, it will have a ‘TimedText’ object on its¬†function parameter. The ‘getText()’ method of this parameter will return¬†the caption text to be rendered at this playback time. If the end time for a caption is reached, this will return a null string. Then, we just hide the TextView captions text control. There is also a ‘getBounds()’ method in the ‘TimedText’ object. Optionally, this may be a non-null ‘Rect’ object. This is a valid Rect object when the input captions file contains positioning information. A rigorous implementation will use this bounds by moving TextView accordingly. One case where this may be used is when an object/person important to the video’s subject matter is on the bottom part of the video which is usually occluded by the captions text.

Pitfalls

The native SRT parser is confused by subtitle information blocks where there is sequence and timing information but the captions text is empty. For instance, as in caption sequence 2 below:

1
00:00:00,000 --> 00:00:00,100
Caption text 1

2
00:00:01,200 --> 00:00:02,000


3
00:00:04,000 --> 00:00:06,000
Caption text 3

One way to fix this is by replacing the ‘\r\n\r\n\r\n’ (assuming Windows style line break) after the end time in sequence 2 with ‘\r\n{Any non-white space character here}\n\r\n’.

QR code generator console application

QRCodeGenerator
QR Code Generator

Source code: https://github.com/sanje2v/QRCodeGenerator

QR code generators are nothing new – they are all over the internet. Yet, I decided to write one as a console application which I didn’t find any (not that it would be groundbreaking in anyway but just to learn how QR code is implemented). This application implements QR code version 1 and encodes only in alphanumeric mode (hence limited to encoding in uppercase English characters).

This project was not intended to cover all implementations of QR code versions and is not likely to be developed further in the future Рbut perhaps, a decoder if I feel like it. Rather, I wanted to understand how QR code is generated. One of the most important (and interesting) component is error correction which  covers some interesting topics like number theory, Galois field and polynomial arithmetic. I had a lot of fun implementing these. An unexpected field I got to explore was masking where the specification tries to remove unwanted patterns Рi.e. patterns which may cause confusion to decoders. The source code for this project will be helpful to anyone starting to learn QR code encoding.

Aside from the official QR code specification document, http://www.thonky.com/qr-code-tutorial/ provides a good overview with walk-through samples for various steps of the encoding process. Not being a math buff, the guide was especially helpful to me on understanding binary polynomial division in Galois field.

In order for the console application to display the square QR code correctly, the program changes console’s font size and type in function ‘SetConsoleAttributes()’. The font index number used in this function is undocumented and was determined by changing Windows 10 console properties and watching the index number change in Windows API. If you are using non-Windows operating system, please change this function to correct your console font to any monospaced font. Also, the function forces the console to black background with bright white foreground as required by the specification.