A detailed synopsis of the book ‘Data Science for Business’, What you need to know about Data Mining and Data-Analytic Thinking, written by Foster Provost and Tom Fawcett, published by O’Reilly Media.
The preface alone, tackling the ever present discussion of cross-over from academia to commerce, illustrates some nifty and jaw-dropping knowledge re-use techniques that put the reader in the right frame of mind to be creative in exploiting big data.
Chapter 1 centres the argument, with a good coverage of the well-known project where Walmart’s competitor Target managed to use their sales data actually predict when their customers were likely to start families and change their buying patterns accordingly. See the work by Duhigg (2012) in the bibliography for the full story.
Chapter 2 addresses the terminology in the hope that we can start standardising the language of this new field. I am not convinced their choices will stand the test of time, but the task certainly needs doing if we are all to speak the same language. There are many names for the same things – because so many fields converge in the field of Data Science. The authors quite rightly go back to the first principals of the pioneers, including my personal hero Claude Shannon, with his work on information theory dating from 1948.
If, as I do, you learn by experience rather than theory, you may be interested in their explanation of the emerging CRISP Data Mining Model: Cross Industry Standard Process for Data Mining, introduced by Shearer, 2000.
By chapter 6 ( Similarity, Neighbours and Clusters ) things get really interesting and certainly stretch what I see in most regular commercial decision making. The authors introduce another ingredient which suddenly makes the whole mixture fizz: The fundamental concept of similarity between data items. Their bibliography leads to the ‘Dictionary of Distances’ by Deza & Deza (Elsevier Science, 2006) which explains the many ways of calculating the distance between 2 things. Let me explain just a few:
- Euclidean distance. If you know your lat and long and you know the lat and long of your customer, think Pythagoras and use squares to approximate how far away they are. As a bird flies.
- Manhattan distance. A better way when customers can’t fly direct, but they have to use a grid of roads to reach you.
- Cosine distance. A calculation used in text classification to measure the similarity of two documents
- Edit distance or the Levenshtein metric. Also used in biology. How similar are 2 gene strings? How similar are 2 pieces of copy? Same calculation.
Chapter 7 gets really personal: how accurate are you yourself? how many decisions do you make? How often are they correct? What is your performance as a data analyst? What is the cost/benefit of all this? and introducing … the confusion matrix.
The subject matter of Chapter 8, on ROC curves, graphs for visualizing, and profit graphs will in truth be well familiar to most but I for one will be re-reading it a few times to scavenge tips for tuning my own algorithms
Chapter 9 turns to such activities as Targeting Online Consumers With Advertisements. Read for the fascinating discussion of the 2013 study by Michal Kosinski, David Stillwell and Thore Graepel on how what people “Like” on Facebook is quite predictive of many unexpected traits
Accompanied by a really cool footnote / academic joke (I think): “For those unfamiliar with Facebook, it is a social networking site that allows people to . . . “
Chapter 10 is on text. (SEO)
Chapter 11 revisits the ‘why’ of all this, in terms of is it worth doing and (chapter 12) does it tell us something we did not really suspect before? Leading to 13: a look at Business Strategy which, quite frankly, is where I would have started if I had written this book.
A cracking book and I cannot recommend it too highly. The information they impart borders on the commercially competitive.
Warning: you do need to have retained your high-school level grasp of maths notation to absorb the contents.