I came, I saw, I took a photograph

I came, I saw, I took a photograph. The NASA rover Curiosity is filming Mars. Computer Vision looks like the next big trend.

Kurt Demaagd, Anthony Oliver, Nathan Oostendorp and Katherine Scott have produced ‘Practical Computer Vision with SimpleCV’, The Simple Way to Make Technology See, published by O’Reilly.

Disclaimer: O’Reilly gave me a free copy of the ebook to review. I shall definitely be keeping it. The book is really simply written but it provoked my imagination in an area of technology that right now is wide open.

What is Computer Vision?

Computer Vision is using cameras and computers to take pictures and analyze things. Consider the more sophisticated image processes of GIMP, Photoshop and PaintShop: edge detection, masking, layering, manipulating. Apply to the moving images of a camera. Combine with other techniques that we know the eye and brain use, apply some (artificial) intelligence. Grail: Perception itself.

What is the difference between SimpleCV and OpenCV?

OpenCV is the collection of algorithms and implementations, mostly written in C, that represent our
collective knowledge on Computer Vision so far. Originally curated by Intel, now open source because we can all be stakeholders in this and the algorithms are simply too precious and too important not to share.

SimpleCV is a high-level framework or wrapper built on top of OpenCV, written in Python. OpenCV is compiled, complicated and ever-changing. SimpleCV is interpreted, simple, and lags slightly behind the state-of-the-art but is nevertheless an excellent learning. experimenting and prototyping tool.

My only concern is that SimpleCV is not event-driven. Vision works by catching something at the edge of the
eye then swivelling focus to study it. Proper emulation of that needs a system of interrupts.

The level of the book?

This book uses SimpleCV, not OpenCV, and it is pitched at a really basic level. Until chapter 10 the authors
cover nothing but building blocks. But by then the reader has a good understanding of what the python
scripting can do. There are illustrated applications that struck me as simultaneously clever examples of
image interpretation but also clear demonstrations of how mechanistic computer algorithms fall far short of
what the living eye can do. It is only in the appendices that the book hints at what you might be able to
achieve with CV.

Is the website a resource of the book or is the book a resource of the website?

I read the book first, then turned to the website second. The full Python and SimpleCV development stack
environment is a 120MG download; it all played nicely for me and within a couple of hours I had a grainy
photo of myself on the webcam up and running and analysed in various ways for features.

Others will come to the SimpleCV framework first, then buy the book as a manual and a way to provide the
authors some small reward for their intense contributions in this open source environment.

What is the current state of the art?

O’Reilly’s last book on this was ‘OpenCv’ by Gary Bradski and Adrian Kaehler, in 2008. Developments over the last 4 years include: accurate speed recording and number plate recognition in traffic control, facebook recognition in social media, smile detection on cameras, image combining in Google Earth.

If they publish again after another 4 years, you will see machine-driven cars, ambient Glasses/Goggles informers. There is also room for progress in areas that we now understand quite well in terms of the eye and the brain. Like: responding faster to movement on the edge of vision, picking out the bright red in a scene, selectively applying fast fourier transformations, adapting to darkness levels and still seeing the same things.

Where next?

I urge you to buy this book. Advance the science. Climb on the bandwagon. Think of it as a rover called Curiosity.



Microsoft® SQL Server® 2012 T-SQL Fundamentals by Itzik Ben-Gan

Last week I agreed to review Microsoft® SQL Server® 2012 T-SQL Fundamentals by Itzik Ben-Gan, for O’Reilly.

Is it a brave move to produce a book on MS SQL in these times? The thought-leaders are all about big data on the network and sqlLite on the device. Or is now the time of the noSql backlash?

Ben-Gan kicks off with a roundup of set theory, the way sql is non-procedural, and even the famous early days of hybrid industry / academics Codd and Date and ‘third normal form’. Interesting, evocative, and possibly quite useful as background to the way the sql programmer must think.

Then straight into the commands, their application, variance between sql versions, and various gotchas. Whether you find this more or less useful than the online SQL books at msdn.com, probably depends on your preferred learning mode. There are even excercises after each chapter, like in a school book. But I cannot imagine many people would go to the trouble of learning SQL fundamentals without real-world use cases of their own.

His descriptions are absolutely accurate, to my knowledge. The only single challenge I would make, based on my many years of experience with Microsoft SQL server, is to his assertion that online transactional processing and datawarehousing are best kept separate. In fact the superb query optimisation and parallel access handling of MS SQL server has made that untrue long ago.

The debugging tips puzzled me at first: how can a set-based data manipulation language have bugs? Then I remembered my first few long-ago struggles with some of the more complex ways of joining data sets. The issues would almost always revolve around the use of ‘null’ and its non-intuitive behaviour in logical expressions. If Ben-Gan’s advice saves you even a few hours of surfing to resolve such questions, you will have paid for the book.

There is even a section on set up and installation but not back-up strategies. My advice: the database server is SO important that you should either plan everything bottom up round your backup and recovery plans – or keep out of this side altogether and get a provider to take responsibility.

He also runs over locking, deadlocks, trouble shooting and isolation levels. But not the real-life key factors in all this, which is the habits and interoperability of different applications that will be accessing the sql server.

SQL server is a work-horse: far superior to mySql in my opinion and still well worth basing your enterprise-grade applications around. I have been running a business 24/7 on an SQL server non-stop for well over a decade under constant access from websites, internal applications and, yes, even entry-level programmers using fundamental T-SQL. Definitely here to stay for some years yet and a technology you should know.

The Data Journalism Handbook By Jonathan Gray, Lucy Chambers, Liliana Bounegru

2012 may well be the year of open news, claims Alastair Dant, of The Guardian. Well yes. Actually news itself has been open for some time now, it is more a question of news and the press.

As a UK ex-pat and ex-newspaper reader I have been watching the News International and Leveson inquiry unfold over Twitter and wondering: why? Few of us read these papers anymore anyway? So I did two things:

  • I flew to the UK to purchase my very own printed newspaper (  the Times )
  • I took up O’Reilly’s offer to review a pre-publication copy of ‘The Data Journalism Handbook’ by Jonathan Gray, Liliana Bounegru, and Lucy Chambers, in exchange for a free copy ( the eBook format, naturally)

On the print side, I am happy to report the Times has hardly changed over the years. Same columnists, same demographic, I imagine all cosily growing old together for as long as the shrinking circulation will support the model.

On the data journalism side, I am happy to report a more youthful approach. This book is full of ideas, initiatives, and enthusiasm.

To an IT professional or a seasoned web worker, some of the journalists’ efforts are hugely amateur. The struggles of picking data out by scraping unstructured web pages remind me of the 1990s before we decided we needed xml and all that followed. The problem of authorities claiming to release data but then using the pdf format comes up time and time again.

But these journalists are honest and they face a responsibility: who will curate the news, who will distribute, verify and interpret the news, who will ultimately safeguard information and therefore freedom in the digital age?

The idea for the ‘Handbook’ came out of a 48 hour workshop at MozFest 2011 in London. The contributors list includes people from such well-known organisations in the field as

  • Open Knowledge Foundation
  • Creative Commons
  • BBC
  • The UK Guardian
  • and even my own local news curator here in the Netherlands, nu.nl.

The story is really of how the profession of Journalism went digital: the opportunities, the challenges, the dead-ends, the false-starts, and above all the unfolding future.

For a journalist, data journalism is the way forward: on all fronts the internet has killed their traditional business, but in this one way they can use their traditional skills to add value and stay relevant to the rest of us.

For a data worker, ie an IT professional, data journalism is the field of techniques to communicate the information within data: ranging from visual tricks through statistical analysis through to understanding accuracy, immediacy and provenance.

In other words, the book is about turning data into meaning.

There are some great stories in the ‘Data Journalism Handbook’. Some to whet your appetite:

  • How Florence Nightingale used infographics to key effect in transforming healthcare
  • The analysis of wikileaks data from USA overseas wars
  • The early (2007) effort of publishing government financial data as COINS (Combined Online Information System)

Journalists, read this book to keep you in touch with what your peers are doing and how they have achieved it, even down to listing working environments, personality and skills profiles, methodologies, tools, languages and data sources that have worked for other people.

Government workers, read this book and stop using pdf!

Programmers, read this book and release the pdf structures

Web publishers, read this book and release more data.  Duncan Geere, of wired.co.uk, quoted in the book:

‘The most important thing you can do with your data is share it as widely and openly as possible.’

The Information Diet, by Clay A. Johnson

The Information Diet. By Clay A. Johnson
A Case for Conscious Consumption
By Clay A. Johnson
Publisher: O’Reilly Media
Released: December 2011

Perfect timing, to publish this book for 2012. like many people I know, I reckoned I had my web news diet down to a fine art. I follow people I know, trust and agree with, and then just read the news they alert me to. Sorted. But then a recent trip to the wilds of Scotland, outside the reach of the web, caused me to a) buy a printed newspaper and b) introspect for a few hours away from the net. In my mind I crystalised the thought that has been nagging me for a while: the news I read is the news I like. I am missing the challenge of different mindsets, the provocation of opposing views, and the mind expansion that comes from reading content I had not explicitly sought, as happens with printed newspapers.
Clay Johnson tackles these issues by drawing the parallel with modern day (specifically American) food consumption and obesity. A novel insight from an information scientist? Or a useful attempt to simplify complex issues by describing them in a way we can all understand?
Clay achieves both, and more. In fact the early part of the book told me more that I expected to learn about modern agriculture and american attitudes to food. But he is in fact from a political background.The insights of this book stem from his experience in a world where information is regularly used to achieve an end, or in circumstances where more than one competing interpretation of the facts is common.
The political discussions in the book yield one or two barnstorming relevations:
  • did you know the term ‘intellectual’ originated as an insult?
  • did you know the Murdoch businesses, and specifically Fox News, actually have a quantifiable formula that explains their superior profit from news compared to their rivals?
The psychological discussions in the book are a little more shaky, with emphasis on selected studies and theories of the ‘the shape of my brain reflects the words that I read’ variety. perhaps selected because they provide the instant gratification sought by the foodie. But there again, Johnson is not primarily a psychologist nor an information scientist.
The final parts of the book recommend some favorite news sites ( always a topical risk in a book ) and some strategies for consuming information. One of which is to seek out not just the information you already know. Good advice. My tip to Johnson would be to seek out the writings of Claue Shannon from 1948 onwards, and look at the subsequent ‘Information Theory’ science that emerged. It covers these issues of information and communication quality, entropy, predictability, and potential to surprise in a way that would nicely complement the angles of this book.
Love the open letter section, headed ‘Dear Programmer’.
Three stars; for getting us all thinking about these issues.

Architecting Mobile Solutions for the Enterprise, by Dino Esposito

Cover shotIt is spring 2012.

  • The telecomm companies are offering data plans that have hammered their call and sms revenues
  • Microsoft are open-sourcing many of the development tools that traditionally delivered their licence revenues
  • iTunes and App stores have brought the supply-to-demand balance to the point where nearly every app is free
  • The public in many countries have deserted websites for mobile completely, and none more so than in the developing economies.

The tipping point is reached.

Now we need to revisit all our enterprise solutions, and anyone with a serious corporate or personal investment in legacy architecture frameworks needs to think hard about the future.

Dino Esposito is one of these people ( he comes from a Microsoft .Net background ) and he has written ‘Architecting Mobile Solutions for the Enterprise’ from real-world experience and with an eye on all our futures.

The first 2 parts of the book cover mobile strategy and architecture. Familiar issues for those of us who are already following the path of migrating mature enterprises to mobile. Dino helpfully recognises that readers will be at different places on the path and suggests a few ways to navigate both his book, and the present architecture of the mobile scene.

Then the third and final part of the book moves from general architectural discussions to specific code examples; and here it really shines. Dino has either taught himself or worked with others who can code in all the major platforms. This is really impressive: full of patterns and tips and gotchas that really help you see the challenges you will face with the present fragmented mobile architecture.

Nice work, and truly educational. Covers mainly Apple, Microsoft and Android. Not much on Blackberry RIM or the other minor players, but at the moment those three cover most of the growth in mobile.

It is truely unusual to see .Net, Silverlight, Objective-C, Android SDK, Monotouch, PhoneGap, HTML5, jQuery mobile covered to such a degree in one place.

If you are unaware of how to architect mobile solutions for the enterprise ( perhaps too busy maintaining legacy enterprise solutions? ) then this book is a must-read for you.

If you can work in one platform but uncertain how to cover the others, this book is a must-read for you.

If you think you can port to mobile just by putting an ‘m.’ in front of your web pages, you also need this book.

My favorite mobile interaction pattern from the book: ‘Guess-Dont-Ask’.

My favorite Dino Esposito quote: “You might want to have a line similar to the following on top of all your
JavaScript files: var GLOBALS = GLOBALS || {};”. Poetry.

Read it while it lasts; some of these architectures are looking very fluid and lets all hope that the solutions will have advanced enough for the 2013 edition to cover a more evolved landscape.

Working with Microsoft Office 365: Brett Hill

This is a highly relevant and topical subject.

I have been looking at how different business units can take advantage of open source,   hosting, and software as a service. So when O’Reilly offered me this book to review I jumped at the chance.

Why, firstly, would you choose Office 365?

Microsoft deal in in-house, licensed and proprietary productivity systems. So this product treads a fine line between offering hosted services and limiting those services to protect their licence income from related products.
You might want to stay with Microsoft if you have a previous investment in Office files, or you value employee understanding of the office formats. Brett makes a good job of explaining the features present and not present in Office 365, and how they compare with the various Microsoft licensing models.

Brett is out of the Microsoft stable; he does not make direct comparisons with competing solutions, apart from occasional references to the superior image of MS-hosted businesses.

If MS Office 365 is for you – and it is worth buying the book just to assist in making that decision – the next question is: How to set it up.

Brett’s experience shows here; he navigates us through the different options, even making honest comparisons of subscription prices with licence pricing. The book helps you match the size of the Office 365 set-up to the size and type of your business; choose the best configurations, and then set up the administration.

  • Buy this book if your business is an existing MS user and you are evaluating cloud or SaaS options. But probably not if you are looking for open source, crm or collaborative software systems.
  • Buy this book if you want to judge the most cost-effective way to move forward with MS Office; but probably not if your requirements go further than office productivity and extend to hosting a commercial web-site or business systems.
  • Buy this book if you are newly tasked with setting up or administering your Office 365 implementation; but probably not so useful if you are already an experienced Office or Sharepoint administrator.

Android Cookbook by Ian F. Darwin; O’Reilly Media

This book is crowd-sourced. Ian Darwin has assembled recipes from himself and over 30 other contributors to build this book.

I like cookbooks. They sit somewhere between official guides and subject-expert web writing. They represent the team effort of co-operative mutual learning as we all share advice and advance the science. Even if you don’t follow the recipe, they stimulate the taste buds and the imagination.

I read this book, all 661 pages, in a single busy day. Downloaded it over breakfast, devoured it through lunch, raced through the recipes over dinner, and finished it off in bed. Can’t remember the last time a book captivated me that much.

So, is ‘Android Cookbook’ for you?

  • Yes, if you need to know what is possible in app development for phones and tablets
  • Yes, if you are an experienced app developer but need some quick answers on new features and sensors
  • Yes, if you a cross-over developer moving from another platform. Maybe one of the many RIM developers watching your user base evaporate right now.
  • Yes, if you are new to professional development but already versed in the Linux, Eclipse, Java stack. Maybe coming out of college.
  • No, if you want to learn how to design attractive user interfaces, or you are looking for productivity advice on using your android. This cookbook is strictly by and for programmers and the screenshots are not pretty and certainly do not follow the rounded-corners philosophy of elegant visual design. But yours can! Follow recipes 6.10 and 6.11. and 7.28

Android Cookbook is thought-provoking. The problem/solution format tends to concentrate on specific tips that are intriguing on their own but hardly of value as stand-alone apps. This has the effect of making the knowledge easy to absorb, and leading the mind to envisage mash-ups of the various tips to create real and useful apps. So for example you could combine recipe 4.9 (Starting a Service After Device Reboot) with 15.3 (Loading a User’s Twitter Timeline Using JSON) and 9.4 (Creating an Advanced ListView with Images and Text) to create a fully functional Twitter client.

Android Cookbook is factually correct. I fact-checked the things I did not already know and they look right.

Android Cookbook is fast moving. The story leaps from solution to solution; each growing on the next. But it is inevitable that there are some time lapses in the solutions and a little overlap too. Recipes 6.5 and 7.2 date from 2010, while 22.2 and 22.3 are already obsolete, yet 22.4 comments on Android Market being folded into Google Play just as the book was going to press. Is it just me or is recipe 8.11 ( Creating a Reusable About Box Class, from Daniel Fowler ) and recipe 19.3 ( Obtaining Information from the Manifest File , from Colin Wilcox ) exactly the same? Buy the book and tell me if you agree.

Android Cookbook is scary in parts. At one point Ian Darwin even shows us how to embed C inside your app and potentially destroy any semblance of robustness you thought you had from the legendary Java sandbox. I was thankful he refrained from showing us a genuine use for bare-metal coding. Instead he filled the gap with a traditional algorithm; in this case how to compute a square root. Iin this case I think Ian is being a little school-masterly. Do not try this tip at home. The correct way to calculate a square root in an Android app is with Math.sqrt(x), not to use the Newton-Raphson iterative method that Ian demonstrates.

Ian likes the dry style first made popular by Kernighan and Ritchie in their 1978 White Book, of which I guess he is clearly fan. Acres of code, seasoned with a few choice wizened peppercorns all the tastier for their sparseness.

Do I miss anything in this book? Well, Ian, when you bring out the next edition can you explain more about:

  • The microphone. How can we integrate sound recording and build our own Shazam?
  • Image processing. How can we turn our cameras into Google streetviews? Is a phone (yet) capable of OCR?
  • Music making.  Lets make music, synthesize sounds, build with Midi
  • Graphics. help us construct our own charts and diagrams
  • Calendar events. Some one in the crowd must know more about integrating with the on-phone calendar.

Is it worth you buying the Android Cookbook?

  • Yes, if you are buying out of your corporate budget this will pay for itself in saved labour within hours.
  • Yes, if you are paying from your own pocket. I wish I had known tip 11.12 before I wrote my last app.
  • Maybe not yet, if you are still to become comfortable with Eclipse and Java
  • Yes, if you are an entrepreneur visualising the future.

Buy and enjoy. Android Cookbook by Ian F. Darwin; O’Reilly Media