VBI Programming in Java


This book is about programming novel computer interfaces by capturing images from a PC's webcam. The idea is to augment (perhaps even replace) the familiar keyboard and mouse with input derived from pictures of the user's movements, facial features, hand and finger gestures, and visual tags such as barcodes. In one picture:

Summary of VBI Programming in Java

Vision-based user interfaces (VBIs) have been a hot research topic for decades, but only in the last five years or so has the hardware and software become cheap enough, fast enough, and feature-rich enough for the technology to also be practical. For instance, it's a rare computing device that doesn't now come with a color mega-pixel camera, capable of recording 30 images (frames) per second.

The software side of VBIs is all about implementing sophisticated computer vision algorithms that are fast enough for (near) real-time processing of images. That problem was addressed to a great extent by the release of version 1 the OpenCV computer vision library (http://opencv.org/) in 2006. Subsequent releases have seen the library ported to a variety of machines, programming languages, and OSes (Windows, Linux, OS X, Android, iOS, and more).

Specifically, this book is about how to write VBI programs using a Java binding of OpenCV called JavaCV (https://code.google.com/p/javacv/) utilizing an ordinary laptop-based webcam. The chapters are divided into seven parts:

Early (sometimes very early) draft versions of the book's chapters can be downloaded from here (see the links below). I'll also be adding new chapters here; chapters which don't appear in the book.

If you're looking for Killer Game Programming in Java then it's here.


What this Book is not About

It's quite useful to list things which this book does not do.

  1. This book is not an introduction to Java. I'm going to assume that you've already done an introductory course on Java (or something similar), and so understand about classes, objects, inheritance, exception handling, basic threads and graphics. A good introductory textbook on Java is Thinking in Java, by Bruce Eckel. It's won awards, and can be downloaded for free from http://www.mindview.net/Books/TIJ/. However, I will be explaining more advanced stuff such as Java Sound, networking, and Java 3D.
  2. This book is not a theoretical introduction to computer vision or human computer interfaces. It's driven by programming examples focused on specific vision-based user interface problems. Of course, when a particular technique (e.g. eigenfaces) comes up, I explain it, but without relying on heavy-duty mathematics.
    There are many excellent academic texts on computer vision. One that I've found useful is:
    Richard Szeliski, Computer Vision: Algorithms and Applications, Springer, 2010; http://szeliski.org/Book/
  3. This book doesn't cover every aspect of OpenCV, which is enormous, and getting bigger by the moment. I cover a lot of topics, but there's always more things to learn. Many times during this book, I'll refer to the OpenCV website (http://opencv.org/) for more information, and to the standard text on OpenCV:
    Gary Bradski and Adrian Kaehler, Learning OpenCV: Computer Vision with the OpenCV Library, O'Reilly Media, 2008; http://shop.oreilly.com/product/9780596516130.do
  4. This isn't a book about Android programming. All my code utilizes a laptop/PC webcam running on MS Windows (tested on XP and version 7, but I couldn't face installing Windows 8 ☺). However, OpenCV, and most of the other libraries I use, are supported across multiple platforms, such as Mac OS X, Linux, and Android. In all cases, I use the Java bindings of the libraries, but most have multiple programming language interfaces.
  5. You won't find page after page of code that you have to type in. All the code examples are available online, accessible from this page. I only discuss the code that contains interesting computer vision algorithms, or more advanced Java programming (e.g. fancy uses of concurrency).

[Hals PIC]

Software Used

Almost all of my examples utilize OpenCV 2.4.5 and JavaCV 0.5. For more details, please read the "readme" file including in each chapter's download.

Both OpenCV and JavaCV have changed many times since I wrote my examples, and unfortunately they will no longer compile under the latest versions of those libraries.

Probably the easiest thing is to download the old versions of OpenCV and JavaCV.

The last version of OpenCV which I know to work is OpenCV 2.4.9, which you can download from http://sourceforge.net/projects/opencvlibrary/files/opencv-win/2.4.9/. This is the version for MS Windows. If you want to try a more recent version, start on the page http://sourceforge.net/projects/opencvlibrary/files/opencv-win/.

The last version of JavaCV which works is JavaCV 0.5 which you can download from http://code.google.com/p/javacv/downloads/detail?name=javacv-0.5-bin.zip&can=4&q=. Alternatively, you can go to the JavaCV download page at http://code.google.com/p/javacv/downloads, and search for "Deprecated Downloads". Many old versions of JavaCV are listed, including the binary, source and examples for v0.5.

Chapters    Last updated 20th November 2013:
This page appeared online.

Dr. Andrew Davison
E-mail: ad@fivedots.coe.psu.ac.th
Back to my home page