OpenCV + Java in Processing

I love Processing. I love OpenCV too. I do not generally love using them together too much: weird wrappers, spotty support, weird linking errors, etc. That, however, has changed because OpenCV 2.4.4 began supporting Java builds, which means that you can use OpenCV in your Processing sketch sans wrapper, sans 3rd party library. You simply get OpenCV installed, create a little library for it, drop in the main .jar and .dylib that the build process creates, and off you go. In the spirit of making that easy, I made a small Processing library that facilitates going between Java and OpenCV and I’d like to explain how you can use it.

OpenCV + Java

First things first, you need to go to the ocvP5 github repo, download it and drop it into the libraries folder of your Processing main sketch folder ( this is probably something like home/Documents/Processing ). Once that’s done, you need to get OpenCV installed. Let’s walk through a few ways to do this:

1) OSX + MacPorts OpenCV

OSX people, you do have MacPorts installed, right? If not, remedy that. Once you’re ready with MacPorts, install OpenCV:

sudo port install opencv +java

The +java is important to make sure that the Java are built. Now, figure out where the Java is:

port contents opencv | grep java

These are the two files that you should copy to your OpenCV installation location:

/opt/local/share/OpenCV/java/* $PROCESSING_HOME/libraries/ocvP5/library

Now you’re ready to go.

2) Build your own OpenCV

This is what you can do if you’re using Linux, Windows, or don’t want to use MacPorts. I’m going to gloss over it a bit because it’s well documented elsewhere on the OpenCV site. First, there’s a checklist of things you need before you build:

Python 2.6 or higher

All of these are installable with a package manager on Linux. On Windows, go ahead and download and run installers. Now you’re ready. Get a terminal and run:

git clone git://
cd opencv
git checkout 2.4
mkdir build
cd build

On Linux/OSX you want to generate a Makefile:


or on Windows a MS Visual Studio* solution

cmake -DBUILD_SHARED_LIBS=OFF -G "Visual Studio 10" ..

Note When OpenCV is built as a set of static libraries (-DBUILD_SHARED_LIBS=OFF option) the Java bindings dynamic library is all-sufficient, i.e. doesn’t depend on other OpenCV libs, but includes all the OpenCV code inside.
Examine the output of CMake and ensure java is one of the modules “To be built”. If not, it’s likely you’re missing a dependency. You should troubleshoot by looking through the CMake output for any Java-related tools that aren’t found and installing them. If CMake can’t find Java in your system set the JAVA_HOME environment variable with the path to installed JDK before running it. E.g.:

export JAVA_HOME=/usr/lib/jvm/java-6-oracle

Now start the build, on Linux/OSX:


or MS:

msbuild /m OpenCV.sln /t:Build /p:Configuration=Release /v:m

Besides all this will create a jar containing the Java interface (bin/opencv-244.jar) and a native dynamic library containing Java bindings and all the OpenCV stuff (lib/ or bin/Release/opencv_java244.dll respectively). Now you simply need to copy opencv-245.jar from build/bin and libopencv_java245.dylib from build/lib into Processing/libraries/ocvP5/library and you’re good to go.

Edit thanks to Kasper Kamperman

On Windows, if you run into some of the issues described here you can check out the alternate way of installing OpenCV on your Windows machine that he describes in the comments [here](


This is really a very simple library that I’m beginning to put together. It’s not going to do much more than help you work with OpenCV code directly in Java and take advantage of the wonderful environment and library set that Processing offers. In many ways it’s very similar to the OpenCV block for Cinder or the ofxCv addon for openFrameworks: only what you need to get the two libraries talking to one another and nothing more. Let’s look at an example to see how this works:

Here’s a classic cascade based face detection example:

import ocv.*;

import org.opencv.core.*;
import org.opencv.calib3d.*;
import org.opencv.contrib.*;
import org.opencv.objdetect.*;
import org.opencv.imgproc.*;
import org.opencv.utils.*;
import org.opencv.features2d.*;
import org.opencv.highgui.*;

import java.util.Vector;

PImage pimg;
Capture cam;
ocvP5 ocv;
CascadeClassifier classifier;

ArrayList<rect> faceRects;

void setup()
  // This is what you'll load if you're loading from MacPorts, otherwise this should be
  // wherever you built the OpenCV libraries
  System.load(new File("/opt/local/share/OpenCV/java/libopencv_java245.dylib").getAbsolutePath());

  // make an ocvP5 object to convert file types (this will do a teeny bit more in the future)
  ocv = new ocvP5(this);  
  size(640, 480);

  String[] cameras = Capture.list();
  cam = new Capture(this, cameras[0]);
  // initialize with the classic face detection cascade file:
  classifier = new CascadeClassifier(dataPath("haarcascade_frontalface_default.xml"));
  faceRects = new ArrayList(); 

void draw() 
  // if there's a new frame in the camera:
  if (cam.available() == true) 
    // get a PImage from the camera
    pimg = cam;
    // convert to OpenCV
    Mat m = ocv.toCV(pimg);
    // we want a grayscale image
    Mat gray = new Mat(m.rows(), m.cols(), CvType.CV_8U);
    Imgproc.cvtColor(m, gray, Imgproc.COLOR_BGRA2GRAY);

    MatOfRect objects = new MatOfRect();

    Size minSize = new Size(150, 150);
    Size maxSize = new Size(300, 300);

    // do the actual conversion, more info<a href=""> here</a>
    classifier.detectMultiScale(gray, objects, 1.1, 3, Objdetect.CASCADE_DO_CANNY_PRUNING | Objdetect.CASCADE_DO_ROUGH_SEARCH, minSize, maxSize);


    // add all the rects to the faceRects array
    for (Rect rect: objects.toArray()) {
      faceRects.add(new Rect(rect.x, rect.y, rect.width, rect.height));

  // draw the image
  image(cam, 0, 0);

  // now draw the detected face regions
  for (int i = 0; i < faceRects.size(); i++) {
    rect(faceRects.get(i).x, faceRects.get(i).y, faceRects.get(i).width, faceRects.get(i).height);

If you’ve worked with raw OpenCV in C++ or Python, this should all look pretty familiar to you. If not, it might look very different from what you’re used to seeing. Most Processing libraries for computer vision work tend towards a few simple carefully wrapped methods. I’m trying to keep this library oriented towards the very minimal, so the only methods I’ll be creating are simple conversion methods to change from Processing to OpenCV and back. Anything else you do can be done in straight-up OpenCV code, like the calls to detectMultiScale() you see in this code. That’s normally wrapped in a “detectFaces()” type method, but that’s a little deceiving because detectMultiScale() can be used to detect anything: faces, eyes, hands, stop signs, airplanes, the list is almost limitless. Exposing a little more of that means that you can start to experiment and explore and that’s a good thing. This might seem tricky at first, but it means that your applications can more easily be modified and extended to do literally anything that OpenCV can do, and porting it to another platform is almost painless.

That’s all for the moment, but I’ll be building out ocvP5 more over the coming weeks, so keep an eye out if you’re interested.

7 Nascent Thoughts on Critical Artwork

Law is often an indequate discourse to publicly and intelligibly express the difficulties of legislating online space, digital privacy, surveillance, and the agency of our creations.

Art is potentially an adequate discourse to do these things.

Critical Design is an adequate discourse to do these things.

We require tactics to query legislation and norms around this politics, in an intelligible, public, and open way. In a way which isn’t couched in rhetoric. Not a strategy, not a critical stance, not in art galleries.

The erosion of public space, laws and practices curtailing individuals ability to document police action, are all of vital importance to expression, communication, socialization, and fundamentally “human-ness”.

We cannot adequately challenge a law without a legal case. The most expedient path to a challenge of a law is for someone to break it.

This is of course, bold, almost a crazed thing to ask of an artist. No one wants to go to jail, face increased surveillance, TSA pat-downs, any of the other potentially life-altering difficulties that a criminal record leads to. Yet I think that art is perhaps the only arena in which a discussion of these issues can take place. It is outside of the realm of direct commerce (by happenstance). It requires and is often granted the permission to be purely provocative and critical. It is one of the last vestiges of pure exploratory rhetoric. For that reason, it’s worth asking what questions a critical art can ask that any other type of investigation cannot ask.


[Design can be …] The stable platform on which to entertain unusual bedfellows. The glue for things that may not be naturally sticky. The lubricant that allows movement between ideas that don’t quite run together. The medium through which we can make otherwise awkward connections and comparisons. The language for tricky conversations and translations.

— Dr Ken Arnold, Head of Public Programs, The Wellcome Trust, London.

Thinking about OOO

I have read Ian Bogost and I’ve begun reading Graham Harman, and it’s becoming entirely clear to me why new media artists are so enamored with their ideas. First, Graham Harman is hard reading if you’re not comfortable with Husserl and Heidegger, but his core concepts, the necessity of considering objects and systems, is undeniably compelling. It makes perfect sense that the populist approach to OO would be taken by a video game developer. To people who make who things which really do go on to do weird unexpected things in the world the idea of a world really being filled with things which are very much independent of our selves and attitudes towards them seems very commonsense. When we can model some of the behavior of wildly disparate things like a magnet, DNA, fiber optics, and potentially lightning, all with the same fundamental mathematical concept then why not posit that things, objects really, are the superset of us and experiences, rather than vice versa?

It get particularly acute when one thinks about the things that we actually create. We can say “what do words do when they’re not being spoken?” but that’s essentially poetic nonsense. A video game however, actually might do things when it’s not being played, it has a relationship to its creator and to its player, but it also is perfectly capable of doing things that we may or may not understand at the time or at any time. It seems to have agency and increasingly computational artists are investigating that agency, the structures that enable that agency, and how we perceive that agency and structure in our own terms. We’re trying make sense of systems around us in strange ways because systems around us are irrupting into our lives in strange ways. As Clement Valla writes on glitches in Google maps:

these images are not glitches. They are the absolute logical result of the system. They are an edge condition

However, going a step beyond that, making things as a condition of thinking seems a natural response to living in a place and set of systems which we have given agency and which we can inspect at a remove, in a strange way. These are our creations, but they are, taken in sum, acting far beyond us, and when we see their processes, it is in a translated artifact. Screenshots, videos, do not accurately represent the state of a system, but they can stand in for it, they can pique our imagination. Aestheticizing these images and this artwork does as much to help us actually comprehend a system as say a Marinetti painting helped Europeans understand the burgeoning military-industrial complexes of late 19th and early 20th century Europe: it doesn’t in the strict sense. That’s an interesting place for a programmer, an engineer of sorts, to be: creating and exploring impressionistic artifacts. It might be though, that it’s the most appropriate place for an engineer/artist to be:

“the human/world relation is just a special case of the relation between any two entities whatsoever.” – Quadruple Object

That’s a great quote, particularly appealing to a programmer: my program interfacing with a user is not entirely dissimilar from my program interfacing with another program. We live in a world where despite our best efforts the motivations and inner workings of others remain rather mysterious, worthy of aestheticization, poetry, and wonder. We also increasingly surrounded by human-scale actors and the artifacts of their actions, in that “we” live in cities, we live quite connected, we live quite instantaneously, and we can either by ourselves or with proxies capture a remarkable amount of the world as it appears to us. We have odd strategies for comprehending this, for example:

“All information is grounds for knowledge, whether empirical or aphoristic, no matter its truth-value” – Meta-Modernism

Which sounds to me an implicit recognition that incoherence, error, invisibility, might just be temporary states of our particular relation to something. Our tools and systems might be able to make more sense of it than we can, having, unlike us, infinite attention spans and no need for sleep or sitcoms. Taking that as the departure point for aesthetic investigation, an active and inquisitive investigation that is conducted through making machines and tools which will form their own opinions on things and which we can query in whimsical or critical ways, seems to me, the most coherent aesthetic I can imagine.

7 quotes germane to Object Oriented Ontology

Ian Bogost

“Ontology is the philosophical study of existence. Object-oriented ontology (“OOO” for short) puts things at the center of this study. Its proponents contend that nothing has special status, but that everything exists equally—plumbers, DVD players, cotton, bonobos, sandstone, and Harry Potter, for example. In particular, OOO rejects the claims that human experience rests at the center of philosophy, and that things can be understood by how they appear to us. In place of science alone, OOO uses speculation to characterize how objects exist and interact.”

Ludwig Wittgenstein

“What is the meaning of the word ‘five’? No such thing is in question here, only how the word ‘five’ is used.”

CS Pierce

“Consider what effects which might conceivably have practical bearings we conceive the object of our conception to have. Then, our conception of these effects is the whole of our conception of the object.”

Paul Churchland

“Your brain is far too complex and mercurial for its behavior to be predicted in any but the broadest outlines or for any but the shortest distances in the future.”

Alain Badiou

“I think multimedia is a false idea because it’s the power of absolute integration and it’s something like the projection in art of the dream of globalization. It’s a question of the unity of art like the unity of the world but it’s an abstraction too. So, we need to create new art, certainly new forms, but not with the dream of a totalization of all the forms of sensibility.”

Bruno Latour

“…scientific and technical work is made invisible by its own success. When a machine runs efficiently, when a matter of fact is settled, one need focus only on its inputs and outputs and not on its internal complexity. Thus, paradoxically, the more science and technology succeed, the more opaque and obscure they become”

Thomas Kuhn

“Contrary to Latours oft-repeated claim that politics has never taken technology seriously, totalitarian regimes stand out from traditional forms of authoritarianism precisely by the role assigned to technology as the medium through which citizens are turned into docile subjects, specifically parts of a corporate whole. While attention has usually focused on totalitarian investments in military technology, of more lasting import have been totalitarian initiatives in the more day-to-day technologies associated with communication”

World Map for data viz

All elements are tagged with nice labels, nations and US states. US states are a bit messy as I did them by hand, but the boundaries are approximately right

I wanted this for a while, mostly for doing “where are people” visualizations, but never could find a nice SVG file that had all of this in it. Now I do. Download it here