OpenCV + Java in Processing

I love Processing. I love OpenCV too. I do not generally love using them together too much: weird wrappers, spotty support, weird linking errors, etc. That, however, has changed because OpenCV 2.4.4 began supporting Java builds, which means that you can use OpenCV in your Processing sketch sans wrapper, sans 3rd party library. You simply get OpenCV installed, create a little library for it, drop in the main .jar and .dylib that the build process creates, and off you go. In the spirit of making that easy, I made a small Processing library that facilitates going between Java and OpenCV and I’d like to explain how you can use it.

OpenCV + Java

First things first, you need to go to the ocvP5 github repo, download it and drop it into the libraries folder of your Processing main sketch folder ( this is probably something like home/Documents/Processing ). Once that’s done, you need to get OpenCV installed. Let’s walk through a few ways to do this:

1) OSX + MacPorts OpenCV

OSX people, you do have MacPorts installed, right? If not, remedy that. Once you’re ready with MacPorts, install OpenCV:

sudo port install opencv +java

The +java is important to make sure that the Java are built. Now, figure out where the Java is:

port contents opencv | grep java

These are the two files that you should copy to your OpenCV installation location:

/opt/local/share/OpenCV/java/* $PROCESSING_HOME/libraries/ocvP5/library

Now you’re ready to go.

2) Build your own OpenCV

This is what you can do if you’re using Linux, Windows, or don’t want to use MacPorts. I’m going to gloss over it a bit because it’s well documented elsewhere on the OpenCV site. First, there’s a checklist of things you need before you build:

The JDK
Python 2.6 or higher
ANT
CMake

All of these are installable with a package manager on Linux. On Windows, go ahead and download and run installers. Now you’re ready. Get a terminal and run:

git clone git://github.com/Itseez/opencv.git
cd opencv
git checkout 2.4
mkdir build
cd build

On Linux/OSX you want to generate a Makefile:

cmake -DBUILD_SHARED_LIBS=OFF ..

or on Windows a MS Visual Studio* solution

cmake -DBUILD_SHARED_LIBS=OFF -G "Visual Studio 10" ..

Note When OpenCV is built as a set of static libraries (-DBUILD_SHARED_LIBS=OFF option) the Java bindings dynamic library is all-sufficient, i.e. doesn’t depend on other OpenCV libs, but includes all the OpenCV code inside.
Examine the output of CMake and ensure java is one of the modules “To be built”. If not, it’s likely you’re missing a dependency. You should troubleshoot by looking through the CMake output for any Java-related tools that aren’t found and installing them. If CMake can’t find Java in your system set the JAVA_HOME environment variable with the path to installed JDK before running it. E.g.:

export JAVA_HOME=/usr/lib/jvm/java-6-oracle
cmake -DBUILD_SHARED_LIBS=OFF ..

Now start the build, on Linux/OSX:

make

or MS:

msbuild /m OpenCV.sln /t:Build /p:Configuration=Release /v:m

Besides all this will create a jar containing the Java interface (bin/opencv-244.jar) and a native dynamic library containing Java bindings and all the OpenCV stuff (lib/libopencv_java244.so or bin/Release/opencv_java244.dll respectively). Now you simply need to copy opencv-245.jar from build/bin and libopencv_java245.dylib from build/lib into Processing/libraries/ocvP5/library and you’re good to go.

Edit thanks to Kasper Kamperman

On Windows, if you run into some of the issues described here you can check out the alternate way of installing OpenCV on your Windows machine that he describes in the comments [here](http://thefactoryfactory.com/wordpress/?p=1093&cpage=1#comment-16875).

ocvP5

This is really a very simple library that I’m beginning to put together. It’s not going to do much more than help you work with OpenCV code directly in Java and take advantage of the wonderful environment and library set that Processing offers. In many ways it’s very similar to the OpenCV block for Cinder or the ofxCv addon for openFrameworks: only what you need to get the two libraries talking to one another and nothing more. Let’s look at an example to see how this works:

Here’s a classic cascade based face detection example:

import ocv.*;
import processing.video.*;

import org.opencv.video.*;
import org.opencv.core.*;
import org.opencv.calib3d.*;
import org.opencv.contrib.*;
import org.opencv.objdetect.*;
import org.opencv.imgproc.*;
import org.opencv.utils.*;
import org.opencv.features2d.*;
import org.opencv.highgui.*;
import org.opencv.ml.*;
import org.opencv.photo.*;

import java.util.Vector;

PImage pimg;
Capture cam;
ocvP5 ocv;
CascadeClassifier classifier;

ArrayList<rect> faceRects;

void setup()
{
  // This is what you'll load if you're loading from MacPorts, otherwise this should be
  // wherever you built the OpenCV libraries
  System.load(new File("/opt/local/share/OpenCV/java/libopencv_java245.dylib").getAbsolutePath());

  // make an ocvP5 object to convert file types (this will do a teeny bit more in the future)
  ocv = new ocvP5(this);  
  size(640, 480);

  String[] cameras = Capture.list();
  cam = new Capture(this, cameras[0]);
  cam.start();
  
  // initialize with the classic face detection cascade file:
  classifier = new CascadeClassifier(dataPath("haarcascade_frontalface_default.xml"));
  
  faceRects = new ArrayList(); 
  stroke(255);
  noFill();
}

void draw() 
{
  // if there's a new frame in the camera:
  if (cam.available() == true) 
  {
    cam.read();
    // get a PImage from the camera
    pimg = cam;
    // convert to OpenCV
    Mat m = ocv.toCV(pimg);
    // we want a grayscale image
    Mat gray = new Mat(m.rows(), m.cols(), CvType.CV_8U);
    Imgproc.cvtColor(m, gray, Imgproc.COLOR_BGRA2GRAY);

    MatOfRect objects = new MatOfRect();

    Size minSize = new Size(150, 150);
    Size maxSize = new Size(300, 300);

    // do the actual conversion, more info<a href="http://docs.opencv.org/modules/objdetect/doc/cascade_classification.html"> here</a>
    classifier.detectMultiScale(gray, objects, 1.1, 3, Objdetect.CASCADE_DO_CANNY_PRUNING | Objdetect.CASCADE_DO_ROUGH_SEARCH, minSize, maxSize);

    faceRects.clear();

    // add all the rects to the faceRects array
    for (Rect rect: objects.toArray()) {
      faceRects.add(new Rect(rect.x, rect.y, rect.width, rect.height));
    }
  }

  // draw the image
  image(cam, 0, 0);

  // now draw the detected face regions
  for (int i = 0; i < faceRects.size(); i++) {
    rect(faceRects.get(i).x, faceRects.get(i).y, faceRects.get(i).width, faceRects.get(i).height);
  }
}

If you’ve worked with raw OpenCV in C++ or Python, this should all look pretty familiar to you. If not, it might look very different from what you’re used to seeing. Most Processing libraries for computer vision work tend towards a few simple carefully wrapped methods. I’m trying to keep this library oriented towards the very minimal, so the only methods I’ll be creating are simple conversion methods to change from Processing to OpenCV and back. Anything else you do can be done in straight-up OpenCV code, like the calls to detectMultiScale() you see in this code. That’s normally wrapped in a “detectFaces()” type method, but that’s a little deceiving because detectMultiScale() can be used to detect anything: faces, eyes, hands, stop signs, airplanes, the list is almost limitless. Exposing a little more of that means that you can start to experiment and explore and that’s a good thing. This might seem tricky at first, but it means that your applications can more easily be modified and extended to do literally anything that OpenCV can do, and porting it to another platform is almost painless.

That’s all for the moment, but I’ll be building out ocvP5 more over the coming weeks, so keep an eye out if you’re interested.

11 thoughts on “OpenCV + Java in Processing

  1. Why is it so difficult to install OpenCV. Is there not a prebuild version, so I don’t have to go through the process of installing all those programs to build OpenCV? The original OpenCV 1 library was really easy to install (including OpenCV) with one installer. http://ubaa.net/shared/processing/opencv/

    Is something more user friendly not possible anymore? I’m trying to find my way, but get stuck on different routes and ways. You provide different steps then this one:
    http://codeanticode.wordpress.com/2011/11/21/opencv-2-in-processing/

    (I know the latter one uses JavaCv, but still OpenCv needs to be installed).

    Thank you for your library, but I’m just wondering and curious why installing OpenCV all needs to involve so much steps.

  2. Maybe some useful info to complete your tutorial. I’ve managed to get your example to work, but in a different way you provided (I think it’s easier). This applies to for Windows (tested on Windows7) users who have access to the pre-build libraries:

    1. https://code.google.com/p/javacv/wiki/Windows7AndOpenCV
    Install the 2 programs and opencv (probably the newer 2.4.6 will do as well).

    2. http://docs.opencv.org/doc/tutorials/introduction/windows_install/windows_install.html#windowssetpathandenviromentvariable

    Set environment variable and path.
    The download to the Path Editor didn’t work, but it’s available as direct download at CNET.

    3. Make a folder called code in your Processing sketch.

    Copy the following files:
    opencv-245.jar (you can find this in opencv/build/java)
    opencv_java245.dll (you can find this in opencv/build/java/x64

    I didn’t test it but if you work with the 32bit version of Processing, you probably should copy the .dll from the opencv/build/java/x32 directory.

    Instead of copying the opencv_java245.dll you could also add the path to that directory in your systems path (with the aforementioned Path Editor program).
    I tried this also for the .jar, but I didn’t succeed with that.

    Change this line in the code:
    System.load(new File(“/opt/local/share/OpenCV/java/libopencv_java245.dylib”).getAbsolutePath());

    to:
    System.loadLibrary(Core.NATIVE_LIBRARY_NAME);
    (Thanks to the example at http://www.magicandlove.com/blog/2013/04/05/the-new-face-detection-and-processing/)

    ====
    Someone else is working as well with the OpenCV java bindings, this now only seems to work with OSX. I just downloaded and tested it and it seems to work directly out of the box (OSX 10.7 – Processing 2.0.1 64 bit):
    https://github.com/atduskgreg/OpenCVPro

    Add also the path: C:\opencv]build\java\x64

  3. I found out that when using the method above and I exported it as an application it didn’t work. In the end the opencv_java245.dll to the lib folder that was generated.

    Next to that I had to install the 64bit JRE because that’s not included in Processing.

    If you don’t have a reason to work 64bit, I think its smart to stick with 32bit.

  4. Yes, Gregs method is probably more Processing-like but I’m really comfortable with the inner workings of OpenCV so for me, I just want a friendlier windowing scheme and access to the Processing libraries. That’s really the thought behind this: making it easier to get access to raw OpenCV in Processing. I’m gonna add the stuff from your comment about the windows build for sure though, thank you very much for that!

  5. Thank you very much! With this tutorial I finally make OpenCV 2.4.6.0 working smoothly with Processing on by 64bit Win8.

  6. Hi Josh,

    As of the official release, OpenCV for Processing works with Windows as well. It ships with the OpenCV binaries built for 32-bit Windows. There was an issue with paths for haarcascade files being correct on Windows for a bit, but that’s fixed now. You can use the full OpenCV API with my library as well: you just have to use the right includes. It has functions for getting at all of the Mats it uses.

    I’d love to collaborate on tutorials/examples. If you’ve got any cool ones that you suggest I look at, I can port them over.

    Cool stuff!

  7. Thanks for the tutorial! I don’t understand the final step of installation however (via macports). What files am I supposed to copy to where? Is it the /opt/local…/java folder to the $PROCESSING_HOME/…ocvP5/library folder or vice versa?

  8. Okay! I figured it out (the contents of /opt/… should be copied into the …./ocvP5/library/ folder), but now I get the error

    “Cannot find a class or type named â??rectâ?”

    Which refers to line 23 of the code above. Is there some library I’m missing (I’m sorry, I’m new to processing)

  9. Thanks a lot for this. I was trying to find out why my dylib was referencing another opencv dylib. Obviously after reading your post I realised I had compiled opencv using shared dylibs. :smile:

  10. One quick thing. If you get an error saying you cannot find “rect” change

    ArrayList faceRects;

    to

    ArrayList faceRects;

    or

    include Rectangle from awt

    import java.awt.Rectangle;

    and switch rect to Rectangle and also

    faceRects.add(new Rect(rect.x, rect.y, rect.width, rect.height));

    to faceRects.add(new Rectangle(rect.x, rect.y, rect.width, rect.height));

    that fixed it for me :)

  11. Hello. How can i fix this problem:

    [ 37%] Built target opencv_ts
    Linking CXX executable ../../bin/opencv_perf_core
    ld: library not found for -lavcodec
    clang: error: linker command failed with exit code 1 (use -v to see invocation)
    make[2]: *** [bin/opencv_perf_core] Error 1
    make[1]: *** [modules/core/CMakeFiles/opencv_perf_core.dir/all] Error 2
    make: *** [all] Error 2

    after

    export JAVA_HOME=/usr/lib/jvm/java-6-oracle
    cmake -DBUILD_SHARED_LIBS=OFF ..

    make

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>