Recently, I came across a situation where, when making a photogrammetric model, up to 2/3rds of each input image was not relevant to the subject being investigated. Considering I planned on doing extensive tests on the dataset, I needed to generate at least partial masks to ensure there wasn’t too much computational redundancy. Considering I had back of a notepad calculated that to do the model runs as planned with these images without masks would take ~21 days, I set to ease the process by building some masks.
Here I’ll present a two camera case, from Middlebury’s stereo datasets, for simplicity. NOTE: I don’t list any code here, but it can be found on an ipython notebook html here, or on github here, or from here.
OpenCV (version 2.4.11 in Python) was my first port of call, and has a very convenient function called ‘findContours’. This scans the image for edges (The input is a ‘threshold’ image previously generated) and pulls out consistent shapes in an image. Considering the area of interest was quite distinct from the rest of the image, this was somewhat easily done, but left a certain amount of relevant data masked. In this example I target a green cone in the stereo scene:
We can call on a second function ‘BoxPoints’ which draws the minimum bounding box for the area which the contour pulled out. This more or less split the image into a single bounding rectangle which enclosed the area of interest. We can specify a maximum and minimum search size (in pixels) too, which will help in the image segmentation. This could be convenient for generating classified images without going through the process manually.
I can think of two situations where this may be convenient for bigger datasets where manual masking might be impractical:
- Where only a portion of the image is relevant to the results. For instance, when shooting an object on stage (such as in Middlebury’s multi-view examples) you can reduce the search space for the feature detectors. This could also be relevant for masking out sky, sea, or other low-value/low-relevance areas in a scene.
- Where camera positions are know from an earlier bundle adjustment, but a higher resolution is desired in only a portion of an scene, or for a specific object.
The masking code provided in the attached ipython notebook html is very basic, and meant to guide any python aficionado’s towards making their own inroads into the ideas presented here. I’ve just presented a minimum working example for those curious enough! Other photogrammetric applications include measuring areas of objects in an image (Particularly relevant for, say, microscopy), or simple object tracking when a camera is moving under stable lighting conditions.