Mangrove canopy using photogrammetry

A good paper released in the open access journal Remote Sensing in Ecology and Conservation today detailed the use of worldview stereo pairs to estimate canopy height within mangroves in Mozambique. One of the factors making this more practical than other forest types is the relative uniformity of the canopy height (“tree height saturates and remains relatively consistent in mature and intact forests”), as well as the fact that they only occur at or around sea level, which allows for confidence to be given in ground control readings as well as the canopy height itself.

The introduction details the use of the NASA Ames stereo pipeline, which has been of some interest to me after coming across a very good description of an implementation of a semi-global matching algorithm written by an engineer who has a colleague working on Ames. SGM is particularly relevant to SfM-MVS photogrammetry (See Hirschmuller’s papers for more detail!), but is also being incorporated into big software packages such as IMAGINE. Zack’s blog is pretty amazing in general and I recommend it be given some attention!

The paper itself lists some pretty great results, and some of the maps generated are beautiful! It serves as a proof of concept for modern photogrammetry, which has come on leaps and bounds, and the potential for back-projecting and reprocessing data in this way is pretty exciting. I’ll be keeping my eye on the journal for future applications to environmental remote sensing!

Monitoring climate from space

I’ve participated in this MOOC over the last few weeks run by ESA on monitoring climate from space which is a good introductory resources for anyone interested in Earth Observation generally. The range of topics is broad and I certainly learned a thing or two so if you’re a novice and are interested in the field I’d certainly recommend giving it a go!

https://www.futurelearn.com/courses/climate-from-space

Ur the cast

Picked this up off twitter, a company which operates a few cameras onboard the ISS. I think the hype video is pretty dramatic, though some of the images produced are very cool, I’ll be keeping my eye on how they progress in the next while as they have stiff competition in the sector!

https://www.urthecast.com/

GLAS – Spaceborne LiDAR

IceSAT was a unique satellite launched in 2003 with the aim to provide accurate topographic information of the Earth’s surface over a number of years. One of it’s aims was recovery of ice sheet mass balance and cloud property information, as detailed on NASA’s page here. Onboard this satellite was an instrument called GLAS, the geoscience laser altimetry system, a spaceborne LiDAR I’ve been meaning to look at for a long time. Last Sunday I decided to give it a look in the afternoon, and after a bit of tinkering with the files available here I produced a point cloud showing the Earth’s topography as seen by GLAS, collated for one month in 2003.

The files are structured that topographic information (The GLAH14 and GLAH15 files I used) are split up for each individual day, each with 14 orbits per file. I’ve presented one of the files below, as seen in CloudCompare, about 1.3 million points.

GLAH14 data collected on the 4th March 2003

As you can see, each day doesn’t recover a detailed scan of the Earth, so a collation of the month gives us a better idea as to the extent of the mission.

GLAH14/15 data for all of March 2003 – ~64,000,000 points

I really like this as not only can you see the completeness of the data (I had to mask lots of points), but it gives you a really good idea of what a low earth orbit looks like. IceSAT orbited about 600km above the Earth’s surface, and this is the pattern produced.

After removing duplicate points we can then compare each of the heights recovered from each laser pulse. For practicality’s sake I just used height above the WGS84 ellipsoid, a term different from height above sea level which is normally used. This which just saved a bit of time, one of a couple of other corners which were cut to produce the cloud. This gives us a global model of surface topography for March 2003, shown below.

The final product is pretty interesting and it was a really fun exercise to do. I’m hosting the model on my webspace also, which is viewable here.

Computer vision applications in industry

Just a quick entry having happened across David Lowe’s teaching page, which has a full range of commercial applications of computer vision systems. Some of them are really cool, such as live tracking for traffic signalling. You can find other nifty applications sorted by industry on this page.

ISS Photogrammetry

I saw Changchang Wu’s model of the Earth from footage taken from the ISS put through VisualSFM on youtube, so I thought I’d repeat the process using some newer footage (Taken with a Nikon D4) over Europe. The image squence (available here) starts somewhere in the English channel and follows the spacecraft all the way south to the red sea and Arabian peninsula.

The ground sample distance for a full frame sensor at 16 megapixels at a flight height of 400km with a 24mm lens is 120m in nadir. The field of view across the width of the sensor is about 600km.

I took every third frame to make up the photogrammetric model, with only 174 being aligned for the model itself. The point cloud produced is available for viewing here, and I hope to do more of these in the future aswell as georeference it! I’ve included a gif below of all the frames used as well as the viewshed from Photoscan.

http://imgur.com/PjCGsPH

Artifical texturing

For problems associated with photoscanning low-contrast objects projecting light onto the scene is one clever way of ensuring a good match for stereo images. It’s nothing new, and remains a focus within computer vision research as it can reduce the need for camera calibration in stereo setups, such as the bumblebee, which are used for industrial applications. Structured light setups have been suggested for low resolution blurry imagery such as those in underwater cameras, where solutions to the correspondence problem are shaky at best. I know of one PhD student who’s focusing on accurate camera calibration using these methods, but I wonder if projecting an artificial texture on an object with low contrast and using multi-view stereo with a self-calibration could be equally as useful. Structured light scanners do just this for two cameras with a laser beam, but I’m thinking about if you could just do it with a home projector or similar and keep it constant throughout images.

The first figure of this web page (from 12 years ago!) exemplifies the idea and can solve for camera positions/intrinsics using the laser. There are other good ideas with this in mind from this University of Washington lecture slide set from 6 years ago, their researchers always seem to be ahead of the game!

What camera for a UAV?

UAV photography has come on leaps and bounds within the last few years, but considering which camera to use often isn’t the focal point (!) of many articles. With this in mind, let’s consider a few stereotypical camera setups, and why we may/may not want to use some in certain situations. Certainly for photogrammetry the high grade cameras are better, but with weight restrictions everything becomes a little more ambiguous. Here I’ll consider three that I’d like to test side by side in the future, and a fourth I’m unlikely to ever see.

1. Go pro hero 4

The quintessential beginner’s UAV/rugged terrain setup, go pros are legendary for how stable their videos are. They weigh 82g and so are a very light choice for mounting onto UAVs, a major consideration for camera selection. While many in the photography community are megapixel mad, often diffraction effects are a more important consideration. 12 Megapixels on a ~6mm wide sensor will likely cause serious softening of edges, and while ground sample distance will be the same as others on paper, ‘spatial resolution’, the ability to resolve individual objects based on the Rayleigh criterion, will be far lower.

Image acquired using a gopro on a rotary UAV

The distortion at the edges is a further nuisance for metric applications, as it needs to be correctly modelled to ensure accuracy within measurements. The example is shot at an angle oblique to the surface, which typically isn’t used in metric applications, but shows the distortions pretty well.

2. Ricoh GR

This is a camera which I see being used more and more for photogrammetric applications, it boasts a high dynamic range and an APS-C sensor (~22mm width) with a well machined lens at a fixed focal length of 18mm. In the last blog post, I spoke about smartplanes surveying a mine in Sweden where they used a Ricoh to achieve some pretty amazing results. It also happens to be the camera used in the sample dataset provided with MicMac, an open-source SfM software which I have talked about on this blog previously. It light enough (168g) that it would fit on, for example, a Conservation Drones fixed wing platform, which would constitute an amazingly low cost setup which could deliver survey-grade results, with some care. I’ve included a sample which was included in the MicMac data at a downsampled .5 MP resolution, but you can see the contrast is great!

1 image from MicMacs example dataset

3. Sony A7S

Part of an emerging crowd of mirrorless cameras known as MILCs (Mirrorless interchangeable lens cameras), I think this could be a good choice for higher grade surveys, considering it’s seemingly ridiculous low light shooting capabilities, full-frame (36mm width) sensor and relatively lower weight (450g body vs, for example a Nikon D810’s 1kg). It makes me wonder of how viable a mixed workflow would be with RTK-GPS, a low cost laser rangefinder and this camera to produce really high quality orthophotos which can be fully georeferenced off two separate data sources. Particularly, I wonder if it would be a good choice for canopy height modelling, as lasers are known to perform worse during daytime, so maybe a dusk survey with a ramped ISO could produce some good results. One advantage over the others is that the lens is interchangeable, so while the Ricoh is somewhat limited in its applications, the A7S could be used for many things. One I’m going to keep in mind for when I happen upon a spare £1,500.

I found a video on vimeo with video footage flown on a DJI zenmuse – A setup around about £3.5k but would produce some amazing accuracy I’m sure.

4. Phase one iXU 180

Not much to say about this, but putting it in to consider the very very top end of the market. Phase one have produced this medium format camera (53.7mm sensor width) with a pricetag of $60,000, though it does tick all the boxes for aerial surveying. It’s about the same weight as the D810 (1kg), so won’t mount on most consumer grade UAVs, though we aren’t really going for low-cost here. I don’t suspect I’ll ever see one with my eyes, but have just grabbed the demo from their website of which I’ll included a downsampled version here for completeness, it’s pretty good looking flying 600m above ground which means a wide field of view, ground sample distance is 5.2cm.

Keep an eye on an upcoming post about various UAV platforms, which I plan to tag onto what I’ve presented here!

The geobusiness show

Last week, ontop of attending the peer review workshop which was the subject of my last post, I was lucky enough to attend the geo business show, a trade fair detailing the latest technologies associated with geomatics and visual environment technologies. It’s become a pretty big event, and I’m happy that I could recognize many familiar faces and companies who were presenting the latest products. I attended about 5 talks during the day, ranging from inspiring to drab, as is often the case in trade fairs error isn’t always the main focus. Nonetheless I’ll detail some of what caught my eye, and stuff I’ll potentially delve into in the future.

Smart planes

A slick presentation from a Swedish company mainly detailing a competition in which they participated which saw several surveying firms trying to get recover the most accurate fit for ground checkpoints from an open pit mine. They used Agisoft Photoscan with which I am very familiar, a Ricoh GR3 with all the settings detailed and a fixed wing platform. Their achieved accuracies given the area being surveyed were phenomenal, and while the sample size was low it was very impressive to see!

English Heritage

English heritage gave a presentation on how they were applying new photogrammetric techniques to monitor assets using a mixture of close range photogrammetry and UAVs. They reckoned that photogrammetric surveys could produce models up to LOD ~ 2.75 in about a third the time as would have taken by traditional means. Considering all the equipment they use is a decent DSLR for close range stuff and a pocket camera for aerial photography it shows just how advanced SfM has become in the last few years.

Ecometrica/Carbomap

A joint presentation by two startups looking to promote ethical management of natural resources. Ecometrica have developed a very nice web GIS for producing reports and statistics on queried areas, their demo involved presentation of Saatchi’s map for the Amazon. Considering very recent reports suggesting deforestation in the Amazon is accelerating contrary to recent investigations, it makes the platform more important than ever. While Saatchi’s map does have its inaccuracies, I was happy to spy a healthy uncertainty in the above-ground biomass estimates, as opposed to many other presentations where uncertainties are often not discussed at length.

Carbomap spoke of a TerraSar-X product which they were using to advise forestry councils on timber recover after storm events. Biomass from RADAR is a concept which will come to fruition in the BIOMASS mission, but does have its detractors. The presentation was great, and made use of a handy web GIS to display results in a handy browser based app.

Tomtom

Tomtom, the car navigation system manufacturer presented on how they query all the data they collect to make useful insights into motorists behaviour. They have over 12 trillion data points which they can data mine to see how traffic flow is changing over time, and with such a big dataset I can imagine city planners would be very interested in this sort of stuff. Some questionable jargon was used (Referring to their model as a ‘fusion engine’), but an interesting talk nonetheless!

Pixalytics

A Plymouth based company which use satellite data to advice on bespoke solutions to seemingly big problems. The presentation detailed a range of products, including mixed algorithm images which were actually pretty cool. Later I visited their stand and they had made prints of some of the images which would be great to hang on your wall, might have to look into having a few made up myself! It was nice to see some satellite remote sensing as I’ve been doing smaller scale stuff recently, will be keeping up with their movements as satellite start-ups are all the rage in the US and seem to have a lot of support in the UK at the minute.

In all it was an interesting experience and I look forward to next years event!

Peer review: The nuts and bolts

As a first year PhD (The earliest of early career researchers) the peer review process is still somewhat veiled in mystery. I’ve had friends who have been through the publication process before, and have seen some quite lazy reviewer’s comments, as well as some really constructive succinct ones. This variation in quality of comments came as a surprise, as their didn’t seem to be a standard format for them, and considering the process takes some time I feel like it’s something worth doing right. This was the only insight I was armed with when I decided to attend Sense about Science’s workshop entitled ‘Peer review: The nuts and bolts’ where we learnt much about the process and more experienced researchers than I shared their stories.

The workshop was split into two group work sessions, one on people’s experiences with peer review and another on the public perception of science with a panel discussion in between. Within my particular group many of the researchers had had some poor experiences with the process, many feeling reviewers’ comments were either very harsh (behind the mask of anonymity) or incomplete. Considering small communities exist within each field of research, some of our group were speculating that competing interests may incentivise certain reviewers to try and discourage publications from certain people/research groups. This was all news to me, and points at a side of the peer review process not really seen by the public.

The second major theme within our group was the difficulty of proceeding when you receive contradictory reviews, which appears to happen quite often. It seems that these comments can sometimes be mutually exclusive which leaves the researcher in the awkward position of having to resubmit without all the recommended edits made. Some communication between authors and reviewers was discussed, which could play an important part in the future of peer review.

With this dissatisfaction being expressed with the current system, alternative models were considered. The discussion took to considering the benefits of a moderated online forum, which actually came up in Irene Hamas portion of the panel debate. Peerage of science, to take one, is a moderated forum offering peer-reviews before submission which seems to satisfy many of the demands being sought after by our group.

After the first section of group work the panel went on to discuss pros/cons of the current peer review system, why it exists and how it is likely to evolve in the future. Elizabeth Moylan opened proceedings by introducing the basics of the system, and talking about how quickly the system is changing. As an outsider to the formal process (having not submitted any papers to date), it was good to hear what I should expect in the future. Mike Smith spoke next, detailing why people submit research and how to constructively take on board reviewer’s comments and not become too dismayed at how harsh some can appear. Speaking from personal experience, he shared a range of comments he had received on his research, which was particularly funny as many were woefully contradictory as alluded to before. Irene Hames concluded by introducing alternative models that currently exist, and how we should be approaching the changing landscape of the peer review model.

The panel were very relatable and open with their critiquing of the current system, acknowledging many of the faults noted within our first group session. A Q&A followed where many of these concerns were raised again, some doubts over the lack of incentive for reviewers to participate in the process in the first place were discussed and recommendations on how to conduct reviews. As I am finding increasingly as I become part of the research community the best answer to most issues is to just ask, which was the overwhelming recommendation from those on the panel.

Like many things in life, it’s easy to criticise, but far more difficult to take those dissatisfactions and present them as rational concerns to those who can change things. Considering how lonely research can be, communication is something I hope to continue to work on not just for my own sanity, but so I can participate in how the reviewing process changes over the course of my research career. The panel were very considered in their recommendations for every issue raised and I think much of the audience were glad to find that they were not alone with their concerns, but that many of these issues had been raised previously and are outstanding faults within peer review.

The second group workshop discussed the public perception of science, an area which I am particularly interested in, having done several long form blog posts to a collaborative website which attempt to digest the cutting edge in my research area, and keeping this blog publicly. We discussed how perceptions differ across various media platforms, how to try and dissuade the use of ‘clickbait’ headlines within media today and how to communicate what peer review is to the public effectively.

These are things I’m sure many scientists think about quite often, but rarely have a platform to discuss them. Within our group we admitted being prone to just accepting facts within certain media depending on many factors (Sometimes you just want to accept what the metro is telling you first thing on a Monday morning, for example), and how we’d like to see a drive for more integrity and better review from the media’s end. Joanne Thomas touched upon this in her summary of science in the media, noting how researchers and research groups have become a part of how science is reported, and not just the science itself.

In all, the workshop was very informative about what I can expect over the next few years as I try to develop my skills as a researcher, and how the process may change over the course of my career. Best of all though is that it was actually really fun, and I met some people I hope to cross paths with again. I’ll leave twitter links for those interested below, I’d best get back to researching!

James

Twitter : Voice of young science, Sense about Science