Activity 11: Ground Control Points (GCPs)

Introduction

This weeks activity was again using Pix4D image processing software but we were focusing on a new capability of the program, GPCs. GCPs are used to tie down and image to the surface of the earth so that it is spatially accurate. In this case we are using them to create a true ortho rectified mosaic. You do not have to use GCPs when processing imagery in Pix4D, however when we compare the accuracy of the images you see why it is important to use the GCPs. There are a couple of different ways to add GCPs and we talked about three of them in class.
The first method is used when the image geolocation and the GCPs have a know n coordinate system in the Pix4d database. The two coordinate systems may be different but Pix4D can automatically match them up. This is the method used the majority of the time and there is very little manual input compared to the other methods. The manual input is marking the GCP markers that are on the ground in each image that Pix4D finds them. Basically all you do is click a GCP number and images that Pix4D found that GCP in pop up and you click as close to the middle of the GCP marker in the image as possible. You have to go through this cycle a couple of times so you can not just hit run and leave however it doesn't take long to mark the GCPs. This is the method we used with out SX260 data of the South Middle School ponds. Figure 1 is an illustration of this process.

Figure 1 Work flow for the first method of using GCPs in Pix4D. Credit to Ethan Nauman for its creation.

The next method that is available in Pix4D when working with GCPs is when the images are not goelocated, or the initial images or GCPs are geolacted in a local coordinate system. This method is more manual entry. Instead of Pix4D picking out the GCPs in the images you have to manually find them and mark them training the program what to look for. As you repeat the process there will be less manual entry but the first part of this method will take some time.Again you can't just place the GCPs and images in Pix4D and let it run you have to be there to mark the GCPs. Figure  2 shows the work flow for this method.

Figure 2 Work flow for the second method. Credit to Ethan Nauman for its creation.

The final method works with any images and GCPs entered regardless of coordinate system or geolocation. If you want to be able to set up a project and let it run this is the best method to use. You will spend more time at the beginning marking the GCPs in the images but once that is done the process can run to completion. Figure 3 is the work flow.

Figure 3 Work flow of method three. Credit to Ethan Nauman for its creation.

Methods

Once you have selected a method to use for incorporating your GCPs the next step is to pick a coordinate system. When picking a coordinate system you want to chose on that is going to give you the most spatial accuracy for the area that you collected images of. In our case we chose UTM NAD 1983 Zone 15. This is a grid pattern coordinate system and zone 15 happens to lie right over the top of Wisconsin. It is a very good coordinate system for small local areas of focus and reduces the distortion compared to say a world wide coordinate system.Our GCPs were collected in NAD 1983 Zone 15 and the output coordinate system was also set to this. The images are in a different coordinate system but like I stated before Pix4D can automatically match the image and GCP coordinate systems.
Like last week images were loaded into Pix4D (see video for procedure) but this week they are already goelocated so adding the flight file was not necessary like when we used the GEMS data. We were given GCP locations which we collected in a previous activity (Activity 4 ) and they were in NAD 83 Zone 15 coordinate system. Given this information we used method one to incorporate the GCPs.
The first thing Pix4D asks for is a coordinate system and using the logic I explained earlier we chose UTM NAD 83 Zone 15 to increase spatial accuracy as much as possible. Then we began working with GCP in the GCP Manual Tie Point Manager (Figure 4). This manager tells you how many images there are that are associated with each one of the GCPs. This is all automated Pix4D finds the GCPs in each image and assigns the image to which ever GCP it is. Pix4D does a decent job of finding the GCPs but this is where the manual input greatly increases the spatial accuracy. We let Pix4D run the initialization step and then begin adding tie points. This is done by selecting the GCP and a group of images will appear (Figure 5). Pix4D picked those images out because it sees that GCP in the images but sometimes it picks images of other GCPs and makes other errors so this is where the manual input is important. You go through a good number of the images for each GCP, find the GCP marker and click as close to middle of it as possible (Figure 6). As you do this Pix4D becomes better trained in what to look for in the images for that GCP. The more images you mark the more tie points will appear next to the GCP number. For this lab we had at least 10 tie points for each GCP. The more tie points the better accuracy.

Figure 4 This is the tie point manager where you see the number of GCPs and next to that you see the number of tie points. Again we were shooting for at least 10 and as you can see we got a lot more than that for most of the GCPs which increases the image accuracy.

Figure 5 These are the images that open when you go in to add tie points for a GCP. You can see that each image is labeled and also if the images were bigger it would say which GCP they relate to. The green x are where Pix4D thinks the center of the GCP marker is. The yellow ring and x is where I manually selected (Figure 6) the middle of the GCP marker. As you go through and select more center Pix4D gets more accurate with the green x and you don't have to correct the location on as many images.

Figure 6 This is showing how to place those yellow markers mentioned in figure 5.

 Making sure you have everything set up correctly is important because if this were for an employer and you set it up wrong you could be wasting half a day or more of time and have to redo it all. One way to check and make sure the project is running correctly is to look at the initial processing report. The report has information about number of images, geolocation and matching, a preview of what the final mosaic will look like, image position, tie points, and image overlap. If any of this information doesn't look right to you make sure you go back and fix the issue before proceeding. The easiest way to tell if everything looks good is to look at the mosaic preview. If this doesn't look right or if the overlap preview is not what you expected make sure you go back and check geolocation of the images because Pix4D may not be correctly arranging the images into the final mosaic. Once you have marked the GCPs in the images and have a good amount of tie points for each the next step is letting Pix4D run a re optimization. This re optimize takes all the GCP corrections into account and creates a new report. This report has the same information as the first but the sections related to tie points and the GCPs will be different. If everything looks good then you let Pix4D run the last 2 last processing steps. Depending on the size of the project this will take a couple hours but can take days so you can expect to leave for a while and come back while it is running. When this is done you will get the final report which shows have the processing went as a whole. I have further explanation of this in my video in activity 10.
To show how much better the spatial accuracy of the final mosaic is when using GCPs we ran the same project witg out the GCPs which is then relying on the GPS in the camera or the geotag of the images to place the mosaic on the face of the earth. This is basically going to be a comparison of how accurate a Topcon survey grade GPS is compared to the GPS in a Canon SX260 camera. You can probably guess which is better.

Discussion

It is no surprise that the mosaic with GCPs is much more spatially accurate then the one with out. This can be attributed to the poor accuracy of the GPS in the camera that is geotaging the photos. It is comparing a survey grade GPS unit which has sub millimeter accuracy both horizontal and vertical to a average digital camera and the accuracy of the built in GPS. This is why when conducting vegetation change analysis or volumetric analysis it is vital to use GCPs so that the images are spatially accurate and consistent from week to week. Figure 7 and 8 are the two mosaics created by Pix4D. Figure 7 is the mosaic where GCPs were used and Figure 8 is just the imagery no GCPS. As you can see there is a pretty big difference. If you look at the imagery compared to the features in the base map figure 7 lines up pretty well and is very close to the same spatially as the basemap. In figure 8 you can see that the imagery isn't even close to lining up with the basemap, again this comes down to the terrible accuracy of the GPS in the camera.

Figure 7 This is the imagery run with the GCPs. There were 6 GCPs and you can see them in the map as the light blue dots. They were spread out evenly in the area that we wanted to image to assure that none of the image was distorted and it is as spatially accurate as possible. If you look at how well the imagery lines up with the basemap it looks like it is within a meter of lining up perfectly which is very good. You can see the much higher detail of the mosaic compared to the basemap which is one of the biggest reasons UAVs are used.

Figure 8 This is the imagery without the GCPs. Right away looking at this image you can tee that it is not spatially accurate. If you look at the parking lot in the left side of the image or the road in the top you notice that the imagery is shifted at least 10 to 15 meters possibly more. If you look very closely you can see the GCP markers on the ground. The light blue dots should be directly over the top of those like they are in figure 7. The blue dots haven;t moved the imagery is off by that much.If you need the imagery to be spatially accurate this data set is useless without the GCPs.

Figure 9 This is a map of the no GCP mosac over the top of the GCP mosaic.Again we see that they aren't close to lining up.

Besides looking at the spatial accuracy of images with or without GCPs and learning how to integrate GCPs into Pix4D this assignment also is looking at how Pix4D functions and how it compares to some other software. I was curios to see how the 3D mosaics created in Pix4D compare to 3D mosaics created in ArcScene. Figure 10 is the an aerial view of the Pix4D 3d mosaic and figure 11 is the same created in ArcScene.If you look at both them from the aerial perspective they look very similar. I think the ArcScene mosaic or figure 11 may have slightly better detail but it is very similar between the two. The bigger difference is when you look at the mosaics from an oblique angle. Figure 12 is the Pix4D mosaic and figure 13 is the ArcScene mosiac. Again they look pretty similar but the biggest difference I noticed is how well they represent 3D objects particularly trees. I think Pix4D does a better job. When looking at the sides of the trees you can still see the texture pattern of the trees compared to the ArcScene mosaic where trees show up as big green blobs.Another thing I noticed is that Arcscene has a hard time representing how tall a 3D object actually is.You can manually exaggerate the heights or have the program set the exaggeration from the extent. When I had it set the extent it was set to 1.7 which made the image look ridiculous. The mosaic in figure 13 to .75 exaggeration and seems to be pretty close to accurate. Pix4D is nice because it creates all the 3D objects as part of the mosaic and there is no need to figure out the exaggeration or lay the image over the DSM to get elevation values, its already done as part of the processing. In that regard I think Pix4D is better than ArcScene at 3D mosaics. 

Figure 10 The 3D mosaic generated in Pix4D

Figure 11 The 3D mosaic created in ArcScene

Figure 12 Oblique view of the Pix4D mosaic

Figure 13 Oblique view of the ArcScene mosaic

Conclusion

Working in Pix4D has been enjoyable. Mostly because it is so easy to create some really cool 3D images that are also very useful. The user interface is great and the help makes solving issues you may encounter pretty easy to fix most of the time.It has much more to offer than we have gone through in these two short labs and I hope to continue learning more functions of the program in this class but also as part of my role as the GEI technician here on campus.

 

Activity 10: Construction of a point cloud data set, true orthomosaic, and digital surface model using Pix4D software.

Introduction

The last activity was using  the GEMS sensor to collect imagery as well as the GEMS Software tool to create mosaics of the images collected. This week we are exploring another image processing software package called Pix4D. We explored some of the simple tasks that Pix4D has to offer. The software functions by finding common points on multiple images. Many .JPG files are loaded into the software and it lays them out in order of time and location they were collected. This can either be done having the images be geotaged as they are collected, which means a lat/long position is assigned to each image, or a .bin file can be loaded seperate of the images. The .bin files contain flight data about the flight path and in the case of the GEMS sensor when images were collected. This flight file is then matched up with the .JPG files and Pix4D can lay them out in order. Once the images are lain out in order Pix4D starts looking for points in multiple images that overlap. Points that appear in two or more images are called key points. These points overlap and align and the higher image overlap you have the more of these key points the software will find resulting in a better quality mosaic and 3D image. 

When it comes to image collection for images that are going to be used in Pix4D there are some considerations that need to be taken into consideration. One of the biggest is that a sufficient amount of image overlap is planned in the flight. For creating 3D images or getting good quality mosaics in Pix4D at least 75% frontal and 60% side lap is required. This is the recommendation for most images but not all it varies with the terrain being collected in the images. When looking at agriculture even more overlap is recommended because of the similarity of the images. Corn for example looks very similar in all the images and Pix4D will have a hard time finding key points so increased overlap is essential to make sure the mosaic still is high quality. This doesn't only apply to agriculture, any uniform surface such as water, sand, snow or trees should should be flown with increased overlap. Another tip that can help improve the quality of the mosaics is to set the exposure settings on the camera so that as much contrast as possible is captured in the images.

Combing two flights into one large mosaic is possible however certain parameters should be met. There are a couple of things that are important the biggest is to make sure that the conditions during the two flights were pretty similar. You don't want it to be sunny one day and cloudy the other day because Pix4D will combine the two flights but they will definitely still look like two separate flight not one big one. The light difference will be very obvious. Also making sure you fly the flights at the same altitude and overlap rate is also important. If the flights are collected at different altitudes you will be dealing with two different scales which will look weird when it is mosaiced. Similar overlap is also important and overlap between the two flight is also important so that areas between the flight don't get left out. 

Ground control points or GCPs are not necessary for Pix4D to process images however they can be useful. When very high spatial accuracy is needed like with volumetric analysis or vegetation change the GCPs assure that all the images are tied to the exact actual location on the earths surface or very close to it depending on the accuracy of the GPS unit. The GPS unit we have on campus has sub mm accuracy so that is very good for increasing the spatial accuracy of the images.

One final part of this software that is nice is the quality report. The quality report basically tells you how well the images you input turned into the mosaic. Figure 1, 2, and 3 are parts of the quality report. 

Figure 1 This is the first portion of the quality report. This is basically an overview of the project as a whole. This portion tells you the project name, when it was processed, how large of any area the image covered and how long the project took to finish. Also included in this part is a preview of what the RGB mosaic is going to look like after processing and how the DSM is going to look. The DSM or digital surface model is dealing with the 3D aspect of the image. In this report you can see that the shelter in the middle of the RGB image stands out as bright red in the DSM which means Pix4D is reading as higher elevation than the rest of the image.One last part is the calibration details which tell you how many of the images were able to be used by the software. It also tells you how many images were geolocated. The higher percentage both these numbers are the better the mosaic will turn out. For this report both categories are at about 95% which is very good.

Figure 2 What figure 2 is showing is how well the area of the mosaic was overlapped. Green areas mean that 5 or more images overlapped in those areas while the yellow and red areas are only 1 to 2 images overlapping. The more green area the better the mosaic and the more accurate the 3D image will be. This image has very good overlap but it is easy to see why it is important to fly a larger area than you actually want to collect data for. The further to the outside of the image you get the less overlap to assure good overlap throughout fly a much larger area than you actually want the data for.

Figure 3 This is an review of how many key points were found between the images. The darker the area the more key points are found the lighter the fewer were found. Again maximizing the number of key points is essential for getting a good mosaic and 3D image.

Methods

For this activity we processed two data sets. One is of the Eau Claire soccer complex here in town using the GEMS sensor and the other is the same location with a Canon SX260 camera. Please watch the video below (Figure 4) to see how these were run and what the final products of each are. Figure 5 is the resulting mosaic from the GEMS sensor project.

Figure 4  This is a video on how to create new project in Pix4D and run imagery you have collected

Figure 5 Mosaic of the Eau Claire Soccer Park collected by the GEMS sensor

The video in Figure 4 was showing the process for running the GEMS data in Pix4D. The same steps are followed in order to run the SX260 except when the images are being added. The images for the SX260 are geotaged meaning they have lat/long information included with them unlike the GEMS images so there is no need to add the .POG file with the trigger locations in it. Figure 6 is the mosaic from the SX260 project.

Figure 6 This the mosaic created when the images collected with the Sx260 camera are run in Pix4D   

Once the projects have run an the mosaics are completed there are many things that you can do to manipulate the mosaics. A few we explored are the creation of "fly" through animations of the mosaics, line measurements, surface area measurements and volume calculation of objects in the mosaic. Figures 7 and 8 are the two animations I created moving through the mosaics.

Figure 7 This is the animation for the GEMS data or Figure 5

Figure 8 This is the animation for the SX260 data or Figure 6

The video in figure 9 explains how to created line features and measurements, polygon measurement and measure the volume of a 3D object. All of these tasks are easy to and didn't take long. The video also discusses how to export the created features so that they can be use in ArcMap to create maps with.

Figure 9 This is a short tutorial on some of the functions in Pix4D. All of these functions were performed on the GEMS data set in the video.

Discussion

I thought that so far Pix4D has been very easy to use. The help menu makes it easy to find basically anything you don't know how to do in the software. How you interact with the software is very easy as well. There are hidden details that need to be paid close attention to when creating and running projects however for a pretty new user of this software I haven't had many problems getting tripped up by the details.

Creating the line, polygon and volume measurements and shape files was very straight forward and easy to understand. I created a few maps from the shape files I made in Pix4D. Figure 10 is the maps with GEMS data and figure 11 is the Sx260 data.

Figure 10 This is the a masked portion of the GEMS data run in Pix4D and brought into ArcMap. You can see the 3 features I created and brought in from Pix4D. They line up very will with the imagery or in other words they are in the same location on this map that they were when I created them in Pix4D. This spatial accuracy and consistency is important for accurately mapping spatial features. If you look at the difference between the GEMS imagery and the base map you can see that the GEMS image is moved slightly left and downward from where the base map is. This is because of different levels of spatial accuracy between the two maps. We won't know which is more accurate until we use GCPs to tie down the imagery to the earths surface which is in the next lab.

Figure 11 This is the masked SX260 data run in Pix4D. Again you can see the features I created in Pix4D and exported to ArcMap. The thing that should be noticed about this map is how poorly the base map and SX260 imagery line up. The SX260 imagery is moved way down and to the right of from where the base map imagery is. This shows how poor the GPS unit in the camera is that geotags the images. The base map, imagery and features are all in the same projections and yet the imagery is off by probably 50 meter or more. This is not good spatial accuracy and should defiantly be tied down with GCPs to place in the correct location. It will be interesting to see how well this issue is corrected by the GCPs next week when we run some SX260 imagery with GCPs.

Pix4D has a lot to offer as a software package and from this small introduction to some of its functions this week I look forward to seeing what else it has to offer. The next project will be incorporating GCPs when processing imagery.