MASI: Modules for Aerial and Satellite Imagery

Version 6.0 ADS Modules

Tutorial

 

 

 

 

 

 

 

说明: D:\VisionOnSky\LOGO\空视\logo.png

 

VisionOnSky Co., Ltd.

www.visiononsky.com

 

File Version: v1.0

March 29, 2023

 


Special Notes:

 

(1) Before starting the tour of MASI software, please be sure that the installation manual and tutorials are fully read and understood. If users need to know and exploit the software deeply, please further read user manual of the software.

 

(2) Users can receive software license through two manners. One is to download license from internet. If this manner is used, users should provide us an e-mail address. We send the password to the e-mail address via the Sense Shield (SS) Licensing platform. If you have not found it, please check your e-mail trash (spam) box. The e-mail text may be written by Chinese, but the English string is actually the password. The account used to log in the SS (Sense Shield) middleware is your e-mail address and the password is the English string sent to your e-mail box. After you download the software license and bind it on your machine, you can use the software even though you disconnect the internet. If the internet connection is not very stable, we recommend that you disconnect the internet, otherwise a timeout error may happen. Using this manner, users can download software package, tutorials, user manuals, installation manual, etc. from website: http://www.visiononsky.com. The other manner is to use a physical dongle to get the software license which is stored in this dongle. You don’t need to provide us an e-mail address and don’t need to log in in the SS middleware, just need to plug the dongle on computer. But you still need to install the SS middleware. Using this manner, users should contact us to get the software package and the package downloaded from the website is invalid for the physical dongle.

 

(3) The file name and the path name to be used in English version of MASI software should be English characters, and the file name and the path name can not include blank space.

 

(4) The full path of file can not be too long (e.g., more than 200 characters). If it is too long, please shorten the full path of file.

 

(5) Please make sure that there is enough free space in the disk where the work directory is located.

 

(6) If text editing is needed, editors such as UltraEdit, NotePad++ are highly recommended.

 

(7) Because of the update of software version, some GUI interfaces of programs illustrated in this tutorial are not the newest, which are changed and adjusted. The latest user manual of the software provides the newest one, please refer to that document.

 

(8) Any feedbacks, suggestions, and problems, please send to the e-mail address: jhyang@vip.163.com

 


History of Versions

 

Version 1.0: finishing the first version of this document, tutorial of ADS modules. Time: Mar. 2023, Author: VisionOnSky Co., Ltd.

 

 


1 The brief introduction of ADS modules of MASI Version 6.0

The functions in MASI Version 6.0 for ADS images can be packaged as some relatively independent modules. The modules include the following functions: automatic stereo matching and generation of highly dense DSM (as well as point clouds), automatic 3D modeling with true color textures, automatic ortho-rectification, automatically transforming DSM to DTM (the height difference between DSM and DTM is the height of building / tree), automatic finding change of surface height (used in the automatic finding of new buildings, the unplanned buildings and the removed buildings, and estimating their corresponding accurate height), automatic mosaic (mosaic of DSMs, mosaic of ortho-rectified images, mosaic of histogram matched images), volume calculation, extraction of building attributes (the center position of building, height of building, number of layers, area of ground, and construction area of building), generation of boundary polygon for ADS images, image displaying, procedural and batch processing for DSM production, interactive editing of DSM/DEM, collecting polygons, and some commonly used tools. These modules for ADS images are shown to users by a main interface (adsMain.exe). After installing of MASI software, users can launch the main interface by clicking the shortcut to adsMain.exe or directly double-clicking the program, adsMain.exe, itself. Then, the corresponding functions can be launched by clicking the icons in the interfaces.

 

Figure 1. The main interface (adsMain.exe) of ADS modules

 

The features of ADS modules in MASI Version 6.0 are as follows:

(1) Supporting all sensors of ADS series: ADS40, ADS80, ADS100;

(2) GUI operating manner and batch processing manner are supported in MASI software. GUI-based operations for DSM generation and ortho-rectification are easy. User can also automatic generate or write batch files including commands to fulfill the pipeline processing for a specific procedure. Distributed computing where multiple tasks can be allocated to multiple machines is also supported.

(3) Modules demanding large computation requirements, such as automatic stereo matching and DSM generation, ortho-rectification, mosaicking, have capability of parallel computing. Thus, the computational resources can be fully exploited.

(4) Supporting different operating systems, either Windows OS or Linux OS. MASI can run on the commonly used PC computer, and cluster computer usually located in data center or supercomputer center. This tutorial is based on Windows OS. If you need the document of usage of MASI on Linux OS, please contact us.

 

For more details about functional features and technique traits, please refer to the document of Product Descriptions and Typical Applied Cases.

 


2 The automatic processing flowchart for ADS image

Figure 2. the automatic processing flowchart for ADS image

 

Currently in MASI Version 6.0 there is not a function of aero-triangulation for ADS images, but the aero-triangulation results which comply with the file format defined by the producer of ADS sensors can be used in MASI software. Experimental results show that the aero-triangulation results from Leica GPro, XPro, and HxMap software packages can be correctly used in MASI software. The aero-triangulation results from any third party software can be used as well. Any problem encountered in the course of using these aero-triangulation results, please contact us immediately.

 

Input for the flowchart provided by MASI ADS modules is ADS L1 imagery. In the flowchart, each step corresponds to a function in ADS modules as follows:

(1) Data preparation. The aero-triangulation results from third party software, e.g., Xpro, and Level 1 (L1) images, etc. usually need to be copied and moved in order to enter this processing flowchart. Therefore, the paths which are the corresponding values of the keywords, such as IMAGE_FILE_NAME, ORIGINAL_ORIENTATION, ADJUSTED_ORIENTATION, and ORIGINAL_CALIBRATION, in the support file (.SUP) for ADS need to be modified. In this step of data preparation, users can modify values of these keywords by using changePathADS.exe and changePathADSForm.exe provided by MASI software. Clicking the icon showing “Data Preparation” in the main interface,

The following menu will pop up.

The command item, Change Paths in SUP file, in the menu is to call the function of data preparation. The GUI program called from the command item is changePathADSForm.exe. Also, the GUI program can be launched by double-clicking the program directly. For more details, please refer to “Usage of the changePathADS.exe and changePathADSForm.exe programs”.

 

(2) Automatic stereo matching and DSM generation. The automatic stereo matching is fulfilled for any stereo pair among three panchromatic views (possible RGB views or Green band if being ADS 100) of ADS images and dense 3D points (users can decide whether the text-formatted point clouds files with extension .xyz are saved by setting the configuration option) are generated. The program tries to extract a 3D point for each pixel (pixel-wise matching). Then, these point clouds are transformed to DSMs and these DSMs are mosaicked. At last, a mosaicked DSM covering the whole image is generated (the DSM before interpolation).

 

Because the size of ADS image is large, image is usually divided into tiles to further process. In the course of stereo matching, intermediate files, i.e. epipolar and disparity images, are generated for each tile. The program determines whether these epipolar and disparity images should be deleted in the light of configuration options after each tile is finished. If point clouds are set to be saved, a text-formatted file of point clouds with extension .xyz is generated for each stereo pair of image tiles. In files of point clouds, each line is for one point, and each line includes the x, y, z coordinates value. At the same time, a list file including all filenames of point clouds extracted from stereo pairs of image tiles is generated. The files of point clouds as well as the generated list file can be used as an input of the program, generateDSM.exe and generateDSMForm.exe, resulting in DSM results with different parameters, such as grid spacing (cell size), no-data value, and the method to select the height value. In such a way that through one time of stereo matching (because stereo matching is a step demanding large computation) the DSM results with different grid spacing are achieved in the light of users’ requirements. The point clouds file with extension, .xyz, can be translated to point clouds with LAS file format according to users’ needs.

 

For ADS sensor, a type of three lines scanner, multiple stereo pairs can be obtained, namely forward-backward stereo, forward-nadir stereo, backward-nadir stereo. Then, multiple list files of cloud points respectively from multiple stereo can be incorporated into a new list file by using the program mergeFilelist.exe or mergeThreelist.exe (or the GUI program, mergeListForm.exe, which can be called via the command item, Merge Lists, under DSM menu; please refer to the tutorial of satellite module). For instance, list files of point clouds respectively from ADS forward-backward stereo, ADS forward-nadir stereo, and ADS backward-nadir stereo are combined. A list file including all files of point clouds is generated. The commands used to merge list files of point clouds are as follows:

>mergeFilelist.exe list1.txt list2.txt merged_List.txt

>mergeThreelist.exe FWD_NAD_points_List.txt BWD_NAD_points_List.txt FWD_BWD_points_List.txt merged_List.txt

The last argument in the command, i.e., merged_List.txt, is the list file after merging.

 

Clicking the icon showing “DSM” in the main interface,

The following menu will pop up.

 

The command item, Stereo Matching, in the menu is to call the function of stereo matching and DSM generation. The GUI program called from the command item is adsStereoForm.exe. Also, the GUI program can be launched by double-clicking the program directly. For more details, please refer to “Usage of the adsStereo.exe and adsStereoForm.exe programs”. The command item, Merge Lists, in the menu is to call the previously mentioned program, mergeListForm.exe, which is used to merge list files.

 

Transforming to DSM and mosaicking: called by the command item, Generate DSM, in the menu. A list file is passed to the function. The point clouds whose filenames are listed in the list file are transformed to DSMs, respectively. Then, these DSMs are mosaicked, resulting in a mosaicked DSM covering the whole strip (the DSM before interpolation). The list file can come from two cases. If point clouds are set to be saved in the course of stereo matching, a list file including all filenames of point clouds extracted from stereo pairs of image tiles is generated. The list file can also be one into which the multiple lists of cloud points respectively from multiple stereo are incorporated by using the program mergeFilelist.exe or mergeThreelist.exe, and mergeListForm.exe.

 

The command item, Generate DSM, in the menu is to call the function of transforming to DSM and mosaicking. The GUI program called from the command item is generateDSMForm.exe. Also, the GUI program can be launched by double-clicking the program directly. For more details, please refer to “Usage of the generateDSM.exe and generateDSMForm.exe programs”.

 

If there are multiple ADS strips for an area, for each strip three stereo pairs exist: forward-backward stereo, forward-nadir stereo, backward-nadir stereo. Other case is that multiple ADS strips are acquired in different phases. In MASI software, two methods are provided to generate highly accurate and highly dense DSM. One method is as follow. Point clouds are set to be saved in the course of stereo matching. All list files of cloud points respectively from multiple stereo pairs are merged (Merge Lists under DSM menu) to generate a list file including all filenames of point clouds extracted from all strips. Then, the merged list file is passed to the function, transforming to DSM and mosaicking (generateDSM.exe and generateDSMForm.exe, Generate DSM under DSM menu). A highly accurate and highly dense DSM is generated. The other method is to mosaic multiple DSMs from multiple strips to produce a highly accurate and highly dense DSM by using the programs, mpimosaic.exe and mpimosaicForm.exe (DSM Mosaic under DSM menu). For each strip, one DSM or several DSMs (multiple stereo pairs are selected for a strip) can be yielded before mosaicking. The GUI program, adsDSMForm.exe, provided by MASI software, can automatically generate a batch file in which multiple steps in the latter method are connected, resulting in procedural and batch processing for ADS images. The former method is flexible and can produce DSM results with different parameters, such as grid spacing (cell size), no-data value, and the method to select the height value according to users’ needs. The latter method is more efficient.

 

To simplify user’s operations and have capability of procedural and batch processing in the course of DSM production, a GUI program called adsDSMForm.exe is provided by MASI software. By the program, multiple steps required for DSM generation of whole block of ADS images, which include stereo matching for multiple strips, mosaic of DSMs of multiple strips, are connected automatically, so that the un-interpolated DSM results covering multiple ADS strips can be generated by means of one clicking. Users can launch the GUI program via the command item of “Pipeline and Batch Processing” in the above pop-up menu. Also, the GUI program can be launched by double-clicking the program directly. For more details, please refer to “Usage of the adsDSMForm.exe program”.

 

(3) Post processing for DSM. The above mosaicked DSM has some areas which have not height value (before interpolation these areas are filled by using no-data values). In this step, these areas with no-data value are interpolated, and outliers (or small spots of errors) in the resultant DSM after interpolation are removed. The output of this step is the product of dense DSM, which is generated by this software. The DSM product can be used in different applications followed (e.g., volume calculation, finding change of surface height, extraction of the height of building / tree).

 

The command item, Post Processing, in the menu is to call the function of post processing for DSM. The GUI program called from the command item is postDSMForm.exe. Also, the GUI program can be launched by double-clicking the program directly. For more details, please refer to “Usage of the postDSM.exe and postDSMForm.exe programs”.

 

(4) Automatic ortho-rectification. The program uses the DSM source to ortho-rectify ADS nadir color image to generate so-called true ortho-image. If true ortho-image needs to be generated, high side overlapping between strips is needed. The needed side overlapping between strips depends on building height on ground. The higher are buildings, the higher is the side overlapping between strips. Generally, the side overlapping between strips is above 60%. An alternative way is that DTM as a height source, instead of DSM, is used to ortho-rectify nadir color images to generate ordinary ortho-image. Clicking the icon showing “Ortho” in the main interface,

The following menu will pop up.

Two operational modes are provided by MASI software: ortho-rectification for one strip and ortho-rectification for multiple strips. The two modes are called via the command items, Ortho for one ADS strip and Ortho for multi ADS strips, respectively, in the above menu. The GUI program called from the command item, Ortho for one ADS strip, is adsOrthoForm.exe. Also, the GUI program can be launched by double-clicking the program directly. For more details, please refer to “Usage of the adsOrtho.exe and adsOrthoForm.exe programs”.

 

The GUI program called from the command item, Ortho for multi ADS strips, is adsBlockOrthoForm.exe. Also, the GUI program can be launched by double-clicking the program directly. In the mode of multiple strips, images of selected multiple strips can be ortho-rectified, these ortho-rectified images are then mosaicked and an ortho-mosaic image covering all these strips is generated. The area along mosaicking seams is feathered in the course of mosaicking. The program which has above functions is adsBlockOrthoForm.exe. It automatically generates a batch file which is used to implement the above steps in the mode of multiple strips. The batch file includes multiple commands of which each employs the program, adsOrtho.exe, to ortho-rectify images of a strip and the last command whose function is to mosaic these ortho-rectified images from multiple strips to generate a total mosaic image covering all input strips. Calling the generated batch file is to execute the procedure. For more details, please refer to “Usage of the adsBlockOrthoForm.exe program”.

 

(5) Comparing surface height and change finding. By comparing DSMs of different phases (generated by MASI software), the program can automatically find change of the surface height and calculate the value of changed height. It can be used in the automatic finding of new buildings, the unplanned buildings and the removed buildings, and estimating their corresponding accurate height. The map of height difference produced in this step can be used in the following extraction of building attributes (the center position of building, height of building, number of layers, area of ground, and construction area of building). Clicking the icon showing “Surface Change” in the main interface,

The operating interface of the function will be launched. The GUI program called from the icon is surfaceChangeForm.exe. Also, the GUI program can be launched by double-clicking the program directly. For more details, please refer to “Usage of the surfaceChange.exe, surfaceChangeForm.exe and buildingHeightForm.exe programs”.

 

(6) Automatic DSM2DTM. The program deletes the non-grounded objects like buildings, trees in DSM and transforms DSM to DTM where the pure ground height is saved. Clicking the icon showing “DSM to DTM” in the main interface,

The operating interface of the function will be launched. The GUI program called from the icon is qefiterForm.exe. Also, the GUI program can be launched by double-clicking the program directly. For more details, please refer to “Usage of the qefilter.exe and qefiterForm.exe programs”.

 

(7) Height estimation of building / tree. The height difference between DSM and DTM (nDSM) is the height of building / tree. The height difference (raster form) produced in this step can be used in the following extraction of building attributes (the center position of building, height of building, number of layers, area of ground, and construction area of building). DSM is generated by the functions in MASI software (including post processing for DSM), and DTM can be produced in the step 6 in this flowchart. Clicking the icon showing “Building height” in the main interface,

The operating interface of the function will be launched. The GUI program called from the icon is buildingHeightForm.exe. Also, the GUI program can be launched by double-clicking the program directly. For more details, please refer to “Usage of the surfaceChange.exe, surfaceChangeForm.exe and buildingHeightForm.exe programs”.

 

(8) Applications. The generated dense DSM, map of height difference, nDSM (i.e., height of building / tree) in above steps can be passed to the application functions which fulfill volume calculation and extraction of building attributes (the center position of building, height of building, number of layers, area of ground, and construction area of building). Clicking the icon showing “App” in the main interface,

The following menu will pop up.

Currently two applications modules are included, i.e., volume calculation and automatic extraction of building attributes in the light of buildings footprints. The command item, Calculate Volume, in the above menu is to call the function of volume calculation. The GUI program called from the command item is calVolumeForm.exe. Also, the GUI program can be launched by double-clicking the program directly. For more details, please refer to “Usage of the calVolume.exe and calVolumeForm.exe programs”. The command item, Extract Buildings’ Attributes, in the above menu is to call the function of extraction of building attributes. The GUI program called from the command item is extBldgAttributesForm.exe. Also, the GUI program can be launched by double-clicking the program directly. For more details, please refer to “Usage of the extBldgAttributes.exe and extBldgAttributesForm.exe programs”. In the course of volume calculation and automatic extraction of building attributes, the bounds in the form of polygon are needed. Users can click the command item, Polygons Drawing Tool, in above menu to launch the interactive tool, collectPolygons.exe, to collect polygons and save them.

 

(9) 3D modeling with true color textures. Images and highly dense DSM generated in previous steps are used to generate 3D TIN models with true color textures. The 3D models are in a form of tiled mesh. The data type of the images which are used for texture mapping is unsigned 8 bit integer. If it is not, user should stretch the images to unsigned 8 bit using the autoStretch.exe program provided by MASI software. Clicking the icon showing “Textured 3D Model” in the main interface,

The following menu will pop up.

The command item, textured 3D modeling, in the above menu is to call the function of 3D modeling. The GUI program called from the command item is adsScene3DForm.exe. Also, the GUI program can be launched by double-clicking the program directly. For more details, please refer to “Usage of the adsScene3D.exe and adsScene3DForm.exe programs”.

 

In addition, other functions are included: generation of boundary polygon for ADS images, image displaying, interactively editing of DSM/DEM, collecting polygons, automatic mosaic (mosaic of DSMs, mosaic of ortho-rectified images, mosaic of histogram matched images), correcting ADS image by using the average height determined from SUP file, transforming RGB image to grey image, rotation of image, reflection of image, transforming point clouds to surface in the form of raster, transforming DSM with raster form to point clouds, image cropping, creating overviews for image, transforming between orthometric height and ellipsoidal height, and projection / re-projection for image based on WGS 84.

 

The method of format transform for point clouds generated by MASI software: the file of point clouds generated by the software is text format, which can be transformed to LAS format through the following steps. In the course of transform, a third party software package OSGeo4W (including the open source library, i.e., libLAS) is exploited. The package can be downloaded from the official website of OSGeo4W. The steps are as follows:

(1) Translation to LAS format: the command, namely txt2las.exe (OSGeo4W package includes the program), in the open source library, libLAS, can be used to translate the point clouds with text format to the LAS file. The corresponding command is as follows:

>txt2las.exe -parse xyz -i filename.xyz -o filename.las

(2) Assigning RGB values: the command, namely las2las.exe (OSGeo4W package includes the program), in the open source library, libLAS, can be used to assign the RGB values from truly ortho-rectified image to the points in LAS point clouds file. The colored point clouds are obtained. One example of the command is as follows:

>las2las.exe -i points.las --color-source ortho.tif -o points_rgb.las --file-format 1.2 --point-format 3 --color-source-scale 256 --color-source-bands 1 2 3

 

 


3 The flowchart sample of ADS images

In the following, the flowchart will be illustrated through a sample. First, the main interface of the ADS modules is launched via the Windows Start menu, i.e., calling the program, adsMain.exe. The launched main interface is as follows:

 

Clicking the icons in the main interface is to call the corresponding functions. The main steps of the flowchart sample are as follows:

 

Step 1: Data preparation

In this sample, there are three strips of ADS100 images and each strip has three views of RGB images. The input datasets passed to MASI software are L1 images and their ancillary files. The following files are included: SUP files, ADS files, ODF files, ODF.ADJ files, CAM camera files and image files with TIF format.

 

In this sample, the directory, E:\tutorial_ADS\ADS100\TLS, is set as work directory and the directory will be work directory for the next following steps. First, the above mentioned SUP files, ADS files, ODF files, ODF.ADJ files, and image files with TIF format are copied to the work directory. Meanwhile, all CAM camera files are placed in the directory, E:\tutorial_ADS\ADS100\TLS\SH100_10524\v001. Then, the programs changePathADS.exe and changePathADSForm.exe are called to change file paths in SUP files and these changed SUP files are saved in the work directory E:\tutorial_ADS\ADS100\TLS. For the functional introduction of changePathADS.exe and changePathADSForm.exe programs, please refer to Chapter 2. Clicking the icon showing “Data Preparation” in the main interface,

The following menu will pop up.

By clicking the command item, Change Paths in SUP file, in the above menu, the GUI program is launched. The operational setting and feedback from the program are as follows:

 


 

 


After finishing the input and output setting, users can click the button of “Execute”. The program is executed and finished very quickly. The blank area of the right side will display the information printed by the command program. Once the execution of the program is finished, a message box will pop up as follows:

In this sample, the changed SUP files are saved in the work directory E:\tutorial_ADS\ADS100\TLS and the filenames of these SUP files do not change. Hitherto, all dataset to be used are prepared and are in the work directory.

 

Step 2: Automatic stereo matching and DSM generation

In this sample, the images for starting stereo matching are RGB color images with three bands. We need firstly transform them to grey images by using the tool, rgbToIntensity.exe, provided by MASI software (the command item, RGB to Intensity, under Tools menu). All color images with TIF format are transformed to grey images and filenames of all types files remain the same as before. Meanwhile, the value of keywords BANDS in .ads file is modified from 3 to 1. If the images for starting stereo matching are panchromatic images of ADS40/ADS80 three views or green band images of ADS100 sensor, the transformation does not need.

 

Clicking the icon showing “DSM” in the main interface,

The following menu will pop up.

 

By clicking the command item, Pipeline and Batch Processing, in the above menu, the GUI program is launched. It can be used to generate the un-interpolated DSM results covering multiple ADS strips by means of one clicking. The graphical interface of the program is as follows:

 

 



 

The operating steps for procedural and batch processing of multiple strips are as follows:

(1) Users need select one type sensor among three types, i.e., ADS100 GRN, ADS100 RGB, and ADS80/40 in the input setting after setting work directory. If the work directory is on network storage, a local directory on the used machine disk also needs to be set. Before be processed, these data files are transmitted to the local directory automatically, and then processing for them are based on the local directory. The results after the whole procedure, which are the total mosaic of DSM, are transmitted back to network storage automatically. The whole procedure is automatic and users do not need any additional manual operations. The reason is that employment of local storage device is more efficient. Please make sure that there is enough storage space. In this sample, the data to be processed is saved on local storage device; thus, a local directory does not need to be set. For different cases, users can determine if the nadir image should be included by checking or unchecking the checkbox of including nadir. After the button of find files automatically is clicked, all SUP filenames of different strips, with different view angles, in the work directory are listed in the grid view below the button. These listed images are suitable for stereo matching. Users can use the icons on the bottom of the grid view to adjust the positions of filename on the list. The adjusting operations include inversing the list order, deleting a selected strip, moving a step toward up or down for a selected strip, and moving to head or real of the list for a selected strip.

(2) Users select and set parameters on the GUI regions of stereo matching, serial or parallel processing, and output setting (the filename of the mosaicked DSM, the grid spacing, and no-data value), respectively. Users can determine whether the files of point clouds are saved on the opened setting interface of configuration options for stereo matching. If parallel processing is selected, number of processors should be inputted as well as the current user name using the Windows OS and password of the user.

(3) On the GUI region of procedural and batch processing, the batch file, which is used to save commands automatically generated for multiple strips, is set here. After clicking of button of Generate file, a batch file with extension .bat, which is used to describe the procedure, is generated. The file needs to be stored in the work directory. If the work directory is on network storage and a local directory has been set, the generated batch file will include commands of copying files and automatically modifying the contents of SUP files.

(4) Clicking the button of execute is to start the processing. Finishing the processing means end of the procedure.

 

In the middle GUI region of the left part, there is a button which is used to open the setting interface of configuration options for stereo matching, which should be set and saved. The following interface will pop up after the button of setting parameters and saving is clicked.

 

 

After all configuration options on the above interface are set, users can click the button, Save Configuration, to save these options and generate the configuration file, ads.conf, which is stored in the work directory. Meanwhile, its full path will be shown in the textbox behind the words of “Config file” in the main form. For the meaning and setting of these options, please refer to the program usage of adsStereo.exe and adsStereoForm.exe.

 

Because ADS sensors have multiple views, multiple stereo pairs can be selected in the course of stereo matching. The main form provides three checkboxes to select multiple stereo pairs. Depending on users’ selections, one or multiple (2 or 3) stereo pairs may need to be processed for each strip. The DSM results generated from each stereo pair will be taken as input for the final mosaicking of all DSMs, resulting in the last DSM results with higher precision and higher density.

 

Serial or parallel processing can be selected for stereo matching. Here is parallel processing of multi-processors and each processor utilizes multiple cores whose number is set in the option, Number of used threads per processor. While serial processing means only one processor is exploited and the only processor also utilizes multiple cores. If parallel processing is selected, number of processors should be inputted as well as the current user name using the Windows OS and password of the user. Next time users can load the information by clicking the load button, and do not need input the information again. In terms of determination of the number of processors, please refer to the program usage of adsStereo.exe and adsStereoForm.exe.

 

The user name and password used in the parallel processing are saved in the file named as pwd.txt (If parallel processing is not used, there is no such file). The file, pwd.txt, can be used to load user name and the password which are used in the parallel processing. These two files are also stored in the work directory.

 

A useful method to decide if arguments, parameters setting and command operations in this step are correct is to check if the objects in the generated epipolar pair are aligned strictly in horizontal direction. Because the epipolar generated for ADS image (a type of linear scanner) is quasi-epipolar, sometimes there are vertical parallaxes about one pixel in epipolar pair, which is a normal case. The vertical swipe function in the module of image viewer, imgViewer.exe, provided by MASI software can be used to view horizontal alignment. For more details, please refer to the program usage of imgViewer.exe.

 

The above setting is valid for all input strips.

 

The meaning of GUI components in the output setting is as follows. DSM mosaic of multiple strips: users should set the filename of the mosaicked DSM of multiple strips (the DSM before interpolation); Grid size: the grid spacing (cell size) of the raster file used to store height value (the value of grid spacing can not be better than the actual ground sample distance, GSD, of image; unit is meter); no-data value: the value used to fill in the area without height value in the raster DSM file (no-data value). The no-data value only supports two cases, 0.0 or -9999.0.

 

The blank area of the right side will display the information printed by the command program, and the printed information will also be saved in a log file named as adsDSM_ddMMMyyyyHHmmss_log.txt. In the filename, dd are two digital numbers indicating a date, MMM are three English characters indicating an abbreviation of a month, yyyy are four digital numbers indicating a year, and HHmmss are digital numbers indicating hour, minute and second in the form of 24 hours. The log file is also stored in the work directory.

 

The contents of the configuration file, ads.conf, generated in the work directory, for stereo matching are as follows:

<?xml version="1.0" encoding="utf-8"?>

<adsStereo>

  <tmpFile>

    <Type>GTiff</Type>

    <Extension>.tif</Extension>

  </tmpFile>

  <Tiling>

    <Tiles>5</Tiles>

    <extendedSize>512</extendedSize>

  </Tiling>

  <ifDeleteTmp>1</ifDeleteTmp>

  <selectHeight>max</selectHeight>

  <heightLimit>

    <ifExternal>1</ifExternal>

    <extendedLength>150</extendedLength>

    <elevationFile>dtm.tif</elevationFile>

  </heightLimit>

  <projType>UTM</projType>

  <zoneSize>6</zoneSize>

  <numCores>6</numCores>

  <Bounding>

    <ifBound>0</ifBound>

    <boundsFile>polygon.shp</boundsFile>

  </Bounding>

  <ifSavePoints>0</ifSavePoints>

</adsStereo>

 

Users can use text editors, such as UltraEdit, NotePad++, to open the generated batch file (.bat file) and check the commands, automatically generated by the program, in the batch file.

 

For more details, please refer to “Usage of the adsDSMForm.exe program”. The DSM result obtained in this step is the DSM before interpolation, which needs to pass to the next step (i.e., post processing for DSM) to yield the final highly dense DSM. Before post processing for DSM, if the un-interpolated DSM has big areas of mismatching (e.g., water body, cloud and snow covered area, area with repeated textures) which can be found by the manner of visual checking, the interactive tool for editing DSM/DEM (i.e., collectPolygons.exe) can be used to edit the DSM with big areas of mismatching. For more details, please refer to “Usage of the collectPolygons.exe program”.

 

Step 3: Post processing for DSM

In this step, these areas with no-data value in the above mosaicked DSM are interpolated, and outliers (or small spots of errors) in the resultant DSM after interpolation are removed. The output of this step is the product of dense DSM, which is generated by this software. The DSM product can be used in different applications followed.

 

By clicking the command item, Post Processing, in the above menu, the GUI program is launched. The graphical interface of the program is as follows:

 


 


 

In the section of “The DSM source”, the DSM before interpolation from the previous step is selected, and in the section of “The DSM result”, the filename of resultant DSM, which will be the outcome of this step of post processing, is set.

 

After the interface is launched, the default values for these parameters are given. In most cases, the default values can be used. The blank area of the right side will display the information printed by the command program. The meaning of parameters is as follows:

The maximum size of the errors to remove: the size of the area of maximum errors to remove. The number, 5, indicates that outliers up to the size of 5 x 5 can be removed. The parameter is integer.

The threshold of height difference: The point (or small areas) where height difference beyond the surroundings is above the threshold is deemed as outliers. The parameter is float and unit is meter.

The size of big block: the size of big block which is used to divide the image. The DSM is so large that block division is required. The parameter is integer.

The size of overlap between big-blocks: the size of overlap between the adjacent big-blocks. The parameter is integer.

According to the features of the terrain of covered area, a filtering method for post-processing is selected among no filtering (suitable for vegetation area), median filtering (suitable for dense urban area), and mean filtering (suitable for smooth terrain). In this sample, median filtering is selected.

 

Again, still by the manner of visual checking, if on the interpolated DSM after post processing small areas of errors is still found, the interactive tool for editing DSM/DEM (i.e., collectPolygons.exe) is continually used for editing. After editing, users can determine if post processing for DSM is required again in the light of their needs. For more details, please refer to “Usage of the collectPolygons.exe program”.

 

Step 4: Automatic (true) ortho-rectification

Clicking the icon showing “Ortho” in the main interface,

The following menu will pop up.

The command items, Ortho for multi ADS strips, in the above menu is to call the function of ortho-rectification for multiple strips. As a result, an ortho-mosaic image covering all these strips is generated. By clicking the command item, Ortho for multi ADS strips, the GUI program is launched as follows:

 

 



 

The concept of work directory is the same as those in the previous steps. All files required in the step should be put in the work directory. Therefore, the directory, i.e., E:\tutorial_ADS\ADS100\TLS, is still set as the work directory in this step.

 

In the input setting, multiple SUP files corresponding to the images to be ortho-rectified are selected. Usually nadir images are selected for ortho-rectification. Either grey image or RGB image can be rectified, if only all types of files (image files with TIF format and their ancillary files, SUP files, ADS files, ODF files, ODF.ADJ files) correspond to each other.

 

Configuration options in the middle GUI region of the left part are the same as those of the program, adsOrthoForm.exe. For the meaning of these options, please refer to the program usage of adsOrtho.exe and adsOrthoForm.exe. The values of configuration options adopted in ortho-rectification are the same for multiple strips. These values are all from the settings in the interface. These configuration options will be saved in a configuration file, adsOrtho.conf, which is stored in the work directory.

 

Tiles: the number of tiles to which the short side of the resultant ortho-rectified image is divided. A tile is a square shape whose size is determined in the program internal by the option set here and the short side of the resultant ortho-rectified image. The option is useful for the situation where the image size is very large and the memory of used computer is limited. The option corresponds to the XML element, Tiles, in the configuration file. File of elevation source: the full file path of the external elevation source used in the ortho-rectification. The option corresponds to the XML element, Elevation, in the configuration file. The file selected in the option, File of elevation source, is the DSM result after post-processing in Step 3. The underlying reference ellipsoid of elevation file must be the same as that adopted in orientation file of ADS sensors. It is WGS 84. The projection method of elevation file must be the same as the one set by users for the option, projection of the ortho-rectified image. The boundary of elevation source covers the region of the images of all strips. If third party elevation source is used, please make sure the sameness of coordinate system.

 

Projection of the ortho-rectified image: currently two types, i.e., UTM and GK (Gauss-Kruger, in some countries it is also called TM), are supported, which are selected by setting UTM and GK, respectively. The underlying reference ellipsoid for the two types of projection must all be WGS 84. The option corresponds to the XML element, projType, in the configuration file. Zone size (degree): zone width for projection, it can be 6 or 3 degree. If GK is selected for projection, it can be 6 or 3 degree; while UTM is selected for projection, it can only be 6 degree. The option corresponds to the XML element, zoneSize, in the configuration file. Number of used threads: setting the number of used threads (CPU cores or the virtual CPU cores by using hyper-threading technology). For the cases where CPU cores are enough, we suggest that number of 4, 6 or 8 is set here. The option corresponds to the XML element, numCores, in the configuration file.

 

After the above options in the interface are set, users can click “Save Configuration” to save these options. These configuration options will be saved in a configuration file, adsOrtho.conf, and next time the options can be loaded from the configuration file by clicking “Load Configuration”. The configuration file, adsOrtho.conf, is also stored in the work directory. If the button of “Execute” is clicked, the configuration file will also be generated in the same way.

 

In the output setting, users should set the output directory and the filename of the output ortho-mosaicked image. In the output directory, the ortho-rectified image for each strip will be stored. The resultant images after ortho-rectification are named in a specific rule: name of SUP file_ortho.extension. The data type of ortho-rectified images, as well as the number of bands, is the same as that of original images, e.g., unsigned 8 for original image, still unsigned 8 for ortho-rectified image. The file format of each ortho-rectified image is set here in the option, Format of each resultant ortho image. At the same time, users should set the image resolution (Unit: m) of the ortho-rectified image. The resolution of ortho-rectified image can not be better than the actual GSD (ground spacing distance). The output ortho-mosaicked image should also be stored in the output directory.

 

After finishing the setting of configuration options, input and output setting, users can click the button of “Execute”. A batch file with extension .bat, which is used to describe the procedure, is automatically generated internally and is automatically called to start the procedure. After the execution of the batch file is finished, ortho-rectified image for each strip and a total mosaic image covering all input strips are generated.

 

A list file named ortho_List.txt is generated, which is saved in the same directory (i.e., the output directory) where the ortho-rectified images are stored. The list file includes, in sequence, all filenames of ortho-rectified images whose order is the same as that in the input setting. Meanwhile, the filenames of input SUP files are, in sequence, written into a list file whose name is supfiles_list_ddMMMyyyyHHmmss.txt. In the filename, dd are two digital numbers indicating a date, MMM are three English characters indicating an abbreviation of a month, yyyy are four digital numbers indicating a year, and HHmmss are digital numbers indicating hour, minute and second in the form of 24 hours. The generated batch file is named as adsOrtho_ddMMMyyyyHHmmss_batch.bat, in which the meaning of ddMMMyyyyHHmmss is the same as above. The list file of SUP filenames and the batch file are both stored in the work directory.

 

The blank area of the right side will display the information printed by the command program, and the printed information will also be saved in a log file named as adsOrtho_ddMMMyyyyHHmmss_log.txt. The meaning of ddMMMyyyyHHmmss is the same as above. The log file is also stored in the work directory.

 

For more details, please refer to “Usage of the adsBlockOrthoForm.exe program”.

 

The contents of the configuration file, adsOrtho.conf, generated in the work directory, for ortho-rectification are as follows:

 

<?xml version="1.0" encoding="utf-8"?>

<adsOrtho>

  <Tiles>3</Tiles>

  <Elevation>E:\tutorial_ADS\ADS100\TLS\3strips_dsm_post.img</Elevation>

  <projType>UTM</projType>

  <zoneSize>6</zoneSize>

  <numCores>8</numCores>

</adsOrtho>

 

Users can use text editors, such as UltraEdit, NotePad++, to open the generated batch file (.bat file) and check the commands (multiple commands of ortho-rectification and the last mosaicking command), automatically generated by the program, in the batch file.

 

Step 5: Comparing surface height and change finding

By comparing DSMs of two different phases, the program can automatically find change of the surface height and calculate the value of changed height. Clicking the icon showing “Surface Change” in the main interface,

The operating interface of change of surface height is launched.

 

Attention: the datasets in the interface may probably be not the results from ADS images. The purpose of this step is to demonstrate how to operate.

 

 

The corresponding DSM files are selected in the sections of the previous DSM and the next DSM, respectively. After processing, the file of height difference is obtained. Users can set two parameters, the height threshold and no-data value.

 

The height threshold: If the height difference is less than the threshold, the difference is not interesting and it will be assigned as no-data value in the resultant file. For the case where the height difference is more than the threshold, the real difference will saved in the resultant file.

 

No-data: no-data is the value used to fill in the area without change of surface height (or invalid area) in the resultant file. No-data can be determined either from input DSM or from the right side list box.

 

Step 6: Automatic DSM2DTM

Clicking the icon showing “DSM to DTM” in the main interface,

The operating interface of automatic DSM2DTM is launched.

 

Attention: the datasets in the interface may probably be not the results from ADS images. The purpose of this step is to demonstrate how to operate.

 

 

The program adopts an algorithm with multilevel triangle-based ground filtering. It deletes the non-grounded objects like buildings, trees in DSM and transforms the DSM to DTM where the pure ground height is saved. The input file is DSM source and the output file is resultant DTM.

 

The meaning of parameters is as follows:

The predefined size: the defined size in which the lowest point is selected as a ground point; unit: the same as the grid spacing (cell size), e.g., 40.0. It means that the lowest point in a grid, whose sizes of width and height are both 40 meters (assuming the unit of grid spacing is meter), is taken as a ground point. The higher is the predefined size, the more levels in the pyramids are generated. Level number of pyramids decrease, then the built triangles is to be dense. This parameter is usually decided by the size of the biggest non-ground object in the terrain area. If the predefined size is less than the size of non-ground object, these objects will not be filtered out. On the contrary, if the predefined size is too large to maintain some terrain details. Thus, users need find a balance between filtering out big non-ground objects and maintaining the terrain details in the light of the requirements of applications.

 

Level, to which level the DSM is filtered. The higher is the level set, the less ground points are generated, the less terrain details remain, and the more efficient is the processing. The lower is the level set, the more ground points are generated, the more terrain details remain, and the less efficient is the processing. Usually the triangulation of mass points and searching a specific triangle in a huge triangulation is inefficient. The level set can not be higher than (the highest level - 1), level is 0-based, e.g., 3.

 

The threshold of distance: the threshold of distance from a point to the corresponding triangle. In the course of filtering, a point whose distance to its corresponding triangle is more than the threshold is deemed as a non-ground point, vice versa, e.g., 1.0 meter. If the value of threshold is low, there are less ground points which will survive in the filtering processing and there are also some risks that the true ground point may be filtered out by taking them as non-ground points.

 

The size of block: for processing large raster, block division is needed. Filtering is fulfilled, block by block, and its size must bigger than two times of overlapping size, e.g., 6144. Usually the default value is used. The reason that block division is used is that a large DSM will result in huge ground points which require huge computation for the filtering processing. The block division is a way which promotes computational efficiency and saves memory consumption.

 

The size of overlap between blocks: the size of overlap between the adjacent blocks, e.g. 128. The overlay ensures the rightness of border of each block. Usually the default value is used.

 

No-data: no-data is the value used to fill in the area without height value. Because each grid in raster file must be assigned a value, for some cases where there are no real height values in some areas, the concept of no-data is introduced. No-data can be determined either from input DSM or from the right side list box in this interface.

 

If there are some non-ground objects which are not filtered out, such as buildings and vegetation areas, or small areas of errors in the resultant DTM in this step, the interactive tool for editing DSM/DEM (i.e., collectPolygons.exe) can be used for editing. For more details, please refer to “Usage of the collectPolygons.exe program”.

 

Step 7: Height estimation of building / tree

Clicking the icon showing “Building height” in the main interface,

The operating interface of height estimation of building / tree is launched.

 

Attention: the datasets in the interface may probably be not the results from ADS images. The purpose of this step is to demonstrate how to operate.

 

 

The corresponding files are selected in the sections of the ground height (DTM) and the surface height (DSM), respectively. After processing, the file of building / tree height is obtained. Users can set two parameters, the height threshold and no-data value.

 

The height threshold: the height being less than the threshold is the value which is not interesting or the possible existing errors. The value will be replaced by no-data in the file of building / tree height. The height being more than the threshold is an interesting and reliable value, which as a true value will be saved in the file of building / tree height.

 

No-data: no-data is the value used to fill in the area without building / tree (or invalid area) in the resultant file. No-data can be determined either from input DSM or from the right side list box.

 

Step 8: Applications

Clicking the icon showing “App” in the main interface,

The following menu will pop up.

Currently two applications modules are included, i.e., volume calculation and automatic extraction of building attributes in the light of buildings footprints.

 

Volume calculation: a polygon in the vector form in which volume is calculated and highly dense DSM are given, the module estimates the volume of objects (e.g., sands, small stones, coal, mineral particles and garbage) occupying the area. For more details, please refer to “Usage of the calVolume.exe and calVolumeForm.exe programs”. Users can click the command item, Polygons Drawing Tool, in above menu to launch the interactive tool, collectPolygons.exe, to collect the polygon and save it.

 

Automatic extraction of building attributes: a vector file with ERSI shape format, which including multiple polygons of building footprint, and the corresponding nDSM dataset are given, the module automatically extracts the center position (x, y coordinates) of building, area of ground, height of building, number of layers, and construction area of building for each building. Moreover, these extracted values are set as new attributes for these polygons. nDSM can either be obtained by subtracting DTM from DSM in the module of buildingHeightForm.exe, or be obtained by calculating difference between two DSMs in the module of surfaceChangeForm.exe. The former can be used to extract building height from the bottom to top, number of layers, etc., while the latter can be used to automatically find new, illegal, added, demolished, or height-exceeded buildings and extract their attributes, such as position, height, layers, ground area, and construction area of these found buildings. For more details, please refer to “Usage of the extBldgAttributes.exe and extBldgAttributesForm.exe programs”. Users can click the command item, Polygons Drawing Tool, in above menu to launch the interactive tool, collectPolygons.exe, to collect polygons on nDSM and save them to a vector file. Polygons of building footprint in vector file can also be from third party source. For both cases, it is required that building footprints are accurately aligned with the actual boundaries of buildings. And, the vector file including polygons and the nDSM file should be geo-coded and the underlying coordinate system is the same.

 

Step 9: 3D modeling with true color textures

Clicking the icon showing “Textured 3D Model” in the main interface,

The following menu will pop up.

By clicking the command item, textured 3D modeling, the GUI program is launched as follows:

 


 


 

All files required in the course of generation of 3D models should be put in the same directory and the directory is set as work directory. The required files are DSM file, images files used for texture mapping and their ancillary files, such as SUP files, ADS files, ODF files and ODF.ADJ files. Users should copy all these files to a directory in advance and set it as work directory.

 

The data type of the images which are used for texture mapping is unsigned 8 bit integer. If it is not, user should stretch the images to unsigned 8 bit using the autoStretch.exe program provided by MASI software (the command item, automatically stretch image to 8 bit, in the above menu to call the function).

 

Attention: (1) Before texture mapping, users should ensure that in these SUP files, the files which the corresponding values of the keywords, such as IMAGE_FILE_NAME, ORIGINAL_ORIENTATION, ADJUSTED_ORIENTATION, and ORIGINAL_CALIBRATION, point to are existent and correspond to these keywords, respectively. If not, users can modify values of these keywords by using changePathADS.exe provided by MASI software. For more details, please refer to the program usage of changePathADS.exe. The required files in the course of texture mapping, such as SUP files, ADS files, ODF files, ODF.ADJ files, image files with TIF format, should be put in the same directory, and the directory is set as work directory. (2) If the program, autoStretch.exe, provided by MASI software, is used to stretch ADS images (with TIF format) used for texture mapping to unsigned 8 bit, the filenames of resultant stretched images and of their ancillary files remain the same as before. Moreover, in .ads files the value of keyword DEPTH is changed from 2 to 1 and the value of keyword BITS is changed from 16 to 8.

 

In the input files, users should select the DSM file and SUP files (or a list file which includes filenames of these SUP files) used for texture mapping. Then, users can set configuration options and other setting. For the meaning of configuration options and operating method, please refer to “Usage of the adsScene3D.exe and adsScene3DForm.exe programs”.

 

Hereto, the whole flowchart for ADS images is finished, including automatic stereo matching and generation of highly dense DSM, automatic (true) ortho-rectification, automatic finding change of surface height, automatically transforming DSM to DTM, height estimation of building / tree, volume calculation, extraction of building attributes, etc. If the functions, such as generation of boundary polygon for ADS images, image displaying, interactive editing of DSM/DEM, collecting polygons, automatic mosaic (mosaic of DSMs, mosaic of ortho-rectified images, mosaic of histogram matched images), correcting ADS image by using the average height determined from SUP file, transforming RGB image to grey image, rotation of image, reflection of image, transforming point clouds to surface in the form of raster, transforming DSM with raster form to point clouds, image cropping, creating overviews for image, transforming between orthometric height and ellipsoidal height, and projection / re-projection for image based on WGS 84, are needed, please refer to user manual of the software.