Issue link: https://geospatial.trimble.com/en/resources/i/1415711
Trimble Software: Working With Large Datasets 3 Data flow through your projects Efficiently working with large datasets requires careful planning of each aspect of the process from data collection through archiving. Every detail needs to be considered. This section describes different phases of the project data flow and how each may be impacted when working with large datasets. Pre-project Planning Understanding the scope of the project prior to data collection is critical. Knowing the project extents and the client's expected final deliverables allows good project planning to optimize the workflow from beginning to end. It is beneficial to ask all pertinent questions in advance to avoid having to guess at what is needed throughout the project. Getting these details in writing with the client helps avoid misunderstandings and frustration later. Data Collection Addressing the data volume at the beginning of a project frequently results in more manageable workflows during processing and delivery of the results. It is useful to determine in advance exactly how much data is needed to create the desired deliverables for the project. Usually, the data capture can be adjusted to provide a sufficient amount of information without collecting the maximum amount of data that each hardware solution can provide. Remember… "More is not necessarily better." ● When using a 3D laser scanner, choose the appropriate scan density to yield the required detail. o There may be relatively small differences between field collection times for different scan densities, but the data management and processing times of the collected scans should be considered before choosing the density. o Some scanning solutions provide the ability to collect scans at lower density for context, along with higher density scans of smaller areas where more detail is needed. o A set of floor plans for a large open building probably would not require the same density as a small plant with fine details that will be modeled. The same number of scans may be needed for both but each will require different scan densities to achieve the desired results. o When making a decision about scan density, it is important that enough data is collected to provide the necessary overlap to compute a good registration of the scans. ● High resolution images can be very large and consume a lot of drive space. o Base your determination of how many images to collect on the desired resolution and coverage for the area. This applies to aerial imagery as well as terrestrial. It is very time consuming to over- collect imagery with most field solutions. o Think about the need to create colorized scans. The process of colorizing scans can be lengthy for large projects and the drive space consumed by the required images can be great. ● It is important to make adequate backup copies frequently. o The extra time to create a backup, and the drive space consumed, are small prices to pay to protect against data loss. o Drive failures, mistakes in project processing, or data corruption while processing the data, can all be overcome if a backup copy of the data is available. o Backing up the raw field data as soon as possible is especially important. Consider creating multiple backups at the end of each field day, so that a re-visit to the site is not necessary.