Sen2Cor - a program for processing images taken from the Sentinel-2 satellite. The article describes how to install, run, and configure it.In the magistracy, I needed to make atmospheric correction of images from the Sentinel-2 satellite. Master in ecology, so it was possible not to delve into physics and mathematics, but to use a ready-made tool. Such a tool was found - Sen2Cor.The problem is that there is no clear instruction for Sen2Cor. The authors of geoinformatics textbooks do not crawl into such jungles as working with specific programs, and the authors of scientific articles, on the contrary, believe that image correction is too simple a part of the technique, therefore they do not describe it in detail. On sites "for dummies" (and not quite for dummies), Sen2Cor is also not written about. The official documentation is confusing: suffice it to say that the “Installation” section on the 26th page of the “User Guide” immediately refers to the 60th page of Release Notes. It took me a while to figure it out.I decided to write an understandable guide to installing, starting and configuring Sen2Cor. The manual is aimed at biologists, ecologists, agricultural specialists, and all those who may need satellite image processing, but for whom it is not the main task. In fact, I share my own experience. I do not guarantee absolute correctness and completeness. Geoinformatics specialists, I think, will advise me something.Strictly speaking, Sen2Cor makes not only an atmospheric correction, but also a correction for the angle of the Sun and the terrain, as well as a thematic classification (scene classification), but for me the most important thing was the atmospheric correction. Therefore, for simplicity, I sometimes write “atmospheric correction” instead of “processing”, although this is inaccurate.Sen2Cor works in three modes: as a standalone console application, as a plugin in the Sentinel-2 Toolbox and as a console application used in PDGS (Payload Data Ground Segment). PDGS is a ground-based data center received from satellites. He is in Italy at a research institute. Theoretically, you can run Sen2Cor in the same mode in which it is launched in the data center, but to run you need data that just can not be downloaded on the Internet, so we will not consider this option. Sentinel-2 Tolobox is also bypassed, because I did not work with this program. It has a graphical interface, so I believe that understanding her work is not difficult. So, this article describes how to use Sen2Cor in console mode.Sen2Cor runs on 64-bit Windows, MacOS, and Linux. I have Linux, so I will show the commands using Linux as an example. However, the only difference is in the installers and in the file paths.Sen2Cor requires 4 GB of RAM. If less, it will start, but may crash in the middle of processing. The consumption of RAM depends on the resolution of the images taken, so if the memory is small and the program gives an error because of this, then you can start processing with a lower resolution. How to do this is described below. There are no similar strict requirements for the processor, but the weaker the processor, the more time the processing will take.Installation
You need to download the installer and run it. Sen2Cor is written in Python, however, the interpreter and all necessary packages are in the installer, so you do not need to install Python yourself and configure the environment.Suppose we have a user user with a home directory /home/user. Create a folder in the home directory spaceshotsin which we will work with the pictures. The full path to the folder will be /home/user/spaceshots.Next, download and run the installer. Open the Sen2Cor pageon the instrument site of the European Space Agency. There are two versions of the program: 2.5.5 and 2.8. Installing 2.8 is better: it consumes less RAM and contains less errors. Version 2.5.5 may be needed if you need to process pictures taken according to the old specifications. Click on "Sen2Cor v2.8". At the end of the page in the “Standalone Installers” section are links for downloading installers. Download the one that suits our operating system. In this case, it is “Linux installer: Sen2Cor-02.08.00-Linux64.run”. Save the file to /home/user/spaceshots.Next, open the console, move to the directory spaceshotsand run the installer.$ cd ~/spaceshots
$ ./Sen2Cor-02.08.00-Linux64.run
By default, the program is installed in the current directory. A folder is created in it Sen2Cor-02.08.00-Linux64, where all the program files are extracted. By and large, the installation comes down to unpacking the archive. In addition, in the home directory is created a folder for configuration files and logs: /home/user/sen2cor/2.8.If there are no errors, then a message should appear in the console «Congratulations, Installation successful...»and a suggestion to test the installation. Let's make a test run:$ /home/user/spaceshots/Sen2Cor-02.08.00-Linux64/bin/L2A_Process -h
This command /home/user/spaceshots/Sen2Cor-02.08.00-Linux64contains the directory where Sen2Cor was installed. The folder bincontains the main executable files of the program, including the main file L2A_Process. The option -hsays that you need to show the help page. As a result of the command execution, the following should appear in the console:usage: L2A_Process.py [-h] [--mode MODE] [--resolution {10,20,60}]
[--datastrip DATASTRIP] [--tile TILE]
[--output_dir OUTPUT_DIR] [--work_dir WORK_DIR]
[--img_database_dir IMG_DATABASE_DIR]
[--res_database_dir RES_DATABASE_DIR]
[--processing_centre PROCESSING_CENTRE]
[--archiving_centre ARCHIVING_CENTRE]
[--processing_baseline PROCESSING_BASELINE] [--raw]
[--tif] [--sc_only] [--cr_only] [--debug]
[--GIP_L2A GIP_L2A] [--GIP_L2A_SC GIP_L2A_SC]
[--GIP_L2A_AC GIP_L2A_AC] [--GIP_L2A_PB GIP_L2A_PB]
input_dir
Sentinel-2 Level 2A Processor (Sen2Cor). Version: 2.8.0, created: 2019.02.20,
supporting Level-1C product version 14.2 - 14.5.
…
If it appeared, the installation went fine and the program is ready for use. The only thing that still does not hurt to do is to simplify the launch of the program. While you need to specify the full path of the file: /home/user/spaceshots/Sen2Cor-02.08.00-Linux64/bin/L2A_Process. It is not comfortable. It is much easier to write simply L2A_Process. However, if you do this right now, an error will appear:$ L2A_Process
bash: L2A_Process:
To fix the error and run Sen2Cor on command L2A_Process, you need to add one command to the file .bashrc, which is located in the home directory (in our case, at the address /home/user/.bashrc). This is a hidden file, so before you open it, you need to enable the display of hidden files in the file manager.Add the .bashrcfollowing lines to the end of the file :
source /home/user/spaceshots/Sen2Cor-02.08.00-Linux64/L2A_Bashrc
The first line is a comment to remember why the second is needed. Each time the system starts, the second line executes the script contained in the file L2A_Bashrc. This script changes some environment variables so that the command L2A_Processbecomes accessible from everywhere, and you do not need to specify the full path to it.For the changes to .bashrctake effect, you need to return to the console and run the command$ source ~/.bashrc
You can also change the place where the program will be installed. To do this, you must specify the option when starting the installer --target. For example, the command
will install Sen2Cor in /usr/share/sen2cor. We run the command from the superuser, because an ordinary user, as a rule, cannot write files to /usr. However, in this case, then you will have to run Sen2Cor with superuser rights or change the access rights to /usr/share/sen2cor.How to install on MacOS and WindowsMacOS , Linux.
Windows — zip-, .
L2A_Process L2A_Process.bat,
bin, , :
C:\Users\user\spaceshots\Sen2Cor-02.08.00-Linux64/L2A_Process.bat -h
Windows :
:\Users\user\sen2cor\2.8. .
,
L2A_Process.bat, Windows
L2A_Bashrc.bat PATH.
Image processing
So, we installed Sen2Cor and made it run on command L2A_Process. Now make an atmospheric correction. Suppose that we have already downloaded and unpacked a satellite image into /home/user/spaceshots. I took a picture of the area west of Yekaterinburg on July 27, 2018. The snapshot file is called L1C_T40VFJ_A016162_20180727T071939.zip . After unpacking, the S2A_MSIL1C_20180727T071621_N0206_R006_T40VFJ_20180727T092607.SAFE folder appeared . Then for atmospheric correction you need to run the following command:$ L2A_Process S2A_MSIL1C_20180727T071621_N0206_R006_T40VFJ_20180727T092607.SAFE/
After pressing Enter, you can relax, because the processing takes a lot of time (on Yandex.Cloud with Intel Cascade Lake and a guaranteed share of 5% CPU - about an hour). Sen2Cor displays information about the progress in the console, so that you will roughly understand what the program is doing at the moment.Log start in consoleSentinel-2 Level 2A Processor (Sen2Cor). Version: 2.8.0, created: 2019.02.20, supporting Level-1C product version 14.2 - 14.5 started ...
Product version: 14.5
Operation mode: TOOLBOX
Processing baseline: 99.99
Progress[%]: 0.00 : Generating datastrip metadata
L1C datastrip found, L2A datastrip successfully generated
Selected resolution: 60 m
Progress[%]: 1.13 : PID-4202, L2A_ProcessTile: processing with resolution 60 m, elapsed time[s]: 2.299, total: 0:00:10.789933
Progress[%]: 1.13 : PID-4202, L2A_ProcessTile: start of pre processing, elapsed time[s]: 0.000, total: 0:00:10.790432
Progress[%]: 1.27 : PID-4202, L2A_Tables: start import, elapsed time[s]: 0.278, total: 0:00:11.068694
Progress[%]: 1.69 : PID-4202, L2A_Tables: band B01 imported, elapsed time[s]: 0.851, total: 0:00:11.919562
Progress[%]: 9.39 : PID-4202, L2A_Tables: band B02 imported, elapsed time[s]: 15.618, total: 0:00:27.537532
Progress[%]: 17.16 : PID-4202, L2A_Tables: band B03 imported, elapsed time[s]: 15.743, total: 0:00:43.280800
…
When the program finishes working, /home/user/spaceshotsnext to S2A_MSIL1C_20180727T071621_N0206_R006_T40VFJ_20180727T092607.SAFE/ , another folder with a very similar name should appear - S2A_MSIL2A_20180727T071621_N9999_R006_T40VF08that we got the program for now. To do this, consider the structure of the Sentinel-2 images.The Sentinel-2 satellite image is a zip archive of several hundred megabytes in size. In our case, the archive is called L1C_T40VFJ_A016162_20180727T071939.zip . Information is encoded in the name: processing level ( L1C ), territory code ( T40VFJ ), orbit number ( A016162), date and time of shooting. Inside this archive there is a folder with a long name that ends with .SAFE. In the * .SAFE folder there is a GRANULE folder, in it there is another folder with a long name, inside which is the IMG_DATA folder, and in it there are 14 images in jp2 format. Among them there are 13 numbered - their names end in B01-B12 (plus one more with the number B8A). These are tiles - images of the earth’s surface, taken at specific wavelengths. For example, B04 is a picture in the red range, and B08 is in the near infrared, and together these two pictures can be used to calculate such a well-known index as NDVI . The name of the 14th image ends with TCI, which means True Color Image - this is a full-color picture, that is, approximately the same as it would have turned out on a regular camera.Sen2Cor receives the * .SAFE folder as source data - it is it that is specified as the only required parameter when starting the program. If instead of the * .SAFE folder you specify something else, for example, the source archive with a snapshot or a specific tile, the program will not start.$ L2A_Process T40VFJ_20180727T071621_B04.jp2
Product metadata file cannot be read.
The fact is that not only the tiles themselves are used for atmospheric correction, but also metadata, and they are in the files inside the * .SAFE folder, and not in the tiles.The result of the program’s work is also a * .SAFE folder with a similar structure: inside there is a GRANULE folder, inside it is another folder in which the IMG_DATA folder is located, and there are corrected tiles in it. Tiles are grouped by spatial resolution: those with a spatial resolution of 10 meters are located in the R10m folder, those with a resolution of 20 meters in R20m and 60 meters in R60m. With certain settings, some permissions are not processed and the corresponding folders are not created - more on that below.The name of the * .SAFE folder resulting from it differs from the original name only in one: at the beginning instead of _MSI L1C_ indicated _MSI L2A _. L1C and L2A are processing level codes. The L1C level means that the image is processed to the Top-of-Atmosphere reflectance level, that is, the pixel values in the image correspond to the reflection recorded at the upper boundary of the atmosphere - where the satellite flies. The L2A level is processing to the Bottom-of-Atmoshpere reflectance level, that is, to the reflection values at the surface of the Earth - as if the atmosphere is absolutely transparent. In reality, of course, this is not so, and atmospheric correction allows you to remove only part of the distortions that occur when light passes through the atmosphere, but in general, the Bottom-of-Atmosphere reflectance values are more consistent with the actual reflection from the earth's surface than the Top-of-Atmosphere values reflectance.In addition to adjusted tiles, Sen2Cor produces several more images: full color (TCI), maps of aerosol optical density of the atmosphere (AOT - Aerosol Optical Thickness) and water vapor content (WV - Water Vapor), a map of thematic classification of surface types (SC - Scene Classification). They are also located in the IMG_DATA folder in directories for the corresponding spatial resolution.If you want the folder with the result of the program to be called not * .SAFE, but in a different way, then you can specify the name yourself using the option --output_dir. Suppose we run Sen2Cor three times with different settings and want to save the results to the folders test1, test2 and test3. Then we indicate --output_dirat startup:
$ L2A_Process --output_dir /home/user/spaceshots/test1 S2A_MSIL1C_20180727T071621_N0206_R006_T40VFJ_20180727T092607.SAFE
- and the corrected image will appear in the catalog /home/user/spaceshots/test1. At the second and third start, we indicate --output_dir /home/user/spaceshots/test2and, respectively --output_dir /home/user/spaceshots/test3.Another useful option is this --resolution. It allows you to set the spatial resolution of the tiles that will be obtained as a result of the program. Sen2Cor can create images with different spatial resolutions - 10, 20 or 60 meters per pixel. By default, 10 and 20 meters are made. Suppose we have enough tiles with a resolution of 60 meters per pixel for exploratory analysis. Then you can specify the target resolution using the option --resolution:$ L2A_Process --resolution 60 S2A_MSIL1C_20180727T071621_N0206_R006_T40VFJ_20180727T092607.SAFE
When launched with these parameters, Sen2Cor will process the image and create adjusted tiles with a resolution of 60 meters. Unlike a standard launch, processing takes much less time - a little more than four minutes. The memory consumption is also less, so if the start with a standard resolution ended in an error due to lack of memory, then you can specify a resolution of 60 meters. Only the R60m folder will be located in the IMG_DATA folder, and inside it there will be tiles with a resolution of 60 meters and some other images: a full-color, thematic map and a map of the content of water vapor.Configuration file
We processed the satellite image using Sen2Cor using the default settings. The resolution and folder for saving work results can be changed using the --resolutionand options --output_dir. There are a few more command line options, however, not all can be configured with them. There are many other parameters that are important in processing, for example: whether to take into account the topography, whether to remove clouds, how to assess the state of the atmosphere. In Sen2Cor, they are configured using the configuration file.If you run Sen2Cor and do not specify the path to the configuration file, as we did before, then the standard configuration file will be used. The standard configuration file is in the home directory - in our case, in the / folder home/user/sen2cor/2.8/cfg. The file is calledL2A_GIPP.xml. L2A stands for processing level, GIPP stands for Ground Image Processing Parameters - parameters for ground processing of the image. This is a regular xml file, inside which there are several sections, and in sections there are elements for different settings. An explanation is written directly in the file for each setting. In addition, a detailed description of the settings is contained in the documentation - however, it is not easy to understand it.You can edit the standard configuration file directly, but it is better to copy it and explicitly indicate when you start the program that you need to use a different configuration file. Copy L2A_GIPP.xml to /home/user/spaceshots, rename it to custom.xml and run the program with this configuration file:$ L2A_Process --GIP_L2A /home/user/spaceshots/custom.xml --resolution 60 S2A_MSIL1C_20180727T071621_N0206_R006_T40VFJ_20180727T092607.SAFE
Please note that the option for the configuration file is called --GIP_L2A- with one letter "P". Since we did not edit the configuration file, the process and the result of the program are no different from starting with the default configuration.There are many different parameters in the configuration file. Let's consider some of them.Logs
Log_Level- a parameter that determines how detailed the log of the program will be. The log is displayed in the console during operation and is additionally stored in the log folder (in our case - /home/user/sen2cor/2.8/log). The default value is INFO. Usually there is no need to change this setting if you have not encountered errors in the program.Accounting for the relief and angle of the sun
DEM_Directory- the folder in which the digital elevation map is located (DEM - Digital Elevetion Model). Processing results using a digital elevation map are usually more accurate. The default is NONE, i.e. the terrain is not taken into account. If you have your own digital elevation map, then indicate the path to it. The folder for DEM should be in the same place as the folders cfgand log, in our case, in the directory /home/user/sen2cor/2.8. Therefore, if you have a digital elevation map, you need to put it in /home/user/sen2cor/2.8/demand specify dem in DEM_Directory. If it is not there, but you still want to use it, you can specify any path (for example, write dem too) and configure the next parameter.DEM_Reference- URL from which to download the digital elevation model, if it is not in the folder specified in the previous paragraph. Sen2Cor developers suggest usinghttp: // data_public: GDdci@data.cgiar-csi.org/srtm/tiles/GeoTIFF/
- This is a link to STRM DEM, a freely distributed digital elevation map with 90-meter spatial resolution. Given that the scale is not much smaller than the satellite images themselves, this model should give a good result.If Sen2Cor sees DEM_Directorythat a value other than NONE is specified, it will try to open files with a bump map from the specified directory. If the files cannot be opened, he will try to download them at the URL specified in DEM_Reference. If it fails to download, an error message will appear in the log and processing will continue without taking into account the terrain. Files downloaded from the URL are saved inDEM_Directoryand are not deleted after the program terminates. Therefore, if you want to process the same image again with the same elevation map (for example, by changing other correction settings), Sen2Cor will not download the digital elevation map again, but will use its offline copy.The option Generate_DEM_Outputallows you to get a separate tile with a digital elevation map as a result of the program. The default is FALSE.The option DEM_Terrain_Correctionpartially disables the use of a digital elevation map: the elevation will continue to be taken into account in thematic classification (SC) and the construction of the AOT map, but not when adjusting reflection values from the surface.If DEM is not used, then specify the parameterAltitude- This is the average height above sea level in the area captured in the picture. Height is indicated in kilometers.Two other parameters related to elevation accounting are BRDF_Correctionand BRDF_Lower_Bound. BRDF stands for bidirectional reflectance distribution function and means an equation that simulates the reflection of light from an opaque surface. Different values BRDF_Correctioncorresponds to different values of one of the coefficients in this equation. In the official documentation, it is recommended in most cases to use 21. It BRDF_Lower_Boundis responsible for another coefficient in the equation, and it can be left by default, unless you know exactly why and why you need to change it.Beautiful picture
The option Generate_TCI_Outputturns on and off the creation of a full-color image. The default is TRUE, but if a beautiful picture is not needed, then you can select FALSE.Accounting for the state of the atmosphere
The parameters Aerosol_Type, Mid_Latitude, Ozone_Contentindicate the state of the atmosphere at the time of the shooting. Aerosol_Typecan be set on the territory: if the continent, then RURAL, if the ocean, then MARINE. Mid_Latitudesimilarly determined by the season: if in the summer, then SUMMER, if in the winter, then WINTER.The parameter Ozone_Contentindicates the ozone content. Ozone content can be found on the World Archive of Ozone and UV Light Online Archive. On the map you can find the station closest to the survey area, then download the data set for the right time and see the measurement results on the day of shooting. However, it is impossible to establish the actual measurement result in the configuration file - you need to select one value from the list for the corresponding time of the year. Therefore, you must first find out the actually measured value, and then choose the closest available. For example, according to the station in Yekaterinburg , on June 27, 2018 (time taken), the ozone content was 316 DU. The explanation in the configuration file states: for summer in the middle latitudes, the available options are 250, 290, 331 (default), 370, 410, 450. The closest to 316 is 331. It comes by default, so in this case nothing can be done do not change. If the picture was taken a few days earlier, when the ozone concentration decreased to 302 DU, should be set Ozone_Contentto 290.In principle, all three parameters - Aerosol_Type, Mid_Latitude, Ozone_Content- in most cases can be set by default - for the AUTO Aerosol_Typeand Mid_Latitudeand 0 for Ozone_Content. In this case, Sen2Cor will determine the most suitable values from the image metadata.Cloud removal
The parameter Cirrus_Correctionsays whether or not cirrus clouds are removed from the image. Works in conjunction with WV_Threshold_Cirrus- this is the water vapor content threshold below which cloud removal is automatically disabled, even if Cirrus_Correctionset to TRUE. Therefore, if there is reason to believe that the water vapor content in the atmosphere at the time of shooting was low, but there are still small clouds, and they need to be removed, then it is better to reduce the value WV_Threshold_Cirrus. WV_Threshold_Cirruscan be from 0.1 to 1.Conclusion
Thus, in this article we examined the following issues: how to install Sen2Cor, how to start it with the default settings, how to change some parameters of the program using the command line and configuration file options. For a more in-depth study of Sen2Cor, you can refer to the official documentation, which is available on the same page as the installers. The official documentation includes four documents: the simpler and clearer Release Note and Software User Manual, and the rather sophisticated technical Product Definition and Input Output Data Definition. The latter file also includes “The full reference of GIPP” as an attached pdf file. I believe that technical experts will find much more useful in these documents.