Processing on AWS #107
Comments
|
Hi @rodolfolotte, |
|
Hey John @johntruckenbrodt ! Thank you so much! I will have a look! Had no idea about this... Many thanks for updating! |
|
Sure, you're welcome. I am closing this for now but feel free to reopen it if you encounter something similar. |
|
@johntruckenbrodt , I'm sorry! Just one more question! The stacking example provided in the repository (see below), does not make any adjust to geocode the images. Why in this case the geocode method is not required? This example is runnable or it is just there to give "an idea" how to use? The geocode method demands the So, my doubt is: I do need to use SNAP to search and download S1 images, or I'm still missing something. PS. I'm currently using S3 AWS as source. |
|
Hi. This example stacks images already processed with pyroSAR geocode. The following line collects all images in VV polarization which have been topographically normalized ( srcfiles = finder(dir_in, ['S1*_VV_*norm_db.tif'])This example is runnable as long as you have a directory containing several images processed by function geocode. |
|
Hi @johntruckenbrodt Thank you! Look, my function is like: where Then, I get this message: I just debuged! Seems to be something regarding the image projection, or probably the file extension! Because the empty variable I'm correct? |
|
A SAR scene usually consists of one or more image files and in most cases several text files containing metadata. If you want to process a SAR scene with the from pyroSAR.snap import geocode
from pyroSAR.ancillary import groupbyTime, seconds
from spatialist.ancillary import finder
list_images = finder('/location', ['*.SAFE'], foldermode=2)
list_images = groupbyTime(images=list_images, function=seconds, time=30)
for group in list_images:
geocode(group, ...)The function spatialist.finder is just for demonstration, but seems not to work for AWS S3 yet as discussed in issue johntruckenbrodt/spatialist#10. |
|
@johntruckenbrodt Ok! Then, i will have a look at SAFE packages in S3 buckets. If I get it done, I will post it here! Thank you so much! |
|
Well, almost there! Hey, @johntruckenbrodt, how are you? I'm sorry to insist on this, but I think we will have a nice contribution after link pyroSAR to use together with AWS repositories. Well, I could not find any other bucket in S3 that could provide to be .zip or SAFE formats, even in Once the folders are done, I compressed (.zip) each of them and finally got the inputs to use Thank you so much! |
|
@johntruckenbrodt I just found the problem! I was not fully agreement to the SAFE format. The following files: were nominated incorrectly (final integer). For this reason, the SNAP, either Before doing some reverse engineering, just asking: would you mind to share the details (theory, not code) of each workflow steps (could be papers, or any other link): Subset, ThermalNoiseRemoval, Apply-Orbit-File, Calibration, Multilook, Terrain-Flattening, Terrain-Correction, and LinearToFromdB? Thank you! |
|
Probably here:
;) |
|
Hi @rodolfolotte, from pyroSAR import identify
location = 's3://sentinel-s1-l1c/GRD/2018/11/11/IW/SV/'
name = 'S1B_IW_GRDH_1SSV_20181111T171503_20181111T171532_013559_019191_0F09'
scene = identify(location + name)
print(scene)How are you searching for scenes in the S3 buckets? It would be very convenient if a user could just search for scenes based on some parameters, e.g. polarization and acquisition date, and get the result as a list that can then be passed to Cheers, |
johntruckenbrodt
changed the title
|
Hi @johntruckenbrodt, how are you? Ok, I will give all insights about the "mismatchings" between Yes, you are right! The respective bucket offers all the individual itens (not SAFE format, either .zip packages), but with different names when you compare to the name conventions (see here). For example, the measurements brings the image discriminated by polarization and mode: I have checked the bucket Then, I thought I should have this done but many other small issues have also to be adapted! First, there is a problem with to get the SAFE folder compressed. But I got that error massage. But if I compress with Ubuntu right click, then I want to share all what I've reached, although it is not tested at all (I mean, Cheers! Many thanks! |
|
Hi @rodolfolotte, from pyroSAR.drivers import SAFE
name = 'S1B_IW_GRDH_1SSV_20181111T171503_20181111T171532_013559_019191_0F09'
scene = SAFE(name)
print(scene)The function Cheers, John |
|
Hello @johntruckenbrodt, everything is fine! Yeah, pretty tough moment we are living! :/ Ok, so... I've created a new repository to search and download images from AWS. It s available here.
but if I pass image by image, then I get all the procedures done: Now, I'm facing a problem with stacking: First, I thought it could be an overlapping issue, but it seems to be something else. I'm debbuging and still did not get a workaround! What I truly want to do with all this, is to get the mean of a set of images over a specific AOI. Let you know! ;) Cheers! |
|
Hi @rodolfolotte, list_images_groups = groupbyTime(images=list_images, function=seconds, time=30)
for job in list_image_groups:
geocode(job, ...)One suggestion: use output_path_aux = output_path + "/group-images-" + str(i) + "/"do this: output_path_aux = os.path.join(output_path, "group-images-{}".format(i))This way it works on all operating systems. Cheers, |
Just did a small example for stacking Sentinel-1 images, but something is still wrong!
The method is mostly what is presented in this repository's example:
Just to make clear, the
srcfilesvariable brings all original sentinel images:['.../iw-vh/S1A_IW_GRDH_1SDV_20180525T093134_20180525T093159_022059_026254_0ED2_iw-vh.tiff', '/.../iw-vh/S1A_IW_GRDH_1SDV_20180525T093159_20180525T093224_022059_026254_DE25_iw-vh.tiff', '.../iw-vh/S1A_IW_GRDH_1SDV_20180525T093224_20180525T093249_022059_026254_28B8_iw-vh.tiff', '.../iw-vh/S1A_IW_GRDH_1SDV_20180530T093955_20180530T094020_022132_0264A6_04BA_iw-vh.tiff', '.../iw-vh/S1A_IW_GRDH_1SDV_20180530T094020_20180530T094045_022132_0264A6_A27A_iw-vh.tiff', '.../iw-vh/S1A_IW_GRDH_1SDV_20180530T094045_20180530T094110_022132_0264A6_D444_iw-vh.tiff']but still getting this error:
Seems to be some projection missing data! But I'm using original image and check with another softwares, which gave me the correct metadata!
The text was updated successfully, but these errors were encountered: