Opened 3 years ago
Last modified 14 months ago
#298 accepted Support
Running preprocessing with previously loaded grib files
Reported by: | tcarion | Owned by: | anphi |
---|---|---|---|
Priority: | minor | Milestone: | flex_extract_v7.1.3 |
Component: | flex_extract | Version: | flex_extract_v7.1 |
Keywords: | Cc: |
Description
Hello,
When I'm running flex_extract, I sometimes get an error at etadot calculation :
Flex_extract error:
Command '['{fe_path}/flex_extract_v7.1.2/Source/Fortran/calc_etadot']' died with <Signals.SIGSEGV: 11>.
I don't know where it comes from, and for debugging purposes, I would like to rerun flex_extract without re-downloading the grib files through mars request, since these files are already available.
Unfortunately, reading the (very good) documentation, I didn't see any way of running only the pre-processing phase of the raw grib files given by mars.
So I wanted to know if this feature could be added (or if it already exists but I'm not aware).
Thank you very much in advance.
Tristan Carion
Attachments (2)
Change History (16)
comment:1 Changed 3 years ago by anphi
- Owner set to anphi
- Status changed from new to accepted
comment:2 Changed 3 years ago by anphi
- Type changed from Enhancement to Support
comment:3 Changed 3 years ago by anphi
comment:4 Changed 3 years ago by tcarion
Dear Anne,
It worked, thank you! I got the same error as before, but at least I know how to run flex_extract with downloaded data files. Thank you very much for your quick support.
Kind regards,
Tristan
comment:5 Changed 3 years ago by anphi
You could send me your CONTROL file and the log output. Then, I'll have a quick look to see if I notice the problem right away. Which mode do you use? Local public? Or Remote (Gateway)?
comment:6 Changed 3 years ago by tcarion
The files are attached. I'm using the local mode with operational data.
comment:7 Changed 3 years ago by pesei
I see that you already use ETA 1, so calc_etadot is not used to calculate the vertical velocity, but it just combines the GRIB files into a single one.
It would be useful to know what still causes the program to crash. Maybe you don't have enough memory? Or you have limits in place that are too low? Does it happen immediately? If it's not memory, please recompile in debug mode and post the output.
comment:8 Changed 3 years ago by anphi
Ok, my best guess after a quick_look is, that these lines in the CONTROL file cause the problem:
TYPE AN FC FC FC FC FC FC FC FC FC FC FC FC FC FC TIME 00 00 00 00 00 00 00 00 00 00 00 00 12 12 12 STEP 00 01 02 03 04 05 06 07 08 09 10 11 00 01 02
Flex_extract is NOT trained to work with data in between a full day or half a day. Did you do this intentional or is it an accident that you tried to get data from 0am to 2pm?
I also think that this special definition of FC TIME and FC STEP without an AN field in between could cause a problem. I never used it this way.
You should try to use the full 24 hours like this:
TYPE AN FC FC FC FC FC FC FC FC FC FC FC AN FC FC FC FC FC FC FC FC FC FC FC TIME 00 00 00 00 00 00 00 00 00 00 00 00 12 12 12 12 12 12 12 12 12 12 12 12 STEP 00 01 02 03 04 05 06 07 08 09 10 11 00 01 02 03 04 05 06 07 08 09 10 11
or
for half a day (you need to use the BASETIME parameter then):
TYPE AN FC FC FC FC FC FC FC FC FC FC FC AN FC FC FC FC FC FC FC FC FC FC FC TIME 00 00 00 00 00 00 00 00 00 00 00 00 12 12 12 12 12 12 12 12 12 12 12 12 STEP 00 01 02 03 04 05 06 07 08 09 10 11 00 01 02 03 04 05 06 07 08 09 10 11 BASETIME 00
Let me know if this works. When this solves the problem, I will need to provide a fix to catch this.
comment:9 Changed 3 years ago by pesei
Oh yes, I think you can't use TYPE=FC with STEP=00 - there is no forecast at time zero! It is just strange that the error only happens in calc_etadot and not earlier.
comment:10 Changed 3 years ago by tcarion
I tried again with TYPE=AN at STEP=00 and it worked, even with getting data from 0am to 2pm. I didn't know flex_extract wasn't trained with data in the middle of the day, I will adapt the way I use it in the future.
Thank you for your support !
comment:11 Changed 3 years ago by anphi
- Milestone set to flex_extract_v7.1.3
comment:12 Changed 2 years ago by pesei
- Resolution set to fixed
- Status changed from accepted to closed
Looks like fixed.
comment:13 Changed 2 years ago by pesei
- Resolution fixed deleted
- Status changed from closed to reopened
Reopen, because it might be useful to catch this condition.
comment:14 Changed 14 months ago by anphi
- Status changed from reopened to accepted
Dear Tristan,
the feature you are looking for is already implemented. I just didn't put the instructions into the documentation yet. I'll have to do this soon.
However, to do the post-processing only, you can use the python script prepare_flexpart.py which is located in {fe_path}/Source/Python/Mods/prepare_flexpart.py.
This was modified so it can be used as a standalone script in addition. It can take the following parameters:
Use this script within the directory where you have the downloaded data files.
Copy the CONTROL file you used into this directory and then call the script with:
Does it work for you?
Best regards,
Anne