table of Contents
0 PEST++ series of articles
[PEST++] 01 Parameter optimization and sensitivity analysis
[PEST++] 02 Automatic calibration of Xin'anjiang model parameters
[PEST++] 03 Uncertainty and sensitivity analysis of hydrological model
[PEST++] 04 User manual notes
This article records the communication process between three friends when they used PEST++, and I attach it here. Hope it will be helpful for readers who learn PEST++. Thank you three friends for their authorization.
1 First friend ZJ
Q: In the tsproc.datfile, when importing the observation value and the measured value sequence, the time range of the observation value must be less than or equal to the time range of the measured value, right?
A: Yes. Only in this way can the observed value be interpolated to the same sequence of the measured value, which is convenient for accuracy comparison and statistics. tsproc.datThe file is used to deal with the situation that the output of the model is a time series, for example, the output of the model is a flow process, a water level process, etc. And it can automatically generate control files necessary for PEST++ automatic calibration .pst. If the output result of the model is only a few values and does not constitute a time series, there is no need to write a tsproc.datfile, and you can directly write a control file according to the requirements of PEST++ .pst.
Q: How to run the calibration program? What is the command input?
A: Both pestpp.exeand tsproc.exeare run in the command line (Command Line), you cannot double-click to run. After configuring the required files for calibration, open the command line tool in the folder where the model program is located (hold down the shiftkey, right-click to select powershell), and the command to run the time series processing program is
.\tsproc.exe tsproc.dat tsproc.rec
The command to run the automatic calibration program is
.\pestpp model.pst
It is recommended to write the command in a batch script .bat, such as the one given in the example 自动率定.bat, so that you do not need to enter the command every time you run it.
Q: Where can I view the calibration results?
A: To .parview the calibration result in the file, open the Notepad, the first column is the calibration value, you don't need to read it later. .senFile viewing sensitivity. As shown in the figure, the value of this column indicates sensitivity. The larger the value, the more sensitive it means that this parameter has a greater impact on the target.
Q: batch.batWhy are these two scripts missing pestpp model .pst?
A: It batch.batis executed repeatedly while the program is running. It is pestpp.execalled by itself , so it can't be written pestpp model.pst. 运行.batNo error will be reported when you run it directly , you can use the console to debug it first, and then use it if there is no error 运行.bat. The main thing is to save every time you paste.
Q: What does this place mean?
A: This means that there will be 44 more runs this time.
Q: If the result of automatic calibration is brought back to the original model and the effect is not good, how should I improve it? Rate it again?
A: When calibrating, the comparison between the measured value and the simulated value cannot take the entire calculation period, and the preheating part needs to be proposed. If the result is still not good, try group.txtto reduce the magnification in the parma.txtmiddle and increase the middle parameter range. If the calibration effect is still not satisfactory, it is the problem of the model itself.
2 Second friend LGH
Q: Where is the problem with this template file?
A: There must be a half-width (English) space between ptf and # in the first line
Q: The question here is that there is a problem with #dk? Can't the following space be increased so much?
A: According to the error message, dkwhat you enter later is the Tabkey, not the space. In this way, pestpp will recognize the dkfollowing Tabtogether as a variable, rather than just dkthis parameter. So you should replace the dkfollowing Tabkeys with half-width spaces. This problem .tploften occurs after using Excel to process data and pasting it into a template file . I hope readers will pay attention.
Q: param.txtIs there any problem with the parameter file ?
A: Remove the # of bp, group.txtand add four groups in it, that is gp1 gp2 gp3 gp4, like the gp1same.
Q: I added the group before, is it wrong?
A: Yes, you should use 0.1 or 0.01 for the next four. 0.01 here refers to multiples, which means that the initial value of the parameter is increased or decreased by 0.01 times each time the program is run.
Q: After the program runs, so many files are output, which file is the result in?
A: model.parInside, the model.recdifference between the simulation and the actual measurement is recorded. .par1 .par2 .par3This type refers to the optimal parameters obtained every time the program is run during the calibration process. The final optimal parameters are in the .parfile. The file name may not be model, but the suffix must be .par.
Q: Is the result of calibrating parameters in this column?
A: Yes.
Q: Same as the initial value given, is the calibration failed?
A: You give a method to calibrate the slot rate for each section, and it is difficult to get the result, because the change of slot rate has little effect on the result. It is recommended that you set the slot rate in sections.
Q: The simulation results of the hydrodynamic model have preheated parts, which are put in the input file. Do they need to be deleted?
A: It is best not to use the preheating part as the calibration part.
Q: How do you identify the data output from the running model to compare with the measured value?
A: In this question, the measured value file is placed in observationstage.ssfit. In order to match the model output file stage.ssfwith the measured value file observationstage.ssf, it is necessary to tsproc.datwrite in the file to compare the data from these two files.
Q: After running the program, an error will be reported. Is there a problem with my input file?
A: Do not have special characters such as Chinese and spaces in the path name, and the text file format is changed to ANSI format.
Q: I only monitored the water level of one section before. If I need to add the flow of another section to the comparison, what needs to be modified? Is tsproc.datthere something to be added?
A: Then you have to output the flow of that section, and then write a file of the measured value of the flow, and finally tsproc.datimport the model into the file to output the flow value.
Q: Does the question here correspond to the part on the right?
A: No, the right is interpolated. It may be that your calculation time step is shorter and the memory is not enough. Or the time step of the measured value is shorter.
Q: model.recSome of the contents are not clear what does it mean? Like here
A: This file is the entire calibration record of PESTPP. These parameters are set in the .pst file. The .pst file is a control file necessary for the operation of PESTPP, which is automatically generated by tsproc.exe running tsproc.dat. You can open the .pst file to view these parameters. As for the physical meaning of the parameters, you need to read the manual of pestpp, which is usually the default.
Q: Here is the average error of the optimal result, right?
A: This records the comparison between the actual measured value and the simulated value. It is an important result. It can be directly imported into excel to draw graphs to visually compare the calibration effect.
3 Third friend XY
Q: Does Xin'anjiang parameter file follow this?
A: What I wrote in my blog is an example, so I don't actually need to do it like this. General type selection noneis fine.
Q: Did you write bat all by yourself?
A: I wrote it myself, just three sentences.
Q: The parameter is used for 4 years for calibration and 2 years for verification. Why? For example, why is it not a 5-year rate, 1-year verification, or 3+3?
A: This is my own decision, you can choose other combinations.
Q: flow.ssfWhat traffic file is it?
A: The .ssfsuffix is the data format required by PESTPP, which can be opened with Notepad. This is the data that the model simulates to flow.
Q: The PEST I installed doesn't seem to see these two files in your package, pestpp.exeand tsproc.exe?
A: I use PEST++, which is an upgraded version of PEST. It is much simpler and more convenient to use than PEST. There is only an automatic calibration program without installation pestpp.exeand a time series processing program tsproc.exe.
Q: tsproc is a tool that converts the difference between simulated values to match the measured values?
A: Yes, it matches the time. At the same time, it can also automatically generate the control file required by pestpp with the suffix name .pst.
Q: These three batch files batch.batare used for parameter estimation, right? batch.bat should be used for inversion? Is it used to set the direction of the rate? Or for dimensionality reduction?
A: It batch.batis automatically called by the program, and no one needs to do it. batch.batIn fact, pestpp is automatically called every time the calibration is performed to xaj.exeplay this role.
Q: allIs n't this ?
A: Either all or test is used for verification.
Q: What is the use of writing test.txt? What is the com_test file for?
A: test.txt is to verify, and xaj.out is the same type of file. It is used for identification of pestpp, you can open it in Notepad. com_test is to write the statistical value of each deviation
Q: What about record.txt, is this something to be configured?
A: The record.txt is automatically output by pestpp and the running record does not need to be manually configured.
Q: Is it run like this in the dos window?
A: Do not use DOS, just click 自动率定.batto start the calibration.
Q: Can I separate into folders like this? No need to configure the environment and path?
A: No, it cannot be separated, it must be in one folder, and the path does not need to be configured.
Q: Why does your article say it takes an hour, and I run it in a few minutes?
A: The running time problem may be that your computer is not installed matlaband cannot run xaj.exe. My Xin'anjiang model is matlabwritten.
Q: Is there only one independent PEST++ pestpp.exe? Why is there so much PEST?
A: Yes, yes, there is only one pestpp.exe, nothing else, no need to install, just put it in the folder, you can use it directly, you don’t need to open the console, just click auto-calibrate.bat to automatically rate deal. pestI haven't read the documents, these seem to separate many functions into multiple exe.
Q: Where did your one come tsproc.exefrom? It doesn’t seem to be in the pest package?
A: This is developed by USGS, specifically for processing time. This is the tsproc document .
Q: I can't find a regularity in the naming of pests. For a while++, for a while PP?
A: pestpp = pest plus plus = pest ++ = PEST++, which is actually the same.
Q: I don't understand these selections. What is the basis?
A: I have explained the selection of parameters in the blog, and I suggest you take a closer look. The above is what I used as an example. Do this when necessary. Generally, just use it without operation none.
Q: Is the% representative an explanation? Is the content of the parameter explained later in Chinese or English? Should be consistent with the model, right?
A:% is not an explanation. It doesn't matter what words follow, as long as the corresponding position is marked in the template file.
Q: Is the out file of xaj a text file? flow.ssfWhere did the file come from?
A: It xaj.outis a text file that matches the model output file flow.ssfwith the obseravedflow.ssftime series of the measured value . flow.ssfIt is the model output file. The three columns are the identifier, time and flow value. This format is xaj.exemodified by me because I pestppneed this format to identify the data. pestppSeveral common formats are given, which are included in the pestpp 4.2.1 manual. Page 27 of this document introduces some of the data file types that tsproc can handle. You can see if there are any. For example, this is with HSPF and Modflow and other models. If not, you have to write a script to convert it to SSF format. Although pestpp does not require programming, it has high requirements on the output file format, which is also a troublesome place for pestpp. If you can modify the output format directly from the model source code, it will be much faster, and it will be much more efficient than converting the output file.
Q: And this ssf file is the output file format of xaj, right? Isn't it the Pest file format requirement?
A: SSF is a requirement of PESTPP. I modified the source program of xaj so that it can directly output this format to meet the requirement of PESTPP.
Q: What exactly does this do in CONTEXT? Is it the character name in the xaj model?
A: The function of CONTEXT is to specify which modules are readable. tsproc.datThere is a CONTEXT keyword at the beginning of the file, and the following modules will be executed if they are consistent with the keyword. When running tsproc.exe, you can also specify keywords to override the keywords in the file. The use of keywords is to write multiple modules into one tsproc.datfor easy editing.
Q: How to run this, the software window of the model pops up?
A: Because this needs to run the model multiple times, in fact it is to automatically change the input parameters, then run the model multiple times, and then automatically compare it. Pestpp calibrates this task. This calibration actually does not require a graphical user interface exe, preferably a model exe with a black screen console.