GitHub

Other output

Next to the NetCDF output there's other ways to output data from the model, these are described below.

JLD2 Output

As an alternative to the NetCDF output, it is also possible to directly output the PrognosticVariables and DiagnosticVariables to a JLD2 file. This might be interesting if you are really interested in the model internals, or also for some machine learning tasks. However, this option doesn't feature any possibilites to regrid or select variables, and it comes with the usual limitations of serialized JLD2 data: SpeedyWeather.jl always has to be in the scope when loading the data and the saved files might only load properly with exactly the same version of SpeedyWeather.jl and Julia as used when saving the data. Its usage is similar to the NetCDF output above:

using SpeedyWeather
spectral_grid = SpectralGrid(trunc=31, nlayers=1)
output = JLD2Output(output_dt=Hour(1))
model = ShallowWaterModel(spectral_grid, output=output)
model.output
JLD2Output
├ status: inactive/uninitialized
├ write restart file: true (if active)
├ path: output.jld2
└ frequency: 3600 seconds

With all options shown below 

@doc JLD2Output

Output writer for a JLD2 file that saves the PrognosticVariables and DiagnosticVariables structs directly to a JLD2 file. All internal scalings and units are still applied to these outputs. Fields are

  • active::Bool

  • path::String: [OPTION] path to output folder, run_???? will be created within

  • run_prefix::String: [OPTION] Prefix for run folder where data is stored, e.g. 'run_'

  • id::String: [OPTION] run identification, added between runprefix and runnumber

  • run_number::Int64: [OPTION] run identification number, automatically determined if overwrite=false

  • run_digits::Int64: [OPTION] run numbers digits

  • run_folder::String: [DERIVED] folder name where data is stored, determined at initialize!

  • run_path::String: [DERIVED] full path to folder where data is stored, determined at initialize!

  • overwrite::Bool: [OPTION] Overwrite an existing run folder?

  • filename::String: [OPTION] name of the output jld2 file

  • write_restart::Bool: [OPTION] also write restart file if output=true?

  • write_parameters_txt::Bool: [OPTION] also write parameters txt file if output=true?

  • write_progress_txt::Bool: [OPTION] also write progress txt file if output=true?

  • output_dt::Second: [OPTION] output frequency, time step

  • merge_output::Bool: [OPTION] will reopen and resave the file to merge everything in one big vector. Turn off if the file is too large for memory.

  • output_prognostic::Bool: [OPTION] output the PrognosticVariables

  • output_diagnostic::Bool: [OPTION] output the DiagnosticVariables as well

  • output_every_n_steps::Int64

  • timestep_counter::Int64

  • output_counter::Int64

  • jld2_file::Union{Nothing, JLD2.JLDFile}

Parameter summary

With output=true as an argument in the run!(simulation) call, the NetCDFOutput by default also writes a parameter summary into parameters.txt in the same folder. This is implemented as a Callbacks (<: SpeedyWeather.AbstractCallback) and can be added independent of NetCDFOutput too. After output=true this callback is found here as ParametersTxt

# run an example simulation with output
simulation = initialize!(model)
run!(simulation, period=Hour(4), output=true)

# now model.output will have added additional "output" writers
simulation.model.callbacks
Dict{Symbol, SpeedyWeather.AbstractCallback} with 3 entries:
  :restart_file   => RestartFile <: AbstractCallback…
  :progress_txt   => ProgressTxt <: AbstractCallback…
  :parameters_txt => ParametersTxt <: AbstractCallback…

but it's activity is by default tied to activity of the NetCDFOutput with you can control with write_only_with_output. Creating such a callback independently

parameters_txt = ParametersTxt(write_only_with_output=false)
ParametersTxt <: AbstractCallback
├ path::String = 
├ filename::String = parameters.txt
└ write_only_with_output::Bool = false

we can add it with a random or manual key as

add!(model, parameters_txt)             # random key
add!(model, :my_key => parameters_txt)  # manual key

But note that callbacks are overwritten with identical keys, otherwise treated independently. Meaning now we have the preexisting :parameters_txt callback and then the two we just added, writing their parameters.txt files one after another, overwriting that same file two times.

Progress txt

Similarly to ParametersTxt, a callback ProgressTxt is by default added with output=true. They can be created independently too

progress_txt = ProgressTxt(write_only_with_output=false, path="myfolder", filename="letsgo.txt")
ProgressTxt <: AbstractCallback
├ path::String = myfolder
├ filename::String = letsgo.txt
├ write_only_with_output::Bool = false
├ every_n_percent::Int64 = 5
└ file::IOStream = IOStream()

and added like

add!(model, :progress_txt => progress_txt)

Restart file

NetCDFOutput also by default writes a restart file, containing the simulation.prognostic_variables that can be read back in with teh StartFromFile initial conditions. Implemented as a callback RestartFile can also be created independently of NetCDFOutput, e.g.

restart_file = RestartFile(write_only_with_output=false, path="folder1", filename="restart.jld2")
RestartFile <: AbstractCallback
├ filename::String = restart.jld2
├ path::String = folder1
├ compress::Bool = true
├ write_only_with_output::Bool = false
└ pkg_version::VersionNumber = 0.17.3

and added like

add!(model, :restart_file => restart_file)

By default path="" will use the folder determined by NetCDFOutput but otherwise you can also provide your own.