coupled model configuration generation

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for soroosh.mani.noaa from gravatar.com soroosh.mani.noaa Avatar for zacharyburnett from gravatar.com zacharyburnett

Unverified details

These details have not been verified by PyPI
Project links
Meta

Project description

CoupledModelDriver

tests build version license style

coupledmodeldriver generates an overlying job submission framework and configuration directories for NEMS-coupled coastal ocean model ensembles.

It utilizes nemspy to generate NEMS configuration files, shares common configurations between runs, and organizes spinup and mesh partition into separate jobs for dependant submission.

Supported models and platforms

  • models
    • circulation models
    • forcings
      • ATMESH
      • WW3DATA
  • platforms
    • local
    • Slurm
      • Hera
      • Stampede2

Usage

Example scripts can be found at examples/<platform>

1. generate configuration directory

The following code (examples/hera/hera_shinnecock_ike.py) creates a configuration for coupling (ATMESH + WW3DATA) -> ADCIRC on Hera, over a small Shinnecock Inlet mesh:

from datetime import datetime, timedelta
from pathlib import Path
import sys

from adcircpy import Tides
from adcircpy.forcing.tides.tides import TidalSource
from adcircpy.forcing.waves.ww3 import WaveWatch3DataForcing
from adcircpy.forcing.winds.atmesh import AtmosphericMeshForcing
from nemspy import ModelingSystem
from nemspy.model import ADCIRCEntry, AtmosphericMeshEntry, WaveMeshEntry

sys.path.append((Path(__file__).parent / '..').absolute())

from coupledmodeldriver.adcirc import write_adcirc_configurations
from coupledmodeldriver.job_script import Platform

# paths to compiled `NEMS.x` and `adcprep`
NEMS_EXECUTABLE = '/scratch2/COASTAL/coastal/save/shared/repositories/ADC-WW3-NWM-NEMS/ALLBIN_INSTALL/NEMS-adcirc_atmesh_ww3data.x'
ADCPREP_EXECUTABLE = '/scratch2/COASTAL/coastal/save/shared/repositories/ADC-WW3-NWM-NEMS/ALLBIN_INSTALL/adcprep'

# directory containing input ADCIRC mesh nodes (`fort.14`) and (optionally) mesh values (`fort.13`)
MESH_DIRECTORY = Path('/scratch2/COASTAL/coastal/save/shared/models') / 'meshes' / 'shinnecock' / 'grid_v1'

# directory containing input atmospheric mesh forcings (`wind_atm_fin_ch_time_vec.nc`) and WaveWatch III forcings (`ww3.Constant.20151214_sxy_ike_date.nc`)
FORCINGS_DIRECTORY = Path('/scratch2/COASTAL/coastal/save/shared/models') / 'forcings' / 'shinnecock' / 'ike'

# directory to which to write configuration
OUTPUT_DIRECTORY = Path(__file__).parent.parent / 'data' / 'configuration' / 'hera_shinnecock_ike'

HAMTIDE_DIRECTORY = '/scratch2/COASTAL/coastal/save/shared/models/forcings/tides/hamtide'

# dictionary defining runs with ADCIRC value perturbations - in this case, a single run with no perturbation
runs = {f'test_case_1': (None, None)}

# initialize `nemspy` configuration object with forcing file locations, start and end times,  and processor assignment
nems = ModelingSystem(
    start_time=datetime(2008, 8, 23),
    end_time=datetime(2008, 8, 23) + timedelta(days=14.5),
    interval=timedelta(hours=1),
    atm=AtmosphericMeshEntry(filename=FORCINGS_DIRECTORY / 'wind_atm_fin_ch_time_vec.nc', processors=1),
    wav=WaveMeshEntry(filename=FORCINGS_DIRECTORY / 'ww3.Constant.20151214_sxy_ike_date.nc', processors=1),
    ocn=ADCIRCEntry(processors=11),
)

# describe connections between coupled components
nems.connect('ATM', 'OCN')
nems.connect('WAV', 'OCN')
nems.sequence = [
    'ATM -> OCN',
    'WAV -> OCN',
    'ATM',
    'WAV',
    'OCN',
]

# initialize `adcircpy` forcing objects
tidal_forcing = Tides(tidal_source=TidalSource.HAMTIDE, resource=HAMTIDE_DIRECTORY)
tidal_forcing.use_all()
wind_forcing = AtmosphericMeshForcing(nws=17, interval_seconds=3600)
wave_forcing = WaveWatch3DataForcing(nrs=5, interval_seconds=3600)

# send run information to `adcircpy` and write the resulting configuration to output directory
write_adcirc_configurations(
    nems,
    runs,
    MESH_DIRECTORY,
    OUTPUT_DIRECTORY,
    nems_executable=NEMS_EXECUTABLE,
    adcprep_executable=ADCPREP_EXECUTABLE,
    email_address='example@email.gov',
    platform=Platform.HERA,
    spinup=timedelta(days=12.5),
    forcings=[tidal_forcing, wind_forcing, wave_forcing],
    overwrite=True,
    use_original_mesh=False,
    verbose=True,
)

This code will generate a directory hera_shinnecock_ike/ with the following structure:

๐Ÿ“ฆ hera_shinnecock_ike/
โ”ฃ ๐Ÿ“‚ coldstart/
โ”ƒ  โ”ฃ ๐Ÿ“œ fort.13
โ”ƒ  โ”ฃ ๐Ÿ“œ fort.14
โ”ƒ  โ”— ๐Ÿ“œ fort.15
โ”ฃ ๐Ÿ“‚ runs/
โ”ƒ  โ”— ๐Ÿ“‚ test_case_1/
โ”ƒ    โ”ฃ ๐Ÿ“œ fort.13
โ”ƒ    โ”ฃ ๐Ÿ“œ fort.14
โ”ƒ    โ”— ๐Ÿ“œ fort.15
โ”ฃ ๐Ÿ“œ nems.configure.coldstart
โ”ฃ ๐Ÿ“œ nems.configure.hotstart
โ”ฃ ๐Ÿ“œ config.rc.coldstart
โ”ฃ ๐Ÿ“œ config.rc.hotstart
โ”ฃ ๐Ÿ“œ model_configure.coldstart
โ”ฃ ๐Ÿ“œ model_configure.hotstart
โ”ฃ ๐Ÿ“œ job_adcprep_hera.job
โ”ฃ ๐Ÿ“œ job_nems_adcirc_hera.job.coldstart
โ”ฃ ๐Ÿ“œ job_nems_adcirc_hera.job.hotstart
โ”ฃ ๐Ÿ“œ setup.sh.coldstart
โ”ฃ ๐Ÿ“œ setup.sh.hotstart
โ”ฃ ๐Ÿ“œ setup_hera.sh
โ”— ๐Ÿ“œ run_hera.sh

Note: the required NEMS configuration files (nems.configure, model_configure) do not yet exist in the run directories (coldstart/, runs/test_case_1/). These will be populated in the next step.

2. run job submission script run_<platform>.sh

Run run_hera.sh:

sh run_hera.sh

This will first create symbolic links to populate configuration directories (by calling setup_hera.sh),

๐Ÿ“ฆ hera_shinnecock_ike/
โ”ฃ ๐Ÿ“‚ coldstart/
โ”ƒ  โ”ฃ ๐Ÿ“œ fort.13
โ”ƒ  โ”ฃ ๐Ÿ“œ fort.14
โ”ƒ  โ”ฃ ๐Ÿ“œ fort.15
โ”ƒ  โ”ฃ ๐Ÿ”— nems.configure -> ../nems.configure.coldstart
โ”ƒ  โ”ฃ ๐Ÿ”— config.rc -> ../config.rc.coldstart
โ”ƒ  โ”ฃ ๐Ÿ”— model_configure -> ../model_configure.coldstart
โ”ƒ  โ”ฃ ๐Ÿ”— setup.sh -> ../setup.sh.coldstart
โ”ƒ  โ”ฃ ๐Ÿ”— hera_adcprep.job -> ../job_adcprep_hera.job
โ”ƒ  โ”— ๐Ÿ”— hera_nems_adcirc.job -> ../job_nems_adcirc_hera.job.coldstart
โ”ฃ ๐Ÿ“‚ runs/
โ”ƒ  โ”— ๐Ÿ“‚ test_case_1/
โ”ƒ    โ”ฃ ๐Ÿ“œ fort.13
โ”ƒ    โ”ฃ ๐Ÿ“œ fort.14
โ”ƒ    โ”ฃ ๐Ÿ“œ fort.15
โ”ƒ    โ”ฃ ๐Ÿ”— fort.67.nc -> ../../coldstart/fort.67.nc
โ”ƒ    โ”ฃ ๐Ÿ”— nems.configure -> ../../nems.configure.hotstart
โ”ƒ    โ”ฃ ๐Ÿ”— config.rc -> ../../config.rc.hotstart
โ”ƒ    โ”ฃ ๐Ÿ”— model_configure -> ../../model_configure.hotstart
โ”ƒ    โ”ฃ ๐Ÿ”— setup.sh -> ../../setup.sh.hotstart
โ”ƒ    โ”ฃ ๐Ÿ”— hera_adcprep.job -> ../../job_adcprep_hera.job
โ”ƒ    โ”— ๐Ÿ”— hera_nems_adcirc.job -> ../../job_nems_adcirc_hera.job.hotstart
โ”ฃ ๐Ÿ“œ nems.configure.coldstart
โ”ฃ ๐Ÿ“œ nems.configure.hotstart
โ”ฃ ๐Ÿ“œ config.rc.coldstart
โ”ฃ ๐Ÿ“œ config.rc.hotstart
โ”ฃ ๐Ÿ“œ model_configure.coldstart
โ”ฃ ๐Ÿ“œ model_configure.hotstart
โ”ฃ ๐Ÿ“œ job_adcprep_hera.job
โ”ฃ ๐Ÿ“œ job_nems_adcirc_hera.job.coldstart
โ”ฃ ๐Ÿ“œ job_nems_adcirc_hera.job.hotstart
โ”ฃ ๐Ÿ“œ setup.sh.coldstart
โ”ฃ ๐Ÿ“œ setup.sh.hotstart
โ”ฃ ๐Ÿ“œ setup_hera.sh
โ”— ๐Ÿ“œ run_hera.sh

and then submit the requested jobs to the queue:

squeue -u $USER -o "%.8i %.21j %.4C %.4D %.31E %.20V %.20S %.20e"

   JOBID                  NAME CPUS NODE                      DEPENDENCY          SUBMIT_TIME           START_TIME             END_TIME
16368044 ADCIRC_MESH_PARTITION    1    1                          (null)  2021-02-18T19:29:17                  N/A                  N/A
16368045      ADCIRC_COLDSTART   11    1  afterany:16368044(unfulfilled)  2021-02-18T19:29:17                  N/A                  N/A
16368046 ADCIRC_MESH_PARTITION    1    1  afterany:16368045(unfulfilled)  2021-02-18T19:29:17                  N/A                  N/A
16368047       ADCIRC_HOTSTART   13    1  afterany:16368046(unfulfilled)  2021-02-18T19:29:17                  N/A                  N/A

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for soroosh.mani.noaa from gravatar.com soroosh.mani.noaa Avatar for zacharyburnett from gravatar.com zacharyburnett

Unverified details

These details have not been verified by PyPI
Project links
Meta

Release history Release notifications | RSS feed

1.7.1.post1

1.7.1 yanked

Reason this release was yanked:

stormevent dependency for speed/quadrant fixes without rmax forecast

1.7.0

1.6.6

1.6.5

1.6.4

1.6.3

1.6.2

1.6.1

1.6.0

1.5.1

1.4.14

1.4.12

1.4.11

1.4.10

1.4.9

1.4.8

1.4.7

1.4.6

1.4.5

1.4.4

1.4.3

1.4.2

1.4.1

1.4.0

1.3.3

1.3.2

1.3.1

1.3.0

1.2.25

1.2.24

1.2.23

1.2.22

1.2.21

1.2.20

1.2.19

1.2.18

1.2.17

1.2.16

1.2.15

1.2.14

1.2.13

1.2.12

1.2.11

1.2.10

1.2.9

1.2.8

1.2.7

1.2.6

1.2.5

1.2.4

1.2.3

1.2.2

1.2.1

1.2.0

1.1.4

1.1.3

1.1.2

1.1.1

1.1.0

1.0.3

1.0.2

This version

1.0.1

1.0.0

0.3.6

0.3.5

0.3.4

0.3.2

0.3.1

0.3.0

0.2.9

0.2.8

0.2.7

0.2.6

0.2.5

0.2.4

0.2.3

0.2.2

0.2.1

0.2.0

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

coupledmodeldriver-1.0.1.tar.gz (16.7 kB view hashes)

Uploaded Source

Built Distribution

coupledmodeldriver-1.0.1-py3-none-any.whl (17.1 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page