Skip to content

Add Parallel I/O Infrastructure#706

Open
SeanBryan51 wants to merge 18 commits intomainfrom
add-parallelio-infrastructure
Open

Add Parallel I/O Infrastructure#706
SeanBryan51 wants to merge 18 commits intomainfrom
add-parallelio-infrastructure

Conversation

@SeanBryan51
Copy link
Copy Markdown
Collaborator

@SeanBryan51 SeanBryan51 commented Mar 10, 2026

This change brings in the interface layer for working with the NetCDF Fortran and ParallelIO (PIO) libraries in CABLE. PIO allows for each MPI rank to participate in I/O operations collectively and is a first step in adding MPI support to the serial offline driver, and eventually, to replace the legacy MPI implementation (#358).

To keep CABLE dependencies as minimal as possible for running in serial mode (without MPI), the interface layer is designed such that PIO support is optional.

To build CABLE with PIO, PIO version 2.6.8 or greater is required.

Type of change

Please delete options that are not relevant.

  • New feature
  • New or updated documentation

Checklist

  • The new content is accessible and located in the appropriate section
  • I have checked that links are valid and point to the intended content
  • I have checked my code/text and corrected any misspellings

Testing

  • Are the changes bitwise-compatible with the main branch? If working on an optional feature, are the results bitwise-compatible when this feature is off? If yes, copy benchcab output showing successful completion of the bitwise compatibility tests or equivalent results below this line.
2026-03-17 12:43:56,608 - INFO - benchcab.benchcab.py:380 - Running comparison tasks...
2026-03-17 12:43:56,634 - INFO - benchcab.benchcab.py:381 - tasks: 168 (models: 2, sites: 42, science configurations: 4)
2026-03-17 12:46:38,124 - INFO - benchcab.benchcab.py:391 - 0 failed, 168 passed

📚 Documentation preview 📚: https://cable--706.org.readthedocs.build/en/706/

@SeanBryan51 SeanBryan51 force-pushed the add-parallelio-infrastructure branch 9 times, most recently from ffc3905 to 0138093 Compare March 10, 2026 21:18
@SeanBryan51 SeanBryan51 changed the title Add parallelio infrastructure Add Parallel I/O Infrastructure Mar 11, 2026
@SeanBryan51 SeanBryan51 force-pushed the add-parallelio-infrastructure branch 4 times, most recently from 05b0997 to c10b30a Compare March 16, 2026 08:48
@SeanBryan51 SeanBryan51 force-pushed the add-parallelio-infrastructure branch 2 times, most recently from f9fc997 to 1d8d61f Compare March 17, 2026 00:42
GCC was updated from 13.x to 14.x to allow for passing a factory
procedure with a polymorphic function result to each test case
subroutine.

See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=118372 for more
details.
@SeanBryan51 SeanBryan51 force-pushed the add-parallelio-infrastructure branch from 1d8d61f to 7f8d450 Compare March 17, 2026 00:48
@SeanBryan51 SeanBryan51 marked this pull request as ready for review March 17, 2026 01:13
@SeanBryan51
Copy link
Copy Markdown
Collaborator Author

SeanBryan51 commented Mar 17, 2026

Hi @Whyborn, now that #700 has been merged, these changes are ready to go in. Do you mind giving this your review?

Copy link
Copy Markdown
Contributor

@Whyborn Whyborn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A lot of the internal functions are missing any commentary. I found a lot of the routines not particularly intuitive, so some commentary explaining human-readably what the routines are doing would be a big help I think. Doesn't have to be public facing documentation, more like internal developer comments.

I find the tests not very "human"- a lot of abstractions, that I didn't find very easy to reason about exactly what they're doing. This could probably be solved by some commentary as well.

@SeanBryan51 SeanBryan51 requested a review from abhaasgoyal March 17, 2026 06:02
@SeanBryan51 SeanBryan51 force-pushed the add-parallelio-infrastructure branch from d1767af to 87f98fd Compare March 17, 2026 06:18
@SeanBryan51
Copy link
Copy Markdown
Collaborator Author

SeanBryan51 commented Mar 18, 2026

Thanks for taking a look Lachlan, I forgot to add docs to cable_netcdf_internal.F90. I realised cable_netcdf_internal.F90 was only really necessary for initialising the I/O handler, so I've trimmed down that file to only include the I/O handler initialisation and renamed it to cable_netcdf_init.F90 in 0f30366.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants