Re: copy-on-write

From: Paul H. Hargrove (PHHargrove_at_lbl_dot_gov)
Date: Tue Jul 28 2009 - 12:16:19 PDT

  • Next message: Ionel Taflan: "Re: BLCR web page"
    Gang Chen,
      When you say "Copy-On-Write mechanism" I am assuming you mean allowing 
    the process(es) to be checkpointed to continue running while the 
    checkpoint is being taken, and relying on the page-protection mechanisms 
    to make copies of any memory that the processes may modify (thus 
    ensuring that the unmodified version are the ones stored in the context 
    file).  If you mean something different, please let me know.
      As presently implemented, BLCR does not use the copy-on-write 
    mechanism as I have described it above.  The current file format has 
    several key attributes of the processes (such as open files) stored 
    after the memory dump, which requires that the process is stopped during 
    the checkpointing to prevent modification of these attributes.  It is 
    possible that a future BLCR version could change the file format to 
    place the memory dump as the last portion saved for a process, which 
    would probably allow a copy-on-write optimization.  However, we have not 
    examined what changes to BLCR would be required.  One concern that comes 
    to mind is that when checkpointing multiple processes in the same 
    checkpoint request, we currently store then serially into a single 
    file.  That serialization would mean that even with copy-on-write we'd 
    need to prevent "later" processes from running until "earlier" ones were 
    checkpointed, making copy-on-write far less useful as an optimization 
    than would be the case without the serialization.  We are planning to 
    eliminate this serialization in the future.
      While BLCR itself does not implement a copy-on-write mechanism, it is 
    possible for a client/user of BLCR (such as an MPI implementation) to 
    perform "forked checkpointing" to accomplish approximately the same 
    thing.  In "forked checkpointing" an application process will fork() in 
    response to the checkpoint request and one of the two processes will be 
    checkpointed while the other continues to run.  Something like this is 
    possible in an MPI runtime because it has control of all the external 
    interactions of the processes, and so can deal with the fact that the 
    process that is checkpointed and the one that keeps running will not 
    have the same pid (depending on the choice of child or parent to keep 
    running some "fixup" may be required at either checkpoint or restart 
    time, respectively, to account for the apparent change of pid).  I don't 
    know if any MPI implementation currently perform forked checkpointing.
    Gang Chen wrote:
    > Hi,
    > Does the BLCR support the Copy-On-Write mechanism? Thank you.
    > -- 
    > Gang Chen
    Paul H. Hargrove                          PHHargrove_at_lbl_dot_gov
    Future Technologies Group                 Tel: +1-510-495-2352
    HPC Research Department                   Fax: +1-510-486-6900
    Lawrence Berkeley National Laboratory     

  • Next message: Ionel Taflan: "Re: BLCR web page"