From: Paul H. Hargrove (PHHargrove_at_lbl_dot_gov)
Date: Tue Jul 28 2009 - 12:16:19 PDT
Gang Chen, When you say "Copy-On-Write mechanism" I am assuming you mean allowing the process(es) to be checkpointed to continue running while the checkpoint is being taken, and relying on the page-protection mechanisms to make copies of any memory that the processes may modify (thus ensuring that the unmodified version are the ones stored in the context file). If you mean something different, please let me know. As presently implemented, BLCR does not use the copy-on-write mechanism as I have described it above. The current file format has several key attributes of the processes (such as open files) stored after the memory dump, which requires that the process is stopped during the checkpointing to prevent modification of these attributes. It is possible that a future BLCR version could change the file format to place the memory dump as the last portion saved for a process, which would probably allow a copy-on-write optimization. However, we have not examined what changes to BLCR would be required. One concern that comes to mind is that when checkpointing multiple processes in the same checkpoint request, we currently store then serially into a single file. That serialization would mean that even with copy-on-write we'd need to prevent "later" processes from running until "earlier" ones were checkpointed, making copy-on-write far less useful as an optimization than would be the case without the serialization. We are planning to eliminate this serialization in the future. While BLCR itself does not implement a copy-on-write mechanism, it is possible for a client/user of BLCR (such as an MPI implementation) to perform "forked checkpointing" to accomplish approximately the same thing. In "forked checkpointing" an application process will fork() in response to the checkpoint request and one of the two processes will be checkpointed while the other continues to run. Something like this is possible in an MPI runtime because it has control of all the external interactions of the processes, and so can deal with the fact that the process that is checkpointed and the one that keeps running will not have the same pid (depending on the choice of child or parent to keep running some "fixup" may be required at either checkpoint or restart time, respectively, to account for the apparent change of pid). I don't know if any MPI implementation currently perform forked checkpointing. -Paul Gang Chen wrote: > Hi, > > Does the BLCR support the Copy-On-Write mechanism? Thank you. > > -- > Gang Chen > -- Paul H. Hargrove PHHargrove_at_lbl_dot_gov Future Technologies Group Tel: +1-510-495-2352 HPC Research Department Fax: +1-510-486-6900 Lawrence Berkeley National Laboratory