From: Pradeep Padala (ppadala_at_eecs_dot_umich_dot_edu)
Date: Tue Jul 26 2005 - 15:15:02 PDT
Hi Paul, Latest mail from Jeff mentioned this (He is busy with some conference and may be late in responding) -------- Original Message -------- Subject: Re: cr Date: Tue, 26 Jul 2005 06:43:22 -0600 From: Jeff Squyres <firstname.lastname@example.org> To: Pradeep Padala <ppadala_at_eecs_dot_umich_dot_edu> Yes, without libaio parallel processes checkpointed / restarted just fine. I see the problem -- only libaio.so.1 exists (not libaio.so). This is why the linker doesn't find it. Did you remove an RPM yesterday or something? IIRC, the libaio.so file is in the libaio-devel RPM...? ---------------------------------- I fixed the aio rpm and I am waiting for him to re-test the mpi programs. Is linking with aio a problem for blcr? -- Pradeep Padala http://ppadala.blogspot.com Paul H. Hargrove wrote: > Sorry to have replied before reading other replies which said the same > thing. > > I just reread the relavent parts of the BLCR sources and see just a few > places where EBUSY might be generated: > > + PID conflict. > + Restore of a FIFO (aka named pipe) in which there is data buffered in > the pipe. A "solution" here would be to delete and recreate the FIFO. > We need a better behavior in BLCR, but can't yet do anything more > intelligent. > + Some "should never happen" file restore cases. > > In all three cases, there should be a warning/error message in the > system log file. Please let me know what you find in /var/log/messages > (or equivalent). > > -Paul > > Paul H. Hargrove wrote: > >> Typically this is an indication that the original pids are (still) in >> use. My guess is that the originaly mpi processes are still running. >> >> -Paul >> >> Jeff Squyres wrote: >> >>> A user was having problems with LAM + BLCR, so I got a guest account >>> on his cluster and gave it a whirl. With my own build of LAM/MPI, >>> I'm able to checkpoint just fine (i.e., I get N+1 checkpoint files). >>> But when I try to restart, I get the following error: >>> >>> [jeff@linf1 ~]$ cr_restart context.4037 >>> cri_syscall(CR_OP_RSTRT_REQ, &req): Device or resource busy >>> cri_syscall(CR_OP_RSTRT_REQ, &req): Device or resource busy >>> cri_syscall(CR_OP_RSTRT_REQ, &req): Device or resource busy >>> cri_syscall(CR_OP_RSTRT_REQ, &req): Device or resource busy >>> >>> What does this mean? >>> >>> I had checkpointed a simple "hello world" MPI application (4 MPI >>> processes) on a single node. >>> >>> The user has already been in contact with Paul -- from his initial >>> post on the LAM list >>> (http://www.lam-mpi.org/MailArchives/lam/2005/07/11015.php): >>> >>> "P.S. I am using a patched version of blcr to make it work on FC4. The >>> patch was given to me by Paul Hargrove." >>> >>> The specific version of BLCR in use is: >>> >>> [jeff@linf1 ~]$ cr_restart --version >>> cr_restart version 0.4.pre1_snapshot_2005_06_27 >>> >>> Sidenote: I notice that cr_checkpoint has a "--version" switch, but >>> it is not listed in "cr_checkpoint --help" (which was somewhat >>> confusing). Ditto for cr_run.