Problem using BLCR with mvapich2

From: Patrice Martinez (patrice.martinez_at_bull_dot_net)
Date: Thu Sep 20 2007 - 04:37:03 PDT

  • Next message: Paul H. Hargrove: "Re: Problem using BLCR with mvapich2"
    I'm trying to run linpack benchmark using blcr and mvapich2 (and Infiniband).

    I'm using:
    mvapich2-1.0 compiled with blcr support
    linpack linked with pvapich, and ofed libs

    I'm using (for this test) a single em64t computer, running a 2.6.21 kernel above a RHEL U4:
    uname -a
    Linux twing #1 SMP Wed Jun 13 10:29:09 CEST 2007 x86_64 x86_64 x86_64 GNU/Linux

    BLCR is compiled with this kernel, the modules are inserted, and the following env vars are set as follows:


    export LD_PRELOAD=/usr/local/lib/

    I started mpdboot:
    mpdboot --ncpus=4

    Then I try to run linpack:
    mpiexec -n 4 ./xhpl
    /usr/local/bin/mpdroot: error while loading shared libraries: cannot open shared object file: No such file or directory
    mpiexec_twing (__init__ 1171): forked process failed; status=127
    CTRL+C Caught... exiting

    It doesn't work, however, the libcr is located:

    ldd /usr/local/bin/mpdroot
            /usr/local/lib/ (0x0000002a95557000)
            /lib64/tls/ (0x000000323fa00000)
   => /usr/lib64/ (0x0000002a95690000)
   => /usr/lib64/ (0x0000002a9579b000)
   => /lib64/tls/ (0x000000323ef00000)
   => /lib64/ (0x000000323ed00000)
            /lib64/ (0x000000323eb00000)
   => /usr/lib64/ (0x0000002a958a6000)

    ldd ./xhpl
   => /lib64/tls/ (0x000000323fa00000)
   => /usr/lib64/ (0x00002b4a28296000)
   => /usr/lib64/ (0x00002b4a283a1000)
   => /usr/local/lib/ (0x00002b4a284ab000)
   => /lib64/tls/ (0x000000323ef00000)
            /lib64/ (0x000000323eb00000)
   => /lib64/ (0x000000323ed00000)
   => /usr/lib64/ (0x00002b4a285b4000)

    Any idea about whta's wrong with this?

    Linpacks runs well if linked with a release of mvapich2 compiled without blcr support.
    Cordialement/Best regards
    Patrice Martinez
    Linux Kernel Architect.
    Bull, Architect of an Open World
    OFFICE : B1-405
    PHONE  : +33 (0)4 76 29 74 69
    EMAIL  : Patrice_dot_martinez_at_bull_dot_net
    ADDR   : BULL, 1 rue de Provence, BP 208, 38432 Echirolles Cedex, FRANCE
    Bull recrute :

  • Next message: Paul H. Hargrove: "Re: Problem using BLCR with mvapich2"