From: Paul H. Hargrove (PHHargrove_at_lbl_dot_gov)
Date: Wed Oct 21 2009 - 16:50:19 PDT
Alan,
Thanks for the patch. Sorry that I've been too busy w/ another project
to look at it until today.
The same problem exists on a ppc64 (and on sparc64 if that port is ever
completed).
Cross-compilation creates a chicken-and-egg with the way the current
configure steps are done because the setting of CR_KARCH comes too late.
So, I offer the attached patch which generalizes yours and fixes the
cross-compilation issue by reordering some of the configure probes.
Please let me know if the attachment works for you as well as your own
version does.
I should note that at least in my own trials on an old ppc64 machine
with FC6, I did *NOT* find the kbuild magic to "just work" and therefore
needed to add KCC='gcc -m64' to the configure command.
-Paul
Alan Woodland wrote:
> Hi,
>
> I've run into another minor problem building. When the
> userland/toolchain is i386 it is possible still to be using an x86_64
> kernel. If I try and configure for this kernel, e.g.:
>
> ./configure --with-installed-libcr --with-installed-util
> --with-components=modules --prefix=/usr --with-linux=2.6.30-2-amd64
>
> Everything appears to be fine, however during building it dies:
>
> CC [M] /usr/src/modules/blcr/cr_module/kbuild/vmadump_i386.o
> /usr/src/modules/blcr/cr_module/kbuild/vmadump_i386.c: In function
> 'vmadump_store_cpu':
> /usr/src/modules/blcr/cr_module/kbuild/vmadump_i386.c:56: error:
> 'struct pt_regs' has no member named 'ds'
> /usr/src/modules/blcr/cr_module/kbuild/vmadump_i386.c:57: error:
> 'struct pt_regs' has no member named 'es'
> /usr/src/modules/blcr/cr_module/kbuild/vmadump_i386.c: In function
> 'vmadump_restore_cpu':
> /usr/src/modules/blcr/cr_module/kbuild/vmadump_i386.c:161: error:
> 'struct pt_regs' has no member named 'ds'
> /usr/src/modules/blcr/cr_module/kbuild/vmadump_i386.c:162: error:
> 'struct pt_regs' has no member named 'es'
> make[7]: *** [/usr/src/modules/blcr/cr_module/kbuild/vmadump_i386.o] Error 1
>
> Presumably it shouldn't be trying to use vmadump_i386.c here...
>
> What's the best way to handle this? On Debian at least we can detect
> this because either the running kernel or the target kernel will have
> amd64 in the name, but that doesn't always have to be the case. We
> could use file to test the kernel image itself? Or look at the length
> of the addresses in System.map for that kernel. Or the presence of
> startup_64 in the kernel could be tested for...
>
> The attached patch seems to work (counting length of kernel symbol
> addresses). Even though it works, I can't actually test the modules it
> produces on any x86_64 machines due to a xen related problem. The
> patch is a little rough around the edges, but is it sane? Any
> suggestions gratefully received!
>
> Kbuild magic does seems to be sorting out the c compiler to build the
> module correctly:
>
> debian-devel64:/usr/src/modules/blcr/cr_module/kbuild# file *.ko
> blcr.ko: ELF 64-bit LSB relocatable, x86-64, version 1 (SYSV), not stripped
> debian-devel64:/usr/src/modules/blcr/cr_module/kbuild# uname -a
> Linux debian-devel64 2.6.30-1-686-bigmem #1 SMP Sat Aug 15 20:10:47
> UTC 2009 i686 GNU/Linux
>
> Thanks,
> Alan
>
--
Paul H. Hargrove PHHargrove_at_lbl_dot_gov
Future Technologies Group Tel: +1-510-495-2352
HPC Research Department Fax: +1-510-486-6900
Lawrence Berkeley National Laboratory
Index: acinclude.m4
===================================================================
RCS file: /var/local/cvs/lbnl_cr/acinclude.m4,v
retrieving revision 1.177.14.4
diff -u -r1.177.14.4 acinclude.m4
--- acinclude.m4 21 Oct 2009 20:24:53 -0000 1.177.14.4
+++ acinclude.m4 21 Oct 2009 23:47:13 -0000
@@ -499,7 +499,7 @@
cr_cvname="${cr_cvname} V=1"
fi
if test x$cross_compiling = xyes; then
- cr_cvname="$cr_cvname ARCH=$CR_ARCH CROSS_COMPILE=$host_alias-"
+ cr_cvname="$cr_cvname ARCH=$CR_KARCH CROSS_COMPILE=$host_alias-"
fi
])
KBUILD_MAKE_ARGS="$cr_cvname"
@@ -1043,8 +1043,11 @@
CR_SET_CACHE_VAR([LINUX_VMLINUX])
CR_CACHE_REVALIDATE([${LINUX_SYMTAB_FILE}],[ksymtab],[kernel symbol table])
fi
- # Now check for consistency w/ the kernel source
- # XXX: Currently just check SMPness. Can this be more aggressive?
+])
+
+# Now check for SYMTAB consistency w/ the kernel source
+# XXX: Currently just check SMPness. Can this be more aggressive?
+AC_DEFUN([CR_LINUX_SYMTAB_VALIDATE],[
AC_MSG_CHECKING([for SMP kernel source])
CR_CACHED_KERNEL_COMPILE([smp_source],[
#include <linux/autoconf.h>
@@ -1073,7 +1076,7 @@
# ------------------------------------------------------
# Helpers for CR_FIND_KSYM() and CR_FIND_EXPORTED_KSYM()
AC_DEFUN([_CR_KSYM_INIT_PATTS],[
- case "$CR_ARCH" in
+ case "$CR_KARCH" in
ppc64)
CR_KSYM_PATTERN_DATA=['[bBdDrRtT] ']
CR_KSYM_PATTERN_CODE=['[dD] '] dnl Function descriptor is data
Index: configure.ac
===================================================================
RCS file: /var/local/cvs/lbnl_cr/configure.ac,v
retrieving revision 1.410.2.10
diff -u -r1.410.2.10 configure.ac
--- configure.ac 23 Aug 2009 22:58:26 -0000 1.410.2.10
+++ configure.ac 21 Oct 2009 23:47:13 -0000
@@ -835,14 +835,37 @@
fi
AC_MSG_RESULT([${CR_MODULE_DIR} ($cr_why)])
+# Checks for matching (we hope) symbol table
+CR_LINUX_SYMTAB
+
+# Setup CR_KARCH to enable 32-bit user-space on 64-bit kernels
+CR_KARCH=$CR_ARCH
+if test $cr_wordsize = 4; then
+ # Count pointer hex digits in the System.map to id a 64-bit kernel
+ # Based on logic provided by Alan Woodland
+ AC_PROG_EGREP
+ AC_MSG_CHECKING([if kernel and user-space wordsize match])
+ if eval $LINUX_SYMTAB_CMD | head | $EGREP '^[[a-f0-9]]{16}' >/dev/null 2>&1; then
+ AC_MSG_RESULT([no (32-bit user-space with 64-bit kernel)])
+ case $CR_KARCH in
+ i386) CR_KARCH=x86_64;;
+ ppc) CR_KARCH=ppc64;;
+ sparc) CR_KARCH=sparc64;;
+ esac
+ else
+ AC_MSG_RESULT([yes])
+ fi
+fi
+AC_SUBST(CR_KARCH)
+
# Configure automake/kbuild glue
CR_CHECK_KBUILD
# Configure module compile flags and perform a sanity check
CR_SET_KCFLAGS
-# Checks for matching (we hope) symbol table
-CR_LINUX_SYMTAB
+# Now we can check if SYMTAB matches the kernel
+CR_LINUX_SYMTAB_VALIDATE
## Check for optional kernel headers...
CR_CHECK_KERNEL_HEADER([linux/syscalls.h])
@@ -1003,7 +1026,7 @@
[mm_context_t],[unsigned long],[vdso])
CR_CHECK_KERNEL_MEMBER([thread_info.sysenter_return],[#include <linux/thread_info.h>],
[struct thread_info],[void *],[sysenter_return])
-if test $CR_ARCH = arm; then
+if test $CR_KARCH = arm; then
CR_CHECK_KERNEL_MEMBER([thread_info.tp_value],[#include <linux/thread_info.h>],
[struct thread_info],[unsigned long],[tp_value])
CR_FIND_KSYM([__kuser_helper_start],[CODE],[extern int __kuser_helper_start(void);])
@@ -1026,7 +1049,7 @@
if test -n "${HAVE_MAP_VSYSCALL}"; then
CR_FIND_KSYM([map_vsyscall],[CODE])
fi
-if test $CR_ARCH = x86_64; then
+if test $CR_KARCH = x86_64; then
CR_FIND_KSYM([syscall32_setup_pages],[CODE],
[extern int syscall32_setup_pages(void *, int);])
fi
@@ -1063,7 +1086,7 @@
# which must have a prototype in asm/pgtable.h (arch-specific)
# If we support ia64 or mips in the future we'll need to add more cases here.
# NOTE: don't try to factor the CR_FIND_KSYM() call; it requires a literal.
-case $CR_ARCH in
+case $CR_KARCH in
sparc*)
CR_FIND_KSYM([mem_map_zero],[DATA])
ZERO_PAGE_SYMBOL=mem_map_zero
@@ -1094,7 +1117,7 @@
CR_FIND_KSYM([do_sigaltstack],[CODE],
[extern int do_sigaltstack(const stack_t *, stack_t *, unsigned long);])
-if test $CR_ARCH = i386 -o $CR_ARCH = x86_64; then
+if test $CR_KARCH = i386 -o $CR_KARCH = x86_64; then
CR_CHECK_KERNEL_TYPE([struct n_desc_struct],[
#include <linux/sched.h>
#include <asm/desc.h>])
@@ -1145,7 +1168,7 @@
fi
fi
-if test $CR_ARCH = i386; then
+if test $CR_KARCH = i386; then
CR_CHECK_KERNEL_MEMBER([pt_regs.xcs],[#include <asm/ptrace.h>],[struct pt_regs],[int],[xcs])
CR_CHECK_KERNEL_MEMBER([pt_regs.xfs],[#include <asm/ptrace.h>],[struct pt_regs],[int],[xfs])
CR_CHECK_KERNEL_MEMBER([pt_regs.xgs],[#include <asm/ptrace.h>],[struct pt_regs],[int],[xgs])
@@ -1493,7 +1516,7 @@
CR_FIND_KSYM([sys_mremap],[CODE],
[extern asmlinkage unsigned long sys_mremap(unsigned long addr, unsigned long old_len, unsigned long new_len, unsigned long flags, unsigned long new_addr);])
CR_FIND_KSYM([do_sigaction],[CODE])
-if test $CR_ARCH = x86_64; then
+if test $CR_KARCH = x86_64; then
CR_FIND_KSYM([load_gs_index],[CODE])
CR_CHECK_KERNEL_CALL([read_pda],[#include <asm/percpu.h>])
@@ -1555,7 +1578,7 @@
fi # cr_build_modules
-if test x"$cr_build_libcr" = xyes && expr $CR_ARCH : ppc >/dev/null; then
+if test x"$cr_build_libcr" = xyes && expr $CR_KARCH : ppc >/dev/null; then
AC_MSG_CHECKING([for work around for bug 2524])
if test "$cr_build_modules,$force_bug2524" = "yes,"; then
if test [`echo $LINUX_VER | sed -e 's/2.6.\([0-9]*\).*/\1/'`] -le 15; then
Index: config/kbuild.mak.in
===================================================================
RCS file: /var/local/cvs/lbnl_cr/config/kbuild.mak.in,v
retrieving revision 1.24
diff -u -r1.24 kbuild.mak.in
--- config/kbuild.mak.in 4 Aug 2008 18:39:29 -0000 1.24
+++ config/kbuild.mak.in 21 Oct 2009 23:47:13 -0000
@@ -1,7 +1,7 @@
# Here are the configuration bits needed in the "caller" to use this file
#SUBDIR = vmadump4
#TARGET = vmadump
-#SOURCES = vmadump_common.c vmadump_@[email protected] optional.h
+#SOURCES = vmadump_common.c vmadump_@[email protected] optional.h
#[optional]EXTRA_CFLAGS = -I$(top_builddir)/include -I$(top_builddir) -I$(srcdir)
#[optional]IMPORTS_FROM = /some/other/kbuild/module/dir /and/another
#
Index: cr_module/Makefile.am
===================================================================
RCS file: /var/local/cvs/lbnl_cr/cr_module/Makefile.am,v
retrieving revision 1.39
diff -u -r1.39 Makefile.am
--- cr_module/Makefile.am 3 Sep 2008 00:29:00 -0000 1.39
+++ cr_module/Makefile.am 21 Oct 2009 23:47:13 -0000
@@ -31,13 +31,13 @@
BPROC_VERSION = "4.0.0pre8"
vmadump_dir = $(top_srcdir)/vmadump4
-vmad_sources = $(vmadump_dir)/vmadump_common.c $(vmadump_dir)/vmadump_@[email protected]
+vmad_sources = $(vmadump_dir)/vmadump_common.c $(vmadump_dir)/vmadump_@[email protected]
vmad_include = -I$(vmadump_dir) -D__NR_vmadump=-1 -DBPROC_VERSION='$(BPROC_VERSION)'
cr_sources = $(my_sources) $(vmad_sources)
INCLUDES = -I$(top_builddir)/include \
-I$(top_srcdir)/include \
- -I$(srcdir)/arch/@CR_ARCH@ \
+ -I$(srcdir)/arch/@CR_KARCH@ \
$(vmad_include)
# echo an arbitrary make/environment variable (needed by kbuild glue)