From: "Chen, Qi" <Qi.Chen@windriver.com>
To: Bruce Ashfield <bruce.ashfield@gmail.com>
Cc: "meta-virtualization@lists.yoctoproject.org"
<meta-virtualization@lists.yoctoproject.org>
Subject: RE: [meta-virtualization][master-next][PATCH] ceph: fix do_compile/do_install failures
Date: Thu, 14 Mar 2024 04:48:06 +0000 [thread overview]
Message-ID: <CO6PR11MB5602322B8DFE4A2C9BD9E9BEED292@CO6PR11MB5602.namprd11.prod.outlook.com> (raw)
In-Reply-To: <CADkTA4MyYTpaOAYocepi23yuyaG7u1X8bdG4xpCyo2GmQrGAnQ@mail.gmail.com>
Hi Bruce,
For convenience, I'll put my replies together, one by one.
* OECMAKE_C/CXX_COMPILER and PY_CC
OECMAKE_C/CXX_COMPILER are used to set CMAKE_C/CXX_COMPILER for cmake to use. These two are used everywhere.
The PY_CC/CXX are here: https://github.com/ceph/ceph/blob/main/cmake/modules/Distutils.cmake#L73
They are set from CMAKE_C/CXX_COMPILER and are used for building pybind: https://github.com/ceph/ceph/blob/main/cmake/modules/Distutils.cmake#L91
The original error I got is:
In file included from /ala-lpggp72/qichen/Yocto/builds/build-poky/tmp/work/core2-64-poky-linux/ceph/18.2.0/build/src/pybind/rados/rados.c:29:
/ala-lpggp72/qichen/Yocto/builds/build-poky/tmp/work/core2-64-poky-linux/ceph/18.2.0/recipe-sysroot-native/usr/include/python3.12/Python.h:23:12: fatal error: stdlib.h: No such file or directory
23 | # include <stdlib.h>
| ^~~~~~~~~~
compilation terminated.
error: command '/ala-lpggp72/qichen/Yocto/builds/build-poky/tmp/work/core2-64-poky-linux/ceph/18.2.0/recipe-sysroot-native/usr/bin/x86_64-poky-linux/x86_64-poky-linux-gcc' failed with exit code 1
So I added '--sysroot' to OECMAKE_C/CXX_COMPILER to fix that error.
I'm not sure if this change fixes other issues. Maybe.
* what broke with 0001-ceph-fix-build-errors-for-cross-compile.patch
The error message is like below:
"""
Traceback (most recent call last):
File "/buildarea2/chenqi/builds/build-poky/tmp/work/core2-64-poky-linux/ceph/18.2.0/ceph-18.2.0/src/pybind/r\
ados/setup.py", line 182, in <module>
**ext_args
^^^^^^^^
NameError: name 'ext_args' is not defined
"""
The patch deleted the ext_args definitions.
* Did you not run into Thrift or python not being detected properly ?
I never got the python detection issue. I did get the Thrift detection issue. I turned off WITH_JAEGER to get rid of that error.
https://github.com/open-telemetry/opentelemetry-cpp/blob/9f3a8eeb1952c75d6d5f74aa4c5e6ed21e1318a8/CMakeLists.txt#L207
* Were those options needed to fix build issues ?
WITH_RADOSGW_SELECT_PARQUET and WITH_RADOSGW_ARROW_FLIGHT are turned off to get rid of the utf8proc dependency.
https://github.com/ceph/ceph/blob/main/src/CMakeLists.txt#L906
I did succeed adding utf8proc recipe, trying to satisfy that dependency. But some further problem appeared and that problem is even more complicated to solve.
The build process tried to download some tar.gz contents, which failed because do_compile defaults to turn network off.
The error message is like below:
"""
FAILED: src/xsimd_ep-stamp/xsimd_ep-download /ala-lpggp72/qichen/Yocto/builds/build-poky/tmp/work/core2-64-poky-linux/ceph/18.2.0/build/src/arrow/cpp/src/xsimd_ep-stamp/xsimd_ep-download
...
-- stdout output is:
-- Downloading...
dst='/ala-lpggp72/qichen/Yocto/builds/build-poky/tmp/work/core2-64-poky-linux/ceph/18.2.0/build/src/arrow/cpp/src/aeec9c872c8b475dedd7781336710f2dd2666cb2.tar.gz'
timeout='none'
inactivity timeout='none'
-- Using src='https://github.com/xtensor-stack/xsimd/archive/aeec9c872c8b475dedd7781336710f2dd2666cb2.tar.gz'
-- Retrying...
...
-- stderr output is:
CMake Error at xsimd_ep-stamp/download-xsimd_ep.cmake:170 (message):
Each download failed!
"""
https://github.com/apache/arrow/blob/347a88ff9d20e2a4061eec0b455b8ea1aa8335dc/cpp/thirdparty/versions.txt
https://github.com/apache/arrow/blob/347a88ff9d20e2a4061eec0b455b8ea1aa8335dc/cpp/cmake_modules/ThirdpartyToolchain.cmake
After checking the above two files, I decided to take the easy way, disabling those two options.
The -DWITH_SYSTEM_ZSTD=ON, together with the zstd dependency, will make ceph to use our zstd.
For -DARROW_GANDIVA=OFF and -DARROW_WITH_RE2=OFF, they were set to get rid of the following error at that time.
Could NOT find re2Alt (missing: RE2_LIB RE2_INCLUDE_DIR)
Looking at the cmake files again, I'm not sure about these two. So I just tried it again. Now I can confirm that these two options are not needed. The build still succeeds without these two options.
* BUILD_DOC=1
The error I got without exporting BUILD_DOC is as below:
"""
In file included from /ala-lpggp72/qichen/Yocto/builds/build-poky/tmp/work/core2-64-poky-linux/ceph/18.2.0/ceph-18.2.0/src/pybind/rados/tmpwk7dy7ow/rados_dummy.c:2:
/ala-lpggp72/qichen/Yocto/builds/build-poky/tmp/work/core2-64-poky-linux/ceph/18.2.0/ceph-18.2.0/src/pybind/rados/../../include/rados/librados.h:22:10: fatal error: netinet/in.h: No such file or directory
22 | #include <netinet/in.h>
| ^~~~~~~~~~~~~~
compilation terminated.
Compile Error: RADOS development headers not found
"""
The error comes from the check_sanity() function: https://github.com/ceph/ceph/blob/main/src/pybind/rados/setup.py#L76
So I set BUILD_DOC to workaround this issue: https://github.com/ceph/ceph/blob/main/src/pybind/rados/setup.py#L76
There might be better solutions. But I was trying to fix things ASAP, so I didn't dig deeper.
Regards,
Qi
-----Original Message-----
From: Bruce Ashfield <bruce.ashfield@gmail.com>
Sent: Wednesday, March 13, 2024 11:57 PM
To: bruce.ashfield@gmail.com
Cc: Chen, Qi <Qi.Chen@windriver.com>; meta-virtualization@lists.yoctoproject.org
Subject: Re: [meta-virtualization][master-next][PATCH] ceph: fix do_compile/do_install failures
On Wed, Mar 13, 2024 at 10:33 AM Bruce Ashfield via lists.yoctoproject.org <bruce.ashfield=gmail.com@lists.yoctoproject.org> wrote:
>
> My build testing definitely got further with this applied!
>
> I'm now hitting a build issue with boost, but I'll double check that
> my oe-core and other layers are up to date before digging in.
>
> Is this the boost problem you saw ?
>
> /include/boost/preprocessor/iteration/detail/limits/local_256.hpp:16:
> multiple definition of `boost::phoenix::placeholders::uarg1';
> src/common/CMakeFiles/common-common-objs.dir/ConfUtils.cc.o:/usr/include/boost/preprocessor/iteration/detail/limits/local_256.hpp:16:
> first defined here
> collect2: error: ld returned 1 exit status
Indeed. I updated my oe-core and it has made it past this build problem.
Bruce
>
> Bruce
>
> On Wed, Mar 13, 2024 at 9:13 AM Bruce Ashfield via
> lists.yoctoproject.org
> <bruce.ashfield=gmail.com@lists.yoctoproject.org> wrote:
> >
> > These questions are for my own understanding, they aren't issues I
> > have with the patch!
> >
> > On Wed, Mar 13, 2024 at 7:08 AM Chen Qi via lists.yoctoproject.org
> > <Qi.Chen=windriver.com@lists.yoctoproject.org> wrote:
> > >
> > > From: Chen Qi <Qi.Chen@windriver.com>
> > >
> > > 1. OECMAKE_C/CXX_COMPILER needs to have sysroot setting in it, because
> > > ceph's cmake files are using it to construct PY_CC. Without it, files
> > > such as stdlib.h cannot be found.
> >
> > Since this used to be cross compiling (but it has been a while since
> > it updated, hence my stubbornness in needing to upgrade it before
> > the release). I'm curious as to why this wasn't already done. The
> > inability to link the test C program showed that this was the case
> > .. but I wasn't able to locate this variable and set it.
> >
> > I would have thought that this is something the cmake bbclass should
> > take care of.
> >
> > Is the variable used for this in Ceph custom ?
> >
> > > 2. re2 is added to deps. Seems no way to disable it.
> >
> > ok. I didn't make it far enough to see that this was needed.
> >
> > > 3. 0001-ceph-fix-build-errors-for-cross-compile.patch is dropped. It's useless
> > > and problematic for new version.
> >
> > What broke with this in place ?
> >
> > > 4. 0001-cmake-add-support-for-python3.11.patch is dropped as it's not needed anymore.
> >
> > Agreed. I was going to drop it as well (in particular since we are
> > python12 and it wasn't failing checks).
> >
> > On this point: Did you not run into Thrift or python not being
> > detected properly ?
> > I was constantly hitting those two errors.
> >
> > > 5. 0001-avoid-to_string-error.patch is added to fix build error, maybe caused by
> > > boost version incompatibility.
> >
> > Looks good. I didn't get far enough to run into this one.
> >
> > > 6. Some cleanups.
> > > 7. A few more options are set.
> >
> > I assume it is these ones (among a few others)
> >
> > > + -DARROW_GANDIVA=OFF \
> > > + -DARROW_WITH_RE2=OFF \
> > > + -DWITH_RADOSGW_SELECT_PARQUET=OFF \
> > > + -DWITH_RADOSGW_ARROW_FLIGHT=OFF \
> >
> > Were those options needed to fix build issues ? I assume so, since
> > they are being turned off.
> >
> > I support turning off everything we can in Ceph :)
> >
> > > 8. BUILD_DOC=1 is exported so that check_sanity() causes compilation error.
> > > This is a workaround and may need further visit in the future.
> >
> > Interesting. What is the sanity error ? Without having hit it that
> > way, that variable indicates that we will be building the docs .. I
> > assume the sanity check is looking to see that they are present
> > and/or that the build of them has been enabled ?
> >
> > > 9. Delete the conditional check for /etc/debian_version which adds '--install-layout deb'
> > > and causes the following error.
> > > error: option --install-layout not recognized
> > > The patch is 0001-delete-install-layout-deb.patch.
> >
> > Sounds good. I didn't make it this far :)
> >
> > > 10. ceph-volume[-systemd] are now in sbin.
> > > 11. cephfs-mirror units are added to avoid package QA issue.
> >
> > Also good.
> >
> > As I mentioned, we'll ignore the TMPDIR QA issues, this is going to
> > be an incremental process to bring ceph back into something that is updated.
> >
> > Bruce
> >
> > >
> > > Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
> > > ---
> > > .../ceph/0001-avoid-to_string-error.patch | 73 +++++++
> > > ...h-fix-build-errors-for-cross-compile.patch | 189
> > > ------------------ ...001-cmake-add-support-for-python3.11.patch
> > > | 31 --- .../ceph/0001-delete-install-layout-deb.patch | 37 ++++
> > > recipes-extended/ceph/ceph_18.2.0.bb | 61 +++---
> > > 5 files changed, 136 insertions(+), 255 deletions(-) create mode
> > > 100644 recipes-extended/ceph/ceph/0001-avoid-to_string-error.patch
> > > delete mode 100644
> > > recipes-extended/ceph/ceph/0001-ceph-fix-build-errors-for-cross-co
> > > mpile.patch delete mode 100644
> > > recipes-extended/ceph/ceph/0001-cmake-add-support-for-python3.11.p
> > > atch create mode 100644
> > > recipes-extended/ceph/ceph/0001-delete-install-layout-deb.patch
> > >
> > > diff --git
> > > a/recipes-extended/ceph/ceph/0001-avoid-to_string-error.patch
> > > b/recipes-extended/ceph/ceph/0001-avoid-to_string-error.patch
> > > new file mode 100644
> > > index 00000000..0b4fc984
> > > --- /dev/null
> > > +++ b/recipes-extended/ceph/ceph/0001-avoid-to_string-error.patch
> > > @@ -0,0 +1,73 @@
> > > +From f807220d13adc0656c30d3207d11c70360b88d06 Mon Sep 17 00:00:00
> > > +2001
> > > +From: Chen Qi <Qi.Chen@windriver.com>
> > > +Date: Wed, 13 Mar 2024 03:14:55 -0700
> > > +Subject: [PATCH] avoid to_string error
> > > +
> > > +Upstream-Status: Pending
> > > +
> > > +Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
> > > +---
> > > + src/rgw/rgw_asio_client.cc | 15 ++++++++-------
> > > + 1 file changed, 8 insertions(+), 7 deletions(-)
> > > +
> > > +diff --git a/src/rgw/rgw_asio_client.cc
> > > +b/src/rgw/rgw_asio_client.cc index a0ec0bf5c..17880eda5 100644
> > > +--- a/src/rgw/rgw_asio_client.cc
> > > ++++ b/src/rgw/rgw_asio_client.cc
> > > +@@ -3,6 +3,7 @@
> > > +
> > > + #include <boost/algorithm/string/predicate.hpp>
> > > + #include <boost/asio/write.hpp>
> > > ++#include <string_view>
> > > +
> > > + #include "rgw_asio_client.h"
> > > + #include "rgw_perf_counters.h"
> > > +@@ -39,11 +40,11 @@ int ClientIO::init_env(CephContext *cct)
> > > + const auto& value = header->value();
> > > +
> > > + if (field == beast::http::field::content_length) {
> > > +- env.set("CONTENT_LENGTH", value.to_string());
> > > ++ env.set("CONTENT_LENGTH", std::string(value));
> > > + continue;
> > > + }
> > > + if (field == beast::http::field::content_type) {
> > > +- env.set("CONTENT_TYPE", value.to_string());
> > > ++ env.set("CONTENT_TYPE", std::string(value));
> > > + continue;
> > > + }
> > > +
> > > +@@ -62,26 +63,26 @@ int ClientIO::init_env(CephContext *cct)
> > > + }
> > > + *dest = '\0';
> > > +
> > > +- env.set(buf, value.to_string());
> > > ++ env.set(buf, std::string(value));
> > > + }
> > > +
> > > + int major = request.version() / 10;
> > > + int minor = request.version() % 10;
> > > + env.set("HTTP_VERSION", std::to_string(major) + '.' +
> > > + std::to_string(minor));
> > > +
> > > +- env.set("REQUEST_METHOD",
> > > +request.method_string().to_string());
> > > ++ env.set("REQUEST_METHOD",
> > > ++ std::string(request.method_string()));
> > > +
> > > + // split uri from query
> > > + auto uri = request.target();
> > > + auto pos = uri.find('?');
> > > + if (pos != uri.npos) {
> > > + auto query = uri.substr(pos + 1);
> > > +- env.set("QUERY_STRING", query.to_string());
> > > ++ env.set("QUERY_STRING", std::string(query));
> > > + uri = uri.substr(0, pos);
> > > + }
> > > +- env.set("SCRIPT_URI", uri.to_string());
> > > ++ env.set("SCRIPT_URI", std::string(uri));
> > > +
> > > +- env.set("REQUEST_URI", request.target().to_string());
> > > ++ env.set("REQUEST_URI", std::string(request.target()));
> > > +
> > > + char port_buf[16];
> > > + snprintf(port_buf, sizeof(port_buf), "%d",
> > > +local_endpoint.port());
> > > +--
> > > +2.42.0
> > > +
> > > diff --git
> > > a/recipes-extended/ceph/ceph/0001-ceph-fix-build-errors-for-cross-
> > > compile.patch
> > > b/recipes-extended/ceph/ceph/0001-ceph-fix-build-errors-for-cross-
> > > compile.patch
> > > deleted file mode 100644
> > > index 9686becb..00000000
> > > ---
> > > a/recipes-extended/ceph/ceph/0001-ceph-fix-build-errors-for-cross-
> > > compile.patch
> > > +++ /dev/null
> > > @@ -1,189 +0,0 @@
> > > -From 4712fe18405ffea31405308357a8e7fca358bcce Mon Sep 17 00:00:00
> > > 2001
> > > -From: Dengke Du <dengke.du@windriver.com>
> > > -Date: Mon, 11 Mar 2019 09:14:09 +0800
> > > -Subject: [PATCH] ceph: fix build errors for cross compile
> > > -
> > > -1. set the cross compile sysroot to find the rocksdb library -2.
> > > correct the install path for library in Distutils.cmake
> > > -
> > > -Upstream-Status: Inappropriate [oe specific]
> > > -
> > > -Signed-off-by: Dengke Du <dengke.du@windriver.com>
> > > -
> > > -Adjust context for v14.2.3
> > > -
> > > -Signed-off-by: He Zhe <zhe.he@windriver.com>
> > > -Signed-off-by: Sakib Sajal <sakib.sajal@windriver.com>
> > > ----
> > > - cmake/modules/Distutils.cmake | 25 +++++--------------------
> > > - cmake/modules/FindRocksDB.cmake | 4 ++--
> > > - src/compressor/zstd/CMakeLists.txt | 2 +-
> > > - src/pybind/cephfs/setup.py | 8 --------
> > > - src/pybind/rados/setup.py | 8 --------
> > > - src/pybind/rbd/setup.py | 8 --------
> > > - src/pybind/rgw/setup.py | 8 --------
> > > - 7 files changed, 8 insertions(+), 55 deletions(-)
> > > -
> > > -Index: ceph-18.2.0/cmake/modules/Distutils.cmake
> > > -=================================================================
> > > ==
> > > ---- ceph-18.2.0.orig/cmake/modules/Distutils.cmake
> > > -+++ ceph-18.2.0/cmake/modules/Distutils.cmake
> > > -@@ -29,17 +29,10 @@
> > > - cmake_parse_arguments(DU "" "INSTALL_SCRIPT" "" ${ARGN})
> > > - install(CODE "
> > > - set(options --prefix=${CMAKE_INSTALL_PREFIX})
> > > -- if(DEFINED ENV{DESTDIR})
> > > -- if(EXISTS /etc/debian_version)
> > > -- list(APPEND options --install-layout=deb)
> > > -- endif()
> > > -- list(APPEND options
> > > -- --root=\$ENV{DESTDIR}
> > > -- --single-version-externally-managed)
> > > -- endif()
> > > - if(NOT \"${DU_INSTALL_SCRIPT}\" STREQUAL \"\")
> > > - list(APPEND options --install-script=${DU_INSTALL_SCRIPT})
> > > -- endif()
> > > -+ list(APPEND options --root=${CMAKE_DESTDIR})
> > > -+ list(APPEND options
> > > -+ --install-lib=${PYTHON_SITEPACKAGES_DIR})
> > > - execute_process(
> > > - COMMAND ${Python3_EXECUTABLE}
> > > - setup.py install \${options}
> > > -@@ -65,7 +58,7 @@
> > > - if(DU_DISABLE_VTA AND HAS_VTA)
> > > - list(APPEND PY_CFLAGS -fno-var-tracking-assignments)
> > > - endif()
> > > -- list(APPEND PY_CPPFLAGS -iquote${CMAKE_SOURCE_DIR}/src/include
> > > -w)
> > > -+ list(APPEND PY_CPPFLAGS -iquote${CMAKE_SOURCE_DIR}/src/include
> > > -+ -w --sysroot=${CMAKE_SYSROOT})
> > > - # This little bit of magic wipes out __Pyx_check_single_interpreter()
> > > - # Note: this is reproduced in distutils_install_cython_module
> > > - list(APPEND PY_CPPFLAGS -D'void0=dead_function\(void\)')
> > > -@@ -135,14 +128,8 @@
> > > - set(ENV{CEPH_LIBDIR} \"${CMAKE_LIBRARY_OUTPUT_DIRECTORY}\")
> > > -
> > > - set(options --prefix=${CMAKE_INSTALL_PREFIX})
> > > -- if(DEFINED ENV{DESTDIR})
> > > -- if(EXISTS /etc/debian_version)
> > > -- list(APPEND options --install-layout=deb)
> > > -- endif()
> > > -- list(APPEND options --root=\$ENV{DESTDIR})
> > > -- else()
> > > -- list(APPEND options --root=/)
> > > -- endif()
> > > -+ list(APPEND options --root=${CMAKE_DESTDIR})
> > > -+ list(APPEND options
> > > -+ --install-lib=${PYTHON_SITEPACKAGES_DIR})
> > > - execute_process(
> > > - COMMAND
> > > - ${Python3_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/setup.py
> > > -Index: ceph-18.2.0/cmake/modules/FindRocksDB.cmake
> > > -=================================================================
> > > ==
> > > ---- ceph-18.2.0.orig/cmake/modules/FindRocksDB.cmake
> > > -+++ ceph-18.2.0/cmake/modules/FindRocksDB.cmake
> > > -@@ -9,9 +9,9 @@
> > > - # ROCKSDB_VERSION_MINOR
> > > - # ROCKSDB_VERSION_PATCH
> > > -
> > > --find_path(ROCKSDB_INCLUDE_DIR rocksdb/db.h)
> > > -+find_path(ROCKSDB_INCLUDE_DIR rocksdb/db.h ${CMAKE_SYSROOT})
> > > -
> > > --find_library(ROCKSDB_LIBRARIES rocksdb)
> > > -+find_library(ROCKSDB_LIBRARIES rocksdb ${CMAKE_SYSROOT})
> > > -
> > > - if(ROCKSDB_INCLUDE_DIR AND EXISTS "${ROCKSDB_INCLUDE_DIR}/rocksdb/version.h")
> > > - foreach(ver "MAJOR" "MINOR" "PATCH")
> > > -Index: ceph-18.2.0/src/pybind/cephfs/setup.py
> > > -=================================================================
> > > ==
> > > ---- ceph-18.2.0.orig/src/pybind/cephfs/setup.py
> > > -+++ ceph-18.2.0/src/pybind/cephfs/setup.py
> > > -@@ -135,20 +135,6 @@
> > > - finally:
> > > - shutil.rmtree(tmp_dir)
> > > -
> > > --
> > > --if 'BUILD_DOC' in os.environ or 'READTHEDOCS' in os.environ:
> > > -- ext_args = {}
> > > -- cython_constants = dict(BUILD_DOC=True)
> > > -- cythonize_args = dict(compile_time_env=cython_constants)
> > > --elif check_sanity():
> > > -- ext_args = get_python_flags(['cephfs'])
> > > -- cython_constants = dict(BUILD_DOC=False)
> > > -- include_path = [os.path.join(os.path.dirname(__file__), "..", "rados")]
> > > -- cythonize_args = dict(compile_time_env=cython_constants,
> > > -- include_path=include_path)
> > > --else:
> > > -- sys.exit(1)
> > > --
> > > - cmdclass = {}
> > > - try:
> > > - from Cython.Build import cythonize
> > > -Index: ceph-18.2.0/src/pybind/rados/setup.py
> > > -=================================================================
> > > ==
> > > ---- ceph-18.2.0.orig/src/pybind/rados/setup.py
> > > -+++ ceph-18.2.0/src/pybind/rados/setup.py
> > > -@@ -130,17 +130,6 @@
> > > - finally:
> > > - shutil.rmtree(tmp_dir)
> > > -
> > > --
> > > --if 'BUILD_DOC' in os.environ or 'READTHEDOCS' in os.environ:
> > > -- ext_args = {}
> > > -- cython_constants = dict(BUILD_DOC=True)
> > > --elif check_sanity():
> > > -- ext_args = get_python_flags(['rados'])
> > > -- cython_constants = dict(BUILD_DOC=False)
> > > --else:
> > > -- sys.exit(1)
> > > --
> > > --cmdclass = {}
> > > - try:
> > > - from Cython.Build import cythonize
> > > - from Cython.Distutils import build_ext
> > > -Index: ceph-18.2.0/src/pybind/rbd/setup.py
> > > -=================================================================
> > > ==
> > > ---- ceph-18.2.0.orig/src/pybind/rbd/setup.py
> > > -+++ ceph-18.2.0/src/pybind/rbd/setup.py
> > > -@@ -133,20 +133,6 @@
> > > - finally:
> > > - shutil.rmtree(tmp_dir)
> > > -
> > > --
> > > --if 'BUILD_DOC' in os.environ or 'READTHEDOCS' in os.environ:
> > > -- ext_args = {}
> > > -- cython_constants = dict(BUILD_DOC=True)
> > > -- cythonize_args = dict(compile_time_env=cython_constants)
> > > --elif check_sanity():
> > > -- ext_args = get_python_flags(['rados', 'rbd'])
> > > -- cython_constants = dict(BUILD_DOC=False)
> > > -- include_path = [os.path.join(os.path.dirname(__file__), "..", "rados")]
> > > -- cythonize_args = dict(compile_time_env=cython_constants,
> > > -- include_path=include_path)
> > > --else:
> > > -- sys.exit(1)
> > > --
> > > - cmdclass = {}
> > > - try:
> > > - from Cython.Build import cythonize
> > > -Index: ceph-18.2.0/src/pybind/rgw/setup.py
> > > -=================================================================
> > > ==
> > > ---- ceph-18.2.0.orig/src/pybind/rgw/setup.py
> > > -+++ ceph-18.2.0/src/pybind/rgw/setup.py
> > > -@@ -134,20 +134,6 @@
> > > - finally:
> > > - shutil.rmtree(tmp_dir)
> > > -
> > > --
> > > --if 'BUILD_DOC' in os.environ or 'READTHEDOCS' in os.environ:
> > > -- ext_args = {}
> > > -- cython_constants = dict(BUILD_DOC=True)
> > > -- cythonize_args = dict(compile_time_env=cython_constants)
> > > --elif check_sanity():
> > > -- ext_args = get_python_flags(['rados', 'rgw'])
> > > -- cython_constants = dict(BUILD_DOC=False)
> > > -- include_path = [os.path.join(os.path.dirname(__file__), "..", "rados")]
> > > -- cythonize_args = dict(compile_time_env=cython_constants,
> > > -- include_path=include_path)
> > > --else:
> > > -- sys.exit(1)
> > > --
> > > - cmdclass = {}
> > > - try:
> > > - from Cython.Build import cythonize
> > > diff --git
> > > a/recipes-extended/ceph/ceph/0001-cmake-add-support-for-python3.11
> > > .patch
> > > b/recipes-extended/ceph/ceph/0001-cmake-add-support-for-python3.11
> > > .patch
> > > deleted file mode 100644
> > > index c72c91b2..00000000
> > > ---
> > > a/recipes-extended/ceph/ceph/0001-cmake-add-support-for-python3.11
> > > .patch
> > > +++ /dev/null
> > > @@ -1,31 +0,0 @@
> > > -From 1060f2e4362ebd6db23870d442dcd158d219ee92 Mon Sep 17 00:00:00
> > > 2001
> > > -From: Yanfei Xu <yanfei.xu@windriver.com>
> > > -Date: Tue, 10 Nov 2020 17:17:30 +0800
> > > -Subject: [PATCH] cmake: add support for python 3.9 and 3.10
> > > -
> > > -add support for python3.9.
> > > -
> > > -Signed-off-by: Yanfei Xu <yanfei.xu@windriver.com>
> > > -
> > > -Add support for python 3.10.
> > > -
> > > -Upstream-Status: Submitted
> > > [https://github.com/ceph/ceph/pull/43630]
> > > -
> > > -Signed-off-by: Kai Kang <kai.kang@windriver.com>
> > > ----
> > > - cmake/modules/FindPython/Support.cmake | 2 +-
> > > - 1 file changed, 1 insertion(+), 1 deletion(-)
> > > -
> > > -Index: ceph-18.2.0/cmake/modules/FindPython/Support.cmake
> > > -=================================================================
> > > ==
> > > ---- ceph-18.2.0.orig/cmake/modules/FindPython/Support.cmake
> > > -+++ ceph-18.2.0/cmake/modules/FindPython/Support.cmake
> > > -@@ -17,7 +17,7 @@
> > > - message (FATAL_ERROR "FindPython: INTERNAL ERROR")
> > > - endif()
> > > - if (_${_PYTHON_PREFIX}_REQUIRED_VERSION_MAJOR EQUAL 3)
> > > -- set(_${_PYTHON_PREFIX}_VERSIONS 3.10 3.9 3.8 3.7 3.6 3.5 3.4
> > > 3.3 3.2 3.1 3.0)
> > > -+ set(_${_PYTHON_PREFIX}_VERSIONS 3.11 3.10 3.9 3.8 3.7 3.6 3.5
> > > -+ 3.4 3.3 3.2 3.1 3.0)
> > > - elseif (_${_PYTHON_PREFIX}_REQUIRED_VERSION_MAJOR EQUAL 2)
> > > - set(_${_PYTHON_PREFIX}_VERSIONS 2.7 2.6 2.5 2.4 2.3 2.2 2.1 2.0)
> > > - else()
> > > diff --git
> > > a/recipes-extended/ceph/ceph/0001-delete-install-layout-deb.patch
> > > b/recipes-extended/ceph/ceph/0001-delete-install-layout-deb.patch
> > > new file mode 100644
> > > index 00000000..91eacfa9
> > > --- /dev/null
> > > +++ b/recipes-extended/ceph/ceph/0001-delete-install-layout-deb.pa
> > > +++ tch
> > > @@ -0,0 +1,37 @@
> > > +From 903bb882a44eb5567f8b1fc7f7c4857c2f03579d Mon Sep 17 00:00:00
> > > +2001
> > > +From: Chen Qi <Qi.Chen@windriver.com>
> > > +Date: Wed, 13 Mar 2024 03:41:47 -0700
> > > +Subject: [PATCH] delete install-layout=deb
> > > +
> > > +Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
> > > +---
> > > + cmake/modules/Distutils.cmake | 6 ------
> > > + 1 file changed, 6 deletions(-)
> > > +
> > > +diff --git a/cmake/modules/Distutils.cmake
> > > +b/cmake/modules/Distutils.cmake index daaae4ba6..e606e3890 100644
> > > +--- a/cmake/modules/Distutils.cmake
> > > ++++ b/cmake/modules/Distutils.cmake
> > > +@@ -30,9 +30,6 @@ function(distutils_install_module name)
> > > + install(CODE "
> > > + set(options --prefix=${CMAKE_INSTALL_PREFIX})
> > > + if(DEFINED ENV{DESTDIR})
> > > +- if(EXISTS /etc/debian_version)
> > > +- list(APPEND options --install-layout=deb)
> > > +- endif()
> > > + list(APPEND options
> > > + --root=\$ENV{DESTDIR}
> > > + --single-version-externally-managed)
> > > +@@ -136,9 +133,6 @@ function(distutils_install_cython_module
> > > +name)
> > > +
> > > + set(options --prefix=${CMAKE_INSTALL_PREFIX})
> > > + if(DEFINED ENV{DESTDIR})
> > > +- if(EXISTS /etc/debian_version)
> > > +- list(APPEND options --install-layout=deb)
> > > +- endif()
> > > + list(APPEND options --root=\$ENV{DESTDIR})
> > > + else()
> > > + list(APPEND options --root=/)
> > > +--
> > > +2.42.0
> > > +
> > > diff --git a/recipes-extended/ceph/ceph_18.2.0.bb
> > > b/recipes-extended/ceph/ceph_18.2.0.bb
> > > index 35188106..ca713bc6 100644
> > > --- a/recipes-extended/ceph/ceph_18.2.0.bb
> > > +++ b/recipes-extended/ceph/ceph_18.2.0.bb
> > > @@ -9,10 +9,10 @@ inherit cmake pkgconfig python3native
> > > python3-dir systemd # pybind mix cmake and python setup environment, would case a lot of errors.
> > >
> > > SRC_URI = "http://download.ceph.com/tarballs/ceph-${PV}.tar.gz \
> > > - file://0001-ceph-fix-build-errors-for-cross-compile.patch \
> > > file://0001-fix-host-library-paths-were-used.patch \
> > > file://ceph.conf \
> > > - file://0001-cmake-add-support-for-python3.11.patch \
> > > + file://0001-avoid-to_string-error.patch \
> > > + file://0001-delete-install-layout-deb.patch \
> > > "
> > >
> > > SRC_URI[sha256sum] = "495b63e1146c604018ae0cb29bf769b5d6235e3c95849c43513baf12bba1364d"
> > > @@ -23,8 +23,13 @@ DEPENDS = "boost bzip2 curl cryptsetup expat gperf-native \
> > > oath openldap openssl \
> > > python3 python3-native python3-cython-native python3-pyyaml-native \
> > > rabbitmq-c rocksdb snappy thrift udev \
> > > - valgrind xfsprogs zlib libgcc \
> > > + valgrind xfsprogs zlib libgcc zstd re2 \
> > > "
> > > +
> > > +
> > > +OECMAKE_C_COMPILER = "${@oecmake_map_compiler('CC', d)[0]} --sysroot=${RECIPE_SYSROOT}"
> > > +OECMAKE_CXX_COMPILER = "${@oecmake_map_compiler('CXX', d)[0]} --sysroot=${RECIPE_SYSROOT}"
> > > +
> > > SYSTEMD_SERVICE:${PN} = " \
> > > ceph-radosgw@.service \
> > > ceph-radosgw.target \
> > > @@ -34,6 +39,8 @@ SYSTEMD_SERVICE:${PN} = " \
> > > ceph-mds.target \
> > > ceph-osd@.service \
> > > ceph-osd.target \
> > > + cephfs-mirror@.service \
> > > + cephfs-mirror.target \
> > > ceph.target \
> > > ceph-rbd-mirror@.service \
> > > ceph-rbd-mirror.target \
> > > @@ -45,15 +52,20 @@ SYSTEMD_SERVICE:${PN} = " \
> > > ceph-immutable-object-cache@.service \
> > > ceph-immutable-object-cache.target \ "
> > > -OECMAKE_GENERATOR = "Unix Makefiles"
> > >
> > > -EXTRA_OECMAKE = "-DWITH_MANPAGE=OFF \
> > > +EXTRA_OECMAKE += "-DWITH_MANPAGE=OFF \
> > > + -DWITH_JAEGER=OFF \
> > > + -DWITH_SYSTEM_ZSTD=ON \
> > > -DWITH_FUSE=OFF \
> > > -DWITH_SPDK=OFF \
> > > -DWITH_LEVELDB=OFF \
> > > -DWITH_LTTNG=OFF \
> > > -DWITH_BABELTRACE=OFF \
> > > -DWITH_TESTS=OFF \
> > > + -DARROW_GANDIVA=OFF \
> > > + -DARROW_WITH_RE2=OFF \
> > > + -DWITH_RADOSGW_SELECT_PARQUET=OFF \
> > > + -DWITH_RADOSGW_ARROW_FLIGHT=OFF \
> > > -DWITH_MGR=OFF \
> > > -DWITH_MGR_DASHBOARD_FRONTEND=OFF \
> > > -DWITH_SYSTEM_BOOST=ON \ @@ -67,34 +79,6 @@
> > > EXTRA_OECMAKE = "-DWITH_MANPAGE=OFF \
> > > -DCMAKE_TOOLCHAIN_FILE:FILEPATH=${WORKDIR}/toolchain.cmake \
> > > "
> > >
> > > -EXTRA_OECMAKE += "-DThrift_INCLUDE_DIR:PATH=${STAGING_INCDIR} \
> > > - -DThrift_LIBRARIES:PATH=${STAGING_LIBDIR} \
> > > - "
> > > -
> > > -# retired options:
> > > -# -DPython3_VERSION=${PYTHON_BASEVERSION}
> > > -# -DPython3_USE_STATIC_LIBS=FALSE
> > > -# -DPython3_INCLUDE_DIR:PATH=${PYTHON_INCLUDE_DIR}
> > > -# -DPython3_LIBRARY:PATH=${PYTHON_LIBRARY}
> > > -# -DPython3_ROOT_DIR:PATH=${PYTHON_SITEPACKAGES_DIR}
> > > -# -DPython3_EXECUTABLE:PATH="${PYTHON}"
> > > -
> > > -CXXFLAGS += "${HOST_CC_ARCH} ${TOOLCHAIN_OPTIONS}"
> > > -CFLAGS += "${HOST_CC_ARCH} ${TOOLCHAIN_OPTIONS}"
> > > -
> > > -export STAGING_DIR_HOST
> > > -
> > > -do_compile:prepend() {
> > > - cmake_runcmake_build --target legacy-option-headers
> > > -}
> > > -
> > > -# do_compile() {
> > > -# ninja -v ${PARALLEL_MAKE}
> > > -# }
> > > -do_compile() {
> > > - cmake_runcmake_build --target all
> > > -}
> > > -
> > > do_configure:prepend () {
> > > echo "set( CMAKE_SYSROOT \"${RECIPE_SYSROOT}\" )" >> ${WORKDIR}/toolchain.cmake
> > > echo "set( CMAKE_DESTDIR \"${D}\" )" >>
> > > ${WORKDIR}/toolchain.cmake @@ -104,16 +88,23 @@ do_configure:prepend () {
> > > echo "set( CMAKE_C_COMPILER_FORCED TRUE )" >>
> > > ${WORKDIR}/toolchain.cmake }
> > >
> > > +do_compile:prepend() {
> > > + export BUILD_DOC=1
> > > +}
> > > +
> > > +do_install:prepend() {
> > > + export BUILD_DOC=1
> > > +}
> > > +
> > > do_install:append () {
> > > sed -i -e 's:^#!/usr/bin/python$:&3:' \
> > > -e 's:${WORKDIR}.*python3:${bindir}/python3:' \
> > > ${D}${bindir}/ceph ${D}${bindir}/ceph-crash \
> > > - ${D}${bindir}/ceph-volume ${D}${bindir}/ceph-volume-systemd
> > > + ${D}${sbindir}/ceph-volume
> > > + ${D}${sbindir}/ceph-volume-systemd
> > > find ${D} -name SOURCES.txt | xargs sed -i -e 's:${WORKDIR}::'
> > > install -d ${D}${sysconfdir}/ceph
> > > install -m 644 ${WORKDIR}/ceph.conf ${D}${sysconfdir}/ceph/
> > > install -d ${D}${systemd_unitdir}
> > > - mv ${D}${libexecdir}/systemd/system ${D}${systemd_unitdir}
> > > mv ${D}${libexecdir}/ceph/ceph-osd-prestart.sh ${D}${libdir}/ceph
> > > mv ${D}${libexecdir}/ceph/ceph_common.sh ${D}${libdir}/ceph
> > > # WITH_FUSE is set to OFF, remove ceph-fuse related units
> > > --
> > > 2.42.0
> > >
> > >
> > >
> > >
> >
> >
> > --
> > - Thou shalt not follow the NULL pointer, for chaos and madness
> > await thee at its end
> > - "Use the force Harry" - Gandalf, Star Trek II
> >
> >
> >
>
>
> --
> - Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end
> - "Use the force Harry" - Gandalf, Star Trek II
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#8637):
> https://lists.yoctoproject.org/g/meta-virtualization/message/8637
> Mute This Topic: https://lists.yoctoproject.org/mt/104902878/1050810
> Group Owner: meta-virtualization+owner@lists.yoctoproject.org
> Unsubscribe:
> https://lists.yoctoproject.org/g/meta-virtualization/unsub
> [bruce.ashfield@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II
prev parent reply other threads:[~2024-03-14 4:48 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-03-13 11:07 [meta-virtualization][master-next][PATCH] ceph: fix do_compile/do_install failures Qi.Chen
2024-03-13 13:13 ` Bruce Ashfield
[not found] ` <17BC552B8186D28C.15086@lists.yoctoproject.org>
2024-03-13 14:33 ` Bruce Ashfield
[not found] ` <17BC5986FB6642B2.27213@lists.yoctoproject.org>
2024-03-13 15:57 ` Bruce Ashfield
2024-03-14 4:48 ` Chen, Qi [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CO6PR11MB5602322B8DFE4A2C9BD9E9BEED292@CO6PR11MB5602.namprd11.prod.outlook.com \
--to=qi.chen@windriver.com \
--cc=bruce.ashfield@gmail.com \
--cc=meta-virtualization@lists.yoctoproject.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).