Package prebuilt binaries

There are specific scenarios in which it is necessary to create packages from existing binaries, for example from 3rd parties or binaries previously built by another process or team that is not using Conan. Under these circumstances, building from sources is not what you want.

You can package the local files in the following scenarios:

  1. When you are developing your package locally and you want to quickly create a package with the built artifacts, but as you don’t want to rebuild again (clean copy) your artifacts, you don’t want to call conan create. This method will keep your local project build if you are using an IDE.

  2. When you cannot build the packages from sources (when only pre-built binaries are available) and you have them in a local directory.

  3. Same as 2 but you have the precompiled libraries in a remote repository.

Locally building binaries

Use the conan new command to create a “Hello World” C++ library example project:

  1. $ conan new cmake_lib -d name=hello -d version=0.1

This will create a Conan package project with the following structure.

  1. .
  2. ├── CMakeLists.txt
  3. ├── conanfile.py
  4. ├── include
  5. │   └── hello.h
  6. ├── src
  7. │   └── hello.cpp
  8. └── test_package
  9.     ├── CMakeLists.txt
  10.     ├── conanfile.py
  11.     └── src
  12.         └── example.cpp

We have a CMakeLists.txt file in the root, an src folder with the cpp files and, an include folder for the headers.

They also have a test_package/ folder to test that the exported package is working correctly.

Now, for every different configuration (different compilers, architectures, build_type…):

  1. We call conan install to generate the conan_toolchain.cmake file and the CMakeUserPresets.json that can be used in our IDE or calling CMake (only >= 3.23).

    1. $ conan install . -s build_type=Release

  2. We build our project calling CMake, our IDE, … etc:

    Linux, macOS

    1. $ mkdir -p build/Release
    2. $ cd build/Release
    3. $ cmake ../.. -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE=../Release/generators/conan_toolchain.cmake
    4. $ cmake --build .

    Windows

    1. $ mkdir -p build
    2. $ cd build
    3. $ cmake ..  -DCMAKE_TOOLCHAIN_FILE=generators/conan_toolchain.cmake
    4. $ cmake --build . --config Release

    Note

    As we are directly using our IDE or CMake to build the library, the build() method of the recipe is never called and could be removed.

  3. We call conan export-pkg to package the built artifacts.

    1. $ cd ../..
    2. $ conan export-pkg . -s build_type=Release
    3. ...
    4. hello/0.1: Calling package()
    5. hello/0.1 package(): Packaged 1 '.h' file: hello.h
    6. hello/0.1 package(): Packaged 1 '.a' file: libhello.a
    7. ...
    8. hello/0.1: Package '54a3ab9b777a90a13e500dd311d9cd70316e9d55' created

    Let’s deep a bit more in the package method. The generated package() method is using cmake.install() to copy the artifacts from our local folders to the Conan package.

    There is an alternative and generic package() method that could be used for any build system:

    1. def package(self):
    2.     local_include_folder = os.path.join(self.source_folder, self.cpp.source.includedirs[0])
    3.     local_lib_folder = os.path.join(self.build_folder, self.cpp.build.libdirs[0])
    4.     copy(self, "*.h", local_include_folder, os.path.join(self.package_folder, "include"), keep_path=False)
    5.     copy(self, "*.lib", local_lib_folder, os.path.join(self.package_folder, "lib"), keep_path=False)
    6.     copy(self, "*.a", local_lib_folder, os.path.join(self.package_folder, "lib"), keep_path=False)

    This package() method is copying artifacts from the following directories that, thanks to the layout(), will always point to the correct places:

    • os.path.join(self.source_folder, self.cpp.source.includedirs[0]) will always point to our local include folder.

    • os.path.join(self.build_folder, self.cpp.build.libdirs[0]) will always point to the location of the libraries when they are built, no matter if using a single-config CMake Generator or a multi-config one.

  4. We can test the built package calling conan test:

    1. $ conan test test_package/conanfile.py hello/0.1 -s build_type=Release
    3. -------- Testing the package: Running test() ----------
    4. hello/0.1 (test package): Running test()
    5. hello/0.1 (test package): RUN: ./example
    6. hello/0.1: Hello World Release!
    7.   hello/0.1: __x86_64__ defined
    8.   hello/0.1: __cplusplus199711
    9.   hello/0.1: __GNUC__4
    10.   hello/0.1: __GNUC_MINOR__2
    11.   hello/0.1: __clang_major__13
    12.   hello/0.1: __clang_minor__1
    13.   hello/0.1: __apple_build_version__13160021

Now you can try to generate a binary package for build_type=Debug running the same steps but changing the build_type. You can repeat this process any number of times for different configurations.

Packaging already Pre-built Binaries

Please, first clone the sources to recreate this project. You can find them in the examples2 repository on GitHub:

  1. $ git clone https://github.com/conan-io/examples2.git
  2. $ cd examples2/tutorial/creating_packages/other_packages/prebuilt_binaries

This is an example of scenario 2 explained in the introduction. If you have a local folder containing the binaries for different configurations you can package them using the following approach.

These are the files of our example, (be aware that the library files are only empty files so not valid libraries):

  1. .
  2. ├── conanfile.py
  3. └── vendor_hello_library
  4.     ├── linux
  5.     │   ├── armv8
  6.     │   │   ├── include
  7.     │   │   │   └── hello.h
  8.     │   │   └── libhello.a
  9.     │   └── x86_64
  10.     │       ├── include
  11.     │       │   └── hello.h
  12.     │       └── libhello.a
  13.     ├── macos
  14.     │   ├── armv8
  15.     │   │   ├── include
  16.     │   │   │   └── hello.h
  17.     │   │   └── libhello.a
  18.     │   └── x86_64
  19.     │       ├── include
  20.     │       │   └── hello.h
  21.     │       └── libhello.a
  22.     └── windows
  23.         ├── armv8
  24.         │   ├── hello.lib
  25.         │   └── include
  26.         │       └── hello.h
  27.         └── x86_64
  28.             ├── hello.lib
  29.             └── include
  30.                 └── hello.h

We have folders with os and subfolders with arch. This the recipe of our example:

  1. import os
  2. from conan import ConanFile
  3. from conan.tools.files import copy
  6. class helloRecipe(ConanFile):
  7.     name = "hello"
  8.     version = "0.1"
  9.     settings = "os", "arch"
  11.     def layout(self):
  12.         _os = str(self.settings.os).lower()
  13.         _arch = str(self.settings.arch).lower()
  14.         self.folders.build = os.path.join("vendor_hello_library", _os, _arch)
  15.         self.folders.source = self.folders.build
  16.         self.cpp.source.includedirs = ["include"]
  17.         self.cpp.build.libdirs = ["."]
  19.     def package(self):
  20.         local_include_folder = os.path.join(self.source_folder, self.cpp.source.includedirs[0])
  21.         local_lib_folder = os.path.join(self.build_folder, self.cpp.build.libdirs[0])
  22.         copy(self, "*.h", local_include_folder, os.path.join(self.package_folder, "include"), keep_path=False)
  23.         copy(self, "*.lib", local_lib_folder, os.path.join(self.package_folder, "lib"), keep_path=False)
  24.         copy(self, "*.a", local_lib_folder, os.path.join(self.package_folder, "lib"), keep_path=False)
  26.     def package_info(self):
  27.         self.cpp_info.libs = ["hello"]

  • We are not building anything, so the build method is not useful here.

  • We can keep the same package method from the previous example because the location of the artifacts is declared by the layout().

  • Both the source folder (with headers) and the build folder (with libraries) are in the same location, in a path that follows:

    vendor_hello_library/{os}/{arch}

  • The headers are in the include subfolder of the self.source_folder (we declare it in self.cpp.source.includedirs).

  • The libraries are in the root of the self.build_folder folder (we declare self.cpp.build.libdirs = ["."]).

  • We removed the compiler and the build_type because we only have different libraries depending on the operating system and the architecture (it might be a pure C library).

Now, for each different configuration we call conan export-pkg command, later we can list the binaries so we can check we have one package for each precompiled library:

  1. $ conan export-pkg . -s os="Linux" -s arch="x86_64"
  2. $ conan export-pkg . -s os="Linux" -s arch="armv8"
  3. $ conan export-pkg . -s os="Macos" -s arch="x86_64"
  4. $ conan export-pkg . -s os="Macos" -s arch="armv8"
  5. $ conan export-pkg . -s os="Windows" -s arch="x86_64"
  6. $ conan export-pkg . -s os="Windows" -s arch="armv8"
  8. $ conan list "hello/0.1#:*"
  9. Local Cache
  10.   hello
  11.     hello/0.1
  12.       revisions
  13.         9c7634dfe0369907f569c4e583f9bc50 (2024-05-10 08:28:31 UTC)
  14.           packages
  15.             522dcea5982a3f8a5b624c16477e47195da2f84f
  16.               info
  17.                 settings
  18.                   arch: x86_64
  19.                   os: Windows
  20.             63fead0844576fc02943e16909f08fcdddd6f44b
  21.               info
  22.                 settings
  23.                   arch: x86_64
  24.                   os: Linux
  25.             82339cc4d6db7990c1830d274cd12e7c91ab18a1
  26.               info
  27.                 settings
  28.                   arch: x86_64
  29.                   os: Macos
  30.             a0cd51c51fe9010370187244af885b0efcc5b69b
  31.               info
  32.                 settings
  33.                   arch: armv8
  34.                   os: Windows
  35.             c93719558cf197f1df5a7f1d071093e26f0e44a0
  36.               info
  37.                 settings
  38.                   arch: armv8
  39.                   os: Linux
  40.             dcf68e932572755309a5f69f3cee1bede410e907
  41.               info
  42.                 settings
  43.                   arch: armv8
  44.                   os: Macos

In this example, we don’t have a test_package/ folder but you can provide one to test the packages like in the previous example.

Downloading and Packaging Pre-built Binaries

This is an example of scenario 3 explained in the introduction. If we are not building the libraries we likely have them somewhere in a remote repository. In this case, creating a complete Conan recipe, with the detailed retrieval of the binaries could be the preferred method, because it is reproducible, and the original binaries might be traced.

Please, first clone the sources to recreate this project. You can find them in the examples2 repository on GitHub:

  1. $ git clone https://github.com/conan-io/examples2.git
  2. $ cd examples2/tutorial/creating_packages/other_packages/prebuilt_remote_binaries

conanfile.py

  1.    import os
  2.    from conan.tools.files import get, copy
  3.    from conan import ConanFile
  6.    class HelloConan(ConanFile):
  7.        name = "hello"
  8.        version = "0.1"
  9.        settings = "os", "arch"
  11.        def build(self):
  12.            base_url = "https://github.com/conan-io/libhello/releases/download/0.0.1/"
  14.            _os = {"Windows": "win", "Linux": "linux", "Macos": "macos"}.get(str(self.settings.os))
  15.            _arch = str(self.settings.arch).lower()
  16.            url = "{}/{}_{}.tgz".format(base_url, _os, _arch)
  17.            get(self, url)
  19.        def package(self):
  20.            copy(self, "*.h", self.build_folder, os.path.join(self.package_folder, "include"))
  21.            copy(self, "*.lib", self.build_folder, os.path.join(self.package_folder, "lib"))
  22.            copy(self, "*.a", self.build_folder, os.path.join(self.package_folder, "lib"))
  24.        def package_info(self):
  25.            self.cpp_info.libs = ["hello"]

Typically, pre-compiled binaries come for different configurations, so the only task that the build() method has to implement is to map the settings to the different URLs.

We only need to call conan create with different settings to generate the needed packages:

  1. $ conan create . -s os="Linux" -s arch="x86_64"
  2. $ conan create . -s os="Linux" -s arch="armv8"
  3. $ conan create . -s os="Macos" -s arch="x86_64"
  4. $ conan create . -s os="Macos" -s arch="armv8"
  5. $ conan create . -s os="Windows" -s arch="x86_64"
  6. $ conan create . -s os="Windows" -s arch="armv8"
  8. $ conan list "hello/0.1#:*"
  9. Local Cache
  10.   hello
  11.     hello/0.1
  12.       revisions
  13.         d8e4debf31f0b7b5ec7ff910f76f1e2a (2024-05-10 09:13:16 UTC)
  14.           packages
  15.             522dcea5982a3f8a5b624c16477e47195da2f84f
  16.               info
  17.                 settings
  18.                   arch: x86_64
  19.                   os: Windows
  20.             63fead0844576fc02943e16909f08fcdddd6f44b
  21.               info
  22.                 settings
  23.                   arch: x86_64
  24.                   os: Linux
  25.             82339cc4d6db7990c1830d274cd12e7c91ab18a1
  26.               info
  27.                 settings
  28.                   arch: x86_64
  29.                   os: Macos
  30.             a0cd51c51fe9010370187244af885b0efcc5b69b
  31.               info
  32.                 settings
  33.                   arch: armv8
  34.                   os: Windows
  35.             c93719558cf197f1df5a7f1d071093e26f0e44a0
  36.               info
  37.                 settings
  38.                   arch: armv8
  39.                   os: Linux
  40.             dcf68e932572755309a5f69f3cee1bede410e907
  41.               info
  42.                 settings
  43.                   arch: armv8
  44.                   os: Macos

It is recommended to include also a small consuming project in a test_package folder to verify the package is correctly built, and then upload it to a Conan remote with conan upload.

The same building policies apply. Having a recipe fails if no Conan packages are created, and the --build argument is not defined. A typical approach for this kind of package could be to define a build_policy=”missing”, especially if the URLs are also under the team’s control. If they are external (on the internet), it could be better to create the packages and store them on your own Conan repository, so that the builds do not rely on third-party URLs being available.