[Scons-users] Building object files in multiple environments

[Dan Čermák]

Thanks a lot for your extensive answer!

Do I understand it correctly that for each "device"/"project" you infer
all modules from the config file and then build them for that specific
project?

I actually wanted it kind of the other way around: all modules are build
once and then linked for each project, but the project specific stuff
should be build with an additional include path.

I must admit that I like your approach very much and I think I could use
something like that, too. Could you give me maybe a hint how your
configuration system roughly works (only if you can do that without
getting into trouble with your employer)?


Cheers,

Dan


Hua Yanghao <huayanghao at gmail.com> writes:

> I think it is much easier to introduce something called "configuration".
> Take a look at the Linux kernel or u-boot Kconfig system, where
> everything is considered equally as source code and then the
> configuration file selects which pieces you want to compile and link
> together.
>
> My current project folder structure is something like this:
> hua at grass:~/git/usw $ tree -L 1
> .
> ├── arch
> ├── boards
> ├── build_list.txt
> ├── common
> ├── configs
> ├── doc
> ├── drivers
> ├── external
> ├── include
> ├── lib
> ├── Makefile
> ├── output
> ├── README.md
> ├── SConstruct
> ├── site_scons
> ├── soc
> ├── test
> ├── TODO.md
> └── tools
>
> 14 directories, 5 files
> hua at grass:~/git/usw $
>
> where in configs/ you have all the configuration file written in
> python, for example configs/qemu_arm_vexpress_bios.py: 1
> #!/usr/bin/env python
>   2 '''
>   3 To execute:
>   4 export QEMU_AUDIO_DRV="none"
>   5 qemu-system-arm -M vexpress-a9 -nographic -kernel
> output/armv7_gcc_full/usw.elf
>   6 '''
>   7 from common import ConfigMeta
>   8 from compiler import gcc_arm_none_eabi
>   9
>  10 class Config(object):
>  11     __metaclass__ = ConfigMeta
>  12     COMPILER = gcc_arm_none_eabi.Compiler()
>  13
>  14     CONFIG_ARCH = "armv7"
>  15     CONFIG_NO_STACK = 1
>  16     CONFIG_BIOS_START = 1
>  17     CONFIG_BOOT_ENTRY = 0x60000200
>  18     CONFIG_LINK_SCRIPT = "arch/armv7/link.ld"
>  19     CONFIG_TEXT_START = 0x00000000
>  20     CONFIG_HEAP_SIZE = 0x100000
>  21     CONFIG_ARM_BASE = 0x1e000000
>  22     CONFIG_UART_BASE = 0x10009000
>  23     CONFIG_PRINT_COLOR = 1
>  24
>  25     CONFIG_IBI_HEADER = 1
>  26     CONFIG_IBI_SIZELIMIT = 0x64000
>  27     # SMP
>  28     CONFIG_CPU_NUM = 4
>  29     CONFIG_IRQ_STACK_SIZE = 0x10000
>  30     CONFIG_FIQ_STACK_SIZE = 0x10000
>  31
>  32     # Module List
>  33     MODULE_LIST = [
>  34         "arch",
>  35         "arch/armv7",
>  36     ]
>
> This is one of my simplest config file, where it selects which module
> (e.g. each folder that have a SConscript is considered a module) is
> specified in the MODULE_LIST and then in SConstruct the SConscript
> file is automatically imported.
> All build that is corresponding to a particular config file has a
> dedicated output folder:
> hua at grass:~/git/usw $ ll output/
> total 28
> drwxr-xr-x  9 hua hua 4096 Nov  8 10:30 linux64_full
> drwxr-xr-x 10 hua hua 4096 Nov  8 10:29 qemu_arm_vexpress
> drwxr-xr-x  4 hua hua 4096 Nov  8 10:30 qemu_arm_vexpress_bios
> drwxr-xr-x 11 hua hua 4096 Nov  8 10:30 xmm7xx0
> drwxr-xr-x 11 hua hua 4096 Nov  8 10:30 xmm7xx0_xxx
> drwxr-xr-x  4 hua hua 4096 Nov  8 10:30 xmm7xx0_xxx_lmu
> drwxr-xr-x  8 hua hua 4096 Nov  8 10:30 xmm7xx0_xxx_yyy
> hua at grass:~/git/usw $
>
> So even for a single device, you could have have different
> configuration, e.g. one for boot loader/bios, one for verification SW,
> one for production SW etc.
>
> The best part I love SCons is that in the output folder there is a
> copy of the file that is actually being selected/used from the
> configuration system so you know exactly what is compiled.
>
> For each individual module the SConscript is extremely simple and
> straightforward:
>
> hua at grass:~/git/usw $ cat boards/vexpress/SConscript
> name = "usw_lib"
>
> Import('cs')
> Import('usw_files')
>
> obj_files = [
>     "board.c",
>     "pl011_device.c",
>     "test_device.c",
>     "pipe_device.c",
> ]
>
> lib_files = [
> ]
>
> if hasattr(cs, "CONFIG_LOAD_BINARY"):
>     obj_files.append(("binary.c", ["cmd.txt", "page_table.bin"]))
>
> ret = usw_files(name, lib_files, obj_files)
>
> Return('ret')
> hua at grass:~/git/usw $
>
> There you just provide a obj_files / lib_files and you can even
> specify dependency files (where you don't really want a Scons parser),
> and even specify customized compilation flags for individual files
> (Kconfig/Kbuild can also do this, but not the explicit dependency
> specification).
>
> I am trying to make this framework (basically Kconfig/Kbuild reduced
> feature set implemented using SCons) open source but company process
> is lengthy ...
> Or maybe it even makes sense to make this kind of feature as part of
> SCons itself?
>
> I hope this helps a little bit.
>
> Best Regards,
> Hua Yanghao
>
> On Wed, Nov 8, 2017 at 10:48 AM, Dan Čermák
> <dan.cermak at cgc-instruments.com> wrote:
>> Hi Folks,
>>
>> I am currently using SCons for a big mono-repo C++ Firmware, where I
>> have lots of common source files and then a directory where the actual
>> firmware for each device is (this is just a .cpp file with device
>> specific configurations and the appropriate high-level logic).
>>
>> For illustration purposes, the directory structure looks something like
>> this:
>> .
>> ├── devices
>> │   ├── Device_A
>> │   │   ├── main.cpp
>> │   │   ├── SConscript
>> │   │   ├── uart.cpp
>> │   │   └── uart.hpp
>> │   ├── Device_B
>> │   │   ├── main.cpp
>> │   │   └── SConscript
>> │   └── SConscript
>> ├── spi
>> │   └── config.cpp
>> ├── uart
>> │   └── uart.cpp
>> ├── util
>> │   └── endian.c
>> └── SConstruct
>>
>> The idea behind this is that everything outside of 'devices' is
>> considered as common files and build into object files. Every
>> subdirectory of devices should have a SConscript that creates a single
>> binary file (this is the firmware for the specific device) that is
>> linked with the common object files.
>>
>> I have achieved this by creating object files for every cpp file not in
>> devices/ and passing them to the SConscript in devices/. However, there
>> is a catch: I would like to build everything under devices with an
>> additional include path (the top level directory of the project) which
>> should not be propagated to the common object files. I therefore tried
>> creating a clone of the environment for each device in
>> devices/SConscript and then doing the following in
>> devices/Device_A/SConscript:
>>
>> Import('env_clone', 'obj')
>>
>> local_obj = env_clone.Object(Glob('*.cpp'))
>> prog = env_clone.Program('Device_A_bin', local_obj + obj)
>>
>> Return('prog')
>>
>> where env_clone is the cloned environment with the additional include
>> path and obj the list of object files that are common to all
>> devices (which have been created in the environment env from which
>> env_clone was cloned). This however causes SCons to complain, that there
>> are object files in multiple environments with the same build command.
>>
>>
>> My guess is, that my solution of cloning environments is not the correct
>> way. Does someone have an idea how to achieve this with SCons?
>>
>>
>> Thanks in advance,
>>
>> Dan
>> _______________________________________________
>> Scons-users mailing list
>> Scons-users at scons.org
>> https://pairlist4.pair.net/mailman/listinfo/scons-users
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users