From summerrain1 at gmx.de Thu Feb 6 11:58:11 2025 From: summerrain1 at gmx.de (summerrain1 at gmx.de) Date: Thu, 6 Feb 2025 17:58:11 +0100 Subject: [Scons-users] Debugging a build project using SCons Message-ID: <32a942ba-36c7-9300-c0e2-a79e6128d7e4@gmx.de> Hello folks, first peek into this list, and been fiddling with a SCons build structure for a somewhat complex project for a couple days. Unfortunately I'm not going the route of learning this system by starting with a small, simple project and making baby steps, but am in the less than enviable position of being tasked to fix a build system left by a "defected" colleague, which used to work, and now doesn't ;) I have been trying to do things like add print or Debug.Trace statements to SConscript files within the deep project hierarchy where I thought illuminating when what happens may be promising. But turns out there is no time correlation between when prints are put on the console and when build steps actually happen, because the prints are apparently all done when the system reads all the SConscript files. Is there still a way to do "printf style debugging" this? Is this even a good idea, or are there better ways to go about it? I have seen one stackoverflow thread with suggestions of installing some fancy IDE to debug the build scripts step-wise based on Python, but this is a headless remote server & I can't do that. I am having problems like the build failing due to some missing file, but no indication in the console output as to why and where it may have failed to get something. Trying to get a picture of things just with grep is eating a lot of time. Though I did fix one of these type of issues which turned out to be hard-coded access tokens to download fixed archives of components, which were expired & the build did not stop there despite those files failing to download - intead things exploding later... (and it wasn't even in SConscript or other clearly script type files, i.e. I'm grep'ing all files...) Regards From mats at wichmann.us Thu Feb 6 12:37:59 2025 From: mats at wichmann.us (Mats Wichmann) Date: Thu, 6 Feb 2025 10:37:59 -0700 Subject: [Scons-users] Debugging a build project using SCons In-Reply-To: <32a942ba-36c7-9300-c0e2-a79e6128d7e4@gmx.de> References: <32a942ba-36c7-9300-c0e2-a79e6128d7e4@gmx.de> Message-ID: <3724f7f4-fa03-41ad-b38c-497b4356093f@wichmann.us> On 2/6/25 09:58, summerrain1--- via Scons-users wrote: > Hello folks, > > first peek into this list, and been fiddling with a SCons build > structure for a somewhat complex project for a couple days. > > Unfortunately I'm not going the route of learning this system by > starting with a small, simple project and making baby steps, but am in > the less than enviable position of being tasked to fix a build system > left by a "defected" colleague, which used to work, and now doesn't ;) Sympathies. Yes, this does happen a fair bit (how I was introduced to SCons about 8 years ago). > I have been trying to do things like add print or Debug.Trace statements > to SConscript files within the deep project hierarchy where I thought > illuminating when what happens may be promising. > But turns out there is no time correlation between when prints are put > on the console and when build steps actually happen, because the prints > are apparently all done when the system reads all the SConscript files. Yes, there's what you could think of as two-pass behavior. All the sconscripts are read and a dependency tree is built. In this phase, all the "pure Python" that is not embedded in callback functions executes. Then the completed tree is handed to the taskmaster which figures out what needs (re)building, and dispatches workers to do so. Everything that actually builds is done via Actions, which don't run until they are needed. Also, the command lines that make up many of the Actions are written in a mini-language that allows for variable substituton, this also does not happen until it's actually needed. For example, the line to build from a C source file with gcc is: '$CC -o $TARGET -c $CFLAGS $CCFLAGS $_CCCOMCOM $SOURCES' That ends up stored in a variable in your construction environment called CCCOM. The variable references are expanded recursively, so _CCCOMCOM, which is: '$CPPFLAGS $_CPPDEFFLAGS $_CPPINCFLAGS' has to be further expanded, etc. This means you have to think a bit about how to debug, as you've found. Debug prints are great for the first phase, and if you're downloading things are you indicate further down, you should be able to diagnose those things fairly easily. > Is there still a way to do "printf style debugging" this? > Is this even a good idea, or are there better ways to go about it? > I have seen one stackoverflow thread with suggestions of installing some > fancy IDE to debug the build scripts step-wise based on Python, but this > is a headless remote server & I can't do that. Some places, prints work fine. > I am having problems like the build failing due to some missing file, > but no indication in the console output as to why and where it may have > failed to get something. Usually, missing header files are due to missing settings in $CPPPATH: https://scons.org/doc/production/HTML/scons-man.html#cv-CPPPATH Sometimes the way the build is set up obscures information, and if so you may wish to remove some of the obscuring. The build actions have corresponding string functions that say what to print when that step is executed (if omitted, the whole command line is emitted). If you're seeing brief lines like Compilng src/foo/x.c you might want to go look for settings of variables like CCCOMSTR and temporarily comment those out. Let us know if these are other kinds of missing files... sometimes generated sources don't get generated, external dependencies don't get put into place, and so forth. We may be able to recognize some messages. > Trying to get a picture of things just with grep is eating a lot of > time. Though I did fix one of these type of issues which turned out to > be hard-coded access tokens to download fixed archives of components, > which were expired & the build did not stop there despite those files > failing to download - intead things exploding later... (and it wasn't > even in SConscript or other clearly script type files, i.e. I'm grep'ing > all files...) As mentioned, getting "externals" or "third party" bits in place can be a challenge, so definitely keep checking that. There are some tricks to getting those in place so SCons can track them, rather than just declaring not found and giving up. From bill at baddogconsulting.com Thu Feb 6 13:51:20 2025 From: bill at baddogconsulting.com (Bill Deegan) Date: Thu, 6 Feb 2025 10:51:20 -0800 Subject: [Scons-users] Debugging a build project using SCons In-Reply-To: <3724f7f4-fa03-41ad-b38c-497b4356093f@wichmann.us> References: <32a942ba-36c7-9300-c0e2-a79e6128d7e4@gmx.de> <3724f7f4-fa03-41ad-b38c-497b4356093f@wichmann.us> Message-ID: If you'd like some consulting help with this, I'm available. ( www.baddogconsulting.com) Can you roll back your SCM (git) to a point where the build still works? (Use git bisect if you're using git) That may be the easiest way to track down what changed. -Bill SCons Project Co-Manager On Thu, Feb 6, 2025 at 9:38?AM Mats Wichmann wrote: > On 2/6/25 09:58, summerrain1--- via Scons-users wrote: > > Hello folks, > > > > first peek into this list, and been fiddling with a SCons build > > structure for a somewhat complex project for a couple days. > > > > Unfortunately I'm not going the route of learning this system by > > starting with a small, simple project and making baby steps, but am in > > the less than enviable position of being tasked to fix a build system > > left by a "defected" colleague, which used to work, and now doesn't ;) > > Sympathies. Yes, this does happen a fair bit (how I was introduced to > SCons about 8 years ago). > > > I have been trying to do things like add print or Debug.Trace statements > > to SConscript files within the deep project hierarchy where I thought > > illuminating when what happens may be promising. > > But turns out there is no time correlation between when prints are put > > on the console and when build steps actually happen, because the prints > > are apparently all done when the system reads all the SConscript files. > > Yes, there's what you could think of as two-pass behavior. All the > sconscripts are read and a dependency tree is built. In this phase, all > the "pure Python" that is not embedded in callback functions executes. > Then the completed tree is handed to the taskmaster which figures out > what needs (re)building, and dispatches workers to do so. > > Everything that actually builds is done via Actions, which don't run > until they are needed. Also, the command lines that make up many of the > Actions are written in a mini-language that allows for variable > substituton, this also does not happen until it's actually needed. For > example, the line to build from a C source file with gcc is: > > '$CC -o $TARGET -c $CFLAGS $CCFLAGS $_CCCOMCOM $SOURCES' > > That ends up stored in a variable in your construction environment > called CCCOM. > > The variable references are expanded recursively, so _CCCOMCOM, which is: > > '$CPPFLAGS $_CPPDEFFLAGS $_CPPINCFLAGS' > > has to be further expanded, etc. > > This means you have to think a bit about how to debug, as you've found. > Debug prints are great for the first phase, and if you're downloading > things are you indicate further down, you should be able to diagnose > those things fairly easily. > > > Is there still a way to do "printf style debugging" this? > > Is this even a good idea, or are there better ways to go about it? > > I have seen one stackoverflow thread with suggestions of installing some > > fancy IDE to debug the build scripts step-wise based on Python, but this > > is a headless remote server & I can't do that. > > Some places, prints work fine. > > > I am having problems like the build failing due to some missing file, > > but no indication in the console output as to why and where it may have > > failed to get something. > > Usually, missing header files are due to missing settings in $CPPPATH: > > https://scons.org/doc/production/HTML/scons-man.html#cv-CPPPATH > > Sometimes the way the build is set up obscures information, and if so > you may wish to remove some of the obscuring. The build actions have > corresponding string functions that say what to print when that step is > executed (if omitted, the whole command line is emitted). If you're > seeing brief lines like > > Compilng src/foo/x.c > > you might want to go look for settings of variables like CCCOMSTR and > temporarily comment those out. > > Let us know if these are other kinds of missing files... sometimes > generated sources don't get generated, external dependencies don't get > put into place, and so forth. We may be able to recognize some messages. > > > Trying to get a picture of things just with grep is eating a lot of > > time. Though I did fix one of these type of issues which turned out to > > be hard-coded access tokens to download fixed archives of components, > > which were expired & the build did not stop there despite those files > > failing to download - intead things exploding later... (and it wasn't > > even in SConscript or other clearly script type files, i.e. I'm grep'ing > > all files...) > > As mentioned, getting "externals" or "third party" bits in place can be > a challenge, so definitely keep checking that. There are some tricks to > getting those in place so SCons can track them, rather than just > declaring not found and giving up. > > > > > _______________________________________________ > Scons-users mailing list > Scons-users at scons.org > https://pairlist4.pair.net/mailman/listinfo/scons-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: