[Scons-users] File / Dir relative paths and SConscriptChdir

Bill Deegan bill at baddogconsulting.com
Fri Sep 7 09:17:20 EDT 2018 

How about?

env.Execute([Mkdir("${TARGET.dir}"),'${TOP}/py/makeversionhdr.py
${HEADER_BUILD}/mpversion.h'])

On Fri, Sep 7, 2018 at 5:59 AM RW via Scons-users <scons-users at scons.org>
wrote:

> forget I said that, it was because I was relying on the Command builder to
> create the directory for the target
> dooh
>
> On Fri, 7 Sep 2018 at 10:53, RW <garlicbready at googlemail.com> wrote:
>
>> the reason for chdir is just because of the location of files
>> it makes porting the old Makefiles easier and just allows for relative
>> paths in the logged output making a bit easier to read
>> I basically have a common script sitting in one directory, and the top
>> level SConstruct sitting somewhere else
>> the layout of the files is partially dictated by the way the existing
>> source is setup
>>
>> The variables do get expanded, but the windows slash isn't handled in the
>> executed script
>> possibly because I'm launching a different python script and it's not
>> handling the slashes in the parameter field
>>
>> env.Execute('${TOP}/py/makeversionhdr.py ${HEADER_BUILD}/mpversion.h',
>> chdir=env.GetLaunchDir())
>>
>> D:\Temp\micropython/py/makeversionhdr.py "build/genhdr/mpversion.h"
>> GEN build/genhdr/mpversion.h
>> Traceback (most recent call last):
>>   File "D:\Temp\micropython\py\makeversionhdr.py", line 107, in <module>
>>     make_version_header(sys.argv[1])
>>   File "D:\Temp\micropython\py\makeversionhdr.py", line 103, in
>> make_version_header
>>     with open(filename, 'w') as f:
>> FileNotFoundError: [Errno 2] No such file or directory:
>> 'build/genhdr/mpversion.h'
>>
>>
>> On Fri, 7 Sep 2018 at 04:31, Bill Deegan <bill at baddogconsulting.com>
>> wrote:
>>
>>>
>>> # You shouldn't need to do this.  Python's open will handle either slash
>>> on windows.
>>> # header = env.subst('${HEADER_BUILD}/mpversion.h')
>>> # header = os.path.normpath(header)
>>>
>>> # you shouldn't need to do this as ${TOP}. will get expanded by execute
>>> # script = env.subst('${TOP}/py/makeversionhdr.py')
>>> # script = os.path.abspath(script)
>>>
>>> # Why do you need chdir?  Do you control the script?
>>> env.Execute("'${TOP}/py/makeversionhdr.py ${HEADER_BUILD}/mpversion.h",
>>> chdir=env.GetLaunchDir())
>>>
>>> -Bill
>>>
>>> On Thu, Sep 6, 2018 at 9:57 PM RW via Scons-users <scons-users at scons.org>
>>> wrote:
>>>
>>>> Hi Bill,
>>>> I've replaced File with os.path.normpath / os.path.abspath
>>>> (to make sure the slashes are the right way round for Windows or Linux)
>>>> and I'm now running it directly in the SConscript instead of as a
>>>> builder since there are no sources for this particular one
>>>> (but there will be for others)
>>>>
>>>> ```
>>>> header = env.subst('${HEADER_BUILD}/mpversion.h')
>>>> header = os.path.normpath(header)
>>>> script = env.subst('${TOP}/py/makeversionhdr.py')
>>>> script = os.path.abspath(script)
>>>> env.Execute(script + ' ' + header, chdir=env.GetLaunchDir())
>>>> ```
>>>>
>>>> I was using File() instead of env.File which is why the variable
>>>> substitution failed from the looks of things
>>>>
>>>>
>>>> On Fri, 7 Sep 2018 at 02:11, Bill Deegan <bill at baddogconsulting.com>
>>>> wrote:
>>>>
>>>>> So much that could be improved with your code example below.
>>>>> Firstly almost no reason to use File() or Dir(), just use a string.
>>>>> Secondly no need to have File(subst...) use File("${TOP}/....")
>>>>>
>>>>> And never have a builder with no sources. If it's no sources that
>>>>> means it can be run when the SConstruct/SConscript is processed, so use
>>>>> Execute.
>>>>> If it can't be run when SConstruct/SConscripts are processed, then it
>>>>> depends on something, and you should list those as sources.  Your builder
>>>>> can pick and choose what of $SOURCES it uses (or nothing).
>>>>>
>>>>> What directory does your SConstruct live in?
>>>>>
>>>>> -Bill
>>>>>
>>>>> On Thu, Sep 6, 2018 at 7:56 PM RW via Scons-users <
>>>>> scons-users at scons.org> wrote:
>>>>>
>>>>>> I've sorted it out now so no worries, I've also removed the
>>>>>> SConscriptChdir() call
>>>>>>
>>>>>> what I've basically got is
>>>>>> a SConscript within a directory such as
>>>>>> D:\Temp\micropython\ports\minimal
>>>>>> (there being multiple ports under different directories
>>>>>>
>>>>>> This then pulls in a seperate common SConscript under a path such as
>>>>>> D:\Temp\micropython\tools\scons\micropy.py
>>>>>>
>>>>>> via SConscript('../../tools/scons/micropy.py', {'env': env})
>>>>>>
>>>>>> because the common script lives in a different directory I needed to
>>>>>> make sure the paths resolved correctly
>>>>>> Putting in a # at the beginning of the path seems to work best
>>>>>> this way it will resolve to D:\Temp\micropython\ports\minimal instead
>>>>>> of D:\Temp\micropython\tools\scons
>>>>>> which is where the top level script lives
>>>>>>
>>>>>> As a simple extract
>>>>>> ```
>>>>>> # TOP Resolves to D:\Temp\micropython
>>>>>> env.SetDefault(TOP=Dir('../..'))
>>>>>>
>>>>>> env.SetDefault(BUILD='build')
>>>>>> env.Replace(HEADER_BUILD='${BUILD}/genhdr')
>>>>>>
>>>>>> # Build the micropython version header
>>>>>> def build_mpversion(target, source, env):
>>>>>>     # Use File to make sure the directory seperator is the correct
>>>>>> way round
>>>>>>     cmd = File(env.subst('${TOP}/py/makeversionhdr.py')).abspath
>>>>>>     cmd += ' ' + target[0].path
>>>>>>     env.Execute(cmd, chdir=env.GetLaunchDir())
>>>>>>
>>>>>> tgt = env.Command('#${HEADER_BUILD}/mpversion.h', [], build_mpversion)
>>>>>> ```
>>>>>>
>>>>>> Note there's a lot more in this file than just this
>>>>>> and part of the initial exercise is to just try and keep the existing
>>>>>> logic / if statements
>>>>>> from the old Makefile intact to make sure it works before I start to
>>>>>> re-arrange stuff to clean it up
>>>>>>
>>>>>> One thing I have noticed is that if you provide a construction
>>>>>> variable as part of the name to a File or Dir
>>>>>> such as File('${TOP}/py/makeversionhdr.py') then run .abspath on it
>>>>>> for the full path
>>>>>> it doesn't look like it expands out the construction variable
>>>>>>
>>>>>> but this does work
>>>>>> env.Replace(TEST1=Dir('../..'))
>>>>>> env.Replace(TEST2='$TEST1\\additionaldir')
>>>>>> test1 = env.subst('$TEST2')
>>>>>> # test1 now equals D:\Temp\micropython\additionaldir
>>>>>>
>>>>>> On Thu, 6 Sep 2018 at 23:39, Bill Deegan <bill at baddogconsulting.com>
>>>>>> wrote:
>>>>>>
>>>>>>> What command line are you seeing?
>>>>>>> Get rid of your SConscriptChdir() call.
>>>>>>> I never use it.
>>>>>>> You can add as an argument to SConscript, but generally the default
>>>>>>> behavior just works.
>>>>>>>
>>>>>>> Chdir generally breaks parallel builds.
>>>>>>> It's possible the default mentioned in the manpage is incorrect. As
>>>>>>> far as I know all paths are specified from dir and chdir is only done if
>>>>>>> explicitly specified to a builder..
>>>>>>>
>>>>>>> On Thu, Sep 6, 2018 at 5:58 PM Bill Deegan <
>>>>>>> bill at baddogconsulting.com> wrote:
>>>>>>>
>>>>>>>> So you have two builders with empty sources?
>>>>>>>> Not good practice.
>>>>>>>>
>>>>>>>> Use Execute if you want it to always run..
>>>>>>>> -Bill
>>>>>>>>
>>>>>>>> On Thu, Sep 6, 2018 at 5:18 PM RW <garlicbready at googlemail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi Bill,
>>>>>>>>> I think I've figured it out
>>>>>>>>> basically I'm trying to use relative paths which are relative to
>>>>>>>>> the top level SConstruct file
>>>>>>>>> instead of being relative to a second called script in another
>>>>>>>>> directory
>>>>>>>>>
>>>>>>>>> the first way around this is to use the directory parameter
>>>>>>>>> tgt = env.Command(File('testfile.txt', directory ='../src1'), [],
>>>>>>>>> build_test)
>>>>>>>>>
>>>>>>>>> the second way around this is to prefix the path with #
>>>>>>>>> tgt = env.Command(File('#testfile.txt'), [], build_test)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, 6 Sep 2018 at 22:10, Bill Deegan <
>>>>>>>>> bill at baddogconsulting.com> wrote:
>>>>>>>>>
>>>>>>>>>> Perhaps you can explain what you're actually trying to do, rather
>>>>>>>>>> than providing a toy example?
>>>>>>>>>> That will help provide more context correct help.
>>>>>>>>>>
>>>>>>>>>> On Thu, Sep 6, 2018 at 4:23 PM RW via Scons-users <
>>>>>>>>>> scons-users at scons.org> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>> I've got another question, this is over the use of
>>>>>>>>>>> SConscriptChdir
>>>>>>>>>>> such as env.SConscriptChdir(0)
>>>>>>>>>>> from what I can gather when specifying a File or Dir object or
>>>>>>>>>>> making a call to a Command builder at least which will automatically
>>>>>>>>>>> convert a string target to a File object.
>>>>>>>>>>>
>>>>>>>>>>> The File / Dir object will always be relative to the current
>>>>>>>>>>> SConstruct file.
>>>>>>>>>>> Not relative to the originating SConstruct file (which is what
>>>>>>>>>>> I'm trying to do)
>>>>>>>>>>>
>>>>>>>>>>> To give an example
>>>>>>>>>>>
>>>>>>>>>>> src1\test1.txt - empty test file
>>>>>>>>>>> src1\test2.txt - empty test file
>>>>>>>>>>>
>>>>>>>>>>> src1\SConstruct
>>>>>>>>>>> ```
>>>>>>>>>>> import os
>>>>>>>>>>>
>>>>>>>>>>> EnsureSConsVersion(3, 0, 0)
>>>>>>>>>>>
>>>>>>>>>>> # Setup the construction env
>>>>>>>>>>> env = Environment(ENV={'PATH': os.environ['PATH']})
>>>>>>>>>>>
>>>>>>>>>>> # Set working directory to not change
>>>>>>>>>>> env.SConscriptChdir(0)
>>>>>>>>>>>
>>>>>>>>>>> # Load in an external script
>>>>>>>>>>> SConscript('../src2/testsubscript.py', {'env': env})
>>>>>>>>>>> ```
>>>>>>>>>>>
>>>>>>>>>>> src2\SConstruct
>>>>>>>>>>> ```
>>>>>>>>>>> # Import the scons environment from the calling script
>>>>>>>>>>> Import('env')
>>>>>>>>>>>
>>>>>>>>>>> def build_test(target, source, env):
>>>>>>>>>>>     # At this point the 'testfile.txt' will be automatically
>>>>>>>>>>> changed to a File class by SCons
>>>>>>>>>>>
>>>>>>>>>>>     # print the path scons thinks this file is located at
>>>>>>>>>>>     print(target[0].path)
>>>>>>>>>>>
>>>>>>>>>>> tgt = env.Command(File('testfile.txt'), [], build_test)
>>>>>>>>>>> env.Default(tgt)
>>>>>>>>>>>
>>>>>>>>>>> tgt2 = env.Command('testfile2.txt', [], build_test)
>>>>>>>>>>> env.Default(tgt2)
>>>>>>>>>>>
>>>>>>>>>>> # Even though SConscriptChdir is set to 0 in the originating
>>>>>>>>>>> env, File uses the current script as it's reference point
>>>>>>>>>>> # D:\Temp\testproject\src2\testfile.txt
>>>>>>>>>>> # D:\Temp\testproject\src2\testfile2.txt
>>>>>>>>>>> ```
>>>>>>>>>>>
>>>>>>>>>>> Is this expected behaviour?
>>>>>>>>>>> perhaps SConscriptChdir only influences when calling exe's
>>>>>>>>>>> rather than the use of File or Dir
>>>>>>>>>>> is there some public API way of setting the current directory
>>>>>>>>>>> for File / Dir calls?
>>>>>>>>>>> thats not the directory of the current SConsscript
>>>>>>>>>>> since builders wrap everything in a File automatically, it might
>>>>>>>>>>> mean I have to make sure all file paths are absolute before hand.
>>>>>>>>>>>
>>>>>>>>>>> Many Thanks
>>>>>>>>>>> Richard
>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> Scons-users mailing list
>>>>>>>>>>> Scons-users at scons.org
>>>>>>>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>> Scons-users mailing list
>>>>>> Scons-users at scons.org
>>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>>
>>>>> _______________________________________________
>>>> Scons-users mailing list
>>>> Scons-users at scons.org
>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>
>>> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20180907/36823f60/attachment-0001.html>

More information about the Scons-users mailing list