[Scons-users] File / Dir relative paths and SConscriptChdir

RW garlicbready at googlemail.com
Fri Sep 7 05:53:16 EDT 2018 

the reason for chdir is just because of the location of files
it makes porting the old Makefiles easier and just allows for relative
paths in the logged output making a bit easier to read
I basically have a common script sitting in one directory, and the top
level SConstruct sitting somewhere else
the layout of the files is partially dictated by the way the existing
source is setup

The variables do get expanded, but the windows slash isn't handled in the
executed script
possibly because I'm launching a different python script and it's not
handling the slashes in the parameter field

env.Execute('${TOP}/py/makeversionhdr.py ${HEADER_BUILD}/mpversion.h',
chdir=env.GetLaunchDir())

D:\Temp\micropython/py/makeversionhdr.py "build/genhdr/mpversion.h"
GEN build/genhdr/mpversion.h
Traceback (most recent call last):
  File "D:\Temp\micropython\py\makeversionhdr.py", line 107, in <module>
    make_version_header(sys.argv[1])
  File "D:\Temp\micropython\py\makeversionhdr.py", line 103, in
make_version_header
    with open(filename, 'w') as f:
FileNotFoundError: [Errno 2] No such file or directory:
'build/genhdr/mpversion.h'


On Fri, 7 Sep 2018 at 04:31, Bill Deegan <bill at baddogconsulting.com> wrote:

>
> # You shouldn't need to do this.  Python's open will handle either slash
> on windows.
> # header = env.subst('${HEADER_BUILD}/mpversion.h')
> # header = os.path.normpath(header)
>
> # you shouldn't need to do this as ${TOP}. will get expanded by execute
> # script = env.subst('${TOP}/py/makeversionhdr.py')
> # script = os.path.abspath(script)
>
> # Why do you need chdir?  Do you control the script?
> env.Execute("'${TOP}/py/makeversionhdr.py ${HEADER_BUILD}/mpversion.h",
> chdir=env.GetLaunchDir())
>
> -Bill
>
> On Thu, Sep 6, 2018 at 9:57 PM RW via Scons-users <scons-users at scons.org>
> wrote:
>
>> Hi Bill,
>> I've replaced File with os.path.normpath / os.path.abspath
>> (to make sure the slashes are the right way round for Windows or Linux)
>> and I'm now running it directly in the SConscript instead of as a builder
>> since there are no sources for this particular one
>> (but there will be for others)
>>
>> ```
>> header = env.subst('${HEADER_BUILD}/mpversion.h')
>> header = os.path.normpath(header)
>> script = env.subst('${TOP}/py/makeversionhdr.py')
>> script = os.path.abspath(script)
>> env.Execute(script + ' ' + header, chdir=env.GetLaunchDir())
>> ```
>>
>> I was using File() instead of env.File which is why the variable
>> substitution failed from the looks of things
>>
>>
>> On Fri, 7 Sep 2018 at 02:11, Bill Deegan <bill at baddogconsulting.com>
>> wrote:
>>
>>> So much that could be improved with your code example below.
>>> Firstly almost no reason to use File() or Dir(), just use a string.
>>> Secondly no need to have File(subst...) use File("${TOP}/....")
>>>
>>> And never have a builder with no sources. If it's no sources that means
>>> it can be run when the SConstruct/SConscript is processed, so use  Execute.
>>> If it can't be run when SConstruct/SConscripts are processed, then it
>>> depends on something, and you should list those as sources.  Your builder
>>> can pick and choose what of $SOURCES it uses (or nothing).
>>>
>>> What directory does your SConstruct live in?
>>>
>>> -Bill
>>>
>>> On Thu, Sep 6, 2018 at 7:56 PM RW via Scons-users <scons-users at scons.org>
>>> wrote:
>>>
>>>> I've sorted it out now so no worries, I've also removed the
>>>> SConscriptChdir() call
>>>>
>>>> what I've basically got is
>>>> a SConscript within a directory such as
>>>> D:\Temp\micropython\ports\minimal
>>>> (there being multiple ports under different directories
>>>>
>>>> This then pulls in a seperate common SConscript under a path such as
>>>> D:\Temp\micropython\tools\scons\micropy.py
>>>>
>>>> via SConscript('../../tools/scons/micropy.py', {'env': env})
>>>>
>>>> because the common script lives in a different directory I needed to
>>>> make sure the paths resolved correctly
>>>> Putting in a # at the beginning of the path seems to work best
>>>> this way it will resolve to D:\Temp\micropython\ports\minimal instead
>>>> of D:\Temp\micropython\tools\scons
>>>> which is where the top level script lives
>>>>
>>>> As a simple extract
>>>> ```
>>>> # TOP Resolves to D:\Temp\micropython
>>>> env.SetDefault(TOP=Dir('../..'))
>>>>
>>>> env.SetDefault(BUILD='build')
>>>> env.Replace(HEADER_BUILD='${BUILD}/genhdr')
>>>>
>>>> # Build the micropython version header
>>>> def build_mpversion(target, source, env):
>>>>     # Use File to make sure the directory seperator is the correct way
>>>> round
>>>>     cmd = File(env.subst('${TOP}/py/makeversionhdr.py')).abspath
>>>>     cmd += ' ' + target[0].path
>>>>     env.Execute(cmd, chdir=env.GetLaunchDir())
>>>>
>>>> tgt = env.Command('#${HEADER_BUILD}/mpversion.h', [], build_mpversion)
>>>> ```
>>>>
>>>> Note there's a lot more in this file than just this
>>>> and part of the initial exercise is to just try and keep the existing
>>>> logic / if statements
>>>> from the old Makefile intact to make sure it works before I start to
>>>> re-arrange stuff to clean it up
>>>>
>>>> One thing I have noticed is that if you provide a construction variable
>>>> as part of the name to a File or Dir
>>>> such as File('${TOP}/py/makeversionhdr.py') then run .abspath on it for
>>>> the full path
>>>> it doesn't look like it expands out the construction variable
>>>>
>>>> but this does work
>>>> env.Replace(TEST1=Dir('../..'))
>>>> env.Replace(TEST2='$TEST1\\additionaldir')
>>>> test1 = env.subst('$TEST2')
>>>> # test1 now equals D:\Temp\micropython\additionaldir
>>>>
>>>> On Thu, 6 Sep 2018 at 23:39, Bill Deegan <bill at baddogconsulting.com>
>>>> wrote:
>>>>
>>>>> What command line are you seeing?
>>>>> Get rid of your SConscriptChdir() call.
>>>>> I never use it.
>>>>> You can add as an argument to SConscript, but generally the default
>>>>> behavior just works.
>>>>>
>>>>> Chdir generally breaks parallel builds.
>>>>> It's possible the default mentioned in the manpage is incorrect. As
>>>>> far as I know all paths are specified from dir and chdir is only done if
>>>>> explicitly specified to a builder..
>>>>>
>>>>> On Thu, Sep 6, 2018 at 5:58 PM Bill Deegan <bill at baddogconsulting.com>
>>>>> wrote:
>>>>>
>>>>>> So you have two builders with empty sources?
>>>>>> Not good practice.
>>>>>>
>>>>>> Use Execute if you want it to always run..
>>>>>> -Bill
>>>>>>
>>>>>> On Thu, Sep 6, 2018 at 5:18 PM RW <garlicbready at googlemail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Bill,
>>>>>>> I think I've figured it out
>>>>>>> basically I'm trying to use relative paths which are relative to the
>>>>>>> top level SConstruct file
>>>>>>> instead of being relative to a second called script in another
>>>>>>> directory
>>>>>>>
>>>>>>> the first way around this is to use the directory parameter
>>>>>>> tgt = env.Command(File('testfile.txt', directory ='../src1'), [],
>>>>>>> build_test)
>>>>>>>
>>>>>>> the second way around this is to prefix the path with #
>>>>>>> tgt = env.Command(File('#testfile.txt'), [], build_test)
>>>>>>>
>>>>>>>
>>>>>>> On Thu, 6 Sep 2018 at 22:10, Bill Deegan <bill at baddogconsulting.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Perhaps you can explain what you're actually trying to do, rather
>>>>>>>> than providing a toy example?
>>>>>>>> That will help provide more context correct help.
>>>>>>>>
>>>>>>>> On Thu, Sep 6, 2018 at 4:23 PM RW via Scons-users <
>>>>>>>> scons-users at scons.org> wrote:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>> I've got another question, this is over the use of SConscriptChdir
>>>>>>>>> such as env.SConscriptChdir(0)
>>>>>>>>> from what I can gather when specifying a File or Dir object or
>>>>>>>>> making a call to a Command builder at least which will automatically
>>>>>>>>> convert a string target to a File object.
>>>>>>>>>
>>>>>>>>> The File / Dir object will always be relative to the current
>>>>>>>>> SConstruct file.
>>>>>>>>> Not relative to the originating SConstruct file (which is what I'm
>>>>>>>>> trying to do)
>>>>>>>>>
>>>>>>>>> To give an example
>>>>>>>>>
>>>>>>>>> src1\test1.txt - empty test file
>>>>>>>>> src1\test2.txt - empty test file
>>>>>>>>>
>>>>>>>>> src1\SConstruct
>>>>>>>>> ```
>>>>>>>>> import os
>>>>>>>>>
>>>>>>>>> EnsureSConsVersion(3, 0, 0)
>>>>>>>>>
>>>>>>>>> # Setup the construction env
>>>>>>>>> env = Environment(ENV={'PATH': os.environ['PATH']})
>>>>>>>>>
>>>>>>>>> # Set working directory to not change
>>>>>>>>> env.SConscriptChdir(0)
>>>>>>>>>
>>>>>>>>> # Load in an external script
>>>>>>>>> SConscript('../src2/testsubscript.py', {'env': env})
>>>>>>>>> ```
>>>>>>>>>
>>>>>>>>> src2\SConstruct
>>>>>>>>> ```
>>>>>>>>> # Import the scons environment from the calling script
>>>>>>>>> Import('env')
>>>>>>>>>
>>>>>>>>> def build_test(target, source, env):
>>>>>>>>>     # At this point the 'testfile.txt' will be automatically
>>>>>>>>> changed to a File class by SCons
>>>>>>>>>
>>>>>>>>>     # print the path scons thinks this file is located at
>>>>>>>>>     print(target[0].path)
>>>>>>>>>
>>>>>>>>> tgt = env.Command(File('testfile.txt'), [], build_test)
>>>>>>>>> env.Default(tgt)
>>>>>>>>>
>>>>>>>>> tgt2 = env.Command('testfile2.txt', [], build_test)
>>>>>>>>> env.Default(tgt2)
>>>>>>>>>
>>>>>>>>> # Even though SConscriptChdir is set to 0 in the originating env,
>>>>>>>>> File uses the current script as it's reference point
>>>>>>>>> # D:\Temp\testproject\src2\testfile.txt
>>>>>>>>> # D:\Temp\testproject\src2\testfile2.txt
>>>>>>>>> ```
>>>>>>>>>
>>>>>>>>> Is this expected behaviour?
>>>>>>>>> perhaps SConscriptChdir only influences when calling exe's rather
>>>>>>>>> than the use of File or Dir
>>>>>>>>> is there some public API way of setting the current directory for
>>>>>>>>> File / Dir calls?
>>>>>>>>> thats not the directory of the current SConsscript
>>>>>>>>> since builders wrap everything in a File automatically, it might
>>>>>>>>> mean I have to make sure all file paths are absolute before hand.
>>>>>>>>>
>>>>>>>>> Many Thanks
>>>>>>>>> Richard
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Scons-users mailing list
>>>>>>>>> Scons-users at scons.org
>>>>>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>>>>>
>>>>>>>> _______________________________________________
>>>> Scons-users mailing list
>>>> Scons-users at scons.org
>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>
>>> _______________________________________________
>> Scons-users mailing list
>> Scons-users at scons.org
>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20180907/bd840c06/attachment.html>

More information about the Scons-users mailing list