[Scons-users] File / Dir relative paths and SConscriptChdir

Fri Sep 7 05:58:49 EDT 2018

forget I said that, it was because I was relying on the Command builder to
create the directory for the target
dooh

On Fri, 7 Sep 2018 at 10:53, RW <garlicbready at googlemail.com> wrote:

> the reason for chdir is just because of the location of files
> it makes porting the old Makefiles easier and just allows for relative
> paths in the logged output making a bit easier to read
> I basically have a common script sitting in one directory, and the top
> level SConstruct sitting somewhere else
> the layout of the files is partially dictated by the way the existing
> source is setup
>
> The variables do get expanded, but the windows slash isn't handled in the
> executed script
> possibly because I'm launching a different python script and it's not
> handling the slashes in the parameter field
>
> env.Execute('${TOP}/py/makeversionhdr.py ${HEADER_BUILD}/mpversion.h',
> chdir=env.GetLaunchDir())
>
> D:\Temp\micropython/py/makeversionhdr.py "build/genhdr/mpversion.h"
> GEN build/genhdr/mpversion.h
> Traceback (most recent call last):
>   File "D:\Temp\micropython\py\makeversionhdr.py", line 107, in <module>
>     make_version_header(sys.argv[1])
>   File "D:\Temp\micropython\py\makeversionhdr.py", line 103, in
> make_version_header
>     with open(filename, 'w') as f:
> FileNotFoundError: [Errno 2] No such file or directory:
> 'build/genhdr/mpversion.h'
>
>
> On Fri, 7 Sep 2018 at 04:31, Bill Deegan <bill at baddogconsulting.com>
> wrote:
>
>>
>> # You shouldn't need to do this.  Python's open will handle either slash
>> on windows.
>> # header = env.subst('${HEADER_BUILD}/mpversion.h')
>> # header = os.path.normpath(header)
>>
>> # you shouldn't need to do this as ${TOP}. will get expanded by execute
>> # script = env.subst('${TOP}/py/makeversionhdr.py')
>> # script = os.path.abspath(script)
>>
>> # Why do you need chdir?  Do you control the script?
>> env.Execute("'${TOP}/py/makeversionhdr.py ${HEADER_BUILD}/mpversion.h",
>> chdir=env.GetLaunchDir())
>>
>> -Bill
>>
>> On Thu, Sep 6, 2018 at 9:57 PM RW via Scons-users <scons-users at scons.org>
>> wrote:
>>
>>> Hi Bill,
>>> I've replaced File with os.path.normpath / os.path.abspath
>>> (to make sure the slashes are the right way round for Windows or Linux)
>>> and I'm now running it directly in the SConscript instead of as a
>>> builder since there are no sources for this particular one
>>> (but there will be for others)
>>>
>>> ```
>>> header = env.subst('${HEADER_BUILD}/mpversion.h')
>>> header = os.path.normpath(header)
>>> script = env.subst('${TOP}/py/makeversionhdr.py')
>>> script = os.path.abspath(script)
>>> env.Execute(script + ' ' + header, chdir=env.GetLaunchDir())
>>> ```
>>>
>>> I was using File() instead of env.File which is why the variable
>>> substitution failed from the looks of things
>>>
>>>
>>> On Fri, 7 Sep 2018 at 02:11, Bill Deegan <bill at baddogconsulting.com>
>>> wrote:
>>>
>>>> So much that could be improved with your code example below.
>>>> Firstly almost no reason to use File() or Dir(), just use a string.
>>>> Secondly no need to have File(subst...) use File("${TOP}/....")
>>>>
>>>> And never have a builder with no sources. If it's no sources that means
>>>> it can be run when the SConstruct/SConscript is processed, so use  Execute.
>>>> If it can't be run when SConstruct/SConscripts are processed, then it
>>>> depends on something, and you should list those as sources.  Your builder
>>>> can pick and choose what of $SOURCES it uses (or nothing).
>>>>
>>>> What directory does your SConstruct live in?
>>>>
>>>> -Bill
>>>>
>>>> On Thu, Sep 6, 2018 at 7:56 PM RW via Scons-users <
>>>> scons-users at scons.org> wrote:
>>>>
>>>>> I've sorted it out now so no worries, I've also removed the
>>>>> SConscriptChdir() call
>>>>>
>>>>> what I've basically got is
>>>>> a SConscript within a directory such as
>>>>> D:\Temp\micropython\ports\minimal
>>>>> (there being multiple ports under different directories
>>>>>
>>>>> This then pulls in a seperate common SConscript under a path such as
>>>>> D:\Temp\micropython\tools\scons\micropy.py
>>>>>
>>>>> via SConscript('../../tools/scons/micropy.py', {'env': env})
>>>>>
>>>>> because the common script lives in a different directory I needed to
>>>>> make sure the paths resolved correctly
>>>>> Putting in a # at the beginning of the path seems to work best
>>>>> this way it will resolve to D:\Temp\micropython\ports\minimal instead
>>>>> of D:\Temp\micropython\tools\scons
>>>>> which is where the top level script lives
>>>>>
>>>>> As a simple extract
>>>>> ```
>>>>> # TOP Resolves to D:\Temp\micropython
>>>>> env.SetDefault(TOP=Dir('../..'))
>>>>>
>>>>> env.SetDefault(BUILD='build')
>>>>> env.Replace(HEADER_BUILD='${BUILD}/genhdr')
>>>>>
>>>>> # Build the micropython version header
>>>>> def build_mpversion(target, source, env):
>>>>>     # Use File to make sure the directory seperator is the correct way
>>>>> round
>>>>>     cmd = File(env.subst('${TOP}/py/makeversionhdr.py')).abspath
>>>>>     cmd += ' ' + target[0].path
>>>>>     env.Execute(cmd, chdir=env.GetLaunchDir())
>>>>>
>>>>> tgt = env.Command('#${HEADER_BUILD}/mpversion.h', [], build_mpversion)
>>>>> ```
>>>>>
>>>>> Note there's a lot more in this file than just this
>>>>> and part of the initial exercise is to just try and keep the existing
>>>>> logic / if statements
>>>>> from the old Makefile intact to make sure it works before I start to
>>>>> re-arrange stuff to clean it up
>>>>>
>>>>> One thing I have noticed is that if you provide a construction
>>>>> variable as part of the name to a File or Dir
>>>>> such as File('${TOP}/py/makeversionhdr.py') then run .abspath on it
>>>>> for the full path
>>>>> it doesn't look like it expands out the construction variable
>>>>>
>>>>> but this does work
>>>>> env.Replace(TEST1=Dir('../..'))
>>>>> env.Replace(TEST2='$TEST1\\additionaldir')
>>>>> test1 = env.subst('$TEST2')
>>>>> # test1 now equals D:\Temp\micropython\additionaldir
>>>>>
>>>>> On Thu, 6 Sep 2018 at 23:39, Bill Deegan <bill at baddogconsulting.com>
>>>>> wrote:
>>>>>
>>>>>> What command line are you seeing?
>>>>>> Get rid of your SConscriptChdir() call.
>>>>>> I never use it.
>>>>>> You can add as an argument to SConscript, but generally the default
>>>>>> behavior just works.
>>>>>>
>>>>>> Chdir generally breaks parallel builds.
>>>>>> It's possible the default mentioned in the manpage is incorrect. As
>>>>>> far as I know all paths are specified from dir and chdir is only done if
>>>>>> explicitly specified to a builder..
>>>>>>
>>>>>> On Thu, Sep 6, 2018 at 5:58 PM Bill Deegan <bill at baddogconsulting.com>
>>>>>> wrote:
>>>>>>
>>>>>>> So you have two builders with empty sources?
>>>>>>> Not good practice.
>>>>>>>
>>>>>>> Use Execute if you want it to always run..
>>>>>>> -Bill
>>>>>>>
>>>>>>> On Thu, Sep 6, 2018 at 5:18 PM RW <garlicbready at googlemail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Bill,
>>>>>>>> I think I've figured it out
>>>>>>>> basically I'm trying to use relative paths which are relative to
>>>>>>>> the top level SConstruct file
>>>>>>>> instead of being relative to a second called script in another
>>>>>>>> directory
>>>>>>>>
>>>>>>>> the first way around this is to use the directory parameter
>>>>>>>> tgt = env.Command(File('testfile.txt', directory ='../src1'), [],
>>>>>>>> build_test)
>>>>>>>>
>>>>>>>> the second way around this is to prefix the path with #
>>>>>>>> tgt = env.Command(File('#testfile.txt'), [], build_test)
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, 6 Sep 2018 at 22:10, Bill Deegan <bill at baddogconsulting.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Perhaps you can explain what you're actually trying to do, rather
>>>>>>>>> than providing a toy example?
>>>>>>>>> That will help provide more context correct help.
>>>>>>>>>
>>>>>>>>> On Thu, Sep 6, 2018 at 4:23 PM RW via Scons-users <
>>>>>>>>> scons-users at scons.org> wrote:
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>> I've got another question, this is over the use of SConscriptChdir
>>>>>>>>>> such as env.SConscriptChdir(0)
>>>>>>>>>> from what I can gather when specifying a File or Dir object or
>>>>>>>>>> making a call to a Command builder at least which will automatically
>>>>>>>>>> convert a string target to a File object.
>>>>>>>>>>
>>>>>>>>>> The File / Dir object will always be relative to the current
>>>>>>>>>> SConstruct file.
>>>>>>>>>> Not relative to the originating SConstruct file (which is what
>>>>>>>>>> I'm trying to do)
>>>>>>>>>>
>>>>>>>>>> To give an example
>>>>>>>>>>
>>>>>>>>>> src1\test1.txt - empty test file
>>>>>>>>>> src1\test2.txt - empty test file
>>>>>>>>>>
>>>>>>>>>> src1\SConstruct
>>>>>>>>>> ```
>>>>>>>>>> import os
>>>>>>>>>>
>>>>>>>>>> EnsureSConsVersion(3, 0, 0)
>>>>>>>>>>
>>>>>>>>>> # Setup the construction env
>>>>>>>>>> env = Environment(ENV={'PATH': os.environ['PATH']})
>>>>>>>>>>
>>>>>>>>>> # Set working directory to not change
>>>>>>>>>> env.SConscriptChdir(0)
>>>>>>>>>>
>>>>>>>>>> # Load in an external script
>>>>>>>>>> SConscript('../src2/testsubscript.py', {'env': env})
>>>>>>>>>> ```
>>>>>>>>>>
>>>>>>>>>> src2\SConstruct
>>>>>>>>>> ```
>>>>>>>>>> # Import the scons environment from the calling script
>>>>>>>>>> Import('env')
>>>>>>>>>>
>>>>>>>>>> def build_test(target, source, env):
>>>>>>>>>>     # At this point the 'testfile.txt' will be automatically
>>>>>>>>>> changed to a File class by SCons
>>>>>>>>>>
>>>>>>>>>>     # print the path scons thinks this file is located at
>>>>>>>>>>     print(target[0].path)
>>>>>>>>>>
>>>>>>>>>> tgt = env.Command(File('testfile.txt'), [], build_test)
>>>>>>>>>> env.Default(tgt)
>>>>>>>>>>
>>>>>>>>>> tgt2 = env.Command('testfile2.txt', [], build_test)
>>>>>>>>>> env.Default(tgt2)
>>>>>>>>>>
>>>>>>>>>> # Even though SConscriptChdir is set to 0 in the originating env,
>>>>>>>>>> File uses the current script as it's reference point
>>>>>>>>>> # D:\Temp\testproject\src2\testfile.txt
>>>>>>>>>> # D:\Temp\testproject\src2\testfile2.txt
>>>>>>>>>> ```
>>>>>>>>>>
>>>>>>>>>> Is this expected behaviour?
>>>>>>>>>> perhaps SConscriptChdir only influences when calling exe's rather
>>>>>>>>>> than the use of File or Dir
>>>>>>>>>> is there some public API way of setting the current directory for
>>>>>>>>>> File / Dir calls?
>>>>>>>>>> thats not the directory of the current SConsscript
>>>>>>>>>> since builders wrap everything in a File automatically, it might
>>>>>>>>>> mean I have to make sure all file paths are absolute before hand.
>>>>>>>>>>
>>>>>>>>>> Many Thanks
>>>>>>>>>> Richard
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Scons-users mailing list
>>>>>>>>>> Scons-users at scons.org
>>>>>>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>> Scons-users mailing list
>>>>> Scons-users at scons.org
>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>
>>>> _______________________________________________
>>> Scons-users mailing list
>>> Scons-users at scons.org
>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20180907/654d3c80/attachment-0001.html>