[Scons-users] SCons misses a dependency which results in incorrect output.

Tal Dayan tal at zapta.com
Wed Jul 16 20:14:40 EDT 2025 

Our system is based on Python plugin classes that provide scons Builders so
I wonder if we can just define our own subclass of Builder that
automatically adds the deletion on errors functionality.

https://github.com/FPGAwars/apio/blob/008b090e92246414353776ef5df4c0dd3e274fab/apio/scons/plugin_ice40.py#L62C16-L62C23

On Wed, Jul 16, 2025 at 5:02 PM Bill Deegan <bill at baddogconsulting.com>
wrote:

> Tai,
>
> If you have a program you're running as the action in a command which is
> erroring out but writing the file, you can work around this issue for the
> time being by wrapping that program with a script which checks the exit
> value of your program, and removing the target files if it exits with an
> error status.
>
> That should be sufficient to get you going again.
>
> -Bill
>
> On Wed, Jul 16, 2025 at 3:18 PM Bill Deegan <bill at baddogconsulting.com>
> wrote:
>
>> Did you read my explanation and how to fix it in your environment
>> assuming command.sh is similar to what's really happening in your build?
>>
>>
>> On Wed, Jul 16, 2025 at 3:16 PM Bill Deegan <bill at baddogconsulting.com>
>> wrote:
>>
>>> It's an enhancement request.
>>>
>>> The bug is in your command.sh script.
>>> I'll change the description and text.
>>>
>>>
>>>
>>> On Wed, Jul 16, 2025 at 12:56 PM Tal Dayan <tal at zapta.com> wrote:
>>>
>>>> Hi Bill, I will file an issue.
>>>>
>>>> I don't think it's safe for SCons to assume that once a program writes
>>>> an output file it can't crash.
>>>>
>>>> On Wed, Jul 16, 2025 at 12:45 PM Bill Deegan <bill at baddogconsulting.com>
>>>> wrote:
>>>>
>>>>> Arguably your command.sh is incorrectly written.
>>>>>
>>>>>  It copies the file and then checks it's contents and exits with error
>>>>> status if there improper.
>>>>>
>>>>> So a quick fix would be to check the input before copying the file.
>>>>>
>>>>> From the URL you posted in other email here's the command.sh
>>>>> #!/bin/bash
>>>>> # Usage: ./command input > output
>>>>>
>>>>> # Copy source to destination
>>>>> cp "$1" "$2"
>>>>>
>>>>> # If the input file stats with 'bad' inject an error AFTER creating
>>>>> the output file.
>>>>>
>>>>> first_line=$(head -n 1 "$1")
>>>>>
>>>>> if [[ "$first_line" == bad* ]]; then
>>>>>   exit 1
>>>>> fi
>>>>>
>>>>> exit 0
>>>>>
>>>>> Change it to this:
>>>>> #!/bin/bash
>>>>> # Usage: ./command input > output
>>>>>
>>>>> # If the input file stats with 'bad' inject an error AFTER creating
>>>>> the output file.
>>>>>
>>>>> first_line=$(head -n 1 "$1")
>>>>>
>>>>> if [[ "$first_line" == bad* ]]; then
>>>>>   exit 1
>>>>> fi
>>>>>
>>>>> # Copy source to destination
>>>>> cp "$1" "$2"
>>>>>
>>>>> exit 0
>>>>>
>>>>> Results:
>>>>>
>>>>> % ./run.sh
>>>>>
>>>>> ----- Iteration 1: file1 = 'good' -----
>>>>>
>>>>> scons: Reading SConscript files ...
>>>>> scons: done reading SConscript files.
>>>>> scons: Building targets ...
>>>>> scons: building `file2' because it doesn't exist
>>>>> ./command.sh file1 file2
>>>>> +-file2
>>>>>   +-file1
>>>>> scons: done building targets.
>>>>>
>>>>> File1
>>>>> MD5 (file1) = d7f986677d9f563bd1794b09d82206a3
>>>>>      1 good
>>>>>
>>>>> File2
>>>>> MD5 (file2) = d7f986677d9f563bd1794b09d82206a3
>>>>>      1 good
>>>>>
>>>>> DBlite:
>>>>> === .:
>>>>> file1: d7f986677d9f563bd1794b09d82206a3 1752694898 5
>>>>> file2: d7f986677d9f563bd1794b09d82206a3 1752694898 5
>>>>>         file1: d7f986677d9f563bd1794b09d82206a3 1752694898 5
>>>>>         2dbc2dce125a753309a27b7d5157aaaa [./command.sh $SOURCE $TARGET]
>>>>>
>>>>> ----- Iteration 2: file1 = 'bad' -----
>>>>>
>>>>> scons: Reading SConscript files ...
>>>>> scons: done reading SConscript files.
>>>>> scons: Building targets ...
>>>>> scons: rebuilding `file2' because `file1' changed
>>>>> ./command.sh file1 file2
>>>>> scons: *** [file2] Error 1
>>>>> +-file2
>>>>>   +-file1
>>>>> scons: building terminated because of errors.
>>>>>
>>>>> File1
>>>>> MD5 (file1) = df207dc9143c6fabf60b69b9c3035103
>>>>>      1 bad
>>>>>
>>>>> File2
>>>>> md5: file2: No such file or directory
>>>>> cat: file2: No such file or directory
>>>>>
>>>>> DBlite:
>>>>> === .:
>>>>> file1: df207dc9143c6fabf60b69b9c3035103 1752694898 4
>>>>> file2: d7f986677d9f563bd1794b09d82206a3 1752694898 5
>>>>>         file1: d7f986677d9f563bd1794b09d82206a3 1752694898 5
>>>>>         2dbc2dce125a753309a27b7d5157aaaa [./command.sh $SOURCE $TARGET]
>>>>>
>>>>> ----- Iteration 3: file1 = 'good' -----
>>>>>
>>>>> scons: Reading SConscript files ...
>>>>> scons: done reading SConscript files.
>>>>> scons: Building targets ...
>>>>> scons: building `file2' because it doesn't exist
>>>>> ./command.sh file1 file2
>>>>> +-file2
>>>>>   +-file1
>>>>> scons: done building targets.
>>>>>
>>>>> File1
>>>>> MD5 (file1) = d7f986677d9f563bd1794b09d82206a3
>>>>>      1 good
>>>>>
>>>>> File2
>>>>> MD5 (file2) = d7f986677d9f563bd1794b09d82206a3
>>>>>      1 good
>>>>>
>>>>> DBlite:
>>>>> === .:
>>>>> file1: d7f986677d9f563bd1794b09d82206a3 1752694899 5
>>>>> file2: d7f986677d9f563bd1794b09d82206a3 1752694899 5
>>>>>         file1: d7f986677d9f563bd1794b09d82206a3 1752694899 5
>>>>>         2dbc2dce125a753309a27b7d5157aaaa [./command.sh $SOURCE $TARGET]
>>>>>
>>>>> There's not presently logic in SCons to delete target files if the
>>>>> associated action yields an error.
>>>>>
>>>>> I think what you want is equivalent to makes .DELETE_ON_FAILURE,
>>>>> there's actually a SO question on this:
>>>>>
>>>>> https://stackoverflow.com/questions/29546276/scons-delete-target-on-failure-of-any-action
>>>>>
>>>>> Please go ahead and file an enhancement request to add equivalent
>>>>> to DELETE_ON_FAILURE, please include your reproducer scripts.
>>>>>
>>>>> -Bill
>>>>>
>>>>> On Wed, Jul 16, 2025 at 10:44 AM Tal Dayan <tal at zapta.com> wrote:
>>>>>
>>>>>> Looking at the end state of scons after invocation #3, the actual md5
>>>>>> of file2 doesn't match its md5 in the dblite.
>>>>>>
>>>>>> https://i.imgur.com/NGco3yQ.png
>>>>>>
>>>>>> On Wed, Jul 16, 2025 at 10:33 AM Tal Dayan <tal at zapta.com> wrote:
>>>>>>
>>>>>>> Hi Keith, I updated the example files here
>>>>>>> https://github.com/FPGAwars/apio/issues/676
>>>>>>>
>>>>>>> They now include the md5 of the files and a dump of .sconsign.dblite
>>>>>>>
>>>>>>> On Wed, Jul 16, 2025 at 10:08 AM Keith Prussing <
>>>>>>> kprussing74 at gmail.com> wrote:
>>>>>>>
>>>>>>>> I suspect it's because `file1` has the same hash in the
>>>>>>>> .sconsign.dblite as the last "good" build (i.e. the first one). Thus
>>>>>>>> you get the line "scons: `file2' is up to date." on the third run.
>>>>>>>> However, I am not an expert in the specifics of SCons' hashing
>>>>>>>> methods.
>>>>>>>>
>>>>>>>> On Wed, Jul 16, 2025 at 12:38 PM Tal Dayan <tal at zapta.com> wrote:
>>>>>>>> >
>>>>>>>> > Hi all,
>>>>>>>> >
>>>>>>>> > We encountered this problem with the nextpnr tool and created
>>>>>>>> here a small and independent example that demonstrates it.
>>>>>>>> >
>>>>>>>> > In the example below, a shell script 'command.sh' reads the
>>>>>>>> source file 'file1' and writes it to the target file 'file2'. However, if
>>>>>>>> the
>>>>>>>> > source file starts with 'bad' it exits with an error code,
>>>>>>>> *after* creating the target file.
>>>>>>>> >
>>>>>>>> > The script `run.sh', runs scons three times with these values of
>>>>>>>> the source file file1 'good', 'bad', and 'good'.  The expectation is that
>>>>>>>> after the third scons run, file2 should contain the value 'good' but it
>>>>>>>> contains the value 'bad'.
>>>>>>>> >
>>>>>>>> > Do we miss anything or is it simply a bug?
>>>>>>>> >
>>>>>>>> > SConstruct:
>>>>>>>> >
>>>>>>>> > ----------------------------------------------
>>>>>>>> >
>>>>>>>> > # SCons environment
>>>>>>>> >
>>>>>>>> > env = Environment()
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > # Copy file1 → file2 using command.sh
>>>>>>>> >
>>>>>>>> > # Inject an error if file1 starts with 'bad"
>>>>>>>> >
>>>>>>>> > file2 = env.Command(
>>>>>>>> >
>>>>>>>> >     target='file2',
>>>>>>>> >
>>>>>>>> >     source='file1',
>>>>>>>> >
>>>>>>>> >     action='./command.sh $SOURCE > $TARGET'
>>>>>>>> >
>>>>>>>> > )
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > # Make 'file2' the default target
>>>>>>>> >
>>>>>>>> > Default(file2)
>>>>>>>> >
>>>>>>>> > ----------------------------------------------
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > command.sh:
>>>>>>>> >
>>>>>>>> > ----------------------------------------------
>>>>>>>> >
>>>>>>>> > #!/bin/bash
>>>>>>>> >
>>>>>>>> > # Usage: ./command input > output
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > # Read from the first argument and copy to stdout
>>>>>>>> >
>>>>>>>> > cat "$1"
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > # If the input file starts with 'bad', inject an error AFTER
>>>>>>>> creating the output file.
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > first_line=$(head -n 1 "$1")
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > if [[ "$first_line" == bad* ]]; then
>>>>>>>> >
>>>>>>>> >   exit 1
>>>>>>>> >
>>>>>>>> > fi
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > exit 0
>>>>>>>> >
>>>>>>>> > ----------------------------------------------
>>>>>>>> >
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > run.sh
>>>>>>>> >
>>>>>>>> > ----------------------------------------------
>>>>>>>> >
>>>>>>>> > #!/bin/bash
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > # Clean up.
>>>>>>>> >
>>>>>>>> > rm -f .sconsign.dblite
>>>>>>>> >
>>>>>>>> > rm -f file[12]
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > echo
>>>>>>>> >
>>>>>>>> > echo "---- Iteration 1: file1 = 'good'"
>>>>>>>> >
>>>>>>>> > echo "good" > file1
>>>>>>>> >
>>>>>>>> > scons
>>>>>>>> >
>>>>>>>> > echo
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > echo "File1"
>>>>>>>> >
>>>>>>>> > cat -n file1
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > echo "File2"
>>>>>>>> >
>>>>>>>> > cat -n file2
>>>>>>>> >
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > echo
>>>>>>>> >
>>>>>>>> > echo "---- Iteration 2: file1 = 'bad'"
>>>>>>>> >
>>>>>>>> > echo "bad" > file1
>>>>>>>> >
>>>>>>>> > cat -n file1
>>>>>>>> >
>>>>>>>> > scons
>>>>>>>> >
>>>>>>>> > echo
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > echo "File1"
>>>>>>>> >
>>>>>>>> > cat -n file1
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > echo "File2"
>>>>>>>> >
>>>>>>>> > cat -n file2
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > echo
>>>>>>>> >
>>>>>>>> > echo "---- Iteration 3: file1 = 'good'"
>>>>>>>> >
>>>>>>>> > echo "good" > file1
>>>>>>>> >
>>>>>>>> > cat -n file1
>>>>>>>> >
>>>>>>>> > scons
>>>>>>>> >
>>>>>>>> > echo
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > echo "File1"
>>>>>>>> >
>>>>>>>> > cat -n file1
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > echo "File2"
>>>>>>>> >
>>>>>>>> > cat -n file2
>>>>>>>> >
>>>>>>>> > ----------------------------------------------
>>>>>>>> >
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > Run log:
>>>>>>>> >
>>>>>>>> > ----------------------------------------------
>>>>>>>> >
>>>>>>>> > $ ./run.sh
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > ---- Iteration 1: file1 = 'good'
>>>>>>>> >
>>>>>>>> > scons: Reading SConscript files ...
>>>>>>>> >
>>>>>>>> > scons: done reading SConscript files.
>>>>>>>> >
>>>>>>>> > scons: Building targets ...
>>>>>>>> >
>>>>>>>> > ./command.sh file1 > file2
>>>>>>>> >
>>>>>>>> > scons: done building targets.
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > File1
>>>>>>>> >
>>>>>>>> >      1 good
>>>>>>>> >
>>>>>>>> > File2
>>>>>>>> >
>>>>>>>> >      1 good
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > ---- Iteration 2: file1 = 'bad'
>>>>>>>> >
>>>>>>>> >      1 bad
>>>>>>>> >
>>>>>>>> > scons: Reading SConscript files ...
>>>>>>>> >
>>>>>>>> > scons: done reading SConscript files.
>>>>>>>> >
>>>>>>>> > scons: Building targets ...
>>>>>>>> >
>>>>>>>> > ./command.sh file1 > file2
>>>>>>>> >
>>>>>>>> > scons: *** [file2] Error 1
>>>>>>>> >
>>>>>>>> > scons: building terminated because of errors.
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > File1
>>>>>>>> >
>>>>>>>> >      1 bad
>>>>>>>> >
>>>>>>>> > File2
>>>>>>>> >
>>>>>>>> >      1 bad
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > ---- Iteration 1: file1 = 'good'
>>>>>>>> >
>>>>>>>> >      1 good
>>>>>>>> >
>>>>>>>> > scons: Reading SConscript files ...
>>>>>>>> >
>>>>>>>> > scons: done reading SConscript files.
>>>>>>>> >
>>>>>>>> > scons: Building targets ...
>>>>>>>> >
>>>>>>>> > scons: `file2' is up to date.
>>>>>>>> >
>>>>>>>> > scons: done building targets.
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > File1
>>>>>>>> >
>>>>>>>> >      1 good
>>>>>>>> >
>>>>>>>> > File2
>>>>>>>> >
>>>>>>>> >      1 bad
>>>>>>>> >
>>>>>>>> > ----------------------------------------------
>>>>>>>> >
>>>>>>>> > _______________________________________________
>>>>>>>> > Scons-users mailing list
>>>>>>>> > Scons-users at scons.org
>>>>>>>> > https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Keith Prussing
>>>>>>>> _______________________________________________
>>>>>>>> Scons-users mailing list
>>>>>>>> Scons-users at scons.org
>>>>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>>>>
>>>>>>> _______________________________________________
>>>>>> Scons-users mailing list
>>>>>> Scons-users at scons.org
>>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>>
>>>>> _______________________________________________
>>>>> Scons-users mailing list
>>>>> Scons-users at scons.org
>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>
>>>> _______________________________________________
>>>> Scons-users mailing list
>>>> Scons-users at scons.org
>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>
>>> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20250716/fb3c621f/attachment-0001.htm>

More information about the Scons-users mailing list