[Scons-users] env[X'] and signatures

Thu Dec 29 18:07:08 EST 2016

Right now its pretty much all python functions instead of separate scripts.
I see ehat you mean with the scripts though... However there can sometimes
be many many many information passed throgh the builder, which passing it
throgh command line makes it a bit messy...

El dv, 30 des 2016 a les 0:02 Bill Deegan <bill at baddogconsulting.com> va
escriure:

> Albert,
>
> Is there a command line for the tagging target?
> Or this a python builder inside your SCons implementation?
>
> Generally I suggest not having complex python builders in your
> implementation, but rather have scripts called by SCons.
> This would simplify the case where your just changing a version as you'd
> make the version be part of the command line to the script.
> Then it would only rerun when the version changed..
>
> -Bill
>
> On Thu, Dec 29, 2016 at 2:51 PM, Albert Arquer <albert.arquer at gmail.com>
> wrote:
>
> Well it's pretty complex as I am sure you can imagine, you have all sorts
> of files. Verilos systemverilog, vhdl, constraints, headers, vendor files
> like xci, qip, etc.
>
> However lets say I have an fpga build flow split in 4 phases.
>   1. Synthesis
>   2. Place & Route
>   3. Tagging. Gets the bitstream from place and route and modifies ONLY
> the contents of a block ram, adding version, timestamp, build information
> strings, etc. This is to go from an unofficial development build (untested)
> to an officially tagged and tested release.
>   4. Configuration image generarion. Get one (or multiple) bitstreams
> along with some hex files for example, some parameters, and generate an SPI
> configuration file to load onto the FPGA.
>
> Now, each of these targets depends on the previous and some extra stuff
> (like hex files and parameters), the synthesis target depends on the hdl
> sources intself. Both synthesis and place and route can add some extra
> sources, constraints for instance.
>
> Lets say I have already built the whole process, so I have the output
> files and they are up-to-date. Now lets say I just want to go to the
> tagging target and modify some tagging information, like the version. I go
> and open the sconscript which defines that target and modify that
> parameter. However that file also defines all other targets, so if my
> targets depend on that sconscript they are all out-dated now!
>
> However the only target that has trully changed is the tagging target (and
> therefore the next and last stage too), what I miss about scons is a way of
> each builder detecting if it's environment has changed since the last build.
>
> What I have done to fix this is have scons hash the whole environment for
> each target and add that hash to the builders signature. This allows me to
> remove the sconscripts from the target's dependencies, and still have the
> target rebuilt when a prameter is changed. At least In our case changes in
> a target's environment are much much more unlikely than the scenario I just
> described, so this aproach means a huge improvement for us.
>
> As I have mentioned in the previous message I would like to be able to use
> dill instead off pickle to serialise.
>
> Albert.
>
> El dj, 29 des 2016 a les 23:32 Bill Deegan <bill at baddogconsulting.com> va
> escriure:
>
> Albert,
>
> I'm very familiar with the EDA space.
> What's the input file?
> What environmnent variables changing would trigger a rebuild for you?
>
> -Bill
>
> On Thu, Dec 29, 2016 at 9:58 AM, Albert Arquer <albert.arquer at gmail.com>
> wrote:
>
> I agree it is not the ideal solution. In fact we use scons for hdl
> synthesis/implementation which can take many hours.
>
> Before we used to make the target depend on the sconscript which declared
> the target. However this caused many innecessary rebuilds, as a single
> sconscript often declares multiple targets, and even if a change in the
> sconscript which only afected a single target was modified, scons would
> rebuild all other targets declared in the sconscript and any others which
> depended on them.
>
> By hashing the environment we can isolate sconscript modifications to the
> targets it actually affects. This is not ideal ofcourse, it would be much
> better for each builder to define somehow which items from the environment
> it will be using. However this problem usually causes build to be repeated,
> not missed, and still counting those cases the innecessary builts are
> drastically reduced (at least in our case).
>
>  There is also the issue on the hashing itself, since not everything is
> hashable in python, and even if an object is hashable that hash might not
> change if the object's class definition has changed for example. I opened
> this thread in stackoverflow a while back trying to find a serialization
> module suited for this purpose, turns out dill would be a good choice but
> it has compatibility issues with scons.
>
>
> http://stackoverflow.com/questions/40640797/compute-class-and-instance-hash/40775525#40775525
>
>
> El dj, 29 des 2016 a les 18:51 Bill Deegan <bill at baddogconsulting.com> va
> escriure:
>
> As a workaround, perhaps add an action string which does nothing, but uses
> the variables you depend on?
> Just a thought.
>
>
> Are the builders external python scripts which get run from the shell by
> SCons? or are the builders python logic run inside scons?
>
> On Thu, Dec 29, 2016 at 8:54 AM, Tom Tanner <trtanner at btinternet.com>
> wrote:
>
> On 27/12/16 18:22, Bill Deegan wrote:
>
>
>
>
> If every builder depends on all values in env['ENV'], then any change will
> rebuild everything.
>
>
> This is almost never warranted.
>
>
>
>
>
>
>
> But then why does scons clear out the supplied environment? It is
> precisely because everything behaves in a way that is unclear and not
> reproducable if you don't know exactly what the environment is.
>
>
>
>
> I think, ideally, a builder (Action really), would specify what parts of
> the Environment() it's sensitive too.
>
>
>
>
>
>
>
> This would be better. If every builder specified which parts of env['ENV']
> it is interested in, and they (and nothing else) got passed through and
> added to the build signature
>
>
>
>
>
>
>
>
>
>
> Remember, while many users are only building source code which is fairly
> quick, SCons is used in other "build" systems where steps can take
> considerable time and resources so extraneous rebuilds are quite costly for
> them.
>
>
>
>
>
> -Bill
>
>
>
>
>
>
>
> I agree but specifying the correct dependencies are how you avoid costly
> rebuilds. Not rebuilding something that has changed can be just as costly
> because instead of finding out about it at the build stage, you find out
> about it when it is on a production machine.
>
>
>
> _______________________________________________
>
>
> Scons-users mailing list
>
>
> Scons-users at scons.org
>
>
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
>
>
>
>
> _______________________________________________
>
> Scons-users mailing list
>
> Scons-users at scons.org
>
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
>
>
>
> _______________________________________________
>
>
> Scons-users mailing list
>
>
> Scons-users at scons.org
>
>
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
>
>
>
>
>
> _______________________________________________
>
> Scons-users mailing list
>
> Scons-users at scons.org
>
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
>
>
>
> _______________________________________________
>
>
> Scons-users mailing list
>
>
> Scons-users at scons.org
>
>
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
>
>
>
>
>
> _______________________________________________
>
> Scons-users mailing list
>
> Scons-users at scons.org
>
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20161229/07544de6/attachment-0001.html>