[Scons-users] Emitters vs. Scanners
Plunket, Tom
tom.plunket at aristocrat-inc.com
Tue Mar 15 14:29:17 EDT 2016
Digging into this issue in order to get a handle of what’s what, I see that there’s a “Current” scanner that seemed to do what I wanted.
I had been instantiating my scanner with SCons.Script.Scanner (which is initialized as SCons.Scanner.Base), but instantiating SCons.Scanner.Current instead seems to have fixed my problem.
Does this result provide any guidance given that the current default behavior is deemed to be incorrect? Rather than SCons.Script.Scanner creating a Scanner.Base maybe it should be instantiating Scanner.Current?
From: Scons-users [mailto:scons-users-bounces at scons.org] On Behalf Of Plunket, Tom
Sent: Monday, March 14, 2016 5:51 PM
To: SCons users mailing list <scons-users at scons.org>
Subject: Re: [Scons-users] Emitters vs. Scanners
Yes, that’s the issue I see with the asterisk in my case that it takes so long to generate the file that it’s always done scanning before being overwritten. As long as the original file that’s getting scanned doesn’t contain any errors the only problem is that it gets scanned twice and maybe some unnecessary targets get built but if the original file contains errors detected during scan it cannot be overwritten because the build halts before regenerating it.
Fingers crossed that I can figure out how to convert the provided SConstruct into a unit test of some sort.
From: Scons-users [mailto:scons-users-bounces at scons.org] On Behalf Of William Blevins
Sent: Monday, March 14, 2016 3:05 PM
To: SCons users mailing list <scons-users at scons.org<mailto:scons-users at scons.org>>
Subject: Re: [Scons-users] Emitters vs. Scanners
Tom,
Does this previously reported issue look related (or exact): http://scons.tigris.org/issues/show_bug.cgi?id=3022
On Mon, Mar 14, 2016 at 10:02 PM, Bill Deegan <bill at baddogconsulting.com<mailto:bill at baddogconsulting.com>> wrote:
I'd concur with what William said. If you're going to hack on this pull from the current default branch and work from there as there's been some non-trivial changes to how all this works in SCons which will be released with the next release.
(Waiting on a couple pull requests to be ready before we publish that)
I'm having a hard time understanding exactly what the issue is you're trying to solve.
If you can make a toy build which demonstrates it, it would be better to have a concrete example.
Thanks,
Bill
On Mon, Mar 14, 2016 at 5:26 PM, Plunket, Tom <tom.plunket at aristocrat-inc.com<mailto:tom.plunket at aristocrat-inc.com>> wrote:
Thank you. I’ll pull the trunk to see how it works in my environment and will start toying with tests.
From: Scons-users [mailto:scons-users-bounces at scons.org<mailto:scons-users-bounces at scons.org>] On Behalf Of William Blevins
Sent: Monday, March 14, 2016 2:10 PM
To: SCons users mailing list <scons-users at scons.org<mailto:scons-users at scons.org>>
Subject: Re: [Scons-users] Emitters vs. Scanners
I assume what you are asking for exists in src/engine/SCons/Node/__init__.py in the Node class per the HG Trunk (not yet released). In earlier versions of SCons, scanner execution happened in the executor directly: src/engine/SCons/Executor.py.
V/R,
William
On Mon, Mar 14, 2016 at 8:47 PM, Plunket, Tom <tom.plunket at aristocrat-inc.com<mailto:tom.plunket at aristocrat-inc.com>> wrote:
Ok, so since I care about this I can spend energy working on it but beyond browsing the sources I don’t have a great handle on where to start on this.
Is this an appropriate forum to ask where I should start poking, where I can put an appropriate test for (or rather against) this behavior, so that I can start the ball rolling on a fix? Should I join another forum for development issues and questions?
From: Scons-users [mailto:scons-users-bounces at scons.org<mailto:scons-users-bounces at scons.org>] On Behalf Of William Blevins
Sent: Monday, March 14, 2016 1:25 PM
To: SCons users mailing list <scons-users at scons.org<mailto:scons-users at scons.org>>
Subject: Re: [Scons-users] Emitters vs. Scanners
It is known, but it is not desired obviously.
On Mon, Mar 14, 2016 at 7:13 PM, Plunket, Tom <tom.plunket at aristocrat-inc.com<mailto:tom.plunket at aristocrat-inc.com>> wrote:
So it’s known (and desirable?) that scanners run on targets that are scheduled for replacement?
From: Scons-users [mailto:scons-users-bounces at scons.org<mailto:scons-users-bounces at scons.org>] On Behalf Of William Blevins
Sent: Friday, March 11, 2016 6:42 PM
To: SCons users mailing list <scons-users at scons.org<mailto:scons-users at scons.org>>
Subject: Re: [Scons-users] Emitters vs. Scanners
A TL;DR for scanners vs emitters. Emitters give explicit target nodes (for the action being done) and scanners find implicit dependency nodes (for the source being acted upon).
V/R,
William
On Sat, Mar 12, 2016 at 2:13 AM, Plunket, Tom <tom.plunket at aristocrat-inc.com<mailto:tom.plunket at aristocrat-inc.com>> wrote:
Ok cool, thanks for the example, I hadn’t thought of lex and yacc as similar to what I’m doing. I see that they both use emitters and don’t have scanners so maybe my understanding of emitter vs. scanner is … nonexistent. ☺
I see the CScanner is put in as a source_scanner. The mention of this option in the documentation is sparse, I had previously put my scanner into the environment with the file extension but it wasn’t attached to a specific builder. Now that I have it attached to the builder I’d hoped it’d fix things but… it doesn’t. ☹
My Scene generate function is step A->B:
myAction = Action(sceneExport, 'Exporting ${SOURCES.file} to ${TARGETS.file}.')
builder = Builder(action = myAction)
env.Append(BUILDERS = { 'Scene': builder })
My Asset generate (step B->C) function sets it up like this (LAUNCH is a command line hardcoded elsewhere):
message = "Packaging ${TARGETS.file}"
myAction = Action(LAUNCH, message)
assetScanner = Scanner(function=assetScan, skeys=['.xml'])
builder = Builder(action = myAction, source_scanner=assetScanner)
env.Append(BUILDERS = { 'Asset': builder })
Print statements in the scanner (“Scanning blahblah.xml”) print before the Scene command string (“Exporting blahblah.aep to blahblah.xml”).
I’ll keep banging on this though and if I can get it going I’ll let everyone know what I did wrong. If I can’t I’ll try to strip this down (that is unfortunately more work than it should be at this point). The SCons source is still a little opaque to me as it seems like components of this process are spread around various folders somewhat wantonly still. ☺ (That’s not a criticism of anything beyond my own understanding!)
-tom!
From: Scons-users [mailto:scons-users-bounces at scons.org<mailto:scons-users-bounces at scons.org>] On Behalf Of Bill Deegan
Sent: Friday, March 11, 2016 3:22 PM
To: SCons users mailing list <scons-users at scons.org<mailto:scons-users at scons.org>>
Subject: Re: [Scons-users] Emitters vs. Scanners
Tom,
Likely DynamicSourceGenerator is overkill for u.
Can you create a small example to demonstrate your problem?
If a->b is entirely predictable, then is there a:
env.MyBuilder('b','a')
env.otherBuilder('c','b')
If SCons doesn't know that b is generated from a, then I can see the issue you're running into.
lex/yacc do similar and have no issues..
-Bill
On Fri, Mar 11, 2016 at 5:07 PM, Plunket, Tom <tom.plunket at aristocrat-inc.com<mailto:tom.plunket at aristocrat-inc.com>> wrote:
I am struggling with an issue where an intermediate file is getting scanned for targets even though it is about to get regenerated. I’m curious if I need to change my Scanner into an Emitter or if there’s something else I should be considering?
The flow is A->B->C, A is the input file which creates B. B is the primary source for C but (B) also lists a bunch of other files that need to go into C. filename.a gets processed into filename.b and that, along with the results of the scan get bundled up into filename.c. The closest software construction analogy would be that A is like a “premake” or “makefile.in<http://makefile.in>” file, B would be the makefile and would list the source files to be compiled, and C is the executable that results.
If B exists, it gets scanned during dependency generation by the scanner, even if A has changed. This is a problem if B refers to invalid or otherwise broken dependencies that have been fixed by the changes to A, since I will not be able to build nor clean these items without manually deleting B.
I’m looking at https://bitbucket.org/scons/scons/wiki/DynamicSourceGenerator but I’m not yet sure that it’ll fix my problem (or maybe it’s overkill) since B exists, it just needs to get recreated before getting scanned. (Could/should it be considered a deficiency to scan a file that’s scheduled for replacement in the first place?)
Thanks and sorry for the corporate junk that the email gateway sticks on the end of this message.
IMPORTANT CONFIDENTIALITY NOTICE:
This E-mail(including any documents referred to in, or attached, to this E-mail) may contain information that is personal, confidential or the subject of copyright or other proprietary rights in favor of Aristocrat, its affiliates or third parties. This E-mail is intended only for the named addressee. Any privacy, confidence, copyright or other proprietary rights in favor of Aristocrat, its affiliates or third parties, is not lost because this E-mail was sent to you by mistake.
If you received this E-mail by mistake you should: (i) not copy, disclose, distribute or otherwise use it, or its contents, without the consent of Aristocrat or the owner of the relevant rights; (ii) let us know of the mistake by reply E-mail or by telephone (US 1-877-274-9661<tel:1-877-274-9661>, or AU +61 2 9013 6000<tel:%2B61%202%209013%206000>); and (iii) delete it from your system and destroy all copies.
Any personal information contained in this E-mail must be handled in accordance with applicable privacy laws.
Electronic and internet communications can be interfered with or affected by viruses and other defects. As a result, such communications may not be successfully received or, if received, may cause interference with the integrity of receiving, processing or related systems (including hardware, software and data or information on, or using, that hardware or software). Aristocrat gives no assurances in relation to these matters. If you have any doubts about the veracity or integrity of any electronic communication we appear to have sent you, please call (US 1-877-274-9661<tel:1-877-274-9661>, or AU +61 2 9013 6000<tel:%2B61%202%209013%206000>) for clarification.
_______________________________________________
Scons-users mailing list
Scons-users at scons.org<mailto:Scons-users at scons.org>
https://pairlist4.pair.net/mailman/listinfo/scons-users
IMPORTANT CONFIDENTIALITY NOTICE:
This E-mail(including any documents referred to in, or attached, to this E-mail) may contain information that is personal, confidential or the subject of copyright or other proprietary rights in favor of Aristocrat, its affiliates or third parties. This E-mail is intended only for the named addressee. Any privacy, confidence, copyright or other proprietary rights in favor of Aristocrat, its affiliates or third parties, is not lost because this E-mail was sent to you by mistake.
If you received this E-mail by mistake you should: (i) not copy, disclose, distribute or otherwise use it, or its contents, without the consent of Aristocrat or the owner of the relevant rights; (ii) let us know of the mistake by reply E-mail or by telephone (US 1-877-274-9661<tel:1-877-274-9661>, or AU +61 2 9013 6000<tel:%2B61%202%209013%206000>); and (iii) delete it from your system and destroy all copies.
Any personal information contained in this E-mail must be handled in accordance with applicable privacy laws.
Electronic and internet communications can be interfered with or affected by viruses and other defects. As a result, such communications may not be successfully received or, if received, may cause interference with the integrity of receiving, processing or related systems (including hardware, software and data or information on, or using, that hardware or software). Aristocrat gives no assurances in relation to these matters. If you have any doubts about the veracity or integrity of any electronic communication we appear to have sent you, please call (US 1-877-274-9661<tel:1-877-274-9661>, or AU +61 2 9013 6000<tel:%2B61%202%209013%206000>) for clarification.
_______________________________________________
Scons-users mailing list
Scons-users at scons.org<mailto:Scons-users at scons.org>
https://pairlist4.pair.net/mailman/listinfo/scons-users
IMPORTANT CONFIDENTIALITY NOTICE:
This E-mail(including any documents referred to in, or attached, to this E-mail) may contain information that is personal, confidential or the subject of copyright or other proprietary rights in favor of Aristocrat, its affiliates or third parties. This E-mail is intended only for the named addressee. Any privacy, confidence, copyright or other proprietary rights in favor of Aristocrat, its affiliates or third parties, is not lost because this E-mail was sent to you by mistake.
If you received this E-mail by mistake you should: (i) not copy, disclose, distribute or otherwise use it, or its contents, without the consent of Aristocrat or the owner of the relevant rights; (ii) let us know of the mistake by reply E-mail or by telephone (US 1-877-274-9661<tel:1-877-274-9661>, or AU +61 2 9013 6000<tel:%2B61%202%209013%206000>); and (iii) delete it from your system and destroy all copies.
Any personal information contained in this E-mail must be handled in accordance with applicable privacy laws.
Electronic and internet communications can be interfered with or affected by viruses and other defects. As a result, such communications may not be successfully received or, if received, may cause interference with the integrity of receiving, processing or related systems (including hardware, software and data or information on, or using, that hardware or software). Aristocrat gives no assurances in relation to these matters. If you have any doubts about the veracity or integrity of any electronic communication we appear to have sent you, please call (US 1-877-274-9661<tel:1-877-274-9661>, or AU +61 2 9013 6000<tel:%2B61%202%209013%206000>) for clarification.
_______________________________________________
Scons-users mailing list
Scons-users at scons.org<mailto:Scons-users at scons.org>
https://pairlist4.pair.net/mailman/listinfo/scons-users
IMPORTANT CONFIDENTIALITY NOTICE:
This E-mail(including any documents referred to in, or attached, to this E-mail) may contain information that is personal, confidential or the subject of copyright or other proprietary rights in favor of Aristocrat, its affiliates or third parties. This E-mail is intended only for the named addressee. Any privacy, confidence, copyright or other proprietary rights in favor of Aristocrat, its affiliates or third parties, is not lost because this E-mail was sent to you by mistake.
If you received this E-mail by mistake you should: (i) not copy, disclose, distribute or otherwise use it, or its contents, without the consent of Aristocrat or the owner of the relevant rights; (ii) let us know of the mistake by reply E-mail or by telephone (US 1-877-274-9661<tel:1-877-274-9661>, or AU +61 2 9013 6000<tel:%2B61%202%209013%206000>); and (iii) delete it from your system and destroy all copies.
Any personal information contained in this E-mail must be handled in accordance with applicable privacy laws.
Electronic and internet communications can be interfered with or affected by viruses and other defects. As a result, such communications may not be successfully received or, if received, may cause interference with the integrity of receiving, processing or related systems (including hardware, software and data or information on, or using, that hardware or software). Aristocrat gives no assurances in relation to these matters. If you have any doubts about the veracity or integrity of any electronic communication we appear to have sent you, please call (US 1-877-274-9661<tel:1-877-274-9661>, or AU +61 2 9013 6000<tel:%2B61%202%209013%206000>) for clarification.
_______________________________________________
Scons-users mailing list
Scons-users at scons.org<mailto:Scons-users at scons.org>
https://pairlist4.pair.net/mailman/listinfo/scons-users
_______________________________________________
Scons-users mailing list
Scons-users at scons.org<mailto:Scons-users at scons.org>
https://pairlist4.pair.net/mailman/listinfo/scons-users
IMPORTANT CONFIDENTIALITY NOTICE:
This E-mail(including any documents referred to in, or attached, to this E-mail) may contain information that is personal, confidential or the subject of copyright or other proprietary rights in favor of Aristocrat, its affiliates or third parties. This E-mail is intended only for the named addressee. Any privacy, confidence, copyright or other proprietary rights in favor of Aristocrat, its affiliates or third parties, is not lost because this E-mail was sent to you by mistake.
If you received this E-mail by mistake you should: (i) not copy, disclose, distribute or otherwise use it, or its contents, without the consent of Aristocrat or the owner of the relevant rights; (ii) let us know of the mistake by reply E-mail or by telephone (US 1-877-274-9661, or AU +61 2 9013 6000); and (iii) delete it from your system and destroy all copies.
Any personal information contained in this E-mail must be handled in accordance with applicable privacy laws.
Electronic and internet communications can be interfered with or affected by viruses and other defects. As a result, such communications may not be successfully received or, if received, may cause interference with the integrity of receiving, processing or related systems (including hardware, software and data or information on, or using, that hardware or software). Aristocrat gives no assurances in relation to these matters. If you have any doubts about the veracity or integrity of any electronic communication we appear to have sent you, please call (US 1-877-274-9661, or AU +61 2 9013 6000) for clarification.
IMPORTANT CONFIDENTIALITY NOTICE:
This E-mail(including any documents referred to in, or attached, to this E-mail) may contain information that is personal, confidential or the subject of copyright or other proprietary rights in favor of Aristocrat, its affiliates or third parties. This E-mail is intended only for the named addressee. Any privacy, confidence, copyright or other proprietary rights in favor of Aristocrat, its affiliates or third parties, is not lost because this E-mail was sent to you by mistake.
If you received this E-mail by mistake you should: (i) not copy, disclose, distribute or otherwise use it, or its contents, without the consent of Aristocrat or the owner of the relevant rights; (ii) let us know of the mistake by reply E-mail or by telephone (US 1-877-274-9661, or AU +61 2 9013 6000); and (iii) delete it from your system and destroy all copies.
Any personal information contained in this E-mail must be handled in accordance with applicable privacy laws.
Electronic and internet communications can be interfered with or affected by viruses and other defects. As a result, such communications may not be successfully received or, if received, may cause interference with the integrity of receiving, processing or related systems (including hardware, software and data or information on, or using, that hardware or software). Aristocrat gives no assurances in relation to these matters.
If you have any doubts about the veracity or integrity of any electronic communication we appear to have sent you, please call (US 1-877-274-9661, or AU +61 2 9013 6000) for clarification.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20160315/4c2b08df/attachment-0001.html>
More information about the Scons-users
mailing list