Clone wiki

SCons / BugParty / IrcLog2008-11-19

17:24:12 * garyo-home (n=chatzill@ has joined #scons 17:24:42 <garyo-home> Hi Greg. 17:27:55 * stevenknight (n=stevenkn@nat/google/x-93de785fa2957b48) has joined #scons 17:28:18 <garyo-home> Hi, Steven. 17:28:24 <stevenknight> hey garyo 17:28:38 * Greg_Noel just got here 17:28:46 <Greg_Noel> Hi, all... 17:28:51 <stevenknight> hi greg 17:29:20 <Greg_Noel> Give me a chance to set up... 17:29:26 <garyo-home> Hi Greg. 17:34:58 <Greg_Noel> I'm ready; shall we proceed? 17:35:32 <garyo-home> yes, let's start. 17:35:39 <stevenknight> all right then... 17:35:53 <garyo-home> 1895 consensus 17:36:02 <garyo-home> 2127 consensus 17:36:06 <Greg_Noel> 1895 and 2127 look like consensus 17:36:37 <garyo-home> 2248: I say maybe invalid due to vs_revamp. 17:36:46 <Greg_Noel> I'll go with research, David 17:36:50 <stevenknight> yes, vs_revamp should take care of it 17:37:00 <stevenknight> probably nicer to the user to close as FIXED by vs_revamp, though 17:37:09 <garyo-home> q: when should we integrate vs_revamp? (Steven: yes, good idea) 17:37:51 <stevenknight> how about right after 1.2? 17:37:56 <Greg_Noel> Can't close it until there's a fix in place; bad practice 17:37:57 <stevenknight> (which we should discuss) 17:38:12 <stevenknight> greg: right, sorry I meant after vs_revamp is integrated 17:38:24 <garyo-home> steven: sounds good. 17:38:41 <stevenknight> okay, so mark this 1.3 p3 keyword 'vs_revamp' ? 17:38:48 <garyo-home> ok\ 17:38:50 <Greg_Noel> ok, 1.3, p?, who? 17:39:13 <Greg_Noel> p3 is ok 17:39:16 <garyo-home> I'll do it, it's only closing the bug when vs_revamp is in. 17:39:20 <Greg_Noel> done 17:39:57 <garyo-home> 2250? 17:40:25 <stevenknight> probably end up with me 17:40:32 <Greg_Noel> Wide range of opinion... 17:40:41 <stevenknight> how about 1.3 p3? (or p4?) 17:41:05 <garyo-home> I'd prefer 2.x, there's too much going on already. 17:41:17 <stevenknight> i can go with 2.x 17:41:30 <Greg_Noel> I still think that a revamp of Configure, which would take over the functionality of not only Configure, but also Option/Variable and other stuff (including getting options from the shell environment), is the right strategy, but as a short term hack, I can agree with a convenience function 17:43:09 <garyo-home> Good. 17:43:37 <Greg_Noel> ok, 2.x, p?, steven? 17:43:43 <garyo-home> 2.x p3 steven 17:44:13 <Greg_Noel> I could also go with p4, not as important as other things. 17:44:24 <garyo-home> either way's ok w/ me 17:44:36 <Greg_Noel> Steven? You're the deciding vote. 17:44:39 <garyo-home> p4 maybe better 17:44:57 <stevenknight> sorry, distracted -- i just broke the chrome build... :-/ 17:45:11 <Greg_Noel> Naughty, naughty... 17:45:18 <garyo-home> can we help? :-) 17:45:14 <stevenknight> 2.x p3 me 17:45:20 <Greg_Noel> done 17:45:24 <garyo-home> 2251: threading issues are hard 17:45:39 <garyo-home> ... to test, at least 17:45:50 <stevenknight> yeah, and I'm far from a threading guru 17:46:10 <stevenknight> but I'm starting to rope more people here into issues like this... 17:46:44 <Greg_Noel> Well, I've been one in the past, but it's nasty work. 17:46:48 <garyo-home> in this case though it seems likely to be something like that, so I'd vote for something that "ought to fix it" even w/o a hard testcase; the testcase should just exercise it a bit. 17:47:03 <Greg_Noel> garyo-home, agree 17:47:05 <stevenknight> garyo-home: agreed, i'll take that approach 17:47:29 <garyo-home> ok, 1.x p3 steven? 17:47:35 <stevenknight> done 17:47:48 <stevenknight> oh, wait, we were saying 2.x, yes? 17:47:50 <Greg_Noel> We don't have a model for making (and keeping) SCons thread-safe and we should. Right now, it's completely ad-hoc and we've been pretty lucky that so few issues have surfaced. How should we go about developing a model, including possible thread locks? 17:48:20 <garyo-home> Greg: without redesigning it for safety? Nearly impossible. 17:48:24 <stevenknight> Greg_Noel, good point, excellent question, I have no idea 17:49:03 <garyo-home> Greg: but thinking about it and discussing it and a few well-placed comments won't hurt :-) 17:49:08 <Greg_Noel> Yeah, but an idea about where to go from here is needed; for example, should this issue have been fixed with a lock rather than a delayed-action flag? No model, so we're working in the dark. 17:49:44 <garyo-home> I agree. 17:50:01 <stevenknight> okay, how about a TASK in the tracker to come up with a model? 17:50:14 <stevenknight> or define it, really 17:50:15 <garyo-home> But we do have a basic model about when threads are created etc. 17:50:18 <stevenknight> at least we track the issue 17:50:24 <Greg_Noel> I'll go for that, but you can't put multiple people on a task. 17:50:25 <garyo-home> steven: can't hurt. 17:50:41 <garyo-home> wiki page in the design doc section? 17:50:48 <Greg_Noel> good idea 17:50:48 <stevenknight> mark it something like 2.0, and either Greg (if you want to drive it) or me (if we want it to lie fallow for a good long while... :-)) 17:50:58 <stevenknight> wiki page ++ 17:51:04 <Greg_Noel> {;-} I'll drive it, then 17:51:15 <garyo-home> great! 17:51:19 <stevenknight> cool, thanks 17:51:55 <Greg_Noel> OK, what did we decide for 2151? 17:52:16 <stevenknight> 2.x p3 stevenknight 17:52:22 <Greg_Noel> done 17:52:26 <garyo-home> good 17:52:28 <stevenknight> and a new TASK for the larger issue of a coherent thread model 17:52:31 <Greg_Noel> yes 17:52:53 <Greg_Noel> "coherent" ==> good word 17:52:46 <garyo-home> 2252: trivial fix 17:52:53 <stevenknight> 2252: consensus 17:52:55 <stevenknight> 2253: moot 17:53:09 <stevenknight> 2254: consensus 17:53:44 <garyo-home> 2255: Greg, can you go w/ Steven's idea? 17:53:51 <stevenknight> 2255: consensus except for Greg -- you okay with the proposal on the table? 17:54:10 * Greg_Noel still catching up; hadn't read all the new comments before 17:55:27 <Greg_Noel> Yes, add compat layer; only the one is needed post 2.0, but the other is, ah, problematic. 17:55:58 <garyo-home> how? 17:56:34 <Greg_Noel> no Python support to get the needed information 17:57:03 <garyo-home> Could it be made to be a noop on old pythons or something? 17:57:42 <stevenknight> if it's a real problem, i'll do something like that rather than spend huge amounts of time on it 17:57:53 <stevenknight> hopefully 1.5.2 support only lives for another couple months anyway 17:57:58 <garyo-home> right. 17:58:31 <Greg_Noel> Are you suggesting that we add a new get_text_contents? I'm not sure I like that solution, unless it becomes a noop on systems that don't need it (memory impact) 17:58:59 <stevenknight> ??? i don't think it should be, IIRC how it was implemented 17:59:12 <stevenknight> it's not going to hang on to the decoded text 17:59:33 <Greg_Noel> I'm not so sure... 18:00:12 <Greg_Noel> how about research rather than committing to a fixed release? 18:00:31 <Greg_Noel> and bring it back to triage when there's more information? 18:00:53 <garyo-home> Not sure what that more information would be. Whether it would be a memory hog? 18:00:57 <stevenknight> well, i can live that, i guess 18:01:08 <stevenknight> if you specify what information you're looking for, i'll bring it back 18:01:17 <garyo-home> I think there's no system on which it's not needed. 18:01:58 <Greg_Noel> Where/how get_text_contents would be used, whether text would be saved/cached, that sort of thing. I18n text is expensive. 18:02:50 <garyo-home> It would be used in scanners. Any utf-8 source code could have this problem. 18:03:02 <stevenknight> just checked the code, the text is not saved/cached 18:03:14 <stevenknight> it's decoded by the scanners as needed 18:03:16 <garyo-home> But I get the point that it takes more memory. 18:03:43 <Greg_Noel> OK, maybe I'm being stubborn. "Memory is infinite and free", right? 18:03:49 <garyo-home> If you had a monster utf-8 resource file (e.g.) it could take 2x the storage to get its text contents. I don't think that's a problem. 18:03:56 <Greg_Noel> 4x. 18:04:10 <garyo-home> true, up to 4x depending. 18:04:21 <stevenknight> sure, but only while scanning 18:04:24 <garyo-home> (or is python always 4x internally?) 18:04:29 <stevenknight> and the alternative is SCons doesn't work at all for you 18:04:33 <garyo-home> yes, only during the scan, then it's gone. 18:04:52 <garyo-home> Steven, I basically agree, this is needed. Just want to tease out all the implications. 18:04:55 <Greg_Noel> (Python always uses 4x on all platforms now) 18:05:00 <stevenknight> okay 18:05:41 <Greg_Noel> I'll go with what you two decide. 18:05:59 <garyo-home> Doesn't seem like memory usage would be a huge problem. I vote for implementing it and testing it on a couple of large builds; we have mem test infrastructure now. 18:06:11 <Greg_Noel> OK, that works, 18:07:01 <garyo-home> 2255: 1.x p2 steven then? 18:07:05 <stevenknight> done 18:07:08 <Greg_Noel> done 18:07:19 <stevenknight> 2256 & 2257: consensus David, 1.3 p3 ? 18:07:25 <garyo-home> yup 18:07:46 <garyo-home> 2258: invalid 18:07:57 <Greg_Noel> done 18:08:13 <garyo-home> 2259 consensus (I'd like this too) 18:08:48 <Greg_Noel> done 18:09:02 <stevenknight> 2260 consensus invalid 18:09:11 <garyo-home> 2260: I feel like it's too "interesting" to just mark it invalid somehow. 18:09:28 <stevenknight> future? 18:09:28 <garyo-home> future? 18:09:31 <garyo-home> :-) 18:09:40 <Greg_Noel> What does Clean() do on a directory? We may already have a fix. 18:09:58 <garyo-home> Good question. 18:10:03 <garyo-home> research, then? 18:10:14 <Greg_Noel> ok, research, who? 18:10:37 <garyo-home> I'd love to but I am overcommitted. 18:11:28 <Greg_Noel> I can check on Clean(), but I have personal stuff coming up, so my time will be limited over the next couple of months 18:12:36 <Greg_Noel> Did we lose Steven again? 18:12:49 <garyo-home> maybe he just doesn't want it either :-~ 18:13:03 <Greg_Noel> Or he could have broken another build... 18:13:12 <garyo-home> maybe. 18:13:45 <stevenknight> yep, broke it again 18:13:46 <Greg_Noel> Let's make it research, me, and I'll toss it back if Clean() won't work. 18:13:53 <garyo-home> ok. 18:14:04 <garyo-home> and then we'll mark it future. 18:14:08 <stevenknight> i'm pretty sure Clean() does it 18:14:09 <garyo-home> thanks! 18:14:16 <stevenknight> agreed, thanks for taking it 18:14:38 <garyo-home> So, discuss 1.2 plans? 18:14:47 <stevenknight> done with that spreadsheet; spend a little time on editlist2005q2? 18:14:48 <Greg_Noel> That concludes this spreadsheet, should we go on? Or do you need to pay attention to your build, Steven? 18:14:51 <stevenknight> oh, 1.2 better 18:15:06 <stevenknight> i have to wait for my second fix to build anyway... 18:16:09 <garyo-home> btw, Greg, have you noticed some decay in the BugParty page? 18:16:18 <stevenknight> 1.2 is overdue, so my inclination is to get a candidate out there 18:16:18 <Greg_Noel> 1.2 is due out 24 Nov 18:16:30 <stevenknight> right, sorry, candidate at least is overdue 18:16:35 <stevenknight> i only sent out the one checkpoint so far 18:16:36 <Greg_Noel> what decay? 18:16:57 <garyo-home> Check it out. Words with missing parts, damaged lists... 18:18:18 <garyo-home> steven: 1.2 candidate any time is fine w/ me. I only wish I had more time to get my fixes in. 18:18:24 <stevenknight> me too 18:18:28 <Greg_Noel> also 18:18:48 <stevenknight> the big thing I'd like to get in is a performance improvement I've been working on for folks here 18:19:05 <stevenknight> it changes the LIBPATH / CPPPATH search from linear (for each .h file for each .o file) 18:19:13 <stevenknight> to O(1) by collapsing the directories into a lookup dictionary 18:19:36 <garyo-home> That sounds good. 18:19:39 <stevenknight> one of our libraries (from an upstream project) has literally ~80 directories in CPPPATH 18:19:49 <stevenknight> and we use Repository() to multiply that x3 18:20:00 <stevenknight> it cut the SCons overhead literally in half 18:20:09 <garyo-home> amazing. 18:20:24 <garyo-home> Were you also looking at a quoting issue? 18:20:33 <stevenknight> yes 18:20:42 <stevenknight> actually, string substitution in general 18:20:54 <stevenknight> but it dovetails with the quoting for command execution 18:20:57 <garyo-home> Ah, right. 18:21:32 <Greg_Noel> (Gary, I see it, it must be recent, I'll check into it.) 18:22:04 <garyo-home> So is 1.2 still possible on 11/24? 18:22:17 <stevenknight> in its more-or-less current state, yes 18:22:23 <Greg_Noel> How big is the change? Should it be kept for a checkpoint post-1.2? 18:22:31 <stevenknight> probably post 1.2 18:22:46 <stevenknight> it moves a bunch of scanning logic from the Node class into the Scanner proper 18:23:00 <stevenknight> so it's potentially impactive and needs some baking time 18:23:15 <stevenknight> actually, Gary, you could try giving it a sanity check if you want 18:23:24 <stevenknight> its in branches/sgk_PathList 18:23:24 <garyo-home> In that case, and given vs_revamp, the sooner we get 1.2 out the sooner both those changes can move into being testable 18:23:35 <Greg_Noel> concur 18:23:33 <garyo-home> steven: I'll try it out this week. 18:23:40 <stevenknight> should be able to point to 18:23:45 <stevenknight> okay, that makes sense 18:23:57 <stevenknight> i'll go ahead and work on the candidate checkpoint after we're done 18:24:04 <stevenknight> i should have some downtime in between breaking builds... 18:24:14 <garyo-home> :-/ 18:24:14 <Greg_Noel> {;-} 18:24:12 <stevenknight> and then ship 1.2 next week 18:24:22 <garyo-home> sounds good. 18:24:22 <Greg_Noel> works for me 18:25:23 <stevenknight> okay, then 18:25:28 <Greg_Noel> (Uh, wow, maybe it's Moin; I've got some other pages with lists that are broken...) 18:25:52 <garyo-home> I think that page has had minor damage for quite a while, but it just got a lot worse. 18:25:46 <stevenknight> any cycles to look at a few 2005q2 bugs, or do we need to wind down? 18:26:00 <garyo-home> I can do a few, Steven. 18:26:07 <Greg_Noel> I've got time 18:27:05 <stevenknight> okay, 1136: 18:27:09 <stevenknight> consensus 1.x p3 stevenknight 18:27:19 <garyo-home> ok 18:27:27 <Greg_Noel> done 18:27:43 <garyo-home> 1140: could Ignore() help here? 18:28:11 <Greg_Noel> Probably not; you want the dependency 18:28:29 <stevenknight> but you can't have that dependency without making a cycle 18:28:37 <stevenknight> it needs to be broken one way or another... 18:28:41 <Greg_Noel> I've had to create fake dependencies to deal with it in the past 18:28:56 <Greg_Noel> That works, but it's a hassle 18:29:09 <garyo-home> You basically want the file to depend on all the other files in the dir except itself, right? 18:29:23 <Greg_Noel> yes, recursively 18:29:26 <stevenknight> off the top of my head, that sounds right 18:29:29 <garyo-home> eek! 18:29:57 <garyo-home> ok, I don't know why this is useful but still sounds like you could make the file depend on the dir and then use Ignore (?) 18:30:17 <garyo-home> But I haven't thought about it a lot so feel free to Ignore me :-) 18:30:47 <Greg_Noel> garyo-home, bad pun! I like it! 18:30:18 <Greg_Noel> The thing with using Glob() for dependencies that Ludwig is working on would solve it, but I don't know if he can do it. 18:31:24 <stevenknight> so someone research this for a good solution? 18:31:37 <Greg_Noel> maybe Ludwig? 18:31:51 <stevenknight> if we go with Ignore(), it should at least be documented as the recommended pattern 18:32:22 <Greg_Noel> Ignore is applied after dependencies; there's still a loop. 18:33:57 <Greg_Noel> (Silence while we contemplate.) 18:34:31 <stevenknight> well, it sounds like a research for someone 18:34:39 <stevenknight> to either find the right code fix or the right doc fix 18:35:29 <Greg_Noel> Let's see if Ludwig is willing. 18:36:08 <stevenknight> okay, sounds good to me 18:36:26 <stevenknight> 1140: resesarch, Ludwig 18:36:27 <Greg_Noel> Gary, you OK with that? 18:36:37 <garyo-home> yes 18:36:40 <Greg_Noel> done 18:36:48 <stevenknight> 1142: FIXED, ludwig 18:36:52 <Greg_Noel> done 18:37:30 <stevenknight> 1143: 2.x p4 steveknight? 18:37:44 <Greg_Noel> 1143, don't use FilterOut; I've got an enhancement with that name on it 18:37:47 <garyo-home> sure! 18:37:58 <Greg_Noel> done 18:37:59 <stevenknight> okay, i prefer env.Remove() myself anyway 18:38:03 <garyo-home> me too 18:38:07 <stevenknight> parallel with list.append => env.Append(), etc. 18:38:51 <stevenknight> 1152: gary, you filed it, so your 2.x p3 trumps 18:39:05 <garyo-home> 1152: sure, count me in. 18:39:10 <Greg_Noel> works for me 18:39:18 <stevenknight> 1152: 2.x, p3, garyo 18:39:18 <stevenknight> done 18:39:36 <stevenknight> 1161: 2.x p2 gregnoel ? 18:40:13 <Greg_Noel> OK, although I'm going to have to digest Steven's comment... 18:40:45 <stevenknight> nah, just spit it back up -- i'm blathering on like i usually do 18:40:44 <garyo-home> Greg: by proxy wrappers you're thinking about -Bstatic/-Bdynamic, right? In which case I agree, this is the same. 18:41:06 <stevenknight> i don't think my comment adds any real value 18:41:27 <Greg_Noel> yes, Bstatic() and Bdynamic() proxy wrappers 18:42:27 <garyo-home> I like tags better than proxy wrappers but if you're implementing it, you choose. 18:42:55 <Greg_Noel> This case needs to return a list, so it needs some sort of wrapper 18:43:24 <Greg_Noel> I'll come up with a proposal, then re-triage it 18:43:27 <garyo-home> I see your point, otherwise something else has to collect the tagged libs, and that gets hairy 18:43:49 <garyo-home> OK, greg re-triage w/ proposal 18:44:50 <stevenknight> done 18:45:00 <Greg_Noel> 1164, looks like the consensus is future 18:45:11 <stevenknight> works for me 18:45:14 <stevenknight> 1164: future, p4? 18:45:18 <Greg_Noel> done 18:45:20 <garyo-home> yes 18:45:31 <stevenknight> 1166: shall we just put it in to get it off the plate? 18:45:48 <stevenknight> we've been kind of doing that for other tools with small user bases 18:45:56 <Greg_Noel> I don't use it, so that's fine with me 18:45:58 <garyo-home> I think there's another bcc ticket lying around, hang on 18:46:45 <garyo-home> yes, there are a few actually. 18:46:59 <Greg_Noel> The -e$TARGET should be first, if the command accepts that, to be consistent with other builders. 18:47:32 <garyo-home> I did 2163 already, so why don't I do 1166 (blind) 18:47:41 <stevenknight> gregnoel: agreed 18:47:47 <Greg_Noel> done 18:47:53 <stevenknight> garyo: thanks: 2163: anytime garyo 18:48:02 <Greg_Noel> yes, anytime 18:48:10 <stevenknight> er, 1166: anytime garyo 18:48:20 <garyo-home> sure. Also I have 2164 but never got any response from the OP. 18:48:24 <garyo-home> Maybe he's moved on. 18:48:41 <stevenknight> seems likely 18:48:48 <stevenknight> 1170: consensus research, david 18:49:18 <Greg_Noel> done 18:49:38 <garyo-home> yes 18:50:03 <Greg_Noel> 1175, another blind change for Gary? 18:50:12 <garyo-home> oh joy 18:50:15 <garyo-home> sure, why not. 18:50:22 <garyo-home> i'm brave 18:50:31 <Greg_Noel> done 18:51:10 <garyo-home> 1176: agree w/ Greg 18:51:32 <stevenknight> 1176: agree 18:51:40 <Greg_Noel> done 18:52:00 <stevenknight> 3: greg filed it, has it been fixed? 18:52:29 <Greg_Noel> damifino, I used a workaround 18:52:06 <stevenknight> i think you're right about gary working on it since then 18:52:40 <garyo-home> I did clean it up a lot. I seem to remember I was skeptical about changing the existing behavior though. Anyway I'll look at it again. 18:52:57 <Greg_Noel> anytime, garyo? 18:53:15 <Greg_Noel> or is research better? 18:53:20 <garyo-home> Would you all be in favor of changing the behavior? Would cause rebuilds... 18:53:36 * Greg_Noel has his hand up 18:53:41 <garyo-home> call it research. Need to see if this case works or not. 18:54:10 <garyo-home> I'd rather it be changed too. 18:54:12 <Greg_Noel> Rebuilds would be very rare... 18:54:43 <Greg_Noel> You OK with that, Steven? Or did the build break again? 18:55:00 <garyo-home> And one more q: if you append [1, 2, 1], should it append 1,2 or 2,1? 18:55:26 <Greg_Noel> You added the option, follow the option. 18:55:44 <garyo-home> That makes sense 18:55:51 <stevenknight> build break 18:56:13 <garyo-home> It's about time for me to go anyway. 18:56:34 <stevenknight> i'm okay with changing behavior if the new is better, and we have a reasonable path 18:56:44 <garyo-home> ok 18:56:48 <stevenknight> so 3: research, garyo 18:56:55 <Greg_Noel> done 18:57:09 <stevenknight> last parting shot: 1180: 1.x, p3, me 18:57:16 <stevenknight> it's a definite problem, we leak tmp*.lnk files 18:57:24 <Greg_Noel> I should go soon as well... 18:57:32 <Greg_Noel> 1180, done 18:57:37 <garyo-home> ok, thanks all! 18:57:38 <stevenknight> sometimes SCons slows down because of huge numbers of leaked files in /tmp or %TMPDIR% 18:57:43 <stevenknight> all right, good progress 18:57:46 <stevenknight> many thanks, guys 18:57:53 <Greg_Noel> er, 1182, David? 18:58:05 <stevenknight> 1182 david, yes 18:58:11 <garyo-home> ok 18:58:12 <stevenknight> they're like potato chips! 18:58:16 <garyo-home> 2 weeks from now, right? 18:58:21 <stevenknight> yes, two weeks 18:58:26 <Greg_Noel> The rest next time, yes, two weeks, after Thanksgiving. 18:58:27 <stevenknight> have a good turkey day 18:58:39 <garyo-home> ok, I'll see you then. Happy Thanksgiving! 18:58:40 <Greg_Noel> same to you guys 18:58:45 <Greg_Noel> cul 18:58:49 <garyo-home> bye 18:58:50 <stevenknight> l8r 18:58:52 * stevenknight (n=stevenkn@nat/google/x-93de785fa2957b48) has left #scons ("Leaving") 18:58:56 * Greg_Noel goes for dinner 18:59:00 * garyo-home has quit ("ChatZilla 0.9.84 [Firefox 3.0.4/2008102920]")