On a more serious note, I think that high bar to getting things into python is a good thing. What if Zed's changes were accepted, as is, into the Python codebase today? What about when the next guy comes with his alternative APIs for option parsing and email along with a new standard for templates? Should they include them, too? How many completely different APIs for the same thing should they include in the standard library?
Any software system as large as Python is going to be imperfect -- it is constrained by backwards compatibility and conflicting requirements. However, Zed is free to build whatever he wants outside the core and make it available to the world. If it catches on enough, maybe his followers will go through the social process to get it into the core. This process helps to ensure consensus, consistency, and quality.
> What about when the next guy comes with his alternative APIs for option parsing and email along with a new standard for templates? Should they include them, too?
If they improve the consistency of the language, sure. Keep the old APIs for a bit with a deprecation warning. Python 3 makes good breakpoint to do this too, as though released, it's still not used for most projects - the core team are making similar post-3.0 changes themselves.
I agree, this is something that shouldn't be solved arbitrarily in a first-come first-served fashion.
However, I don't think this is something that will ever be solved or even addressed as long as Pythonistas continue to pretend Python is super consistent.
A good first step towards fixing these kinds of issues would be to admit the issues exist at all.
So you've missed the rounds of "name 5 things you don't like about Python" posts by prominent people in the community, many of which harp on various modules in the standard library?
That there are things in need of improvement is not disputed. What to do to accomplish the improvement, well, that's a topic to go 'round and 'round about...
You are welcome. Actually, the inclusion of your processing module in 2.6/3.0 is an excellent example of a non-core developer getting code into the Python distribution. Your blog postings on the process (especially http://jessenoller.com/2009/01/28/multiprocessing-in-hindsig...) really show how challenging it is to add something to a system that big (and ported to so many platforms). I appreciate the work you've put into processing (and one of these days I'll get around to using it).
Overall, the Python community strikes me as a bunch of really pragmatic, low-key people who have learned the hard lessons in the trenches of software engineering. To me, this is almost more important than the details of the language.
Some people really need to be on the offensive to get things done. They need to feel like someone else is the enemy and they are going to defeat them. It seems to me like Zed puts himself in the position of David vs the open-source-community-Goliath on purpose so that he can be on the offensive. He gives himself an enemy, throws down the gauntlet and declares that he can and is going to make things better. I'm not one to argue that he picks the best motivational targets, calling people out and demeaning a hard-working community seems pretty negative, but damn when Zed says he is going to deliver, Zed delivers. Laser focused, intelligent and hard-working. I would love to work with Zed, hire him or have him contribute to one of my projects, but damn, I'd be terrified of pissing him off.
Zed's the enemy of crap. Whether it's inconsistency in languages, poorly created enterprise software development practices (The ACL is dead), or egotism and leeching from others work without acknowledgement (Rails is a Ghetto).
If I pissed him off, I'd listen to why he's pissed off, because he's good at identifying crap.
"Zed's the enemy of crap. Whether it's inconsistency in languages, poorly created enterprise software development practices (The ACL is dead), or egotism <b>coming from others (Zed's so fucking awesome)</b> and leeching from others work without acknowledgement (Rails is a Ghetto)."
There. Fixed that for you.
Zed is has a really, amazingly, big hat. Too bad there is next to no cowboy under it.
I may share your position on Zed, but people here generally dislike comments that don't add to the discussion, or are blatant attacks on someone; your post fits both perhaps?
Zed may have his quirks but he's a good programmer and is right more often than not about these things.
Guido's reply is classic: "you can't criticize my work unless you fix everything you complain about." This is a nice way to avoid having to debate the merits of the criticisms, but doesn't really advance the dialog (or the code) very much.
"I will provide my fucking awesome fix on the condition you include and recognize my fucking awesomeness. If you don't promise that, I will submit nothing"
Thus avoiding a clear review of Zed's shortcomings.
Zed is in denial. He can't grasp the fact that he is not that fucking awesome. And that, perhaps, he never really was awesome.
Who is alienating whom? It sure sounds like Zed is offering to do something that he doesn't have to do, that could potentially be more work for him to maintain, and that could easily lead to a lot more political trouble in the long run.
The way I see it, Zed made a (correct and polite) criticism of Python. Someone childishly taunted back at his criticism with the typical "where's the patch?" defensiveness, and he's now responding in a rather polite fashion. If Zed is making a mistake, it's that he's engaging in the debate at all. He's expressed his opinion -- if the Python people are so petty that they can't handle constructive criticism, the best response is to ignore them and move on.
I would hesitate to call his code "excellent". How about chunked encoding for mongrel? Here (http://www.ruby-forum.com/topic/142852) is his naive rant about it, but it turns out that there are real reasons for CE. I would describe his code as a ghetto.
Jesse gave a lightning talk at PyCon this past year, blowing off some steam about the people that complain about Python on Slashdot/Reddit/Hacker News without submitting a patch. The video is here if you're interested. It's actually kinda funny: http://pycon.blip.tv/file/1931026/
He suggested a new Python license that would forbid people complaining about the language without first submitting a patch. If people did this they would be "Van Lindberged". Van Lindberg is the Python software foundation's lawyer.
Jesse proceeded to showcase the "Van Lindberg engine." It was a javscript bookmarklet that took a comments page on reddit that complaining about python, and superimposed pictures of Van Lindberg all over it. The reddit page had then been "Van Lindberged" or "Lindbergified".
It didn't sound like Guido was trying to start shiznit with Zed. It sounded like he was making light of the situation, and suggesting that Jesse "Van Lindberg" Zed for his comments.
Van Lindberging is funny because it's an extreme response to what some devs see as a real problem, opinions without patches. Zed's pointing out he does in fact have patches, and can have more patches if they're likely to be accepted.
He has no patches. I see nothing on python-dev, bugs.python.org, etc. All I see is someone saying "I have code which is the balls, awesome". That's it.
Then he should probably raise that fact on python-ideas if he hasn't already. The SMTP library hasn't been tried and tested to any significant degree in the real world, so the likelihood of having it accepted right now is slim -- but it's the usual approach for getting stuff like this on the radar...
He will not do it. He doesn't like to play with others or have his brilliant ideas scrutinized by who he considers lesser people.
So, the way he has to prove me, and others who agree with me, is
- to provide the said patches (a plus, if they don't collide with already existing libraries, let's call Zed's SMTP "zsfasmtp") in publicly accessible libraries
- accept criticism and patches from others
- work with the Python maintainers to help them include his fucking awesome libraries in a future Python release
And that's it.
Until then, we all would gain if only he did shut the fuck up.
Programming? Nerds? Open Source Software? People running businesses? Hacker News?
I dunno, what do you think is common to all of these situations? :P
(point being, zed surely could stir the pot less, but he's not changing what's in the pot. And if two pots happen to be similar, i don't know that that's his fault)
being an asshole doesn't have that affect. It's very easy to find flaws in anyone's work. For instance, almighty zed didn't implement chunked encoding in mongrel. You don't see me running around calling him names and putting down the work he's given to the community to boost my ego. It wouldn't have any positive affect at all.
"It wouldn't have any positive affect at all."
Really? You mean you are saying it is impossible that because someone high profile like zed shaw pointed out some factual deficiencies in a product, that it would never light a fire under their ass to fix them....because the manner in which he did it was rude? But, if he was polite about it, it might get fixed?
I can see how politeness would be more successful, but I am comparing causing visible public scrutiny while being an asshole vs no public scrutiny.....and you say this approach never could and never has worked?
Think about what you said for a second. You start by saying that zed is "high profile". You then assert that if he were polite there would be no public scrutiny.
The truth is that one must create these false hypotheticals in order to justify zed being an asshole.
"You start by saying that zed is "high profile". You then assert that if he were polite there would be no public scrutiny."
Huh? Maybe try a re-read of my post.
I'm not justifying him being an asshole, I am countering your assertion that if you are an asshole while pointing out legitimate shortcomings, that it will have no positive affect.
I re-read, here is what you said "but I am comparing causing visible public scrutiny while being an asshole vs no public scrutiny"
This is a false choice as is clear by your statement that zed is high profile. If he were polite there would still be public scrutiny so your binary hypothetical is meaningless.
For some reason, I cannot infer the same after reading that article. Probably, I'm low on caffeine. What sequence of events do you visualize in near future? They will deny and he will walk away? I guess this time the whole ball game is different. Zed has placed himself in a different situation now :)
Pythonists will turn up their nose when they will see Zed walking down the same street. He won't be invited to dinner. People will hide their babies when he shows up. This is just the beginning.
He seems to be trying to become a "center of gravity" in python-land by being controversial. Why not just work with the community in a productive non-confrontational way?
Lots of opportunity for pop-psychology here... but best to let it pass...
Hah, yeah I think I'd be more impressed if Zed had actually submitted some patches and had them refused. Why did this blog post have to happen now, before Zed was actually wronged?
Uh, what if every would-be open source contributor wrote a grandstanding blog post promising only to submit patches if the maintainer(s) publicly eat crow for an offhand remark they made ... :)
Zed is a gifted hacker of software systems, but I don't really think he's all that good dealing with other mammals.
I'm sure he's a nice guy -- he actually seems like the kind of person I'd be friends with -- I just think it's appropriate to sort of call him out on the grandiose blog posts :)
I'm all for it to the extent that Zed is building his personal brand and (in essence) being a self-promoter. If he really feels self-righteous about it, then that's a bit worrisome.
Zed vs Guido is exactly the sort of David vs Goliath story that makes for entertaining reading...
"It seems to me he's justified in not wanting to waste his time writing patches that wouldn't be accepted anyways."
He is free to provide packages that replace core python libraries. The fact he doesn't want to do so and would only give his contributions if they are included in the standard Python libraries is childish, at best.
Zed was actually quite polite [but strong] in that post. For the first time I'm starting to believe that he has kept the promise of changing. I might subscribe again.
From the article: "I will make a long bet right now, that no matter how nice I am, no matter how many people use my software, no matter how much better it is than anything Python has that’s similar, Guido will not accept them as changes."
Man, how do you think we got optparse in the first place? People got sick of using getopt to parse command-line arguments (now there's a C-style option-parsing library for you) and somebody said "Hey, there's this third-party tool called Optik (http://optik.sf.net) that's much better than anything Python has that's similar, let's add it." And so, after some messing around with evaluating alternatives (http://www.python.org/community/sigs/retired/getopt-sig/) Optik was adopted as optparse.
I see no reason why Zed's shiny, improved replacements couldn't become part of the standard library if they're as good and as popular as he claims they'll be, so long as they're independently reusable and not all bundled together in some 'lamson' hierarchy.
I agree. I'm not sure what the fuss is all about - it's not like the standard library was coded by the Python core monks in seclusion a thousand years ago. Much of it has grown and taken in new things over time.
And it's not the end of the world if his changes aren't accepted. He'll still have his own libraries, and people can enjoy them through PyPI.
That said, Guido is definitely very picky about changes to the language itself. I think that's a good thing.
While it can be taken too far, there are reasons for that attitude, too. I think the most important one is that until you sit down and try to write a patch that actually meets all the relevant needs, you don't have good odds of making a really strong engineering criticism. Any idiot can sit on the sidelines and throw stones.
For instance, Zed says he wrote an options parser and it sounds like he wants it to ship with Python. Well, has he made the API reverse compatible? If not, is his really so much better that it's worth breaking reverse compatibility? Should we now ship two option parsers? These are questions that are easy to elide over in a blog post, but must be solved in the process of building a real patch, and criticisms that come from a true, honest attempt to build a real patch are much more valuable.
(Also, note the distinction I draw between simply replacing the option parser Python ships with because you have something better, vs. balancing the desire to not break old code against the better choice. Presumably Zed doesn't have years of code written against the options parsing module that Python ships with; those who do will have rather different priorities when it comes to deciding whether to ship a new one, if it breaks their code.)
At work, I frequently come across old code, go "This is stupid!", and replace it. Somewhat less frequently, I come across old code, say "This is stupid!", but discover in the process of replacing it why it was done that way. (No comments, of course.) It would be easy for me to go yell "It's stupid!" in both cases, but until I actually spend some time fixing it, my criticism is objectively less valuable.
There are other reasons too, this is just what I think is the best.
Let me repeat my first point so it is both my first last: This attitude can be taken too far. But the seed of the attitude is there for good reason.
See also: http://www.userland.com/whatIsStopEnergy (Note I don't think Zed has "stop energy"; he has code and he is pushing for things to be done. But I think this is related to the "show me the patches" issue, because in the general open source case it is one of the easiest ways to throw stop energy at somebody.)
Just regarding "stop energy", the thing I love about software and startups is you can just do something. You don't to convince a corporate manager, nor an academic panel, nor the board of developers. All you need do is make something useful, and let the people know about it who need it.
Yes, you still have to convince people - but they aren't gateway guardians whose good opinion you must seek, but people who have the problem that you are trying to solve. Your interests are aligned.
you made so many good (written) points, yet accrued so few (karma) points. i know my own vacuous comment will be down-voted, but it's worth it to say Good work.
Just one point - the 'Where are your patches' attitude takes for granted that the implementation is more important than the design. This might be good approach at the beginning to bootstrap a library or language - but after it gains momentum then it becomes clear that API design is at least as important as the implementation of those APIs. And if design is important - then we need to accept that design critique is an important part of the Open Source development process.
Lately I have been searching for online API design guidelines - and the results were really rather dissapointing - there aren't many good online resources on that subject (or at least they are not easy to find). But all that I found were stressing how important it is to write use cases and example code using the API before wrtiting the implementation. I wonder what OS projects would accept use cases and example code as patches.
Implementation is a lot less forgiving than design. Therefore, requiring a working implementation is a way to sort out a large amount of (unworkable) design ideas.
True - that's a very good point explaining why the loose coordination of Open Source works so good. But it also has limits - so to reach a next level we need to think how to incorporate design into that view. Maybe if we are more strict with our methods we can make the design also easier to objectively evaluate and sort out those that don't meet the standards.
I know I'm missing the point, but my main reaction to this is naive admiration that Zed can complain about a language in a blog post and get flamed by the freakin' creator of said language.
You acknowledge that the email systems in Python need an overhaul and that my work will seriously be the basis of the effort.
Well, who's gonna promise that up front? Maybe Guido would be interested if Zed said, "You acknowledge that the email systems in Python need an overhaul and that my work will seriously be considered as the basis of the effort."
It's interesting to compare the Python situation to that of Perl.
With Perl, there's a dozen solutions on CPAN for every problem, and you can pick and choose to find the best one. In Python, there's usually one that's had the blessing from above, and like-it-or-lump-it, that's the one everyone's using.
But the end result is, with Perl, in the long run, even though it's more work, the community ends up eventually filtering through the choices and homes in on the nearst-optimal solution. With Python, the higher-ups make the decision and you get what you get.
What's striking is that even with all its faults, Perl still can often win. Python fans look down at the playing field and they say, "How can this be? How is Perl still scoring goals? Python's got more equipment, a more detailed playbook, fancier training facilities, bigger crowds of fans, even snazzier jerseys. Perl's team is scruffy, scarred, and unkempt; they train on old equipment, and half the time they don't even seem to be taking the game seriously. Yet they still manage to score!"
As an aside: I like how even though Zed said he's done with the over-the-top persona, the real Zed is still showing through like always. :)
With Perl, there's a dozen solutions on CPAN for every problem, and you can pick and choose to find the best one. In Python, there's usually one that's had the blessing from above, and like-it-or-lump-it, that's the one everyone's using.
I think you have a misconception about Python.
The equivalent to CPAN in the Python world is not the std library but PyPI (easy_install).
Zed's whining is inappropiate, he should just get to work. The current recommended practice to get something into the stdlib is:
1. Build it as an external package.
2. If it's good people will use it, it becomes popular, bugs get fixed.
3. Eventually it enters the stdlib (potentially obsoleting whatever package previously did the task)
Yes, the stdlib has accumulated a lot of cruft over the years due to the higher ups not adhering to this procedure. That's only more reason to tighten the criteria and accept new stuff only when it has proven itself in the wild - and not because a particular name is printed on it (be it Zed or someone else's).
Do you expect that the canonical Perl 6 implementation of the standard library (I guess this is whatever will come with Rakudo?) will suffer from this follow-up problem about the same as Perl 5, or less-so?
In what project in what universe are outside contributions not subjected to more stringent quality control than those made by people who are already trusted members of the development community? How could it possibly be any different if you don't want your tree cluttered with garbage?
The post is saying the garbage is coming from within because trusted members can get away with worse code.
Enforcing different standards of quality control is unproductive since the weakest link in the chain determines the overall quality. Now, certainly you should scrutinise outside contributions more strongly than trusted contributions, but this isn't the argument.
The difference is that code from an existing developer is going to be maintained: that is, the existing developer will stay around to take care of it.
Code from an outside developer will just be committed and never touched again in many cases, since the outside developer doesn't intend to stay and work on the project. If you don't believe me, look at the history of the ffmpeg project: there are enormous swaths of unmaintained code (and uncommitted code) for exactly this reason.
Thus, it is often reasonable to have higher code standards for code whose author does not intend to maintain it into the future.
Great code is still subject to entropy. Shit will break. Mail servers will start using some proposed SMTP extension, someone will find a nasty bug or two, there will be security problems and inconsistencies and inefficiencies discovered.
Programmer A has been working with me for a long time, has done a lot of B- to B+ code. I trust that what he does is going to be up to that level.
Programmer B has been working with me for a long time and consistently turns in C level code. It isnt great, it isnt horrid. Call him pedestrian. When I get something from him, I know what it is going to be. Adequate. Not everything has to be great, most things move forward on adequate because there are too many things to get for B and A level people to do them all.
Programmer C is someone I don't know it all. Maybe I've heard of them by reputation, maybe not. Doesn't much matter, there is no trust there, I don't know what I'm getting. If I'm responsible, I'm going to vet their code closely. Make sure they follow coding conventions, best practices, etc. This takes time on my part. And my time is limited because now I am BDFL or some similarly ridiculous title that means "He's a busy guy".
Is it worth breaking backwards compatibility to have fancy libraries distributed in the Python core?
Zed is employing a false contradiction: he can make his patches available and people can use them regardless of them being or not included in the core Python.
In fact, my modules that are now core started this way.
Does anyone else aside from zed have burning issues with optparse? I'm genuinely curious. I use it quite a lot, but have never really had to push the boundaries of what optparse can do. As an example, my optparse code usually looks something along these lines:
Zeds way (look in the lamson source: it is in lamson.args and well documented) gives you a bi more freedom.
So it passes out any options it finds by default and lets you decide what to do with it - you only have to call a function if you want to set a default.
It lets you run args.parse_and_run_command to directly run a function and pass it the cmdline options as a dict.
It's all very handy and is totally compatible with previous cmdline structures!
I think it's a neat option & hopefully someone will convince Guido to have it in Python core in the future :) (doubt it though)
Grab the lamson source and dig it out to get a better idea.
I use it frequently and I like the output it creates. I sure as hell like python's better than C#'s (== none at all and people doing all kinds of crazy broken crap inlined into Main(string[] args).
I've never managed to use optparse without the docs open though. I'm not sure that's a defect, but it seems like room for improvement.
Can someone (zed?) point at a library in any other language that's better?
"I will make a long bet right now, that no matter how nice I am, no matter how many people use my software, no matter how much better it is than anything Python has that’s similar, Guido will not accept them as changes."
That's the same "not invented here" pattern that happened with Parc Smalltalk, and which persisted until a few years ago. (4 companies later.)
I've had similar frustrations with PrototypeJS. I've tried contributing, but them choosing to include the patches is solely based on their mood as far as I can tell. I've even went so far to completely rewrite the AJAX portion (without drastic changes and shifting to better event handling+aborts)...
There are a lot of frustrations in doing that and it pretty much being moot on the politics of the community.
Here's the thing: Go to New York. Go to any random street corner. Wait until traffic starts heading your way, then step into the middle of the street. Guess what will happen?
I'll tell you what: You'll be called a stupid ass, a moron, a fuckin' tourist, etc. You know what else? All those "rude" New Yorkers will have also just saved your life. So, do you thank them or take offense?
On a more serious note, I think that high bar to getting things into python is a good thing. What if Zed's changes were accepted, as is, into the Python codebase today? What about when the next guy comes with his alternative APIs for option parsing and email along with a new standard for templates? Should they include them, too? How many completely different APIs for the same thing should they include in the standard library?
Any software system as large as Python is going to be imperfect -- it is constrained by backwards compatibility and conflicting requirements. However, Zed is free to build whatever he wants outside the core and make it available to the world. If it catches on enough, maybe his followers will go through the social process to get it into the core. This process helps to ensure consensus, consistency, and quality.
For a thoughtful response from a core Python developer, see Brett Cannon's blog post: http://sayspy.blogspot.com/2009/05/staleness-of-standard-lib....