Getting off the island in 2013

Submitted by Larry on 31 December 2012 - 12:13am

And so 2012 draws to a close. The world didn't end, to the disappointment of many. In some ways it was an eventful year, in others rather ho-hum follow-ups to the excitement of 2011.

In the Drupal world, though, 2012 will go down as the year Drupal finally began replacing NIH with PIE. Compare Drupal's 8.x branch a year ago with it today. A year ago, we had one class of 3rd party PHP code, in an uninteresting corner of the update system. Today, it contains 3rd party code from no less than five different projects: Symfony, Doctrine, Guzzle, Assetic, and Twig. Those libraries are included via Composer, the new and ground-breaking PHP-library-management-system-that-actually-finally-works. Code from at least one more should be landing soon.

Drupal developers, working on Drupal, have contributed to a number of other projects, including Symfony and Symfony CMF, and because of the degree of code sharing happening in the PHP world now have indirectly contributed to a half-dozen other major projects, too. Drupal 8, aside from the technological advances it will offer over Drupal 7, also represents perhaps the largest cultural shift in Drupal or PHP history.

Are you ready for 2013, Drupal? Really?

Like many PHP projects, Drupal has been its own island for years. A large, active, vibrant island, but an island. But in 2012, the PHP archipelago began to form into continents. With the bridges built by Composer and by the "open islands" made possible by the PHP Framework Interoperability Group and PSR-0, collaboration and sharing between PHP projects has never been higher.

That represents both a threat to Drupal's traditional island-based culture, but also an incredible opportunity. We have an incredible opportunity to, as Greg Dunlap has put it before, "get off the island" and learn from others outside of our community. No matter how big the Drupal community, the PHP community is larger.

So, I put this challenge to the Drupal community: Make your New Year's Resolution to get off the island in 2013. By that I mean get involved in the wider PHP community, both to learn from it and to share with it. Let's be honest, Drupal has done some pretty amazing things, both technically and as a community, that we can and should share with the rest of the web development world. At the same time, we need enough humility to accept that there are way better ideas floating around out there than exist in Drupal, and we should be open minded enough to learn from them or adopt them wholesale.

In 2013, make it your goal to attend at least two non-Drupal web development conferences. Large, small, doesn't matter. If you can, present at least one of them; maybe about Drupal, maybe not. Connect with people outside of Drupal. Meet fresh faces; expose yourself to new ideas; share your awesome ideas with a wider world than just the followers of Druplicon.

Then come back to Drupal Island wiser, more worldly, and more able to continue to drive Drupal to be the leading CMS on the market.

For my part, I will be attending at least Sunshine PHP in February; I'm still looking for other conferences to speak at this year as well. (Suggestions and invites welcome, of course.) And all Drupalers have a unique and awesome opportunity this May to attend Symfony Live Portland, directly next door to DrupalCon Portland. Kick off your island hopping in Portland with the trifecta of conferences: DrupalCon, Symfony Live, and WebVisions. Attend at least 2 of them, or just do the combo pass. Help build bridges between PHP communities.

Perhaps the world did end in 2012: The world that consisted of only Drupal. That world is indeed gone. The new world of 2013 involves collaboration and sharing across dozens of projects. Let's make Drupal a leader in this brave new world.

Drupal
PHP
DrupalCon
Symfony2

chx (not verified)

31 December 2012 - 12:51am

Permalink

Drupal always did things better. Always. When there was a KSES sechole, every major PHP project was affected but Drupal was not. That's a fact and a hard fact. When there will be a Symfony sechole we will be equally affected. What is the win? I think that we added Symfony because we had no idea on how to write modern OOP code and it's like the lined paper you use in school to learn writing. Now that we have learned we should in Drupal 9 remove as much as we can and replace with leaner, faster and better documented code. Lean and fast were once the Drupal calling words, the words that got me involved and we have completely lost our way and all this code we added doesn't help. Not at all. I hope we can get back on track for D9, finally.

@chx You are right that Drupal developed initially as lean and fast code, that was because the initial development team usually 'rolled their own' so to speak.

However the ensuing success and growth of the project and the community meant that as core developers moved on or burnt out nobody ultimately 'knew' the whole code...this meant that managing ongoing development and encouraging new developers was becoming difficult.

So I reckon that actually getting off an island, using 3rd party products and attracting other developers is healthy for Drupal and so makes the project manageable in the long term. :)

Did Drupal do things better when it adopted a custom PHP based template engine that required graphics designers to become PHP coders?

Did its homegrown approach to XSS prevent the steady stream of security issues that continue to appear?

Did its mixing of router logic and menu navigation do anything but put both CMS UI and site structure in an unnecessary straightjacket?

Was its 'unique' approach to rewriting SQL queries on the fly ever anything but an enormous headache?

Was its 'universal' node system ever a good base to build on, when so many core features avoided it like the plague?

Was "get X into core" ever anything but an enormous undertaking, showing contrib's deficiency as a proving ground?

And above all: was there ever room and flexibility to swap out components to find the optimal solution for each problem, or was everyone unnecessarily forced behind a single solution when multiple were required?

How can you really evolve a lean and fast architecture, when you're limited to extending a monolithic core through a flat namespace of hooks, which requires a political process simply to extend, but a miracle to restructure and reorganize?

It is ironic that a community that so loudly extolled the virtues of open source in reducing maintenance costs, did everything it could to increase its own.

Dig1 (not verified)

31 December 2012 - 5:33am

Permalink

Poignant and thought-provoking...

Well said.

"When there was a KSES sechole, every major PHP project was affected but Drupal was not. "

Jesus man, that was over 4 years ago, is that the only thing you are clinging to? You don't work within core, so ultimately, why is it a huge deal to you? If Drupal makes a change to adopt or leverage something that will bring more developers to the table, what is the harm? That is a force multiplier of programmers who are, or could, or are willing, to work back at core and contrib which seems to be sorely lacking right now after the D7 burnout. I am not trying to make you feel bad because you contribute a lot of good stuff, but at the same time, Drupal needs to be adding coders within its ranks instead of continuously see old guard move on without new people interested in helping it grow.

The more the code is internalized, the more Drupal isolates itself from others being able to contribute in because its not a widely agreed upon pattern, library or API, its just a few peoples thoughts on a nightmare-ish router system or pain in the ass theming system, or half implemented model system (Entity), or a core that makes thousands of PHP calls and hundreds of mysql calls per page load that just don't need to happen. From its contrib, Drupal has received a pass on these major flaws, but eventually (now), all of that needs to be addressed. This is the first step.

It's not about 'making patches' or whatever to 'make it better'. It requires a paradigm shift in thought and architecture. You can leverage open source solutions and still be on top of 0day exploits. Sure it may mean more releases, but is that such a terrible thing? Its Drupal's job to be the best option it can be- its someone elses job to keep their site up to date as required.

I would be interested in seeing Drupal becoming a GitHub project, because it feels too 'close to the chest' on d.org, and it feels much much easier to hop in and contribute/add patch pull request reviews etc there than an issue queue 200+ posts deep per issue.

Anonymous makes a lot of good points but I can't really take them seriously since I don't know where it comes from. Too bad.

No one 'known' in the core community, but left name out to focus on the points and not the person commenting. I have no agenda anyway- I just want to see Drupal get better like everyone else. But the 'Drupal was awesome at D5 because XYZ' style of thinking is just holding it back from evolving. There are always growing pains.

It's not a matter of being "known", your first post (assuming that was the same person) was more or less ok to post as anonymous since it was more objective than not. I'm not a fan of the process but I can live with it.

The second one isn't ok to have as anonymous, you're addressing chx directly in an opinionated way while using "I" to refer to yourself. It is only fair to have everyone know who's involved in a discussion. I respect (and mostly agree with) your opinions but I disagree with how you did it this time around.

And using OAuth Connector module could allow us to authenticate each d.o account with Github so that creating a new project simply created a github repo and linked it on our end. In fact, the project pages could look pretty much the same as they do now, except that data on open issues and links to project browsing would pulls from and redirect to GitHub :)

I spent some time trying to put together a proof of concept, but there's still an issue with OAuth Connector that keeps it from working with GitHub... didn't have time to work through it
https://github.com/patcon/drupalgithub/

Yes, I also feel like it puts it 'out there' - so any passer by who hops in may be apt to offer a fix or suggestion because you have developers from all walks of life cruising GitHub who aren't necessarily ever looking through Drupal.org. Fresh sets of eyes and minds.

I guess I would be a bit careful about pushing Drupal into GitHub. One of the 'post' attractions to me about Drupal has a lot to do with the fact that all the contrib modules are standardized around GPL. I don't want to get into deep legal reasoning at this point, but maintaining that GPL requirement and controlling it through the d.o hosting of the Drupal codebase, is, in my opinion, an extreme strength of Drupal. I believe it is important to preserve that strength.

This is NOT to diminish the technical strengths, community strengths, or any other strengths of Drupal and it's community.

Thanks
Kurt

Totally valid point Kurt, but this can easily be enforced with the contents call in github's Repos API :)
http://developer.github.com/v3/repos/contents/

So we could quite simply "hide" project pages if the LICENSE.txt isn't correct, or we probably even allow push-button license addition/update using the Git Data API:
http://developer.github.com/v3/git/
http://developer.github.com/v3/git/blobs/#create-a-blob

OK, seems pretty simple at least via the API:

# Install pip, python pkg manager
curl https://raw.github.com/pypa/pip/master/contrib/get-pip.py | sudo python
# Install httpie, curl for humans
sudo pip install httpie
# Get the tree objects of the 7.x branch of drupal repo
http GET https://api.github.com/repos/drupal/drupal/git/trees/7.x
# Use the blob hash of LICENSE.txt from the previous results, with Accept header for plaintext
http GET https://api.github.com/repos/drupal/drupal/git/blobs/94fb84639c4b6ff359e47a124d8fb4a3aba7a386 Accept:application/vnd.github-blob.raw --body

Welcome to FOSS. If you find a fault in the libraries you use, contribute to fixing them. Please. If you are not interested in that, perhaps it's time to re-evaluate what FOSS means to you. Sharing code freely (as in speech) is powerful. It is a dangerous attitude to think we should simply peak out, learn a few things, and duck back in. Please, reconsider. What if you are the one who could solve the next major Symfony issue? You would be helping D8 and who else? A massive community of developers. Perhaps your solution will be a pattern which can be applied at an even greater scale. The entire web community could benefit from your exposure to the issue and your willingness to contribute solutions in *that* scope.

Is Drupal truly FOSS? Can we participate in a larger community? Same question?

I think that to say Drupal always did things better is a bit of rose-tinted-glasses hindsight mixed with a little bit of hubris.

I just spent the whole morning updating several D6 sites to 6.27 because of a sechole in 6.26. Secholes exist and they will always exist in software and to think that Drupal is somehow above that is just not true.

Furthermore, Drupal's raison d'etre as an open source project is to leverage the efforts of others so as not to have to reinvent the wheel yourself every time you need a feature in a website. I think opening the gates of the Drupal world a bit to Symfony was the smartest move that Drupal as a community has made in a long time, and has got me very excited to work with and contribute to D8. If we've found great code by another great community out there that is willing to bend over backwards to make it work within Drupal, why on earth would we say "no thanks"?

Lastly - my personal hope for D9 is that we'll stop ripping half of the API out by the roots with every major release. If the biggest headache for Drupal's adoption in the market is lack of Drupal talent (http://denver2012.drupal.org/keynote/dries-buytaert), then stop yanking the rug out from under everyone every two years.

I fully agree with not rewriting the entire system from scratch every version. However, that requires first shifting Drupal to an architecture that allows us to do that. Backward compatibility is a feature that must be planned for in advance.

That said, one of the reasons I'm pushing such invasive changes for Drupal 8 is that they should make it easier for us to slow down API breakage without greatly slowing down feature innovation, assuming we're willing to do that. The big battle for Drupal 9 will be to solidify the gains we've made, finish the job (lots of conversions will only be part way through, I fear), but not rearchitect the whole system again. That will be fun times. :-)

I have to be honest that I was doubtful at first, but that's my natural reaction to almost everything in life :)

Seeing all the changes that happened so far, I'm getting more and more excited and it's reopening my eyes on the PHP world in general as I can say I got kind of 'stuck' and missing the OO goodies that slowly got into PHP while working with full time Drupal almost 7 years now. Not that I regret it, I was lucky having other (although smaller) projects in much more mature languages during this time.

Let's compare it to cars: Drupal 8 is going to be like switching from a two-stroke engine to a V12 Rolls. That's maybe exaggerating as out of the box D8 won't look *that* fancy, but under the hood, on my .. :)

Thank you for driving this forward and everyone involved so far!

Very well said, Larry! I began doing just what you say last year and had my eyes opened and my horizons broadened by the good folks at the PHP Poland conference. The first event on my calendar for this year is the PHP BeNeLux Conference (http://www.phpbenelux.eu/) in Antwerp, January 25 & 26.

We've done some things well so far in Drupal-Land, but we don't have a monopoly on smart people. Every time we can learn something new, or debate why particular technical decisions were made, or see how others solve similar problems is a chance for all of our projects to do better. Cross-pollination makes for stronger plants!

It's no surprise that you feel this way, as you've been great at getting off the island in recent years, but it's really awesome to see this post, and see that it's something you take so seriously. Sticking my head out of Drupal Narnia is the #1 best thing I've done in the past 2 years. Sometimes it's made me feel like I have super-powers, because it's historically been a pretty rare thing to do within the community, but I just know people are warming up to it :)

Anyhow, if you're looking for another conference, I HIGHLY recommend devopsdays. And lookee here -- there just so happens to be one happening in NYC in a few weeks. I've been to a few and they're always humbling and inspiring :)
http://devopsdays.org/events/2012-newyork/

Seriously, let me know if you're going and that might be the thing that puts me over the edge and gets me to NYC :)
@patconnolly

Oh hey, was talking really informally with @KrisBuytaert about hosting one in Toronto. If I can find some folks to follow through with, I would definitely be dogging you to adventure up here :)

Because it's really hard to get people really get involved in the Drupal community in Vietnam (code contribution and presenting wise) our team at Geekpolis contributed a lot to the organization of the national PHP Day in Vietnam this year and gave 2 presentations that showcases Drupal's power easy to understand for PHP people. We also built the event site with COD and received some big compliments from the crowd.

If the community is an island and a very small one here in Vietnam, you don't have that many to talk with anyway so the best thing to do is to dive into other projects/communities to learn from their cool tricks and evangelize the Drupal thing. We are joining local code dojo sessions (similar to code retreat) and get in contact with even more people, JAVA, .NET and what not, and do also learn a lot from them as well.

The next thing we will do is to reach out to the LINUX freaks, talk with them and check how overlapping interests can get us closer to them. The more flexible and agile you are the more you get out of it and this should make us strong, not because we are the top of the pop (CMS) kings. We have probably the most open-minded spirit already but still only when we receive people. We need to go and give to others too, even they don't need it (yet). :)

It's indeed a two way exchange all the time.
[Drupal] %anything% (for and/or from) [others].

nixonparker (not verified)

5 February 2013 - 5:48am

Permalink

Drupal is a platform for the website development. It has improved many criteria which are really good to use in website.