Drupal 7 battle plans

So, Dries wants to know what our Drupal 7 battle plans are. I think this is the first version where I'll have explicit battle plans before hand rather than just "whatever I come up with along the way". :-) So, for those playing along at home, here's my goals for Drupal 7:

• Move Drupal to PDO
• Introduce a function registry
• Begin to solve the "Data API problem"
• Whatever I come up with along the way

Move Drupal to PDO

This should come as no surprise to anyone, especially given previous blog entries and DrupalCon presentations I've given. :-) There is little doubt at this point that PDO is the way forward for PHP database handling in general. It's not perfect, and there are stil some inconsistencies between different databases (most notably with Oracle and LOB handling), but it's a much cleaner interface than the many disparate APIs that came before it. It also offers a number of nifty features that people have either been asking for or that we could find use for.

Work on this front is about 50% complete already. I need to refactor a few things yet for easier maintenance, and then get to work on Postgres and SQLite drivers for a PDO-based extension. How these changes with affect the Schema API I am not sure yet, and that's one thing I plan to discuss with Barry Jaspan next week as part of the Data API pow-wow/sprint Palantir is hosting.

And before anyone asks, Oracle is not on my immediate radar for two key reasons: 1) It's a difficult system to support, because it requires different field handling than other database systems; 2) It's not Free/free, therefore I have no way of testing against it in the first place. My hope is to be able to offer an API behind which someone else can implement Oracle support.

Introduce a function registry

I've bounced this idea off of a few other developers, and the response so far has been positive. In Drupal 6, we introduced the concept of menu handler files, where functions called from a menu handler could live in a separate file that is included conditionally only when needed, thus decreasing the amount of code to be parsed on any given page load. (We later added it to theme functions, too.)

The next step is to remove the manual declaration step. PHP is capable of parsing itself, so we can have Drupal introspect itself and build up a list of all functions or classes that are called indirectly. We can then lazy-load anything that is called indirectly: page handlers, theme functions, forms, nearly any hook, nearly any alter-function, any class or interface (which PHP 5 supports properly now, recall)... almost all of Drupal. Think of it as the love child of Drupal 6 menu-split and Karoly's old "split mode" concept.

With proper code organization, then, we can eliminate most of the bootstrap phase of a Drupal page request, which is well-documented (Yay, GHOP!) as being the slowest part of the request lifecycle because we're loading so much code that we never actually use. As a nice bonus, we also get a fully-dynamic registry of what modules implement what hooks; there's all sorts of things we could do once we know that.

I already have working proof-of-concept code in my sandbox, implemented as a module. The only catch at the moment is that it suffers from the same "death by a thousand small queries" problem as the path module currently does. Feedback on how to solve that issue is most welcome.

The Data API problem

I am far from the first person to want to tackle this issue. It's well-accepted that we need to somehow clean up our "entity systems": Nodes, Users, Files, Comments, etc. The APIs are wildly inconsistent and in current incarnations are very fragile, slow, and too-tightly-coupled to other systems like the Form API.

I have no expectation of being able to solve this challenge alone, of course. That's why next week several Drupal developers will be camping out in the conference room at Palantir.net here in Chicago to see if seven heads are better than one in establishing long-term battle plans for data handling. The guest list includes Larry Garfield, Barry Jaspan, Karoly Negyesi, Karen Stevenson, Yves Chedmois, Moshe Weitzman, and Nedjo Rogers. That is, six really smart luminaries of the Drupal world plus me. :-) Afterward, Karoly and I will be spending some quality time together to try and hash out code along whatever lines we come up with at the data-sprint.

Expect a full run-down on what we're able to accomplish (if anything) in Boston. You are going to be there, right?

Whatever I come up with along the way

On the off chance I actually have time to think about anything else, I'd also like to see about getting more fine-grained filter control into core. In nearly every case, I want filters to apply to only certain node types. Right now I'm solving that with a very touchy module I wrote called filter by node type, but various other people have similar modules floating about contrib. That to me is a sure sign that such functionality belongs in core. Ideally, every textarea on a node, including the body as well as any CCK textfields, could specify a limited set of input formats that it supports. If anyone else wants to pick up this issue, please do so! I won't be offended, trust me. :-)

What else? I'll figure it out once I use Drupal 6 enough to know what annoys me so that I can fix it.

Drupal 6 is going to rock. It's going to rock so much that it will be really hard to make Drupal 7 rock even more, and that's going to take the combined effort of over 1000 Drupal developers.

I love a challenge...