As Drupal is in the process of considering how to restructure code to best leverage the PSR-0 standard, I figured it would be wise to take a quick survey of how some other major projects organize their code bases. This is not a complete rundown of every project, simply roughly comparable notes for those areas Drupal is currently discussing. I am posting it here in the hopes that it will be useful to more than just Drupal.
Note: This is based on one evening's work of poking around. If you work with one of these projects and have more information to provide or want to correct a mistake I made, please do so in the comments!
CakePHP
http://cakephp.org/
http://book.cakephp.org/2.0/en/getting-started/cakephp-folder-structure…
Cake is not a PHP 5.3 system, but its classes appear to be structured in a very PSR-0-friendly way.
Cake has a top level "lib" directory that is "we wrote this, go away" (similar to /core for Drupal). There's another top-level "vendor" directory that is explicitly for 3rd party stuff. There is also a second Vendor directory for app-specific third party stuff. It looks like that's similar to sites/all/modules vs. sites/default/modules, as Cake can host multiple apps on one install.
Cake stores its unit tests in the src/Cake/Test directory. There is an app/Test directory pre-created to hold application-specific tests.
Symfony2
http://symfony.com/
http://symfony.com/doc/current/glossary.html#term-vendor
http://symfony.com/doc/current/cookbook/workflow/new_project_git.html
http://symfony.com/doc/current/cookbook/bundles/best_practices.html#tes…
Symfony-namespaced code lives in /src. Non-Symfony-namespaced code that ships with Symfony is placed in /vendor. Curiously, it is not included in the GitHub repository nor are there git submodules. Instead, there is a vendors.php command line script that git clones the dependent repositories. Essentially it's a manual git submodule setup. I do not understand why.
The documentation (links above) suggests that users should install their own 3rd party dependencies in /vendor as well. I... really don't think that's a good idea. :-)
Symfony namespaces its tests to Symfony\Tests (note the plural), but stores them in a /tests directory, not in /src where Symfony itself goes. They are in a mirroring namespace tree to the code being tested.
Bundles (the closest equivalent to Drupal modules) ship with their own tests in a Tests directory/namespace, below the directory/namespace of the bundle.
Silex
Silex has a /src directory that contains Silex itself, the Silex-namespaced classes. Any non-Silex namespaced code (Pimple, Symfony2 components, Doctrine, Monolog, etc. It's actually a very similar list to Symfony the framework.) lives in /vendor. In the development repository they are referenced as git submodules.
Silext namespaces its tests to Silex\Tests (note the plural), but stores them in a /tests directory, not in /src where Silex itself goes.
CodeIgnighter
CodeIgnighter is not a PHP 5.3 PSR-0 framework, so its code is not organized for it. I mention it anyway for completeness.
At the top level are an /application directory for a particular install and a /system directory for CI-provided code. The /system directory appears to be clustered in a way that would likely fit a PSR-0 model fairly easily and contains almost exclusively class-per-file files.
The /application directory is setup as a skeleton of a working application. There is even a directory called third_party, the use of which I presume is as one would expect.
I can't seem to find unit tests in the main repository. There's a custom base class for one, but there doesn't appear to be an obvious place to put system-provided tests nor app-specific tests. Odd. Perhaps I'm missing something.
Zend Framework 2
https://github.com/zendframework/zf2
Zend-namespaced code lives under /library (singular). There's a lot of it. :-) There is also a /modules directory that appears to be for additional 3rd party code components, which have their own /library directory that restarts at a Zend\ namespace. (So /modules/ZendFramework1Mvc/library/Zend/[lots of stuff].)
It's not clear from the code repository if there is a canonical place to put 3rd party libraries that are not Zend modules.
The main framework, as well as each module, has a tests/Zend directory that contains a huge number of test classes, mirroring the path and namespace of the class being tested.
Composer
http://packagist.org/
http://nelm.io/blog/2011/12/composer-part-1-what-why/
Composer isn't a framework project. It's an application to automate managing dependencies. Although its documentation only hints at it, when you tell composer to resolve and download dependencies for you it will do so and place them all into a vendor/ directory. That does not appear to be configurable.
However, that directory is not the direct root of each package's class tree but the root of the package's download. The exact location within vendor/ where each class tree starts varies with the package. Composer provides its own class loader that has all of that sorted out already for you, but if you have your own class loader then you have to work that out yourself.
Summary
Project | own-namespace | external-namespace | 3rd party | tests |
---|---|---|---|---|
CakePHP | /lib | N/A | /vendor | src/Cake/Test, app/Test |
Symfony2 | /src | /vendor | /vendor | /tests |
Silex | /src | /vendor | /vendor | /tests |
CodeIgnighter | /system | N/A | /application/third_party | N/A |
Zend Framework 2 | /library | N/A | /modules (seems to require placeholder module?) | /tests, module/X/tests |
And Composer puts everything in /vendor.
namespacing
I like Kohana (Which is Kohana based), but think it should be /core. Less typing.
For the love of sanity, I would not use PHP Namespace scheme. Over the time it has been introduced, the more I think it was a bad idea. More typing and more chances of collisions. A decent directory structure and decent auto loader will go a long way.
USE \Mylib\Xyz,
\Monkey\Xyz as MonkeyXyz;
$xyz = new Xyz();
$mxyz = new MonkeyXyz();
Proper directory structure and autoloader:
$xyz = new mylib_xyz();
$mxyz = new monkey_xyz();
/$0.02
Namespacing
Well, a decent directory sturcture and decent autoloader is exactly the point of namespaces, and PSR-0 in particular. :-)
Drupal has already decided to adopt PSR-0 namespaces. The question now is "so what do we do with it", essentially.
Kohana doesn't appear to be a PSR-0 framework, and I had enough non-PSR0 frameworks already. :-) That's why I didn't include it.
Yeah - It was a total free
Yeah - It was a total free form somewhat related comment. =)
Basically I would do /core for "core" code =)
er free = free form.
er free = free form.
Underscores are a ridiculous
Underscores are a ridiculous choice when namespaces are available. If you don't want to alias classes, just use the qualified names:
<?php
$xyz = new \MyLib\Xyz();
$mxyz = new \Monkey\Xyz();
?>
No need for stupid underscores, and no need to use aliases if you don't like them.
It works exactly like your "proper" example, doesn't have underscores, and will work OUT OF THE BOX with spl_autoload()
So no having to use the 'USE'
So no having to use the 'USE' statement?
The USE statement is just so
The USE statement is just so you can make an Alias to a Class, it's not mandatory.
I suggest you read http://php.net/manual/en/language.namespaces.importing.php
"users should install their
"users should install their own 3rd party dependencies in /vendor as well. I... really don't think that's a good idea. :-)"
It's common practice (see GNU/Linux for example) and it's only normal way, because otherwise you will need to store all dependencies in your repository and check each minute, if one of them was not updated.
composer dependency manager
composer dependency manager an sf2 boundels(self contained apps) seems to me the best tools available, becouse relying on an automation tool and not only on conventions ensure better consinstency
From documentation it seems
From documentation it seems that bundle looks a lot like our Drupal module.
I realise it's part of the
I realise it's part of the PSR-0 spec, but "vendor" implies something has been sold. How does that sit with free, open-source software? Or do these terms not have to relate precisely to English definitions?
PPI Framework
Larry asked me to create a comment on this blog post.
PPI Framework
http://www.ppi.io
PPI is a web application framework that's fully namespaced, PSR-0 compliant and running on PHP5.3+
It's code base is built upon libraries such as Symfony2 and Doctrine2, so by default it has great Vendor support.
The framework is in PPI/ folder
The tests are in the PPI/Test/ folder
The vendor libraries are in the Vendor/ folder.
The skeleton application glues all this together in a very simple and accessible manner.
Thanks,
- Paul
The vendor dir is
The vendor dir is configurable with Composer. Just add this to your composer.json:
{
"config": {
"vendor-dir": "some-other-vendors"
}
}
Also, if the vendor package you are installing defines and bin scripts, you can customize where those are installed
{
"config": {
"bin-dir": "bin"
}
}
Composer corrections
You can configure where composer stores libraries it installs by default:
"config": {
"vendor-dir": "custom/path/for/vendor"
}
See https://github.com/composer/composer/blob/master/README.md
We don't recommend changing it, to keep things familiar across projects, but it is possible. Custom installers allow you to install some types of packages in a different location than others, if that is desired.
Furthermore composer does not only generate an autoloader but also a namespace map (a simple array). So if you want to use your own autoloader you can still use that array to get all the information about namespaces and directories installed by composer.
Since composer is meant for third party code only, any directory format for your own code (src/, lib/, tests/ etc.) works fine with composer. You can actually add any such directory to composer's automatically generated autoloader rather easily as well.
PEAR2
PEAR2 uses src/ for own code, nothing for 3rd party code (since it's only self-contained components) and tests/ for unit tests
Symfony2
I believe you misunderstood the way Symfony2 is handled. The Symfony2 github repository only contains the "library" code, and that sits in src/ as you pointed out. However, when you want to create a Symfony project, you use https://github.com/symfony/symfony-standard or other "distribution". Those distributions are what is really more comparable to Drupal or most other frameworks. It is the application shell in which you put your code (in src/), and it contains the Symfony library in vendor/ as well as all other third-party code.
As far as your application is concerned, Symfony is an implementation detail, not your code, there is no reason it should be treated differently than other vendors. That is also why Composer puts everything in vendor/, because you should ideally never have to worry about this stuff and go dig into it.
Exactly! The author didn't
Exactly! The author didn't get the essential of Symfony2 and Composer, they introduced a open standard for package management, rather than build an isolate kingdom of it's own, like every other php framework ever did.
I think it's a great thing for php community, one thing that every good language should have (Rails' bundler and rvm, Python's easy_install and virtualenv, Nodejs' npm, etc. )
Agreed
I agree that a more viable shared package mechanism is a good thing for PHP. We're trying to coax Drupal toward at least less of an isolated kingdom. It's an uphill battle, as it would be for any project of that size. The goal with this post is to just get a "casual feel" for what is happening outside the castle walls.
It looks like /src and /vendor are the clear conventions, informal though the may be. I didn't want to go into whether Symfony2 or Composer specifically are as magically awesome as you claim. :-) (Although I kinda like both of them so far.)
Aura Project for PHP 5.4
The Aura project provides independent library packages for PHP 5.4+. Aura is not a new one, but the second major rewrite of the Solar PHP Project by Paul M Jones ( The father of PHP Framework Bench Marks https://github.com/pmjones/php-framework-benchmarks ) .
The Aura packages can be used standalone, in concert with each other, or combined into a full-stack framework of their own.
The Aura project also includes a system that composes the independent packages into a full-stack framework. All the Aura packages resides in /package folder .
All the packages are unit tested and aims for 100 % code coverage, it has its own tests folder . Aura follows the psr-0 standard and all the source resides in the src folder with the psr-0 standard.
Currently the third party libraries can be kept in the include folder of the system. As all are using the term vendor I have asked @pmjones to look whether we need a change in the name, so the community has a common name for libraries outside it.
Project own-namespace external-namespace 3rd party tests
Aura /src /include /include /tests
Source Code : https://github.com/auraphp
Website : http://auraphp.github.com/
IRC : #auraphp
Groups : https://groups.google.com/forum/#!forum/auraphp
Version : Beta 1.0
Hello
The Aura project provides independent library packages for PHP 5.4+. Aura is not a new one, but the second major rewrite of the Solar PHP Project by Paul M Jones ( The father of PHP Framework Bench Marks https://github.com/pmjones/php-framework-benchmarks ) .
The Aura packages can be used standalone, in concert with each other, or combined into a full-stack framework of their own.
Try Onion
Try Onion, which is PEAR-compatible bundler (builder), pretty simple to use. :-)
Check here:
https://github.com/c9s/Onion
Good one
Yes Onion is a Good one to try .
Please actually talk w/developers from the projects you survey
Caveat: I'm project lead for Zend Framework.
It's entirely clear from reading your analysis of ZF2's project structure that you:
I'll clarify the structure for you, but I have to wonder how well you evaluated other projects, and which projects had folks you were already collaborating with. If you're going to do any sort of honest evaluation, you should definitely state where you're getting your information, and also definitely state how you tried to get in contact with developers from the project -- this ensures an open and transparent process, but also serves as good information to the developers of projects, as they then know how others try and reach out to them, and whether or not they ultimately succeeded.
Also, editorial comments about the relative sizes of projects should be kept to a minimum -- size of a library -- be it small or large -- should not be the consideration. What they offer you and your project is what is of interest here.
Now, as for ZF2 structure...
Suggested project structure is as follows:
Third party code should go under the "vendor" directory. There are no assumptions made about how code in that directory is installed, and the assumption instead is that you will instantiate appropriate autoloaders in your bootstrap to load this code. In terms of the Zend Framework 2 distribution itself, you would install it as you would any other 3rd party library -- and thus under vendor/.
Modules are the bits and pieces you assemble to create your application. These may be anything -- library code, public assets, or MVC code. MVC code will always be delivered via modules. Our recommended module directory structure looks like this:
The "Module.php" should contain a class named "Module" under the {ModuleName} namespace. The ZF2 module manager will consume this to expose the module to the application. However, there is nothing saying that a module must contain MVC code or be specific to the ZF2 MVC layer; this is simply another way to distribute code.
Typically PHP source code, under {ModuleName}/src/, should follow PSR-0 naming conventions, allowing you to use a PSR-0-compliant autoloader. (We actually also recommend shipping a classmap file and/or a file containing a callback for use with spl_autoload_register() to make the module easy to consume in non-ZF2 applications.)
As such, 3rd-party modules will be located under vendor/, while the modules you write specifically for your application will be under module/.
The "modules" directory you found in the ZF2 distribution is a set of code we've removed from the core ZF library and which we intend to distribute as stand-alone ZF2 modules. One of which, the ZendFramework1Mvc module, contains the previous ZF MVC implementation, which we've removed from core in order not to confuse new developers, but retained as a module to provide a potential migration path for developers of ZF1 applications. This module follows the same conventions for modules that I outliend above.
See the intro
Matthew, as I said in the beginning of the post this was not a formal rundown or analysis, just my notes from casually poking through websites and git for about 3 hours. It was not intended to be anything more than that, and I wasn't trying to "privilege" any particular project. I was just looking for trends. You're right, I didn't really talk to devs from any of the projects, although I've been talking to Symfony folks lately since Drupal is adopting some Symfony2 components. That said, thank you for the more detailed explanation of how Zend 2 is organized.
It looks like you're using a similar model to what Symfony2 does, with some of the same terms used. It's actually a bit closer to what Drupal may end up using, although more structured.
For the code under src, assuming it's PSR-0, do you just live with the extra seemingly redundant directories that can result, or do you have some other solution to it? Also, is there any expectation that the Zend namespace is used for anything other than official, "core" modules?
Map-based autoloader
The amount of attention devoted to this is ridiculous. And pressuring a diverse project structure as that of Drupal into a PEAR scheme will of course incur some drama. Is that really worth the buzzword compliance?
Why not use an automap-based autoloader? That's most independent from the discrepancies in directory structures. It might need some professional coding to get progressive updates for directory scans, but the autoloading lookup itself is heaps speedier than with any sort-of rigid directory traversal scheme.
Have one
Drupal 7 has a map-based autoloader. It's far too brittle in practice, and has a dependency on the database to work properly. (That's where the map is stored.) We wan't to address both of those issues, and try to more broadly adopt emerging PHP standards and conventions.
map-base -> cache
Whatever we do, we will still probably end up with a "map-based" autoload cache, for performance reasons. This cache will probably not be stored in the database, though.
The real reason for all this talk, imo, is to agree on standards for how classes should be named and namespaced, and where they should live.
PHP Frameworks (and tools
PHP Frameworks (and tools like composer) are not the only thing using a vendor directory. For example, Ruby on Rails uses a vendor directory as a place for 3rd party code. Using the vendor directory is fairly common and if Drupal continues to deviate from that it would be an exception. Cake, Symfony2, Silex, ZF2, and Rails are all examples here using the vendor directory. Is there a good reason not to use the vendor top level directory name? Ya know, aside from how we handle things (modules/themes) where one version could be in profiles, sites/all, or sites/foo and this would deviate from that pattern (which I'm ok with).
Something to think about with these frameworks is that the entire project isn't always supposed to live within your sites web root. For example, with symfony2 you will often point the web root at the web directory inside a symfony2 install. The src directory is not web accessible. Distinctions like this are important to understand when making our naming conventions.