gistfile1.txt

Problem/Motivation restated via user stories
If I download a distro from drupal.org, or if I write make files to accompany an install profile according to drupal.org's guidelines, and then I build my code base using the build-profile.make file I end up with a directory structure that looks like this:

docroot/profiles/my-profile/modules/contrib/views
docroot/profiles/my-profile/modules/contrib/ctools
docroot/profiles/my-profile/modules/contrib/some-contrib-project-i-maintain
docroot/profiles/my-profile/modules/contrib/etc.
docroot/sites/all/modules/ <-- empty

This is fine for someone who downloads a distro from d.o, installs it, and never contributes. But for maintainers, and potential contributors, this directory structure creates an huge amount of unnecessary friction.

As a distro maintainer, I organize my distro the way drupal.org suggests I should. I eat my own dogfood, running an instance of my own distro as a production website. The site lives in a single git repository. When I do work on my own site (my own instance of my distro), getting changes out of my site repo and into either my install profile repo or any related contrib projects I maintain is a big headache. Common Git tricks like putting a whole project repo inside my site repo get a lot more complicated when working with an install profile. Git has a few standard solutions to problems like mine: git-subtree, git subtree merge strategy, git subtmodules. But none of these solutions is a good fit for me when contrib project directories and stored inside my install profile, because I don't want any of these files to get tangled up in my stand-alone install profile repo. As a result, I have to put a lot of thought work into inventing a hacky workaround, my workaround inevitably involves a number of time consuming steps, which all make it costly and frustrating to contribute work from my personal project back to the community through the distro I maintain.

As a potential contributor to a distro I use, I have similar problems to the ones described above. It's confusing and difficult for me to do development work on my own instance of the distro and then figure out how to roll patches or push my changes up to a drupal.org sandbox, with out getting my changes tangled up with stuff that should not be included in the install profile repo.

I've been building distros and maintaining sites running on distros for five years. I know these problems are not insurmountable. But these obstacles for maintainers, contributors, and the product owners who pay for their work (hopefully) are real and costly.

Solution
The problems described above evaporate when we organize distros just slightly differently, like this:

docroot/profiles/my-profile/modules/ <-- only contains custom modules that are not stand-alone projects, which live inside install profile repo
docroot/sites/all/modules/contrib/ctools
docroot/sites/all/modules/contrib/some-contrib-project-i-maintain
docroot/sites/all/modules/contrib/etc.

Or, a slight variation I've come to prefer is this:

docroot/profiles/my-profile/modules/ <-- only contains custom modules that are not stand-alone projects, which live inside install profile repo
docroot/sites/all/modules/my-profile/ctools
docroot/sites/all/modules/my-profile/some-contrib-project-i-maintain
docroot/sites/all/modules/my-profile/etc.
docroot/sites/all/modules/contrib/ <-- not polluted by distro, available for site maintainers to put instance-specific contrib stuff here

Organizing a repo ^^ this way makes it trivial for maintainers and contributors to work on install profiles the way we do with modules and themes. If I want to work on my project by dropping a full project repo inside my site repo, I can do this, whether I'm working on my install profile or my contrib module. And if I want to use a more standard Git approach--submodule, git-subtree, or subtree merge--I can do that easily too. I don't have to do anything strange or different when I'm doing development on and install profile to prevent getting other projects' files tangled up in my commit history or patches.

Possible resolutions
Here are four possible resolutions for drupal.org's distro builder. (I'm sure other people can think of more.)

1. Enable distro / install profile maintainers to put everything into build-my-profile.make. Then update d.o's packager to package up a distro like this:
`drush5 make build-my-profile.make built-distro`. (My least favorite option, introduces other anti-patterns and funny details.)

2. Enable maintainers to provide a top-level make file and use Drush 7's --no-recursion flag (requires backport of no-recursion flag). Build.make files would look like this:
includes[base] = build-myprofile.make
includes[contrib] = drupal-org.make

Builds could be done like this:
`drush5 make build.make built-distro --no-recursion`

--no-recursion obviously creates problems if a contrib module includes a make file. This isn't insurmountable, the make file can be explicitly included in build.make. But it's not ideal.

3. Enable install profiles to leverage Drush 7's contrib_destination property in drupal-org.make.

4. Change the default handling of drupal-org.make so it defaults to putting included modules and themes in sites/all.

Proposed resolution
#3, supporting contrib_destination seems like the path of least resistance. It's an elegant Drush Make-based solution. It's supported in Drush 7.x. And it doesn't force anybody to change what they're doing or how they organize their distros without knowing what's going on.

If we could go back in time, I'd suggest #4. But now that the current pattern has been established, I think this would cause as much confusion and frustration as we're eliminating.