Software

Thanks to the work of the Drupal security team, we released Drupal Commerce 7.x-1.10 on September 10 to address an information disclosure vulnerability. Last week we released a companion module to that update, Commerce Username Update, to help administrators manage the username update the release requires. The new version also includes a handful of minor bug fixes and a new feature to better support free order notifications on the checkout form.

Read more to learn more about the patched vulnerability and new feature.

Migrating from major version to major version of Drupal core has always been a significantly large task for all but the simplest sites. The upgrade path that has traditionally been part of Drupal core has always been limited in what it can do, so most sites were forced to use alternative methods to migrate configuration and content. Sometimes these migrations were manual, sometimes automated, and most often a combination of the two.

Drupal 8 aims to greatly reduce the friction of migrating sites from Drupal 6 and Drupal 7 by adopting a proven and extensible approach to site migrations. The Migrate module has been the go-to tool for migrating a large number of sites to Drupal 7 from earlier versions of Drupal as well as from other content management systems (including custom ones.)

This blog post aims to provide an overview of how the migration system in Drupal 8 works, our current progress, and how new contributors can get involved. The Migrate in Core initiative began in earnest about a year ago at DrupalCon Prague, when it was decided to use some code and concepts from the Migrate and Drupal-to-Drupal Data Migration modules as a starting point for a new and improved upgrade path.

At the current time, the Drupal 6 to Drupal 8 migration is almost complete, while the Drupal 7 to Drupal 8 migration is just getting started. There are a few blocking issues that we're trying to get past in the next couple of weeks (including files migration and link field migration). We feel that we'll be able to leverage much of the work we've done on the Drupal 6 to Drupal 8 migration for the Drupal 7 to Drupal 8 migration. In fact, we have a great issue for a new contributor to help us kick of the Drupal 7 work just waiting for someone to tackle.

-->

read more

It's the community behind the open source Drupal Content Management project that really sets it apart.  Thousands of developers around the world take part in building Drupal, and building the web with Drupal. This post peers behind the scenes at what makes the community tick.

Topic: DrupalLocation: Japan

I spent the last week in Japan. The goal was two-fold: meet with the Drupal community to understand how we can grow Drupal in Japan, and evaluate the business opportunity to incorporate an Acquia subsidiary in Japan (we already offer Acquia Cloud in Japan using Amazon's Tokyo data center).

I presented at two Drupal meetups in Japan; spent the week meeting with members of the Drupal community, Drupal agencies, large system integrators (IBM, Accenture, Hitachi, Fujitsu, Ci&T and SIOS) and the Japanese government. In between meetings, I enjoyed the amazing food that Japan has to offer.

The community in Japan is healthy; there are some noteworthy Japanese Drupal sites and there are passionate leaders that organize meetups and conferences. The Japanese Drupal community is bigger than the Chinese Drupal community but compared to North America and Europe, the Japanese Drupal community is relatively small; the largest Drupal agency I met with employs 20 developers.

The large system integrators, with the exception of Ci&T, have not done any Drupal projects in Japan. We're way behind our competitors like Sitecore, Adobe Experience Manager and SDL in this regard. All of them enabled the large system integrators to sell and use their products. It was great to meet with all the system integrators to make them aware of Drupal, and the potential it could have to their business. It's clear the large system integrators could benefit from an Open Source platform that allows them to move faster and integrate with more systems.

The biggest challenge is the lack of Japanese documentation; both marketing materials as well as developer documentation. Most of the Japanese do not have much confidence in their English speaking ability and struggle to use Drupal or to participate on drupal.org. My recommendation for the Japanese Drupal community is to organize regular translation sprints. Translating one or more of the best-selling English Drupal books to Japanese could also be a game-changer for the community.

Another problem has been the historic challenges with drupal.jp. The anonymous owner of the domain drupal.jp claims that drupal.jp is the official Drupal site in Japan (it's not officially approved) and runs it without much regard or consultation with the broader Japanese Drupal community. I promised the Japanese community to help fix this.

I returned from my trip feeling that the Japanese market offers a great opportunity for Drupal. Japan is the world's third-largest economy, after the United States and China. With continued leadership, Drupal could be huge in Japan. I’d love that, as I would like to go back and visit Japan again.

You want to learn HTML and CSS, or maybe you just need a refresher on the current state of web technology—where should you start? This is a question we get asked a lot at Drupalize.Me. Our theming and module development videos often assume that you're familiar with basic HTML and CSS. But not everyone is, and you've got to start somewhere. At the moment, creating HTML and CSS vidoes is low on our list of priorities. We're not saying we'll never do it, but for now we serve our members best by rocking Drupal-specific content. And there's already a lot of great resources available for HTML and CSS. I thought I'd share some of my favorites here. I also tapped the Lullabot "hivemind" for additional recommendations.

The most insecure part of a Drupal website is typically the theme. Drupal 8 is using Twig as its template layer. This is a massive leap forward. It's no longer possible to put SQL queries in a template file!

Furthermore, Drupal 8 is now taking advantage of a security feature of Twig: autoescape. Every variable in a Twig template will be escaped if it is not marked as safe. This makes it much harder to introduce XSS vulnerabilities.

Unfortunately any HTML that your module produces will end up being double-escaped and the HTML itself will be visible instead of the browser's rendering of it. The quick and dirty way to fix this problem is to wrap your string with \Drupal\Component\Utility\SafeMarkup::set:

<?php
$output = SafeMarkup::set('<div class="my-module">' . $my_variable . '<div>');
?>

But this defeats the whole point of using autoescape. The correct approach is that all HTML created by a module should be declared in a Twig template. This means that all the variables are guaranteed to be escaped. It is also very easy to implement.

First you need to declare the template in your hook_theme():

<?php
function my_module_theme(array $existing, $type, $theme, $path) {
  return array(
    'my_module_my_template' => array(
      'template' => 'my-template',
      'variables' => array(
        'variable1' => NULL,
        'variable2' => NULL,
      ),
    ),
  );
}
?>

You then need to create a Twig template file, for example my_module/templates/my-template.html.twig:

{#
/**
* @file
* Default theme implementation for my template.
*
* Available variables
* - variable1: The first variable.
* - variable2: The second variable.
*/
#}
<div class="my-template">
  This is the first variable: <b>{{ variable1 }}</b>.
  This is the second variable: <i>{{ variable2 }}</i>.
</div>

When you need to generate your html you should use the drupal_render() function:

<?php
$render = array(
  '#theme' => 'my_module_my_template',
  '#variable1' => t("First"),
  '#variable2' => t("Second"),
);

$output = drupal_render($render);
?>

Strings returned by drupal_render() are automatically marked as safe and will not be escaped again.

See why global non-profit organization Teach For All chose our osCaddie solution var switchTo5x = false;stLight.options({"publisher":"dr-75626d0b-d9b4-2fdb-6d29-1a20f61d683"});

As of 06:58 UTC today, September 19, there are zero Drupal 8 beta blockers. This means that, after more than nine months of focused effort, we are almost ready to release the first Drupal 8 beta!

When will Drupal 8.0.0-beta 1 be released?

Today (September 19), we have released one more Drupal 8 alpha, Drupal 8.0.0-alpha15. This alpha can be treated as a "beta release candidate". If no additional beta blockers are identified in the next 10-14 days, we will then tag the first beta! (If we do discover additional beta blockers, then we will evaluate them and adjust our timeline.)

What does beta mean?

Betas are good testing targets for developers and site builders who are comfortable reporting (and where possible, fixing) their own bugs, and who are prepared to rebuild their test sites from scratch if necessary. Beta releases are not recommended for non-technical users, nor for production websites.

See Dries' original announcement about the beta for more information on the beta and the criteria for beta blockers. The explanation of the Drupal 8 release management tags explains the differences between critical beta blockers and other issues impacted by the beta phase.

How can I help? Help stabilize the beta

The beta is an important milestone for Drupal 8. Help test the final alpha for critical and potentially beta-blocking bugs, and take extra care to avoid introducing regressions during this pre-beta window.

Beta deadline issues (complete by September 28)

This final pre-beta window is our final chance to complete beta deadline issues. As a reminder, changes to the following have a beta deadline:

1. Non-critical changes to the core data model. (See the beta-to-beta upgrade path and data model stability policy and the beta-to-beta-upgrade path critical task for ongoing discussion of what is included in the Drupal 8 data model, how we will handle small data model additions, and when we will support a beta-to-beta upgrade path).
2. Non-critical, backward-compatibility-breaking changes to the public APIs of the following critical subsystems:
   • The Configuration system
   • The Entity Field API
   • The Plugin API
   • The Menu and Routing APIs
3. Other broad, non-critical changes that significantly break backward compatibility, at core maintainer discretion.

Beta deadline issues can be committed up until Sunday, September 28, after which there will be a freeze to ensure stability. If you have questions or concerns about completing a particular change, speak to a core maintainer about it soon.

If you know of issues that would introduce any of these changes, add the "beta deadline" issue tag so that contributors can find and help complete them before the beta. The following issues are particular priorities:

• #2002138: Use adapters for supporting typed data
• #2305397: Field type (item) classes implementing AllowedValuesInterface::getPossible(Values|Options)() is incompatible with Views and possibly other use cases
• #2312093: Rename FieldInstanceConfig to FieldConfig

(Also see the full queue of known beta deadline issues.)

Keep in mind that API and schema additions may still be made during the beta phase, at core maintainer discretion. Limited API and data model changes will also happen during the beta phase, though core maintainers will try to isolate these changes to non-fundamental APIs or critical bug fixes. (See the ongoing beta-to-beta-upgrade path discussion.)

Beta target issues

"Beta target" issues are issues that we hope to complete early during the beta phase, but can still be added to Beta 2 or later. These are the next priority after important beta deadline issues. We especially need to work on:

• #2341323: Figure out what to do in Views when entity schema changes
• #2140511: Configuration file name collisions silently ignored for default configuration

(Also see the full queue of known beta target issues).

Thank you!

Many thanks to the 234 contributors who have helped resolve our 177 beta blockers in Drupal 8, and to the incredibly dedicated Drupal 8 branch maintainers. Your focus and effort is helping us build a solid Drupal 8 beta and, going forward, a better release.

In our daily work we all make mistakes in our git commits. Sometimes this errors can easily be repaired just by reverting our commits. But if we are working in a public repository and we have accidentally pushed some sensitive information, we now have a problem.

That sensitive information is in our repository history and anybody who has the enough time to explore can gain access to that. Our clients or even ourselves are now dealing with a privacy issue.

We can always try to repair that commit in our local environment and push our code again using the --force parameter. But we know, when you do that, a kitten dies. And if your team members already pushed something, everything in the repository will be messed up.

So the best option is to try and fix this in a more elegant way that allow us to erase all the traces of our mistake, but preserves repository integrity.

Git provides the filter-branch command, but sometimes this powerful tool becomes too complicated and slow. In trying to find an easier way to do it, finally came across the BFG Repo-Cleaner.

This tool is an alternative to git filter-branch that provides a faster and easier way to clean git repositories. It is written in Java, so you need to make sure you have JRE 6.0 or above installed. To clean your repository you only have to follow the steps below:

Clone your repository using the --mirror option. Beforehand, you should repair manually your mistakes in the repository.

1 $ git clone --mirror git://example.com/my-repo.git

Now, download BFG and execute it against your cloned repository.
1 $ java -jar bfg.jar --strip-blobs-bigger-than&nbsp;1M&nbsp;my-repo.git
This step will remove all the blobs bigger than 1MB from your repository.

Once the index has been cleaned, examine your repository's history and then use the standard git gc command to strip out the unwanted dirty data, which Git will now recognise as surplus to requirements:
1 2 3 $ cd my-repo.git $ git reflog expire --expire=now --all $ git gc --prune=now --aggressive

Finally, once you're happy with the updated state of your repo, push it back up
1 $ git push

If everything went well, your repository won't include any of the accidentally committed files.

Here you have some common examples to use with Drupal:
Delete all files named 'id_rsa' or 'id_dsa' :
1 $ java -jar bfg.jar --delete-files id_{dsa,rsa} &nbsp;my-repo.git

Delete database dumps:
1 $ java -jar bfg.jar --delete-files *{mysql,mysql.gz}

Delete files folder:
1 $ java -jar bfg.jar --delete-folders files

We have to remark that BFG assumes that you have repaired your repository before executing it. You need to make sure your current commits are clean. This protects your current work and gives you peace of mind knowing that the BFG is only changing your repo history, not meddling with the current files of your project.

Finally, here you have some useful related links:

• BFG repository in GitHub: https://github.com/rtyley/bfg-repo-cleaner
• BFG project page: http://rtyley.github.io/bfg-repo-cleaner
• GitHub doc - Remove Sensitive data: https://help.github.com/articles/remove-sensitive-data
• Git fitler-branch documentation: https://www.kernel.org/pub/software/scm/git/docs/git-filter-branch.html
• Git Tools - Rewriting History: http://git-scm.com/book/en/Git-Tools-Rewriting-History

Tags: Drupal Planet

Kyle Hofmeyer talks with Cathy Theys (yesct), Michael Schmid (schnitzel), Emma Karayiannis (emma.maria) about sprints, which occur frequently at Drupal events.

• Drupalcon Amsterdam
• Amsterdam Sprints
• First Time Sprinters Workshop
• Setting Up Your Local for Testing Patches
• Drupal Events Calendar

Episode Number: 169

In this tutorial you will learn about the Honeypot module. The Honeypot modules is a SPAM prevention module that uses a hidden form field to catch SPAM bots from posting onto your site. This tutorial shows you how to configure the module to work on various forms on your site.

Tags: DrupalFormsWebformDrupal 7Drupal PlanetSpam Prevention

After hours of searching Google, lots of trial and error, and a bunch of grumbling, I had a breakthrough and finally figured out how to get Source Maps to work under Chrome and Compass. The problem is that this functionality has been around for over a year in various forms in the pre-release versions of Sass and Chrome. As such, many of the posts I found were out dated and didn't work with the current, stable versions. So this post is partially to document the process for myself (and a small victory lap!), but hopefully someone else will get something out of it.

The admin toolbar is a user's first look at Drupal. A complex, cluttered toolbar gives the n00b a sense that things in this site are too much to handle. A clean, well-curated interface that presents content tasks first gives the n00b a sense that Drupal is easy to use. Drupal 8 promises a simplified toolbar that gives the user the this cozy sense of belonging.

What's new with Drupal 8?

The big news this week is we're still on one beta-blocker. Patches for the remaining beta blocker are coming rapidly with @effulgentsia, @plach and @fago working hard to get it over the line. Could we have zero beta blockers by DrupalCon?

Other keys issues to land this week include Remove ArrayAccess from FormState - never again deal with random arrays - rejoice - $form_state is a first-class object!. Thanks to @timplunkett and others who helped get this through. If you have any contrib projects accessing $form_state in an array fashion eg $form_state['values']['fooey']; then you need to familiarize yourself with the change record.

In a further sign that Drupal 8 is maturing into a modern HTTP framework, we now have support for a stack-php based middleware this will allow us to clean up how page caching, conent negotiotiaton, implementing ban.module's functionalty, options requests and various other elements of the request processing pipeline work. For more information on middlewares see Stackphp.com and this article or see the list of existing middlewares supported by stack-php, and therefore likely to be compatible with Drupal.

In the same vein Modularize kernel exception handling brought some much needed cleanup to to the way we handle exceptions and enables contrib modules to easily add their own exception handling, particularly for custom REST formats.

Over in Convert UnitTestBase to PHPUnit and Remove UnitTestBase, @sun, @Berdir and @tim.plunkett have been working towards removing Simpletest-based Unit tests. There are plenty of sessions around the future of testing at Drupalcon Amsterdam so be sure to check these out if testing is your thing.

The Consensus Banana is moving full-steam ahead with loads of issues resolved to move classes out of preprocessing and into templates landing this week. Meanwhile in Split Seven's style.css into SMACSS categories @LewisNyman has been making great strides towards bringing sanity to Seven's CSS structure.

@WimLeers, @alexpott and @chx worked tirelessly in Add cacheability metadata to access checks to harmonize our access-checking systems and add cacheability to the access results in the form of an AccessResultInterface, great work!

Over in Remove text_processing option from text fields, expose existing string field types as plain text in UI @Berdir, @Wim Leers, @dawehner consolidated our text field types, an important change for Site Builders.

Finally, PHPStorm 8 has been released with lots of support for Drupal 8 APIs!

Where's Drupal 8 at in terms of release?

Since the last Drupal Core Update on Sept. 4, we've fixed 19 critical issues and 24 major issues, and added 12 criticals and 19 majors. That puts us overall at 97 release-blocking critical issues and 644 major issues.

Where can I help? Top criticals to hit this week

Each week, we check with core maintainers and contributors for the "extra critical" criticals that are blocking other work. These issues are often tough problems with a long history. If you're familiar with the problem-space of one of these issues and have the time to dig in, help drive it forward by reviewing, improving, and testing its patch, and by making sure the issue's summary is up to date and any API changes are documented with a draft change record, we could use your help!

• SqlContentEntityStorage::onFieldStorageDefinition(Create|Update|Delete)() is broken for base fields is the final step of the our last beta blocker. Reviews and manual testing welcomed.
• [Meta] Untangle Drupal 8 page rendering lays out the final steps to standardize the page building process. While it's not the result the SCOTCH initiative was hoping for it does line up the pieces needed to allow contrib to experiment with layout mechanisms in a clean and supported way. There are a bevy of child-issues for you to sink your teeth into.

There are also several beta deadline issues that, while not quite critical, will need to be done before the beta if they're to be done at all. The following beta deadline issues are especially important:

• #2002138: Use adapters for supporting typed data will improve developer experience for the Typed Data API, which in turn improves numerous other APIs.
• Field type (item) classes implementing AllowedValuesInterface::getPossible(Values|Options)() is incompatible with Views and possibly other use cases will make an important difference for Views as well as contributed modules like Rules.
• #2312093: Rename FieldInstanceConfig to FieldConfig is a final step to bring consistency and clarity to naming in the Field API.

More ways to help

• Now that we're nearing beta, its time to turn our attention to release-blocking criticals.
• Beta target issues are issues that can be added to Beta 1, Beta 2, or later, but would be best done sooner rather than later for solid beta releases.
• With a looming beta, now we can ramp up our efforts on contrib modules - there's a sprint at Amsterdam just for that - so put your name on the list if this is your thing.

As always, if you're new to contributing to core, check out Core contribution mentoring hours. Twice per week, you can log into IRC and helpful Drupal core mentors will get you set up with answers to any of your questions, plus provide some useful issues to work on.

You can also help by sponsoring independent Drupal core development.

Notable Commits

The best of git log --since "2014-09-04" --pretty=oneline (200 commits in total):

• Issue 2333113 by effulgentsia, plach: Add an EntityDefinitionUpdateManager so that entity handlers can respond (e.g., by updating db schema) to code updates in a controlled way (e.g., from update.php).
• Issue 1857256 by dawehner, xjm, tim.plunkett, jibran, ParisLiakos, hussainweb, pcambra, ekes, InternetDevels, rhabbachi, rdrh555, tstoeckler, oadaeh, Gábor Hojtsy, vijaycs85: Fixed Convert the taxonomy listing and feed at /taxonomy/term/%term to Views.
• Issue 2333501 by swentel | marcvangend: Implement ThirdPartySettingsInterface in EntityView|FormDisplay.
• Issue 1740492 by dawehner, damiankloip, dasjo, xjm: Implement a default entity views data handler.
• Issue 2331019 by slashrsm: Implement ThirdPartySettingsInterface in Vocabulary.
• Issue 2320157 by moshe weitzman, Wim Leers, penyaskito, tim.plunkett: Generate placeholder content for Field types - essentially devel generate in core.
• Issue 2329485 by damiankloip, dawehner: Allow permissions.yml files to declare 'permission_callbacks' for dynamic permissions.
• Issue 1898478 by joelpittet, Cottser, lokapujya, m1r1k, jstoller, er.pushpinderrana, duellj, organicwire, jessebeach, idflood, Jalandhar, Risse, derheap, galooph, mike.roberts, tlattimore, nadavoid, LinL, steveoliver, chakrapani, likin, killerpoke, EVIIILJ, vlad.dancer, podarok, m86 | c4rl: Menu.inc - Convert theme_ functions to Twig.
• Issue 1915056 by Arla, Berdir, amateescu | catch: Use entity reference for taxonomy parents.
• Issue 2321745 by larowlan, tim.plunkett: Add #type => 'path' that accepts path but optionally stores URL object or route name and parameters.
• Issue 474004 by mdrummond, kim.pepper, Wim Leers, jibran, tim.plunkett, joachim | JohnAlbin: Add options to system menu block so primary and secondary menus can be blocks rather than variables - essentially menu block module in core.
• Issue 2068331 by roderik, slashrsm, pcambra, Sharique, piyuesh23, vijaycs85 | plach: Convert comment SQL queries to the Entity Query API.
• Issue 2226493 by Berdir, Wim Leers, m1r1k, mr.baileys, andypost, scor, cbr, joelpittet: Apply formatters and widgets to Node base fields.
• Issue 2302563 by chx, dawehner: Fixed Access check Url objects.

You can also always check the Change records for Drupal core for the full list of Drupal 8 API changes from Drupal 7.

Drupal 8 Around the Interwebs

• Joe Schindler of Drupalize.me gave us Unravelling the Drupal 8 Plugin System an expansive run-down on Drupal 8's plugin architecture.
• Addison Berry, also of Drupalize.me tells us about what to expect now that a Drupal 8 beta is so close
• Dcycle gave us An approach to code driven development in Drupal 8
• Hernani Borges de Freitas of Acquia tells us about Web services in Drupal 8 Core
• Josef Dabernig of the D8 Rules team gives us a rundown on all the comings and goings around Rules in Drupal 8 in Amsterdam

Drupal 8 in "Real Life"

• September 19 - October 3: DrupalCon Amsterdam in Amsterdam, The Netherlands is going to be amazing with lots of opportunities to learn about and get involved with Drupal 8. Be sure to sign up for the extended sprints!
• September 18 - Milwaukee:
  Drupal414 meetup: Building a Drupal 8 theme with new fangled awesomeness - Marc Drummond will present on building a Drupal 8 theme
• September 18 - Washington, DC:Drupal 8 code sprint with Forum One - join Forum one to work on Drupal 8.
• September 20 - Helsinki: Join Wunderkraut to sprint on Drupal 8
• October 17 - Austria: Drupal 8 starter day a free workshop style introduction to Drupal 8

Whew! That's a wrap!

Do you follow Drupal Planet with devotion, or keep a close eye on the Drupal event calendar, or git pull origin 8.0.x every morning without fail before your coffee? We're looking for more contributors to help compile these posts. You could either take a few hours once every six weeks or so to put together a whole post, or help with one section more regularly. Read more about how you can volunteer to help with these posts!

Finally special thanks to KatteKrab for assisting with compiling this edition.

Feature Motivation

The granularity of cache expiration in Drupal has been a long-standing problem.

One can have the most effective cache in the world, but if it clears entirely on any content change, it is not really workable. A “page” in Drupal can have blocks, listing, entities, regions, and many other objects. When one contained item changes, the container of that item needs to be fully rebuilt; often, that is the whole page, a problem requiring a much-needed solution.

Why can't we just rebuild the parts that have actually changed?

Consider what would be the best case scenario here. Assume that every item listed above can be cached separately. Now if one single entity changes, the following would be our "perfect" page request:

1. Drupal bootstraps.
2. Drupal builds the page.
3. Drupal notices that only the “content” region has changed and retrieves the remaining regions from cache.
4. Drupal re-builds the content region.
5. Drupal notices only one listing in the content region has changed and retrieves the remaining blocks from cache.
6. Drupal builds the “missing” block.
7. The block contains a listing of entities.
8. Drupal re-builds the listing, and entity_view() is called on these entities.
9. Drupal retrieves all entities except the changed one from cache.

We would have a bootstrap, then we would see just one region call, one block call, one listing call, and one entity building call. Is this really possible?

Yes and no.

There are certain implementation limitations – especially around page assets – and a unified caching strategy needs to take them into account.

State of the Art

Render Caching is the saving of HTML content in a storage cache, while retaining assets like CSS and JS files and other “out-of-band” data. It can be used for reconstructing the page content, without changing the state the page would have without render caching active. The render cached HTML markup needs to be removed from the cache, or updated in the cache when the objects used for generation of the markup change.

Part 2 of a 2-part conversation with Angie Byron in front of the cameras at NYC Camp 2014, held at United Nations headquarters in New York. In this part of our conversation, we talk about improvements in the Drupal developer- and learning-experience thanks to the major changes under the hood in Drupal 8; the "PHP Renaissance"; and about being welcomed "back into the fold" of the greater PHP world thanks to the nature of Drupal 8 being a sort of "meta project" (my words) that includes parts of many others.

This summer, ThinkShout was named the 9th Fastest Growing Private Company in Portland, Oregon. Admittedly, this came as sort of a shock to me and Lev. Over three and a half years, we’ve grown the company from two dudes renting desks in an incubator space to a full-time team of 17 professionals averaging 10 years of experience each. But most of the time, it doesn’t feel like we’ve come up with any secret sauce for running a successful business. We try to listen to our employees and our peers in the industry. We partner with nonprofit clients trying to make the world a better place, and we do our best to treat them with integrity in all aspects of our work - from our design and engineering practices to our approach to project management and our billing process.

Perhaps there are a few things that we do particularly well. We win our fair share of work in coopetition with our peers in the nonprofit tech industry. But then again, in talking with our friends at ZivTech, Gorton Studios, Aten Design Group, Jackson River, Forum One Communications, and others, what we consistently hear is that there is more than enough work to go around in the world of technology for good.

How is that? What are the mechanics of "the small business of open source" that work for all these firms and, more importantly, for their customers?

You could tackle this question from a number of angles. Conventional wisdom suggests that the business value of open source, both in the for-profit and nonprofit sectors, is simply: you get a lot of stuff for free. This is true. With open source, you avoid licensing fees. With an open source platform such as Drupal, you can pick and choose among literally tens of thousands of free tools and extensions.

But this perspective speaks only to open source’s initial customer appeal. It speaks to the coarse outlines of the sales cycle of open source. You can’t just build a sustainable business off the idea that a platform like Drupal provides a lot of stuff for free.

So, what is the guiding principle of growing a small business around open source? For us, it boils down to:

Our customers benefit tremendously from their own contributions to open source.

Put a bit more bluntly:

Our customers benefit from paying us to give away their code.

To illustrate this point, I’d like to run through some of our numbers as a small business of open source. The following statistics are relative to the publication date of this blog post:

• ThinkShout’s team includes 8 engineers with a total of 4,269 commits to contributed modules hosted on Drupal.org.

• We actively maintain 24 contributed modules that represent over 100,000 lines of code.

• The combined installation base of these modules (i.e., the aggregated number of websites using each of these modules) is over 42,000.

Interestingly, our staff has grown 750% from the end of our first quarter. The growth in adoption of our contributed tools by the community (1,400% since the end of Q1 2011) tracks closely with the growth of our quarterly revenues (1,300%).

Obviously, these statistics are only related anecdotally. On their own, they don’t prove that our open source contributions drive our success.

Still, it is interesting to look at some of the statistics around our client work:

• Averaging the last four $80-$150K Drupal website projects we’ve completed, a typical project in this price range is made up of 36,289 lines of custom code and exported site configuration (excluding the theme, or the implementation of the graphic design).

• In addition, most of these projects include the release of 1,000-5,000 lines of contributed code. In other words, clients who engage with us on these sorts of projects pay for us either to contribute back major features to a contributed module, or to release one or two stand-alone modules.

• But what’s most interesting is that each of these projects leverage at least 4 to 5 of the contributed modules that other clients have paid for on previous projects. Each leverages around 20,000 lines of contributed code that our team has written for similar use cases.

Of course, these statistics reflect the significant cost savings to our clients in going with an open source solution. They get a lot of free stuff…

But more importantly, our clients benefit from becoming financially invested in the direction and cultivation of these open source projects. By becoming committers, they are ensured that their requirements will continue to be prioritized in the future development of the modules their website depend on. They benefit too by the tens, sometimes hundreds, of open source developers reviewing their contributed code for bug fixes and improvements.

Moreover, by releasing the code that powers their websites, our clients connect with other organizations with similar requirements who, in turn, will often contribute additional features to these projects over time. As a case in point, we released the Leaflet module, a web-based mapping solution, on behalf of the Intertwine Alliance in the summer of 2011. The Alliance paid for the initial release, which included a modest yet highly-innovative feature set. Since then, 36 different Drupal developers have contributed code to improve the module, and over 4,500 websites have adopted it.

Similarly, we initially released RedHen CRM, a native CRM solution built entirely within Drupal, for a small cohort of nonprofits with similar needs. In this case, we actually got these nonprofits to work together to brainstorm and prioritize their CRM requirements. Pooling funding, this group helped us launch RedHen publicly in the spring of 2012. Since then, over a dozen of our clients have continued to invest in RedHen, and it has been adopted on over 850 websites around the world. Most impressively, those initial clients that funded our early work on RedHen continue to upgrade their sites with each new release, benefitting from hundreds of thousands of dollars of free development.

Client investments in open source present many less tangible benefits as well. While our developers are inspired to support our mission-driven nonprofit customers, they are particularly excited (and therefore, keenly focused) when their work leads to open source contributions. Not only does this lead to high-quality engineering, it helps reduce turnover among our engineering team, which can be costly for clients. Among our team’s contributors to Drupal, our average turnover is less than half of the industry standard in information technology.

For readers of this post who represent agencies that also contribute to open source, I’d be curious if your data and experience track to ours. Business and nonprofit readers whose teams have paid for open source contributions, I’d be curious to hear your stories as well. After all, open source is all about transparency, which we believe makes us a more effective business.

Episode Number: 168

Following up on the previous Daily Dose of Drupal episode on the Entity Registration module, this episode looks at some of the additional Entity Registration add on modules.

In this episode you will learn:

Tags: DrupalDrupal 7Drupal Planet

In a recent project I got the opportunity to tweak Drupal’s Apache solr queries.In this blog p

After Drupal Camp Lahore and Drupal Camp Islamabad earlier this year, I was once again inivited to Drupal Camp Islamabad to present a session on Drupal 8 as a framework.