Software

It's common knowledge in the Drupal community that Apache Solr (and other text-optimized search engines like Elasticsearch) blow database-backed search out of the water in terms of speed, relevance, and functionality. But most developers don't really know why, or just how much an engine like Solr can help them.

I'm going to be writing a series of blog posts on Apache Solr and Drupal, and while some parts of the series will be very Drupal-centric, I hope I'll be able to illuminate why Solr itself (and other search engines like it) are so effective, and why you should be using them instead of simple database-backed search (like Drupal core's Search module uses by default), even for small sites where search isn't a primary feature.

As an aside, I am writing this series of blog posts from the perspective of a Drupal developer who has worked with large-scale, highly customized Solr search for Mercy (example), and with a variety of small-to-medium sites who are using Hosted Apache Solr, a service I've been running as part of Midwestern Mac since early 2011.

Why not Database?

Apache Solr's wiki leads off it's Why Use Solr page with the following:

If your use case requires a person to type words into a search box, you want a text search engine like Solr.

At a basic level, databases are optimized for storing and retrieiving bits of data, usually either a record at a time, or in batches. And relational databases like MySQL, MariaDB, PostgreSQL, and SQLite are set up in such a way that data is stored in various tables and fields, rather than in one large bucket per record.

In Drupal, a typical node entity will have a title in the node table, a body in the field_data_body table, maybe an image with a description in another table, an author whose name is in the users table, etc. Usually, you want to allow users of your site to enter a keyword in a search box and search through all the data stored across all those fields.

Drupal's Search module avoids making ugly and slow search queries by building an index of all the search terms on the site, and storing that index inside a separate database table, which is then used to map keywords to entities that match those keywords. Drupal's venerable Views module will even enable you to bypass the search indexing and search directly in multiple tables for a certain keyword. So what's the downside?

Mainly, performance. Databases are built to be efficient query engines—provide a specific set of parameters, and the database returns a specific set of data. Most databases are not optimized for arbitrary string-based search. Queries where you use LIKE '%keyword%' are not that well optimized, and will be slow—especially if the query is being used across multiple JOINed tables! And even if you use the Search module or some other method of pre-indexing all the keyword data, relational databases will still be less efficient (and require much more work on a developer's part) for arbitrary text searches.

If you're simply building lists of data based on very specific parameters (especially where the conditions for your query all utilize speedy indexes in the database), a relational database like MySQL will be highly effective. But usually, for search, you don't just have a couple options and maybe a custom sort—you have a keyword field (primarily), and end users have high expectations that they'll find what they're looking for by simply entering a few keywords and clicking 'Search'.

Originally posted on Yellow Pencil's blog. Follow @kimbeaudin on Twitter!

In my last Drupal blog post  I talked about how you can alter an existing view query with hook_views_query_alter, but what if you want to display a result set (your own ?view?) from a custom SQL query?

Well, here's how.

Completed Drupal site or project URL: http://www.chathamhouse.org

Chatham House, home of the Royal Institute of International Affairs, is an independent policy institute and world-leading source of independent analysis, informed debate and influential ideas on how to build a prosperous and secure world for all.

They decided to rebuild their website to better promote their independent analysis and new research on international affairs, engaging with their audiences and disseminating their output as widely as possible in the interests of ultimately influencing all international decision-makers and -shapers.

They wanted a modern, responsive website with a clean design that would provide a better user experience and increased traffic. Additionally, one of the requirements was to integrate the website with their events and membership management software.

Chatham House have used Drupal as their CMS of choice since summer 2011. Their previous Drupal 6 based website was suffering from intermittent performance issues, a dated, non-responsive design and did not integrate with Chatham House’s internal membership management software.

Chatham House worked with Torchbox to produce a new, modern and responsive design on a new Drupal 7 instance. A lot of previous custom code was replaced with tried and tested combinations of contrib modules and a large, rolling content migration (more than 10,000 items) was required to move content from the old Drupal 6 site to the new Drupal 7 site.

Chatham House’s membership management software was also integrated with their new Drupal 7 website for user authentication and events content.

Key modules/theme/distribution used: Display SuiteFeaturesFeedsElectionLinkitMediaMenu blockMenu Trail By PathNagios monitoringCustom BreadcrumbsContextRabbit HoleSecure PagesoEmbedWorkbenchViewsWysiwygmothership

As usual, for my blog posts on the Appnovation website, I’d like to kick off this article off with a disclaimer:

var switchTo5x = false;stLight.options({"publisher":"dr-75626d0b-d9b4-2fdb-6d29-1a20f61d683"});

‘This is the nicest Drupal camp I’ve ever been to.’ These words are quoted from Steve Purkiss, who told this to me on the last Drupalaton night during the cruise party. I think this describes the Drupalaton experience the best - Drupal, summer, fun at the same time.

Topic: DrupalDrupalCon

For my DrupalCon Amsterdam keynote, I want to try something slightly different. Instead of coming up with the talk track myself, I want to "crowdsource" it. In other words, I want the wider Drupal community to have direct input on the content of the keynote. I feel this will provide a great opportunity to surface questions and ideas from the people who make Drupal what it is.

In the past, I've done traditional surveys to get input for my keynote and I've also done keynotes that were Q&A from beginning to end. This time, I'd like to try something in between.

I'd love your help to identify the topics of interests (e.g. scaling our community, future of the web, information about Drupal's competitors, "headless" Drupal, the Drupal Association, the business of Open Source, sustaining core development, etc). You can make your suggestions in the comments of this blog post or on Twitter (tag them with @Dries and #driesnote). I'll handpick some topics from all the suggestions, largely based on popularity but also based on how important and meaty I think the topic is.

Then, in the lead-up to the event, I'll create discussion opportunities on some or all of the topics so we can dive deeper on them together, and surface various opinions and ideas. The result of those deeper conversations will form the basis of my DrupalCon Amsterdam keynote.

So what would you like me to talk about? Suggest your topics in the comments of this blog post or on Twitter by tagging your suggestions with #driesnote and/or @Dries. Thank you!

The greatest thing about all the refactoring in Drupal 8 is that, in general, a lot of those special Drupalisms used nowhere else were thrown out and replaced by sound design patterns, industry best practices and concepts that newcomers from other branches of programming will have an easy time of recognizing and using. While I can understand that this is an annoyance for some who have got used to the Drupalisms (and who haven't got a formal education in programming), as someone with a CS degree and a background in Java I was overjoyed at almost anything new I learned about Drupal 8, which, in my opinion, just made Drupal so much cleaner.
But, of course, this has already been discussed in a lot of other blog posts, podcasts, sessions, etc., by a lot of other people.

What I want to discuss today is one of the few instances where it seems this principle was violated and a new Drupalism, not known anywhere else (as far as I can tell, at least – if I'm mistaken I'd be grateful to be educated in the comments), introduced: plugin derivatives.
Probably some of you have already seen it there somewhere, especially if you were foolish enough to try to understand the new block system (if you succeeded, I salute you!), but I bet (or, hope) most of you had the same reaction as me: a very puzzled look and an involuntary “What the …?” In my case, this question was all the more pressing because I first stumbled upon plugin derivatives in my own module – Frédéric Hennequin had done a lot of the initial work of porting the module and since there was a place where they fit perfectly, he used them. Luckily, I came across this in Szeged where Bram Goffings was close by and could explain this to me slowly until it sank in. (Looking at the handbook documentation now, it actually looks quite good, but I remember that, back then, I had no idea what they were talking about.)
So, without (even) further ado, let me now share this arcane knowledge with you!

What, and why, are plugin derivatives? The problem

Plugin derivatives, even though very Drupalistic (?), are actually a rather elegant solution for an interesting (and pressing) problem: dynamically defining plugins.
For example, take Search API's "datasource" plugins. These provide item types that can be indexed by the Search API, a further abstraction from the "entity" concept to be able to handle non-entities (or, indeed, even non-Drupal content). We of course want to provide an item type for each entity type, but we don't know beforehand which entity types there will be on a site – also, since entities can be accessed with a common API we can use the same code for all entity types and don't want a new class for each.
In Drupal 7, this was trivial to do:

<?php
/**
 * Implements hook_search_api_item_type_info().
 */
function search_api_search_api_item_type_info() {
  $types = array();
  foreach (entity_get_property_info() as $type => $property_info) {
    if ($info = entity_get_info($type)) {
      $types[$type] = array(
        'name' => $info['label'],
        'datasource controller' => 'SearchApiEntityDataSourceController',
        'entity_type' => $type,
      );
    }
  }
  return $types;
}
?>

Since plugin definition happens in a hook, we can just loop over all entity types, set the same controller class for each, and put an additional entity_type key into the definition so the controller knows which entity type it should use.

Now, in Drupal 8, there's a problem: as discussed in the previous part of this series, plugins now generally use annotations on the plugin class for the definition. That, in turn, would mean that a single class can only represent a single plugin, and since you can't (or at least really, really shouldn't) dynamically define classes there's also not really any way to dynamically define plugins.
One possible workaround would be to just use the alter hook which comes with nearly any plugin type and dynamically add the desired plugins there – however, that's not really ideal as a general solution for the problem, especially since it also occurs in core in several places. (The clearest example here are probably menu blocks – for each menu, you want one block plugin defined.)

The solution

So, as you might have guessed, the solution to this problem was the introduction of the concept of derivatives. Basically, every time you define a new plugin of any type (as long as the manager inherits from DefaultPluginManager you can add a deriver key to its definition, referencing a class. This deriver class will then automatically be called when the plugin system looks for plugins of that type and allows the deriver to multiply the plugin's definition, adding or altering any definition keys as appropriate. It is, essentially, another layer of altering that is specific to one plugin, serves a specific purpose (i.e., multiplying that plugin's definition) and occurs before the general alter hook is invoked.

Hopefully, an example will make this clearer. Let's see how we used this system in the Search API to solve the above problem with datasources.

How to use derivatives

So, how do we define several datasource plugins with a single class? Once you understand how it works (or what it's supposed to do) it's thankfully pretty easy to do. We first create our plugin like normally (or, just copy it from Drupal 7 and fix class name and namespace), but add the deriver key and internally assume that the plugin definition has an additional entity_type key which will tell us which entity type this specific datasource plugin should work with.

So, we put the following into src/Plugin/SearchApi/Datasource/ContentEntityDatasource.php:

<?php
namespace Drupal\search_api\Plugin\SearchApi\Datasource;

/**
 * @SearchApiDatasource(
 *   id = "entity",
 *   deriver = "Drupal\search_api\Plugin\SearchApi\Datasource\ContentEntityDatasourceDeriver"
 * )
 */
class ContentEntityDatasource extends DatasourcePluginBase {

  public function loadMultiple(array $ids) {
    // In the real code, this of course uses dependency injection, not a global function.
    return entity_load_multiple($this->pluginDefinition['entity_type'], $ids);
  }

  // Plus a lot of other methods …

}
?>

Note that, even though we can skip even required keys in the definition (like label here), we still have to set an id. This is called the "plugin base ID" and will be used as a prefix to all IDs of the derivative plugin definitions, as we'll see in a bit.
The deriver key is of course the main thing here. The namespace and name are arbitrary (the standard is to use the same namespace as the plugin itself, but append "Deriver" to the class name), the class just needs to implement the DeriverInterface – nothing else is needed. There is also ContainerDeriverInterface, a sub-interface for when you want dependency injection for creating the deriver, and an abstract base class, DeriverBase, which isn't very useful though, since the interface only has two methods. Concretely, the two methods are: getDerivativeDefinitions(), for getting all derivative definitions, and getDerivativeDefinition() for getting a single one – the latter usually simply a two-liner using the former.

Therefore, this is what src/Plugin/SearchApi/Datasource/ContentEntityDatasourceDeriver.php looks like:

<?php
namespace Drupal\search_api\Plugin\SearchApi\Datasource;

class ContentEntityDatasourceDeriver implements DeriverInterface {

  public function getDerivativeDefinition($derivative_id, $base_plugin_definition) {
    $derivatives = $this->getDerivativeDefinitions($base_plugin_definition);
    return isset($derivatives[$derivative_id]) ? $derivatives[$derivative_id] : NULL;
  }

  public function getDerivativeDefinitions($base_plugin_definition) {
    $base_plugin_id = $base_plugin_definition['id'];
    $plugin_derivatives = array();
    foreach (\Drupal::entityManager()->getDefinitions() as $entity_type_id => $entity_type_definition) {
      if ($entity_type_definition instanceof ContentEntityType) {
        $label = $entity_type_definition->getLabel();
        $plugin_derivatives[$entity_type_id] = array(
          'id' => $base_plugin_id . PluginBase::DERIVATIVE_SEPARATOR . $entity_type_id,
          'label' => $label,
          'description' => $this->t('Provides %entity_type entities for indexing and searching.', array('%entity_type' => $label)),
          'entity_type' => $entity_type_id,
        ) + $base_plugin_definition;
      }
    }
    return $plugin_derivatives;
  }

}
?>

As you see, getDerivativeDefinitions() just returns an array with derivative plugin definitions – keyed by what's called their "derivative ID" and their id key set to a combination of base ID and derivative ID, separated by PluginBase::DERIVATIVE_SEPARATOR (which is simply a colon (":")). We additionally set the entity_type key for all definitions (as we used in the plugin) and also set the other definition keys (as defined in the annotation) accordingly.

And that's it! If your plugin type implements DerivativeInspectionInterface (which the normal PluginBase class does), you also have handy methods for finding out a plugin's base ID and derivative ID (if any). But usually the code using the plugins doesn't need to be aware of derivatives and can simply handle them like any other plugin. Just be aware that this leads to plugin IDs now all potentially containing colons, and not only the usual "alphanumerics plus underscores" ID characters.

A side note about nomenclature

This is a bit confusing actually, especially as older documentation remains unupdated: The new individual plugins that were derived from the base defintion are referred to as "derivative plugin definitions", "plugin derivatives" or just "derivatives". Confusingly, though, the class creating the derivatives was also called a "derivative class" (and the key in the plugin definition was, consequently, derivative).
In #1875996: Reconsider naming conventions for derivative classes, this discrepancy was discussed and eventually resolved by renaming the classes creating derivative definitions (along with their interfaces, etc.) to "derivers".
If you are reading documentation that is more than a few months old, hopefully this will prevent you from some confusion.

Image credit: DonkeyHotey

Drupal Remote Dashboard (DRD) fully supports Drush and it does this in two ways: DRD provides all its actions as Drush commands and DRD can trigger the execution of Drush commands on remote domains. This blog post is part of a series (see part 1 of 4) that describes all the possibilities around these two powerful tools. This is part 2 which describes on how to trigger any of DRD's actions from the command line by utilizing Drush.

In open source, something magical happens when like-minded people meet. You find out somebody else is dealing with a similar problem, you combine ideas and before you know it an ad hoc working group has formed to fix the problem. It's a public secret that at Drupalcon the good stuff happens in the BOF sessions.

As the Drupal community has grown we've seen new community events spring up that catered to whole groups of people that before weren't able to meet and talk:

Last Friday, we published the start to our OPCache module for Drupal. While it's still lacking a few features to be ready for a tagged release, we thought it'd be a good idea to get some working code out there.

The goal for this module is to allow Drupal site administrators to clear their opcode cache if they are using the PHP OPcache extension (also known as Zend OPcache or Zend Optimizer+) as well as provide an interface similar to the Memcache Admin module.

As of right now, the only implemented feature is cache flushing. We understand that you might be running on multiple webservers (we build a lot of sites that run on multiple webservers), so we included the ability to flush caches on all of your webservers at once.

Go try the module and feel free to submit patches!

Some time ago, I released Human Queue Worker, a module that takes the concept of the Drupal Queue system, but where the processing of the items is done by human users rather than an automated process. I say 'takes the concept'; it in fact uses the Drupal Queue to create and claim queue items, but instead of declaring your queue with hook_cron_queue_info(), you declare it to Human Queue Worker as a queue that humans will be working on.

This was written for my current project and for a fairly specific need, and I didn't imagine many sites would be using it. However, it has an obvious and popular application: approving comments. I always figured it would be nice if someone wrote a little module to define a comment processing human queue.

Well, that someone is me, and the time is now. You see, I'm an idiot: when I set up this new blog site of mine, I totally forgot to set up a CAPTCHA, and then when I added Mollon, I didn't set it up properly. So this site has a few hundred spammy comments that I need to delete.

The problem is that comment management takes time. Unless there are some magical area of the core UI I've completely missed, I can either visit each node and delete them one by one, or use the comment admin form. There, I can mass-delete the ones with obvious spammy titles, but all the others will still need individual inspection.

The Human Queue UI simplifies this hugely. There's just one page for the queue. When you go to that page, you're presented with an item to process. In the case of comment approval, that's the comment itself, plus the parent node and parent comment to give you some context. To process the comment, click one of two buttons: 'Publish' or 'Delete'. The comment is dealt with, and the form reloads, with a brand new comment for you to process. Which means that the only clicking you do is the action buttons: Publish; Delete; Publish; Delete. (Though with the amount of spam on my site, it's probably Delete; Delete; Delete, like the Cybermen.)

I've not timed it, but I reckon I can probably go at quite a rate. And that's with just one of me: the core Queue system guarantees that only one worker can claim an item at any one time, and that applies to human workers too. So if another user were to work the queue too, by going to the same page, they would be getting shown different comments to work on, and we'd work through the comments at twice the rate.

Now I just need to find a compliant friend and make them into my worker drone. If you're interested, please don't post a comment!

“Contributing to Drupal is life-changing.” Dries, you couldn’t be more right. At DrupalCamp Singapore 2014, developers from all over Asia had the opportunity to spend a day focused on learning, collaborating and preparing for Drupal 8. One takeaway was being reminded of the fact that Drupal 8 still needs significant help from its community to be...

It's funny how quick things happen - Really it is!

Just a week ago I posted I am a Follower & Thinker describing some of my experiences from Open Source. Then, just days later, someone had left me a message in an open Drupal chatroom. What happened after is the result of a chain of interesting - but more or less isolated - events and situations.

Quick background

I'm currently spending some of my time working on an open initiative called Baksteg. I have quite a bit of experience with Drupal and for me it is more than good enough to build the site I have in mind with. Just recently I begun doing some real prototyping of ideas too. Most of the testing have gone very well, but some not so and for those I have started to seek the online community more - poking for help to find solutions or alternative ways.

Online communities are everywhere

www.drupal.org is a fantastic resource to start with. Many problems can quickly be solved doing a quick search there, or it's related sites such as groups.drupal.org and api.drupal.org. Then, when you struggle to find useful help by just searching, you have already started to find other channels to communicate on. In fact you find people using, working with, Drupal everywhere these days. That even includes the same social networks everyone else uses, such as Twitter, Facebook and LinkedIn. Personally I prefer Twitter as it fills my needs and interests good enough, both with Drupal and other ones.

Over the last few months I have worked on tuning my own use of Twitter. I wanted it less as a *megaphone* and more a communication tool to have meaningful, while at the same time a bit entertaining, conversations on. It has worked out really well, while also - as an welcomed bonus - helped me much better appreciate what others actually share out there. My two main feeds are now @tsvenson (personal, mainly in English) and @Baksteg (mainly in Swedish). They both play important roles for my daily needs, which - totally coincidently - works really well with how I now see open source collaboration works out to ;)

Kinda one of the original Social Networks, not that long time ago

However this story happened in the Swedish Drupal-channel on IRC - a social network for geeks and nerds since long ago. It was from kristofferw_ (Kristoffer Wiklund) asking me about this entity ID *problem* I was having troubles with. Some days earlier I had posted a description of it in the Swedish Drupal Group. While that post resulted in some nice advices and ideas, they all turned out to be dead ends. Still, it gave me opportunity to play around with some other modules that later will be used.

- Practice is always good I'm told ;)

Kristoffer was one of them who had helped there. We also go further back, including several Drupal events around. Thus we already know each other, at least when it comes to Drupal stuff. I explained that testing been good, but the work had to, for good reasons, be *pushed* forward. That's when Kristoffer offered to help and write a simple module, if nothing else just to get some coding experience poking around.

As things often end up then, when passionate nerds and geeks find an interesting problem or challenge, brainstorming starts and ideas flows back and forth.

IRC is an important channel in the Drupal community, but it is not because of it's fancy features. It's its simplicity that makes it into such a useful tool to use as a communication hub. A hub that connects us to worldwide chatrooms that can run in the background only to be brought forward when needed or when we have a moment to spare. Or just as inspiration...

Just following the discussions is itself educational and often spurns new ideas. There are many different specialized chatrooms too, one simply called #Drupal-support, most often filled with several hundreds of users, helping each other. It is in these chatrooms many of the toughest challenges with improving the project is ironed out.

A new project is born

It is also here many new projects are born, small as larger ones. As Kristoffer and I began to talk, we quickly found a much more interesting approach. This one had much better potential and many more use cases as well as better flexibility and UX benefits too - So we created a sandbox project on drupal.org. It is now used so that we can experiment further in a better collaborative, open and efficient way.

If this module turns out the way we think it will, then we can take the next step and apply for it to become a Full project. This is a form of quality assurance process that includes us behind the project too, but to get there we need to pass gateways. These are not put in place to stop us though, quite the opposite. Many members in the community voluntary spend their own time to help others to pass. The whole guided process is filled with tools, tips and personal advices about how to make the project work as good as possible, not just for one self but for others too.

Once a full project, access to new features to organize and administrate is granted. Includes proper name space and better ways to manage versions and releases. These are features rarely needed when just poking around and testing ideas in the sandbox.

What I have also learned is what an amazing way to improve my own skills this is. Not just coding related, but also the way to collaborate and how great knowledge transfer can work too. All while at the same time get to know new interesting people and getting exposed to new cultures and ideas.

For me the Drupal community encapsulates all this, and more. Then, taking its size and success into count, it is a pretty remarkable achievement that shows how things can be done quite differently.

Add this to the mix:

• Drupal is used on millions of sites
• Drupal probably already generates a multi billion dollar ecosystem around it

 

Still there is room for practically anyone, like myself, to feel welcomed and included.

Even if it just starts out as a learning experience!

What this module does

At first glance it looks to do little more than adding an extra step, when creating new content, while hiding parts of the form from displaying. That's right, that is basically what it does and one of its main purposes.

Some might now say - Hey, you just going to end up with tons of garbage content! - and they would be right too. Sure, this is going to make it quicker to create a lot of content - Yes, it can certainly be used for that!

But it also makes some other quite interesting new things possible - This is why:

• Entities in Drupal have unique ID-numbers, which can be used for all sorts of interesting things.
• A new entity doesn't get its own ID until after it has been saved the first time.

 

Therein we find my initial problem! There where no smooth way to get around the fact: Users filling in those forms must remember and manually save once before adding content to certain fields. Worse, it would be tricky to notice, when forgotten, as in most cases the data would evaluate, just with something much different than the entity ID needed.

As I played around, with several other modules to see if it was possible to circumvent this somehow, I always hit the same brick-wall. Problem was: Every new idea that showed promise resulted in a solution with tons of complexity, not just in one place but several. Gladly, that complexity was what Kristoffer and I could avoid with just a little bit collaborative brainstorming!

What this module now does is simple:

1. Hijacks entity create (just for content types yet)
2. Allows to limit to content create form down to only display the Title field
3. Creates the new entity
4. Immediately reopens it in normal edit

 

Content type settings:

 

Adding a new node:

 

Thanks to this, I can now avoid displaying any fields that needs the entity ID, minimizing the risk of mistakes. At the same time it also means this new - pre create content - step can be used to only show the bare essential fields while opening up to new interesting possibilities. Any rarely used, and other optional, fields can be dealt with better as they come into play.

This new, less complex, solution will also be much easier to improve upon. It has actually already allowed me to visualize and identify a whole bunch of interesting uses and UX-benefits. It will improve for many roles, not the least site builders and content editors.

So, for me, this is no garbage creator. Instead I see how a small improvement can open up many new creative ways to work and collaborate on content. However that depends, after all it is still just a sandbox project on drupal.org a kind of - Open Source playground - for nerds and geeks like myself.

Not everything is as limited as what is usually read on the tin. For me, this is a few good examples of how things in Open Source can work out just nicely.

Note: What *specifics* Kristoffer gets out of this I know little of. Those specifics are his to share and actually not that important for me - As long as I sense we both get something of value out of the collaboration.

Since our last update, May 2014, the #d8rules initiative was able to complete funding for Milestone 1 and has made major development progress with two unstable releases and loads of commits on our GitHub repository.

Thanks to 142 great supports, we were able to raise $ 15k on Drupalfund. In addition to that, Technocrat stepped in with buying our second largest sponsor package "Reusable component providers" which helps us cover 100 hours of Rules 8.x development. All in all, we are excited to say that Milestone 1 is now fully funded thanks to the crowd funding and 10 companies sponsoring #d8rules.

If you are interested in more background on the #d8rules Drupalfund campaign, make sure to Virginia's blog post with all the details. 

#d8rules Rulers & invoices info

As promised, everybody who has donated $65+ on the Drupalfund will get one of the fancy Rulers provided by Nico from Ausgetrock.net! We just finished production & packaging and will send them out by the next week!

Development update

Development has mainly focused on working with Drupal's context system. Rules actions and conditions need parameters (for example a node object), which is represented as plugin context as used in Drupal core. They also need to provide variables (for example an entity load action will provide the loaded node object), which is implemented as provided plugin context in Rules itself. Provided context might be interesting for other Drupal 8 modules as well, so this might be moved out of Rules either into Drupal core or a Ctools-like project in D8 contributed space.

Currently we try to figure out all parts of the Rules core engine, how context is passed around and how data selectors are applied (example: node:uid:entity:mail:value to access the email adress of a node author). We implemented prototypes of those system parts, but the API is still experimental and we expect to change and improve things here a lot.

A number of action and condition plugins have been ported from Drupal 7 with the help of new Rules contributors, thank you to fubhy, paranojik, jibran and ndewhurst!

We are currently working towards milestone 1 of our roadmap and we are doing monthly unstable releases to keep you up to date with development progress.

Drupalaton training & sprints: Porting Actions

After a great training/sprint session at DrupalCamp Alpe-Adria Portoroz we finished porting most of the conditions. This weekend, at Drupalaton we just delivered our second training session to get contributors up to speed with Rules in Drupal 8. Topics delivered include our git pull request workflow and the new and shiny things about Drupal 8 including the plug-in system, CMI and many more.

Follow [META] Port all actions to 8.x to find out about the current status of actions being ported. Our getting started google doc provides steps in addition to the documentation available on fago's Rules GitHub repository. Upcoming sprint: DrupalCon Amsterdam

• See the sprints information & make sure to fill in at the #d8rules section in the sign-up sheet.
• Drupal 8 Contrib Module Update

Thank you all for contributing.

Josef / dasjo on behalf of the #d8rules team

Drupal 8 is not far off from being released, and you may have heard some chatter about the differences in how you create custom modules. The reality is that, while there are some differences, it's not really that hard to wrap your head around them. This article provides a gentle introduction to creating forms in Drupal 8, and highlights the differences and similarities to how you would do this in previous versions of the platform.

• Drupal
• Drupal 8
• forms
• Form API
• Controllers
• OOP

What are You Going to Learn This Month?

Topics: Tools, Technique, Education

Almost half a year ago, with the help of the Drupal.org Content Working Group and lawyers, the Drupal Association started working on a Drupal.org Terms of Service (ToS) and Privacy Policy. After a number of drafts and rewrites, we are now ready to introduce both documents to Drupal.org users. Read the announcement on Drupal.org for more information.

Lately I've been very interested in declarative programming. Last week Smashing Magazine published my article on Declarative Programming and the Web, and last weekend I gave a talk at DrupalCamp Colorado on "Footless Drupal" in which I talked about how Drupal 8 is using declarative programming for configuration; how the Config in Code (CINC) module is aiming to, among other things, backport that to Drupal 7; and how it's already possible to use custom configuration workflows outside the default Drupal interfaces.

If you're not already familiar with declarative programming and/or the Drupal 8 configuration API, I recommend reading my declarative programming article in Smashing (of course I do), as well as Drupal's configuration API documentation before continuing. All caught up? Great, let's talk about some next steps for expanding declarative programming in Drupal.

Custom Configuration Tools

First, we need more tools for managing configuration, with interfaces designed around more specific workflows. We're using a wide variety of approaches to building Drupal sites, and with a standard configuration format, there's no reason we shouldn't have the same variety of configuration interfaces. Do you define your content types in spreadsheets? Me too. I made an interface for that. Do you test your menus as static HTML? Great, let's make an interface that converts HTML menus to Drupal menu config. Do 90% of your views show all content of a given type? Let's auto-generate those Views configs.

While there's still a lot changing in Drupal 8, and some of that will likely impact configuration structures, those structures are stable enough and simple enough that we should be building our own tools around them today. And as CINC gets closer to D8 config API parity, we can use more and more of the same configuration in D7 as well. These tools can be written entirely outside Drupal, in a completely different language if you prefer, which then exports to YAML. Or you can take advantage of the information available within Drupal, i.e. current site configuration and content, and build modules with new interfaces. The world is our new Drupal configuration playground. Let's get playing.

Declarative Forms

But there's also no reason declarative programming in Drupal needs to stop at configuration. Drupal has a wide variety of non-configuration concepts that could benefit from declarative approaches. One area I've been thinking a lot about lately is forms. Drupal's form API is already mostly declarative, but it's built around PHP arrays, and we interact with form arrays with imperative code. We could probably learn a lot by comparing Drupal forms to existing standards for declarative forms, like XForms. But simply taking a form array and formatting it as YAML would be a good start. (Or we could do it as JSON, as Amitai Burstein has suggested).

I'm sure declarative forms in Drupal would end up being a large and complex project, but it would come with large benefits as well. Many form alters apply universally to a given form, so they might as well be editing the original form definition directly. That wouldn't make sense in community module code, but moving the form definition outside code would allow such form alters to happen directly in the configuration. The same way we can now do \Drupal::config('node.type.page')->set('name', 'Basic page')->save(), we could do form alters with something like \Drupal::form('formid')->set('mytextfield.#title', 'My Text Field')->save(). Or with a YAML workflow, simply editing a YAML file would edit the form. That's already a huge benefit, as it would open up many form alters to a much wider group of implementers. Remember when we edited forms directly in HTML? A YAML form interface could give us that same ease of editing while still maintaining the power of Drupal's form API.

And just like with declarative configuration, declarative forms would open up a wider world of use cases. Forms created in Webform or the field API could be easily reused in custom modules. And the same forms could be built or used entirely outside Drupal, opening the door for more form-building tools focused on specific workflows.

Hopefully this is enough to get more people excited about taking advantage of the parts of Drupal that already have declarative interfaces, and also pushing declarative programming even further in Drupal. I'm continuing to work on these ideas and will likely have more examples to share soon, but I'd really like to see more people playing here. If you're already working or interested in working on declarative programming in Drupal, let's talk.

This week the podcast gets a little bit meta by discussing how to find Drupal news. Addison Berry is joined by three different community news sources: Mike Anello of DrupalEasy podcast, Bob Kepford of The Weekly Drop newsletter, and Chris Weber from Drupal and Coffee in the Morning hangout and the Google+ Drupal community. We discuss why following the news is important, and how we manage to keep up with it.

• Panels Video Tutorials
• DrupalEasy Podcast
• The Weekly Drop Newsletter
• Google+ Drupal Community
• Drupal Planet
• Drupal.org RSS Feeds
• JavaScript Weekly
• Node Weekly
• Shoptalkshow Frontend Podcast

Will you be coming to DrupalCon Amsterdam?

DrupalCon is where all the big magic happens. From sprints that improve the project to training that brings in new talent, attending DrupalCon is the best way to get involved, get connected, and give back.

Take a break from your summer holiday and secure your place at DrupalCon before ticket prices rise on 8 August (that's tonight!) at 23:59 Amsterdam local time. Save the €50 you'd pay on a late ticket and make sure you buy your tickets now to take advantage of the regular rate. You can even buy for your company, with our easy prepaid tickets.

On the fence about attending?

If you're having trouble deciding whether to attend, we suggest you check out the schedule of sessions, BoFs, parties, and more, see who's going to be in Amsterdam, what you can learn, and how you can help the project while you're there. We've even got Cory Doctorow keynoting the event on Wednesday! What's not to love about that?

If you're having trouble convincing your boss to send you, check out these resources we've put together to help you get your manager's buy-in on the conference.

There will be great sessions, fantastic training opportunities, a quickly-filling business summit, a community summit, tons of sprints, and lots of fun at DrupalCon Amsterdam.

Have more questions? Check out the @DrupalConEur Twitter handle for breaking news and new information, or let us know if you need help.

We hope to see you in Amsterdam!