Software

RestoreAHackedSite tl;dr

Rollback a server backup (files and database) from before October 15th 2014.

No server backup?

1. Run "git status" to find new and modified files.
   • Delete new files
   • Checkout modified files
2. Thouroughly check files directory for anything unusual.
3. Make sure the .htaccess file in the files directory restricts code execution
4. Restore database from pre Oct. 15th backup
5. Update Drupal Core to latest release

... Read on for details...

I think I might have been hacked. What do I do?

Hi, this is Brian Lewis with Modules Unraveled.

As you probably already know, there was a huge security fix released for Drupal 7 on October 15th (SA-CORE-2014-005). The patch to update Drupal is actually quite small, but the implications of not updating your site are massive. As a matter of fact, if you haven't already updated your site, chances are you have already been hacked. There were automated programs systematically attacking Drupal sites hours after the fix was released. In this video I'm going to show you how to find out whether or not your site has been hacked. And if so, I'll walk you through what you need to do now, to reduce the damage done.

There are two ways to find out whether your site has been hacked. With "git status" and by searching the database.

• Run "git status" inside Drupal root
  • This will show us any files that have been modified since our last commit. On the live server, there shouldn't be any, so anything listed here, I know is a result of being hacked.
  • This is a huge reason you should be using version control on your site. If you're not, you can try to re-download every module, theme and library you have and download a fresh copy of the version of Drupal core that you had before the attack and replace all of those on your server. I'm hesitant to recommend this as a full fix though, because there may be hidden files, or files in places you don't think to look. Really, my recommendation is a full re-install. If you're in this situation, I'm sure you don't want to hear that, but I hope this gives you a reason to look into Git.
• Search for "file_put_contents" in database
  • If there is a result. You've been hacked.
  • Click "Browse".
  • Click the "BLOB" link under "access_arguments". This should download a file to your local machine.
  • Open that file with a text editor.
  • Notice that only one file is listed. There may be others that need to be deleted.
• If there are no extra files in your git repo, and no results in database search. You're not hacked. Update Drupal Core now! Or at least do the hotfix mentioned here as a temporary measure.
• Delete/checkout all files listed by "git status" (Also check your files directory. The files directory should not be in Git, but that means there's no easy way to view new and modified files, but they could have been placed there. By default, the .htaccess file that is in that directory prevents php code from being executed, but Michael said he has seen an attack that modified that .htaccess file. So, you need to check your site.)
• Restore Database (Otherwise thouroughly check Users, Node, etc.)
• Install latest Drupal Core update

Recap:

1. Run "git status" to find new and modified files.
   • Delete new files
   • Checkout modified files
2. Thouroughly check files directory for anything unusual.
3. Make sure the .htaccess file in the files directory restricts code execution
4. Restore database from pre Oct. 15th backup
5. Update Drupal Core to latest release

Updates:

1. Drupal security team member Greg Knaddison (greggles) wrote up a great guide on what to do when you get hacked. He includes things I didn't mention like making a forensic copy of your site to inspect later, and notifying site stakeholders. You can read that here.

Tags: Securityplanet-drupal

Start:  2014-11-05 09:00 - 2014-11-10 18:00 America/Los_Angeles Sprint

https://2014.badcamp.net/event/core-dev-sprint

We have a great tradition of extended sprints around big Drupal events. Given that a lot of the Drupal core and contrib developers fly in for these events, it makes a lot of sense to use this opportunity to start sooner and/or extend our stay and work together in one space on the harder problems.

BADCamp is next up! BADCamp organizers and sponsors continue to recognize the need for sprints as part of the schedule and are providing space on Wednesday (TBA), great sprint location Thursday through Sunday at the Palace of Fine Arts, and a space on Monday after also (TBA). There are already various sprints signed up including Multilingual, Drupal 8 Criticals, Panopoly, and Frontend. We are really friendly and need all kinds of expertise!

Now is the time to consider if you can be available and book your travel and hotel accordingly!

Join the sprinters -- sign up now! Practical details

Dates

Nov 5 to 10 (entire BADCamp and one extra day before and after).

Times and locations

Day/Time Location Weds Nov 5, 9am to TBA Location, TBA Thurs Nov 6 - Sun Nov 9, 9am to 6pm every day, only open daytime Sprint room at the venue: Palace of Fine Arts, address. Mon Nov 10, 9am to TBA Chapter Three, TBA

Subscribe as calendar events

We even created a Google Calendar that you can subscribe to (4j8sqq5fphhpgmrtfl8t2ggkes@group.calendar.google.com) which contains all the sprint venue information at any point in time.

-->Subscribe as calendar events

We even created a Google Calendar that you can subscribe to (4j8sqq5fphhpgmrtfl8t2ggkes@group.calendar.google.com) which contains all the sprint venue information at any point in time.

-->

Sponsors

Acquia (@acquia) and Chapter Three (@chapter_three).

Looking for sponsors We are looking for more sponsors to be able to pay for extra expenses. If you are interested sponsoring or if you need sponsors to cover expenses, please contact me at https://drupal.org/user/4166/contact -->Looking for sponsors We are looking for more sponsors to be able to pay for extra expenses. If you are interested sponsoring or if you need sponsors to cover expenses, please contact me at https://drupal.org/user/4166/contact -->Frequently asked questions What is a sprint?

Drupal sprints are opportunities to join existing teams and further Drupal the software, our processes, drupal.org and so on.

Do I need to be a pro developer?

No, not at all. First of all sprints include groups working on user experience, designs, frontend guidelines, drupal.org software setup, testing improvements, figuring out policies, etc. However you can be more productive at most sprints if you have a laptop.

Why are there 6 consecutive days of sprints?

This is a time when many people in the Drupal community get together. We try to use this time to share our knowledge as well as further the platform in all possible ways. Therefore there is almost always an opportunity and a place to participate in moving Drupal forward.

What if I'm new to Drupal and/or sprinting, how can I join?

If you feel new and would love helping hands, please attend. It can take a day or a couple days to get set up and find something to work on. But, the great thing is, we have plenty of days to do that, find a group to work with, and get things really moved forward together.

Further questions?

Ask me (YesCT), I am happy to answer.

#node-427578 .picture, #node-427578 h3 { display: none; } #node-427578 .field-type-datestamp { margin: 0 0 2em 0; } #node-427578 dl { margin-bottom: 1em; } #node-427578 dd { margin-top: 0.5em; } #node-427578 h3.content { display: block; }

Published: Wed, 10/22/14Download this episodeDutch Drupal Foundation

• What is the Dutch Drupal Foundation?
  • Dutch Drupal Foundation supports Dutch Drupal Community. We’re groing. Trying to handle work in a professional way in a voluntary way. Lead double lives: daytime job versus community commitments.
• How is this different from the Drupal Association?
• How has that been going for you?
• Would you recommend other regional areas start something similar?

Events

• What events have you organized recently?
  • Drupal Events in Dutch speaking part of EU (Belgium and Netherlands) became big. E.g. DrupalJam (350ppl), Drupal Training Day (250 students). Relatively large events become routine. Now ppl spreading cross border on tour to share experience
• Tell me about DrupalJam. What is it, and how long have you been putting it on?
  • DrupalJam largest event in Netherlands. Started small in 2007 with 30 persons and some pizza boxes in a basement. Now 350 ppl / €25K. Outgrown study dorm and pizza’s. Also started Drupal Training Day (largest worldwide) and Drupal Splash Awards. Lot of work and commitment. Getting more serious. Growing pains expected
• What are some difference in paid events vs. free events?
• What has the attendance been like for free events?
• Where does the income come from?
• How do you get sponsors?
• When do you recommend charging a ticket price to attendees?
• How have you seen your community transition from coders who just come to code for fun, to people who are making their living using Drupal?
  • Come for Drupal, stay for code. Sounds fun. It is. But business side of things and commitment to making this work professionally is different cake.
  • Balance between business side and community side, which is a recurring topic for everything we do in the community. Drupal has become enterprise platform, community still many smaller agencies w/ different values (see incline on Gartner oct. 2014)
• What does the Drupal landscape look like in europe
  • Pan EU events: Frontend United, CxO, Drupal Government Days Brussels 2009
  • DrupalEU

Episode Links: Bert Boerland on drupal.orgImre Gmelig Meijling on drupal.orgBert Boerland on TwitterImre Gmelig Meijling on TwitterBert on LinkedInStitchting Drupal in NederlandsDrupalJamDrupal Training DayTags: drupalcommunityeventsplanet-drupal

Part 2 of 2 – Kris Vanderwater (EclipseGc), Acquia’s Developer Evangelist, and I got together in a Google Hangout to catch up on our impressions of DrupalCon Amsterdam. We prepared a list of our top ten sessions from the Con for you to catch up with at home (technically nine sessions and one “other cool thing”). In our list, there’s a little something for most everyone, from coders, to themers, to site builders, to those of us who pitch sell Drupal to clients – but we would recommend all of these sessions to anyone involved in Drupal. See how the other side lives!

BADCamp (Bay Area Drupal Camp) will take place at the Palace of Fine Arts, November 6-9. Kicking off with 6 summits, free Drupal training, and a two full days of sessions, BADCamp is an excellent opportunity to immerse yourself in Drupal for several days and have a blast doing it.   A number of CiviCRM developers and providers will participate in the non-profit summit on Thursday, November 6th (register now to attend!) and a CiviCRM BoF will be organized on Saturday, November 8 along with the scheduled sessions.   If you plan to attend, let us know in the comments. Based on the participation we may organize a mini-sprint or a community get-together.

• CiviCRM
• Drupal

Ansible and Drupal Development

As I mentioned in my hello world post, I've been learning Ansible via Jeff Geerling's great book Ansible for Devops. When learning new technologies, there is no substitute for diving in and playing with them on a real project. This blog is, in part, the byproduct of my efforts to learn and play with Ansible. Yet embedded within that larger goal were a number of additional technical requirements that were important to me, including:

Wed, 10/22/2014 - 09:23 aaron

I estimate hundreds of thousands of Drupal websites now have backdoors; between ten and fifty percent of all Drupal websites. Automated Drupageddon exploits were in the wild within hours of the announcement. Updating or patching Drupal does not fix backdoors that attackers installed before updating or patching Drupal. Backdoors give attackers admin access and allow arbitrary PHP execution.

read more

Last month at the Central Denver Drupal meeting, Nick Switzer from Elevated Third showed how they are using a structured spreadsheet format for describing their Drupal configuration in a way that makes it easy to build. They based their spreadsheet format on a template Palantir published a while ago, and someone mentioned Lullabot has been using something similar. This looked to me a lot like what we were doing at Aten, even though we had missed the de facto standard that was developing. We are now using that de facto standard.

This was particularly interesting to me because I've been doing a lot of work lately around declarative interfaces and standardized Drupal configuration. Spreadsheets are declarative and CINC has a working YAML import, so when we got to the question and answer portion of the presentation, I knew exactly what I wanted to ask: "Why are we still building Drupal sites manually when these spreadsheets contain everything we would need to automate it?"

No one offered a reason not to automate this process, so I volunteered to present at this month's meeting and show an automated process that did not yet exist. I have since built that process. It still needs a lot more testing and bug fixes, but it's already a compelling alternative to the traditional Drupal site building process.

Sheet2Module

Sheet2Module takes a Google spreadsheet and produces a Drupal module that will create the configuration described therein. The exported modules use YAML files for configuration, which works natively in Drupal 8, and works in Drupal 7 with the CINC YAML submodule. With a standard spreadsheet format, Sheet2Module, and CINC YAML, you can build a reasonably complex Drupal site configuration in a few minutes. The process looks like this:

1. Describe your Drupal configuration in a Google spreadsheet.
2. Use Sheet2Module to auto-generate a module from that spreadsheet.
3. Enable that module to auto-generate your Drupal configuration.
4. (Optional) Spend the hours you would otherwise spend on Drupal configuration helping improve this process.

Both Sheet2Module and CINC YAML almost certainly have bugs, as they've had very limited testing. Both are open source (CINC on Drupal.org, Sheet2Module on GitHub), and patches and pull requests will be met with enthusiastic appreciation. Beyond my appreciation, I'm convinced custom-tailored interfaces like this are the future of Drupal configuration, and you have a lot to gain from helping shape that future.

Outside code contributions, simply trying out the process and giving feedback is very useful, and a good way to make sure this works for your own workflow. Even the incomplete current solution will likely save you hours on your next Drupal build, and you can still manually add any configuration that doesn't work automatically. So you have nothing to lose and hours to gain by trying it out.

Drupal Spreadsheet Standard

I suspect there are more than a few shops already using a similar spreadsheet format to describe Drupal configuration, so before we go too far down the path of building tools around this format, we should turn this into a real, documented community standard. To that end, I've started creating a Drupal Configuration Spreadsheet Standard on GitHub. If you're already using spreadsheets to describe your Drupal configuration, take a look at the documentation and contribute your own format improvements to the wider community. If you're just getting started using spreadsheets to describe your Drupal configuration, this is a good place to start.

Own Your Process

Even if you're not using spreadsheets to describe Drupal configuration, it's worth taking a look at this automation for ideas on how you can improve your own process. I've mentioned before that the declarative format for Drupal configuration adopted in Drupal 8 (and available Drupal 7 with CINC) allows us all to customize our workflows. I'm going to keep mentioning it until this becomes common enough in the Drupal community that it's boring to mention. But for now, this is still a new and exciting space to be working in, and you should join the fun.

27 Questions (and Answers) from My First Drupal 8 Site Build I recently built my first site with Drupal 8, off of the public beta. It was a great experience. I kept a list of questions as I worked, and wrote down the answers when I found them. matt Tue, 10/21/2014 - 13:10

As developers we always are looking for ways to become more efficient. After all, time is money.

At Studio.gd we love the Drupal ecosystem and it became very important to us to give back and participate.
Today we're proud to announce a new module that we hope will help you !

Inline Entity Display module will help you handle the display of referenced entity fields directly in the parent entity.
For exemple if you reference a taxomony "Tags" to an Article node, you will be able directly in the manage display of the article to display tags' fields. It can become very usefull with more complex referenced entity like field collection for exemple.

VOIR LE MODULE : https://www.drupal.org/project/inline_entity_display

Features

- You can control, for each compatible reference field instances, if the fields from the referenced entities would be available as extra fields. Disabled by default.

- You can manage the visibility of the referenced entities fields on the manage display form. Hidden by default.

- View modes are added to represent this context and manage custom display settings for the referenced entities fields in this context {entity_type}_{view_mode} Example: "Node: Teaser" is used to render referenced entities fields, when you reference an entity into a node, and you view this node as a teaser if there are no custom settings for this view mode, fields are rendered using the default view mode settings.

- Extra data attributes are added on the default fields markup, so the field of the same entity can be identified.

Compatible with Field group on manage display form.

Compatible with Display Suite layouts on manage display form.

Requirements

- Entity API
- One of the compatible reference fields module.

Tutorials

simplytest.me/project/inline_entity_display/7.x-1.x
The simplytest.me install of this module will come automatically with these modules: entity_reference, field_collection, field_group, display suite.

VOIR LE MODULE : https://www.drupal.org/project/inline_entity_display

We are currently developping a similar module for Drupal 8 but more powerful and more flexible, Stay tuned !

Blink Reaction's Director of IT, Kenny Silanskas takes a look at why content is crucial when it comes to creating responsive design. Here Kenny breaks it down with a few easy sports analogies.

One of many new features in Drupal 8, made possible by the configuration management system, is the ability to add a default image style to your theme, instead of needing to use a module in tandem with your theme, or creating the image style by hand. Here's a look at working with this new feature in Drupal 8.

read more

Don't you know that, too? You or your team is building a site and during this process all implemented parts are styled through templates and CSS. The CSS files (at best you are using a CSS preprocessor like SASS) are getting bigger, more sophisticated and even more confusing - not to mention that these files are getting almost unmaintainable and more and more error-prone.

drupal planet englishCSS

TL;DR: I've created a fork of Pressflow for the purposes of conversation and analysis -- https://github.com/btopro/Presser-Flow-FORK

History lesson

Part 1 of 2 – Kris Vanderwater (EclipseGc), Acquia’s Developer Evangelist, and I got together in a Google Hangout to catch up on our impressions of DrupalCon Amsterdam. We prepared a list of our top ten sessions from the Con for you to catch up with at home (technically nine sessions and one “other cool thing”). In our list, there’s a little something for most everyone, from coders, to themers, to site builders, to those of us who pitch sell Drupal to clients – but we would recommend all of these sessions to anyone involved in Drupal. See how the other side lives!

With the incorporation of many Symfony components into Drupal in its 8th version, we are seeing a shift away from many Drupalisms towards more modern PHP architectural decisions. For example, the both loved and hated hook system is getting slowly replaced. Plugins and annotations are taking away much of the need for info hooks and the Symfony Event Dispatcher component is replacing some of the invoked hooks. Although they remain strong in Drupal 8, it’s very possible that with Drupal 9 (or maybe 10) hooks will be completely removed.

In this article we are going to primarily look at how the Symfony Event Dispatcher component works in Drupal. Additionally, we will see also how to invoke and then implement a hook in Drupal 8 to achieve similar goals as with the former.

To follow along or to get quickly started, you can find all the code we work with here in this repository. You can just install the module and you are good to go. The version of Drupal 8 used is the first BETA release so it’s preferable to use that one to ensure compatibility. Alpha 15 should also work just fine. Let’s dive in.

What is the Event Dispatcher component?

A very good definition of the Event Dispatcher component can be found on the Symfony website:

The EventDispatcher component provides tools that allow your application components to communicate with each other by dispatching events and listening to them.

I recommend reading up on that documentation to better understand the principles behind the event dispatcher. You will get a good introduction to how it works in Symfony so we will not cover that here. Rather, we will see an example of how you can use it in Drupal 8.

Continue reading %Drupal 8 Hooks and the Symfony Event Dispatcher%

Every Friday at noon Pacific (3pm New York, 9pm Berlin, 6am Saturday in Sydney) I will be in #drupal-contribute helping people fix critical issues. I will prepare 2-3 issues with up to date, actionable issue summaries, familiarize myself with the problems and the suggested solution in the issue so that I can answer questions.

If you're someone who has already worked some in the Drupal.org issue queue (so are familiar with patches, coding standards, etc.), even if your experience is not in the core queue, please come by! It's helpful if you know something of Drupal 8 as well, but not necessary.

If you're new to contributing to Drupal in general, you can go to https://www.drupal.org/core-mentoring for a session or two to learn the skills you need to fix critical issues. If you're new to Drupal 8, https://api.drupal.org/api/drupal/8 is a great starting point.

Hope to see you there!

As I mentioned in my recent post, I got a chance to upgrade the drupal.org ELK stack last week. In doing so, I got to take a look at a Logstash configuration that I created over a year ago, and in the course of doing so, clean up some less-than-optimal configurations based on a year worth of experience and simplify the configuration file a great deal.

The Drupal.org Logging Setup

Drupal.org is served by a large (and growing) number of servers. They all ship their logs to a central logging server for archival, and around a month’s worth are kept in the ELK stack for analysis.

Logs for Varnish, Apache, and syslog are forwarded to a centralized log server for analysis by Logstash. Drupal messages are output to syslog using Drupal core’s syslog module so that logging does not add writes to Drupal.org’s busy database servers. (@TODO: Check if these paths can be published.) Apache logs end up in/var/log/apache_logs/$MACHINE/$VHOST/transfer/$DATE.log, Varnish logs end up in/var/log/varnish_logs/$MACHINE/varnishncsa-$DATE.log and syslog logs end up in /var/log/HOSTS/$MACHINE/$DATE.log. All types of logs get gzipped 1 day after they are closed to save disk space.

Pulling Contextual Smarts From Logs

The Varnish and Apache logs do not contain any content in the logfiles to identify which machine they are from, but the file input sets a path field that can be matched with grok to pull out the machine name from the path and put it into the logsource field, which Grok’s SYSLOGLINE pattern will set when analyzing syslog logs.

Filtering on the logsource field can be quite helpful in the Kibana web UI if a single machine is suspected of behaving weirdly.

Using Grok Overwrite

Consider this snippet from the original version of the Varnish configuration. As I mentioned in my presentation, Varnish logs are nice in that they inclue the HTTP Host header so that you can see exactly which hostname or IP was requested. This makes sense for a daemon like Varnish which does not necessarily have a native concept of virtual hosts (vhosts,) whereas nginx and Apache default to logging by vhost.

Each Logstash configuration snippet shown below assumes that Apache and Varnish logs have already been processed using theCOMBINEDAPACHELOG grok pattern, like so.

filter { if [type] == "varnish" or [type] == "apache" { grok { match => [ "message", "%{COMBINEDAPACHELOG}" ] } } }

The following snippet was used to normalize Varnish’s request headers to not include https?:// and the Host header so that therequest field in Apache and Varnish logs will be exactly the same and any filtering of web logs can be performed with the vhost andlogsource fields.

filter { if [type] == "varnish" { grok { # Overwrite host for Varnish messages so that it's not always "loghost". match => [ "path", "/var/log/varnish_logs/%{HOST:logsource}" ] } # Grab the vhost and a "request" that matches Apache from the "request" variable for now. mutate { add_field => [ "full_request", "%{request}" ] } mutate { remove_field => "request" } grok { match => [ "full_request", "https?://%{IPORHOST:vhost}%{GREEDYDATA:request}" ] } mutate { remove_field => "full_request" } } }

As written, this snippet copies the request field into a new field called full_request and then unsets the original request field and then uses a grok filter to parse both the vhost and request fields out of that synthesized full_request field. Finally, it deletesfull_request.

The original approach works, but it takes a number of step and mutations to work. The grok filter has a parameter calledoverwrite that allows this configuration stanza to be considerably simlified. The overwrite paramter accepts an array of values thatgrok should overwrite if it finds matches. By using overwrite, I was able to remove all of the mutate filters from my configuration, and the enture thing now looks like the following.

filter { if [type] == "varnish" { grok { # Overwrite host for Varnish messages so that it's not always "loghost". # Grab the vhost and a "request" that matches Apache from the "request" variable for now. match => { "path" => "/var/log/varnish_logs/%{HOST:logsource}" "request" => "https?://%{IPORHOST:vhost}%{GREEDYDATA:request}" } overwrite => [ "request" ] } } }

Much simpler, isn’t it? 2 grok filters and 3 mutate filters have been combined into a single grok filter with two matching patterns and a single field that it can overwrite. Also note that this version of the configuration passes a hash into the grok filter. Every example I’ve seen just passes an array to grok, but the documentation for the grok filter states that it takes a hash, and this works fine.

Ensuring Field Types

Recent versions of Kibana have also gotten the useful ability to do statistics calculations on the current working dataset. So for example, you can have Kibana display the mean number of bytes sent or the standard deviation of backend response times (if you are capturing them – see my DrupalCon Amsterdam slides for more information on how to do this and how to normalize it between Apache, nginx, and Varnish.) Then, if you filter down to all requests for a single vhost or a set of paths, the statistics will update.

Kibana will only show this option for numerical fields, however, and by default any data that has been parsed with a grok filter will be a string. Converting string fields to other types is a much better use of the mutate filter. Here is an example of converting the bytes and the response code to integers using a mutate filer.

@TODO: Test that hash syntax works here!

filter { if [type] == "varnish" or [type] == "apache" { mutate { convert => { [ "bytes", "response" ] => "integer", } } } }

Lessons Learned

Logstash is a very powerful tool, and small things like the grokoverwrite parameter and the mutate convert parameter can help make your log processing configuration simpler and result in more usefulness out of your ELK cluster. Check out Chris Johnson’s post about adding MySQL Slow Query Logs to Logstash!

If you have any other useful Logstash tips and tricks, leave them in the comments!