Version Control Blog

Drupal development team has released Drupal 5.2 on July 26, 2007. It fixes two security vulnerabilities, so it is highly recommended that you upgrade as soon as possible. Many Drupal installations often contain extra Drupal modules, and almost always the also contain local customizations.

Question arises: how to upgrade your Drupal installations

• timely,
• safely,
• with confidence that none of your local customizations are lost;
• without need to remember each line that was edited and re-applying those edits to new version;
• without need to drastically change your workflow (very little discipline is needed actually);

We show that Git solves those problems easily.

Table of Contents

• Assumptions
• Lines of development
• Importing Drupal source code
• Importing Drupal modules
• Local customizations of Drupal
• Upgrading Drupal
  • Part I, drupal
  • Part II, drupal-and-modules
  • Part III, drupal-production
  • Part IIIb: resolving conflicts
• Links
• Final words

It is well-known that Git is a distributed version control system that was created by Linus Torvalds to help with the development of Linux kernel. Distributed version control systems, such as Git, are contrasted with centralized version control systems, such as Subversion. Linux kernel development is characterized by hundreds of contributors and several dozens of development sub-projects, all spread out across the Internet. The repositories contain thousands of files and many thousands of revisions.

We show that Git is actually capable of handling much more lightweight problems, without any unnecessary overhead, with only half a dozen of commands to remember.

Assumptions

You will need:

• command line access to any modern Linux distribution or FreeBSD installation;
• basic knowledge of Linux command line and file system;
• sufficiently good knowledge of Drupal installation and maintenance;
• installed Git (consult the documentation);

The code which you will work with resides in working copies. All history of code that you keep in Git is stored in repositories, which is just a single subdirectory with a special name (.git/), residing at the top level of working copy. By default repositories and working copies are bound very tightly (unlike in Subversion).

We assume that you use Drupal 5.x with several modules, and several (or many) local customizations. Drupal and modules could be upgraded separately at any time. Local customizations could be rather extensive.

Our real installation which will be used as an example uses Drupal 5.1 (painlessly upgraded to 5.2 immediately after the release). We use the following:

• Adsense module;
• Google Analytics module;
• Print Friendly pages (which also had a security vulnerability up to a recently released 5.x-1.2 version).

Our local customizations are simple one-line fixes, e.g.:

• single line added to robots.txt, to exclude our “obsolete content” page from attention of spiders;
• several redirects for legacy URLs added to top-level .htaccess file;
• PHP set_time_limit() function is not allowed by our hosting provider, so its occurrences are commented out;
• sites/default/settings.php, where we have specified our Mysql server login and password (this is obviously most common customization);

The changes are few, but we expect them to accumulate over time, adding bug-fixes and such like.

Lines of development

Basically, you have three lines of development: Drupal, modules and your own. Correspondingly, we shall use three working copies (each with its own repository):

• drupal, containing pristine Drupal 5.x versions;
• drupal-and-modules, containing most recent version of Drupal together with current version of modules you use;
• drupal-production, containing most recent version of Drupal, current version of modules, and your local customizations.

Those three repositories will be chained together, with changes propagated from left to right. So,

• after you commit new version of Drupal to the drupal working copy, Git can merge changes automatically to drupal-and-modules, and then to drupal-production;
• also, when you commit new modules to drupal-and-modules, Git can merge changes automatically to drupal-production;
• your local customizations are committed to drupal-production;
• drupal-production is what actually gets copied to the web server.

That will be the complete workflow, basically.

NB: if you already have a Drupal installation, you will only need to remember its version, and extra modules you have installed. Then we will reproduce that installation, and cleanly import your existing changes: you will also have a chance to clearly look at them.

Importing Drupal source code

Let’s start from previous version of Drupal, so that we could simulate the upgrade.

$ mkdir ~/src
$ cd ~/src
$ wget http://ftp.drupal.org/files/projects/drupal-5.1.tar.gz

$ mkdir drupal (that will be our first working copy)

NB: your tar may not support this command line: what we need is to
  extract the tar archive into directory named `drupal’
$ tar --strip-components=1 -C drupal -zxv -f drupal-5.1.tar.gz
$ cd drupal
(verify that you actually have Drupal source correctly extracted:
`index.php’ and `INSTALL.txt’ should be here

$ git-init
Initialized empty Git repository in .git/

$ git-add .
$ git-commit -a -m “Drupal 5.1 imported”
[…]
 create mode 100644 themes/pushbutton/tabs-option-off.png
 create mode 100644 themes/pushbutton/tabs-option-on.png
 create mode 100644 update.php
 create mode 100644 xmlrpc.php
$ git-tag drupal-5.1
Now you have a working copy, and a repository in `.git/’
  directory.  Verify that using ls.

$ git-status

# On branch master
nothing to commit (working directory clean)

That’s all with the first working copy (drupal) for now.

Importing Drupal modules

Now we shall create second working copy, called drupal-and-modules.

$ cd ~/src

$ git-clone drupal drupal-and-modules
Initialized empty Git repository in /home/alexm/src/drupal-and-modules/.git/
remote: Generating pack…
Done counting 329 objects.
Deltifying 329 objects…
remote:  100% (329/329) done
Indexing 329 objects…
 100% (329/329) done
Resolving 32 deltas…
 100% (32/32) done
remote: Total 329 (delta 32), reused 0 (delta 0)

Download and extract several modules (the same versions that are used in your current installation), and commit them to Git.

$ cd drupal-and-modules
$ cd modules
$ wget http://ftp.drupal.org/files/projects/adsense-5.x-1.6.tar.gz

$ tar zxfv adsense-5.x-1.6.tar.gz
[…]
adsense/po/adsense.pot
adsense/po/adsense_help-inc.pot
adsense/po/general.pot
adsense/po/ru.po
adsense/LICENSE.txt
$ git-add adsense
$ git-commit -m “Adsense 5.x-1.6 imported” adsense
[…]
 create mode 100644 modules/adsense/po/adsense.pot
 create mode 100644 modules/adsense/po/adsense_help-inc.pot
 create mode 100644 modules/adsense/po/general.pot
 create mode 100644 modules/adsense/po/ru.po

Repeat the “wget, tar, git-add, git-commit” routine with other modules (e.g., google_analytics and print).

So, our second working copy contains Drupal and modules. We may verify it in several ways.

a) We may look at the entire log of our changes, using git-log.

$ git-log
commit 1fb029d1466f1d3ee405bbee3025640f0add0c90
Author: Alexey Mahotkin <alexm@mynd.rinet.ru>
Date:   Sun Jul 29 02:32:29 2007 +0400

    Print 5.x-1.1 imported

commit 542fbb792f68523120158bf8a65cb754d08ef906
Author: Alexey Mahotkin <alexm@mynd.rinet.ru>

Date:   Sun Jul 29 02:31:40 2007 +0400

    Google Analytics 5.x-1.2 imported

commit 5c9a015a0070971f8380a4a2aa452363f1437f05
Author: Alexey Mahotkin <alexm@mynd.rinet.ru>
Date:   Sun Jul 29 02:23:20 2007 +0400

    Adsense 5.x-1.6 imported

commit b2d7641c66ff6a597b822745f5148e406ef86913
Author: Alexey Mahotkin <alexm@mynd.rinet.ru>
Date:   Sun Jul 29 02:08:27 2007 +0400

    Drupal 5.1 imported

b) We may look at the brief list of files that was added and committed in that working copy, using git-diff and diffstat.

$ git-diff drupal-5.1 | diffstat
 adsense/LICENSE.txt                      |  274 +++++++
 adsense/README.txt                       |   96 ++
 adsense/adsense.info                     |   10
 adsense/adsense.install                  |   55 +
[…]
 print/print.info                         |    8
 print/print.module                       |  382 +++++++++
 print/print.node.tpl.php                 |   53 +
 print/print.profile.tpl.php              |   44 +
 37 files changed, 6206 insertions(+)

c) We may see that git-status shows all the .tar.gz files which we downloaded. They are not needed and could be deleted.

Local customizations of Drupal source code

Finally, we create the third working copy, drupal-production, and populate it with our local modifications (or just copy files from existing installation here).

$ cd ~/src
$ git-clone drupal-and-modules drupal-production
Initialized empty Git repository in /home/alexm/src/drupal-production/.git/
remote: Generating pack…
Done counting 380 objects.
Deltifying 380 objects…
Indexing 380 objects… done
 100% (380/380) done80) done
remote: Total 380 (delta 47), reused 326 (delta 32)
 100% (380/380) done
Resolving 47 deltas…
 100% (47/47) done

Now, copy your existing installation into drupal-production, or just edit all the necessary files. In the future, you will edit the files in that working copy as needed.

NB: important security issue: you have to edit the .htaccess file and add the following line (in bold) near the end, before the final RewriteRule:

  #RewriteRule module.php index.php?q=%1 [L]

  RewriteRule ^\.git - [F]

  # Rewrite current-style URLs of the form ‘index.php?q=x’.
  RewriteCond %{REQUEST_FILENAME} !-f
  RewriteCond %{REQUEST_FILENAME} !-d
  RewriteRule ^(.*)$ index.php?q=$1 [L,QSA]

</IfModule>

This is needed to protect the repository which you may accidentally (or deliberately) put on the Web. Without this line it is possible for the attacker to learn your MySQL login and password, which would be unfortunate.

If you have new files in your installation, add and commit them to git, using git-status:

$ git-status
# On branch master
# Untracked files:
#   (use “git add …” to include in what will be committed)
#
#       NEW-FILE


$ git-add NEW-FILE

Now you may take a look at what is actually changed in your existing installation (and/or what edits you’ve just made), using git-diff. Our own changes would look slightly like this:

$ git-diff
diff –git a/.htaccess b/.htaccess
index 46232a7..cf15272 100644
— a/.htaccess
+++ b/.htaccess
@@ -74,6 +74,9 @@ DirectoryIndex index.php
   # the rewrite rules are not working properly.
   #RewriteBase /drupal

+  RewriteRule ^index\.html / [L,R=301]
+  RewriteRule ^books\.html /books/ [L,R=301]
+
   # Rewrite old-style URLs of the form ‘node.php?id=x’.
   #RewriteCond %{REQUEST_FILENAME} !-f
   #RewriteCond %{REQUEST_FILENAME} !-d
diff –git a/css/styles.css b/css/styles.css
new file mode 100644
index 0000000..25cf3c4
— /dev/null
+++ b/css/styles.css
@@ -0,0 +1,8 @@
+body {
+       background: white;
+       margin-left: 2em;
[… etc., etc… ]

Now, you have to commit all your changes. You may either commit everything entirely, with a single message:

$ git-commit -a -m “Imported existing installation”
Created commit 2d5853b: Imported existing installation
 3 files changed, 4 insertions(+), 2 deletions(-)

Upgrading Drupal

We’re just few steps away from the actual upgrade of your Drupal installation. It will almost certainly be easier than initial preparation!

Our plan for upgrading is as follows:

• commit new version of Drupal to the drupal working copy (risk-free);
• automatically merge changes to the drupal-and-modules working copy (almost certainly risk-free);
• automatically merge changes to the drupal-production working copy (difficulty will depend on the amount of your changes and amount of changes between versions;

Upgrading Drupal, pt. I, drupal

We simply unpack new Drupal archive into the first working copy:

$ cd ~/src
$ wget http://ftp.drupal.org/files/projects/drupal-5.2.tar.gz

$ tar --strip-components=1 -C drupal -zxv -f drupal-5.2.tar.gz
$ cd drupal

(verify that you have correctly extracted new Drupal source by
  looking into version number in CHANGELOG.txt)

$ git-status
(if this shows any “Untracked files”, you should add them to the
  commit with git-add)

If you are interested, you may look at the exact changes made by Drupal developers by executing git-diff.

Let’s commit new version:

$ git-commit -a -m “Drupal 5.2 imported”
Created commit f1aa870: Drupal 5.2 imported
 90 files changed, 859 insertions(+), 511 deletions(-)
$ git-tag drupal-5.2

We are ready to move to the next part.

NB: There is one more reassuring thing to tell: Git combines the important property of immutability of history with the ability to controllably rollback commits without leaving almost any trace. That means that if you mistakenly committed something that shouldn’t have been, you could use

$ git-reset –soft HEAD^
(this rolls back last active commit)

$ git-checkout -f
(this reverts the working copy to a previous (good) commit

Upgrading Drupal, pt. II, drupal-and-modules

Let’s merge changes into second working copy, drupal-and-modules:

$ cd ~/src
$ cd drupal-and-modules
$ git-pull
remote: Generating pack…
remote: Done counting 269 objects.
Result has 135 objects.
remote: Deltifying 135 objects…
 100% (135/135) done: ) done
Indexing 135 objects…
remote: Total 135 (delta 103), reused 0 (delta 0)
 100% (135/135) done
Resolving 103 deltas…
 100% (103/103) done
103 objects were added to complete this thin pack.
* refs/remotes/origin/master: fast forward to branch ‘master’ of /home/alexm/src/drupal/
  old..new: b2d7641..989fff6
Merge made by recursive.
 .htaccess                                     |   30 +++-
 CHANGELOG.txt                                 |   18 ++-
 INSTALL.txt                                   |   12 +-
[…]
 themes/garland/style.css                      |   14 ++-
 themes/garland/template.php                   |    2 +
 update.php                                    |   12 +-
 90 files changed, 859 insertions(+), 511 deletions(-)

You may now verify by looking into CHANGELOG.txt that this working copy was upgraded to Drupal 5.2. You may also see that all your extra modules: adsense, google_analytics, and print are still there.

As in the previous case, you may roll-back this git-pull operation with git-reset –soft HEAD^ and git-checkout -f.

Upgrading Drupal, pt. III, drupal-production

Upgrading the third and final working copy is exactly the same as in the previous case.

The only risk is that your changes may conflict with changes made by Drupal developers. Conflict is a technical term – there is a simple and straight-forward procedure resolving this commit. Next subsection is dedicated to this.

$ cd ~/src
$ cd drupal-production
$ git-pull
Auto-merged includes/common.inc
Auto-merged includes/locale.inc
Merge made by recursive.
 .htaccess                                     |   30 +++-
 CHANGELOG.txt                                 |   18 ++-
 INSTALL.txt                                   |   12 +-
[…]
 themes/garland/print.css                      |    2 +
 themes/garland/style.css                      |   14 ++-
 themes/garland/template.php                   |    2 +
 update.php                                    |   12 +-
 90 files changed, 859 insertions(+), 511 deletions(-)

The first two lines of git-pull output show that your changes were automatically merged with changes made in Drupal 5.2. You may wish to look at those files and make sure they look ok.

Thus, we are done. You should probably test your new upgraded Drupal code before actually putting it into production. Now we could just put this working copy to directory where web-server will see it.

Upgrading Drupal, pt. IIIb: resolving conflicts

You may skip this session on first reading

Sometimes merging can lead to the textual conflicts, which should be intelligently resolved by a human being.

$ cd ~/src
$ cd drupal-production
$ git-pull

Auto-merged .htaccess
CONFLICT (content): Merge conflict in .htaccess
Auto-merged includes/common.inc
Auto-merged includes/locale.inc
Automatic merge failed; fix conflicts and then commit the result.

First rule of resolving conflicts: DON’T PANIC.

You may always either restore from backup or execute the command git-checkout -f which will restore from the previous (good) commit.

Let’s look at the conflicting file .htaccess, looking for the lines that start with <<<<<<<, =======, and >>>>>>>, called “conflict markers”. The conflicting fragment may look like this:

<<<<<<< HEAD:.htaccess
  RewriteRule ^\.git - [F]

  # If your site can be accessed both with and without the prefix www. you
  # can use one of the following settings to force user to use only one option:
=======
  # If your site can be accessed both with and without the 'www.' prefix, you
  # can use one of the following settings to redirect users to your preferred
  # URL, either WITH or WITHOUT the 'www.' prefix. Choose ONLY one option:
>>>>>>> f1c6337a3a0ab4860578177754358e29186e425d:.htaccess

First part of this fragment is YOUR code, the second part is the code from Drupal. We have to decide which part (or combination thereof) we leave in the file. We have four choices:

• leave our code, removing conflict markers and second part;
• leave Drupal code, removing conflict markers and first part;
• leave some combination of both parts, removing conflict markets and editing both parts;
• remove both parts and conflict markers altogether;

In this case we will go the hardest way, leaving our code (RewriteRule directive) and new version of comment from Drupal code (second part of conflicting fragment).

We remove conflict markers and edit both parts, so that the following remains:

  RewriteRule ^\.git - [F]

  # If your site can be accessed both with and without the 'www.' prefix, you
  # can use one of the following settings to redirect users to your preferred
  # URL, either WITH or WITHOUT the 'www.' prefix. Choose ONLY one option:

We mark the conflict as resolved, using

$ git-add .htaccess

Then as usual, we look at the files to verify that they are ok, and commit the results of merge:

$ git-commit -a -m “Drupal 5.2 merged”
Created commit 1fc2102: Drupal 5.2 merged

Links

On Git:

• Git homepage;
• Git documentation, including the following pages for every tool we’ve used in this article:
  • git-init
  • git-add
  • git-commit
  • git-tag
  • git-status
  • git-clone
  • git-log
  • git-diff
  • git-reset
  • git-checkout
  • git-pull
• Git Documentation Wiki: tutorials, handbook and HOWTO documents;
• Articles on Git in the Version Control Blog;

Drupal and extra modules:

• Drupal home;
• Drupal 5.2 release announcement;
• Adsense module for Drupal;
• Google Analytics module for Drupal;
• Print Friendly Pages for Drupal;

Version control in general

• Version Control Blog.

Final words

We successfully use the method described above to upgrade our Drupal installation. If you have questions, comments or suggestions, please comment here or write us to squadette@gmail.com, we’ll try to help.

Please consider digging this article or adding points at reddit if you find it useful.

If you’ve moved your Drupal installation to Git, taking inspiration from this article, please consider linking here from your blog or home site.

Thank you!

Recent posts on similar topics

• Dave Dribin: "Choosing a Distributed Version Control System" - February 10th, 2008
• AccuRev streams vs branches - January 14th, 2008
• Robin Luckey: "The World's Oldest Source Code Repositories" - October 18th, 2007
• Karl Fogel, Ben Collins-Sussman, on distributed version control - October 11th, 2007
• Tim O'Reilly: Why Congress Needs a Version Control System - October 7th, 2007

This entry was posted on Thursday, August 2nd, 2007 at 2:17 am and is filed under GIT, branching and merging, Use cases, Drupal. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.