Direct Drupal 5 to Drupal 7 Migration in 24hrs

With Drupal 7 around the corner, a lot of Drupal developers and users are surely looking at their lingering still-on-Drupal-5 sites and thinking, "surely I might be able to skip Drupal 6 and upgrade directly to Drupal 7?"

Most all experienced Drupal developers would say (to end-users), "not possible". This is because modules only provide upgrade paths from one major version to the next, and just about all Drupal modules received large rewrites between Drupal 5 and Drupal 6. Without help from module developers, it's not extraordinarily difficult to do a direct Drupal 5 to Drupal 7 migration, but you are going to be building out a lot more things manually.

Of all the projects I'm working on, only my personal blog was still on Drupal 5. As this is rather embarrassing, I figured at least it would be a good experience with direct migration from D5 to D7. Here are my experiences and the approach I used in this upgrade.

  1. Don't bother with upgrading to D6 at all.
    With a jump this big, it seemed like a huge headache to download all of my Drupal 5 modules in their Drupal 6 versions. Especially because some modules have actually released a later D5 version than I was running, I'd actually need to upgrade my D5 modules first, then upgrade to D6, then upgrade to D7. No thanks.
  2. Install a clean Drupal 7 site, clean database.
    You have to start somewhere, so I downloaded Drupal 7 RC2 along with essential modules for my site: Views (and CTools), Webform, WYSIWYG, Mollom, and Devel.
  3. Rebuild the basics.
    This includes building out Taxonomy vocabularies, Content types, and all the fields on those types. Fortunately with modules like Select options, ImageField, and FileField built into core, this goes much quicker than you might expect.
  4. Scale back unneeded modules.
    I couldn't believe some of the modules I'd installed for tiny bits of functionality on my site. I had Date module installed for a single date which is never used for sorting or in Views. I had Link module for entering URLs but no title. I converted both of these to simple text fields using the now-provided core Text module. Other things that made sense for me included converting Upload module to D7's File into a "field_files" field. More on that in step 6.
  5. Throw away your theme.
    My Drupal 5 site was a sub-theme of Zen. Not only is a D7 version of Zen not even out, but API differences in the theme layer make existing tpl.php files pretty much worthless. I started over with everything, including my CSS files but still copy/pasted as needed from my old theme.
  6. Write a custom upgrade module.
    The biggest step in moving from D5 to D7 like this is getting everything that can't be rebuilt manually (like content, files, and comments) over to the new site. For this I wrote a Drupal 7 module that uses Batch API to migrate all my existing nodes and comments to the new site. The basic idea is that my Drupal 7 site would be set up with access to two databases in settings.php, "default" and "legacy". This module would manually select data from the D5 site, construct nodes, files, comments, etc. into objects and call node_save() on the Drupal 7 database. This lets Drupal worry about putting all the information into separate DB tables that the new Field module is prone to use (two database tables per field). I've attached the module I used to migrate my content to this article.
  7. Rebuild your Views.
    If you've upgraded from Drupal 5 to 6 before, you're probably aware that the upgrade path from Views 1 (D5) to Views 2 (D6) was a crazy one. Essentially every view has to be rebuilt into Views 2, but Views provided a tool to help reconstuct these views. If you go straight from Views 1 to Views 3 (D7), don't expect this upgrade tool to work at all.
  8. Work through the bugs.
    At this point, trying to use Drupal 7 for a project is still a dicey bet. The first problems I ran into was a conflict between Drupal core (RC2) and Views (dev branch). Apparently Drupal core changed an API post RC2 (surprise!) that caused Views arguments to break. The solution for me was to upgrade to both dev branches of Drupal 7 and Views 3. If you get stuck, search the issue queues of the module that is giving you trouble. Drupal 7 has a lot more early-adopters than previous versions of Drupal.

So there you go. I started upgrading last night and now I'm composing my first D7 entry on the live site. Obviously this approach requires a signficant amount of technical knowledge. Even for experienced Drupal developers, writing a Drupal 7 module that migrates all content can be an intimidating learning experience due to the extensive API changes from Drupal 6. Nonetheless I hope that my own experience in upgrading my relatively simple blog might help others in facing the challenging task of bridging 3+ years of Drupal advancement in a single sweep.

Attached is my module that provides the following:

  • A form at admin/content/custom-import for performing the import.
  • Migration of D5 Upload module files to a File module field named "field_files".
  • Migration from file paths to D7 stream wrapper paths (URIs).
  • Migration of D5 Nodes including converting "Stories" to "Articles", Upload to File fields, Link module to Text module, and Taxonomy terms into Term Reference Fields (note that I manually migrated Vocabularies and terms prior to creating the upgrade module).
  • Migration of D5 Comments to D7 Comments with the comment field as a D7 Field.
  • While all content is recreated on a clean Drupal site all nodes, files, and comments maintain their same identifiers.

Happy upgrading!

Comments

Nice work. And thanks for the writeup. I agree that skipping a major version of Drupal is a reasonable way to proceed. Folks should look at migrate.module to help them do that. It is a bit more structured than a one off import thats custom built. Migrate is what Economist, Examiner, etc. use. So it can handle big and little projects.

I've used Migrate a few different times and found that for the most part it's more learning than it's worth. Either approach (entirely custom or migrate) still requires coding. In D7 the new consistency in save functions (like comment_save()) make custom scripts even easier than before. Unless I'm doing a lot of migrations (or continuous pulling from another system), the custom approaches take me significantly less time. Not to mention I'm not dependent on the stability of other modules for my migration.

I used your 'custom import' code as a starting point for migrating a fairly simple Drupal 5 site to Drupal 7, and after some tinkering, it worked like a charm. Thank you!

I've used this approach successfully on a 4.7 to 6 migration. Some caveats, but generally better than rebuilding the site as D5 which I was never going to use.

Hey Gerhard, is there any possibility you could contact me and share more information about your 4.7 to 6 migration? I'm trying to figure out options for migrating a 4.7 site to Drupal 7. You can contact me at ben at quilted.coop.

The custom module is a huge life/time saver. Mucho Thanks.

Very glad I found this - web surfing always gets me at least one good wave. :) Thanks, Nate.

Thanks for the module and this page, but I have not been able to make it work. I get the following error message: An AJAX HTTP error occurred. HTTP Result Code: 500 Debugging information follows. Path: /?q=batch&id=5&op=do StatusText: Service unavailable (with message) ResponseText: PDOException: SQLSTATE[42S02]: Base table or view not found: 1146 Table 'cando_default.files' doesn't exist: SELECT COUNT(fid) AS count FROM {files} files; Array ( ) in custom_import_batch_files() (line 86 of /var/www/candobetter.local/sites/all/modules/custom_import/custom_import.module). (ends) My best understanding is the databases are supposed to be configured in ./site/default/settings.php Could someone please tell me if I am wrong? Assuming I am right, here is my database configuration: $databases['default']['legacy'] = array( 'driver' => 'mysql', 'database' => 'cando_legacy', 'username' => 'candolocal', 'password' => '#4^7dsa%8-', 'host' => 'localhost', 'prefix' => '', ); $databases['default']['default'] = array( 'driver' => 'mysql', 'database' => 'cando_default', 'username' => 'candolocal', 'password' => '#4^7dsa%8-', 'host' => 'localhost', 'prefix' => '', ); 'default' is the largely empty mysql database into which I intend to import 'legacy'. Could someone tell me if they can spot an error above, or, if not, what the cause of the problem might be?

Hey Daggett, The database string should be like this: $databases = array ( 'default' => array ( 'default' => array ( 'database' => 'newDB', 'username' => 'root', 'password' => 'root', 'host' => 'localhost', 'port' => '', 'driver' => 'mysql', 'prefix' => '', ), ), 'legacy' => array( 'default' => array ( 'database' => 'oldDB', 'username' => 'root', 'password' => 'root', 'host' => 'localhost', 'port' => '', 'driver' => 'mysql', 'prefix' => '', ), ), );

Is there a way to import drupal 5 user information (username, password, email and profile details) and then exporting to drupal 7?

This helped me a lot. I've never used the batch API before and this code was great for me. Looking at the example code, I think each batch method (i.e. custom_import_batch_comments) is only called once. When I'd expect it to be called multiple times. The $limit value is set, but never actually used in sql, so it runs over the full set.

hi i'm New to drupal.can you provide me the step by step process to migrate this

I got the same question how did you set up so Drupal can access the legacy database, I'm kind of confuse how to implement that.

sorry, couldn't make it work.....drupal 6 to 7

First a big thanks for your script. Adapted it to my site and mostly works fine. Only with the comments I have some troubles as table "node_comment_statistics" doesn't get populated and so $comment_count gives an error. For sure I can rebuild it manually with devel module but still don't have information about last comments. Any idea how to fix that?

My opinion is that not everyone will do this. The procedure is quite cumbersome and not only for beginners. May need to consider this. www.miere-bucovina.ro

I have installed custom_import to a fresh D7 installation but I can't see any option available in the modules page connected to this module. Am i doing something wrong? Is there any information on how to use this module somewhere?

I really impressed of this informative post your share . This helped me a lot. Let us to visit my site. http://bit.ly/uRo0bj

Thanks a billion for this Quicksketch. I am using it now to migrate my blog from 5 to 7 and it is a real time saver.

Hi Quicksketch! Did you give Permalink the guide on how to implement this? Can you please share to me too. Thanks in advance!

An AJAX HTTP error occurred. HTTP Result Code: 500 Debugging information follows. Path: /html/web/MyD7Copy/batch?id=16&op=do StatusText: Service unavailable (with message) ResponseText: PDOException: SQLSTATE[42S02]: Base table or view not found: 1146 Table 'clientcopyfeed.content_field_image' doesn't exist: SELECT f.* FROM {content_field_image} f WHERE (vid = :db_condition_placeholder_0) ; Array ( [:db_condition_placeholder_0] => 52 ) in custom_import_batch_nodes() (line 181 of /var/www/html/web/MyD7Copy/sites/all/modules/contrib/custom_import/custom_import.module).Uncaught exception thrown in shutdown function.PDOException: SQLSTATE[42S02]: Base table or view not found: 1146 Table 'clientcopyfeed.batch' doesn't exist: UPDATE {batch} SET batch=:db_update_placeholder_0 WHERE (bid = :db_condition_placeholder_0) ; Array ( [:db_update_placeholder_0] => a:13:{s:4:"sets";a:1:{i:0;a:14:{s:7:"sandbox";a:3:{s:8:"progress";i:0;s:12:"current_node";i:0;s:3:"max";s:5:"56080";}s:7:"results";a:0:{}s:7:"success";b:0;s:5:"start";d:1328618817.522551059722900390625;s:7:"elapsed";i:0;s:5:"title";s:14:"Importing data";s:8:"finished";s:28:"custom_import_batch_finished";s:12:"init_message";s:24:"Initializing.<br/>&nbsp;";s:16:"progress_message";s:29:"Completed @current of @total.";s:13:"error_message";s:22:"An error has occurred.";s:3:"css";a:0:{}s:5:"total";i:3;s:5:"count";i:2;s:5:"queue";a:2:{s:4:"name";s:17:"drupal_batch:16:0";s:5:"class";s:10:"BatchQueue";}}}s:16:"has_form_submits";b:0;s:10:"form_state";a:3:{s:10:"programmed";b:0;s:7:"rebuild";b:0;s:8:"redirect";N;}s:11:"progressive";b:1;s:11:"current_set";i:0;s:3:"url";s:5:"batch";s:11:"url_options";a:0:{}s:10:"source_url";s:27:"admin/content/custom-import";s:8:"redirect";N;s:5:"theme";s:5:"seven";s:17:"redirect_callback";s:11:"drupal_goto";s:2:"id";s:2:"16";s:13:"error_message";s:97:"Please continue to <a href="/html/web/MyD7Copy/batch?id=16&amp;op=finished">the error page</a>";} [:db_condition_placeholder_0] => 16 ) in _batch_shutdown() (line 537 of /var/www/html/web/MyD7Copy/includes/batch.inc).

An AJAX HTTP error occurred. HTTP Result Code: 500 Debugging information follows. Path: /batch?id=4&op=do StatusText: Service unavailable (with message) ResponseText: PDOException: SQLSTATE[42S22]: Column not found: 1054 Unknown column 'f.nid' in 'on clause': SELECT f.*, n.created AS created FROM {files} f INNER JOIN {node} n ON f.nid = n.nid WHERE (fid > :db_condition_placeholder_0) ORDER BY f.fid asc; Array ( [:db_condition_placeholder_0] => 0 ) in custom_import_batch_files() (line 90 of C:\vhosts\test8\sites\all\modules\custom_import\custom_import.module).

Thanks for the informative writeup. I've just installed Drupal 7 and have activated your module. But I'm not able to find it anywhere after I activate it. Could you please tell me how do I go about using the module?

Speak with your own friends http://www.onlyhervelegerdresses.com concerning your own plans to fix.

From what I've read, the Migrate module makes it pretty easy to do imports with node / user references...since there's a bit of a chicken and egg problem with old nids / new nids. Is it worth it to adjust your code to implement something like this? Or do you think migrate is the way to go?

From what I've read, the Migrate module makes it pretty easy to do imports with node / user references...since there's a bit of a chicken and egg problem with old nids / new nids. Is it worth it to adjust your code to implement something like this? Or do you think migrate is the way to go? http://dryscalphomeremedies.us http://mymorningsicknessremedies.net http://constipationhomeremedies.us http://myburnthefatfeedthemusclereview.com/ http://myfatburningfurnacereview.us/ http://myeatstopeatreview.us/ http://xn--hyrafilmpntet-kfbn.nu/ http://xn--bralnutanskerhet-4nbl.nu/ http://xn--lnapengarmedbetalningsanmrkningar-41ck.nu/ http://ireviewgaminglaptops.com http://howtopottytrainatoddler.us

Looks like you blog has been plagiarized by this person here - http://sscreativa.com/sstech/?p=15 P.S: Your post has been quite helpful. Thank you!

Folks should look at migrate.module to help them do that. It is a bit more structured than a one off import thats custom built. Migrate is what Economist, Examiner, etc. use. So it can handle big and little projects.website localization

I'm hoping to use the Drupal-to-Drupal Data Migration module soon! http://drupal.org/project/migrate_d2d

I wish I would have found this a month ago. Thanks for all your work in Drupal.

Add new comment

Plain text

  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
  • Allowed HTML tags: <em> <strong> <cite> <blockquote> <code>