Migrating content from TYPO3 to Drupal6

Part -1: Migrating News content

Srijan Technologies Pvt. Ltd., India

November 19, 2009

Srijan Technologies Pvt. Lt., India 1

Introduction
Migrating from one CMS to another is, surprisingly, something that requires doing more often than one would
expect. The reasons for doing so vary from changes in customer requirements to the availability of better
features in a newer/different CMS to programmer preference. One of the crucial stages during such migrations
is the transfer of data across databases without compromising the integrity of the data. In this article, we have
detailed a subset of this problem by explaining how we successfully effected a migration of content from
TYPO3 to Drupal6.

The problem with migrating CMS data

In an ideal world, we would have figured out a common denominator for organizing information, In which case
moving information would be a close to trivial task. However, the effective organization of information is a
topic under research and continuous improvement. This means that the theories to organize information are
numerous and experimental and as a result the techniques and tools available to effect the organization are
also exponentially numerous. Thus we have equally powerful knowledge models, but each effective within
certain boundary conditions.

In such an atmosphere of flux, it becomes necessary to find ways of making information transferable between
concepts and techniques.

REALITY CHECK

To be realistic, we need to understand that while it might be possible to migrate data, the transfer of the
intelligence that creates information from this data might not always be possible to a satisfactory degree.

In this tutorial we will be looking at a subset of the problem. Specifically, let's look at a "simple" case of
migrating NEWS data from TYPO3 to Drupal6.

Background

Kalaallit Nunaata Radioa (KNR), the Greenlandic broadcasting corporation is a radio and TV station funded by
the government of Greenland. Their existing portal running on TYPO3, needed to be migrated to Drupal6.
Drupal was chosen primarily for three reasons:

1. The modules and extensions available for Drupal makes it possible to integrate numerous features
and functions such as photo gallery, Video & audio integration, advanced forum, blog, etc. It was
found that Drupal would meet all the feature requirements with only minimal customization of
modules.

2. The implementation time was estimated to be comparatively less in the case of Drupal.

3. Content Administration is quite easy in Drupal. Drupal's web-based administrative interface allows
even non-technical users to easily manage, organize and publish content. To maintain content
integrity, user roles and permissions can be easily defined to control the access and capabilities of
different user types.

Among other things, KNR wanted to import old news entries from TYPO3 to the new Drupal6 system to
preserve the old content. We (Srijan Technologies) were contracted to implement the complete migration
from TYPO3 to Drupal6.

Srijan Technologies Pvt. Lt., India 2

Process of Migration - How we did it
As in any data migration exercise, the steps we followed to implement the migration were:

 Studying the source system

 Studying the capabilities of the target system

 Identifying and mapping the elements that can actually be migrated

 Exporting the source data

 Transferring intelligence (in the form of relationships) from source to target

 Importing the data into the prepared target system

Step 1: Studying the source system

The TYPO3 source application uses the tt_news module to implement the news feature. The content came in
two flavours, Greenlandic and Danish. This step involved discovering data sources such as database tables
(data types, sizes), file system and the relationships that tie the data together, in the form of taxonomies or
categories. This resulted in a list of content types their attributes and an ER diagram that established data
relationships [Fig 1].

Fig 1: TYPO3 tt_news ER Diagram

Step 2: Studying the capabilities of the target system

Once the organization of the source information has been understood to a reasonable extent, the next logical
step was to understand the system into which the information needed to be moved. This led us to identify the
data structures, models and abstraction concepts that the target system follows. In the current case, Drupal6
was the chosen target system, within which the CCK module could be used to accommodate the data coming
out of TYPO3's tt_news module.

Srijan Technologies Pvt. Lt., India 3

Step 3: Identifying and mapping the elements that can actually be migrated

Based on the first two steps, we then proceeded to create a high-level mapping of the data between the
source and target systems [Fig 2].

Fig 2: Data field mapping

This brought us to a stage where we could create a migration strategy based on what information could be
migrated to what extent and what level of technical expertise would be required to execute the migration.

Step 4: Exporting the source data

The source data could now be exported to a common format that could then be imported into the target
system. In this case, we exported the NEWS data as CSV into a file since the node_import module which we
use in step 7 allows importing of data from CSV or TSV files. The scripts we used to export the news data from
the TYPO3 database can be downloaded here.

The code archive contains the following files:

 class.dblib.php : Holds the functions that are used to query the database and print the extracted data
to a CSV file.

 importnews.php : User level script that can be run from a command line. Uses functions in
class.dblib.php

Step 5: Transferring intelligence

We then prepared the target Drupal6 system to receive the data by installing the required modules, viz.

 Content Construction Kit (cck): To create custom node type to hold news stories

 date: Date/time field type for the CCK module

 imagefield: Provides image upload field for CCK

Srijan Technologies Pvt. Lt., India 4

  filefield: Provides file upload field for CCK

 i18nodeimport: Provides language/translation support to node import. Download module here

Having installed the required Drupal6 modules to receive the data, we created content-types and taxonomies
and categories to make the imported data useful within the new system. The Content Type 'NEWS' was
created with CCK fields for primary "image" and "subheader". Taxonomy was used for "author" and "author
email" to allow for auto-completion of author details from references of authors who already exist in the
database [Fig 3].

Fig 3: Content type creation using CCK

Step 6: Importing the data into the prepared target system

The import of data into the target Drupal6 system was effected using the node_import module. This module
provides a friendly interface to map data fields between the source and target and also to manage the
migration of data. The node_import module allows the import of a set of (drupal) nodes from CSV (Comma
Separated Values) or TSV (Tab Separated Values) file using a “wizard” interface spanning eight steps.

i. Select Content Type created in step 6 to accommodate news data.

ii. Select the CSV file which contains the data to import. Note: You can select a previously uploaded file
or upload a new file now.

iii. Detail the format of the CSV file. The options tell node_import how to interpret the CSV file. Note:
You can select one of the predefined file formats or enter the record separator, field separator, text
delimiter and escape character yourself.

iv. Map the fields in the CSV to the content type fields in Drupal6 by selecting the columns in the CSV file
that contain the data for the field to which it is to be mapped [Fig 4].

Srijan Technologies Pvt. Lt., India 5

 Fig 4: Mapping TSV columns to CCK fields

v. Set some additional options. At the minimum you would need to mention the separator for taxonomy
terms.

vi. Set default values for each content type field in Drupal6. The default value is used when the field is
not mapped to a file column or when the data in the mapped file column is empty for a record.

vii. A great feature that the node_import module provides is the capability to preview the changes before
actually importing the content into the database. It gives us a chance to rectify any errors or work
around them by tweaking the settings in the previous screens [Fig 5].

Fig 5: node_import preview screen

The number of records to preview can also be changed to give a more comprehensive view of the
imported data. Note: Press Reload if you change the number of records to preview.

viii. This is the last chance to verify that the options selected by you are indeed what you want. You can
start the import process by clicking Start import [Fig 6].

Srijan Technologies Pvt. Lt., India 6

 Fig 6: node_import progress screen

Conclusion
While we only looked at a subset of the information migration problem and one solution, the node_import
module in combination with the Content Construction Kit provides powerful features that make migrating to
Drupal easier than it might originally seem.

ABOUT SRIJAN TECHNOLOGIES

Srijan is an award winning IT consulting company engaged in designing and building corporate websites,
portals, intranets and web applications using open source software.

It works with frameworks such as TYPO3, Drupal, Symfony, “Ruby on Rails” and Django, and also deploys and
manages Linux servers that run these applications.

GETTING MORE INFO

If you have any comments/questions about this document please feel free to contact Rashmi on rashmi@srijan.in

For any business enquiries please contact Rahul on rahul@srijan.in

Srijan Technologies Pvt. Lt., India 7