FOSDEM '09 is a free and non-commercial event organized by the community, for the community. Its goal is to provide Free and Open Source developers a place to meet.

Neil Drumm
Day Sunday
Room H.1302
Start time 13:00
End time 14:00
Duration 01:00
Event type Podium
Track Drupal
Language English
Importing data with job queue and import manager

Continuously importing data, of any size or number of sources, needs infrastructure. Neil Drumm wrote Job Queue and Import Manager to queue and manage imports.

He will show how to use these modules to get the basics out of the way.

Neil Drumm will also talk about a few strategies he found successful for data conversion. The presentation will not be able to cover how to get everything into nodes, but will teach how to think about converting data.

At MAPLight, Neil managed importing data from GovTrack, OpenSecrets, the FEC, the Iowa Legislature and other government entities. Some updates happen within 30 minutes of an action in Congress, while others need to be run monthly, or reports as-needed.