Launching the Africa Media Online Preservation Cloud

Creating a secure and fail-proof hosting environment for digital heritage is the problem we have been working to solve for years. In this past year, however, we have been able to take significant strides in this regard. We are now celebrating with the launch of what we are calling our Preservation Cloud.

Dell servers with disk arrays in the AMO Server Room. The room is temperature controlled, has biometric access and 24-hour monitoring, has a full fire suppressant system and has backup power and backup internet and the data is automatically backed up to an off-site location.

Everyone speaks about “The Cloud” or “Cloud-based computing” but many of us don’t know exactly what that is. All we know is that our data is out there in the ether somewhere and not on our own computer. While that is certainly true, what we often fail to understand is that every bit and byte of data needs to reside on some computer somewhere and the computers have to have a physical location. Most commonly “Clouds” are made up of servers connected to disk arrays that sit in a data centre somewhere are then made accessible on the internet. That data may be synchronised across a number of servers sitting in data centres in different parts of the World.

The automated fire suppressant system can be seen in the background and the back of one of the server racks in the foreground

The challenge we had at Africa Media Online is how do we build a “cloud” where data is never deleted. For most cloud-based computing, data is not kept for the long term. Like a desktop computer, one is constantly creating files and often changing them or deleting them. It tends to grow over time, but there is no requirement for the files to be secured and maintained exactly as they are for generations to come. That, however, is the challenge of digital archiving. To archive a digital file one wants to know that it will be maintained exactly as it is for the foreseeable future and migrated at the appropriate time to keep with new standards. Keeping every file exactly as it is when first loaded means that storage capacity has to grow and grow and grow… There is no end to its growth. And one needs to know that the files are not being corrupted in any way or overwritten or deleted or catch a virus or fall prey to ransomware or become obsolete.

So over the past year, we have taken our core of servers and disk arrays and we have grown the infrastructure around it ensuring there is backup power, backup internet, and multiple backups of the data. We have also established an external web server and in time that will be synchronised with our existing internal web server to ensure seamless uninterrupted service. Right now this is what our Preservation Cloud looks like with our MEMAT digital asset management system running on top of it.

What we call the Infrastructure Layer in this diagram is the Preservation Cloud. The first column shows our external web server that resides in a data centre in Midrand, Gauteng. This is the server that is browsed when one is on a MEMAT site. The second column refers to what is in the Africa Media Online Server Room. This resides in a separate building separated by a road from the Backup Room (see the third column) which is in our main offices. These rooms are connected by a fibre cable. Backups then, are automatically offsite. The Server Room is fitted with biometric access and monitoring, fire suppressant systems and other security measures. It also has an inverter and batteries that provide 12 hours of backup power and a backup generator that can support the batteries when they are running low. The Server Room hosts the main storage servers, the internal web server and the processing server that is used to process material that is loaded to the MEMAT Digital Vault. The Backup Room has our LTO tape libraries including the capability to back up to both LTO6 and LTO7 tapes. In addition we have a system to burn data to Bluray disks to give us an offline backup. LTO tapes and Bluray disks, then, are stored in a safe, 20 km away in a rural area (column 4).

The Africa Media Online backup room with the inverter and backup power batteries in the foreground the the server rack with a switch and LTO tape library (not visible) and Bluray burner. This room is temperature and humidity controlled.

On top of the Infrastructure Layer is the Operating System layer. Like a desktop computer that may have Mac OS or Windows as its operating system, these servers have a common Open Source Linux-based operating system. On top of that sits the MEMAT layers of Asset Management Layer (AML), Cataloguing Layer (MEMAT Metadata App) and Presentation Layer (PL). In this way, MEMAT together with our Preservation Cloud provide a solid digital repository for the long-term preservation of your digital collections as well as reliable ready access.