ArchiVault: Replacements System
There are times when files need to be replaced. Usually this is when a better version of the file becomes available. To do this we need a Replacements System where identified files are replaced by an updated version.
ArchiVault and Memat: Embargo System
The Embargo System will enable administrators to embargo select media files for a period of time or indefinitely. They will also be able to allow certain users to view embargoed files.
Memat: Require Recipients of Shared Selections to Signup or Login
Part of growing a user base for a digital archive is to ensure that potential users sign up in order to access material enabling usage momentum to grow as users share with potential users
For some time now we have been working on an integration between PageFlow and Memat allowing a further curation capability adding to the existing galleries, timelines and blogs.
Memat: One Click Signup
We want to make it super easy to sign up to a digital archive using one’s Facebook, LinkedIn or Google account and we want to simplify the signup form for those signing up with email
ArchiVault and Memat: Advanced Image Search
Advanced image search enabling searching by time, place, keywords, similarity and other features
ArchiVault and Memat: Boolean Search Capability
The ability to use Boolean search operators will make more advanced search a possibility enhancing the findability of content.
ArchiVault and Memat: CSV Metadata Export and Import
The ability to export the metadata associated with a collection as a CSV file so that it can be worked on in bulk and then reuploaded to update the metadata
Preservation Cloud: Migration of Web Servers to AWS
Migration of Preservatio web servers to Amazon Web Services to be able to take advantage of AWS technologies
Memat: Elementor Integration
Elementor is an intuitive WordPress website builder that will give our Memat clients broad latitude to design their the web interface to their Preservatio system. No more needing to keep to a prescribed web interface.
We want to create a system where users can leave comments against media items. A submitted comment is sent to the administrator who can approve or reject the comment. If approved, the comment will be made available alongside the media item.
ArchiVault: Generation System
Certain digitisation and digital processing projects are for clients who do not have a Memat site so material is not being read into the Digital Vault but simply processed to generate derivatives and then made available for delivery. This system needs to be in place to enable that.
ArchiVault: Delivery System
Once material is ingested into the Digital Vault, the ability to deliver projects to clients is dependent on a Delivery System where digital files can be read out for transfer to a client hard drive for delivery.
ArchiVault: Uploads System
Preservatio preserves and makes available preservation quality digital files. The files, then, are large and require significant computational resources to deal with. Uploading these files has traditionally been a bottle neck for us with clients having to wait while the files are queued to upload. On top of that, the only people who can operate the system are the IT Team. For the rest of the AMO Team, particularly those interfacing with clients, progress on uploads has been something of a black box where we can see no detail and so find it hard to manage expectations. This development is to change that, to put a user interface in place that allows the Production Team to manage uploads themselves and to be able to view progress along the way. This is the first of a number of developments toward putting uploads into the hands of our clients and even their users.
The Calendar allows those with administration status to add galleries to dates on a calendar. This furthers the curation capability of Memat adding to galleries, timelines, blogs and stories.
Memat: Editable Browse Display
This development has enabled the ability of administrators to determine the first level to be displayed in the Browse page. Level 1 is the name of the digital archive and Level 2 the name of the fonds or collections within the archive. Administrators can now make Level 2 the default starting point for the Browse page.
ArchiVault: Migrate Storage Fabric to S3 Buckets
Since the days of Memat 3, preservation data has been stored in our Digital Vault in what we called “Security Boxes.” More recently, Amazon Web Services invented what they call the Simple Storage Service (S3) which uses “Buckets”. This is now an extremely widely used and supported protocol so we have deemed it wise to migrate our data from Security Boxes to S3 Buckets.
Preservation Cloud: Consolidate Local Servers
Having commissioned three new remote generating servers we put effort into consolidating and decommissioning a local server and there will be more decommissioning of older infrastructure to follow as we modernize the Preservation Cloud infrastructure.
Preservation Cloud: Commission New Generation Servers in Remote Data Centre
Power issues in Pietermaritzburg as a result of aging municipal infrastructure was making running servers reliably difficult, particularly when those servers need to run through 10s and 100s of thousands of pages to generate derivatives and run actions such as Optical Character Recognition (OCR). We made the decision in January 2021 to commission three new servers in a remote data centre in another city to ensure reliable throughput.
Memat: Ability for Admin to set Universal Viewer Permissions
By default, users need to be logged in to the system in order to view content in the Universal Viewer, that allows the zooming in to photographs or the searching within a document. This development enabled users with Admin status to enable users to view select content in the Universal Viewer without being logged in.
Memat: Admin Ability to Turn Comp Downloads On or Off
Comping images are low resolution versions of files. Prior to this development, only the AMO IT Team could turn this function on or off. Now users with admin status can control whether this option is presented or not.
ArchiVault: IntraDoc Search Multiword Search
When we first produced our new IntraDoc Search functionality, the search itself was limited to a single word. This development ensured that multiple words can be searched together.
Memat: Arrangement Path in Browse Results Clickable
At the top of any browse results page is the display of the archival arrangement. This development enabled elements in this path to be clickable so that new results appear according to the level clicked.
Memat: Information Page Update
On the Memat web interface items returned in search results, browse results or in galleries have an “i” icon that can be clicked on to see further information. This development enabled all associated information to be displayed and further actions to be taken from within the information page.
April - November 2020
ArchiVault: IntraDoc Search Indexing
Having developed our new IntraDoc Search processing pipeline to enable the search within a document online, we began to index collections working systematically through each one for months. The one outstanding collection is the ANC Archive with its upwards of 1.8 million pages.
April - September 2020
Having implemented the Browse system on our client’s Memat web interface, we noticed that the archival arrangement for many collections was not consistent or, in some cases, carefully thought through. We therefore, took it upon ourselves to work through every collection establishing a sustainable archival arrangement and getting every collection in order, an order that can be maintained and extended in future digitisation and digital archiving projects.
Memat: Bulk Download
In implementing a trackable order and downloads system, the original specification did not include the ability to download a number of files at one time. One had to download each one separately, frustrating when one had 10s of files to download. This development fixed that allowing for a zipped folder of files to be downloaded.
While we had web statistics on the system prior to this we had not implemented a user-friendly web interface that allowed system administrators to view and analyze their visitor patterns.
February - June 2020
With rapid development undertaken in the latter half of 2019, the Preservatio system had become unstable with clients experiencing anomalies and our IT Team drowning in a flood of tickets. The crisis brought about new development protocols and an all-out effort starting in the first quarter of 2020 to test and retest every function on every system and to systematically fix every error and as part of that, a ruthless determination to maintain a single code base across all sites. South Africa’s Lockdown gave us space to focus all our effort on this task and by mid-year 2020 the deluge of tickets had become a trickle and all systems were fully operational and behaving as they should.
ArchiVault: IntraDoc Search
In the course of 2019 we had engaged a UK-based company that was on the cutting edge of innovations in the heritage sector to create a system that allowed documents to be searched online using IIF protocols. In spite of the great cost associated with the development of that system, its functioning was poor and the company’s service was even worse. It was a black box that our own team was unable to work on – frustrating for us and really frustrating for our clients. In desperation we developed our own system in a two week period, a system that works more efficiently than the original, costs very much less and most importantly, we can fix ourselves!
Preservation Cloud: Backup Web Server
The Web Server is the server connected to the internet that users are accessing when they visit a Preservatio based digital archive. Originally we hosted our web server in our own data centre in Pietermaritzburg, but with challenges of power and internet, several years ago we outsourced the web server to a database in Johannesburg. Another move to ensure maximum uptime for the digital archives we care for, we commissioned another web server in a different data centre that would mirror the original web server and take over in the eventuality of the primary server going down for any reason.
Memat: Integration of Timeline JS
We wanted to extend the ability of the Memat web interface to curate materials from the digital archive. The Galleries system had been our only curation feature in what we call the Memat CL (Curation Layer) from Memat 1 to 3. Now in Memat 4 we wanted to extend the curation functionality and the first feature in this regard was enabling administrators to build timelines on an Open Source system called Timeline JS. Files from the digital archive can now be placed on the timeline along with explanatory text. This project was funded by the Constitution Hill Trust.
We had developed a browse system on the Memat Metadata App for drilling down through the arrangement of a digital archive. We wanted to bring this functionality to the Memat PL as a new way in which clients could interact with the collection.
Memat: Establish Staging System for testing code
In an attempt to deal with instability issues around code updates on the Memat Presentation Layer, the web interface to Preservatio, we established a Staging System – a test environment where code could be fully tested prior to being released.