Content Protection is paramount for broadcasters and content owners as consumers rapidly devour content across the open internet. As well as the ever-prevalent challenge of delivering content across multiple devices, resilience and robustness are vital in enabling content owners to remain competitive and engaging to their viewers.
At VUALTO, we are continuously working to evolve our DRM infrastructure and working with key infrastructure partners such as AWS, Elastic Search, Spotinst and Ansible is crucial. One of our most recent partnerships has enabled us to enhance the scalability, resilience and robustness of our Multi-DRM solution VUDRM. This partnership evolved after I read the following blog by Spotinst. It offers an intelligent solution which when implemented with VUDRM, would enable us to offer an evolved product/service for our Digital Rights Management clients.
The Proof of Concept provided by Spotinst gave us 3 dedicated master nodes and 8 dedicated data nodes, more than enough to handle all the data we needed long into the future and easily extensible too. Now it needed to become a true part of our VUDRM product. I chose Ansible partly because I have used it before and also because it has modules for both AWS and Spotinst and good documentation. Ansible is effective in developing reliable, repeatable, idempotent orchestrations.
So how did I stick it all together? Here’s an overview of my approach in Ansible (bullet points are the Ansible modules used):
1.Make sure AWS is configured – This is rarely done more than once per cluster.
Get a session
Create a VPC
Create the subnets
Create the Security Group
2. The tricky bit when using Ansible with Spotinst is Spotinst wants to launch AMIs for you, whereas Ansible wants to configure and manage hosts. I could have let Spotinst create my hosts and then used Ansible to identify them and configure them how we needed. This got complicated particularly for the common case of Spotinst re-spinning instances which you want to come up quickly. Instead, I implemented a build AMI Ansible script that would run up an instance in AWS, call any another Ansible script to provision it, and then save the instance as a new AMI. This means I can create provisioning Ansible scripts to create AMIs for anything I want Spotinst to run in an Elastigroup.
Build AMI – Based off the default AWS Ubuntu 18.04 AMI
3. Now I needed to do all the things to actually create an ElasticSearch cluster on an Ubuntu image and call it with the build AMI script.
Install and configure Elastic Search
Add and mount extra disks (data nodes only)
Install and configure authentication
Configure log rotation
4. Lastly I created a load balancer and then finally I could call Spotinst.
Create the Elastic Group
And there you have it, an easily reproducible, highly available, resilient cluster.
But it’s never that easy!
Rebuilding the AMI each time you want to make changes to the cluster is slow… really, really slow. There are so many stages:
- Check the AWS environment.
- Request the base Ubuntu image is started.
- Wait for the allocation.
- Update and build everything. Your 1 small change to a config file is finally applied.
- Save the new AMI. Update Spotinst.
- Wait for Spotinst to create the cluster, then find out you’ve added a typo!!!
To improve this, I added a number of shortcuts in the Ansible scripts e.g. if you’re only changing the Spotinst config you don’t need to rebuild the AMI. Also, I got Ansible to check if there is a previous AMI, and start from that. All this adds complexity to the scripts but saves hours in unnecessary rebuilding.
ElasticSearch does not enable security by default, it uses xpack. For this subsystem we did not need any of the extended features of xpack. Alternatively, Search Guard was better placed to give us the features we required. Only a few extra tweeks to the Ansible implementation are required to install and configure Search Guard and some certificates. I also wrote a small API to allow us to manage users but it was just the normal flow of development, nothing too taxing and gave us the security and control we needed.
Continuous development of our DRM infrastructure (VUDRM) and all other VUALTO products is fundamental to us. As a team, we actively research cutting edge solutions to ensure that our offering remains at the forefront. We are driven by offering our clients intelligent and value for money solutions.