Welcome to This Week in DevOps where I bring you the top news in the DevOps world every week. Don’t forget to Subscribe if you’re reading this on the Blog to get this info straight to your inbox each week. If you’re receiving this via email, thanks for being a subscriber!
AWS
AWS::CloudFormation – Import Existing Resources into a CloudFormation Stack
This is a huge feature for anyone that regularly works with existing systems and CloudFormation on AWS. The ability to import resources instead of recreating new ones is a huge timesaver.
This has been one of the top requested CloudFormation features for a while now so it’s good to see AWS responding to customer feedback. Here’s hoping it works as well as advertised since imports in other tools like Terraform can be a bit hit or miss.
AWS::DataExchange – The new AWS DataExchange was launched this week
This is perhaps the biggest announcement this week since it’s a new AWS product aimed at data and machine learning scientists and engineers. While most of this data was available elsewhere, licensing wasn’t always clear or straightforward and accessibility/formatting was always an issue.
Having this data on AWS with easy to understand licensing and payment models is a huge step forward for organizing these data sets. Whether the data sets will be easy to export for use outside of AWS is unclear at this point, but hopefully AWS will keep things as open as possible.
It looks like many sets will be available for a 1 month free trial with monthly or possibly yearly licensing thereafter. This should be great for startups experimenting with various models and data sets as it lowers the barriers to entry. Smart move on the part of AWS since it will likely result in long term subscriptions.
AWS::EMR – Insert, Update, Delete Data on S3 with Amazon EMR and Apache Hudi
Another data related update that should be interesting for anyone using EMR. This appears to be prompted by data privacy regulations such as GDPR and CCPA which is forcing companies to think about record level delete from their data lakes.
Hudi was developed at Uber before being open sourced and supports Spark, Hive, and Presto integrations on EMR out of the box.
AWS::SystemsManager – Now supports custom and prebuilt automation sequences
Systems Manager now allows you to run pre-built automation scripts as well as custom sequences written in Python or Powershell. The pre-built scripts seem to cover a lot of common use cases such as putting an ASG (Auto Scaling Group) into standby mode while the ability to create custom scripts means you can cover almost any use case.
While I would still bet on tools like Terraform to supercede the need for this in the long run, this is great if you are still running or supporting lots of custom or unmanaged infrastructure.
GoogleCloud
GoogleCloud::CloudRun – Managed Knative based Serverless framework is now General Availability
Based on Knative, CloudRun is a fully managed serverless framework for GKE. CloudRun also supports Anthos and has a free trial tier until May 14, 2020. It’s unclear what the pricing will be after the free trial period.
Google also announced that they have partnered with CircleCI, CloudBees, Datadog, Dynatrace, GitLab, JFrog, New Relic, Octarine, Palo Alto Networks, PureSec, Sumo Logic, and Sysdig. This means that existing systems using these tools will be able to integrate with CloudRun without major changes.
GoogleCloud::Batch – Batch for GKE is now in preview
Batch allows optimization of workloads at scale. It supports Autoscaling and GPUs out of the box so it should be useful for high throughput data jobs. Google seems to be working closely with High Performance Computing vendors to enable integrations with Batch on GKE so keep an eye on out for future announcements.
Azure
Azure::ContainerRegistry – Preview of diagnostic and audit logs
Azure::ContainerRegistry – Preview of repository scoped permissions
2 new preview announcements this week for Azure Container Registry. First up is diagnostic and audit logs which will help open the enterprise market and along with that we have an announcement for repository scoped permissions which is also a feature clearly aimed at enterprise.
This enterprise focus will be surprising to nobody since Azure is owned by Microsoft. This focus does seem to pay off for them as they have captured a large portion of the enterprise market. Hopefully they’ll also continue to roll out innovative features and work on useability and reliability as these are common pain points brought up by Azure users.
Hashicorp
Hashicorp::Vault – Announcing the release of version 1.3
Hashicorp::Vault – Announcing the release of Vault Helm Chart 0.2.0
The focus this week at Hashicorp seems to be on Vault as they’ve announced a new version of Vault shortly after the announcement of a new Vault Helm Chart. The latest Vault release includes some open source features and some Enterprise Features available to paid users only. These include Active Directory enhancements, Entropy Augmentation, Oracle support and a new CLI called Vault Debug which enhances debugging for Vault implementations.
Congrats you made it to the end! If you enjoyed this then please email me at cfornari@startopsgroup.com me and let me know what else you would like to see included in these weekly updates. This includes companies to watch or specific technologies or concepts you’d like to hear more about.
Leave a Reply