Last week, AWS announced its free-of-charge online conference schedule, but seeing as it’ll be an unusual re:Invent in an unusual year, we wanted do something a little different with our “preview.”
We invited some of our top cloud experts to chime in on what they’re expecting from AWS re:Invent 2020 and what they’re hoping for: from trends to product announcements to the sessions.
I already mentioned when AWS announced re:Invent would be virtual this year due to Covid how disappointed I would be to miss out on IRL meetups. That hasn’t changed.
But, I’m finding as November 30 approaches, I’m actually getting excited to be spending time in the online AWS playground seeking out everyone using #reinvent and #reinvent2020 hashtags. Feel free to DM me at @iamondemand or check out our re:Invent- and AWS-related content on Twitter at @weareIOD. I’m happy even to do a virtual reinvent happy hour. Let’s meet up!
In the meantime, read our experts’ takes below!
Bruno Amaro Almeida
Principal Architect, Advisor & AWS APN Ambassador
Far away from its traditional location, under the vibrant lights of Las Vegas, this year’s AWS re:Invent conference will go fully virtual and be three times as long. Spread over three weeks and completely free, the 2020 edition will be a completely different and interesting experience. Nobody can truly guess what features will be announced, but the past few years can give us some good insights.
I expect this year’s announcements to be driven by three key themes: built-in partner integrations, bridging the gap to non-AWS environments, and a smooth experience for different customer types (e.g., developers, data scientists, data analysts).
During the past 12-18 months, AWS has come out with new services featuring built-in third-party integrations, such as Event Bridge, Lambda Extensions, and, most recently, Gateway Load Balancer. It’s clear that AWS is highlighting its technology partners, and making an effort to integrate them more closely within the AWS ecosystem, as well as provide a greater developer experience.
At this re:Invent, I expect a lot more features coming out for key services, including Lambda, EKS, and CloudWatch, that enable you to leverage partner technologies out of the box without complex integration logic.
As the uncontested market leader year after year, AWS has always had a very dominant cloud position. Multi-cloud and hybrid were never truly part of AWS’ offering strategy. But the rapid rise of players such as Google Cloud and Microsoft Azure has forced AWS to re-think this approach. The launch of AWS Outposts last year was a welcome addition to address hybrid architecture scenarios. Yet, with Azure and Google making big bets in Kubernetes and providing a holistic approach toward hybrid and multi-cloud environments with their Arc and Anthos offerings, it’s fair to expect AWS will be expanding its managed Elastic Kubernetes Service (EKS) to bridge the gap to non-AWS environments.
The great range of AWS managed services in areas like data science and analytics, plus the wider adoption in big enterprises, means that different customer types, such as developers, data scientists, and business analysts, are using the AWS web-interface console in their day-to-day. Providing a smooth experience that can cater to both technical and less-technical audiences is thus critical. For engineering teams, I expect improved capabilities on CI/CD and for Lambda and Container services to come closer together. In general, it’s fair to expect the announcement of additional services that provide a more unified experience, especially around data science; low-code and cloud governance, such as tools for cost management; and services similar to the recently launched Glue DataBrew and Honeycode.
Looking forward to AWS re:Invent 2020!
Ben Rogojan
Data Scientist and Data Engineer Consultant
In the past few years, AWS has provided much of the functionality needed by data professionals. For example, if you’re a data engineer, Redshift started supporting stored procedures about a year and a half ago. They also introduced federated queries, which made it easier to query across data warehouses, data lakes, and operational databases. This was great for anyone who needed to do ad hoc analyses across data sources.
But despite many of the new changes AWS has made to improve the experience of data professionals, there’s still a lot of room for improvement. For example, I wish AWS would provide data scientists and engineers improved data lineage and metadata tools, as well as a better collaborative experience for queries.
AWS manages so much data in Redshift, S3, RDS, and a host of other databases. It surprises me that it doesn’t also offer a more substantial data-lineage tool—not just to help data engineers track where data’s coming from, but to also help them better track which processes interact with the data along the way.
This could include some open-source or external ETL processes like Airflow and Fivetran. There are often a lot of gaps in terms of where transformations are happening unless you use AWS Glue, and even then, once pipelines become long and complex, it can be difficult to assess where data’s coming from in your cloud architecture.
Another helpful feature would be a more collaborative query editor. AWS started to offer a decent query editor back in 2019. But it doesn’t really have the collaborative and quick data-visualization editing capabilities some third-parity tools have like PopSQL. Having a query editor where you can easily share queries, comment, have version control, and put together quick data visualizations is very handy when it comes to working for large companies. It allows everyone on a team to gain a solid understanding of what data is being used, as well as what logic is being implemented.
As a data scientist and data engineer, our workflows are slightly different than software engineers (which is what AWS feels more geared for). We need to constantly work together and visually explore our data. Hopefully, in 2021, AWS will start taking the needs of data professionals into account.
Adam Bertram
Microsoft MVP, Automation Engineer and DevOps Trainer
EBS for Lambda
Earlier this year, Amazon introduced EFS storage for Lambda, which allows files to persist between Lambda invocations. This would allow developers to load a large file, break it into chunks, and have several Lambda functions work on those chunks in parallel, among other things, but not really persist data beyond that. By allowing Lambda functions to access EBS volumes as well, developers could write functions to persist data or even trigger an output from an EC2 instance.
In-Place Engine Upgrades for RDS
Let’s face it: Upgrading your database is awful. Forget about the snapshots and restoring; if you’re managing infrastructure with Terraform or CloudFormation, those providers will delete the database before recreating it by default, taking your data with it! An in-place update would allow data engineers and DBAs to breathe easier when a new Postgres or MySQL version comes out and your developers want to use it.
Fargate as a Standalone Service
When Google Cloud Run came out, it quickly became one of the most popular services on that platform. Developers wouldn’t have to worry about how to scale their containers anymore; they could just hand it to GCP and let it figure out the rest. AWS offers a similar service with ECS Fargate—no worrying about the underlying file system or instances. But you as a developer still need to define a task with pre-set memory, CPU, and auto-scaling rules.
Automating those pieces and making Fargate into something that can be used without worrying about how to configure ECS would go a long way toward simplifying containerized apps in AWS.
Better ETL for SageMaker
Last year at re:Invent, Amazon put a lot of resources into promoting their machine learning and Big Data offerings. This year, a good complement to that would be better ETL offerings and integration with SageMaker. You can already start to see the pieces coming together with the recent announcement of AWS DataBrew, an addition to the Glue service, but it’ll be interesting to see what other announcements come out tying data to the pipelines that use it.
Samir Behara
Senior Architect and Microsoft MVP
AWS re: Invent is just around the corner, and the excitement has already begun. Even though I’ll miss attending this fantastic conference in person, the three-week virtual format will be an excellent experience. Year after year, AWS has surprised everyone with their creative product launches, diverse focus areas, and increased pace of innovation. This makes it extremely challenging to predict what’s in store for this year. However, I do have a wish list of things I’ll be on the lookout for during the event:
- New capabilities to help organizations transition their legacy workload to the cloud and speed up their migration journey in a cost-effective manner.
- Managed tools from the Kubernetes ecosystem, like managed Prometheus, managed Grafana, and managed observability toolings, to help drive broader adoption of Kubernetes among enterprises.
- Simplification of database migration—for DynamoDB, ElastiCache, Elasticsearch, Aurora, and other datastores—from one AWS account to another with minimal downtime.
- Service mesh technologies like Istio/Envoy have seen incredible adoption among enterprises and have helped reduce the complexity of microservice architectures. I’ll be watching for new feature enhancements in the service mesh landscape and a future roadmap for AWS App Mesh.
- Simplification of onboarding for existing accounts onto AWS Control Tower to set up multi-account AWS environments and follow best practices.
- Cloud-native tools and technologies have been growing rapidly. I’ll be keeping an eye on the new software listings and integrations allowed in AWS Marketplace related to Data Analytics, Serverless, Networking, DevOps, CI/CD, IoT, Blockchain, and Speech Recognition.
- Innovations in the Artificial Intelligence, Machine Learning, and Big Data space.
AWS re: Invent 2020 is going to be intriguing in a lot of ways. The COVID-19 pandemic has affected people’s lives across the globe, and every organization has been forced to modify their business operations. There has been an increased demand for e-commerce, telehealth, online learning, and information technology. I’m looking forward to seeing how AWS can positively impact and work with small and medium-sized businesses to help them embark on a digital transformation journey.
Demand for this is surging with each passing day, so it will be worth following how AWS plans to scale its operations to meet this demand. As people continue to work remotely going into 2021 in the hope of improving the economy, it will be interesting to see how AWS taps into increasing customer needs and continues to enhance its cloud-computing platform by adding more capabilities.
