Since 2015, Intellias has been a technology partner of the world’s top provider of mapping data and location-based services. As an integral part of one project for this client, the Intellias team adapted the architecture of an automotive-grade navigation solution to comply with the Navigation Data Standard (NDS). The key objective of NDS is to standardize map formats to enable compatibility and interoperability among a growing variety of NDS-compliant applications and devices.
The NDS map format relies on SQLite technology. Thus, the database for the navigation solution actually consists of several product databases, which are further divided into update regions. This supports flexible and consistent versioning for NDS databases and makes it possible to integrate data from different suppliers into one map.
Our client’s initial database was deployed on Oracle Exadata, but our team discovered some disadvantages in the course of its maintenance.
- Limited scalability
Exadata required our client to buy new virtual machines for every three to four projects and featured considerable delivery times (reaching half a year) for new projects.
- Low performance
Oracle Exadata could compile three update regions in 24 hours. In addition, compiling one region influenced the performance of other product databases.
- High costs
Expensive Exadata storage turned out to be inefficient for the project, as data was not used after compilation.
The development and production environments were duplicated.
Cloud optimization services delivered
Once we identified the key business drivers for migration, our next milestone was to select a cloud environment that would provide the most value in terms of efficiency, scalability, operability, and cost savings. We assessed the Amazon Relational Data Service (RDS) and Amazon Elastic Cloud Compute (EC2) and opted for EC2 to get more flexibility and control over the database.
Our team also presented our client with an effective strategy for continuous cloud infrastructure optimization according to a three-stage plan. Any improvements to production are driven by a thorough analysis by the DevOps team and a trial run in the development environment with the cloud optimization services. This cloud optimization strategy ensures quick, seamless, and on-time continuous improvement without any loss of data or other issues in production.
Migration from Oracle Exadata to Amazon EC2
During the active development phase and cloud optimizations of the project, Intellias found an opportunity to dramatically improve the project’s overall performance and scalability as well as reduce cloud automotive infrastructure costs by migrating the existing infrastructure to Amazon Web Services. Migrating the map compilation pipeline, clipping environments, and testing framework to Amazon EC2 resulted in
- cutting the overall build generation time in half, which allowed for cloud optimizations of the map build timeframe with nightly builds
- running an unlimited number of projects in parallel in fully isolated cloud automotive environments
- launching new projects in two weeks instead of six months
- cutting the compilation cost by 55%.
Continuous improvement of automotive management cloud software and infrastructure
Our team of DevOps engineers is constantly looking for opportunities to bring additional value to our client by focusing on continuous improvement and operational efficiency of the automotive management cloud software. To optimize infrastructure costs and project processes, we established cloud optimization services, daily monitoring of instances, system-wide performance, resource utilization, operational health, and cloud spend. Every morning, we thoroughly analyze logs, metrics, events, and automated notifications to transform actionable insights from Amazon into tangible performance improvements and cost reductions.
S3 storage efficiency
One of our approaches to optimizing the cost of cloud automotive infrastructure is conducting a regular S3 storage audit, resulting in changes to storage lifecycle management. In order to achieve cost efficiency and cloud optimization and avoid unnecessary spending, we follow best practices for the S3 buckets lifecycle, configuring AWS to downgrade the objects class and archive or delete unused data. As part of security enhancements and storage rationalization, we also restricted access to S3 buckets for team members who don’t contribute to project development.
When planning for the transition from project development to maintenance, the challenge was to find the golden formula enabling us to effectively deliver value on an ongoing basis with high quality and reduced cost. To ensure that customers receive updated maps in a timely manner and to save costs on infrastructure, we decided to adopt a continuous integration approach, eliminating the issue of production builds, and switch to weekly releases, combining map compilation for all updated regions with the execution of auto-tests.
To minimize the number of failed builds, reduce the compilation time, and decrease the size of the update package, we introduced a deterministic compiler for cloud optimization that allows us to verify only parts of the map affected by the latest commits. Having this feature in place, we reduced the scope of map testing and efforts per nightly build by more than a third.
Monitoring compilation time
Reducing compilation time is an essential part of optimizing cloud infrastructure resources and one of the ways to achieve considerable cost savings. We built a service responsible for shutting down resources if the compilation time exceeds 12 hours or the resources aren’t required. Implementing this feature allowed us to balance resource availability to reduce costs and provide our DevOps engineers with an automated tool for detecting build issues that can be escalated to the development team for further analysis and resolution.
To bring on extra capacity only when needed and cut costs for cloud automotive infrastructure, we used next-generation Amazon EC2 instances, including Reserved Instances, On-Demand Instances, and Spot Instances. This optimization saved $200 per instance used, which is 82 percent of the original cost.
We surpassed our goals and optimized the project infrastructure to reap all possible benefits of cloud optimization technology. The whole project migration from Oracle Exadata to Amazon EC2 took over a year and was followed by continuous improvements to the cloud infrastructure. By leveraging the AWS cloud and establishing a continuous improvement process, we significantly strengthened database performance and scalability while reducing the overall total cost of cloud infrastructure ownership by 98.5%.