The challenge
Decision-making can be overwhelming when data comes from multiple sources with different structures, and standardization demands a lot of manual work. Consolidated data with a unified structure is the key to delivering an exceptional customer experience.
The solution
The automated population of a new Data Mart was implemented, creating a unified data structure for all countries. This ensures that KPIs and all information used for support and decision-making are standardized.
The Data Mart was built using a Serverless Architecture on the AWS Cloud to facilitate maintenance, scalability, and data availability. Below, we list some of the services used in the solution:
Storage & Data
- Amazon S3: For scalable object storage, including data files to be queried by Amazon Athena.
- Amazon Redshift Serverless: For Historical Data.
- Amazon Athena: For querying Databases & Tables.
- AWS Lake Formation: For centralized permission management and data sharing.
- AWS Direct Connect: For a dedicated network connection to AWS.
Compute & Code
- AWS Lambda: Serverless Python Functions used to clean, process, and ingest data, generate mappings, and monitor the solution.
- AWS CodeCommit: Used for Source Control Management.
- AWS CodeBuild: Used for deployments needed for the solution.
Security
- AWS Identity and Access Management (IAM): Manages identities and their permissions, configured to follow least-privilege principles.
- Amazon GuardDuty: Continuously monitors the AWS account for anomalous behaviors and events.
Monitoring
- AWS CloudTrail: Tracks Administrative Events in the account.
- Amazon CloudWatch & Amazon Simple Notification Service (Amazon SNS): Monitors Metrics and Admin events logged by CloudTrail, and uses an Amazon SNS topic to notify interested parties if needed.
- AWS Cost Explorer: Tracks costs and supports reporting.
Results
Up-to-date consolidated data ready for reporting and other analytical tasks to provide a better customer experience for customers in all 15 countries.
