Case study: Online Payments
Data Lake Modernization
Architecture Roadmap & Strategy
A global online payments provider had to update its business intelligence and analytics infrastructure as its Hadoop platform had grown to 30 Petabytes. The company needed to scale to support upwards of 100 Petabytes of data with advanced security, data governance and Spark analytics.
The company contacted Cloudwick, the leading provider of bimodal digital business services and solutions to the Global 1000, to perform a current and future state assessment. Cloudwick provided provide a gap analysis with architectural recommendations for its next generation 100 Petabyte architecture to support advanced analytics.
- The company gained an architectural roadmap for scaling its data and analytics platform
- It improved its data platform and analytics performance to meet SLAs for mission-critical jobs
- It reduced job failure rate for key jobs by 85%
Seeking an Architecture Recommendation
A global online payments system utilizing Hadoop was at risk of falling behind its competitors due to capacity, performance, inability to meet SLAs, and a shortage of skilled personnel. First, its 24-petabyte production cluster was at 80% storage capacity, triggering analytics failures, causing performance issues and data ingestion loss, and hindering the company from storing any more data beyond six months. Next, the company was understaffed compared to its peers and industry standards by at least 7-12 big data experts, including solution and performance architects. Finally, the company’s Hadoop distribution was outdated in terms of security, governance, management and monitoring, putting the company at tremendous risk of losing critical data like consumer information, monthly reports, and more, so a new architecture was necessary to support ongoing business intelligence and analytics requirements.
The organization called on Cloudwick, the leading provider of bimodal digital business services and solutions to the Global 1000, for help with its long-term infrastructure and analytics strategy. Cloudwick’s big data experts started by interviewing the company’s line of business (LOB) leaders and stakeholders for insight on the current state of the system, use case requirements, and past and present weaknesses.
From Assessment to Roadmap
Cloudwick performed a thorough review of the infrastructure, analyzing the Hortonworks Hadoop platform, overall system architecture including data flow, ingestion, replication process, messaging system and more, to determine and provide the company with a thorough assessment and gap analysis. From these, Cloudwick presented a final architecture recommendation and implementation roadmap.
Cloudwick’s architecture recommendation included a market review and industry competitive analysis, vendor analysis, product overviews (features, capabilities, etc.), ecosystem dependencies, and technology stack. These were compared to solutions implemented by the company’s peers – organizations in the financial industry with similar storage and job count requirements – to highlight the capabilities of the recommended architecture compared to industry leaders, demonstrating that all its requirements would be met.
With Cloudwick’s platform and architecture recommendation, resources, and storage analysis tool:
- The company has a 24 month plan for scaling its data-driven business
- Performance issues no longer negatively impact the company’s SLAs
- Analytics failure rate has been reduced by 85%
- The organization has a platform that provides real-time analytics at scale