Architecting for Today’s Hybrid Analytic Ecosystem

The quest for a magical cure-all is not new. In the days of the old west, a medicine wagon would roll into town, and a slick salesman would emerge, pitching his latest panacea, promising that his miracle elixir could treat anything from a hangnail to heart disease. The claims were alluring. But they were based on half-truths and exaggerations that exploited the desperation of settlers, some of whom were desperately ill, and others terrified at the thought of being unprepared for tragedy.
 
Over the past 50 years, many technologies promoted as modern-day cure-alls for data and analytics have proliferated. These technologies usually demonstrate that they are very good at some specific aspect of the ecosystem. Then, at great cost, they are dramatically over-positioned to take on roles where their weaknesses become painfully apparent.
 
A modern data and analytics ecosystem for a large enterprise must support many different requirements, including:
 

• Low-cost data storage and processing for high-volume, unrefined data
• Self-service data provisioning for experimentation and ad-hoc analysis
• Shared data, accessed by a variety of production applications and workloads
• Data curation, integration, and ongoing management
• Numerous forms of reporting and analytics from basic reporting to advanced visualization and machine learning

 
Expecting one tool to support all these needs is unreasonable. There are technologies that are good at meeting some of these requirements but not others. If you acquire a technology because it is good at, for example, low-cost data storage or self-service data provisioning, does that mean it will support the intense response time and resiliency requirements of business-critical applications at the scale required for a large enterprise? Our collective experience with the Hadoop ecosystem should have clearly answered this question. In other cases, a tool may be very good at spinning up a small, departmental data solution that can grow without much friction, but will the expense of scaling that tool for enterprise-class, mixed-workload analytics give the CFO heart palpitations?
 
Teradata was founded over 40 years ago to meet a specific need – providing the performance, reliability, and scalability to address the high-intensity data requirements of large enterprises. Since then, Teradata technology has continuously evolved, consistently improving core strengths while integrating seamlessly with other elements of a modern architecture. Within the larger data and analytics ecosystem, Teradata Vantage does the heavy lifting. And through open standards and engineering partnerships, Vantage fits within a best-of-breed approach for data and analytics. For example, end users can leverage popular tools and languages such as R, Python, SAS, and Jupyter while “pushing down” intense processing into the database. This enables the solution to support thousands of users and a variety of workload types while allowing analysts to speak in their native language. In addition, Teradata works with all major data management tool suppliers to optimize the connections, thus leveraging the best tools possible to ensure the trustworthiness of data resources.   
 
When positioned and leveraged correctly, Vantage keeps overall costs under control in a way that is not possible with other tools attempting to address the same need. For example, unlike other technologies, adding additional capacity, especially at the massive scale required for modern enterprises, does not require exponential growth in costs. And Vantage can directly access data in popular object store technologies such as Amazon’s S3 or Microsoft’s Azure Blob Storage for storing vast amounts of raw or “cold” data that is less frequently accessed, further controlling costs.
 
As much as we would like to believe in a modern-day cure all for data and analytics – or Santa Claus, the Tooth Fairy, or the Easy Button – it simply doesn’t exist. Today’s analytic requirements call for a hybrid architecture that blends together multiple technologies and design patterns, enabling the right tools and technologies to be used for the right purposes. Unifying solutions, such as Vantage, enable a carefully selected mix of tools to be used together in a coordinated and coherent ecosystem, specifically aligned to the requirements of the business. This is the most effective strategy, and the most economical.