Big Data

At the center of "Data Warehouses" are Massively Parallel Processing (MPP) databases and supporting Data Lakes.  Alexicon is a partner with Looker and part of the Amazon APN network to deliver Big Data solutions (Hadoop, Spark and MPPs) that scale in the cloud.  Popular big data tools are Looker, Amazon SageMaker, Amazon S3, Amazon Athena and Amazon Redshift.


Hadoop and Spark (Big Data and Data Science)

Major and mid-size companies are positioned to benefit from these tools.  MPP databases are more affordable for EDWs and even Hadoop and Spark can be used inexpensively through Amazon as a cloud service to process big and massive data sets.  Companies that do not want to put their data in the cloud are bringing Hadoop, Spark and an MPP in-house.

MPP Environments

MPP databases are typically used in major companies for large EDWs.  These are truly large integrated database management systems for Business Intelligence and Analytics reporting.  Major corporations need these systems to ingest and analyze bigger datasets quicker to increase company competitiveness through gained intelligence.  Hadoop and Spark (the truly Big Data systems) can be used to feed MPPs for company-wide reporting.

MPP databases can have millions or billions of records that users run reports directly against.  We have seen over 40 data sources integrated in these databases.  Sizing the initial MPP environment and adding capacity once the system is being used can be a difficult task without professional 3rd party assistance.  Alexicon provides consulting services for big data environments.  Our specialties are architecture, governance, enterprise data models, Extract, Transform and Loading (ETL), computations, Performance Management (goals/targets) and Process-based structures.

We provide performance-based testing to determine the most cost effective mix of onsite and cloud analytics and data management solutions for current and future needs.

Complete System Approach

We take a complete system approach by providing integrated reporting and work management methods to ensure the entire system process is well coordinated.  By using a Process-based Approach, metrics and charts are used to understand how queries are processed through the complete system.  System Throughput and performance inhibitors are the focus to improve overall performance.  Below is a diagram that shows the database and major elements that influence performance:

With the MPP databases at the center of all the action, we focus on loading, administration and user queries to ensure peak periods are resourced correctly or work is moved to better suited times.

MPP Uses

Our experience is that large organizations with high query counts need planned and coordinated workload management to provide optimal performance.  For global companies, workload management is critical considering different time zones, required loads and user queries.  Working in cloud enviroments involves minimizing costs around large custered systems.

Integrated Processes and Reporting

Out-of-the-box reports for databases, ETL and BI tools are like typical data mart/warehouse builds for business users.  We need integration from multiple source applications for a complete system picture.

Workload Management Areas

MPPs have also become a costly asset to manage and deserves the attention and opportunity to optimize hourly performance for a lean structure that avoids unnecessary compute time or over storage.

Contact Us









Home

Indicator Analytics

Enterprise Analytics

Data Science

Lean Six Sigma

Contact Us

   

© 2019 Alexicon Corporation.  All rights reserved.