The AWS Partner Network (APN) has endorsed ElevationData, the CloudGeometry company specializing in data engineering and data science pipelines, as a Designated Service Provider for Amazon Kinesis. This recognition follows the recent APN certification of CloudGeometry AWS Database Migration Services.

Data Unbound 

Distributed big data storage technology is widely accepted in modern computer architectures. The new data imperative is finding new, more productive, and versatile approaches to processing and analyzing data in almost any form. Kinesis offers uniquely rich and powerful hosted cloud services to unlock that data, streaming from virtually any source to any destination. 

Kinesis can ingest real-time data such as IoT telemetry data,  social media feeds, video, audio, application logs, website clickstreams, and more. Its great strength is that it lets you instantly respond by processing data as it arrives, instead of having to collect all your data first. Kinesis scalability provides a whole new dimension for what used to be called "big data". It handles any amount of streaming data and process data from hundreds of thousands of sources, with very low latencies. For any data streamed -- from a source (aka "producer") to a destination (aka "consumer")  -- Kinesis can either store it immediately or kick off a successor data stream. 

Kinesis Benefits

The combination of compute and data management via Kinesis data streaming is both backbone and spinal cord of any modern distributed application. It pays off particularly well with our many SaaS customers, especially turbocharging Data Science needs. It’s hard to overstate how useful this can be:

  • Flexibility and scalability: An easy to build, cost-effective system where you pay only for the data you pass through and reprocess, and for compute resources you actually use.  Speaking of compute, you can also optimize costs by implementing spot instances. Use that for batch style processing that requires a threshold of high performance only during for a few hours a day. 
  • Continuous data flow versatility: As the name implies, Kinesis is not just a simple message broker tool ( sort of like physics:  but the wave and a particle). Kinesis is really a data processing ecosystem that provides a range of tools responsible for ingestion, processing, collecting/caching (e,g, for further computations and data analysis by 3-rd party services/apps such as Apache Spark over EMR), time-series data, and monitoring. 
  • Integrate everything with everything: The versatility of Kinesis to take data from one place and put it in another is matched by being able to take data from anywhere and put it anywhere, over and over again.  Excellent capabilities for integration with other AWS services and tools: Starting with S3, RDS/Aurora, Lambda, SES, SNS, Redshift, and more. The Amazon SDK for Kinesis helps developers to build client Connectors, essentially customized tools with built-in Kinesis integration. It's also straightforward to use with Java and SQL. 
  • Specially optimized streaming types: Kinesis comes in four flavors, each with a specialized data modality.
  • Video securely streams video from connected devices to AWS
  • Streams parallelizing continuous of capture gigabytes of data per second from hundreds of thousands of sources
  • Firehose captures, transforms, and loads data streams into AWS data stores for near real-time analytics with existing BI tools
  • Analytics use SQL or Java without having to learn new programming languages or processing frameworks.

What this adds up to is that Kinesis is super powerful in just about any Data Analytics pipelines, easily built by implementing AWS Kinesis services, with minimum efforts. The end result is a system with ultra-reliable scalability and a high level of security -- at a surprisingly affordable price, as you pay only for what you use. Applying the pay-as-you-go formula for computation resources is literally a big deal because compute performance Is make-or-break for timely in-depth data analysis. An implementation after implementation, we have found it to be the key to unlocking data-driven high-performance applications. It's hard to overstate how much more cost-effective than dedicated servers and storage. 

CloudGeometry Kinesis Use Cases 

What we've seen again and again across our client is that Kinesis is more than (silly analogy alert) the data equivalent of MacGyver with a Swiss Army knife;  it's an endless supply of on-demand data MacGyvers, each with a hyper automated Swiss army knife plus built-in SDK. Here are just a few examples we have built out for our clients

  • Health Care real-time processing pipelines combining relational and streaming data to trigger personalized medicine recommendations
  • Monitoring solar panels with real-time data processing pipeline to drive machine learning behind proactive maintenance planning
  • AdTech traffic analysis to drive audience matching and bid optimization including social streams and viral mentions
  • Cost allocation to attribute fine-grained infrastructure billing for each individual Kubernetes cluster (instance, disk, VPC) to calculate charges for each tenant
  • Dashboarding to span multiple heterogeneous data sources, structuring visualization to provide coherent decision making
  • IoT data acquisition and utilization management for monetizing physical and digital assets 

One particularly productive applications of Kinesis is in bridging traditional relational data within heterogeneous environments, expanding transactional data into new use cases. Another key strategy is in using containerization and serverless design patterns, which affords even greater versatility for various ways in which data can be used and reused to support different business processes as any SaaS application matures (Update: we are now also designated as Certified Service Provider for Lambda Services by the AWS Partner Network). 

Data at the heart of SaaS 

Delivering software-as-a-service to solve strategic business problems can make it easier and faster to deliver value to customers and profit to investors than ever before. There are many ways to describe this, but let's boil it down to three key elements:

  • Customer acquisition via frictionless onboarding: more automation, fewer less manual, error-prone steps
  • Tiered engagement and pricing to segment markets & monetize subscriptions
  • Built-in customer feedback for instant engagement or data-driven business strategies

Data is the backbone of any SaaS business. As the SaaS model gains wider adoption beyond startups and unicorns, driving growth for established Enterprises expanding the reach of their products and services, Kinesis will play an ever more critical role. At CloudGeometry, our specialty is to continue solving data problems at the same time as we create new data opportunities.

‍