Managed Data Platform

Students

Our Competency Creation Platform is built on top of MDP

Data
  • Embrace Data locality – Edge or Central
  • Acquire from any source
  • Acquire any type
  • Build scalable real-time data pipelines
Infrastructure
  • Cluster & Resource Management across Clouds
  • Container Orchestration
DataScience, ML&DL
  • Data Science Workbench
  • Machine and Deep Learning library Catalog
  • Production ML/DL pipelines
  • GPU Support
Applications
  • API/Microservices Management (Service Mesh)
  • API Gateway

Technologies

Cluster & Resource Management

We use Apache Mesos with the Cook scheduler for Spark jobs.

Data pipelines

Spark, Flink, Kafka, HDFS, and Cassandra as primary tools for building data pipelines for ML/DL.

Container Orchestration

Kubernetes and Marathon (depending on use case).

Machine and Deep Learning

Kubeflow and Kubeflow pipelines as DL/DL Toolkit with JupyterHub on Kubernetes.

Microservices Management

Istio for Service Mesh and Kong as the API Gateway. Knative is the chosen FaaS platform for Serverless.