📉📊Selection of tools for working with Big Data
Drill - Layers on top of multiple data sources, allowing users to query a wide range of information in a variety of formats, from Hadoop sequence files and server logs to NoSQL databases and cloud-based object stores.
Druid (https://druid.apache.org/) is a real-time analytics database that provides low query latency, high concurrency, multi-user capabilities, and instant visibility into streaming data. According to its proponents, multiple end users can simultaneously query data stored in Druid without any performance impact.
HPCC Systems is a big data platform developed by LexisNexis and open sourced in 2011. In accordance with its full name - High-Performance Computing Cluster - the technology is essentially a cluster of computers created on the basis of standard hardware for processing, managing and delivering big data.
Iceberg is an open table format used to manage data in lakes, achieved in part by tracking individual files of information in tables rather than directories. Created by Netflix for use with its petabyte-sized tables, Iceberg is now an Apache project. Iceberg is typically "used in production, where a single table can contain tens of petabytes of data."
Kylin is a distributed information storage and analytics platform for big data. It provides an analytical information processing (OLAP) engine designed to work with very large data sets. Because Kylin is built on top of other Apache technologies, including Hadoop, Hive, Parquet and Spark, its proponents say it can easily scale to handle large volumes of data.
Samza is a distributed stream processing system created by LinkedIn and is currently an open source project managed by Apache. The system can run on top of Hadoop YARN or Kubernetes, and a standalone deployment option is also offered. According to the developers, Samza can process "several terabytes" of data state information with low latency and high throughput for fast analysis.