Big Data Projects

Popular open source big data projects

Distributed File System

• Hadoop HDFS

Distributed Storage

• HBase

Distributed Computing

• Hadoop MapReduce Cloudera
• Storm Yahoo
• Spark Databricks
• Flink Ververica

Distributed Message System

• Kafka Confluent
• RocketMQ Alibaba

SQL

• Hive

Search

• Solr
• ElasticSearch Elastic

Distributed Coordination

• Zookeeper
• Curator

Ingestion

• Flume

APIs

• Beam
• OpenMessaging

Others

• Apex DataTorrent
• Alluxio Alluxio
• Presto
• Impala
• Mesos
• Samza
• Kylin Kyligence
• GearPump
• Pig
• Oozie
• Azkaban
• Cassandra
• MongoDB
• Neo4j
• Sqoop
• Ganglia
• Nagios
• Docker
• RocksDB
• OpenTSDB
• Phoenix

General

• Nginx
• Tomcat
• MySQL
• Redis
• HAProxy
• ActiveMQ
• Memcached
• PostgreSQL
• RabbitMQ
• LVS
• dnsmasq

Language

• Java
• SQL
• Scala
• Python
• BASH/Shell
• JavaScript