Apache FlumeTM

Apache Flume is a distributed and reliable system for efficiently collecting, aggregating, and moving large amounts of log or event data from many sources to a centralized data store like MapR Data Platform.

Flume agents ingest incoming streaming data from one or more sources, including avro, thrift, exec, JMS, netcat, and syslog. Data ingested by a Flume agent is passed to a sink, which is most commonly a distributed file system like Hadoop. Multiple Flume agents can be connected together for more complex workflows by configuring the source of one agent to be the sink of another.