6/19/2023 0 Comments Flume stanfordSome of the notable features of Flume are as follows −įlume ingests log data from multiple web servers into a centralized store (HDFS, HBase) efficiently. It guarantees reliable message delivery.įlume is reliable, fault tolerant, scalable, manageable, and customizable. The transactions in Flume are channel-based where two transactions (one sender and one receiver) are maintained for each message. When the rate of incoming data exceeds the rate at which data can be written to the destination, Flume acts as a mediator between data producers and the centralized stores and provides a steady flow of data between them.įlume provides the feature of contextual routing. Using Apache Flume we can store the data in to any of the centralized stores (HBase, HDFS). Here, Apache Flume comes to our rescue.įlume is used to move the log data generated by application servers into HDFS at a higher speed. To do so, they would need to move the available log data in to Hadoop for analysis. Applications of FlumeĪssume an e-commerce web application wants to analyze the customer behavior from a particular region. It is principally designed to copy streaming data (log data) from various web servers to HDFS. Apache Flume - Introduction What is Flume?Īpache Flume is a tool/service/data ingestion mechanism for collecting aggregating and transporting large amounts of streaming data such as log files, events (etc.) from various sources to a centralized data store.įlume is a highly reliable, distributed, and configurable tool.
0 Comments
Leave a Reply. |