Configuring individual components

After defining the flow, you need to set properties of each source, sink andchannel. This is done in the same hierarchical namespace fashion where you setthe component type and other values for the properties specific to eachcomponent:

  1. # properties for sources
  2. <Agent>.sources.<Source>.<someProperty> = <someValue>
  3.  
  4. # properties for channels
  5. <Agent>.channel.<Channel>.<someProperty> = <someValue>
  6.  
  7. # properties for sinks
  8. <Agent>.sources.<Sink>.<someProperty> = <someValue>

The property “type” needs to be set for each component for Flume to understandwhat kind of object it needs to be. Each source, sink and channel type has itsown set of properties required for it to function as intended. All those needto be set as needed. In the previous example, we have a flow fromavro-AppSrv-source to hdfs-Cluster1-sink through the memory channelmem-channel-1. Here’s an example that shows configuration of each of thosecomponents:

  1. agent_foo.sources = avro-AppSrv-source
  2. agent_foo.sinks = hdfs-Cluster1-sink
  3. agent_foo.channels = mem-channel-1
  4.  
  5. # set channel for sources, sinks
  6.  
  7. # properties of avro-AppSrv-source
  8. agent_foo.sources.avro-AppSrv-source.type = avro
  9. agent_foo.sources.avro-AppSrv-source.bind = localhost
  10. agent_foo.sources.avro-AppSrv-source.port = 10000
  11.  
  12. # properties of mem-channel-1
  13. agent_foo.channels.mem-channel-1.type = memory
  14. agent_foo.channels.mem-channel-1.capacity = 1000
  15. agent_foo.channels.mem-channel-1.transactionCapacity = 100
  16.  
  17. # properties of hdfs-Cluster1-sink
  18. agent_foo.sinks.hdfs-Cluster1-sink.type = hdfs
  19. agent_foo.sinks.hdfs-Cluster1-sink.hdfs.path = hdfs://namenode/flume/webdata
  20.  
  21. #...