쥬 잡동사니

Scheme sinkScheme = new TextLine( new Fields( "word", "count" ) );
Tap sink = new Hfs( sinkScheme, outputPath, SinkMode.REPLACE );
// the 'head' of the pipe assembly
Pipe assembly = new Pipe( "wordcount" );
// For each input Tuple
// parse out each word into a new Tuple with the field name "word"
// regular expressions are optional in Cascading
String regex = "(?<!\\pL)(?=\\pL)[^ ]*(?<=\\pL)(?!\\pL)";
Function function = new RegexGenerator( new Fields( "word" ), regex );
assembly = new Each( assembly, new Fields( "line" ), function );
// group the Tuple stream by the "word" value
assembly = new GroupBy( assembly, new Fields( "word" ) );
// For every Tuple group
// count the number of occurrences of "word" and store result in
// a field named "count"
Aggregator count = new Count( new Fields( "count" ) );
assembly = new Every( assembly, count );
// initialize app properties, tell Hadoop which jar file to use
Properties properties = new Properties();
AppProps.setApplicationJarClass( properties, Main.class );
// plan a new Flow from the assembly using the source and sink Taps
// with the above properties
FlowConnector flowConnector = new HadoopFlowConnector( properties );
Flow flow = flowConnector.connect( "word-count", source, sink, assembly );
// execute the flow, block until complete
flow.complete();  

Pipe type

Purpose

Input

Output

Pipe

 instantiate a pipe; create or name a branch

 name

a (named) pipe

 SubAssembly

create nested subassemblies

 Each

 apply a filter or function, or branch a stream

 tuple stream (grouped or not)

 a tuple stream, optionally filtered or transformed

 Merge

 merge two or more streams with identical fields

 two or more tuple streams

 a tuple stream, unsorted

 GroupBy

 sort/group on field values; optionally merge two or more streams with identical fields

one or more tuple streams with identical fields 

 a single tuple stream, grouped on key field(s) with optional secondary sort

 Every

 apply aggregator or buffer operation

 grouped tuple stream

 a tuple stream plus new fields with operation results

 CoGroup

 join 1 or more streams on matching field values

 one or more tuple streams

 a single tuple stream, joined on key field(s)

 HashJoin

 join 1 or more streams on matching field values

 one or more tuple streams

 a tuple stream in arbitrary order