Custom Stream Lookup
Lookup collected rows using a custom index
First, the step reads all records coming in through the lookup gate and indexes them in memory. It uses the function supplied in the indexer setting to calculate the index key. If multiple rows yield the same key, only the latest row is indexed.
When all rows are indexed, the step reads records coming in through the in gate, and performs one lookup per incoming row. The lookup key setting is evaluated for each incoming row and used as the lookup key.
The row found by looking up the key is made available as results.lookup.
Function invoked on each lookup row. Currently processed lookup row is passed into the function. The function’s return value is the key at which the row is indexed.
All comparable values including
The function has the signature
Evaluated once on step initialization
The key to lookup in the index for the current input row.
Evaluated for every input row
The row found by looking up the current lookup key. If no row is indexed at the lookup key, the value is