How to select, rename and retype fields in Apache Hop
What this shows
Select Values does three separate jobs that happen to share one dialog: it picks and renames fields, it removes them, and it changes their types. Each job has its own tab, and mixing them in a single transform is the usual source of confusion.
Reference documentationThe complete list of Select Values options is documented in the Apache Hop manual, which is the authoritative reference.hop.apache.org →Sample pipeline
The pipeline in the video is Apache Hop's own sample, select-values-basic.hpl, which ships in the Hop samples project under transforms/.
Transcript
Full transcript
Select Values does three separate jobs that happen to share one dialog: it picks and renames fields, it removes them, and it changes their types. Each job has its own tab, and mixing them in a single transform is the usual source of confusion.
The same three rows of test data - an id, a description and a date - are fed to three Select Values transforms, each using one tab.
Select and Alter picks the fields to keep and renames them. The order of this table becomes the order of the fields downstream, so it reorders as well. Be careful: once you list anything here, fields you leave out are dropped, unless you tick the box to include unspecified fields.
Remove is the opposite approach: name only what you want gone and everything else passes through untouched. When you are dropping one field from a wide stream this is far less brittle than listing every field you want to keep.
Meta-data changes a field's type without changing which fields flow through. Here a date held as text is converted to a real Date. This is where you give a format mask, and where conversion can fail - which is why the pipeline sends failures to their own output.
Running the pipeline sends the same rows through all three transforms.
Four of the five rows converted cleanly and one did not, arriving at Output Cast Errors instead. Type conversion is the one job of these three that can fail at runtime, so it is worth handling deliberately rather than letting the pipeline stop.