Component mapping
This page is kept in the importer's repository, next to the code it describes, and copied here.
Every Talend component the importer has a converter for, the Apache Hop transform or workflow action
it becomes, and how the migration report classifies it. The registry this describes is
TalendConverterFactory (pipeline transforms) and WorkflowActionConverterRegistry (workflow
actions); ComponentMappingDocTest fails when a component registered there is missing here.
Status is the status the migration report gives the component:
- AUTO — converted by a dedicated converter, with no note asking for review. AUTO says what the importer did; it does not guarantee that the result behaves identically.
- PARTIAL — converted, with a
REVIEWnote about one of its settings. - MANUAL — converted only in part, or kept as a placeholder: a
TODOorREVIEW REQUIREDnote says what to do by hand. Some components are MANUAL by name, whatever their settings: the loops and catchers Hop has no equivalent for, checkpoints,tFlowMeter,tSetGlobalVar,tSetEnv, stored procedures, joblets,tFileInputRegex,tFileCompare,tCreateTemporaryFileand the partial FTP components. They are always flagged. - UNSUPPORTED — no converter, or the converter failed.
"AUTO; PARTIAL when ..." in the table means the converter adds a REVIEW note in that case; the
other cases are AUTO. The notes in a transform's or action's description use a fixed set of
markers, which the report finds wherever they are: TODO, REVIEW REQUIRED and MANUAL REVIEW REQUIRED make the component MANUAL, REVIEW PARTIAL; a NOTE is information that needs no check
(a setting that only affects the console, for example) and does not change the status. Two notes can apply to any component: a password Talend encrypted (left
empty in Hop, PARTIAL) and a file name or path with Java the importer cannot translate (PARTIAL).
The tJava family is the exception to the notes: it is AUTO when every line of its code was
translated and MANUAL otherwise, and it is always flagged for review.
Components not listed here have no converter: they become a Dummy placeholder that keeps their
name, position and hops, and are reported UNSUPPORTED. So does a component whose converter throws,
with the error in its description. Every placeholder delivers the output schema its component
declares, so that the transforms after it start: one that starts a flow (a catcher, an unsupported
input) becomes a Data Grid with that schema and no rows, one in a flow adds the declared columns its
input lacks as typed nulls. Components without a converter include
the ELT and SQL template components (tELTMysqlMap, tSQLTemplate*),
tSocketInput / tSocketOutput, tPOP, tMomInput / tMomOutput, tLibraryLoad and
tAddLocationFromIP.
"Workflow:" marks the action the component becomes when it runs from a generated workflow (a control-flow, pre-job or post-job workflow, or a job made only of file and FTP actions).
File names, paths and other text settings go through one translator for every component: literals,
context.x, globalMap.get("x") and current-date formatting (TalendDate.getDate,
TalendDate.formatDate(..., TalendDate.getCurrentDate()), SimpleDateFormat(...).format(new Date())), which becomes ${TALEND_DATE_<pattern>}, set when the job starts (see the README). A
file name with other Java is flagged for review.
Components
| Talend component | Hop transform / action | Status | Notes |
|---|---|---|---|
tAddCRCRow | Add a checksum | AUTO | |
tAdvancedFileOutputXML | XML Output (Advanced) | AUTO | The LOOP table's first element is the loop element, with or without a column. A GROUP element groups the rows on its first own column (Hop compares one field, Talend all of them). A DOCTYPE or stylesheet only with "Add Document type". Hop does not indent like Talend. |
tAggregateRow | Memory Group By | AUTO; PARTIAL when an output column is never written | Group columns written to an output column of another name, or a schema in another order, get a Select Values after it. An output column no table row writes gets Java's default (false/0 for a non-nullable primitive, null otherwise) and a review note. |
tAggregateSortedRow | Memory Group By | AUTO; PARTIAL when ROW_COUNT is set | Hop does not need sorted input. Talend writes the last group when the ROW_COUNT-th row arrives, Hop when the rows end (the same only when ROW_COUNT is the number of rows): a REVIEW note says so. |
tAssert | Java Filter, false rows to a Write to Log | AUTO; MANUAL when the condition cannot be a Java Filter | A Row Generator bootstrap when nothing feeds it. |
tAssertCatcher | Placeholder with the catcher's schema (no rows) | MANUAL | Hop has no assert event bus: the placeholder emits no rows (TODO). Each tAssert logs its failures (Write to Log). |
tBufferInput | — (hops rewired to the paired tBufferOutput's source) | AUTO | Omitted from the pipeline. |
tBufferOutput | Copy rows to result | AUTO | A parent tRunJob's executor passes the rows on as its output rows. |
tCheckpoint | User Defined Java Class | MANUAL | Talend recovery checkpoints have no Hop equivalent; see the report appendix. |
tCheckpointEnd | Dummy placeholder | MANUAL | See the report appendix. |
tCheckpointStart | Dummy placeholder | MANUAL | See the report appendix. |
tChronometerStart | Workflow: JavaScript (stores the start time) | AUTO | |
tChronometerStop | Workflow: JavaScript (logs the elapsed time) | AUTO | |
tContextDump | Data Grid (snapshot of the context) | AUTO | |
tContextLoad | Row Denormaliser → Replace null value → Set Variables | AUTO; MANUAL without key / value columns or context variables | Sets the job's variables in the root workflow. |
tConvertType | Select Values (metadata) | AUTO; PARTIAL with AUTOCAST off or MANUALTABLE | |
tCreateTemporaryFile | Workflow: Set Variables + Create File | MANUAL | TEMPLATE placeholders approximated. |
tDenormalize | Memory Group By (concatenate) | AUTO; PARTIAL with MERGE | MERGE=false keeps every value; MERGE=true each value once, sorted (Talend: first-seen order; a REVIEW note says so). |
tDie | Abort; workflow: Abort when reached by a trigger or without rows | AUTO | A tDie that starts its subjob (no input rows) is a workflow action, also in a job of tDie/tWarn and actions only. |
tDTDValidator | Workflow: DTD Validator | PARTIAL | Needs XML_ALLOW_DOCTYPE_DECL=Y (the report's Hop settings list it): with Hop's default the action refuses the DOCTYPE and reports every file invalid. |
tExternalSortRow | Sort Rows (+ User Defined Java Expression → Text File Output) | AUTO | Hop sorts in-process (not SORT_CMDPATH); the sorted rows are also written to FILENAME as Talend's external sort leaves it (FIELDSEPARATOR, \r\n, no header or quotes). ADD_EOF is not reproduced. |
tExtractDelimitedFields | Split Fields; User Defined Java Class for a regular-expression separator | AUTO | Output columns named like input columns are copied (FIELD too: a copy of it is split), the others take the split values. With no other columns the row is only copied. FIELDSEPARATOR is a regular expression (Talend's String.split): one fixed text is split on by Split Fields, anything else by the class. A String column gets "" for an empty or missing value. |
tExtractPositionalFields | String Cut → Select Values | AUTO | Output columns named like input columns are copied, FIELD's column is left unset (null), the other columns are cut from FIELD with PATTERN. ADVANCED_OPTION formats are not applied. |
tExtractRegexFields | User Defined Java Class → Select Values | AUTO | One row per match of REGEX found in FIELD (Java find); groups fill the columns the input does not have. Rows without a match go to the REJECT link (error output) or nowhere. |
tExtractXMLField | Filter Rows → Get data from XML (XML source in a field) → Select Values | AUTO; PARTIAL when DIE_ON_ERROR is off without a REJECT link, an output column is never written, or an XPath is Java the importer cannot translate | XMLFIELD, LOOP_QUERY, MAPPING (QUERY, NODECHECK), LIMIT, IGNORE_NS as for tFileInputXML. An output column without QUERY is copied from the input column of its name (unset when there is none); rows whose XML is null give nothing. Rows whose XML does not parse go to the REJECT link (error handling) with the XML in errorXMLField; Hop's error message is not Talend's. |
tFileArchive | Workflow: Zip File | AUTO | |
tFileCompare | User Defined Java Class; workflow: File Compare | MANUAL | Binary compare in the workflow. Paths and messages are translated like other Talend text and resolved by the transform. |
tFileCopy | Workflow: Copy Files (Move Files with REMOVE_SOURCE) | AUTO | |
tFileDelete | Workflow: Delete File | AUTO | |
tFileExist | Workflow: File Exists | AUTO | |
tFileExists | Workflow: File Exists | AUTO | Alias. |
tFileFetch | Workflow: HTTP or Copy Files | AUTO; PARTIAL with POST, UPLOAD or a path with Java | The pipeline keeps a Dummy. |
tFileInputARFF | User Defined Java Class | AUTO | Reads the ARFF file as Talend's code does with Weka 3.4 (header, dense and sparse records, quotes, comments, ?); column i takes attribute i: a String the nominal, string or date value, a number the value cast to the column's type (a missing value 0). |
tFileInputDelimited | Text File Input | AUTO | |
tFileInputExcel | Microsoft Excel Input | AUTO | A numeric cell read into a String column is written as Talend does (12345678, not 12345678.0). |
tFileInputFullRow | Text File Input (one field per line) | AUTO | |
tFileInputLDIF | User Defined Java Class | AUTO; PARTIAL with a List column | The class parses LDIF as Talend's ldapjdk reader does (folding, comments, base64, change records) and fills the columns as Talend's code does. |
tFileInputMSDelimited | User Defined Java Class per linked schema | AUTO | Talend's reader: record type at COLUMNINDEX, parent keys (counter or KEY_COLUMN_INDEX) in a child's last column, CARDINALITY checks, CSV_OPTION and USE_MULTISEPARATORS. Each schema reads the file in its own transform, so the order between record types is not kept. |
tFileInputMSPositional | User Defined Java Class per linked output (and REJECT) | AUTO | Header at HEADER_RECORD_POSITION, PATTERN per schema with CHECK_ROW_SIZE, the parent's PARENT_KEY_COLUMN in a child's KEY_COLUMN, REJECT rows (errorCode 1 parse error, 2 unknown header), HEADER/FOOTER/LIMIT as Talend computes them. The order between outputs is not kept. |
tFileInputMSXML | User Defined Java Class per linked schema | AUTO; PARTIAL with GENERATION_MODE SAX | Each schema reads the file in its own transform, so the order between the schemas' rows is not kept. GENERATION_MODE SAX is read like Dom4j (flagged for review). |
tFileInputMail | User Defined Java Class (Jakarta Mail) | AUTO; MANUAL for an MSG file | MIME (.eml) files: the MAIL_PARTS headers and body as Talend's generated code reads them, attachments saved to ATTACHMENT_PATH. MSG (Outlook .msg) becomes a placeholder with a TODO: Hop has no Apache POI HSMF for a User Defined Java Class. |
tFileInputPositional | Text File Input (fixed width) | AUTO | |
tFileInputProperties | Text File Input (key = value) | AUTO | |
tFileInputRegex | Text File Input → Regex Evaluation | MANUAL | |
tFileInputXML | Get data from XML | AUTO; PARTIAL when an XPath is Java the importer cannot translate | LOOP_QUERY and each MAPPING QUERY are Java expressions: literals and context variables (${x}, resolved by Get data from XML) are translated. NODECHECK ("Get Nodes") gives the node's XML (result type single node); a missing node gives an empty string where Talend gives null. |
tFileList | Get File Names; with an ITERATE link, Get File Names → Pipeline or Workflow Executor running the iterated part per file | AUTO when it drives an iteration; MANUAL otherwise | Each file's values reach the iterated part as <tFileList>_CURRENT_FILEPATH, _CURRENT_FILE, _CURRENT_FILEDIRECTORY, _CURRENT_FILEEXTENSION and _NB_FILE (see the README). In a row flow (with tIterateToFlow, say) it gives one row per file instead of globalMap values (REVIEW REQUIRED). |
tFileOutputDelimited | Text File Output (with CSV options: User Defined Java Expression → Select Values → Text File Output) | AUTO | With CSV options every value is quoted, as Talend's CSV writer does: values that are not Strings are written as text first. |
tFileOutputExcel | Microsoft Excel Writer | AUTO | |
tFileOutputLDIF | User Defined Java Class | AUTO; PARTIAL with an output flow | The class writes the records as Talend's code does (CHANGETYPE, MODIFYCONFIG / MULTIVALUECOLUMNS, base64, WRAP folding) and passes the rows on. |
tFileOutputMSDelimited | User Defined Java Class reading each input flow as an info stream | AUTO | Each child row written under its parent row (KEY_COLUMN = parent's PARENT_KEY_COLUMN), depth first; FIELDSEPARATOR, per-schema FIELD_SEPARATOR, CSV_OPTION (every value quoted). Child rows without a parent are not written, as in Talend. |
tFileOutputMSPositional | User Defined Java Class reading each input flow as an info stream | AUTO | As tFileOutputMSDelimited, each value fitted to the schema's PATTERN size with PADDING_CHAR, ALIGN and KEEP. |
tFileOutputMSXML | User Defined Java Class (one info stream per input) | AUTO; PARTIAL with namespaces, MAP, or an input without a LOOP element | Inputs are read in merge order and the document is built as Talend's generated code does. Namespaces are written by prefix (flagged for review). |
tFileOutputPositional | User Defined Java Class → Select Values → Text File Output | AUTO | Each field is padded with PADDING_CHAR, aligned and cut (KEEP) to its FORMATS size by the class, as Talend does. |
tFileOutputXML | XML Output (+ User Defined Java Class with SPLIT) | AUTO | Booleans are written true/false. With SPLIT, a class after the XML Output gives the split files Talend's names (name0.xml, name1.xml, ...: the number before the last . of the canonical path) once the last one is closed; no rows leave no file. |
tFileProperties | Get File Names → Select Values → Calculator → User Defined Java Expression (→ Load File → Checksum) → Select Values | AUTO | mode_string is r/- then w/- from File.canRead()/canWrite() and mtime_string is Date.toString(), as Talend's code sets them. |
tFileRowCount | Get Files Rows Count | AUTO; PARTIAL with IGNORE_EMPTY_ROW or an ENCODING | <cid>_COUNT is published as a variable (Set Variables) when the job reads it. |
tFileTouch | Workflow: Create Folder + Create File | PARTIAL | Creates the file when it is missing; an existing file's modification time is not updated (REVIEW). |
tFileUnarchive | Workflow: Unzip File | AUTO | |
tFilterColumns | Schema Mapping | AUTO | |
tFilterRow | Filter Rows; Java Filter for advanced-mode expressions | AUTO; MANUAL when neither can express the condition | A condition neither can express becomes a Dummy that passes every row to both outputs (MANUAL REVIEW REQUIRED). The REJECT flow's errorMessage names every condition, where Talend names only those a row failed with several AND conditions (REVIEW). |
tFixedFlowInput | Data Grid; values it cannot hold make it a chain: Row Generator → Get Variables (context) → Add sequence (Numeric.sequence) → Get System Info (TalendDate.getCurrentDate / getDate) → Fake data (TalendDataGenerator names, US addresses) → Select Values in schema order. TalendDate.parseDate of literals folds to a value with its pattern; other expressions leave the column empty with a TODO. An ARRAY cell with a parenthesis is one expression (never split on its commas); otherwise it is a list of literals, picked row by row. NB_ROWS from the context becomes the Row Generator's limit (${x}); one the importer cannot read gives one row and a review note. | AUTO; MANUAL when an expression is left empty | |
tFlowMeter | Dummy (pass-through) | MANUAL | The rows it reads are the meter's count, which a tFlowMeterCatcher's logging pipeline reports. |
tFlowMeterCatcher | Logging pipeline: Pipeline Logging → the meters' messages matched with Hop's line counts → the catcher's columns → its subjob; a Pipeline Log for the job's pipelines | PARTIAL | REVIEW: Hop's log channel ids for the pids, the default context; reference count from the link's target or source. A job whose Stats & Logs settings catch the meters keeps a placeholder (as Talend, no rows). See design patterns. |
tStatCatcher | Logging pipeline: Pipeline Logging / Workflow Logging → the job's and the components' begin and end matched with what Hop logged → the catcher's columns → its subjob; a Pipeline Log and/or Workflow Log for the job's files | PARTIAL | REVIEW: a component's begin and end come when its pipeline or workflow ends; Hop's log channel ids for the pids; the default context. A job whose Stats & Logs settings catch the statistics keeps a placeholder (as Talend, no rows). |
tFlowToIterate | Its input rows → Pipeline or Workflow Executor running the iterated part per row; otherwise an unconnected Dummy placeholder | AUTO when the iteration is split; MANUAL otherwise | Each column reaches the iterated part as <row>.<column>, or under its own keys without the default mapping. |
tForeach | Data Grid of its values → Pipeline or Workflow Executor; otherwise an unconnected Dummy placeholder; workflow: Dummy | AUTO when the iteration is split; MANUAL otherwise | Split only when the values are literals; the value reaches the iterated part as <tForeach>_CURRENT_VALUE. |
tFTPDelete | Workflow: FTP Delete | AUTO | |
tFTPFileExist | Workflow: File Exists on an ftp:// / sftp:// path | MANUAL | |
tFTPFileList | Get File Names on ftp:// / sftp://; workflow: Shell | MANUAL | |
tFTPGet | Workflow: Get a file with FTP | AUTO | |
tFTPPut | Workflow: Put a file with FTP | AUTO | |
tFTPRename | Workflow: Shell (curl / sftp batch) | MANUAL | |
tFuzzyMatch | Fuzzy Match; User Defined Java Class for every-match or phonetic matching | AUTO | MATCHING is the matched lookup value(s), VALUE the distance(s) or phonetic code. Levenshtein with a unique match is a Fuzzy Match; other settings run Talend's algorithm in a User Defined Java Class that reads the lookup flow as an info stream. Joined matches follow the lookup rows' arrival order. |
tGroovy | Script (Groovy engine), after a one-row Row Generator | AUTO; PARTIAL when a VARIABLES value is Java | CODE runs once with the VARIABLES in its binding; with an incoming flow it is the start script. |
tGroovyFile | Script (Groovy engine), after a one-row Row Generator | AUTO; PARTIAL when a VARIABLES value is Java | The script evaluates GROOVY_FILE once, in the same binding as the VARIABLES. |
tHashInput | De-serialize from file | AUTO; MANUAL for other shapes | When triggers order every writer of its store before it; several writers: one De-serialize from file each, joined by Append streams in run order. The control-flow workflow deletes the files when the job ends. Other shapes: Dummy source placeholder with a TODO. |
tHashOutput | Serialize to file | AUTO | A file per store and run under ${java.io.tmpdir}/hop-talend-hash; Dummy (pass-through) when no later subjob reads it. |
tIntervalMatch | User Defined Java Class (lookup flow as info stream) | AUTO | One row per main row with LOOKUP from the interval that holds MAIN_COLUMN, null when none does. With overlapping intervals the first in (min, max) order wins. |
tInfiniteLoop | Dummy placeholder | MANUAL | ITERATE loop: the part it iterates runs once. Rebuild it as a Hop workflow loop. |
tJava | User Defined Java Class; Set Variables when its code only puts translatable values in globalMap | AUTO when the code is fully translated; MANUAL otherwise | Always flagged for review: untranslated code is kept as a comment, listed per idiom with its CODE line. |
tJavaFlex | User Defined Java Class | AUTO when the code is fully translated; MANUAL otherwise | Start part on the first call, main part per row, end part after the last row, sharing variables (the start's become fields); without input the main part runs once. Without input the three parts run in one block, so a loop the start opens and the end closes runs the main part, and sends its row, on every turn. With an input, a block opened in one part and closed in another is left for review. Always flagged. |
tJavaRow | User Defined Java Class | AUTO when the code is fully translated; MANUAL otherwise | Always flagged for review. rowN names, assignments to input fields and the output row kept from row to row (a column not set keeps the previous row's value) translate. |
tJoin | Stream Lookup (→ Filter Rows) → Select Values | AUTO; PARTIAL for uneven JOIN_KEY pairs or unwritten columns | One row per main row, in main order; main columns by position, LOOKUP_COLS under their output names. With inner join, unjoined rows go to the REJECT link (dropped without one). One lookup row per key. |
tLogCatcher | Placeholder with the catcher's schema (no rows) | MANUAL | No Stats & Logs bus in Hop: the placeholder emits no rows; see the report appendix. |
tLogRow | Write to Log | AUTO | |
tLoop | Data Grid of the counter → Pipeline or Workflow Executor; otherwise an unconnected Dummy placeholder; workflow: Dummy | AUTO when the iteration is split; MANUAL otherwise | Split only for a for loop with literal bounds and at most 10,000 iterations; the values reach the iterated part as _CURRENT_VALUE and _CURRENT_ITERATION. While loops stay placeholders. |
tMap | Calculator + Select Values; Stream Lookup / Merge Join chains; a filter per output (Filter Rows, Java Filter or User Defined Java Class) | AUTO; PARTIAL or MANUAL by its notes | Falls back to a Select Values with a TODO when an expression cannot be translated. A join key that is an expression is computed before the lookup; a catch-output-reject table gets the rows no other output's filter accepted. An expression or filter calling the project's own routines (code/routines, not system) runs them: the routine's source is carried into the User Defined Java Class. An output filter is never dropped: one that cannot be translated stays, unchanged, in a Java Filter with a TODO (the pipeline does not start until it is rewritten); an input table's own filter is reported as a TODO. |
tMsgBox | Write to Log | AUTO | |
tMysqlBulkExec | Execute SQL, once: Talend's LOAD DATA LOCAL INFILE statement | AUTO; PARTIAL for DATA_ACTION UPDATE | REPLACE / IGNORE, CHARACTER SET from ENCODING, the separators, enclosure and escape character, the schema's columns, as the template writes the statement; TABLE_ACTION as an SQL action before the pipeline. The component's own Hop connection gets allowLoadLocalInfile=true (Talend's URL has it); the MySQL server needs local_infile on. A BIT column (the schema's DB type) is read into @<cid>_<column> and set with CAST(... AS UNSIGNED), and columns go by their database name (originalDbColumnName), as the template does. DATA_ACTION UPDATE (a temporary table) is not converted. |
tMysqlLastInsertId | Dummy; the Table Output of the tMysqlOutput right before it returns the auto-generated key as last_insert_id, without batch updates | AUTO; PARTIAL when that tMysqlOutput uses extended inserts (Talend's default) or last_insert_id is not Integer/Long; MANUAL (TODO placeholder) after anything else | Talend runs SELECT LAST_INSERT_ID() on the shared connection for each row: the key generated for the row just inserted, which is what the Table Output returns. A row that generates no key fails in Hop (Talend reads the previous value). |
tMysqlOutputBulk | User Defined Java Expression (each row's line) → Text File Output (the lines as they are) | AUTO | Every value enclosed, \N for null, a Boolean as 1/0, a Date with its pattern, other values with String.valueOf; a backslash and the enclosure escaped with a backslash (always with RECORDS_CONTAIN_NULL_VALUE), or the enclosure doubled when it is the escape character; ROWSEPARATOR, ENCODING, APPEND, CREATE as Talend. |
tMysqlOutputBulkExec | tMysqlOutputBulk's two transforms → Memory Group By (one row once the file is closed) → Execute SQL with tMysqlBulkExec's statement | AUTO; PARTIAL for DATA_ACTION UPDATE | The separators are copied as the virtual component's TEMPLATE_PARAMs copy them; TABLE_ACTION as an SQL action before the pipeline. BIT columns as for tMysqlBulkExec. Not converted, as for tMysqlBulkExec: DATA_ACTION UPDATE. |
tMysqlSCD, tPostgresqlSCD | Table Input + Select Values (the current rows, read once) → Stream Lookup on the source keys → User Defined Java Expression (Talend's comparisons, a 1/0 gate per statement) → Execute SQL per row: type 1 UPDATE, type 2 close, INSERT, type 3 UPDATE | AUTO; PARTIAL for SK_CREATION ROUTINE, TABLE_ACTION (MySQL) or a REJECT link | Talend's statements and order (t<Vendor>SCD_main.javajet), with the vendor's differences (ScdVendor): MySQL compares a Date within 999 ms and gives a new type 2 version the current row's type 0 values; PostgreSQL compares a Date with equals, takes the input's type 0 values, puts the table in SCHEMA_DB ("schema"."table") and takes a DB_SEQUENCE key from nextval('schema.sequence'). Not Hop's Dimension lookup/update: it cannot find a current row whose end date is NULL and writes its own end dates. Differences: a source key that comes twice in one run is compared with the dimension as it was before the run; DIE_ON_ERROR off (skip a failing row) and the REJECT flow are not converted; SK_CREATION ROUTINE becomes the table's MAX + 1 (flagged); TABLE_ACTION is not converted (flagged); the first key of an empty type 2 dimension whose rows are all closed is MAX + 1 where Talend starts at 1. Other vendors' SCD components (tMSSqlSCD, tOracleSCD, …) have no converter. |
tNormalize | Split field to rows → Select Values | AUTO | |
tReplace | Replace in String | AUTO; MANUAL when its ADVANCED_SUBST rules cannot be read | |
tReplaceList | User Defined Java Class, lookup flow as info stream | AUTO | Every LOOKUP_SEARCH_COLUMN value of the lookup, in lookup order, replaced by its LOOKUP_REPLACEMENT_COLUMN (replaceAll of the quoted search, (?i) with CASE_INSENSITIVE) in the COLUMN_OPTIONS columns whose type matches the lookup's, as Talend does. |
tReplicate | Dummy, copying rows to every output | AUTO | |
tRowGenerator | Data Grid; values it cannot hold make it a chain: Row Generator → Get Variables (context) → Add sequence (Numeric.sequence) → Get System Info (TalendDate.getCurrentDate / getDate) → Fake data (TalendDataGenerator names, US addresses) → Select Values in schema order. TalendDate.parseDate of literals folds to a value with its pattern; other expressions leave the column empty with a TODO. | AUTO; MANUAL when an expression is left empty | |
tRSSInput | Get data from XML | AUTO; PARTIAL with an article date filter or a limit from the context | |
tRunJob | Pipeline Executor; workflow: Run Pipeline / Run Workflow | AUTO | The child job is resolved within the import. The rows a child writes to tBufferOutput become the executor's output rows (with no input row, or PROPAGATE_CHILD_RESULT). |
tSampleRow | Sample Rows | AUTO; PARTIAL when RANGE is an expression | RANGE becomes sorted, merged line ranges (Talend accepts overlapping or unsorted ranges, Hop does not); tokens Talend ignores are left out. A RANGE expression needs review. |
tSchemaComplianceCheck | Data Validator, REJECT as error hop | AUTO; PARTIAL with a REJECT flow | The same rows are rejected, but their errorCode / errorMessage are the Data Validator's codes and wording, not Talend's (a review note says so when there is a REJECT flow). Identical codes and messages would need a full port of Talend's checker into a User Defined Java Class: the code its main template runs is generated by tSchemaComplianceCheck_begin.javajet, about 567 lines. |
tSendMail | Workflow: Mail | AUTO | |
tServerAlive | Workflow: Ping / Telnet | AUTO | |
tSetEnv | Set Variables | MANUAL | Process environment becomes ROOT_WORKFLOW variables. Without an input flow the values are the variables' default values, so they are set. |
tSetGlobalVar | Set Variables | MANUAL | Dynamic globalMap use needs review. Inside a flow the rows pass on and the last row sets the variables; without a flow the values are the variables' default values. |
tSleep | Workflow: Wait for (delay); Delay row when it receives rows | AUTO; PARTIAL when PAUSE is Java in a row flow | In a row flow it waits PAUSE seconds for every row, as Talend does; a PAUSE that is a Java expression is flagged for review. |
tSortRow | Sort Rows | AUTO | |
tSSH | SSH | AUTO | |
tSystem | Workflow: Shell | AUTO | |
tUniqRow | Unique Rows (HashSet); User Defined Java Class for case-insensitive keys or one duplicate per key | AUTO | First row of each key to UNIQUE (main output), the others to DUPLICATE (error output), input order kept. |
tUnite | Append chain (two or more inputs) or pass-through | AUTO | |
tWaitForFile | Workflow: Wait for File | AUTO | |
tWaitForSqlData | Workflow: Wait for SQL | AUTO | |
tWarn | Write to Log; workflow: Write to Log when reached by a trigger or without rows | AUTO | A tWarn that starts its subjob (no input rows) is a workflow action, also in a job of tDie/tWarn and actions only. |
tWebServiceInput | Web Service Lookup with a Row Generator bootstrap | MANUAL | REVIEW REQUIRED: load the WSDL in the transform to align the input and output fields. |
tWriteXMLField | XML Output (Advanced), value mode | AUTO | |
tXMLMap | Dummy placeholder (pass-through, schema kept) | MANUAL | The mapping is not converted: a TODO asks to rebuild it with XML Output (Advanced) or by hand. |
tXSDValidator | Workflow: XSD Validator | AUTO | |
tXSLT | User Defined Java Class running javax.xml.transform as Talend's template does (once, or per row with an input flow) | PARTIAL (REVIEW: the Saxon version) | The stylesheet and the XML are read as files, so a relative xsl:include, xsl:import or document() resolves next to them as in Talend (Hop's XSL transformation action resolves it against its working directory, which is why it is not used). Saxon when it is on the classpath (Talend's factory), the JDK's otherwise; PARAMS; the output file overwritten; <cid>_OUTPUT_FILEPATH / _OUTPUT_FILENAME as variables up to the root workflow; OVERRIDE_INPUT_XML_FILE / OVERRIDE_INPUT_XSL_FILE / OVERRIDE_OUTPUT with columns; CACHE_SOURCE reads the XML into memory first. Talend runs Saxon-HE 10.6, Hop 2.19 ships Saxon 8.7 (lib/core), whose format-number differs (1234 for Talend's 1234.0): with Saxon-HE 10.6 in its place the output is Talend's, hence the REVIEW note. |
Database components
Every database vendor maps the same way, on a Hop connection named after the Talend repository connection (or one per database), with the vendor's Hop database type. Generic JDBC components use a Hop generic connection with the component's URL and driver class.
| Kind | Hop transform / action | Status |
|---|---|---|
t<Vendor>Input, tDBInput, tHSQLDbInput | Table Input (<name>_query) → Select Values that gives the Boolean and numeric columns the schema's types (Talend reads each column with its schema type); select items get the schema's names as aliases (Talend reads columns by position); HSQLDB in memory or in-process becomes embedded H2 | AUTO; PARTIAL when QUERY is a Java expression |
t<Vendor>Output, tDBOutput, tHSQLDbOutput | Table Output, Insert / Update (also tPostgresqlOutput's UPSERT), Update or Delete by DATA_ACTION; TABLE_ACTION as an SQL action run before the pipeline (in <job>_control_flow.hwf for the main pipeline) | AUTO |
t<Vendor>Row, tDBRow | Execute SQL, once per row when rows flow in; a prepared statement (USE_PREPAREDSTATEMENT) and a query concatenated from the row's columns ("... '" + row1.city + "' WHERE id = " + row1.id) bind the columns to ? parameters; other Java in a per-row query is flagged for review; USE_PREPAREDSTATEMENT with values that are not columns, and QUERYSTORE, become a placeholder | PARTIAL: every one carries a REVIEW note (Talend's REJECT, COMMIT_EVERY and batching differ); MANUAL for a placeholder |
t<Vendor>SP, tDBSP | Call DB Procedure (a function only with IS_FUNCTION); RECORDSET arguments not mapped | MANUAL |
t<Vendor>Connection | Workflow Executor running a Check DB Connections sidecar; workflow: Run Workflow on it | AUTO |
t<Vendor>Commit, t<Vendor>Rollback | Workflow: SQL COMMIT / ROLLBACK; run the job with the local-transactional run configuration (the report lists the jobs). A final commit nothing runs after is left out (Hop transforms commit their own work) | PARTIAL: the REVIEW note asks for the transactional run |
t<Vendor>Close | Left out (Hop closes connections itself); a MySQL or PostgreSQL connection the job opened with auto-commit off and closes with work no commit or rollback followed: Workflow: SQL ROLLBACK (Talend's close discards that work) | AUTO; PARTIAL when the connection is opened in another job, the vendor's driver decides, or the triggers do not give its place in the run order (REVIEW) |
tCreateTable<Vendor>, tCreateTable | Workflow: SQL with the DDL; the pipeline keeps a Dummy. Names the database reserves are quoted as Hop quotes them | AUTO; MANUAL (TODO, a Dummy action) without a table or a connection |
The registered components, by vendor:
| Vendor | Input | Output | Row | SP | Connection | Commit | Rollback | Close | Create table |
|---|---|---|---|---|---|---|---|---|---|
| Generic | tDBInput | tDBOutput | tDBRow | tDBSP | tCreateTable | ||||
| HSQLDB | tHSQLDbInput | tHSQLDbOutput | |||||||
| Access | tAccessInput | tAccessOutput | tAccessRow | tAccessSP | tAccessConnection | tAccessCommit | tAccessRollback | tAccessClose | tCreateTableAccess |
| AS400 | tAS400Input | tAS400Output | tAS400Row | tAS400SP | tAS400Connection | tAS400Commit | tAS400Rollback | tAS400Close | tCreateTableAS400 |
| DB2 | tDB2Input | tDB2Output | tDB2Row | tDB2SP | tDB2Connection | tDB2Commit | tDB2Rollback | tDB2Close | tCreateTableDB2 |
| Exasol | tExasolInput | tExasolOutput | tExasolRow | tExasolSP | tExasolConnection | tExasolCommit | tExasolRollback | tExasolClose | tCreateTableExasol |
| Firebird | tFirebirdInput | tFirebirdOutput | tFirebirdRow | tFirebirdSP | tFirebirdConnection | tFirebirdCommit | tFirebirdRollback | tFirebirdClose | tCreateTableFirebird |
| Greenplum | tGreenplumInput | tGreenplumOutput | tGreenplumRow | tGreenplumSP | tGreenplumConnection | tGreenplumCommit | tGreenplumRollback | tGreenplumClose | tCreateTableGreenplum |
| Informix | tInformixInput | tInformixOutput | tInformixRow | tInformixSP | tInformixConnection | tInformixCommit | tInformixRollback | tInformixClose | tCreateTableInformix |
| Ingres | tIngresInput | tIngresOutput | tIngresRow | tIngresSP | tIngresConnection | tIngresCommit | tIngresRollback | tIngresClose | tCreateTableIngres |
| Interbase | tInterbaseInput | tInterbaseOutput | tInterbaseRow | tInterbaseSP | tInterbaseConnection | tInterbaseCommit | tInterbaseRollback | tInterbaseClose | tCreateTableInterbase |
| JDBC | tJDBCInput | tJDBCOutput | tJDBCRow | tJDBCSP | tJDBCConnection | tJDBCCommit | tJDBCRollback | tJDBCClose | tCreateTableJDBC |
| MSSql | tMSSqlInput | tMSSqlOutput | tMSSqlRow | tMSSqlSP | tMSSqlConnection | tMSSqlCommit | tMSSqlRollback | tMSSqlClose | tCreateTableMSSql, tCreateTableMSSQL |
| Mysql | tMysqlInput | tMysqlOutput | tMysqlRow | tMysqlSP | tMysqlConnection | tMysqlCommit | tMysqlRollback | tMysqlClose | tCreateTableMysql |
| Netezza | tNetezzaInput | tNetezzaOutput | tNetezzaRow | tNetezzaSP | tNetezzaConnection | tNetezzaCommit | tNetezzaRollback | tNetezzaClose | tCreateTableNetezza |
| Oracle | tOracleInput | tOracleOutput | tOracleRow | tOracleSP | tOracleConnection | tOracleCommit | tOracleRollback | tOracleClose | tCreateTableOracle |
| ParAccel | tParAccelInput | tParAccelOutput | tParAccelRow | tParAccelSP | tParAccelConnection | tParAccelCommit | tParAccelRollback | tParAccelClose | tCreateTableParAccel |
| Postgresql | tPostgresqlInput | tPostgresqlOutput | tPostgresqlRow | tPostgresqlSP | tPostgresqlConnection | tPostgresqlCommit | tPostgresqlRollback | tPostgresqlClose | tCreateTablePostgresql |
| SAPHana | tSAPHanaInput | tSAPHanaOutput | tSAPHanaRow | tSAPHanaSP | tSAPHanaConnection | tSAPHanaCommit | tSAPHanaRollback | tSAPHanaClose | tCreateTableSAPHana |
| Snowflake | tSnowflakeInput | tSnowflakeOutput | tSnowflakeRow | tSnowflakeSP | tSnowflakeConnection | tSnowflakeCommit | tSnowflakeRollback | tSnowflakeClose | tCreateTableSnowflake |
| SQLite | tSQLiteInput | tSQLiteOutput | tSQLiteRow | tSQLiteSP | tSQLiteConnection | tSQLiteCommit | tSQLiteRollback | tSQLiteClose | tCreateTableSQLite |
| Sybase | tSybaseInput | tSybaseOutput | tSybaseRow | tSybaseSP | tSybaseConnection | tSybaseCommit | tSybaseRollback | tSybaseClose | tCreateTableSybase |
| Teradata | tTeradataInput | tTeradataOutput | tTeradataRow | tTeradataSP | tTeradataConnection | tTeradataCommit | tTeradataRollback | tTeradataClose | tCreateTableTeradata |
| VectorWise | tVectorWiseInput | tVectorWiseOutput | tVectorWiseRow | tVectorWiseSP | tVectorWiseConnection | tVectorWiseCommit | tVectorWiseRollback | tVectorWiseClose | tCreateTableVectorWise |
| Vertica | tVerticaInput | tVerticaOutput | tVerticaRow | tVerticaSP | tVerticaConnection | tVerticaCommit | tVerticaRollback | tVerticaClose | tCreateTableVertica |
Handled outside the registry
| Talend component | What the importer does | Status |
|---|---|---|
tPrejob, tPostjob | The components they trigger become <job>_pre.hwf / <job>_post.hwf, run around the main pipeline by <job>_orchestration.hwf | AUTO |
tIterateToFlow | Folded into the row stream: its source's rows flow straight to its target, through a field-align step when the schemas differ. Only behind a tFileList or tFTPFileList; elsewhere its hops are left unresolved | PARTIAL behind a file list; UNSUPPORTED otherwise |
Joblets (jblt_*) | Pipeline Executor on the imported joblet pipeline; workflow: Run Pipeline | MANUAL |
TRIGGER_INPUT, TRIGGER_OUTPUT | Dummy topology markers inside a joblet | — |
Control-flow triggers (OnSubjobOk, OnComponentOk, RunIf, …) are not components: they become
the hops of <job>_control_flow.hwf, a RunIf through an Evaluation action, an OnComponentError
or OnSubjobError through a Success action that clears the handled error. Deactivated components
and links are left out. See
design-patterns.md.