PDI pre-migration audit risks
The audit (hop-import -t pdi-audit, or File → Audit a PDI project before a migration…)
writes pre-migration-audit.md and audit.json. The report has four sections:
| Section | What it holds |
|---|---|
| Inventory | Jobs (transformations and jobs alike), reusable parts (the sub-transformations Mapping steps call), steps and entries, jobs no other job runs, steps with code and their lines of code |
| Expected conversion | The count per status, from Apache Hop's Kettle import run into a temporary folder |
| Risks | Each risk found, its level, the number of jobs it is in, what it means and what reduces it |
| Not in the audit | What the files cannot tell: schedules, JDBC drivers and plugins in PDI's folders, a repository's content |
audit.json holds the same, with a few examples of each risk.
Risks
| Risk | Level | What it means | What reduces it |
|---|---|---|---|
| Code that calls the Pentaho API | HIGH | User Defined Java Classes and scripts that use PDI's classes do not compile or run in Hop until they use Hop's API | Port each one to Hop's API (org.apache.hop) and test it on sample rows |
| Steps and entries without a Hop plugin | HIGH | Hop opens them as missing plugins | Decide per type: a Hop transform that does the same, a plugin, or a rebuild |
| Steps the import did not write | HIGH | Apache Hop's Kettle import wrote no transform or action for them | Check the import log for these files |
| Connections no file defines | HIGH | Steps and entries name a connection that is in no file and not in the shared.xml given, so there is nothing to connect to | Import with the project's shared.xml, or define the connections in Hop's metadata |
| Clustered steps | HIGH | Hop has no cluster schemas: the steps run in one process | Check the volumes; scale out with more copies of a transform, a remote run configuration, or Apache Beam |
| Steps to replace, or to install as plugins | MEDIUM | Hop has no plugin with their PDI id; some exist as separate plugins | Install the plugin in every Hop that runs the project, or replace the step |
| Database types without a Hop database plugin | MEDIUM | The connections are imported as Generic connections, without a URL or driver | Install the database plugin, or set the URL and driver class of each connection |
| Connections with one name and different settings | MEDIUM | Hop keeps one connection per name; the import saves the first settings it meets | Decide per connection which settings are right; give the others their own name or variables |
| Log tables | MEDIUM | The import leaves out transformation, step and job log tables | Log with Hop's Pipeline Log and Workflow Log, or execution information; move the reports that read the old tables |
| Execution on slave servers (Carte) | MEDIUM | The import leaves out slave servers; the steps and entries run locally | Run them on a Hop Server through a remote run configuration |
| Partitioned steps | MEDIUM | The import leaves out partition schemas | Define the partition schemas again in Hop's metadata and set them on the transforms |
| Metadata Injection | MEDIUM | Hop has Metadata Injection, but the injection keys of some transforms differ from PDI's | Open each injection in Hop and check that every key is mapped |
| References through a PDI repository | MEDIUM | Hop has no repository: these steps and entries name a job or transformation in the repository, not a file | Export the repository to files and check each reference |
| Variables the project does not define | MEDIUM | Their values come from kettle.properties, the scheduler or the command line | Put a value for each in the Hop environment of every stage |
| KETTLE_ settings in kettle.properties | MEDIUM | The import copies them as variables, but Hop reads its settings under HOP_ names | Set the HOP_ equivalent of each one |
| SQL with variables | MEDIUM | Only a run shows what the queries become | Run the converted files with the production values on a test database |
| Operating system commands | MEDIUM | The commands must exist, and behave the same, where Hop runs | Check each command on the target servers |
| Names and values with .ktr or .kjb | LOW | The import replaces .ktr and .kjb in all text: file references, but also step names, URLs and constants | Check the steps the report lists, and any value that should keep .ktr or .kjb |
| JDBC drivers Hop does not ship | LOW | These connections need their driver in Hop's lib/jdbc | Copy the drivers from PDI's lib/ folder into every Hop that runs the project |
| Hard-coded file paths | LOW | Drive letters, shares and absolute paths must exist where Hop runs | Move the paths into variables of the environment |
| Formula steps | LOW | They run as Hop's OpenFormula transform, which evaluates PDI's formula syntax | Compare the results of the formulas on test data |
| Data Validator steps | LOW | Not every validation carries over | Test each validation with rows that must fail |
The levels say how much a risk can cost, not how often it occurs. Read the number of jobs next to each one: one HIGH risk in two files can be less work than a MEDIUM one in two hundred.
Why and how to move, and how we help: Migrations to Apache Hop, and from Pentaho PDI in particular.