I created an ExecuteRuleEngine processor for Apache Nifi. It allows to check the incomming flow file against defined business rules.
Checks are done on the individual field level. You can check the length of values, if the value is within an upper/lower limit, if it's smaller, greater or equal a given value, if it's in a given list of values, if it matches a regular expression, if it's a number and much more.
The flow file's content can be one or multiple rows of CSV data. The processor splits the rows into the individual fields and executes the business rules on each row of data. For each row of data a flow file is created.
The results of the ruleengine execution for each row of data are assigned to the flow file attributes: how many group of rules passed or failed, how many rules passed or failed, how many actions ran, etc.
Based on these attributes one can then filter the flow files as illustrated below. The GetFile processor retrieves the file, the ExecuteRuleEngine processor executes the business rules and the RouteOnAttribute processor is used to filter the flow files based on the ruleengine results (which are in the flow file attributes).
You can filter the data and route it to another processor. Or you can also update and filter the data and route it.
The busines rules can be defined and orchestrated using the Business Rules Maintenance Tool - a web application which is freely available. Once the logic is complete, it can be exported into a single project zip file. This file is used by the ExecuteRuleEngine processor.
Now if the logic changes - because the project zip file is external to Nifi - you won't have to change your Nifi flow - just update the logic in the Business Rules Maintenance Tool and export the project. So you won't need to hardcode logic in your flow and constantly adjust the flow if logic changes.
All components are available on my GitHub account. All is open source under the Apache licence.
Carpe Diem