You will need to have at least one Kafka broker running. If you run Kafka on a different machine, then apply the settings in the properties file.
The program reads data from a kafka topic named "travel_discount". The data is processed using the ruleengine and then the data is output to a topic named "travel_discount_ruleengine".
The business logic (rules) will do the following:
- check if the person travelling is below the age of 3. If so a 100% discount is applied.
- check if the person is 50 or older and if so apply a 25% discount
- Data from the "destination_region" field is translated to the appropriate three letter region code (lookup)
- Calculate the discounted price
I have Json formatted data in the source topic like this (sample):
{
"id": 1,
"age": 1,
"destination_region": "Asia",
"destination_airport": "SIN",
"price": 100,
"frequent_traveller": 0
}
{"id": 1,"age": 1,"destination_region": "Asia","destination_airport": "SIN","price": 100,"frequent_traveller": 0}
{"id": 2,"age": 12,"destination_region": "America","destination_airport": "DEN","price": 150,"frequent_traveller": 0}
{"id": 3,"age": 33,"destination_region": "Africa","destination_airport": "JNB","price": 120,"frequent_traveller": 0}
{"id": 4,"age": 24,"destination_region": "Europe","destination_airport": "FRA","price": 200,"frequent_traveller": 1}
{"id": 5,"age": 58,"destination_region": "Europe","destination_airport": "AMS","price": 190,"frequent_traveller": 0}
{"id": 6,"age": 45,"destination_region": "America","destination_airport": "LAS","price": 175,"frequent_traveller": 0}
{"id": 7,"age": 46,"destination_region": "America","destination_airport": "JFK","price": 230,"frequent_traveller": 0}
{"id": 8,"age": 41,"destination_region": "Europe","destination_airport": "HAM","price": 140,"frequent_traveller": 0}
{"id": 9,"age": 78,"destination_region": "Europe","destination_airport": "GVA","price": 170,"frequent_traveller": 0}
{"id": 10,"age": 27,"destination_region": "Europe","destination_airport": "MAD","price": 180,"frequent_traveller": 0}
{"id": 11,"age": 2,"destination_region": "Europe","destination_airport": "DUS","price": 210,"frequent_traveller": 1}
{"id": 12,"age": 78,"destination_region": "Asia","destination_airport": "TYO","price": 260,"frequent_traveller": 1}
{"id": 13,"age": 28,"destination_region": "America","destination_airport": "ORD","price": 300,"frequent_traveller": 0}
{"id": 14,"age": 23,"destination_region": "America","destination_airport": "CLE","price": 160,"frequent_traveller": 0}
{"id": 15,"age": 33,"destination_region": "America","destination_airport": "GIG","price": 245,"frequent_traveller": 0}
bin/kafka-console-producer.sh --broker-list localhost:9092 --topic travel_discount < [json filename]
bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic travel_discount --from-beginning
The complete logic is encapsulated in a project. Projects contain groups of rules, subgroups, rules and actions.
Here is a screenprint of the content of one rulegroup:
At the bottom there are three actions. The first sets the discount percentage to 25% if the rulegroup passes and the next sets it to 0% if the rulegroup fails. The last action calculates the actual discount price based on the regular price and the discount percentage.
The content of the whole project looks like shown below. It contains three rulegroups. These groups contain the rules/logic to calculate the discount percentages and adjust the region.
From the web UI the project can be exported into a single zip file containing all the logic.
Compared to the original records in the Kafka input topic there are fields added:
- the discount_price was set by the actions
- the field destination_region_short was mapped from the destination_region
- the discount_price was calculated from the original price and the discount percentage.
This is what the result looks like in the output Kafka topic. The first record has no discount because the age is above 78 but the destination region is Asis - no discount according to the rules.
Whereas the second record gets the 25% price reduction because the age is greater than 78 and the destination region is Europe.
{
"price": 260,
"discount_price": 260,
"destination_region": "Asia",
"id": 12,
"destination_region_short": "ASI",
"discount_percent": 0,
"frequent_traveller": 1,
"age": 78,
"destination_airport": "TYO"
}
....
{
"price": 170,
"discount_price": 127.5,
"destination_region": "Europe",
"id": 9,
"destination_region_short": "EUR",
"discount_percent": 25,
"frequent_traveller": 0,
"age": 78,
"destination_airport": "GVA"
}
You can also specify a second output topic for logging the detailed results of the execution of the rules (set it in the properties file). The data and the results including the relevant message why the rule failed or passed will be in the topic. Watch out that you will get one output message for each rule and input message. If you have 1000 messages and 10 rules, you will get 10000 messages in the detailed results topic.
Hope you enjoyed it. I will work on more details to also allow processing CSV and Avro format soon. Another idea is to route data to a topic based on the results of the logic (passed or failed).
And finally I will try to integrate the ruleengine also in Kafka Connect, so that you can apply complex business logic while running your Kafka Connect ingestion from a file or database or other sources. That would give us an enormous amount of flexibility to process data: filter, check, route and massage data . All without writing code and the logic is kept centrally in one place for maintenance and review.
Check my other repositories on Github for the ruleengine and the Business Rules Maintenance tool.
Please send your feedback.
Carpe Diem