Added Global ‘null’ Variable to the BKJEP Parser
I got a visit from a developer who is using a shared Java library of mine - BKit, and two components of it, specifically: the BKTable and the expression parser - BKJEP, itself based on the JEP parser while it was Open Source and free. They have since closed the source on JEP and upgraded it's capabilities, but for what we need it is more than adequate, and I really don't want to get into a licensing situation with a group that started Open Source and then closed it down. They sound like they're on the brink of closing shop.
Anyway, the developer wanted to return a null value for the computed column. Originally, the idea of the computed column was to give the ability to the user of the BKTable a way to create a simple computed column without having to do it themselves. They could say "Give me a column called 'Area' that is 'width * height'" - where the columns called 'width' and 'height' were already defined in the table. When either 'width' or 'height' of any row in the table changed, the resulting 'Area' column was also updated. Nice, but not Rocket Science.
It got more complicated when they wanted to have Excel-like functions. Things like dateFormat() to turn a Date object into a String for comparison... isNull() to test if the value is null... and then if_then() to give it a simple conditional. And it's this last one that's opened the particular can of worms I'm dealing with right now.
When data is set in the expression, it's impossible to give it a true null value. The core JEP parser simply doesn't allow it. So we encoded the null to be a special String (__null__ if it matters, and it doesn't) so that it could be 'set' and 'get' from the expression. We also created the isNull() function so that you could test for it. So far so good. But what if the output of the if_then() is to be a null? Ah... there we have an issue.
There was no way to create a properly encoded null value within a JEP expression. This developer was the first to ask for it. There's a lot of good that could be done with this variable defined in all expressions... but there's a lot of danger as well.
Say the user snuck an expression past us that generated a true null value. Then used that value in the test: x == null... well... the value of 'x' would not have been encoded to the proper string constant and so it'd fail. So we have the problem that there are cases where something that should work - won't work. The easiest way to think about this is to just be careful and use the variable 'null' only when you have to. And test... test... test.
But with that, I think this adds more than it harms, so I added it in.
UPDATE: wow... that was a lot harder than I had originally thought. The first cut I had seemed to work, but it really didn't. I had assumed that within JEP the constants and the variables were kept apart as the first are immutable and the second aren't. But in fact, they are the same storage class. So... when I scanned through the variables in the BKTable's method, I still caught them and they didn't appear as column headers and so the expression was tossed out.
I had to change the way the BKTable's method worked. When I got a variable from the expression I had to check to see if it had a defined value. If it did, then it was a constant as only constants had defined values at this point in the processing of the expression. This skipping was essential to get the expressions not only passed in, but also properly evaluated. I ran all the tests on the table and they all came up OK. Whew! That was a lot more complex than I had originally thought.