Using Java Classes from Jython Scripts in FDMEE (formatting dates)

Hi all,

I recently got an inquiry regarding the use of the datetime module from a Jython import script in FDMEE.
In my last post I showed how import scripts and custom/event script use a different version of Jython. The datetime module was introduced in Jython 2.3 so our colleague was getting an issue when trying to import it from the import script (which actually runs 2.2.1 as we saw)

Besides any workaround to update sys.path with new path so we can import and use the module (I'm still testing it...), I would like to encourage you to use Java classes from your FDMEE scripts when needed.
Jython is the implementation of Python for Java so one of its main pros is that you can use Java classes from Jython scripts.

How can this help us? This is a very good thing for developers as you can reuse Java libraries to perform complex operations. Sometimes alternative solutions using Jython may not be available due to limitations in version used by FDMEE as it actually happens with our today's example: datetime

Let's say that I would like to import the current date in ISO 8601 format in my description field (YYYY-MM-DD)
If I have a look to what I can use in Jython, we find the datetime module. Our script would be simply:

But, as we already showed, our import script cannot find the module and therefore an error is raised when importing it:

We can test same function in Eclipse using Jython 2.5.1 (same as custom/event scripts) and we get the expected result:

Using Java Classes to get formatted date
Not working so let's try another approach...using Java Classes!
Which classes? just bit of googling...

• java.text.SimpleDateFormat
• java.util.Date
• java.lang.Exception

How do we use them from our import script in Jython?

I like using aliases for classes imported as it simplifies my code. As Jython modules, we first need to import the Java classes, and then adjust the code to Jython syntax (it's not Java syntax but Java used from Jython).

For example, we don't use new operator to create object instance of our class like we do in Java or use types when declaring variables. To create a new object of SimpleDateFormat class using pattern "yyyy-MM-dd" we do:
df = java.text.SimpleDateFormat(pattern)
while in Java we would do:
SimpleDateFormat df = new SimpleDateFormat(pattern)

The import script to return current date formatted as "YYYY-MM-DD" could be implemented as follows:

We just need to assign our import script to Description field (Desc2 in my case):

And run the import! 

As you can see, using Java classes from our scripts (not only import but event and custom) can take our FDMEE application to the next level. If you have issues with using Jython modules available in other Jython versions rather than instance used by FDMEE, this approach can be a good option. It can even perform better.
Will it always the best approach? not really, you should take into consideration that FDMEE currently runs on Java Platform 1.6.0_37. I recently had a complex requirement for importing data from database with Chinese Collation which is not supported in Java 1.6 but in Java 1.7. But as I say, everything has solution but death :-)

Enjoy!

Import scripts do not use same Jython version as Event/Custom scripts (PSU510)

Hi there,

You may have not noticed the following issue (good) but I think it's good to know it in case you face it and don't want to waste a long time troubleshooting :-)

Import and Event/Custom scripts are currently using different versions of Jython.

Before we start with details I'm sure you have noticed that FDMEE currently uses Jython 2.5.1. If not, just set log level to 5, run any FDMEE workflow, and have a look to the FDMEE process log:

So how did I come across this issue? I was trying to use a nice Python module called csv which provides functionality to easily manage csv files. It can be very useful to work with text files having quotes, delimiters, amounts having field delimiter as thousands/decimal separator, etc. I had some issues with the amount's thousands separator as you can see in the following example:

Basically split function ignores that fields are quoted as comma delimiter for numbers  is also the split delimiter.

I'm not going to talk about the csv module today but I encourage you to have a look and play with it. This module was introduced in Python 2.3 as you can see in the following documentation:

This is a good starting point.

Import Script using the csv module

When you use a module you have to import it. In my dummy import script, I imported the csv module as follows:

Then executed my DLR and got this ugly message saying that the module didn't exist! I had tested before from my IDE using Jython 2.5.1 so it was quite weird.

In the image below you can see how execution from Notepad++ succeeded:

Custom/Event Script using the csv module

Was the error due to FDMEE not being able to access the module? I wasn't sure so I created a dummy custom script to confirm it:

When I executed the script I could see my data line was successfully parsed and I could see the amount as expected:

Ok, so it works from custom script (I also tested from event script) but it doesn't from import scripts. Troubleshooting time!

Looking into the issue
I tried many different debug code to show Jython details in order to see if there were any issues with visibility of modules. Finally, one line of my debug code threw some lights...
If we add the following lines to both import and custom/event script we will get the Jython version used by each script:
import sys
fdmAPI.logDebug("Jython Version: %s" % str(sys.version_info))

Now it's just a matter of having a look to the FDMEE process log:

Jython Version for Custom/Event script

Jython Version for Import script

So here we have it! Import scripts use Jython 2.2.1 while Event/Custom scripts use Jython 2.5.1.
That means we cannot use the csv module from Import scripts as it was introduced in version 2.3. This was the reason our import script raised the exception we found.

To me it's no the fact they use different versions but not sharing same Jython space. Why? You cannot easily shared variables between Event and Import scripts as we could do in FDM Classic with PVarTemp variables. There are some approaches to do it but they are not straight forward as sharing variables between Event scripts for instance.

BTW as we have been talking about the csv module, I have been using it in different custom scripts to massively load FDM Classic artifacts into FDMEE:

I don't know if this issue will be solved in next patch PSU520 or 11.1.2.4 but it will be so no need to worry about it. It was just a good occasion to show some troubleshooting and Python modules.

Regards!

Quick Update
I got some question in post http://akafdmee.blogspot.com/2013/10/writing-and-testing-pythonjython.html
regarding the import of datetime module. Here you have another example of a module not available in 2.5 (https://docs.python.org/2/library/datetime.html)

Tips & Tricks: Using external PY modules in our FDMEE Scripts

Hi all,

some time ago someone (I will call him Agent Brody from now on) asked in the community forum about having multiple BefImport event scripts... Is that possible?

In FDM Classic we could use API function SCRIPTENG.fCompileScript for that purpose. In that way we could have one master script calling functions/subs declared in other "slave" scripts as needed.

Good approach for implementing a custom library function that you could port from one customer to another. Definitely, that solution was helping us to improve our scripting model and maintenance... a lot.

So in our process of forgetting FDM Classic, we may be wondering about same functionality in Jython...

There are different ways but I don't want to go into much detail today. If we think from programming perspective, I would be thinking in having different Jython scripts (py files) so I can call functions declared in script A from script B...that's all.

How can I do this in Jython? Luckily we have the import statement which you can read about in Jython's site.

We will start with a simple custom library called BefImport_Lib.py which has one function declared. This function will show a message to the user. As I will be using the API function fdmAPI.showCustomMessage, I need to pass fdmAPI object as parameter to that function.

As a simple test, I just want to show a message when running the BefImport event script.

What steps do I need to perform?

1. Import sys module
2. Append folder with custom library to sys.path (so Jython can find the custom library)
3. Import our custom library BefImport_Lib.py (so we can use functions declared in it)
4. Call function customShowMsg(fdmAPI) defined in the custom library

 

You have more details in link above about how Jython works internally when importing modules. In a simply way, the objects within the sys.path list tell Jython where to search for modules.

So let's run our import step in FDMEE...

Let's have a look to the folder where custom library resides:

What is that BefImport_Lib$py.class?
In simple words, Jython compiles the custom py module when the module is imported and the function is called. As result you have this pretty $py.class file :-)

After I change code in my custom library...the old code is still being used!!!
First thing you notice when working with your custom library is that you need to append new code as needed or maybe you are my hero and you get all working with no errors in the first shot!
So I want to change the message (I know, I could have passed a parameter but then I have to spend more time in this blog entry and it's summer time...sun, beach, beer, family, not necessary in that order)

If I run the import again...Houston, Houston! we have a problem! the old message is shown instead of the new one. Then you investigate and you realize that either removing the $py.class file, or compiling the custom library using py_compile module does not help you. It seems that code is cached (memory) after module is imported :-(

Don't think I didn't try to restart FDMEE Service. I did and that works but I didn't want that solution.

Sometimes you need a divine light, and I found it with my colleague Giacomo :-) he provided some suggestions and we found the one working for our case! if we think as we should (I mean with some logic), we would say...what about re-importing the module? Wait, I'm already importing it so how can I re-import the module to get the new code in place? and the correct word is... R_L_AD! (if you take more than 5 seconds then you can read this)

So let's include reload(module) statement after our import module:

And run now the import...

Oleeeeeeeee, we got it! new code is used. That's what we wanted! The cached code is replaced by our new code and we get our new message.

I think this approach can improve our scripting solution and make it clearer and better maintainable. Not only Jython but you can import also custom Java libraries which may give you more flexibility when interacting with Hyperion applications. Don't forget that when you are in Production you don't really need to reload the custom library.

I would like to say thank you to Giacomo. As he says "the import machinery is very implementation-specific"
Thanks to him and all his Python expertise things are easier for me :-)
if you want to learn about Python/Jython and all related, don't forget to visit his http://blog.pythonaro.com/!

And thanks to Brody!

Note: here you have the site I linked for reload explanation in case somebody removes it

Important updates:

• Adding to sys.path: before adding our folder to list sys.path we need to check if element exists in that list. Otherwise folder path will be added every time the event script is executed

• It could be that reload(module) does not work as expected. Sometimes it does not work effectively. In fact, it has been removed from Python 3.x and moved to the imp module. Feedback from Python experts is that Python does not provide an effective support for reloading or unloading of previously imported modules as modules references makes it impractical to reload a module because references could exist in many places of our scripts. There is another solution consisting in removing module from memory but initially loaded modules (Python Cookbook). You can execute this code before importing the custom library module.

Tips & Tricks: Showing all DLRs in Data Load Rule page

Hi all,

I had some customers complaining about having to switch POV in order to execute different DLRs (Data Load Rules) for different Categories.
By default, when you navigate to DLR page you will see all DRLs for current POV Category:

One of the nicest features in FDMEE is the Show option which is available in different pages. In the DLR page we can use this option to show DLRs for all categories regardless the POV Category. In the following example we can see 3 DLR (2 for Actual Category and 1 for Budget Category): 

If you have different categories (Actual, Budget... or whatever) this option may be useful for you :-)

Enjoy!