Bye bye KScope...

Back to Spain, ,back to jet lag, few days more working and then vacation :-)

I would really like to say thanks very much to all ODTUG people (Cameron, Opal, Natalie, Tony ... all of you) You really do a great job!
I really enjoyed meeting people there and attending sessions. Nice to meet personally other FDMEE guys (Doug, Tony, etc.)

Kscope has also been a great event to learn lot of things. It's great to see how people communicate and share knowledge with the audience. And not all is work...I discovered really good dancers @ EMP museum.

I've been thinking about the blog and I'm going to restructure it a little bit in order to make it more useful.

I'm going to include different post series that hopefully will help you not to be scared about FDMEE and especially Jython. Initially I will try to cover:

• Jythoning with FDMEE (my new word...)
• Migrating from FDM Classic
• Tips & Tricks
• FDMEE basics
• Troubleshooting
• Using Source Adapters

Any other suggestions?

Hasta pronto!

KScope14 at Seattle - here we go!!!

KScope started today! After couple of days enjoying Seattle and San Juan Islands... it's kScope Time!

I will be attending and quite happy that I can meet lot of people.

If you want to have a chat, you can find me at Booth 312

PS. It seems that I brought nice weather from Spain :-)

Migrating from Classic FDM to FDMEE - Episode 0

Migrating or not migrating, that's the question :-)

I have been discussing with many customers about FDMEE migration and I would like to share my feedback with you.
First thing we need to bear in mind is that we are migrating to a "new" product based on different platform and providing several new functionality that will help us to get a better performance in our data load process.
Typical questions will be:

• Do we have a migration tool (next > next > next ...)?
• Can I reuse all metadata defined in my Classic FDM application (locations, import formats, etc.)?
• What about my mappings? can I migrate them?
• My application has lots of customization using VBScript, can I migrate them to FDMEE?
• I have built some custom reports in FDM, can I reuse them in FDMEE?
• What about Batch Loader?
• So how can I migrate then?
• and thousands more...

For me, there are two migration steps:

1. Technical migration
2. Functional migration

Let me clarify this. 

During the technical migration we will install FDMEE in our new architecture. We will register our source and target systems, configure credentials for ERP systems in ODI if needed, etc...

Why do I scope the rest of the migration as functional? Because I consider this migration a kind of once-in-a-lifetime opportunity to review your application design and get the best performance. Why would my application need to be reviewed? Here you have some reasons:

• Classic FDM has some limitations that are now covered by FDMEE. For example, when multi-dimensional mappings were needed, most of the people used mapping scripts which don't perform so well. In FDMEE, we have a new mapping type called Multi-Dimensional mapping. This new mapping type will probably replace all your mapping scripts. In addition, you can use look-up dimensions in FDMEE to support your multi-dimensional mappings.
• The more locations you have, the more you will have to maintain. With new data load rules you may simplify your location structure. Two locations for two different file types may be replaced by one location and two data load rules having each one import format.
• Import process is almost always the bottle neck in Classic FDM. Mapping process is performed during the import process and 90% of the customers having performance issues during import follow this rule: Wrong mapping logic design -> poor performance during import (I have seen source files with 1000 rows taking more than 30 minutes...) It's a perfect chance to review your mapping logic in order to get the best performance. Make use of new mapping types!!! multi-dimensional, #SQL, and #FORMAT mask mappings.
• You will have to re-write your scripts (import, mapping, event, and custom). Even if you keep VB for event and custom scripts, you will have to do some changes as VB Script is replaced by VB .NET. On the other hand, if you decide to use event and custom scripts in Jython...you will have to completely re-write them. Import scripts (dump) support Jython only. Integration scripts are not supported anymore as import scripts so they need to be implemented in BefImport event script (Jython or VB .NET). And lastly, VB mapping scripts are not supported anymore so you will have to re-write them as Jython or SQL mapping scripts (or replace by multi-dimensional mappings). Being said that, you will have a great opportunity to review your code which sometimes is not as efficient as you think. Also, if you forgot commenting your code, don't forget to do it now :-)

Some answers...

• Do we have a migration tool (next > next > next ...)? Not in 11.1.2.3.x. It will be probably be included in 11.1.2.4 but we don't know what will be available for migration yet. The tool will be mainly for FDM metadata (locations, import formats, mappings...)
• Can I reuse all metadata defined in my Classic FDM application (locations, import formats, etc.)? Yes you can. Some of them have been slightly re-designed but the core functionality is in FDMEE as well. 
• What about my mappings? can I migrate them? Explicit, In, Between, Like mappings also exist in FDMEE. You will have to re-write your mapping scripts.
• My application has lots of customization using VBScript, can I migrate them to FDMEE? Already answered above:-)
• I have built some custom reports in FDM, can I reuse them in FDMEE? Crystal reports have been replaced by RTF templates created in BI Publisher. You can reuse the SQL queries but will have to rebuild the report in RTF format. I don't think migration utility will cover reports.
• What about Batch Loader? Open batches are now available as a type of batch definition. Same functionality so you will not have many problems migrating them. 
• So how can I migrate then? There is no unique answer for this question. It all depends on your application(s). In some cases it will be better to start from scratch in FDMEE so you can improve your design. You can also build custom SQL process to move objects such as import formats or locations between FDM and FDMEE databases.

Some advice...

• Please, take your time for the migration
• Review your design. Poor design in FDM will be a poor design in FDMEE so don't expect a big performance improvement if you keep it
• Make use of new FDMEE functionality. It may take some time to rebuild your application but you will be happy with results
• Find an experienced resource for application assessment. I would suggest someone not involved in the design of your current application. If I built the application I will hardly find any design issue
• Work from high level to detailed level. That will help you to have a better understanding of your integration flow
• Do you need 10 days? I would not be worry if you need some time to assess/review/migrate. Performing well during these stage will make you save lot of time/money :-)

Which is your feedback? Please comment!