Upgrading E-Business Suite to 12.2.14 - FND_VAULT again

Mid of September Oracle releases the latest E-Business Suite 12.2 RUP 12.2.14.

Tons of new features

As described in the official release notes, the new release contains tons of new functionalites in almost all application modules as well as a lot of new Enterprise Command Centers. Finally, some technical issues with Analytics Publisher 12c fonts got fixed.

So far, we just found a single regression bug with 12.2.14.

FND_VAULT issues again

Similar to when upgrading to 12.2.13 last year with a different customer, we unfortunately ran into some issues with FND_VAULT again. This is especially problematic since the package behind this is not only wrapped but also highly obfuscated. That makes any "private" bug hunting difficult, and you're typically completely in the hands of Oracle development for any analysis as well as a fix (even Oracle Support does nothing besides forwarding messages). And that may take time.

The error we ran into while applying the 12.2.14 patch was while applying AFVLTPCY.sql:

ORA-06502: PL/SQL: numeric or value error
ORA-06512: at "APPS.FND_VAULT", line 288
ORA-06512: at "APPS.FND_VAULT", line 260
ORA-06512: at "APPS.FND_VAULT", line 2203
ORA-06512: at "APPS.FND_VAULT", line 2325
ORA-06512: at "APPS.FND_VAULT", line 2338
ORA-06512: at line 220        

A first attempt to resolve this was to apply 36842768 (merge=yes) together with the 12.2.14 patch (36026788) to get the latest version of FND_VAULT related packages:

/* $Header: AFSCVL8B.pls 120.8.12020000.24 2024/08/05 23:58:18 emiranda ship $        

This didn't help unfortunately though.

With the help of some special debug patch we found out, that there seems to be a regression with fnd_vault entries longer than 30 characters. By removing those as follows (from the run edition):

exec fnd_vault.del('FND_SMTP','[email protected]');
commit;        

With that, we were able to successfully restart the failed worker with adctrl.

You can find problematic entries by temporarily removing the policies preventing selects on fnd_vault_data and querying in that table for too long records (lenghtb>30).

Of course, you have to be sure that this data is not so important and at least not necessary for first tests. In my case, only FND_SMTP used as part of FND Output delivery was affected; furthermore, talking to the users showing up confirmed that they hadn't been using this functionality on purpose.

Summary

While Oracle has not fixed the issue ( Bug 37083953 - 12.2.14 RUP FAILURE: ORA-06502 ERROR IN FND_VAULT DURING PATCH APPLICATION) as of today (beginning of October 2024), I've at least been able to identify the cause and find kind of an (ugly and not ready for production) workaround. With that, you can at least complete the patching cycle and check if everything else is working.

Other than that, all testing with 12.2.14 looks very smooth; so far, we haven’t run into any functional issues during testing.