Friday, February 7, 2014

Installing TNPM 1.3.2 with RHEL 5.9 Issues and Workarounds

Hi, I'm currently installing a fresh TNPM 1.3.2 platform and I believe some of the issues I found could be of interest. So, as I find them, I will list them here.


Oracle 32bit client MUST be installed on the 64bit server as well

Despite the documentation states the opposite ( see here ), it is mandatory to have the 32bit installed, otherwise, the topologyEditor and deployer won't be able to connect to the database. You MUST use the 32bit JDBC version when installing the topologyEditor and you MUST configure on the topologyEditor the ORACLE_HOME_ROOT property to point to your 64bit oracle install and ORACLE_HOME property to point to the 32bit client (and not the 64bit installed server) for you database host


Missing package for topologyEditor installation

If you try to install the topologyEditor and get this message:


"Graphical installers are not supported by the VM. The console mode will be used instead..."

It is because some 32bit library is missing. In my case, for RHEL 5.9 the missing 32bit libraries were libXres.i386 and libXtst.i386

Please notice that the same error doesn't occur for RHEL 5.5...


Missing packages for provisoinfo GUI on Datamart

If you see the following error message when opening the provisoinfod:

"java.lang.UnsatisfiedLinkError : awt (An exception was pending after running JNI_OnLoad)"

Please make sure all libX... packages have the 64bit and 32bit installed

# yum list installed | grep libX
libX11.i386                                1.0.3-11.el5_7.1            installed
libX11.x86_64                              1.0.3-11.el5_7.1            installed
libXTrap.i386                              1.0.0-3.1                   installed
libXTrap.x86_64                            1.0.0-3.1                   installed
libXau.i386                                1.0.1-3.1                   installed
libXau.x86_64                              1.0.1-3.1                   installed
libXaw.i386                                1.0.2-8.1                   installed
libXaw.x86_64                              1.0.2-8.1                   installed
libXcursor.i386                            1.1.7-1.2                   installed
libXcursor.x86_64                          1.1.7-1.2                   installed
libXdmcp.i386                              1.0.1-2.1                   installed
libXdmcp.x86_64                            1.0.1-2.1                   installed
libXext.i386                               1.0.1-2.1                   installed
libXext.x86_64                             1.0.1-2.1                   installed
libXfixes.i386                             4.0.1-2.1                   installed
libXfixes.x86_64                           4.0.1-2.1                   installed
libXfont.i386                              1.2.2-1.0.4.el5_7           installed
libXfont.x86_64                            1.2.2-1.0.4.el5_7           installed
libXfontcache.i386                         1.0.2-3.1                   installed
libXfontcache.x86_64                       1.0.2-3.1                   installed
libXft.i386                                2.1.10-1.1                  installed
libXft.x86_64                              2.1.10-1.1                  installed
libXi.i386                                 1.0.1-4.el5_4               installed
libXi.x86_64                               1.0.1-4.el5_4               installed
libXinerama.i386                           1.0.1-2.1                   installed
libXinerama.x86_64                         1.0.1-2.1                   installed
libXmu.i386                                1.0.2-5                     installed
libXmu.x86_64                              1.0.2-5                     installed
libXp.i386                                 1.0.0-8.1.el5               installed
libXp.x86_64                               1.0.0-8.1.el5               installed
libXpm.i386                                3.5.5-3                     installed
libXpm.x86_64                              3.5.5-3                     installed
libXrandr.i386                             1.1.1-3.3                   installed
libXrandr.x86_64                           1.1.1-3.3                   installed
libXrender.i386                            0.9.1-3.1                   installed
libXrender.x86_64                          0.9.1-3.1                   installed
libXres.i386                               1.0.1-3.1                   installed
libXres.x86_64                             1.0.1-3.1                   installed
libXt.i386                                 1.0.2-3.2.el5               installed
libXt.x86_64                               1.0.2-3.2.el5               installed
libXtst.i386                               1.0.1-3.1                   installed
libXtst.x86_64                             1.0.1-3.1                   installed
libXv.i386                                 1.0.1-4.1                   installed
libXv.x86_64                               1.0.1-4.1                   installed
libXxf86dga.i386                           1.0.1-3.1                   installed
libXxf86dga.x86_64                         1.0.1-3.1                   installed
libXxf86misc.i386                          1.0.1-3.1                   installed
libXxf86misc.x86_64                        1.0.1-3.1                   installed
libXxf86vm.i386                            1.0.1-3.1                   installed
libXxf86vm.x86_64                          1.0.1-3.1                   installed

Using oracle instance name different than PV

If you need to use another instance name, please make sure that you are defining EXACTLY the same value everywhere, specially the correct letter case. This is case-sensitive for the database instance creation!!

CheckDB script fails mentioning the os version is not supported

As you can see in the image below, the RHEL 5 updates 1 to 10 are supported by TNPM 1.3.2, but the checkdb script expects version 5.5U (hardcoded on checkdb.ini). Change the value in the checkdb.ini file and it should work.




Manual copy and execution of install packages

When installing TNPM, the deployer assumes you will enter the password for the application user on all remote servers and the root user. This is used to copy and execute the install package remotely. Now, this works fine when you have those passwords, but, in a real company environment, it will be unlikely you will receive them.

It is common to be able to do a "sudo" to the required application user and to transfer files using ssh shared keys among the servers (so no password needed).

Unfortunately, the deployer is not capable of transferring files remotely using ssh shared keys (this was identified as an APAR on Nov 2011...)

So, the solution is to disable the "remote file transfer" and "remote command execution" and do the work manually. The only problem is that there is another bug related to the install package that should be copied over (APAR opened on Jan 2012...)

Below you can find the steps necessary to manually create and execute the packages.

On the deployer disable the "remote file transfer" and the "remote execution". After, execute the installation plan step by step. It will give you an error for each step related to remote hosts. Once you see the error, right-click on the step and check the command output. You will see something mentioning you have to copy over the files under /tmp/ProvisoConsumer/(...)/runtime directory. Follow the instructions below for each step.

Deployer step: OS check

On the deployer host as root user:

  1. # cd /tmp/ProvisoConsumer/Plan/MachinePlan_<remote_host>/00001_CheckSys_OS_step_1_1 /runtime
  2. # mkdir package
  3. # cd package/
  4. # cp ../../bin/* .
  5. # cd ../../
  6. # tar cvf runtime.tar runtime/
On the remote host as application user:

  1. # mkdir -p /tmp/ProvisoConsumer/Plan/MachinePlan_<remote_host>/00001_CheckSys_OS_step_1_1/runtime
  2. # cd /tmp/ProvisoConsumer/Plan/MachinePlan_<remote_host>/00001_CheckSys_OS_step_1_1
  3. # scp <deployer_host>:/tmp/ProvisoConsumer/Plan/MachinePlan_<remote_host>/00001_CheckSys_OS_step_1_1/runtime.tar .
  4. # tar xvf runtime.tar
On the remote host as root user:

  1. # cd /tmp/ProvisoConsumer/Plan/MachinePlan_<remote_host>/00001_CheckSys_OS_step_1_1/runtime
  2. # ksh run.sh
If successfully executed (return code(s) = 0), go back to the deployer and change the step status as success for the "preparation" step and the "execution" step.

Deployer step: Database check

On the deployer host as root user:

  1. # cd /tmp/ProvisoConsumer/Plan/MachinePlan_<remote_host>/00002_CheckSys_DB_step_1_2/runtime
  2. # mkdir package
  3. # cd package/
  4. # cp ../../bin/* .
  5. # cd ../../
  6. # tar cvf runtime.tar runtime/
On the remote host as application user:

  1. # mkdir -p /tmp/ProvisoConsumer/Plan/MachinePlan_<remote_host>/00002_CheckSys_DB_step_1_2/runtime
  2. # cd /tmp/ProvisoConsumer/Plan/MachinePlan_<remote_host>/00002_CheckSys_DB_step_1_2
  3. # scp <deployer_host>:/tmp/ProvisoConsumer/Plan/MachinePlan_<remote_host>/00002_CheckSys_DB_step_1_2/runtime.tar .
  4. # tar xvf runtime.tar
On the remote host as root user:

  1. # cd /tmp/ProvisoConsumer/Plan/MachinePlan_<remote_host>/00002_CheckSys_DB_step_1_2/runtime
  2. # ksh run.sh
If successfully executed (return code(s) = 0), go back to the deployer and change the step status as success for the "preparation" step and the "execution" step.

Deployer step: JRE check

On the deployer host as root user:

  1. # cd /tmp/ProvisoConsumer/Plan/MachinePlan_<remote_host>/00003_JRE_step_2_1/runtime
  2. # mkdir package
  3. # cd package/
  4. # cp <tnpm_image>/v132/proviso/RHEL/jre/RHEL5/ibm\-java\-jre\-6.0\-8.1\-linux\-i386.tgz .
  5. # cd ../../
  6. # tar cvf runtime.tar runtime/
On the remote host as application user:

  1. # mkdir -p /tmp/ProvisoConsumer/Plan/MachinePlan_<remote_host>/00003_JRE_step_2_1/runtime
  2. # cd /tmp/ProvisoConsumer/Plan/MachinePlan_<remote_host>/00003_JRE_step_2_1
  3. # scp <deployer_host>:/tmp/ProvisoConsumer/Plan/MachinePlan_<remote_host>/00003_JRE_step_2_1/runtime.tar .
  4. # tar xvf runtime.tar
On the remote host as application user:

  1. # cd /tmp/ProvisoConsumer/Plan/MachinePlan_<remote_host>/00003_JRE_step_2_1/runtime
  2. # ksh run.sh
If successfully executed (return code(s) = 0), go back to the deployer and change the step status as success for the "preparation" step and the "execution" step.

Deployer step: Remote DataChannel setup

On the deployer host as root user:

  1. # cd /tmp/ProvisoConsumer/Plan/MachinePlan_<remote_host>/00004_DataChannel_step_8_1/runtime
  2. # mkdir package
  3. # cd package/
  4. # cp -R <tnpm_image>/v132/proviso/RHEL/DataChannel/* .
  5. # cd ../../
  6. # tar cvf runtime.tar runtime/
On the remote host as application user:

  1. # mkdir -p /tmp/ProvisoConsumer/Plan/MachinePlan_<remote_host>/00004_DataChannel_step_8_1/runtime
  2. # cd /tmp/ProvisoConsumer/Plan/MachinePlan_<remote_host>/00004_DataChannel_step_8_1
  3. # scp <deployer_host>:/tmp/ProvisoConsumer/Plan/MachinePlan_<remote_host>/00004_DataChannel_step_8_1/runtime.tar .
  4. # tar xvf runtime.tar
On the remote host as application user:

  1. # cd /tmp/ProvisoConsumer/Plan/MachinePlan_<remote_host>/00004_DataChannel_step_8_1/runtime
  2. # ksh run.sh
If successfully executed (return code(s) = 0), go back to the deployer and change the step status as success for the "preparation" step and the "execution" step.

Deployer step: Remote Datamart setup

On the deployer host as root user:

  1. # cd /tmp/ProvisoConsumer/Plan/MachinePlan_<remote_host>/00003_DataMart_step_7_1/runtime
  2. # mkdir package
  3. # cd package/
  4. # cp -R <tnpm_image>/v132/proviso/RHEL/DataMart/RHEL5/* .
  5. # cd ../../
  6. # tar cvf runtime.tar runtime/
On the remote host as application user:

  1. # mkdir -p /tmp/ProvisoConsumer/Plan/MachinePlan_<remote_host>/00003_DataMart_step_7_1/runtime
  2. # cd /tmp/ProvisoConsumer/Plan/MachinePlan_<remote_host>/00003_DataMart_step_7_1
  3. # scp <deployer_host>:/tmp/ProvisoConsumer/Plan/MachinePlan_<remote_host>/00003_DataMart_step_7_1/runtime.tar .
  4. # tar xvf runtime.tar
On the remote host as root user:

  1. # cd /tmp/ProvisoConsumer/Plan/MachinePlan_<remote_host>/00003_DataMart_step_7_1/runtime
  2. # ksh run.sh
If successfully executed (return code(s) = 0), go back to the deployer and change the step status as success for the "preparation" step and the "execution" step.

Deployer step: Remote TIP (TCR) setup

Please install TCR manually using its own installer and using the application username. After installing, mark this step as successful.

DO NOT install TCR using the deployer, once it will install the portal as root and as if that was not unsafe enough for an application portal (usually a public portal), it will execute chmod -R 777 <tip_home> as the last step. (yes, you can yell now...) 

Deployer step: Remote Dataview setup

On the deployer host as root user:

  1. # cd /tmp/ProvisoConsumer/Plan/MachinePlan_<remote_host>/00004_DataView_step_10_1/runtime
  2. # mkdir package
  3. # cd package/
  4. # cp -R <tnpm_image>/v132/proviso/RHEL/DataView/RHEL5/* .
  5. # cd ../../
  6. # tar cvf runtime.tar runtime/
On the remote host as application user:

  1. # mkdir -p /tmp/ProvisoConsumer/Plan/MachinePlan_<remote_host>/00004_DataView_step_10_1/runtime
  2. # cd /tmp/ProvisoConsumer/Plan/MachinePlan_<remote_host>/00004_DataView_step_10_1
  3. # scp <deployer_host>:/tmp/ProvisoConsumer/Plan/MachinePlan_<remote_host>/00004_DataView_step_10_1/runtime.tar .
  4. # tar xvf runtime.tar
On the remote host as application user:

  1. # cd /tmp/ProvisoConsumer/Plan/MachinePlan_<remote_host>/00004_DataView_step_10_1/runtime
  2. # ksh run.sh
If successfully executed (return code(s) = 0), go back to the deployer and change the step status as success for the "preparation" step and the "execution" step.

Deployer step: Remote Dataload setup

On the deployer host as root user:

  1. # cd /tmp/ProvisoConsumer/Plan/MachinePlan_<remote_host>/00003_DataLoad_step_9_1/runtime
  2. # mkdir package
  3. # cd package/
  4. # cp -R <tnpm_image>/v132/proviso/RHEL/DataLoad/* .
  5. # cd ../../
  6. # tar cvf runtime.tar runtime/
On the remote host as application user:

  1. # mkdir -p /tmp/ProvisoConsumer/Plan/MachinePlan_<remote_host>/00003_DataLoad_step_9_1/runtime
  2. # cd /tmp/ProvisoConsumer/Plan/MachinePlan_<remote_host>/00003_DataLoad_step_9_1
  3. # scp <deployer_host>:/tmp/ProvisoConsumer/Plan/MachinePlan_<remote_host>/00003_DataLoad_step_9_1/runtime.tar .
  4. # tar xvf runtime.tar
On the remote host as root user:

  1. # cd /tmp/ProvisoConsumer/Plan/MachinePlan_<remote_host>/00003_DataLoad_step_9_1/runtime
  2. # ksh run.sh
If successfully executed (return code(s) = 0), go back to the deployer and change the step status as success for the "preparation" step and the "execution" step.

21 comments:

  1. Hi

    I have Redhat 6.4 as my operating system and I want to install TNPM 1.3.3. In installtion guide I found that redhat 5.9 version is required. I tried to check software product compatibility report for TNPM 1.3.3, but I could not get 1.3.3 as an option to check. Can I continue with redhat 6.4 or I will have to change OS.

    ReplyDelete
  2. Hi

    I would not advise to install it on RHEL 6.4. You may face many issues related to missing 32 bit packages (not listed on the documentation), as well as having to change parts of the installation script.
    Now, I personally never installed TNPM 1.3.3, so I cannot say it would work without problems. But, considering they have just released it I would never install it in production before having at least fixpack 2 or 3 released. This would avoid many problems and headaches.

    ReplyDelete
  3. Hi Danilo,

    I am starting TNPM 1.3.2 installation. I have Oracle 11.2.0.3(64 bit). Can I go ahead with this version of Oracle server or I need Oracle 11.2.0.2(64 bit)?

    ReplyDelete
    Replies
    1. Hi, I would not advise so. I know 11.2.0.2 is already a little old, but you risk having problems with IBM support if using something different. Please remember you need 64bit for the server and 32bit for the client, and the 32 bit client MUST be also installed on the database server, otherwise the topologyEditor and deployer will not work.

      Delete
  4. Hi danilo, I started 64bit oracle 11.2.0.2 server installation. Installation completed successfully but I am not able to connect to database pv..when I start listener it gives
    Instance "PV" status unknown. .
    When I created new database 'test'..I could connect to that database. .so can I drop this pv database and create again..does tnpm creates any tables in this database. .

    ReplyDelete
    Replies
    1. Hi, just to be sure, did you also create the topology and installed using the deployer? The installation of oracle using IBM scripts will only install the binaries and the listener, but will not create the database instance (PV). This is done later using the deployer. The only test that will succeed after oracle installation and listener configuration is "tnsping PV 10". Also it is normal to see the "status UNKNOWN" message when starting the listener before creating the instance itself using the deployer.

      Delete
    2. Hi Danilo,
      Thanks for your reply. :) I had not run deployer and was trying to connect to PV database.

      When I run script for oracle server installation it shows oracle release as 11.2.0 whereas in guide it is mentioned as it should show as 11.2.0.2. I am providing path to oracle 11.2.0.2 download directory.
      So will it give any error when I run deployer.
      Note: When Installation gets completed, it shows version as 11.2.0.2.

      Delete
    3. Yep...the official guide does not match the real life, as usual :) Just continue the installation and it should be fine.

      Delete
  5. Hi Danilo,
    Thanks for your reply. :)
    I have successfully installed oracle client and server using TNPM Scripts as mentioned in TNPM Installation guide.
    I am trying to run deployer using topology editor, but it fails at database check.
    It is giving following errors:
    a) ./check_db Server New PV ORA11
    ERROR! Found an Oracle instance running with PID: 1
    b) ./check_db Client New PV ORA11
    -->ERROR! 64 bit /opt/oracle/product/11.2.0.2/bin/sqlplus not listed in check_db.ini.
    I tried stopping instance by using dbshut.But Stil problem persists.Please suggest.
    Note: Client and Server is installed in separate ORACLE HOME.i.e.
    For Server ORACLE_HOME=/opt/oracle/product/11.2.0.2
    For Client ORACLE_HOME=/opt/oracle/product/11.2.0-client32

    ReplyDelete
    Replies
    1. Hi, sorry for the late answer, but I was travelling abroad. I believe both errors are related to the same issue: You have to install the oracle client 32 bit as well on the database server (I believe you did) and then set the correct values on the topology editor (please see on the top of the page). The check_db script will verify if the 32bit client version is installed before continuing.

      Delete
  6. Hi Danilo,

    I am installing TNPM 1.3.2 as a minimal deployment on redhat 5.5(64bit).
    Installer is giving following error during dataload install step on the deployer:

    Failed retrieving configuration from database. User: 'PV_COLL' DB Environment: 'PV' with error: Database client libraries are inaccessible. Either the Object Engine has not been linked with the required libraries, or the DLLs for those libraries are not in your search path. For more information, consult the release notes for this Connect product

    I referred to this link:
    https://www.ibm.com/developerworks/community/forums/html/topic?id=77777777-0000-0000-0000-000014629080
    But in this link, as user said issue was of OS enviornment(CentOs as os) which is not my case.But I have Oracle Server and Client installed in a different ORACLE_HOME.Is this creating a problem?
    For Server ORACLE_HOME=/opt/oracle/product/11.2.0.2
    For Client ORACLE_HOME=/opt/oracle/product/11.2.0-client32

    ReplyDelete
    Replies
    1. Hi, I've never installed it as a minimal deployment, so I cannot say much here. But some things you can try are:
      - Check .profile for the oracle user on the user home dir (maybe it is not pointing to the correct oracle binaries - 64bit).
      - Check the oracle home permission. You should be able to execute sqlplus using the tnpm user and not the oracle user.

      Delete
    2. Hi, Were you able to fix this issue? Even we are trying minimal installation and facing same issue. Installation failing at the last step and gives same error when tried starting dataload and datachannel components.

      Delete
    3. Hi, I don't have any other suggestion. It is quite possible this is a bug on the minimal installation procedure. You should contact IBM to check on that.

      Delete
    4. For future reference I've just hit this issue with TNPM 1.4 and it was caused by SELINUX file ACLs - i.e. I forgot to reboot after disabling SELINUX. This meant that the Oracle client libraries could not be accessed by the DataLoad / by pvuser.

      Delete
  7. Dear Danilo,
    I am trying to install TNPM 1.3.1 on redhat 5.5 with oracle 10. While running Installation in topology editor I got the error in Install proviso database.The log file is-

    READ FIRST: When encountering problems with this step, you should run
    READ FIRST: the diagnostic utility available at:
    READ FIRST:
    READ FIRST: $ORACLE_BASE/admin/skeleton/bin/checkextc
    READ FIRST:
    READ FIRST: This utility will assist you in possible problems with
    READ FIRST: your setup. Most likely, there is an issue with your
    READ FIRST: listener or tns configuration for ExtProc. Please consult
    READ FIRST: the PROVISO installation guide for further details on the
    READ FIRST: setup of ExtProc.
    READ FIRST:



    ORACLE : Creating library LibExtCall ...


    Library created.


    ORACLE : Creating function ExternalCall ...


    Function created.


    ORACLE : Calling function ExternalCall ...

    Declare
    *
    ERROR at line 1:
    ORA-28575: unable to open RPC connection to external procedure agent
    ORA-06512: at "SYSTEM.EXTERNALCALL", line 1
    ORA-06512: at line 5

    Can anyone please help me on this?

    ReplyDelete
    Replies
    1. Hi Kaushal,

      Please double check your tnsnames.ora and listener.ora files for the EXTPROC entry. Personally I've never faced this error, but it should be related to that. Bellow the links for the version 1.3.2. Please adapt it to your current system and try again:
      => http://www-01.ibm.com/support/knowledgecenter/SSBNJ7_1.3.2/com.ibm.tnpm.doc/oracle/ttnpm_ora_configoraclelistener.html
      => http://www-01.ibm.com/support/knowledgecenter/SSBNJ7_1.3.2/com.ibm.tnpm.doc/oracle/ttnpm_ora_configoraclenetclient.html

      Delete
    2. Hi Danilo,

      Does i need to upgrade oracle 10.2.0.2 to 10.2.0.4 using oracle 10.2.0.4, for the installation of TNPM 1.3.1? I have double checked the tnsnames.ora and listener.ora but the problem still persist.Can be the problem related to upgrading oracle 10.2.0.2 ?

      Thanks

      Delete
    3. Hi Kaushal,

      I believe you should keep the oracle 10.2.0.2. Anyway, maybe you should think about using TNPM 1.3.2 instead with oracle 11, once TNPM 1.3.1 is quite old and should go out of life soon.

      Delete
  8. Hi Danilo

    I'm dooing an installation of TNPM 1.3.2 on Unix Version : Red Hat Enterprise Linux Server release 5.5 (Tikanga) on a distributed environnement .

    we have performed on installation through the deployer and now that the HAM configuration have been added, the deployer can not find any difference in the topology and refuse to install the HAM regardless the deployer option used .

    Even a local installation with the following option does not work
    -DPrimary=true \
    -DTarget=DL \


    Do you have any clue how to debug this or force such ham configuration ?

    Stephane

    ReplyDelete
    Replies
    1. Hi Stephane, sorry for the late response (I guess I missed this one :). I believe by now you should have found the answer already. If that is the case, please share it with us.

      Kind regards

      Danilo

      Delete