SevOne and TNPM. A technical comparison.

Some people have asked me about SevOne and how does it compare to TNPM. Some even ask if they should migrate from TNPM to SevOne. My answer is...it depends.

So, to help you take your decision, I prepared the table below. Please keep in mind that it is not my intention to say which one is better (I have my personal opinion though), but only to show what are the differences (and common) points between them and let you take your own decision.

Item	IBM TNPM	SevOne
Architecture	5 specialized core components: Database DataMart DataView DataChannel DataLoad	1 core component: SevOne appliance
Installation	Very complex. Can take a long time (days to weeks)	Very simple. Appliance or VM based. It is a matter of hours to install and start using
Upgrade	Very complex. Can take a long time (days to weeks)	Simple. Press the update button and it is done (complex architectures may demand some special care)
Scalability	Via hardware upgrade on each core component	Via cluster. Add another appliance side by side in cluster mode
SNMP collection	Using collection formulas. Can be easily customized	Using certified collection formulas. Usually needs SevOne intervention
BULK collection	Using PVLINE (proprietary) format	Using xStats component
Component Discovery	Using discovery formulas. Can be easily customized and allow custom properties	Not flexible. SevOne always discovers everything it can using SNMP walk. Allow custom properties
Component enrichment	Using inventory hooks	Using API
Licensing	Based on “class of devices” being monitored. Very complex	Per discovered and enabled object
Technical Portal	WebSphere based. Allow some flexibility and customization	Cannot be customized or changed
Services Portal	Supported	Not supported. Another solution needs to be used
Multi-tenancy	Supported	Supported
Reports	Cannot be created by the user	Can be created and shared by the user
Backup	Database backup	Not existent. To avoid data loss, another appliance needs to be paired in high availability
Cross collection	Supported, but complex	Supported
Grouping components (service like measurements)	Supported, but complex. Can use rules to group components	Supported but each component needs to be maintained manually or via the API
Historical data	Limited by the database storage size. Purge scripts can be used.	Usually limited to 1 year. This can be changed, but it is highly dependent on the local storage available on each appliance
Authentication	LDAP or local	LDAP, RADIUS, TACACS
Authorization	Based on roles	Based on roles
Data volume	Report generation and data processing are subject to speed degradation as the data volume increases	Report generation and data processing are NOT subject to speed degradation as the data volume increases
Data aggregation	Pre-calculated and aggregated before stored in the database. Stored raw data and aggregated data side by side	Only raw data is stored. Aggregated data is calculated during report rendering
Data export	Can be done using DataAccess component	Not trivial. API allows export of reports in CSV format, but not built for huge data export

Error truncating partition

If for any reason you had an issue with your TNPM datachannel LDR that took more than 3 days to solve, you may find the following error on a LDR walkback file once it starts processing the backlog:

END OF WALKBACK20165: Error truncating partition
ORA-06512: at "PV_ADMIN.PVM_ERROR", line 137
ORA-06512: at "PV_ADMIN.PVM_DATALOAD", line 3558
ORA-06512: at line 1

This is caused due to the fact that the LDR is trying to do changes on a db partition that is already at the READONLY state. You can confirm it by searching the following on the oracle trace log:

$ORACLE_BASE/diag/rdbms/pv/pv/trace/proviso_PV_LDR_01.log

The Lowest Level Error Code is:
ORA-00372: file 487 cannot be modified at this time
ORA-01110: data file 487: '/(...)/PV_C01_1DGA_000_2014082500_001.dbf'

Unfortunately, the LDR will not proceed until the issue is solved and will keep generating walkbacks.

To solve the problem:

1) Connect to the oracle database as PV_ADMIN

2) Execute the query (replace the correct tablespace_name value as showed on the oracle trace log) :

select tablespace_name, status from dba_tablespaces where tablespace_name like '%C01_1DGA_000_2014082500%';

3) You should get two columns, one with the tablespace name and the other with "READ ONLY"

4) Change the tablespace to READ/WRITE executing the following (use the tablespace name returned by the previous sql query):

alter tablespace tablespace_name read write;

like:

alter tablespace C01_1DGA_000_2014082500 read write;

5) Bounce the LDR and it should work fine

Piped merge error - what is wrong?

Last week a friend of mine came to me to ask about a strange error he was getting on TNPM. Basically, he had many gaps on report data for all devices, and it was apparently intermittent.

The error message on the log was the following:

V1:3353 2014.10.22-02.08.43 UTC LDR.1-21884:9897        SQLLDR  2 SQL Loader started

V1:3354 2014.10.22-02.08.43 UTC LDR.1-21884:13196       SQLLDR  3 Starting Piped Merge

V1:3355 2014.10.22-02.09.10 UTC LDR.1-21884:13196       MERGE_ERROR     GYMDC10118F Piped Merge Error: No such device or address:Transfer error

V1:3356 2014.10.22-02.09.10 UTC LDR.1-21884:13196       SQLLDRKILL      GYMDC10102W Killed sqlldr pid=a UnixProcess (Inactive: exitStatus nil, Error: Success) , result=a UnixProcess (Inactive: exitStatus nil, Error: Success)

V1:3357 2014.10.22-02.09.10 UTC LDR.1-21884:9897        ORASQLLDR       GYMDC10104F  ErrorCode=nil CommandLine=$ORA_HOME/11.2.0-client32/bin/sqlldr userid=PV_LDR_01/xxxx@pv log=...datachannel/LDR.1/state/2014.10.22-00/MERGED~000.1DGA.BOF.log control=...datachannel/LDR.1/loader.gagg.ctl logErrors=

After some investigation, it was evident that for some reason, the unix pipe created during the data merge was getting corrupted.

It is important to know that the LDR component has two options for merging and loading the data files. On the topology editor, if the option "USE_PIPE" is false, it will generate an intermediate file with the merged data and then use this file to upload via oracle sqlldr. If "USE_PIPE" is true, it will create a unix pipe and the oracle sqlldr will use it to load the files. Some people say that using the pipe is faster, because you don't have to create the intermediate file, but this can cause issues as well, as we will see.

The TNPM system I mentioned was using the pipe method for quite a long time before the issue occurred. So it, should be something in the system itself that had changed. And indeed, it was.

When using the pipe method, the pipe pointer is created under /tmp/LDR.X wher X is the LDR channel number. This works fine if the /tmp is mounted locally, but, if for any reason, the IT team decides to mount it using a remote data store... well, you will have problems. This was exactly what happened. The IT team decide to mount the /tmp using a remote data store for the VMware cluster. Once the datachannel was running on the cluster, the pipe load was affected.

So, we deactivated the pipe (USE_PIPE=false) on the topology editor, deployed the topology and the problem was solved.

I could not find a way to change where to create the pipe pointer, so it must be hard-coded somewhere. If you know how to do it, let me know :)

link($DC_HOME/bin/visual,CMGR_visual) failed with error 18

If you ever installed the old version of TNPM (Proviso), you know that it was not possible to split the datachannel binaries from the data using two different locations. This was not very smart, considering that it is a common practice among many companies.

Since version 4.4.1 (I believe), this option exists and you can configure different locations for your datachannel binary files and the data files.

But, there is a catch if you use different partitions for the split (what is also the common practice...your data partition is usually remote mounted from the company data cluster).

The limitation is on the visual binary. You cannot execute it from a remote partition. If you try so, you will receive the following error on the screen:

link($DC_HOME/bin/visual,CMGR_visual) failed with error 18

This happens for all the tools that use the visual binary to bootstrap themselves (cmgr, amgr, frmi, etc...).

Fortunately, the solution is very simple:

1) Go to the $DC_HOME/bin folder
2) Open the run script used to start the tool. For instance "cmgr" for the CMGR component.
3) Go to the end of the script and add the line in red right above the last line:

cd $DC_BIN_HOME

$DC_BIN_HOME/pvexec CMGR_visual $DC_BIN_HOME/visual -nologo -noherald $DC_BIN_HOME/dc.im -headless -a CMGR "$@"

This will make sure you go back to the partition where your datachannel binaries are installed, before executing the visual command.

You have to do the same for all run scripts.

That's it.

Datachannel component is not listed on "dccmd status all"

Suppose you cannot see BCOL.1.1 when you execute dccmd status all. Below, some things to investigate:

1) Try to run it manually. Run:
dccmd start BCOL.1.1

2) Check the proviso.log. Run:
grep "BCOL\.1\.1" proviso.log and search for any error message

3) If you see a WALKBACK error on the log, search for the walkback file and read the first lines to see the main reason for the walkback 

4) Check if the component configuration exists in the database. Run:
dccmd debug CMGR "self dbCfgPrint" | grep BCOL.1.1

This should show the component configuration in the database. Check for any mistakes and correct it using the topologyEditor

If you don't see any value, the component was not created on the topologyEditor or was not saved correctly in the database.

5) Check if any filesystem is full. Run:
df -kh and make sure no partition has 100% utilization

6) Restart some datachannel manager components (cmgr, amgr, cns) and try again.

If you still cannot find the reason, it is a good moment to open a ticket towards IBM 

Tivoli Integrated Portal (TIP) portlet authorization

If you ever created custom pages in TIP, you may have encountered authorization issues. This is because you have to set the user authorization on different levels. The official documentation guides you on creating roles, groups, users and setting their relationships, but doesn't mention about portlet authorization.

Each page you create can include a portlet. The most common is the Web Widget, necessary to open a web page. By default, only the "administrator" role has access to it. So, if you create a page, add the web widget, and give access to non admin users to the page, when they try to access it, they will have a blank (grey) page and no error message displayed.

When that happens, go to the server log "SystemOut.log" and look for the following error:

"doStartTag() user does not have permissions for view mode"

Usually this is related to the portlet authorization. You will have to configure it as well.

To configure the portlet authorization, go to "Settings -> portlets" and click on the portlet. This will open the portlet configuration page. Click "Next" until you find the Security tab:

You can now add the correct roles to the authorization containers. Select "User" and add the role for you final user. Click "Next" and "Finish".

Now you can login again with the final user, open the page and the portlet will work fine.

TNPM 1.3.2 support for IE9 and Firefox ESR10

This link explains how to add support for Internet Explorer 9 and Firefox ESR10, but it is quite old.

I've just installed the last fixpacks for TCR and TIP and the steps I followed can be seen below:

Download the necessary packages

From IBM fixcentral download the following (please match the correct bit set for your installation):

• Tivoli Netcool Performance Manager 1.3.2.0-TIV-TNPM-IF0042
• Tivoli Integrated Portal 2.2.0-TIV-TIP-Linux64-FP0011
• Tivoli Integrated Portal 2.2.0-TIV-TIP-FITSuit-FP00011
• Tivoli Common Report 2.1.1.0-TIV-TCR-Linux64-FP2

Install in the following sequence (mandatory)

1) Tivoli Netcool Performance Manager 1.3.2.0-TIV-TNPM-IF0042

      Follow the instructions here

2) Tivoli Integrated Portal 2.2.0-TIV-TIP-Linux64-FP0011

      Follow the instructions on the 22011-fixpack-guide-PDF.pdf file (downloaded together with the packages from fixcentral)

3) Tivoli Common Report 2.1.1.0-TIV-TCR-Linux64-FP2

      Follow the instructions on the TCR211_FP2_Readme.txt file (downloaded together with the packages from fixcentral)

NOTE1: If for any reason you want to install using the console mode (Install.bin -i console), it still requires a valid DISPLAY set.

NOTE2: If for any reason you want to install using the silent mode (Install.bin -i silent -f <response_file>), you will not see any output at all on the shell, but the fix pack will run. You can follow up the process using the logs on <TCR_home>tipv2Components/TCRComponent/logs