Rrdtool aggregate target has "no valid data" after move to github version

xefil · May 25, 2017, 3:23pm

Btw, I’m noticing those errors. Does it have something to do with weathermap or the system?

had no valid data, according to WeatherMapDataSource_rrd

Thanks,

Simon

howie · May 25, 2017, 6:24pm

During a poll? Literally like that, with no node or link names? It’s a weathermap error, but you’d expect to see the target that it’s talking about.

xefil · May 25, 2017, 11:02pm

Hello @howie,
The complete log is like this:

2017-05-26 01:00:30 - WEATHERMAP WARNING: [Map 2] report_vmware_storage_monthly.conf: Not enough output from RRDTool. [WMRRD09]
2017-05-26 01:00:30 - WEATHERMAP WARNING: [Map 2] report_vmware_storage_monthly.conf: ReadData: NODE MYNODE, target: gauge:/data/cacti/rra/vcloud-vc01_<snip>23018.rrd:ramtot:- had no valid data, according to WeatherMapDataSource_rrd [WMWARN70]

What could be the cause?
The map is so not correctly shown.

Simon

howie · May 25, 2017, 11:04pm

That seems to be saying that rrdtool isn’t producing the expected output. Run with DEBUG logging, and you’ll get some lines just before that one that show you exactly what is returned from rrdtool.

xefil · May 26, 2017, 12:50pm

Hello @howie,
The previous version was 0.97a if that’s an important info. And here the complete log of a node in DEBUG:

 2017-05-26 14:43:59 - WEATHERMAP DEBUG: [performDataCollection@WeatherMapDataItem.class.php:211] [Map 2] report_vmware_storage_monthly.conf: ReadData for NODE VM-CUSTOMERS:
 2017-05-26 14:43:59 - WEATHERMAP DEBUG: [collectDataFromTargets@WeatherMapDataItem.class.php:256] [Map 2] report_vmware_storage_monthly.conf: ReadData: New Target: gauge:/data/cacti/rra/vcloud_vmtot_23021.rrd:vmon:- on config line 91 of /data/cacti/plugins/weathermap/configs/report_vmware_storage_monthly.conf
 2017-05-26 14:43:59 - WEATHERMAP DEBUG: [readData@WMTarget.class.php:114] [Map 2] report_vmware_storage_monthly.conf: ReadData for NODE VM-CUSTOMERS (WeatherMapDataSource_rrd 1)
 2017-05-26 14:43:59 - WEATHERMAP DEBUG: [ReadData@WeatherMapDataSource_rrd.php:404] [Map 2] report_vmware_storage_monthly.conf: Special DS names seen (vmon and -).
 2017-05-26 14:43:59 - WEATHERMAP DEBUG: [ReadData@WeatherMapDataSource_rrd.php:421] [Map 2] report_vmware_storage_monthly.conf: SCALING result by 1
 2017-05-26 14:43:59 - WEATHERMAP DEBUG: [ReadData@WeatherMapDataSource_rrd.php:473] [Map 2] report_vmware_storage_monthly.conf: RRD ReadData: Target DS names are vmon and -
 2017-05-26 14:43:59 - WEATHERMAP DEBUG: [wmrrd_read_from_real_rrdtool_aggregate@WeatherMapDataSource_rrd.php:152] [Map 2] report_vmware_storage_monthly.conf: RRD ReadData: VDEF style, for NODE VM-CUSTOMERS
 2017-05-26 14:43:59 - WEATHERMAP DEBUG: [wmrrd_read_from_real_rrdtool_aggregate@WeatherMapDataSource_rrd.php:194] [Map 2] report_vmware_storage_monthly.conf: RRD ReadData: Running: /usr/bin/rrdtool graph /dev/null -f '' --start now-14d --end start+1209600 DEF:in=/data/cacti/rra/vcloud_vmtot_23021.rrd:vmon:AVERAGE VDEF:agg_in=in,MAXIMUM "PRINT:agg_in:'IN %lf'"
 2017-05-26 14:43:59 - WEATHERMAP WARNING: [Map 2] report_vmware_storage_monthly.conf: Not enough output from RRDTool. [WMRRD09]
 2017-05-26 14:43:59 - WEATHERMAP DEBUG: [wmrrd_read_from_real_rrdtool_aggregate@WeatherMapDataSource_rrd.php:244] [Map 2] report_vmware_storage_monthly.conf: RRD ReadDataFromRealRRDAggregate: Returning (null,null,0)
 2017-05-26 14:43:59 - WEATHERMAP DEBUG: [ReturnData@plugin-base-classes.php:88] [Map 2] report_vmware_storage_monthly.conf: RRD ReadData: Returning ({null}, {null}, 0)
 2017-05-26 14:43:59 - WEATHERMAP WARNING: [Map 2] report_vmware_storage_monthly.conf: ReadData: NODE VM-CUSTOMERS, target: gauge:/data/cacti/rra/vcloud_vmtot_23021.rrd:vmon:- had no valid data, according to WeatherMapDataSource_rrd [WMWARN70]
 2017-05-26 14:43:59 - WEATHERMAP DEBUG: [collectDataFromTargets@WeatherMapDataItem.class.php:269] [Map 2] report_vmware_storage_monthly.conf: Invalid data?
 2017-05-26 14:43:59 - WEATHERMAP DEBUG: [performDataCollection@WeatherMapDataItem.class.php:237] [Map 2] report_vmware_storage_monthly.conf: ReadData complete for NODE VM-CUSTOMERS: 0 0
 2017-05-26 14:43:59 - WEATHERMAP DEBUG: [performDataCollection@WeatherMapDataItem.class.php:241] [Map 2] report_vmware_storage_monthly.conf: ReadData: Setting 0,0 for NODE VM-CUSTOMERS

His definition:

 NODE VM-CUSTOMERS
    LABEL {node:this:bandwidth_in:%0.0f}
    LABELFONT 40
    INFOURL /graph.php?rra_id=all&local_graph_id=21472
    OVERLIBGRAPH /graph_image.php?rra_id=0&graph_nolegend=true&graph_height=100&graph_width=300&local_graph_id=21472
    TARGET gauge:/data/cacti/rra/vcloud_vmtot_23021.rrd:vmon:-
    POSITION 311 205

And rrdtoo command:

 # rrdtool tune /data/cacti/rra/vcloud_vmtot_23021.rrd.rrd
 DS[vmon] typ: GAUGE     hbt: 600        min: 0.0000     max: 10000.0000
 DS[vmtot] typ: GAUGE    hbt: 600        min: 0.0000     max: 10000.0000

Thanks!

Simon

xefil · May 29, 2017, 7:48am

Good morning @howie,

Does the error above give you some hints on there the problem could reside?

Executing manually the rrdtool graph command, i get:

 /usr/bin/rrdtool graph /dev/null -f '' --start now-14d --end start+1209600 DEF:in=/data/cacti/rra/vcloud_vmtot_23021.rrd:vmon:AVERAGE VDEF:agg_in=in,MAXIMUM "PRINT:agg_in:'IN %lf'"
 ERROR: bad format for imginfo

If I execute the SAME command on the old server, I got the data result.
Version on the old server:

 RRDtool 1.4.7 @ CentOS release 5.8 (Final)

Version on the new server:

RRDtool 1.4.8 @ CentOS Linux release 7.3.1611 (Core)

Many thanks for your support,

Simon

howie · May 29, 2017, 12:04pm

It appears that rrdtool 1.4.8 had a change that stops the aggregation functions working the same way.

http://git.meleeweb.net/gentoo/portage.git/tree/net-analyzer/rrdtool/files/1.4.8-Allow-empty-imginfo-string-format.patch?id=d457637474aa99202962f2ad423086c8667df548

I’ve just tested with 1.5.5, and it works again, so I guess the answer is to move either forwards or backwards from 1.4.8. One of the changelog items for 1.4.9 is “properly verify validity of user supplied format strings” which might be the fix.

xefil · May 29, 2017, 12:08pm

I’ve found rpm for 1.6.0 but the poller never ends. I’ll look for a different version. Trying to compile it I get errors on perl libraries. Let me search for a better one. I’ll let you know soon.

Thanks,

Simon

xefil · May 29, 2017, 12:23pm

Hello @howie,
I’ve updated to version 1.5.5.
Now it looks better, I mean, the cacti poller doesn’t hang and the weathermap poller get its data, but the map is again wrong. Same “invalid data” error. Here the debug log:

2017-05-29 14:17:58 - WEATHERMAP DEBUG: [performDataCollection@WeatherMapDataItem.class.php:210] [Map 2] report_vmware_storage_monthly.conf: -------------------------------------------------------------
2017-05-29 14:17:58 - WEATHERMAP DEBUG: [performDataCollection@WeatherMapDataItem.class.php:211] [Map 2] report_vmware_storage_monthly.conf: ReadData for NODE VM-CUSTOMERS:
2017-05-29 14:17:58 - WEATHERMAP DEBUG: [collectDataFromTargets@WeatherMapDataItem.class.php:256] [Map 2] report_vmware_storage_monthly.conf: ReadData: New Target: gauge:/data/cacti/rra/vcloud_vmtot_23021.rrd:vmon:- on config line 121 of /data/cacti/plugins/weathermap/configs/report_vmware_storage_monthly.conf
2017-05-29 14:17:58 - WEATHERMAP DEBUG: [readData@WMTarget.class.php:114] [Map 2] report_vmware_storage_monthly.conf: ReadData for NODE VM-CUSTOMERS (WeatherMapDataSource_rrd 1)
2017-05-29 14:17:58 - WEATHERMAP DEBUG: [ReadData@WeatherMapDataSource_rrd.php:404] [Map 2] report_vmware_storage_monthly.conf: Special DS names seen (vmon and -).
2017-05-29 14:17:58 - WEATHERMAP DEBUG: [ReadData@WeatherMapDataSource_rrd.php:421] [Map 2] report_vmware_storage_monthly.conf: SCALING result by 1
2017-05-29 14:17:58 - WEATHERMAP DEBUG: [ReadData@WeatherMapDataSource_rrd.php:473] [Map 2] report_vmware_storage_monthly.conf: RRD ReadData: Target DS names are vmon and -
2017-05-29 14:17:58 - WEATHERMAP DEBUG: [wmrrd_read_from_real_rrdtool_aggregate@WeatherMapDataSource_rrd.php:152] [Map 2] report_vmware_storage_monthly.conf: RRD ReadData: VDEF style, for NODE VM-CUSTOMERS
2017-05-29 14:17:58 - WEATHERMAP DEBUG: [wmrrd_read_from_real_rrdtool_aggregate@WeatherMapDataSource_rrd.php:194] [Map 2] report_vmware_storage_monthly.conf: RRD ReadData: Running: /usr/bin/rrdtool graph /dev/null -f '' --start now-14d --end start+1209600 DEF:in=/data/cacti/rra/vcloud_vmtot_23021.rrd:vmon:AVERAGE VDEF:agg_in=in,MAXIMUM "PRINT:agg_in:'IN %lf'"
2017-05-29 14:17:58 - WEATHERMAP DEBUG: [wmrrd_read_from_real_rrdtool_aggregate@WeatherMapDataSource_rrd.php:210] [Map 2] report_vmware_storage_monthly.conf: > 'IN 365.004444'
2017-05-29 14:17:58 - WEATHERMAP WARNING: [Map 2] report_vmware_storage_monthly.conf: Not enough output from RRDTool. [WMRRD09]
2017-05-29 14:17:58 - WEATHERMAP DEBUG: [wmrrd_read_from_real_rrdtool_aggregate@WeatherMapDataSource_rrd.php:244] [Map 2] report_vmware_storage_monthly.conf: RRD ReadDataFromRealRRDAggregate: Returning (null,null,0)
2017-05-29 14:17:58 - WEATHERMAP DEBUG: [ReturnData@plugin-base-classes.php:88] [Map 2] report_vmware_storage_monthly.conf: RRD ReadData: Returning ({null}, {null}, 0)
2017-05-29 14:17:58 - WEATHERMAP WARNING: [Map 2] report_vmware_storage_monthly.conf: ReadData: NODE VM-CUSTOMERS, target: gauge:/data/cacti/rra/vcloud_vmtot_23021.rrd:vmon:- had no valid data, according to WeatherMapDataSource_rrd [WMWARN70]
2017-05-29 14:17:58 - WEATHERMAP DEBUG: [collectDataFromTargets@WeatherMapDataItem.class.php:269] [Map 2] report_vmware_storage_monthly.conf: Invalid data?
2017-05-29 14:17:58 - WEATHERMAP DEBUG: [collectDataFromTargets@WeatherMapDataItem.class.php:256] [Map 2] report_vmware_storage_monthly.conf: ReadData: New Target: gauge:/data/cacti/rra/vcenter01-tn_mclink-net_it_vmtot_23033.rrd:vmon:- on config line 121 of /data/cacti/plugins/weathermap/configs/report_vmware_storage_monthly.conf
2017-05-29 14:17:58 - WEATHERMAP DEBUG: [readData@WMTarget.class.php:114] [Map 2] report_vmware_storage_monthly.conf: ReadData for NODE VM-CUSTOMERS (WeatherMapDataSource_rrd 1)
2017-05-29 14:17:58 - WEATHERMAP DEBUG: [ReadData@WeatherMapDataSource_rrd.php:404] [Map 2] report_vmware_storage_monthly.conf: Special DS names seen (vmon and -).
2017-05-29 14:17:58 - WEATHERMAP DEBUG: [ReadData@WeatherMapDataSource_rrd.php:421] [Map 2] report_vmware_storage_monthly.conf: SCALING result by 1
2017-05-29 14:17:58 - WEATHERMAP DEBUG: [ReadData@WeatherMapDataSource_rrd.php:473] [Map 2] report_vmware_storage_monthly.conf: RRD ReadData: Target DS names are vmon and -
2017-05-29 14:17:58 - WEATHERMAP DEBUG: [wmrrd_read_from_real_rrdtool_aggregate@WeatherMapDataSource_rrd.php:152] [Map 2] report_vmware_storage_monthly.conf: RRD ReadData: VDEF style, for NODE VM-CUSTOMERS
2017-05-29 14:17:58 - WEATHERMAP DEBUG: [wmrrd_read_from_real_rrdtool_aggregate@WeatherMapDataSource_rrd.php:194] [Map 2] report_vmware_storage_monthly.conf: RRD ReadData: Running: /usr/bin/rrdtool graph /dev/null -f '' --start now-14d --end start+1209600 DEF:in=/data/cacti/rra/vcenter01-tn_mclink-net_it_vmtot_23033.rrd:vmon:AVERAGE VDEF:agg_in=in,MAXIMUM "PRINT:agg_in:'IN %lf'"
2017-05-29 14:17:58 - WEATHERMAP DEBUG: [wmrrd_read_from_real_rrdtool_aggregate@WeatherMapDataSource_rrd.php:210] [Map 2] report_vmware_storage_monthly.conf: > 'IN 100.000000'
2017-05-29 14:17:58 - WEATHERMAP WARNING: [Map 2] report_vmware_storage_monthly.conf: Not enough output from RRDTool. [WMRRD09]
2017-05-29 14:17:58 - WEATHERMAP DEBUG: [wmrrd_read_from_real_rrdtool_aggregate@WeatherMapDataSource_rrd.php:244] [Map 2] report_vmware_storage_monthly.conf: RRD ReadDataFromRealRRDAggregate: Returning (null,null,0)
2017-05-29 14:17:58 - WEATHERMAP DEBUG: [ReturnData@plugin-base-classes.php:88] [Map 2] report_vmware_storage_monthly.conf: RRD ReadData: Returning ({null}, {null}, 0)
2017-05-29 14:17:58 - WEATHERMAP WARNING: [Map 2] report_vmware_storage_monthly.conf: ReadData: NODE VM-CUSTOMERS, target: gauge:/data/cacti/rra/vcenter01-tn_mclink-net_it_vmtot_23033.rrd:vmon:- had no valid data, according to WeatherMapDataSource_rrd [WMWARN70]
2017-05-29 14:17:58 - WEATHERMAP DEBUG: [collectDataFromTargets@WeatherMapDataItem.class.php:269] [Map 2] report_vmware_storage_monthly.conf: Invalid data?
2017-05-29 14:17:58 - WEATHERMAP DEBUG: [performDataCollection@WeatherMapDataItem.class.php:237] [Map 2] report_vmware_storage_monthly.conf: ReadData complete for NODE VM-CUSTOMERS: 0 0
2017-05-29 14:17:58 - WEATHERMAP DEBUG: [performDataCollection@WeatherMapDataItem.class.php:241] [Map 2] report_vmware_storage_monthly.conf: ReadData: Setting 0,0 for NODE VM-CUSTOMERS
2017-05-29 14:17:58 - WEATHERMAP DEBUG: [add_note@base-classes.php:46] [Map 2] report_vmware_storage_monthly.conf: Adding note inpercent='0' to VM-CUSTOMERS
2017-05-29 14:17:58 - WEATHERMAP DEBUG: [add_note@base-classes.php:46] [Map 2] report_vmware_storage_monthly.conf: Adding note max_bandwidth_in='1' to VM-CUSTOMERS
2017-05-29 14:17:58 - WEATHERMAP DEBUG: [add_note@base-classes.php:46] [Map 2] report_vmware_storage_monthly.conf: Adding note max_bandwidth_in_cfg='1' to VM-CUSTOMERS
2017-05-29 14:17:58 - WEATHERMAP DEBUG: [add_note@base-classes.php:46] [Map 2] report_vmware_storage_monthly.conf: Adding note outpercent='0' to VM-CUSTOMERS
2017-05-29 14:17:58 - WEATHERMAP DEBUG: [add_note@base-classes.php:46] [Map 2] report_vmware_storage_monthly.conf: Adding note max_bandwidth_out='1' to VM-CUSTOMERS
2017-05-29 14:17:58 - WEATHERMAP DEBUG: [add_note@base-classes.php:46] [Map 2] report_vmware_storage_monthly.conf: Adding note max_bandwidth_out_cfg='1' to VM-CUSTOMERS
2017-05-29 14:17:58 - WEATHERMAP DEBUG: [colourFromValue@WeatherMapScale.class.php:140] [Map 2] report_vmware_storage_monthly.conf: Finding a colour for value 0 in scale none
2017-05-29 14:17:58 - WEATHERMAP DEBUG: [add_note@base-classes.php:46] [Map 2] report_vmware_storage_monthly.conf: Adding note inscalekey='' to VM-CUSTOMERS
2017-05-29 14:17:58 - WEATHERMAP DEBUG: [add_note@base-classes.php:46] [Map 2] report_vmware_storage_monthly.conf: Adding note inscaletag='' to VM-CUSTOMERS
2017-05-29 14:17:58 - WEATHERMAP DEBUG: [add_note@base-classes.php:46] [Map 2] report_vmware_storage_monthly.conf: Adding note inscalecolor='#ffffff' to VM-CUSTOMERS
2017-05-29 14:17:58 - WEATHERMAP DEBUG: [colourFromValue@WeatherMapScale.class.php:140] [Map 2] report_vmware_storage_monthly.conf: Finding a colour for value 0 in scale none
2017-05-29 14:17:58 - WEATHERMAP DEBUG: [add_note@base-classes.php:46] [Map 2] report_vmware_storage_monthly.conf: Adding note outscalekey='' to VM-CUSTOMERS
2017-05-29 14:17:58 - WEATHERMAP DEBUG: [add_note@base-classes.php:46] [Map 2] report_vmware_storage_monthly.conf: Adding note outscaletag='' to VM-CUSTOMERS
2017-05-29 14:17:58 - WEATHERMAP DEBUG: [add_note@base-classes.php:46] [Map 2] report_vmware_storage_monthly.conf: Adding note outscalecolor='#ffffff' to VM-CUSTOMERS
2017-05-29 14:17:58 - WEATHERMAP DEBUG: [performDataCollection@WeatherMapDataItem.class.php:210] [Map 2] report_vmware_storage_monthly.conf: -------------------------------------------------------------

Ideas?

Thanks,

Simon

howie · May 29, 2017, 12:34pm

This is failing differently now

It looks like it is expecting more lines than it should. I’ve just pushed a new version that is happy with 1 line of output, but the change is simply to change the ‘1’ to a ‘0’ on line 218 of WeatherMapDataSource_rrd.php

I think this is the combination of two problems - between 1.4 and 1.5, rrdtool removed a blank line, and also I think most people use both in and out, so this has probably been around for a while.

xefil · May 29, 2017, 1:11pm

Hello @howie,

Another step forward, but not enough
I’ve made a sync of the git-version. The error is gone, but I’ve data of one node replicated to other nodes.
The map is a table with some data exported. The map2, which we are analyzing, has the same value on the same line, even if the target are different. Looking on the log in debug mode, I can notice as only the first node executes an “rrdtool graph” command on his rrd file. The other not.

Look on the differences.
node1 (VMCUSTRM):

2017-05-29 15:01:02 - WEATHERMAP DEBUG: [performDataCollection@WeatherMapDataItem.class.php:210] [Map 2] report_vmware_storage_monthly.conf: -------------------------------------------------------------
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [performDataCollection@WeatherMapDataItem.class.php:211] [Map 2] report_vmware_storage_monthly.conf: ReadData for NODE VMCUSTRM:
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [collectDataFromTargets@WeatherMapDataItem.class.php:256] [Map 2] report_vmware_storage_monthly.conf: ReadData: New Target: gauge:/data/cacti/rra/vcloud_vmtot_23021.rrd:vmon:- on config line 91 of /data/cacti/plugins/weathermap/configs/report_vmware_storage_monthly.conf
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [readData@WMTarget.class.php:114] [Map 2] report_vmware_storage_monthly.conf: ReadData for NODE VMCUSTRM (WeatherMapDataSource_rrd 1)
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [ReadData@WeatherMapDataSource_rrd.php:400] [Map 2] report_vmware_storage_monthly.conf: Special DS names seen (vmon and -).
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [ReadData@WeatherMapDataSource_rrd.php:417] [Map 2] report_vmware_storage_monthly.conf: SCALING result by 1
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [ReadData@WeatherMapDataSource_rrd.php:469] [Map 2] report_vmware_storage_monthly.conf: RRD ReadData: Target DS names are vmon and -
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [wmrrd_read_from_real_rrdtool_aggregate@WeatherMapDataSource_rrd.php:154] [Map 2] report_vmware_storage_monthly.conf: RRD ReadData: VDEF style, for NODE VMCUSTRM
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [wmrrd_read_from_real_rrdtool_aggregate@WeatherMapDataSource_rrd.php:196] [Map 2] report_vmware_storage_monthly.conf: RRD ReadData: Running: /usr/bin/rrdtool graph /dev/null -f '' --start now-14d --end start+1209600 DEF:in=/data/cacti/rra/vcloud_vmtot_23021.rrd:vmon:AVERAGE VDEF:agg_in=in,MAXIMUM "PRINT:agg_in:'IN %lf'"
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [wmrrd_read_from_real_rrdtool_aggregate@WeatherMapDataSource_rrd.php:211] [Map 2] report_vmware_storage_monthly.conf: > 'IN 365.004444'
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [wmrrd_read_from_real_rrdtool_aggregate@WeatherMapDataSource_rrd.php:221] [Map 2] report_vmware_storage_monthly.conf: MATCHED: IN 365.004444
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [wmrrd_read_from_real_rrdtool_aggregate@WeatherMapDataSource_rrd.php:245] [Map 2] report_vmware_storage_monthly.conf: RRD ReadDataFromRealRRDAggregate: Returning (365.004444,0,0)
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [ReturnData@plugin-base-classes.php:88] [Map 2] report_vmware_storage_monthly.conf: RRD ReadData: Returning (365.004444, 0, 0)
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [readData@WMTarget.class.php:138] [Map 2] report_vmware_storage_monthly.conf: Collected data 365.004444,0.000000
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [collectDataFromTargets@WeatherMapDataItem.class.php:264] [Map 2] report_vmware_storage_monthly.conf: Adding 365.004444 to total for channel 0
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [collectDataFromTargets@WeatherMapDataItem.class.php:264] [Map 2] report_vmware_storage_monthly.conf: Adding 0.000000 to total for channel 1
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [collectDataFromTargets@WeatherMapDataItem.class.php:267] [Map 2] report_vmware_storage_monthly.conf: Running totals: 365.004444 0
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [performDataCollection@WeatherMapDataItem.class.php:237] [Map 2] report_vmware_storage_monthly.conf: ReadData complete for NODE VMCUSTRM: 365.004444 0
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [performDataCollection@WeatherMapDataItem.class.php:241] [Map 2] report_vmware_storage_monthly.conf: ReadData: Setting 365.004444,0 for NODE VMCUSTRM
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [add_note@base-classes.php:44] [Map 2] report_vmware_storage_monthly.conf: Adding note inpercent='36500.4444' to VMCUSTRM
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [add_note@base-classes.php:44] [Map 2] report_vmware_storage_monthly.conf: Adding note max_bandwidth_in='1' to VMCUSTRM
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [add_note@base-classes.php:44] [Map 2] report_vmware_storage_monthly.conf: Adding note max_bandwidth_in_cfg='1' to VMCUSTRM
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [add_note@base-classes.php:44] [Map 2] report_vmware_storage_monthly.conf: Adding note outpercent='0' to VMCUSTRM
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [add_note@base-classes.php:44] [Map 2] report_vmware_storage_monthly.conf: Adding note max_bandwidth_out='1' to VMCUSTRM
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [add_note@base-classes.php:44] [Map 2] report_vmware_storage_monthly.conf: Adding note max_bandwidth_out_cfg='1' to VMCUSTRM
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [colourFromValue@WeatherMapScale.class.php:140] [Map 2] report_vmware_storage_monthly.conf: Finding a colour for value 36500.4444 in scale none
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [add_note@base-classes.php:44] [Map 2] report_vmware_storage_monthly.conf: Adding note inscalekey='' to VMCUSTRM
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [add_note@base-classes.php:44] [Map 2] report_vmware_storage_monthly.conf: Adding note inscaletag='' to VMCUSTRM
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [add_note@base-classes.php:44] [Map 2] report_vmware_storage_monthly.conf: Adding note inscalecolor='#ffffff' to VMCUSTRM
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [colourFromValue@WeatherMapScale.class.php:140] [Map 2] report_vmware_storage_monthly.conf: Finding a colour for value 0 in scale none
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [add_note@base-classes.php:44] [Map 2] report_vmware_storage_monthly.conf: Adding note outscalekey='' to VMCUSTRM
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [add_note@base-classes.php:44] [Map 2] report_vmware_storage_monthly.conf: Adding note outscaletag='' to VMCUSTRM
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [add_note@base-classes.php:44] [Map 2] report_vmware_storage_monthly.conf: Adding note outscalecolor='#ffffff' to VMCUSTRM
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [performDataCollection@WeatherMapDataItem.class.php:210] [Map 2] report_vmware_storage_monthly.conf: -------------------------------------------------------------

Node2 (VMCUSTTN):

2017-05-29 15:01:02 - WEATHERMAP DEBUG: [performDataCollection@WeatherMapDataItem.class.php:210] [Map 2] report_vmware_storage_monthly.conf: -------------------------------------------------------------
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [performDataCollection@WeatherMapDataItem.class.php:211] [Map 2] report_vmware_storage_monthly.conf: ReadData for NODE VMCUSTTN:
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [collectDataFromTargets@WeatherMapDataItem.class.php:256] [Map 2] report_vmware_storage_monthly.conf: ReadData: New Target: gauge:/data/cacti/rra/vcloud_vmtot_23020.rrd:vmon:- on config line 99 of /data/cacti/plugins/weathermap/configs/report_vmware_storage_monthly.conf
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [readData@WMTarget.class.php:114] [Map 2] report_vmware_storage_monthly.conf: ReadData for NODE VMCUSTTN (WeatherMapDataSource_rrd 1)
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [ReadData@WeatherMapDataSource_rrd.php:400] [Map 2] report_vmware_storage_monthly.conf: Special DS names seen (vmon and -).
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [ReadData@WeatherMapDataSource_rrd.php:417] [Map 2] report_vmware_storage_monthly.conf: SCALING result by 1
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [ReturnData@plugin-base-classes.php:88] [Map 2] report_vmware_storage_monthly.conf: RRD ReadData: Returning (365.004444, 0, 0)
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [readData@WMTarget.class.php:138] [Map 2] report_vmware_storage_monthly.conf: Collected data 365.004444,0.000000
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [collectDataFromTargets@WeatherMapDataItem.class.php:264] [Map 2] report_vmware_storage_monthly.conf: Adding 365.004444 to total for channel 0
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [collectDataFromTargets@WeatherMapDataItem.class.php:264] [Map 2] report_vmware_storage_monthly.conf: Adding 0.000000 to total for channel 1
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [collectDataFromTargets@WeatherMapDataItem.class.php:267] [Map 2] report_vmware_storage_monthly.conf: Running totals: 365.004444 0
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [performDataCollection@WeatherMapDataItem.class.php:237] [Map 2] report_vmware_storage_monthly.conf: ReadData complete for NODE VMCUSTTN: 365.004444 0
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [performDataCollection@WeatherMapDataItem.class.php:241] [Map 2] report_vmware_storage_monthly.conf: ReadData: Setting 365.004444,0 for NODE VMCUSTTN
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [add_note@base-classes.php:44] [Map 2] report_vmware_storage_monthly.conf: Adding note inpercent='36500.4444' to VMCUSTTN
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [add_note@base-classes.php:44] [Map 2] report_vmware_storage_monthly.conf: Adding note max_bandwidth_in='1' to VMCUSTTN
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [add_note@base-classes.php:44] [Map 2] report_vmware_storage_monthly.conf: Adding note max_bandwidth_in_cfg='1' to VMCUSTTN
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [add_note@base-classes.php:44] [Map 2] report_vmware_storage_monthly.conf: Adding note outpercent='0' to VMCUSTTN
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [add_note@base-classes.php:44] [Map 2] report_vmware_storage_monthly.conf: Adding note max_bandwidth_out='1' to VMCUSTTN
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [add_note@base-classes.php:44] [Map 2] report_vmware_storage_monthly.conf: Adding note max_bandwidth_out_cfg='1' to VMCUSTTN
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [colourFromValue@WeatherMapScale.class.php:140] [Map 2] report_vmware_storage_monthly.conf: Finding a colour for value 36500.4444 in scale none
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [add_note@base-classes.php:44] [Map 2] report_vmware_storage_monthly.conf: Adding note inscalekey='' to VMCUSTTN
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [add_note@base-classes.php:44] [Map 2] report_vmware_storage_monthly.conf: Adding note inscaletag='' to VMCUSTTN
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [add_note@base-classes.php:44] [Map 2] report_vmware_storage_monthly.conf: Adding note inscalecolor='#ffffff' to VMCUSTTN
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [colourFromValue@WeatherMapScale.class.php:140] [Map 2] report_vmware_storage_monthly.conf: Finding a colour for value 0 in scale none
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [add_note@base-classes.php:44] [Map 2] report_vmware_storage_monthly.conf: Adding note outscalekey='' to VMCUSTTN
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [add_note@base-classes.php:44] [Map 2] report_vmware_storage_monthly.conf: Adding note outscaletag='' to VMCUSTTN
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [add_note@base-classes.php:44] [Map 2] report_vmware_storage_monthly.conf: Adding note outscalecolor='#ffffff' to VMCUSTTN
2017-05-29 15:01:02 - WEATHERMAP DEBUG: [performDataCollection@WeatherMapDataItem.class.php:210] [Map 2] report_vmware_storage_monthly.conf: -------------------------------------------------------------

The nodes are those:

NODE VMCUSTRM
    LABEL {node:this:bandwidth_in:%0.0f}
    LABELFONT 40
    INFOURL /graph.php?rra_id=all&local_graph_id=21472
    OVERLIBGRAPH /graph_image.php?rra_id=0&graph_nolegend=true&graph_height=100&graph_width=300&local_graph_id=21472
    TARGET gauge:/data/cacti/rra/vcloud_vmtot_23021.rrd:vmon:-
    POSITION 311 205

NODE VMCUSTTN
    LABEL {node:this:bandwidth_in:%0.0f}
    LABELFONT 40
    INFOURL /graph.php?rra_id=all&local_graph_id=21471
    OVERLIBGRAPH /graph_image.php?rra_id=0&graph_nolegend=true&graph_height=100&graph_width=300&local_graph_id=21471
    TARGET gauge:/data/cacti/rra/vcloud_vmtot_23020.rrd:vmon:-
    POSITION 520 205

I’ve slightly changed node names and rrd paths to avoid sharing hostnames.
Can you see as node2 is not executing the rrdtool command and it gest the values from node?

Then there are other issues but seems the same type.

Thanks!

Simon

howie · May 29, 2017, 2:30pm

You’re right! OK - another push. When re-organising the data source plugins a couple of weeks ago, I introduced this problem - if a target for a particular plugin works, but the next one fails, you can (I didn’t check all the plugins, but applied the same fix) get the previous target’s data.

That does still leave one question - it’s not at all clear from the logs why the second target failed. It doesn’t seem to even try and collect data.

xefil · May 29, 2017, 3:02pm

Hello!

Updating…

]# git pull origin database-refactor
remote: Counting objects: 19, done.
remote: Compressing objects: 100% (1/1), done.
remote: Total 19 (delta 18), reused 19 (delta 18), pack-reused 0
Unpacking objects: 100% (19/19), done.
From https://github.com/howardjones/network-weathermap
 * branch            database-refactor -> FETCH_HEAD
Updating 34fa63f..78afd19
Fast-forward
 lib/datasources/WeatherMapDataSource_cacti.php      |  16 ++------
 lib/datasources/WeatherMapDataSource_cactihost.php  |   3 ++
 lib/datasources/WeatherMapDataSource_cactithold.php |   2 +
 lib/datasources/WeatherMapDataSource_dbsample.php   |   2 +
 lib/datasources/WeatherMapDataSource_dsstats.php    |   2 +
 lib/datasources/WeatherMapDataSource_fping.php      |   3 ++
 lib/datasources/WeatherMapDataSource_mrtg.php       |   3 ++
 lib/datasources/WeatherMapDataSource_rrd.php        | 359 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++    ++-------------------------------------------------------------------------------------
 lib/datasources/WeatherMapDataSource_snmp.php       |   3 ++
 lib/datasources/WeatherMapDataSource_snmpv2c.php    |   3 ++
 lib/datasources/WeatherMapDataSource_snmpv3.php     |   3 ++
 lib/datasources/WeatherMapDataSource_static.php     |   3 ++
 lib/datasources/WeatherMapDataSource_tabfile.php    |   3 ++
 lib/datasources/WeatherMapDataSource_time.php       |   3 ++
 lib/datasources/WeatherMapDataSource_wmdata.php     |   3 ++
 15 files changed, 221 insertions(+), 190 deletions(-)

aaaand now all other maps are generated correctly! YEAH!
Even on-mouse-over-graph works again (I have not asked about this issue before to not ask too much :P)

I’ve only a last issue, with the original map. The “Map2” is a clone of “Map1” but has this directive:

SET rrd_start now-14d
SET rrd_period 1209600
SET rrd_aggregate_function MAXIMUMSET rrd_start now-14d

Is this not jet implemented on this version?

Thanks @howie!!

Simon

howie · May 29, 2017, 3:16pm

Is this a typo or what you actually have?

I think it should be:

SET rrd_aggregate_function MAXIMUM
SET rrd_start now-14d

which should work fine.

Also, if this is the only difference between the two maps, you can avoid keeping two copies in sync by adding this one setting in the Cacti map management interface instead, and having two entries pointing to the same map.

xefil · May 29, 2017, 3:22pm

Typo sorry:

SET rrd_start now-14d
SET rrd_period 1209600
SET rrd_aggregate_function MAXIMUM

The result is empty.

2017-05-29 17:19:08 - WEATHERMAP DEBUG: [wmrrd_read_from_real_rrdtool_aggregate@WeatherMapDataSource_rrd.php:196] [Map 2] report_vmware_storage_monthly.conf: RRD ReadData: Running: /usr/bin/rrdtool graph /dev/null -f '' --start now-14d --end start+1209600 DEF:in=/data/cacti/rra/vcenter01_vmtot_23030.rrd:vmon:AVERAGE VDEF:agg_in=in,MAXIMUM "PRINT:agg_in:'IN %lf'"
2017-05-29 17:19:08 - WEATHERMAP DEBUG: [wmrrd_read_from_real_rrdtool_aggregate@WeatherMapDataSource_rrd.php:214] [Map 2] report_vmware_storage_monthly.conf: > 'IN 108.500278'
2017-05-29 17:19:08 - WEATHERMAP WARNING: [Map 2] report_vmware_storage_monthly.conf: RRD Aggregate ReadData: Not enough output from RRDTool (0 lines). [WMRRD09]
2017-05-29 17:19:08 - WEATHERMAP DEBUG: [ReturnData@plugin-base-classes.php:88] [Map 2] report_vmware_storage_monthly.conf: RRD ReadData: Returning ({null}, {null}, 0)
2017-05-29 17:19:08 - WEATHERMAP WARNING: [Map 2] report_vmware_storage_monthly.conf: ReadData: NODE VM-MCL-TRENTO, target: gauge:/data/cacti/rra/vcenter01_vmtot_23030.rrd:vmon:- had no valid data, according to WeatherMapDataSource_rrd [WMWARN70]
2017-05-29 17:19:08 - WEATHERMAP DEBUG: [collectDataFromTargets@WeatherMapDataItem.class.php:269] [Map 2] report_vmware_storage_monthly.conf: Invalid data?

Simon

howie · May 29, 2017, 8:37pm

(I could swear I replied to this earlier…)

I managed to get the logic backwards with the earlier change regarding line counts in rrdtool output. Please try the latest.

xefil · May 30, 2017, 10:39am

Good morning @howie,

Well, seems corrected now, many many thanks for your GREAT support!
Sorry for having bothered you with all my weathermap issues!

I wish you a great day

Simon

howie · May 30, 2017, 10:50am

Not a problem! If it’s solving a bug in the software, I especially want to hear about it!