[dns-stats-users] Hedgehog dns data not showing.

John Dickinson jad at sinodun.com
Tue May 19 15:00:54 UTC 2015


What happens if you press the Today button and regenerate the plot?

John

> On 19 May 2015, at 15:04, matt singh <matt.singh at centralnic.com> wrote:
> 
> /
> usr/local/var/hedgehog/data/refile_and_grok.stdout
> 
> Tue May 19 14:50:01 BST 2015 SCRIPT COMMENCED
> Tue May 19 14:50:02 BST 2015 START: Server-A/Node-1
> Tue May 19 14:50:02 BST 2015 DONE: Server-A/Node-1
> Tue May 19 14:50:02 BST 2015 SCRIPT COMPLETE
> 
> 
> 
> 
> /usr/local/var/hedgehog/data/<Server>/<Node>/dsc-extractor-xml-db.out
> 
> ----------------------- 2015-05-19 14:50:02 BST ----------------------
> *** Database connected
> *** Processing server: server_a
> *** Processing node:   Node-1
> *** Found:             5 .xml files to process.
> 
> 2015-05-19 14:50:02 BST: Starting:  ./incoming/1432043160.dscdata.xml
> 2015-05-19 14:50:02 BST: Moved:     ./incoming/1432043160.dscdata.xml to ./done/incoming/dscdata/1432043160.dscdata.xml
> 2015-05-19 14:50:02 BST: Completed: ./incoming/1432043160.dscdata.xml                   4 remaining (0 failures).
> 
> 2015-05-19 14:50:02 BST: Starting:  ./incoming/1432043220.dscdata.xml
> 2015-05-19 14:50:02 BST: Moved:     ./incoming/1432043220.dscdata.xml to ./done/incoming/dscdata/1432043220.dscdata.xml
> 2015-05-19 14:50:02 BST: Completed: ./incoming/1432043220.dscdata.xml                   3 remaining (0 failures).
> 
> 2015-05-19 14:50:02 BST: Starting:  ./incoming/1432043280.dscdata.xml
> 2015-05-19 14:50:02 BST: Moved:     ./incoming/1432043280.dscdata.xml to ./done/incoming/dscdata/1432043280.dscdata.xml
> 2015-05-19 14:50:02 BST: Completed: ./incoming/1432043280.dscdata.xml                   2 remaining (0 failures).
> 
> 2015-05-19 14:50:02 BST: Starting:  ./incoming/1432043340.dscdata.xml
> 2015-05-19 14:50:02 BST: Moved:     ./incoming/1432043340.dscdata.xml to ./done/incoming/dscdata/1432043340.dscdata.xml
> 2015-05-19 14:50:02 BST: Completed: ./incoming/1432043340.dscdata.xml                   1 remaining (0 failures).
> 
> 2015-05-19 14:50:02 BST: Starting:  ./incoming/1432043400.dscdata.xml
> 2015-05-19 14:50:02 BST: Moved:     ./incoming/1432043400.dscdata.xml to ./done/incoming/dscdata/1432043400.dscdata.xml
> 2015-05-19 14:50:02 BST: Completed: ./incoming/1432043400.dscdata.xml                   0 remaining (0 failures).
> 
> *** Database disconnected
> ----------------------- 2015-05-19 14:50:02 BST -----------------------
> 
> 
> I have changed 
> log_statement to 'all'.  Its now logging all the queries (for the last 12 hours window)
> 
> May 19 15:02:51 localhost postgres[6279]: [2-1] LOG:  00000: connection received: host=[local]
> May 19 15:02:51 localhost postgres[6279]: [2-2] LOCATION:  BackendInitialize, postmaster.c:3544
> May 19 15:02:51 localhost postgres[6279]: [3-1] LOG:  00000: connection authorized: user=www-data database=hedgehog
> May 19 15:02:51 localhost postgres[6279]: [3-2] LOCATION:  PerformAuthentication, postinit.c:230
> May 19 15:02:51 localhost postgres[6279]: [4-1] LOG:  00000: statement: set datestyle to ISO
> May 19 15:02:51 localhost postgres[6279]: [4-2] LOCATION:  exec_simple_query, postgres.c:915
> May 19 15:02:51 localhost postgres[6279]: [5-1] LOG:  00000: statement: set statement_timeout to 300000;
> May 19 15:02:51 localhost postgres[6279]: [5-2] LOCATION:  exec_simple_query, postgres.c:915
> May 19 15:02:51 localhost postgres[6279]: [6-1] LOG:  00000: statement: PREPARE getpltdetails (INTEGER) AS SELECT name, title, description, plot_id FROM dsc.plot WHERE id=$1;
> May 19 15:02:51 localhost postgres[6279]: [6-2] LOCATION:  exec_simple_query, postgres.c:915
> May 19 15:02:51 localhost postgres[6279]: [7-1] LOG:  00000: statement: EXECUTE getpltdetails(1);
> May 19 15:02:51 localhost postgres[6279]: [7-2] DETAIL:  prepare: PREPARE getpltdetails (INTEGER) AS SELECT name, title, description, plot_id FROM dsc.plot WHERE id=$1;
> May 19 15:02:51 localhost postgres[6279]: [7-3] LOCATION:  exec_simple_query, postgres.c:915
> May 19 15:02:51 localhost postgres[6279]: [8-1] LOG:  00000: statement: select attnotnull from pg_attribute where attrelid=18903 and attnum='2'
> May 19 15:02:51 localhost postgres[6279]: [8-2] LOCATION:  exec_simple_query, postgres.c:915
> May 19 15:02:51 localhost postgres[6279]: [9-1] LOG:  00000: statement: select attnotnull from pg_attribute where attrelid=18903 and attnum='5'
> May 19 15:02:51 localhost postgres[6279]: [9-2] LOCATION:  exec_simple_query, postgres.c:915
> May 19 15:02:51 localhost postgres[6279]: [10-1] LOG:  00000: statement: select attnotnull from pg_attribute where attrelid=18903 and attnum='6'
> May 19 15:02:51 localhost postgres[6279]: [10-2] LOCATION:  exec_simple_query, postgres.c:915
> May 19 15:02:51 localhost postgres[6279]: [11-1] LOG:  00000: statement: select attnotnull from pg_attribute where attrelid=18903 and attnum='7'
> May 19 15:02:51 localhost postgres[6279]: [11-2] LOCATION:  exec_simple_query, postgres.c:915
> May 19 15:02:51 localhost postgres[6279]: [12-1] LOG:  00000: statement: PREPARE by_node_all_nodes (INTEGER, INTEGER, TIMESTAMPTZ, TIMESTAMPTZ) AS select to_timestamp((extract(epoch from sq.sx)::int / (extract(epoch from $4::timestamp - $3::timestamp + interval '1 minute')/1440)::int )*(extract(epoch from $4::timestamp - $3::timestamp + interval '1 minute')/1440)::int) AS x, sq.skey as key, avg(sq.sy) as y from (SELECT d.starttime AS sx, n.region AS skey, sum(d.value)/60.0 AS sy FROM dsc.data d, dsc.node n WHERE d.node_id = n.id AND d.server_id=$1 AND d.plot_id=$2 AND d.starttime>=$3 AND d.starttime<=$4 GROUP BY sx, skey) as sq GROUP BY x, key;
> May 19 15:02:51 localhost postgres[6279]: [12-2] LOCATION:  exec_simple_query, postgres.c:915
> May 19 15:02:51 localhost postgres[6279]: [13-1] LOG:  00000: statement: PREPARE getdatasetids (INTEGER) AS SELECT dataset_id FROM dsc.plot WHERE id=$1;
> May 19 15:02:51 localhost postgres[6279]: [13-2] LOCATION:  exec_simple_query, postgres.c:915
> May 19 15:02:51 localhost postgres[6279]: [14-1] LOG:  00000: statement: EXECUTE getdatasetids('1');
> May 19 15:02:51 localhost postgres[6279]: [14-2] DETAIL:  prepare: PREPARE getdatasetids (INTEGER) AS SELECT dataset_id FROM dsc.plot WHERE id=$1;
> May 19 15:02:51 localhost postgres[6279]: [14-3] LOCATION:  exec_simple_query, postgres.c:915
> May 19 15:02:51 localhost postgres[6279]: [15-1] LOG:  00000: statement: select attnotnull from pg_attribute where attrelid=18903 and attnum='8'
> May 19 15:02:51 localhost postgres[6279]: [15-2] LOCATION:  exec_simple_query, postgres.c:915
> May 19 15:02:51 localhost postgres[6279]: [16-1] LOG:  00000: statement: select typname, typcategory from pg_type where oid = 1007
> May 19 15:02:51 localhost postgres[6279]: [16-2] LOCATION:  exec_simple_query, postgres.c:915
> May 19 15:02:51 localhost postgres[6279]: [17-1] LOG:  00000: statement: EXECUTE by_node_all_nodes(1, '21', timestamptz '2015-05-18 03:03', timestamptz '2015-05-18 15:02');
> May 19 15:02:51 localhost postgres[6279]: [17-2] DETAIL:  prepare: PREPARE by_node_all_nodes (INTEGER, INTEGER, TIMESTAMPTZ, TIMESTAMPTZ) AS select to_timestamp((extract(epoch from sq.sx)::int / (extract(epoch from $4::timestamp - $3::timestamp + interval '1 minute')/1440)::int )*(extract(epoch from $4::timestamp - $3::timestamp + interval '1 minute')/1440)::int) AS x, sq.skey as key, avg(sq.sy) as y from (SELECT d.starttime AS sx, n.region AS skey, sum(d.value)/60.0 AS sy FROM dsc.data d, dsc.node n WHERE d.node_id = n.id AND d.server_id=$1 AND d.plot_id=$2 AND d.starttime>=$3 AND d.starttime<=$4 GROUP BY sx, skey) as sq GROUP BY x, key;
> May 19 15:02:51 localhost postgres[6279]: [17-3] LOCATION:  exec_simple_query, postgres.c:915
> May 19 15:02:51 localhost postgres[6279]: [18-1] LOG:  00000: statement: select attnotnull from pg_attribute where attrelid=18872 and attnum='7'
> May 19 15:02:51 localhost postgres[6279]: [18-2] LOCATION:  exec_simple_query, postgres.c:915
> May 19 15:02:51 localhost postgres[6279]: [19-1] LOG:  00000: disconnection: session time: 0:00:00.212 user=www-data database=hedgehog host=[local]
> May 19 15:02:51 localhost postgres[6279]: [19-2] LOCATION:  log_disconnections, postgres.c:4434
> 
> 
> 
> On 19/05/15 14:50, John Dickinson wrote:
>>> On 19 May 2015, at 14:38, matt singh <matt.singh at centralnic.com>
>>>  wrote:
>>> 
>>> Hello,
>>> I have installed Hedgehog 2.0.0b2 successfully, however I am now capturing dsc XML data from one dns node using dsc.
>>> Then importing it every 5 mins into the database using;
>>> 
>>> sudo -u hedgehog /usr/local/bin/refile_and_grok.sh -c
>>> 
>>> My node is showing up ok in the web interface.
>>> But I am not seeing any results in the web interface.  All I see is "Sorry... your query didn't return any results.  Please try re-generating the graph with new selection criteria".
>>> I am looking back over the last week for results, but nothing is showing up.
>>> I can see the "data" table in the postgresql database is growing everytime there is an import.
>>> I have verbose logging on for postgresql and its not showing any errors.  Apache is not showing any errors either.
>>> Any ideas what I'm overlooking?
>>> 
>> Hi Matt,
>> 
>> Please could you check the contents of /usr/local/var/hedgehog/data/refile_and_grok.stdout and /usr/local/var/hedgehog/data/<Server>/<Node>/dsc-extractor-xml-db.out?
>> 
>> Also you can tell postgresql to log all queries with the /etc/postgresql/9.3/main/postgresql.conf and changing the log_statement line.
>> 
>> HTH
>> John
>> 
> 
> -- 
> Matt Singh
> System Administrator
> CentralNic Group plc
> Innovative, Reliable and Flexible Registry Services
> for ccTLD, gTLD and private domain name registries
> 
> https://www.centralnic.com/
> 
> 
> London Stock Exchange Symbol: CNIC
> 
> CentralNic Group plc is a company registered in England and Wales with
> company number 8576358. Registered Offices: 35-39 Moorgate, London
> EC2R 6AR.
> 



More information about the dns-stats-users mailing list