Page 1 of 1

Creating a DashBox Development DB

Posted: Sun Dec 03, 2017 9:23 pm
by Mustang65
Being new at this.. I will have a bunch of questions so here they come.

I am inching my way through setting up the system, fine tuning it and I would like to take it to the next level. I am not one that likes to mess with the real time data, so I would like to create a database for running queries against... a Reports dB. To start off with, I would like to use my laptop (running Windows 10) until I get the Raspberry Pi server back on line again.

First things first, I do not see the dash box on my network (using File Manager), like the HD's other computers, laptops, tablets... Am I supposed to?

How can I make a copy of the DashBox dB on my laptop, so I can start creating some queries? I see it creates the backup file as a zip file ( I have already made a backup to see how it works). Can that dB get extracted from the zip file into a location of my choice?

The perfect scenario would be to have the backup program do the backup and have an option to create a Development Db (from the backup) if requested, and create it in a desired location.

I believe I read that there are different levels of data that can be saved... minute, day.., how do I know what my settings are for this. Please do not ask me what my initial entries were. I looked at the settings page but that only had the options to set up the level. Is there a page that displays the current settings that are being used?

Well that is enough to get me started.

Re: Creating a DashBox Development DB

Posted: Sun Dec 03, 2017 11:33 pm
by Teken
Mustang65 wrote:Being new at this.. I will have a bunch of questions so here they come.

I am inching my way through setting up the system, fine tuning it and I would like to take it to the next level. I am not one that likes to mess with the real time data, so I would like to create a database for running queries against... a Reports dB. To start off with, I would like to use my laptop (running Windows 10) until I get the Raspberry Pi server back on line again.

First things first, I do not see the dash box on my network (using File Manager), like the HD's other computers, laptops, tablets... Am I supposed to?

How can I make a copy of the DashBox dB on my laptop, so I can start creating some queries? I see it creates the backup file as a zip file ( I have already made a backup to see how it works). Can that dB get extracted from the zip file into a location of my choice?

The perfect scenario would be to have the backup program do the backup and have an option to create a Development Db (from the backup) if requested, and create it in a desired location.

I believe I read that there are different levels of data that can be saved... minute, day.., how do I know what my settings are for this. Please do not ask me what my initial entries were. I looked at the settings page but that only had the options to set up the level. Is there a page that displays the current settings that are being used?

Well that is enough to get me started.
Hello Don,

- The DB will not show up as a Windows network attached device because it runs a flavor of Linux. It will of course show up on the router, modem, firewall, scan tool etc. And of course if you enter the IP address in any web browser you can access and view it.

- The hardware Dash Box (DB) is not intended to be replicated on to another piece of hardware. As this would take away the funding for the actual software development. Brultech offers a slimmed down version of the DB software in their download section however. If you would like to have another similar working copy the best method is to deploy one of many 3rd party software solutions which operate on a RPi etc.

Also keep in mind the DB can stream and relay that same data to local and cloud hosted services like SEG.

The software that resides on the hardware version of the DB is a premium product. Many years and countless hours have been devoted by the Brultech team to bring it where it is now. The annual sales of the DB hardware is what funds this development.

Re: Creating a DashBox Development DB

Posted: Mon Dec 04, 2017 9:49 am
by Mustang65
I have no problem with that except that where is documentation regarding this information? I looked through a bunch of posts and found nothing. I can do the queries, but need to know what is available and how to initiate the process of getting to the data. Is there documentation with what is necessary to get access to the data? Is there road map like the GEM user manual?

Point me in a direction where I can take the documentation get what ever I need and get on with the queries. Trying to piece part the instructions from 10 or 20 threads is not my idea of a guide.

Point me in the right direction.

Thanks again

Re: Creating a DashBox Development DB

Posted: Mon Dec 04, 2017 10:58 am
by ben
We don't have a guide on this yet.

You can either query the database directly using the backup user (dbbackup) or restore the database into your own PostgreSQL database.

If you're dealing with historical data and want to run some long-running queries, backup and restore will be your best bet as this won't cause any interruptions to the DashBox and your query might get targeted by the OOM killer (as there's no swap on the DB due to increased write-cycles).

The database isn't the cleanest, haven't had enough time to pretty it up so there's a few unused columns, etc.

Everything you want to do is likely a combination of the following tables:
- channel (channel_id, channel_name, ctype, and maybe hide would be the interesting columns)
- minutedata (partitioned by day)
- hourdata
- daydata

ctype is as follows:
- 0 Energy
- 1 Temperature
- 2 Pulse
- 3 Voltage
- 4 NET
- 6 Outdoor Weather add-on

They're linked by channel_id and dates are in UTC. The partitioning is a little off as it uses UTC-time instead of local time. If you're dealing with the minutedata, you're likely best off addressing the dated tables directly instead of minutedata as a whole.

This query for example should return kWh by the hour for today for all energy channels (EST).

Code: Select all

SELECT date, channel_name, kwh
FROM hourdata
INNER JOIN channel ON channel.channel_id = hourdata.channel_id
WHERE ctype IN (0,4) 
AND hide = 0
AND date > '2017-04-12 05:00:00';
The minutedata equivalent would be:

Code: Select all

SELECT date, channel_name, watt, kwh
FROM "minutedata2017-04-12"
INNER JOIN channel ON channel.channel_id = "minutedata2017-04-12".channel_id
WHERE ctype IN (0,4) 
AND hide = 0
AND date > '2017-04-12 05:00:00';