btmon, emoncms and GEM - Volts, Current, Power Factor

Post any 3rd party software here.
sub3marathonman
Posts: 95
Joined: Fri Feb 11, 2011 9:32 am

Re: btmon, emoncms and GEM - Volts, Current, Power Factor

Post by sub3marathonman » Fri Feb 19, 2016 10:41 pm

Interesting that wci68 has been working on this simultaneously as I have.

With great thanks to awtivy, I've finally figured out a bit clearer what is going on with btmon and the GEM.

It would be great to have the btmon updated just a bit to include the volts and amps, and it could be as part of a question and answer option if somebody knows how to program that.

I have followed awtivy's instructions, but it seems that there is an error of omission for 1768, as it didn't work until I put that in. (I have it listed as ch%d_amp, I'm not sure if you changed it to a different name from _amp what would happen. It shows up on the emoncms inputs though as _a and not _amp and I'm not sure why but it didn't really matter to me as long as it seems to work.) Also, for line 1720 I changed the number of channels from 32 to 40, in order to generate more inputs to work with in emoncms for calculating feeds. I was also able to get volts appended too from awtivy's revised 6/23/2015 posting. (And for those who need basic instructions, such as myself, the line numbers are for guidance only and aren't typed in as part of the instructions. I also made a comment with the # date initials method so that I could easily search and see which lines were changed. And finally I changed the name of btmon.3.1.1.py to btmon.3.1.1a.py so I would still have the original btmon present to double-check with.)

Code: Select all

932        FILTER_CURRENT = 'current'

1720       self.NUM_CHAN = 40

1766       elif fltr == FILTER_CURRENT:
1767           for x in range(1, self.NUM_CHAN + 1):
1768                c.append('ch%d_amp' % x)

1800       for x in range (1, self.NUM_CHAN + 1):
1801             cpkt['ch%d_a' % x] = self._convert(rpkt[486+2*(x-1):486+2*x]

3760       for idx, c in enumerate (PACKET_FORMAT.channels(FILTER_PE_LABELS)):
3761          data.append('%s_a:%.2f' % (mklabel(osn, c), p[c+'_a']))

3768    data.append('%s:%.1f' % (mklabel(osn, 'volts'),p['volts']))
Also, emoncms was initially set up and running with btmon without the amperage channels and the voltage, but seemed to do fine adding them with the modified btmon when it initially and subsequently was running.

(Revised 2/20/2016 to correct mistake on line 1801 (originally incorrectly listed as 487 instead of 486.)
Last edited by sub3marathonman on Sat Feb 20, 2016 2:02 pm, edited 1 time in total.
wci68
Posts: 43
Joined: Thu Dec 17, 2015 4:31 pm

Re: btmon, emoncms and GEM - Volts, Current, Power Factor

Post by wci68 » Sat Feb 20, 2016 11:53 am

You can see my changes in-context on github: https://github.com/WaterByWind/mtools/c ... 79273d07f7 You can get there from the list of pull requests on the original master (https://github.com/matthewwall/mtools/pulls)

There really isn't much needed.

Essentially the GEM (and ECM*) output a fixed-size record that includes a list of values. The details are described in the GEM Packet Format Manual under the Developer tab on the GEM software downloads page. There are no names or other tags output from the GEM - just a set of raw values (for the BIN* formats, including BIN48-NET-TIME used by btmon).

In text-based outputs the GEM does have a standard for naming, and it appears btmon is similar. The difference is that the text-based GEM names start with a single-letter tag while the names used by btmon end with the single letter tag: "a_08" vs "ch8_a". There is no relationship between these however and they are independent.

btmon extracts those raw values from the binary packet into a list of key/value pairs. The keys are arbitrary names and it looks like originally '_amp' was used for those (ecmread.py?) but there is code elsewhere to reconcile these to '_a'. '_amp' probably shouldn't be used, at least not without checking the other classes to ensure they'll actually use them.

However, you actually don't even need this at all since you never use that list. On 1766-1768 you create a list of names for 'current', but you don't actually reference that list elsewhere. My changes show how this could be used but really it isn't needed. Instead you are using the 'pelabels' list which works just fine. This list is just the channel names without any additional tag (_w, _wh, _a, etc), with that tag being added to the name when it is sent/published/stored. Your lines 1800-1801 create the key/value pairs for the currents (amperage), and your 3760-3761 add those key/value pairs to the string to be published to emoncms. In both cases, the '_a' is added to the key name at that time.

c.append() is simply a method to add a key name when building the "schema" (a list called 'c'). Any names added here are used elsewhere for processing and sending/publishing data.

The proper reference point for your line 1801 should be '486' instead of '487', which had been noted in a corrective followup by awtivy and can be confirmed using the GEM Packet Format guide noted earlier. (I did double-check this) Your amperages will be incorrect if using 487.


It isn't clear why you would change the number of channels in btmon from 32 to 40, since there would be no data in those other "fields". This value has no implication on emoncms or any other service, other than publishing eight additional sets (w, wh, a) of empty data. In emoncms you would first use an input, published by btmon, and add a process to create a feed. You would add more processes to that feed if needed. If you need more "channels to calculate feeds" in emoncms from what it sounds like you would use virtual feeds. For instance if you have both legs of your service monitored on separate GEM channels but want emoncms to display a single set of power and energy values and graphs you could use a virtual feed to add the two feeds from the inputs together.


One note for using btmon with current versions of emoncms: You need to set 'oem_node' in your config file since leaving it empty will cause an endless stream of new inputs to be created with each publication. This has been noted by a few in the OEM forums. Unless you have more than one GEM (or ECM) using 'oem_node = 1' is sufficient.
sub3marathonman
Posts: 95
Joined: Fri Feb 11, 2011 9:32 am

Re: btmon, emoncms and GEM - Volts, Current, Power Factor

Post by sub3marathonman » Sat Feb 20, 2016 2:01 pm

wci68 wrote:
The proper reference point for your line 1801 should be '486' instead of '487', which had been noted in a corrective followup by awtivy and can be confirmed using the GEM Packet Format guide noted earlier. (I did double-check this) Your amperages will be incorrect if using 487.


It isn't clear why you would change the number of channels in btmon from 32 to 40, since there would be no data in those other "fields". This value has no implication on emoncms or any other service, other than publishing eight additional sets (w, wh, a) of empty data. In emoncms you would first use an input, published by btmon, and add a process to create a feed. You would add more processes to that feed if needed. If you need more "channels to calculate feeds" in emoncms from what it sounds like you would use virtual feeds. For instance if you have both legs of your service monitored on separate GEM channels but want emoncms to display a single set of power and energy values and graphs you could use a virtual feed to add the two feeds from the inputs together.
Thank you wci68, you're absolutely right, as awtivy's previous revised post shows, it should be 486 and not 487, and I'm revising my previous posting to reflect this so nobody will be confused.

Yes, I thought about virtual feeds for emoncms, but there is almost no documentation on how to use anything, and help is just as lacking, so I just did it the primitive way that seems to work. I was also needing the original input value for two or more separate feeds, so if I had calculated one feed I'd then have to go backwards and "uncalculate" back to the original input value. There is so much potential with the emoncms program, but without a manual it is just trial and error and spending time on it when I can.
Post Reply