Smarthome data logging with RPi/domoticz/influxdb/grafana

02 Dec, 2018 (Updated: 14 Dec, 2018)

When we got a smart meter and solar panels I wanted to record this data. Along the way I added more sensors and I’ve extended the software

Goals

Record (smart) meter data and graph it (now including: electricity smart meter via USB, PV via bluetooth, heat meter via machine vision, water meter via line sensor, temperature via EvoHome)
Log data in generic, long-lasting format locally (i.e. no externally hosted solution)
Robust
Interactive graphs on smartphone (domoticz, plot.ly, Highcharts, dygraphs)
Customisable graphs (like grafana)
Cheap/low-power solution (i.e. no Intel NUC)
Versatile wrt interfaces (e.g. bluetooth/wifi/GPIO/USB)
Not used to control or actuate. For that I use COTS products that work and don’t crash :p

Architecture

Data model

Units

I use only SI units for storing meter data so I can combine e.g. heat meter (native unit MJ) with smart meter (native unit Wh) with PV (native unit kWh).

Power: W, J/s
Energy: J
Temperature: deg C
Wind: m/s
Wind direction: deg
Voltage: V
Water: liter
Rain: mm

Database scheme

Influxdb can only combine / add / subtract data from the same measurement. Keep this in mind when designing your scheme.

Generic format: <unit>, type=<quantity>, device=<datasource>, subtype=<device qualifier> value1=X value2=Y value3=Z <timestamp>.

Fully worked out, this looks like:

    Power,type=elec,device=<kaifa/sma/smartplug> energy=X power=Y
    Power,type=heat,device=<landisgyr> energy=X
    Power,type=radiation,device=solar energy=X
    temperature,type=heating,device=livingroom actual=X setpoint=Y
    temperature,type=device,device=sma actual=X
    temperature,type=weather,device=XXweatherloc actual=X
    wind,type=weather,device=XXweatherloc wind=X
    winddirection,type=weather,device= XXweatherloc winddir=X
    voltage,type=elec,device=sma, DC=X AC=Y
    water,type=potable,device=sensus water=X
    rain,type=weather,device=XXweatherloc rain=X

Optionally this could all be in one table (called ‘measurement’ in influxdb parlance)

Polling cadence

Temperature rooms: 1x/5 minutes, resolution: 0.5 deg C
Power: 1x/min
PV: 1x/min
Water: on-demand incremental
Heat: 1x/minute Data usage at 1 measurement/minute is: 1440min/day * 2byte/measurement ±3kb/day/metric = 1 MB/yr/metric.

Software architecture

Easy access visualization: domoticz
Database: influxdb
Full visualization only accessibly locally: grafana
Selected visualizations remotely accessible on smartphone: Dygraphs
Data collection: domoticz with pushers to influxdb, as well as custom scripts to push directly to influxdb.

Implementation

Implementation occurred more or less chronologically in this order. I’ve written the text including hindsight to fix initial mistakes:

Installing domoticz on Rpi
Extending domoticz with influxdb/grafana
Connecting sensors to Rpi/domoticz

Alternatives

Before settling on the above solution I’ve considered (and rejected) the following alternative solutions for various reasons:

hass.io + influxdb: NOK hass.io has no influxdb export, and appears too heavy (never actually tried)
OpenHab: NOK extremely slow on RPi0, briefly tried on RPi3B but still slow.
prometheus: not as widely supported as influxdb. Might actually be better because of more powerful query language
rpi-monitor / pimatic: appears not mature enough
vis.js / Dygraphs / Highcharts / Rickshaw / Plotly: too complicated

Sources

#Linux #smarthome