Smarthome data logging with RPi/domoticz/influxdb/grafana

02 Dec, 2018 (Updated: 14 Dec, 2018)

When we got a smart meter and solar panels I wanted to record this data. Along the way I added more sensors and I’ve extended the software

Goals ¶

Record (smart) meter data and graph it (now including: electricity smart meter via USB, PV via bluetooth, heat meter via machine vision, water meter via line sensor, temperature via EvoHome)
Log data in generic, long-lasting format locally (i.e. no externally hosted solution)
Robust
Interactive graphs on smartphone (domoticz, plot.ly, Highcharts, dygraphs)
Customisable graphs (like grafana)
Cheap/low-power solution (i.e. no Intel NUC)
Versatile wrt interfaces (e.g. bluetooth/wifi/GPIO/USB)
Not used to control or actuate. For that I use COTS products that work and don’t crash :p

Architecture ¶

Data model ¶

Units ¶

I use only SI units for storing meter data so I can combine e.g. heat meter (native unit MJ) with smart meter (native unit Wh) with PV (native unit kWh).

Power: W, J/s
Energy: J
Temperature: deg C
Wind: m/s
Wind direction: deg
Voltage: V
Water: liter
Rain: mm

Database scheme ¶

Influxdb can only combine / add / subtract data from the same measurement. Keep this in mind when designing your scheme.

Generic format: <unit>, type=<quantity>, device=<datasource>, subtype=<device qualifier> value1=X value2=Y value3=Z <timestamp>.

Fully worked out, this looks like:

    Power,type=elec,device=<kaifa/sma/smartplug> energy=X power=Y
    Power,type=heat,device=<landisgyr> energy=X
    Power,type=radiation,device=solar energy=X
    temperature,type=heating,device=livingroom actual=X setpoint=Y
    temperature,type=device,device=sma actual=X
    temperature,type=weather,device=XXweatherloc actual=X
    wind,type=weather,device=XXweatherloc wind=X
    winddirection,type=weather,device= XXweatherloc winddir=X
    voltage,type=elec,device=sma, DC=X AC=Y
    water,type=potable,device=sensus water=X
    rain,type=weather,device=XXweatherloc rain=X

Optionally this could all be in one table (called ‘measurement’ in influxdb parlance)

Polling cadence ¶

Temperature rooms: 1x/5 minutes, resolution: 0.5 deg C
Power: 1x/min
PV: 1x/min
Water: on-demand incremental
Heat: 1x/minute Data usage at 1 measurement/minute is: 1440min/day * 2byte/measurement ±3kb/day/metric = 1 MB/yr/metric.

Software architecture ¶

Easy access visualization: domoticz
Database: influxdb
Full visualization only accessibly locally: grafana
Selected visualizations remotely accessible on smartphone: Dygraphs (influxdata.com)
Data collection: domoticz with pushers to influxdb, as well as custom scripts to push directly to influxdb.

Implementation ¶

Implementation occurred more or less chronologically in this order. I’ve written the text including hindsight to fix initial mistakes:

Installing domoticz on Rpi (vanwerkhoven.org)
Extending domoticz with influxdb/grafana
Connecting sensors to Rpi/domoticz

Alternatives ¶

Before settling on the above solution I’ve considered (and rejected) the following alternative solutions for various reasons:

hass.io + influxdb: NOK hass.io has no influxdb export, and appears too heavy (never actually tried)
OpenHab: NOK extremely slow on RPi0, briefly tried on RPi3B but still slow.
prometheus: not as widely supported as influxdb. Might actually be better because of more powerful query language
rpi-monitor / pimatic: appears not mature enough
vis.js / Dygraphs / Highcharts / Rickshaw / Plotly: too complicated

Sources ¶

#Linux #Smarthome