Prometheus at Home - on Mac OS

I have an old circa Mid-2010 that I am planning on using to control my Form 1 (presuming I can get resin for it) and various other things.

It struck me as a good idea to use it as a Prometheus server for my IoT research projects as part of BlueTrain Software Ltd.

Our golden trio is Prometheus, Prometheus PushGateway (for holding the job data to be scraped by Prometheus) and Grafana for graphing. Ideally putting alerts on metrics would be a good idea (we have air-con, so underneath the house suddenly dipping in temperature in the summer or heating up in the winter is a red flag a pipe has burst - for example).

Prometheus is in brew, so installing it is pretty simple.

brew install prometheus

Don't install it as a service as yet - it will install based on your login account, not on boot (which you want). You could leave it against your login account if you auto-login on boot, but that is terribly insecure, so let's not do that.

You could also use Docker for these, but Docker won't run on machine boot either, so it isn't useful here.

Prometheus Gateway isn't so easy -  you have to download the binary and copy it the right place. I grabbed the 386 darwin binary and moved it to /usr/local/bin. It runs on port 9091, so thats fine.

Now to get them both running as launchd agents. For the PushGateway, use something like this:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>org.prometheus.pushgateway</string>
    <key>ProgramArguments</key>
    <array>
        <string>/usr/local/bin/pushgateway</string>
    </array>
    <key>KeepAlive</key>
    <true/>
</dict>
</plist>

it will just run the push gateway with the default arguments.

You need to put that in /Library/LaunchDaemons - name it org.prometheus.pushgateway.plist and then load it as sudo: sudo launchctl load org.prometheus.pushgateway.plist  (from the /Library/LaunchDaemons directory).

Trap for young players - these plist files must be owned by root:wheel if you intend to run them with sudo, so chown root:wheel X.plist your files before trying to load them.

To run prometheus, I created a /usr/local/etc/prometheus.yml file and then a modified /Library/LaunchDaemons/org.prometheus.plist file, they looked like these respectively.

# my global config
global:
  scrape_interval:     15s # By default, scrape targets every 15 seconds.
  evaluation_interval: 15s # By default, scrape targets every 15 seconds.
  # scrape_timeout is set to the global default (10s).

  # Attach these labels to any time series or alerts when communicating with
  # external systems (federation, remote storage, Alertmanager).
  external_labels:
      monitor: 'codelab-monitor'

# Load and evaluate rules in this file every 'evaluation_interval' seconds.
rule_files:
  # - "first.rules"
  # - "second.rules"

# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: 'prometheus'

    # Override the global default and scrape targets from this job every 5 seconds.
    scrape_interval: 5s

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
    - targets: ['localhost:9090']
  - job_name: 'push-gateway'

    # Override the global default and scrape targets from this job every 5 seconds.
    scrape_interval: 5s

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
    - targets: ['localhost:9091']

prometheus plist file:

<?xml version="1.0" encoding="UTF-8"?>
  <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
  <plist version="1.0">
  <dict>
    <key>Label</key>
    <string>homebrew.mxcl.prometheus</string>
    <key>ProgramArguments</key>
    <array>
      <string>/usr/local/opt/prometheus/bin/prometheus</string>
      <string>--config.file=/usr/local/etc/prometheus.yml</string>
    </array>
    <key>RunAtLoad</key>
    <true/>
    <key>KeepAlive</key>
    <false/>
    <key>StandardErrorPath</key>
    <string>/usr/local/var/log/prometheus.err.log</string>
    <key>StandardOutPath</key>
    <string>/usr/local/var/log/prometheus.log</string>
  </dict>
</plist>

Again, you need to load this plist file in the same was you did above.

As you can see, prometheus logs in /usr/local/var/log - so check its running.

For grafana, its pretty straight forward again:

brew install grafana

then go into the Cellar for it, in my case 5.4.2:

➜  5.4.2  pwd
/usr/local/Cellar/grafana/5.4.2
➜  5.4.2  ls
CHANGELOG.md                LICENSE.md                  README.md                   homebrew.mxcl.grafana.plist
INSTALL_RECEIPT.json        NOTICE.md                   bin                         share
➜  5.4.2  sudo cp homebrew.mxcl.grafana.plist /Library/LaunchDaemons 
➜  5.4.2  cd /Library/LaunchDaemons 
➜  LaunchDaemons  sudo chown root:wheel homebrew.mxcl.grafana.plist 
➜  LaunchDaemons  sudo launchctl load homebrew.mxcl.grafana.plist 

RF2500 in more detail

I got myself a macro lens for my EOS350D for taking photos of electronics, so here the RF2500 is in more detail.

Ordering RF2500's from Alibaba

With the success of the 43oh.com guys in ordering from Alibaba, I thought I'd like to organize some - Tim from MindKits said he'd buy some as well (20 - go there soon if you want some). They are pretty tiny, but solderable. They are designed so you can put header pins in them and solder them onto boards as modules or fit a plastic cap over them of some kind if you want to stand it upright. Once I get my MezzoMill, I'll print a board out that will let me use them more easily, but for now I will just solder wires into them and stick them in a breadboard :-)

My intention is to write drivers for the RaspberryPi for them and also for the MSP430.

I am expecting I'll need to use the BusPirate I ordered from SeeedStudio (to support Dangerous Prototypes, the only reason I didn't order from MindKits who get it via SparkFun don't pay DP a bean for it) to debug my SPI work.

Buying from China on eBay

With good stories from other people, I decided to try my hand at purchasing some items on eBay that I wanted in my kit - and first to arrive is the 16x2 backlist LCD based on the Hitachi HD44780. It took around a month to get them here (New Zealand) from China, a little longer than they said, but it was Chinese New Year in between so I did expect that. They were certainly not the cheapest of the ones on there, but many didn't ship here, so they were ruled out.

They were well packed with plenty of bubble wrap and arrived undamaged.

I soldered one of them up with header pins and attached them to my bread board, which immediately pushed the not soldered well enough ones back out again.

So on the second go I was successful with all but the final pin which I was worried about the soldering anyway - I am using a very fine tip on my Hakko and am very pleased with its performance.

Wiring it all up, figuring out how the LCD backlight is turned on and skipping the trimpot all made for a nice experience. I used the sample Arduino sketch just to check the device out to ensure it was working as expected - I didn't want to go through the same with the MSP430 Launchpad as I haven't put headers on my board yet.

Works exactly as advertised and i'd recommend the seller for people looking for these LCD devices.

Radio

I dislike the Arduino platform for one main reason - its focus on for loops with delays. I understand that the Atmel chips can work primarily on interrupts and in lower power, but the MSP430 Launchpad from TI is really focused on low power. Its not a board for beginner programmers (though EasyMSP may help that). Why low power? I know not every board wants or needs this, but radio in my opinion is where it is at - building something and then having it having no connectivity is just plain weird. Every single project I think of, I want to have it being monitored. I have yet to find the perfect radio kit - I think I'll want a sensor net of RF2500's (as they are cheap boards - around $US2) - they have a range of 30-50 metres and something like an RFM22B ($US13, SPI) or Doji ($NZ30, UART) board for longer range. And there is always RFID for ultra short ranges (<20cm).

PCB really need to be easy

So although we managed to get the circuit working, as soon as we stuffed it inside the mouse it short circuited and parts started failing, which is what I expected unfortunately. So we decided to try and get a perf board and went to Jaycar and got a selection of two different types - one with the copper all along and one with the copper in various patterns. Unfortunately, with this mouse case it really showed up one of the things that continues to frustrate me - which is that it really needed its own PCB.

I'm not at all experienced in PCB layout or manufacture, and one of the reasons I have stayed away for so long is the mess. Chemical processes using etching at home are just plain nasty, and all the processes people seem to be going through to get around this still seem messy and nasty.

So over the xmas period, I found there was a Kickstarter project called MezzoMill - a CNC machine that a guy (David) has put a huge amount of effort into making a really good method for just printing from Eagle. The Kickstarter failed unfortunately, which is pretty sad, but I talked to David directly and he said ok to build a MezzoMill for me and shipping it out here. This, I am totally buzzed about - more so than the 3d printer (Replicator) that I ordered from MakerBot.

Time will see if its just another folly, but hopefully not!

One thing I think that really drove me as a teenager, and drives me still is my love of doing projects. Projects can be simple, short - they can be as simple as making some sausages, cleaning up a section of the garden or writing some software to do math sheet generation for my nearby school (mathsheetsfor.me) for instance.

This holiday break (Summer is Jan-March here in New Zealand), as I was cleaning up my office of old junk and came across some old mice. Pre-laser style. My 9 year old was helping me and asked me which pile they should go in. I told him to snip the end off with the pliers and put them in the "make" cupboard. He asked me why - I told him about the Mouse Bot I had seen in the 2nd edition of Make Magazine.

That started quite a learning curve for me. We got through the first parts together fairly easily - hollowing out the dead mouse, but the electronics were a bit nastier - particularly as the parts I had available didn't match the project. Particularly difficult were the relay (which ended up needing to be wired up completely different to the diagrams), finding out the original circuit in the magazine was wrong, and getting the right transistor.

All in all, I learnt an awful lot. I particularly learned that soldering wires together is a horrible business and that my 9 year old can solder better than I can. We still don't quite have it working - the bounce circuit seems to have died but given all that cramming, I can see why.