A package for sensor data

As mentioned earlier, i thought on how to transport the sensor data my wireless nodes gather.

As i want the nodes to be able to run from battery power, they should consume as few energy for transmitting data as possible. While my former experiments with TCP/IP were pretty exciting (its awesome that you can run a website wirelessly off an 10€ battery driven uC!) its hard to get a node to become a low power device once you have to take care of all the TCP/IP protocol stack.

So i switched to a very basic network layer, based on the RFM12 lib. When there is data to transmit, it is packed to a simple protocol structure and transmitted over the air as bytestream. I wrote a little class to do all the packing and unpacking work named RadioNodePdu.

Each measurement is represented as the following datastructure:

So for each measurement reading, we have its type (as 8 Bit int) and its value (as 16 Bit int). As a header we then just add the ID of the sending node and pack all to a compact linear bytestream:

This bytestream then is transmitted to the accesspoint, which then decodes it and translates it back to the sensor data. As the length of each channel structure stays the same, we can very easily get the number of channels from the received bytestream:

The single channels then can be accessed very easily by their offsets:

After the data is received by the accesspoint, its receive is acknowledged and then it is converted to a JSON string and posted to my monitoring server via a HTTP POST request. Then a PHP skript jumps in and does the further processing of the data.

The nice thing about this is, that with the type of data transmitted, my monitoring system can automatically "learn" the new channels, once a new node goes online. It creates a new "datafeed" and automatically assigns the correct physical unit and scaling factors to the measurement data, so you just have to name the channel.

Next processing steps can follow!

Protocol thoughts

As explained in my previous post, i will look for a way to push information into my monitoring system. I have not yet decided which protocol and format to use.

There are at least two protocol layers you have to care about: The application layer and the transport layer. While the application layer carries the payload data like sensor readings or actor commands, the transportation layers job is to ensure a correct transmission of that data.

For the application layer,  i thought about using something like XML or JSON. While these "full-grown" standard protocols offer the possibility to use the hole bunch of development tools and libraries, they add a lot of processing and transmission overhead. When thinking of wireless sensor networks, you always have to keep in mind the low bandwith of their connection and their sometimes very limited energy ressources. It would be cool to have a sensor node running of a mini solar panel for example. For this reason, i currently prefer a more simple binary protocol that just carries the data needed. 

Same thoughts apply to the transportation layer. First thing that came to my mind was "hey, lets just use TCP/IP". Cool things about IP and TCP are: it's

• stable
• provides a secure transportation layer
• is easy to program with all programming languages (standard sockets)
• easy to add a encryption layer and firewalling
• and you got big pile of standard debug, development and monitor solutions for it

so at first i thought it would be the perfect solution for my wireless nodes. But there are a number of disadvantages when using TCP/IP.

First: Overhead. Both in transport data volume and processing efford. There is a lot of information involved that is needed to keep the internet running - e.g. routable addresses

Second disadvantage is: TCP tries to ensure that all data send to the remote receiver gets submitted, using various tricks like ACKs, sequence numbers, the three-way-handshake, retransmissions, growing timeouts and congestion control. When you are transmitting data from a sensor network this does not always make sense. While some events should be transmitted with high security (like "the mailbox was filled" or "open the garage door") some other just get outdated too fast. When you transmit a temperature reading every 5 seconds or so, it just makes no sense to try to resubmit a lost transmission over and over again, as there might be a new sensor reading while the last one still is not transmitted. You better just try to submit it for a few times, and if that does not work, you just wait for the next reading and try again. Its sometime no problem to loose readings, its not critical.