How to Use MQTT & Raspberry Pi to Make the Ultimate IoT hub

in recipes

9109691433_d6138da512_k

A Raspberry Pi makes a really great central hub for small IoT networks. It is very inexpensive, it consumes very little power and it has no fans or other mechanical parts that might cause failure. This post shows you the steps to get up and running with an MQTT broker, and node-red, a great visual programming tool, which works great with ThingStudio.

Part 1 – MQTT

STOP PRESS: The latest versions of libwebsockets are incompatible with the latest versions of mosquitto. The last known good versions of libwebsockets and mosquitto are:

http://mosquitto.org/files/source/mosquitto-1.4.4.tar.gz

and

http://git.libwebsockets.org/cgi-bin/cgit/libwebsockets/snapshot/libwebsockets-1.4-chrome43-firefox-36.tar.gz

 

Mosquitto is very popular MQTT broker, and works well on small footprint devices. Unfortunately, the current version that comes with Raspbian is somewhat out of date and we will need at least the 1.4 version, so we will have to install from the source distribution.

Skills required:

  • Using the bash command line
  • Basic editing of text files with vi or nano

Steps:

– Update your distribution

– Install pre-requisite software

You can do the above all on one line, but I prefer to see each one work individually.

– Get source code for libwebsockets

-Unpack

 

-Change directory

-Create build directory

-Build

– Install

– Rebuild library cache.

– Change directory to home

– Get the source code for mosquitto.

– Unpack the code.

– Change directories

– Edit config.mk

change the line

to

Make mosquitto

Install mosquitto

– Make the configuration directory

– Copy the default config file over to /etc

– Enable websockets

Add the following two lines to /etc/mosquitto/mosquitto.conf

– Add a user for mosquitto

– Reboot

Part 2 – Node-red

This part is really easy, both for you, and for me, because I don’t have to write anything. Go to nodered.org,  (http://nodered.org/docs/hardware/raspberrypi.html) and follow the instructions. When you completed this, you’ll be ready to create some great solutions with your Raspberry Pi and ThingStudio.

I make stuff.

40 Comments

  1. Gilson

    Hi,
    Followed instruction to install 1.4 but still can’t create a connection.
    When I run mosquitto with the verbose option can see error message:

    1428979794: New connection from 192.168.1.204 on port 1883.
    1428979794: Socket error on client , disconnecting.

      1. Margot

        I have the same error but I’m running mosquitto 1.4 on a raspberry pi. mosquitto is able to open a websockets port on 9001 and listen. I can connect fine but I get that same socket error message. I’m using an example javascript Paho code found here: https://www.eclipse.org/paho/clients/js/

        here is my mosquitto.conf:
        allow_anonymous true
        autosave_interval 1800
        persistence true
        persistence_file m2.db
        persistence_location /var/lib/mosquitto/
        #connection_messages true
        #log_timestamp true
        #log_type all
        #log_dest file /var/log/mosquitto/mosquitto.log

        listener 1883

        listener 9001 127.0.0.1
        protocol websockets

        —-

        Any ideas? Thanks for the help!

  2. Ryan

    When installing libwesockets I get the following error – file cannot create directory: usr/local/lib/pkgconfig. Maybe need administrative privilages
    Sorry, but I’m a Linux newb

  3. Gilson

    Hi Mike,
    I was trying to configure the connection from things tudo on my notebook browser on 192.168.1.204 to the Pi with mqtt broker 1.4 on 192.168.1.6. This was when using port 1883, but 9001 did not work either.

  4. Mike Alport

    Hi,
    I got an error with the cmake .. command:
    Any suggestions?
    Terminal response is as follows:
    pi@Rpi8 ~/libwebsockets/build $ cmake ..
    — The C compiler identification is GNU 4.6.3
    — Check for working C compiler: /usr/bin/gcc
    — Check for working C compiler: /usr/bin/gcc — works
    — Detecting C compiler ABI info
    — Detecting C compiler ABI info – done
    — CMAKE_TOOLCHAIN_FILE=”
    — Found Git: /usr/bin/git
    Git commit hash: 408f537
    — Performing Test HAVE_inline
    — Performing Test HAVE_inline – Success
    — Performing Test HAVE___inline__
    — Performing Test HAVE___inline__ – Success
    — Performing Test HAVE___inline
    — Performing Test HAVE___inline – Success
    — Looking for bzero
    — Looking for bzero – found
    — Looking for fork
    — Looking for fork – found
    — Looking for getenv
    — Looking for getenv – found
    — Looking for malloc
    — Looking for malloc – found
    — Looking for memset
    — Looking for memset – found
    — Looking for realloc
    — Looking for realloc – found
    — Looking for socket
    — Looking for socket – found
    — Looking for strerror
    — Looking for strerror – found
    — Looking for vfork
    — Looking for vfork – found
    — Looking for getifaddrs
    — Looking for getifaddrs – found
    — Looking for dlfcn.h
    — Looking for dlfcn.h – found
    — Looking for fcntl.h
    — Looking for fcntl.h – found
    — Looking for in6addr.h
    — Looking for in6addr.h – not found
    — Looking for inttypes.h
    — Looking for inttypes.h – found
    — Looking for memory.h
    — Looking for memory.h – found
    — Looking for netinet/in.h
    — Looking for netinet/in.h – found
    — Looking for stdint.h
    — Looking for stdint.h – found
    — Looking for stdlib.h
    — Looking for stdlib.h – found
    — Looking for strings.h
    — Looking for strings.h – found
    — Looking for string.h
    — Looking for string.h – found
    — Looking for sys/prctl.h
    — Looking for sys/prctl.h – found
    — Looking for sys/socket.h
    — Looking for sys/socket.h – found
    — Looking for sys/stat.h
    — Looking for sys/stat.h – found
    — Looking for sys/types.h
    — Looking for sys/types.h – found
    — Looking for unistd.h
    — Looking for unistd.h – found
    — Looking for vfork.h
    — Looking for vfork.h – not found
    — Looking for zlib.h
    — Looking for zlib.h – found
    — Looking for 4 include files stdlib.h, …, float.h
    — Looking for 4 include files stdlib.h, …, float.h – found
    — Looking for stddef.h
    — Looking for stddef.h – found
    — Check size of pid_t
    — Check size of pid_t – done
    — Check size of size_t
    — Check size of size_t – done
    — Performing Test HAVE_VISIBILITY
    — Performing Test HAVE_VISIBILITY – Success
    — Found ZLIB: /usr/lib/arm-linux-gnueabihf/libz.so (found version “1.2.7”)
    ZLib include dirs: /usr/include
    ZLib libraries: /usr/lib/arm-linux-gnueabihf/libz.so
    Compiling with SSL support
    CMake Error at /usr/share/cmake-2.8/Modules/FindPackageHandleStandardArgs.cmake:97 (MESSAGE):
    Could NOT find OpenSSL, try to set the path to OpenSSL root folder in the
    system variable OPENSSL_ROOT_DIR (missing: OPENSSL_LIBRARIES
    OPENSSL_INCLUDE_DIR)
    Call Stack (most recent call first):
    /usr/share/cmake-2.8/Modules/FindPackageHandleStandardArgs.cmake:288 (_FPHSA_FAILURE_MESSAGE)
    /usr/share/cmake-2.8/Modules/FindOpenSSL.cmake:298 (find_package_handle_standard_args)
    CMakeLists.txt:508 (find_package)
    — Configuring incomplete, errors occurred!

  5. Gilson

    Hi Mike,

    Yes, mosquitto.conf has:

    listener 9001
    protocol websockets

    listener 1883
    protocol mqtt

    When I start the broker I see the following:

    pi@raspberrypi ~ $ mosquitto -v
    1429133679: mosquitto version 1.4.1 (build date 2015-04-13 22:31:17-0300) starting
    1429133679: Using default config.
    1429133679: Opening ipv4 listen socket on port 1883.
    1429133679: Opening ipv6 listen socket on port 1883.
    1429133679: Warning: Address family not supported by protocol
    1429133681: New connection from 127.0.0.1 on port 1883.

    Thanks,
    Gilson

  6. Peter Scargill

    Hi there

    As you know I’ve successfully installed Mosquitto + websockets from your instructions – with the minor change that I’ve added in the last two lines to get port 1883 back – and as we know that all works.

    So I’ve just done a FRESH install on a brand new Pi2 and followed the instructions again. I also copied my passwords.txt file from the other Arduino (2 WinSCP windows up in Windows 8 and you can just do 2 hops to copy from one to the other, checking permissions at the destination.

    I’ve also copied the script across to run it as a service but when I tried that nothing happened.
    So I went to /pi/mosquitto-1.4.1 and ran “mosquitto -c /etc/mosquitto/mosquitto.conf” and I got the message back…
    bash: mosquitto: command not found

    There is a file called mosquitto that is supposed to be somewhere else – maybe a directory with “local” in it – but for the life of me I can’t remember where that is supposed to be. Any ideas why I’d be having this issue? (prefixing above with sudo makes no difference).

    Pete.

  7. Mike Alport

    Hi Mike,
    Re my build problem, I did:
    “sudo apt-get install openssl”
    as you suggested and got:
    “openssl is already the newest version”
    Redid the make and still getting the same error:
    pi@Rpi8 ~/libwebsockets/build $ cmake ..
    — CMAKE_TOOLCHAIN_FILE=”
    Git commit hash: 408f537
    ZLib include dirs: /usr/include
    ZLib libraries: /usr/lib/arm-linux-gnueabihf/libz.so
    Compiling with SSL support
    CMake Error at /usr/share/cmake-2.8/Modules/FindPackageHandleStandardArgs.cmake:97 (MESSAGE):
    Could NOT find OpenSSL, try to set the path to OpenSSL root folder in the
    system variable OPENSSL_ROOT_DIR (missing: OPENSSL_LIBRARIES
    OPENSSL_INCLUDE_DIR)
    Call Stack (most recent call first):
    /usr/share/cmake-2.8/Modules/FindPackageHandleStandardArgs.cmake:288 (_FPHSA_FAILURE_MESSAGE)
    /usr/share/cmake-2.8/Modules/FindOpenSSL.cmake:298 (find_package_handle_standard_args)
    CMakeLists.txt:508 (find_package)
    — Configuring incomplete, errors occurred!

    This is on a new RPi2 – even tried purging openssl and reinstalling it.
    Any more ideas?
    Seems to be a system variable problem?

    1. Mike Karliner

      I’m not close to a raspberry pi today, but I think you may need openssl-dev,
      rather than just openssl. For those of you who don’t know, many Debian
      packages come in two versions, one for runtime use, and one for developing
      with, typically ‘-dev’, for when you want to link their libraries with something
      else. My apologies for not thinking of this earlier. I did write this article
      starting with a blank Raspbian distribution, so I don’t understand why
      this is happening to you.

      Mike

    2. Mike Karliner

      I thought I’d answered this but maybe I haven’t.
      It’s saying that it can’t find the include directory for openssl
      Check that you have files under /usr/include/openssl.
      If they have been installed somewhere else, you can use export OPENSSL_ROOT_DIR=somewhere
      to point cmake at them.

      Sorry for delay in replying.

      Mike

  8. Bruce

    Thanks for the tutorial, I updated the MQTT on my copy of the OS from thethingbox.io and now I have it talking to thingsud.io

    1. paulisloud

      Hey Bruce,

      Glad to hear the tutorial worked for you and that you’re up and running. We’ve got lots of documention here (http://demo.thingstud.io/docs) to help you take what you’ve got even further, and a forum (http://forum.thingstud.io) where you can chat with us and other users. We just setup the forum this week, so it’s a bit bare at the moment, but we’re actively monitoring it, so if you run in to any snags or just want to chat about what you’re working on, drop us a line and we’ll get back to you shortly. Good luck with your projects!

  9. gs

    hi,

    first of all congratulations on tutorial.
    i need some help please,
    i build mosquitto with websockets := yes and when i add to mosquitto.conf
    “listener 8080
    protocol websockets”
    mosquitto give me:
    “[FAIL] mosquitto is not running … failed!”

    thanks,
    gs

  10. gambituk

    I had a lot of trouble with this install.. the following errors..

    cc sub_client.o client_shared.o -o mosquitto_sub -L../lib ../lib/libmosquitto.so.1
    make[1]: Leaving directory ‘/home/pi/mosquitto-1.4.2/client’
    make[1]: Entering directory ‘/home/pi/mosquitto-1.4.2/src’
    cc -Wall -ggdb -O2 -I. -I.. -I../lib -DVERSION=”\”1.4.2\”” -DTIMESTAMP=”\”2015-05-29 12:57:21+0200\”” -DWITH_BROKER -DWITH_TLS -DWITH_TLS_PSK -DWITH_UUID -DWITH_BRIDGE -DWITH_PERSISTENCE -DWITH_MEMORY_TRACKING -DWITH_SYS_TREE -DWITH_WEBSOCKETS -DWITH_EC -c mosquitto.c -o mosquitto.o
    mosquitto.c: In function âmainâ:
    mosquitto.c:275:101: error: expected expression before â,â token
    mosquitto.c:290:54: error: expected expression before â)â token
    mosquitto.c:368:88: error: expected expression before â)â token
    Makefile:15: recipe for target ‘mosquitto.o’ failed
    make[1]: *** [mosquitto.o] Error 1
    make[1]: Leaving directory ‘/home/pi/mosquitto-1.4.2/src’
    Makefile:21: recipe for target ‘mosquitto’ failed
    make: *** [mosquitto] Error 2

    i tried installing 1.4.2 and 1.4.1 with the same errors, but after some help from Michael Karliner and finding something on another site, i reinstalled libwebsockets using the directions at the following link, http://www.xappsoftware.com/wordpress/2015/05/18/six-steps-to-install-mosquitto-1-4-2-with-websockets-on-debian-wheezy/ which uses a verified release version of the library and afterwards all was ok. .

  11. cash

    Hi Mike,

    Thanks for the tutorial.
    1. Last version of libwebsockets doesn’t compile with mosquitto for me, therefore I used an earlier version. This also written on mosquitto web site.
    I suggest to check this gist https://gist.github.com/taka-wang/1c47cde3e4c9c2d83156, this worked for me.
    2. Could you please add instructions on how to define users and passwords and of course how to add SSL into this soup…

    Thanks A Lot
    Cash

  12. cash

    One more thing: What to do to roll back? For example: I installed libwebsockets 1.3 and there is 1.4. How can I recompile everything and build mosquitto with the newest version?

    Thanks

  13. Ryan

    I’m getting a compile error when using setting WITH_WEBSOCKETS:=yes. It seems to continue compiling if I leave this as no. This happens with both 1.4.1 and 1.4.2.

    make[1]: Entering directory ‘/root/mosquitto-1.4.1/src’
    cc -Wall -ggdb -O2 -I. -I.. -I../lib -DVERSION=”\”\”” -DTIMESTAMP=”\”2015-06-23 20:57:49+0200\”” -DWITH_BROKER -DWITH_TLS -DWITH_TLS_PSK -DWITH_UUID -DWITH_BRIDGE -DWITH_PERSISTENCE -DWITH_MEMORY_TRACKING -DWITH_SYS_TREE -DWITH_WEBSOCKETS -DWITH_EC -c mosquitto.c -o mosquitto.o
    mosquitto.c: In function ‘main’:
    mosquitto.c:275:101: error: expected expression before ‘,’ token
    mosquitto.c:290:54: error: expected expression before ‘)’ token
    mosquitto.c:368:88: error: expected expression before ‘)’ token
    Makefile:15: recipe for target ‘mosquitto.o’ failed
    make[1]: *** [mosquitto.o] Error 1
    make[1]: Leaving directory ‘/root/mosquitto-1.4.1/src’
    Makefile:21: recipe for target ‘mosquitto’ failed
    make: *** [mosquitto] Error 2

  14. Baden Morgan

    Hi
    I get the same error as ryan:
    mosquitto.c: In function ‘main’:
    mosquitto.c:275:101: error: expected expression before ‘,’ token
    mosquitto.c:290:54: error: expected expression before ‘)’ token
    mosquitto.c:368:88: error: expected expression before ‘)’ token
    Makefile:15: recipe for target ‘mosquitto.o’ failed
    make[1]: *** [mosquitto.o] Error 1
    make[1]: Leaving directory ‘/home/pi/mosquitto-1.4.1/src’
    Makefile:21: recipe for target ‘mosquitto’ failed
    make: *** [mosquitto] Error 2
    I’ve done a little looking around and people seem to say its the libwebsocket library that is causing problems

    1. Mike Karliner

      As so many people are reporting this problem, I thought I’d
      give this link to the mosquitto.org site which states it concisely
      http://mosquitto.org/2015/05/mosquitto-and-current-unreleased-libwebsockets-branch/

      And to quote….

      The current unreleased libwebsockets master branch defines the VERSION macro in its header files. I believe this to be a bug in libwebsockets.

      This bug causes compilation of mosquitto with websockets support to fail.

      Please use a released version of libwebsockets, either 1.2, 1.3 or 1.4. Mosquitto will compile with all of these versions.

      I do not recommend using an unreleased version of libwebsockets, the project is not shy about making ABI/API incompatible changes between releases so it is impractical to provide support for.

      I’ve checked it myself and using the stable libwebsockets I get mosquitto to compile without problems.

  15. Shre

    Hi Mike,
    I have two questions.
    1. When I install and try to run libwebsocket-test-server, I get a message :
    “libwebsocket init failed” and “Error on binding port to 7681(-1 98)”

    Im not sure whats causing this. Can you help me understand the possible issue.

    2. Its not clear why we need MQTT for web socket, isn’t libwebsocket providing all that we need for implementing a web socket. Don’t take me wrong. It is question out genuine ignorance

    Thanks.

    1. Mike Karliner

      Hi Shre,

      apologies for not answering sooner, WordPress doesn’t seem to be sending
      notifications to me.

      1) I think that this error may because the port has got locked, does, (did)
      the problem go away on a reboot?

      2) The reason for MQTT is that websockets just go from point to point, so they
      would be OK if all you ever wanted to do was talk to one device from your phone,
      and only to your phone from your device. MQTT is multi-point to multi-point, so you
      only evenr need one socket in use at your device, and one websocket on your phone, but many
      phones/laptops/tablets would be able to (potentially) access your device and your
      phones/laptops/tablets can access many devices at the same time.

      1. Shre

        Thanks for the clarifications. I was able to get the libwebsocket up and running with the test programs. However, when I try to embed this library into the code Im using, Im struck. So, Im looking for alternatives for this library to get websocket functionality. Hopefully, I will be able to see my sensor data on the browser soon.

  16. Neil

    I have the same problem compiling as the folks, above. I’m a little confused. Do I need to uninstall websockets and install an older version? Any chance I can get a little hand holding with the location of the old version? Same repo?

    1. Mike Karliner

      First apologies to all for late reply. I’ve not been getting WordPress notifications for some reason.
      I’ve ammended the post to give explicit details of which versions to use.

  17. venkat

    Hi Mike,
    Thanks for this guide.
    The guide seems to abruptly end after the “Part 2” section above. After Installing Node-Red, how do we continue to go an to “completing the tutorial”. Is there a project in the Node-Red link that is meant to be the “continuation of this guide” ?
    Thanks.

    1. Mike Karliner

      We mean to make post on using ThingStudio with node-red.
      It’s not really for us to write node-red tutorials.
      However, once you have got the basics of node-red it is really easy.
      Just send the output of a node-red flow to an MQTT output node,
      and set up the topic as a ThingStudio feed. You can do the same thing
      in reverse for data from ThingStudio to node-red.

  18. daniel

    Thank you for the great article, Mike.

    I have a question now that Mosquitto 1.4.4 is out. Do I still need to compile a version with liwebsockets in order to have that functionality? Or does Mosquitto 1.4.4 have the websockets functionality already compiled with the install?

    Thanks

  19. Cezar Sacomani

    Hello,
    I followed all the steps and am having trouble, you will be able to help me?

    The error is:

    root @ webbaby: / etc / mosquitto # mosquitto -c /etc/mosquitto/mosquitto.conf
    1445001385: mosquitto version 1.4.4 (build date 10/16/2015 08: 28: 17-0300) starting
    1445001385: Config loaded from /etc/mosquitto/mosquitto.conf.
    1445001385: Opening ipv4 listen socket on port 9001.
    1445001385: Opening ipv6 listen socket on port 9001.
    1445001385: Opening websockets listen socket on port 1883.
    1445001389: New connection from 192.168.0.17 on port 9001.
    1445001389: Socket error on client , disconnecting.
    1445001390: New connection from 192.168.0.17 on port 9001.
    1445001390: Socket error on client , disconnecting.

  20. DonL

    I had same issue with the errors in the “cmake ..” Step.
    Fixed it by running “sudo apt-get install libssl-dev”
    This loaded the missing files for OpenSSL. All good now.
    Running latest Jessie download on rpi B.

  21. Pierre

    If anyone is copy-pasting the lines:

    There is an extra space in “sudo apt-get install  uuid-dev” (after install)
    with that you’ll get an error indicating it can’t find uuid-dev.

Leave a Reply

Your email address will not be published. Required fields are marked *