Section outline

  • Analyzing IoT air quality data streams using Kafka and Jupyter Notebooks

    In this tutorial you will learn how to use Kafka and Jupyter notebooks to process and analyze streams of sensor data (particulate matter, PM2.5).

    After you have completed this tutorial, you will know how to

    • use Docker to install and run Apache Kafka and Jupyter Notebooks on your local computer

    • use Python to access and download PM2.5 sensor data from the Open Sensemap project

    • simulate a PM2.5 sensor data stream that runs against Kafka and how to analyze that data stream for monitoring air quality

    The module is structured as follows:
    1. Overview
    2. Background on IoT, sensor data streams and the air quality parameter particulate matter (PM2.5)
    3. Installing and using Apache Kafka and Jupyter Notebooks for analyzing PM2.5 data streams
    4. Wrap up

    The OER is designed for students and professionals who want to improve their skills in developing applications for near-real-time data. Users should have some basic knowledge of Python and it would not be bad to have some experience with Docker and Jupyter notebooks as well. However, the OER guides you through all these technologies and can also be used to gain some initial practical experience with them.

    How to use the OER Module

    Simply download the PDF file, read and follow the tutorial..

    License Statement

    You are free to use, alter and share the tutorial under the terms of the CC-BY-SA 4.0 license, unless explicitly stated otherwise for specific parts of the content. The tutorial can be referenced as follows: “OER-SpatialDataStreaming”, OER4SDI project /  University Münster, CC BY-SA 4.0.

    All logos used are generally excluded.

    Any code provided with the tutorial can be used under the terms of the MIT license. Please see the full license terms.

    Authors and funding

    The tutorial has been developed at the Institute for Geoinformatics, University of Münster. Authors are Jaskaran Puri and Albert Remke, with contributions from Sandhya Rajendran and Thomas Kujawa. The latest version of this tutorial is always available on GitHub. We hope you will use GitHub issues to provide feedback and suggest improvements.

    The OER4SDI project has been recommended by the Digital University NRW and is funded by the Ministry of Culture and Science NRW.