Gathering Data

Real-Time Information

A Raspberry Pi 1 was connected alongside PMS5003 and BME280 sensors to gather air quality, temperature, humidity, and pressure information from the environment around the UC Riverside area in Riverside, CA.

The data you see displayed in the homepage is real-time and gathered 10 seconds ago, at most.

Crash-redundant Systems Programming

Crash recovery

The Raspberry Pi runs a Raspberry Pi OS Lite to keep a lower CPU and memory load, and several scripts and systemd services were created to recover from crashes. All of this results in a 24-7 data-gathering effort that can be implemented anywhere.

Crashes and systemctl service stops causes an immediate Python script restart, and the data gathering process resumes. The Raspberry Pi is connected to a Uninterruptible Power Supply (UPS) to prevent loss of power.

Lack of connection errors are handled through a robust Python, Exception-based programming. Exceptions are carefully placed so that samples remain in-memory throughout a lack of connection error.

DynamoDB Unproccessed Data

The AWS Lambda function that uploads data to DynamoDB has built-in redundancy handled through an exponential "back off" method to upload data to DynamoDB in 25-sized BatchWriteCommand PUT requests. If any failure occurs in the upload, my lambda function retries the upload 5 times, waiting in exponential intervals (100ms, 200ms, 400ms, 800ms, ...). This ensures greatly improved chances that the upload request is successful.

Data Format

The BME280 sensor gathers temperature, humidity, and pressure data, once per sample

BME280 data format:

• temperature: float
• humidity: float
• pressure: float

PMS5003 data format:

• count_03: int
• count_05: int
• count_10: int
• count_25: int
• count_50: int
• PM1.0 factory: int
• PM2.5 factory: int
• PM10.0 factory: int
• PM1.0 outside: int
• PM2.5 outside: int
• PM10.0 outside: int

Note: "factory" vs "outside" refers to the sensor calibration. Factory calibration is apt for indoor use, and "outside" calibration is for outdoor environments. This Raspberry Pi sits indoors, so the "factory" measurement is more accurate.