My role
- Took over as the project's sole QA, covering embedded firmware, physical devices, mobile apps, the web admin panel, backend services, and AWS IoT.
- Validated sensor-to-controller BLE traffic, controller-to-backend MQTT messages, and the data exposed to mobile and web clients.
- Tested OTA updates and rollback behavior for both the controller and BLE water sensors.
- Built reusable Python automation for the backend API and for FRD-mapped device functions, then reviewed extensions implemented by two interns.
Technical challenges
- A value shown incorrectly in the mobile app could originate in the sensor payload, BLE characteristic, controller, MQTT message, backend mapping, or WebSocket contract.
- OTA state had to remain consistent across firmware and mobile. One defect allowed an update to be started, the app closed and reopened, and a second update initiated even though firmware already handled the first request.
- Device automation needed to verify more than command responses: tests also read firmware logs, BLE characteristics, MQTT topics, and payloads to confirm the resulting state.
My approach
I built a local Python API suite with pytest and requests: 232 automated tests mapped directly to the FRD. Separately, I designed 4,793 negative backend checks that were executed from the command line with curl and cross-checked in Postman, with the results kept in the test documentation.
For physical-device coverage, I created a Python framework that sent MQTT and BLE commands and determined pass or fail from firmware logs, BLE characteristics, MQTT topics, and payloads. Two interns added more FRD-mapped cases under my review, bringing the suite to roughly 90% of the mapped device requirements. The framework remained available for continued regression testing.
For integration failures, I compared the data at each boundary and worked directly with the relevant developers. A three-amigos session with the mobile and backend developers traced an incorrect water-status display to incompatible data structures across the backend-to-mobile WebSocket path. For the duplicate OTA flow, the mobile app was changed to block another update start while one was already active.
System flow
Simplified device-to-cloud flow. Each arrow was a boundary checked independently - sensor payload, BLE characteristic, MQTT message, backend mapping, WebSocket contract - after a mobile-app display bug turned out to originate several hops upstream.
Results
- The 232-test API suite became a reusable pre-release regression check. It runs in roughly 40-50 minutes, replacing work that would take days to repeat manually.
- The 4,793 negative backend checks exposed validation, schema, and functional gaps across the backend.
- The physical-device framework automated roughly 90% of the FRD-mapped cases and could be rerun whenever new firmware arrived. Repeating the same requirements manually would take at least a full day.
- The incorrect water-status display and duplicate OTA-update flow were fixed and successfully retested. The completed platform was released.
Stack
More case studies