LiteML-Edge: A Contract-Centered Framework and Methodology for Replay-Driven 1:1 Python--Firmware Validation in TinyML
Keywords:
Contract-Based Validation, Python--Firmware Validation, Reproducibility, TinyMLAbstract
Tiny Machine Learning (TinyML) enables datadriven inference on resource-constrained microcontrollers; however, offline evaluation and firmware execution are often treated as loosely coupled stages, leaving reproducibility gaps between reference experiments and on-device behavior. This paper presents LiteML-Edge, a contract-centered TinyML framework and methodology for replay-driven 1:1 Python–Firmware Validation. LiteML-Edge integrates training, optimization, quantization, artifact export, firmware deployment, and per-stage validation while exporting replay artifacts, preprocessing rules, tensor I/O metadata, output mapping, post-processing, and the Rolling-24 metric policy as a shared validation contract. The framework supports fully quantized, fully floating-point, and hybrid tensor interfaces within a single workflow. The evaluation covers one environmental sensing scenario, a fixed W = 24 policy, and the TensorFlow-to-TFLM deployment path. On a WEMOS LOLIN32 (ESP32) and a NUCLEO-F411RE (ARM Cortex-M4), case studies based on a multilayer perceptron (MLP), a long shortterm memory network (LSTM), and a compact one-dimensional convolutional model (Conv1D Tiny) showed how the framework localizes agreement or mismatch across preprocessing inputs, model payloads, raw outputs, final predictions, and Rolling-24 metrics. Under replay on both targets, MLP and Conv1D Tiny achieved contract agreement across the critical path, whereas LSTM preserved input-stage agreement but showed its first reproducible divergence at the raw-output stage. Field execution is retained as operational evidence collected under the same contract, complementing replay without replacing it for equivalence claims. The resulting workflow makes validation failures more attributable, reproducible, and auditable before deployment decisions are made, while distinguishing contract conformance across platforms from hardware-dependent deployment costs.
Downloads
References
F. M. Aymone and D. P. Pau, “Benchmarking In-Sensor Machine Learning Computing: An Extension to the MLCommons-Tiny Suite,” Information, vol. 15, no. 11, Art. no. 674, 2024, doi: 10.3390/info15110674.
A. Elsts and R. McConville, “Are Microcontrollers Ready for Deep Learning-Based Human Activity Recognition?,” Electronics, vol. 10, no. 21, Art. no. 2640, 2021, doi: 10.3390/electronics10212640.
R. Immonen, T. Hämäläinen, and K. E. J. Leppänen, “Tiny Machine Learning for Resource-Constrained Microcontrollers,” Journal of Sensors, vol. 2022, Art. no. 7437023, 2022, doi: 10.1155/2022/7437023.
N. Schizas, A. Karras, C. Karras, and S. Sioutas, “TinyML for Ultra-Low Power AI and Large Scale IoT Deployments: A Systematic Review,” Future Internet, vol. 14, no. 12, Art. no. 363, 2022, doi: 10.3390/fi14120363.
P. van Kempen, R. Stahl, D. Mueller-Gritschneder, and U. Schlichtmann, “MLonMCU: TinyML Benchmarking with Fast Retargeting,” in Proc. 2023 Workshop on Compilers, Deployment, and Tooling for Edge AI (CODAI ’23), 2023, doi: 10.1145/3615338.3618128.
V.-E. Baciu, J. Stiens, and B. da Silva, “MLino bench: A Comprehensive Benchmarking Tool for Evaluating ML Models on Edge Devices,” Journal of Systems Architecture, vol. 155, Art. no. 103262, 2024, doi: 10.1016/j.sysarc.2024.103262.
P.-E. Novac, G. Boukli Hacene, A. Pegatoquet, B. Miramond, and V. Gripon, “Quantization and Deployment of Deep Neural Networks on Microcontrollers,” Sensors, vol. 21, no. 9, Art. no. 2984, 2021, doi: 10.3390/s21092984.
M. de Prado, M. Rusci, A. Capotondi, R. Donze, L. Benini, and N. Pazos, “Robustifying the Deployment of tinyML Models for Autonomous Mini-Vehicles,” Sensors, vol. 21, no. 4, Art. no. 1339, 2021, doi: 10.3390/s21041339.
C. M. Bhushan et al., “Deploying TinyML for Energy-Efficient Object Detection and Communication in Low-Power Edge AI Systems,” Scientific Reports, vol. 15, Art. no. 44299, 2025, doi: 10.1038/s41598-025-27818-9.
B. Taylor, V. Sanz Marco, W. Wolff, Y. Elkhatib, and Z. Wang, “Adaptive Deep Learning Model Selection on Embedded Systems,” in Proc. 19th ACM SIGPLAN/SIGBED Int. Conf. on Languages, Compilers, and Tools for Embedded Systems (LCTES ’18), New York, NY, USA: ACM, 2018, pp. 31–43, doi: 10.1145/3211332.3211336.
H.-Y. Lin, “Embedded Artificial Intelligence: Intelligence on Devices,” Computer, vol. 56, no. 9, pp. 90–93, 2023, doi: 10.1109/MC.2023.3280397.
H. Ren, D. Anicic, X. Li, and T. Runkler, “On-device Online Learning and Semantic Management of TinyML Systems,” ACM Transactions on Embedded Computing Systems, vol. 23, no. 4, Art. no. 55, pp. 1–32, 2024, doi: 10.1145/3665278.
M. Gulati, K. Zandberg, Z. Huang, G. Wunder, C. Adjih, and E. Baccelli, “TDMiL: Tiny Distributed Machine Learning for Microcontroller-Based Interconnected Devices,” IEEE Access, vol. 12, pp. 167810–167826, 2024, doi: 10.1109/ACCESS.2024.3492921.
E. Manor and S. Greenberg, “Custom Hardware Inference Accelerator for TensorFlow Lite for Microcontrollers,” IEEE Access, vol. 10, pp. 73484–73493, 2022, doi: 10.1109/ACCESS.2022.3189776.
Z. Huang, K. Zandberg, K. Schleiser, and E. Baccelli, “U-TOE: Universal TinyML On-Board Evaluation Toolkit for Low-Power IoT,” in Proc. 2023 12th IFIP/IEEE Int. Conf. on Performance Evaluation and Modeling in Wired and Wireless Networks (PEMWN), 2023, doi: 10.23919/PEMWN58813.2023.10304946.
G. Delnevo, S. Mirri, C. Prandi, and P. Manzoni, “An Evaluation Methodology to Determine the Actual Limitations of a TinyML-Based Solution,” Internet of Things, vol. 22, Art. no. 100729, 2023, doi: 10.1016/j.iot.2023.100729.
M. Ersoy and U. Şansal, “Analyze Performance of Embedded Systems with Machine Learning Algorithms,” in Trends in Data Engineering Methods for Intelligent Systems, J. Hemanth, T. Yigit, B. Patrut, and A. Angelopoulou, Eds., vol. 76, pp. 231–236, 2021, doi: 10.1007/978-3-030-79357-9_23.
M. A. Hasanpour, M. Kirkegaard, and X. Fafoutis, “EdgeMark: An Automation and Benchmarking System for Embedded Artificial Intelligence Tools,” Journal of Systems Architecture, vol. 167, Art. no. 103488, 2025, doi: 10.1016/j.sysarc.2025.103488.
O. Rainio, J. Teuho, and R. Klén, “Evaluation Metrics and Statistical Tests for Machine Learning,” Scientific Reports, vol. 14, Art. no. 6086, 2024, doi: 10.1038/s41598-024-56706-x.
J. Kwon and D. Park, “MAIL: Micro-Accelerator-in-the-Loop Framework for MCU Integrated Accelerator Peripheral Fast Prototyping,” Applied Sciences, vol. 15, no. 3, Art. no. 1056, 2025, doi: 10.3390/app15031056.
P. Nandi, K. R. Anupama, A. Bajaj, S. Shukla, T. Musale, and S. Kachadiya, “Performance Evaluation of Machine Learning Algorithms on System on Chips in Wearables for Healthcare Monitoring,” Procedia Computer Science, vol. 218, pp. 2755–2766, 2023, doi: 10.1016/j.procs.2023.01.247.
Z. Huang, K. Zandberg, K. Schleiser, et al., “RIOT-ML: Toolkit for Over-the-Air Secure Updates and Performance Evaluation of TinyML Models,” Annals of Telecommunications, vol. 80, pp. 283–297, 2025, doi: 10.1007/s12243-024-01041-5.
S. Heydari and Q. H. Mahmoud, “Tiny Machine Learning and On-Device Inference: A Survey of Applications, Challenges, and Future Directions,” Sensors, vol. 25, no. 10, Art. no. 3191, 2025, doi: 10.3390/s25103191.
J. Pan, J. Tang, M. Caniza, J.-M. Héraud, E. Koay, H. K. Lee, et al., “Correlating Indoor and Outdoor Temperature and Humidity in a Sample of Buildings in Tropical Climates,” Indoor Air, vol. 31, pp. 2281–2295, 2021, doi: 10.1111/ina.12876.
