As a third party in the EU Horizon TrialsNet project, SmartRDI designed and built a mobile Augmented Reality system for outdoor Point-of-Interest enrichment, leveraging 5G/B5G infrastructure, federated learning on edge nodes, and ARCore to overlay real-time contextual information on landmarks as users pass them. In the final trial (10–21 March 2025, 70 users on the POLITEHNICA Bucharest campus), 6 of 7 KPIs and all 8 KVIs were validated.
The Goal
TrialsNet's mission is to demonstrate 5G and Beyond-5G capabilities at scale in real trial environments. SmartRDI's use case was specifically chosen to stress-test both the network's throughput and latency, and the AI pipeline's ability to run privacy-preserving federated learning across edge nodes.
The app identifies outdoor landmarks in real time, such as monuments and building facades, and overlays rich information and mixed-reality objects without requiring the phone to connect to a remote cloud for inference.
Why It's Hard
Combining augmented reality (which demands sub-20ms latency), federated learning (which requires continuous model sync without centralising raw images), and edge computing (which requires dynamic workload routing) into a single Android application is a significant systems integration challenge.
Our solution routes each inference request through a contextual decision engine, determining whether to run locally on-device, offload to the nearest 5G edge node, or defer to the aggregated global model, invisible to the end user. Federated learning is coordinated with Flower, while on-device training runs through LiteRT, so models improve without raw images ever leaving the phone.
70
Users · 10 sessions · 2 weeks (10–21 March 2025)
19 ms
RTT latency validated (indoor 5G Lab)
99.2%
AI accuracy · 94.7% F1 score achieved
526 Mbps
Peak downlink · 600 Mbps best lab run
How the system works
A YOLOv11s model in TFLite runs continuously on the device camera feed. The user can toggle between detection and segmentation modes, with location and ARCore scene semantics captured alongside each frame.
Recognition runs entirely on-device via TFLite. Users give thumbs up/down feedback on each result; negative feedback triggers local storage of the image and its metadata for later training.
Training runs in the background, only while the device is idle and charging, via LiteRT. Only weight updates leave the phone, never raw images, transmitted to the server through Flower.
A Flower server on an HPE ProLiant DL360 Gen10 (8× Xeon CPUs, 32 GB RAM), reachable over VPN, aggregates the updates. Payloads are a compact 10–30 MB and transmit within seconds, even outdoors.
The refreshed global model is pushed back for ARCore to detect and annotate PoIs. Inference can also be offloaded to the edge: a Galaxy S23 runs detection in 121 ms on-device versus 5 ms on the edge GPU, valuable for low-end devices.
Users earn incentives for contributing quality data (new PoI captures, environment scans). This drives continuous model improvement while keeping users engaged with the city around them.
Trial KPI Results
Each target measured against the validated trial result. The single gap, uplink throughput, came from outdoor 5G coverage at the POLITEHNICA campus, not a system failure.
Downlink Throughput
526 Mbps
✓ Lab validated
Target ≥200 Mbps · 526 Mbps (lab) / ~102 Mbps (outdoor)
Uplink Throughput
20.2 Mbps
✗ Not validated
Target ≥100 Mbps · upload limited by outdoor 5G coverage at the POLITEHNICA campus, not a system failure.
App Round-Trip Latency
19 ms
✓ Validated
Target <20 ms
AI/ML Accuracy
99.2%
✓ Validated
Target ≥80%
AI/ML Precision (mAP75)
90%
✓ Validated
Target ≥80%
Recall
100%
✓ Validated
Target ≥70%
F1 Score
94.7%
✓ Validated
Target ≥75%
Per-PoI Results
All (aggregate)
99.2%
Accuracy · Recall 100% · mAP50 99.5% · mAP50-95 80.0%
Aula Magna
98.6%
Accuracy · Recall 100% · mAP50 99.5% · mAP50-95 88.8%
Time Column
99.4%
Accuracy · Recall 100% · mAP50 99.5% · mAP50-95 67.8%
Rectorate
99.5%
Accuracy · Recall 100% · mAP50 99.5% · mAP50-95 83.4%
Training used AdamW, batch 16, an 80/10/10 split, and YOLOv11s in FP16 converted to TFLite. Labels were crowdsourced via Timeworx.io: 1,000+ contributors, 15,000+ tasks, 5 annotators per task with IoU consensus.
Trial Site
POLITEHNICA Bucharest · 10–21 March 2025 · 70 users
Controlled campus environment with rich architectural diversity: faculty buildings, monuments, outdoor installations. The final trial ran 70 users across 10 sessions over two weeks, calibrating the federated model in a geofenced area with known ground-truth PoIs.
Project Milestones
Trial plan, resource allocation, and partner integration agreements finalised.
System architecture, API contracts, and ML model baseline established.
Working Android app with local FL model, ARCore overlay, and 5G edge offloading integration.
Controlled lab environment KPI measurements, model performance analysis, and UX evaluation.
System validated against the KPIs in a controlled 5G testbed environment with simulated user load.
70 users across 10 sessions (10–21 March 2025) on the POLITEHNICA Bucharest campus, using Orange Romania 5G SA/NSA infrastructure. 6 of 7 KPIs and all 8 KVIs validated. Final report and presentation delivered to the consortium.
What's Next
Participants asked for an AR map of nearby points of interest, route guidance and discovery challenges. Full market readiness is estimated at about two years.
The route to market is partnerships with municipal authorities, cultural institutions and tourism boards, rather than selling directly to end users.
Technology Stack
Value Dimensions
Societal: Education, cultural accessibility, community engagement with local heritage, social mobility through technology democratisation.
Environmental: Optimised resource usage through on-device/edge compute, avoiding unnecessary cloud round trips.
Economic: 80% of participants said they would use the app again, and 71.5% would keep using it with a paid subscription. Trial insights point to partnerships with institutions, rather than end-user fees, as the stronger route to market.
KVI Results
Ease of use
92.8%
found it easy to interact with PoIs
AR navigation comfort
90%
comfortable with the AR features
Cultural connection
94.3%
felt it enhanced appreciation of local heritage
Cultural resonance
92.9%
said content connected to their cultural background
Edutainment
88.5%
felt it was an enriching learning experience
Societal sustainability
90%
believe it makes cultural knowledge more accessible
Digital inclusion
97.2%
consider it accessible to a broad range of devices
System stability
95.7%
felt features were stable and responsive
Overall experience
91.4%
positive overall rating
Return intent
80%
would use it again if available
All 8 KVIs were validated against the ≥75% target. The weakest signal was the business model: 71.5% would continue with a paid model, but only 48.5% saw it as clearly sustainable long-term.
Deviations & Achievements
Aula Magna was added to the Rectorate and Time Column, expanding the dataset and training scope.
A second YOLOv11s model, beyond the planned detection-only scope, improved the AR experience and drew positive user feedback.
The trial scaled up, leading to refined UI suggestions including AR maps, crowdsourced PoI descriptions, and gamification features.
Initially unplanned, it confirmed feasibility for low-end devices (5 ms on the edge vs 121 ms on-device).
Vilnius municipality expressed interest; scope expanded to smart tourism and digital heritage beyond campuses.
EU Funding
TrialsNet is funded under the Horizon Europe Smart Networks and Services Joint Undertaking (JU-SNS-2022), Grant Agreement No. 101095871. SmartRDI participated as a third party, contributing the mobile AR use case and trial execution in Bucharest.
Funded by the European Union. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the granting authority. Neither the European Union nor the granting authority can be held responsible for them.
Public Engagement
15
Dissemination events (workshops, demos, showcases)
1,000+
Students reached via live university lecture demos
44,200+
Social media followers across X, LinkedIn, Facebook
28,600+
Social media impressions
Demos were integrated into 8 UPB courses (Parallel & Distributed Algorithms, Cloud Computing, Applied Informatics I/IV, Modern Distributed Systems, and others), with presentations at MobyLab and Bucharest GDG. Coverage spanned the Timeworx.io blog (42,000+ followers), a SmartRDI YouTube demo, and a Medium article.