Between 2022 and 2023, I was part of an Erasmus+ research project at AFN Norte Porto. The goal: build a VR training system that helps people overcome public speaking anxiety by providing real-time feedback on their performance.
It was the most interdisciplinary project I have ever worked on. And it taught me things I still use every day.
The Core Idea
The system puts a user in a virtual room with an audience. As they speak, computer vision algorithms analyze their posture, gestures, and movement. If they pace too much, the system nudges them. If they freeze, it encourages them. If they avoid eye contact, it adapts the scenario.
The technical challenge was not any single component. It was getting all the components to work together in real time.
The Architecture
We built:
- A Unity VR application for the training environment
- A C# / .NET backend for session management and data persistence
- An Angular web dashboard for administrators to configure scenarios
- PostgreSQL for storing session data and progress tracking
- Computer vision pipelines for analyzing user behavior
The VR app sent telemetry data to the backend during sessions. The CV models processed video frames and returned feedback parameters. The backend coordinated everything and stored results for longitudinal tracking.
What Surprised Me About VR Development
Motion sickness is a design constraint, not a bug. Every interaction, every camera movement, every UI element had to be evaluated for its impact on user comfort. This forced a level of design discipline that I rarely see in web or mobile development.
Performance margins are razor-thin. In VR, you have about 11 milliseconds per frame at 90 FPS. Exceed that and the user feels sick. This means every asset, every script, every shader is a potential problem. Optimization is not optional.
The Research Publication
We published the results in Procedia Computer Science in 2024 (doi: 10.1016/j.procs.2024.06.416). The paper covers the system architecture and initial findings on its effectiveness for phobia mitigation.
What I did not expect was how different academic writing is from technical documentation. In engineering, you explain how something works. In research, you have to prove that it works, with data, methodology, and statistical significance. The rigor required was eye-opening.
The Takeaway
This project reinforced something I already suspected: the hardest problems are not technical. They are understanding the user, their fears, their context, and what actually helps them. The VR scenarios and the CV algorithms were just tools.
Building software that changes how someone feels, not just what they do, is a completely different category of engineering. And it is much more rewarding.