Between 2022 and 2023, I was part of an Erasmus+ research project at AFN Norte. The goal: build a VR training system that helps people overcome public speaking anxiety by providing real-time feedback on their performance.
It was the most interdisciplinary project I have ever worked on. And it taught me things I still use every day.
The Core Idea
The system puts a user in a virtual room with an audience. As they speak, computer vision algorithms analyze their posture, gestures, and movement. If they pace too much, the system nudges them. If they freeze, it encourages them. If they avoid eye contact, it adapts the scenario.
The technical challenge was not any single component. It was getting all the components to work together in real time.
The Architecture
We built:
- A Unity VR application for the training environment
- A C# / .NET backend for session management and data persistence
- An Angular web dashboard for administrators to configure scenarios
- PostgreSQL for storing session data and progress tracking
- Computer vision pipelines for analyzing user behavior
The VR app sent telemetry data to the backend during sessions. The CV models processed video frames and returned feedback parameters. The backend coordinated everything and stored results for longitudinal tracking.
What Surprised Me About VR Development
Motion sickness is a design constraint, not a bug. Every interaction, every camera movement, every UI element had to be evaluated for its impact on user comfort. This forced a level of design discipline that I rarely see in web or mobile development.
Performance margins are razor-thin. In VR, you have about 11 milliseconds per frame at 90 FPS. Exceed that and the user feels sick. This means every asset, every script, every shader is a potential problem. Optimization is not optional.
User testing is humbling. We tested with real people who had public speaking anxiety. Some could not complete the first session. The system was too intense. We had to redesign the onboarding experience three times before it worked for most users. You cannot predict how people will react to VR. You have to test and iterate.
The Research Publication
We published the results in Procedia Computer Science in 2024 (doi: 10.1016/j.procs.2024.06.416). The paper covers the system architecture and initial findings on its effectiveness for phobia mitigation.
What I did not expect was how different academic writing is from technical documentation. In engineering, you explain how something works. In research, you have to prove that it works, with data, methodology, and statistical significance. The rigor required was eye-opening.
The Takeaway
This project reinforced something I already suspected: the hardest problems are not technical. They are understanding the user, their fears, their context, and what actually helps them. The VR headset and the CV algorithms were just tools.
Building software that changes how someone feels, not just what they do, is a completely different category of engineering. And it is much more rewarding.