¶ Augmented Reality Mural
WINLAB Summer Internship 2023
Group Members: Alexander Clarke, Ana Obradovic, Arya Shetty, Nicholas Clarke, Pavan datta Vedire Reddy
¶ Project Objective
The objective of this project is to create a collaborative augmented reality application which will allow remote users to collaboratively create a virtual mural. This project will be created using the Unity game engine and deployed to AR platforms such as the Microsoft Hololens.
In addition to creating the software that will manage the generation and rendering of the mural, another important component of this project is designing an infrastructure that will allow for very low latency collaboration. If time allows, one of the end goals of this project is to deploy it using an "edge cloud" architecture using the OpenAR Cloud project.
Students should get started with this project by familiarizing themselves with the Unity game engine and Hololens development tools.
¶ Weekly Progress
Week 1:
Presentation Link: https://docs.google.com/presentation/d/1FjPUL7XrPDCdE972nUbW7S74IX70BCDgDVR_eASt9H0/edit?usp=sharing
- Started to familiarize ourselves with the Unity Software and Hololens.
- We started with connecting the Hololens to the Unity app through remote connection.
- Once we were able to see the project scene through the Hololens we tested limits of the environment
Week 2:
Presentation Link: https://docs.google.com/presentation/d/1LQVxsd-889PpTrCxg7lbWw3VhyYNj-niy3aS0FdTTnw/edit?usp=sharing
- Set up essential MRTK (mixed reality toolkit) packages in Unity
- Downloaded and deployed app on Hololens
- Moved and resized objects
- Used eye tracking feature for selection
- Used voice commands to switch color of objects
- Added template menu
- Wrote code for drawing application using mouse
- Worked on iterations of line drawings (2D & 3D)
Week 5:
Presentation Link: https://docs.google.com/presentation/d/1w9NXW46-_z1c2cesLaWOiW27Q5gIE-tJYjH5PGD_khY/edit?usp=sharing
- More work with MQTT, setting up unity client
- More work on save system, and transferring game object files to JSON format
Week 6:
Presentation Link: https://docs.google.com/presentation/d/1QC8dDQiOfP1Gx0GNxpk489A0xPRhpyjq8akN6ApKZss/edit?usp=sharing
- Used MQTT to send JSON files from different devices
- Implemented saving lines by position change
- Fixed UI and optimized brushes
- Set up Node-Red to manage MQTT topics
Week 7:
Presentation Link: https://docs.google.com/presentation/d/1I73XR4envKSiZ2wyW71QvwwqoPtDomBs0DI1XrCHSLs/edit?usp=sharing
- Two player collaboration
- File Server
- Allow for multiple users to draw incrementally
- New menu flow and shape tools
Week 8:
Presentation Link: https://docs.google.com/presentation/d/1zA52CP9krTh-wP0SNyQAoHZ3nT4O-MO8huG1WbuqBug/edit?usp=sharing
- Fixed bugs to reduce lag
- Allowed for multiple users to draw simultaneously
- Mesh, shape, and drawing functionality refined
- Applied save/load features to different textures
Week 9:
Presentation Link: https://docs.google.com/presentation/d/1yg9vJt0DxlAo2eEjRVaes9K5XTRtt0YS3PNbwkqtp7c/edit?usp=sharing
- Fixed more bugs
- Deployed working single player app on hololens
- Put broker on a linux VM hosted by orbit nodes
- Created presentation and poster for final week
Week 10:
Presentation Link: https://docs.google.com/presentation/d/1yg9vJt0DxlAo2eEjRVaes9K5XTRtt0YS3PNbwkqtp7c/edit?usp=sharing
- https://youtu.be/V-7BNvZp4og
- Created demo video
- Final app testing
- Commenting code for next year
