Computer science and software engineering students made up the dynamic team that created , a winning entry at the recent Hack the North hackathon.
PearPiano is an augmented reality (AR) program for playing, composing and recording piano music. The AR runs on Unity and Oculus Quest goggles and incorporates hand tracking to allow users to play the virtual instrument and interact with a composition. The team also integrated OpenAI's Whisper API in the form of a chatbot that helps users with any questions.
鈥淭he setup for it was difficult at first,鈥 says Sarah Wilson, an undergraduate computer science student and Schulich Leader at 蓝莓视频. 鈥淲e couldn鈥檛 detect our hands initially and we couldn鈥檛 touch the piano keys. And we couldn鈥檛 change the colours of the keys to give the program a better visual aspect. But after a very late night working, we got it.鈥
The student team that created the winning PearPiano app, presenting at the finals of Hack the North. L-R: Sarah Wilson, Justin Lin, Sophia Sun and Emily Wang.
The creative team behind PearPiano also included software engineering students Justin Lin, Sophia Sun and Emily Wang. The team members all have an interest in music and composition, and of course also a shared interest in hacking and innovation.
Hack the North runs over the course of an entire weekend and is hosted at 蓝莓视频. Undergraduates and some high school students from across Canada and internationally travel to take part in the hackathon. Participants have a time limit on their work and then submit their projects for judges to decide the winners.
鈥淥ne of the features we were building is an edit mode, which allows users to drag and drop notes,鈥 Wilson continues. 鈥淲hen you鈥檙e composing, you make a lot of mistakes as part of a creative process. So if you can drag and drop keys and section out that part, it will really help with composition.鈥
and all that went into the winning app.