Advaith Narayanan, Yufan Wang, and Omeed Sabouri
USA
-
Completed in
2024
Finally, for our 3rd-place (Bronze) winner, we have “Pantheon Meets,” by highschoolers Advaith Narayanan, Yufan Wang, and Omeed Sabouri.
Inspired by Narayanan’s grandfather who is hearing-impaired, our 3rd-place-winning trio decided to create Pantheon Meets. Their project seeks to overcome the brutal communication barriers that the hearing-impaired population face. Opting to create an online meeting platform, Team Pantheon Meets focuses extensively on fostering not only accessibility but true community.
Their niche approach to addressing the too-often underserved disabled community is something the EQ Hacks judges are proud to showcase.
An idea that meets the soul of sign language.
Pantheon Meets is an online meeting platform designed to facilitate real-time connections through a web-based service. Similar to platforms like Zoom and WebEx, Pantheon Meets offers a range of features that enhance virtual interactions, including the ability to toggle cameras and microphones, share screens, and utilize a public whiteboard for collaborative purposes. Most importantly though, our judges were most impressed by the attention-to-detail in curating extra features for hearing-impaired or deaf users.
Efficient Closed Captioning : The Pantheon Meets team was able to implement an accurate and efficient automatic captioning system, which converts spoken words into text. This feature was built through Javascript’s built-in speech-to-text translator and routed the data to the captions box.This ensures that all participants, regardless of hearing ability, can follow the conversation easily. Even more innovative, users who are deaf or hard of hearing can actively engage in meetings by typing messages into a chat box that integrates with the existing captioning system. As such, their nonverbal contributions would be seamlessly included in meeting dialogue.
American Sign Language (ASL) Detection: Pantheon Meets also incorporates advanced technology to recognize and transcribe American Sign Language (ASL)—which is commonly used by the hearing-impaired community as a form of communication . More specifically, the Pantheon Meets team created a classifier that analyzes hand gestures and converts them into text that is included in the aforementioned captioning system. This is done through the display of the user’s hand nodes and linking the detector with the video feed system on the platform.
Video Conferencing: The bulk of the meeting platform was built using a backend video-meeting provider. The Pantheon Meets team utilized AgoraRTC—allowing them to send all video data necessary for ASL classification through their hosted servers. Additionally, common conferencing features such as screen sharing was implemented through the WebRTC tool. Furthermore, the team was able to purchase their own domain and assign each individual meeting its own personal meeting ID through their platform.
Did the team meet its match? Sorry for the puns.
The Pantheon Meets project incorporated an array of high-level techniques and features as part of their fruition. Given this limited amount of time,the trio had to split their project into multiple moving parts. Sabouri was in charge of the front-end development and captioning, while Naryanan was working on the video streaming system, as well as the backend. Simultaneously, Wang innovated the AI-powered phrase classification system.
Throughout the course of the competition, a main issue the Pantheon Meets team faced was the compatibility and performance of website hosting platforms. Because Cloudflare was not working, the team had to transition to Vercel in the middle of the competition—requiring extra time and effort to set up a new environment. Moreover, the creation of Pantheon Meets American Sign Language (ASL) phrase classifier required “thousands of lines of Python code” for data generation and model training. In fact, the team decided to challenge themselves and develop their own phrasal epenthesis algorithm—relying on machine learning to increase the accuracy of the ASL transcriber.
And, although website and classifier development required significant effort, the team also had the challenge of linking the interface of their website with these intricate, backend features.
Nonetheless, despite the challenge of compartmentalization, the Pantheon Meets team combined all technical features to form a functional video conference platform. Specifically, we appreciated the versatility shown by the team, as they shifted platforms to Vercel when Cloudfare was not functioning well.
More importantly, our judges were also completely awestruck by the development of the ASL phrase classifier. This accomplishment highlighted the team's technical prowess and represented EQ Hacks’ core tenet of innovation—not merely imitation. Creating a phrasal recognition algorithm different from the one used in the status quo is something the judges have not seen before!
Ultimately, we are extremely proud that, through their social good project, the Pantheon Meets team was able to build practical knowledge about website hosting, client/socket/API connections, and even algorithmic development. These niche considerations involved in web development are a sureproof way to ensure tech team members can tackle similar, large-scale projects in the future!
And for our future competitors considering a larger project similar to Pantheon Meets, learn from Sabouri’s words:
“We learned that we should have a more organized approach to implementing [our multitude] of features, as a lot of work has to be done [for the phrase detection to properly work embedded in the website]”
When planning an elaborate project, for the sake of cohesion, we encourage participants to create a detailed and organized plan for implementation. Overall, if there is any conclusion to be drawn about this project and trio, it’s that their platform truly MEETS this moment and strengthens the ability of the disabled community to lead more accessible lives.
So what happens next time we meet PantheonMeets? Again, sorry for the puns.
The next time we MEET the Pantheon Meets team hopes to collaborate with their educational institutions to utilize their platform for ASL community members. Beyond that, the team seeks to rigorously improve their product to eventually market as a startup company. Nonetheless, at the end, a product inspired by the community, returns back to the same community.
Likewise, as Sabouri suggests, “If you want a problem to solve, I recommend looking in your own community. We came up with the idea of an ASL meeting platform through the idea of one of our team members who had a hearing-impaired grandfather. “
Community is after all, the only rock-solid thing we have. In terms of technological development, it's the main reason why we at EQ Hacks are excited to see more future socially beneficial projects!
Check out the Pantheon Meets team's work here:
If you liked this article and enjoyed Advaith Narayanan, Yufan Wang, and Omeed Sabouri's project, feel free to leave a comment of support on their DevPost page.
If you want the chance to get your own featured article, join our ongoing competitions.
