Breaking the Sound Barriers - Lighting Up the Future of Alert Systems

Digital Solutions have changed our lives for good. Be it the travel apps that let us travel from one place to another or the instant messaging and video-calling applications through which you can reach out to your friends living across seven seas. Can you imagine even a day without such applications? Well for some of us, it has been years waiting, to fully use these technologies on their own. All because of a gap in the designing of such applications - the accessibility gap.

Before we start, I have a small task for you. Look around you. Pick anything. Now try to analyze that product from the perspective of any special-needs individual (blind, deaf, impaired motor, dumb, dyslexic). Can they use this thing the same way as you can? in 90% of the cases, they won’t be able to. This difference arises due to a lack of inclusivity and accessibility while designing such solutions.

Let’s talk about one such solution that most of us rely on for our day-to-day work - our notifications system. What is common about an incoming call on your phone, someone ringing your doorbell, and a new message alert on your smartphone? A sound is generated to alert you about the same. The “ting-tong” that grabs your attention. However such interactions are not inclusive, keeping out the deaf and hard-of-hearing individuals. To cover this gap, I present to you Alertify - a smart alert system that transforms your smart home lighting into a visual alert system. Now, imagine your room lights turning blue to alert you about your ringing doorbell. Sounds interesting right? Read ahead as we transform this idea into a reality.

What is the problem?

A majority of the alerting system relies on sound-based communications to alert the users. Be it a ringing phone or the buzzing bell of your home. This poses a huge problem for deaf and hard-of-hearing individuals who have impaired auditory senses. Due to this, such individuals are dependent on others around them to alert them. To better understand the problem, imagine you are a deaf person living alone and expecting someone to come to your home. As of now, you need to wait by standing at the door beforehand since you can’t hear the ringing bell. While a notification alert from your favorite social media app can wait, you can’t afford to miss an alert about an incoming cyclone/flood from your local government. Not only this, haven’t we come across our grandparents finding it hard to hear incoming phone calls? This is the pain point I plan to solve using Alertify.

Why is this problem worth solving?

Notifications play a huge role in our day-to-day lives. Ever since the covid pandemic, our lives have gone digital. That means, most of our work has been digitalized. Confined to our homes, and glued to our phones, it has been getting difficult to stay away from our digital devices. For some, it is a medium to stay connected with the global world, for others it is a means to order their food, cabs, and whatnot. All of the above activities are stitched together with a common string of push notifications to notify users of real-time updates. This brings us back to the flaw in the design of notification systems in general. Be it our phones, laptops, or the doorbells of our homes. They rely heavily on our auditory senses to notify us by using sound waves. For deaf and hard-of-hearing individuals with impaired auditory senses, this corresponds to increased dependence on others to stay alerted or potentially missing out on important notifications. This restricts them from leading a normal life like others for no fault of their own. One may think it is only the deaf people facing such problems. However, the ground reality shows that the affected individuals include those suffering from temporary deafness too - ranging from people having ear infections to genZ and their music-during-work obsession. Your mind may be running around thinking about the potential alternatives to sound-based alert mechanisms like smartphone notification LEDs, vibrating sensors, etc. Continue reading as we compare those solutions with Alertify.

The Solution

The idea was simple - to convert a smart home lighting system into a visual alerting system that will notify users about incoming alerts. Since there are a lot of smart bulbs available in the market by different brands, my first intuition was to use existing smart bulbs and configure them for my use case. However, these bulbs can only be controlled via their own apps or smart assistants like Alexa and Google Assistant. Inside the bulbs, they use Bluetooth and WiFi-based communications to interact with the user’s phone. It was equally difficult to reverse-engineer them swiftly and added a lot of friction in the development of a custom-built application to control them. Thus preventing any external developer to drive innovation and find new use cases for their products. Determined to innovate, I took out the engineer inside me and developed a prototype of my own as a proof-of-concept, ditching the existing smart bulbs for now. After some brainstorming, I decided to make a simple LED blinking device that will listen to incoming notifications on the connected phone and blink a particular LED based on the type of notification received. Having tinkered with Arduino in the past, I decided to use an Arduino UNO board along with a Bluetooth HC-05 module for building the hardware part of the project. Due to my limited exposure to app development, I decided to take the help of MIT App Inventor to build my custom mobile application.

Sounds simple right? What I thought would be a cool weekend project ended up taking a week to complete. With no prior experience with app development, I first set out to explore no-code app development tools. This was when I stumbled upon MIT App Inventor. Now the only problem was, that this tool was meant to be used as a learning resource to teach school-going students about app development. Hence had limited support for complex functionalities. However, thanks to the vibrant community of developers, a lot of exciting new features are possible with the help of third-party extensions. For my project, I needed to listen to incoming notifications on the user’s phone by utilizing their notification service. While this support was not inbuilt, with the help of a wonderful notification listening extension by Taifun, I was able to move forward with my project. As of now I was reading the incoming notifications, filtering them out as per my needs, then assigning each notification a numeric priority value. This data was then sent to the connected Arduino board with Bluetooth for further processing.

Programming an Arduino is a whole new world. Here you are crippled by the limited space and computing power of the microcontroller. I had this O(n2) side of mine, which the Arduino’s O(1) personality didn’t really approve of. Everything had to be optimized to ensure smooth and low-latency operations. That meant keeping the logic on board as simple as possible. With this in mind, I scraped my initial thoughts of processing the data on the Arduino board and instead revamped the application side logic to process it on the user’s phone and only send minimal encoded data - enough to trigger a response. In simpler words, my Arduino board need not know whether the incoming notification is a WhatsApp message or a phone call. All it needs to know is whether the incoming notification priority is higher than the previous one or not. And if so, which LED no. to blink? Having worked with REST APIs where the majority of the data processing is on the server side, it was a bit challenging for me at first to optimize the data transfer as much as I could and send only the pre-processed information to the Arduino (backend). It required a shift in my mental model.

With my Android app ready and the Arduino code successfully compiled, it was time to fit the final block in the puzzle - making the connections. Armed with jumper wires, breadboards, and powerful microcontrollers, I set out connecting them like some little pieces of legos (no Jerry Rig was harmed during the process). As I was seeing the prototype coming together in front of my eyes, I felt like the doctor from Toy Story 2 fixing Woody, working with my small yet powerful tools. With all the pieces together, it was time for the most important part - testing the system. As the power began flowing through the Arduino, turning up the Bluetooth module with its blinking LED, I was doing my final checks to make sure of any loose connections and fixing them. I bring up my phone, connect it to the system and I wait. Wait for an incoming message. I haven’t been this anxious waiting for someone’s message as I was this time. Suddenly I received a message on WhatsApp and instantly I could see the white LED on my prototype blinking. Finally! My prototype was working as I expected it to work. Was it my eureka moment? Have I broken the ceiling? Okay, enough exaggeration. Let’s get back to the ground. I have made a minimalistic project that can listen to incoming notifications on my phone and blink respective LEDs to notify me about the same without looking at the phone or relying on its sound-based notifications to get alerted. Not only that, you can assign a priority to different types of notifications. E.g. An incoming phone call will have a higher priority than a reminder from your calendar app. This behavior can be customized using the custom-built app. Secondly, the users can even filter what notifications to listen to. Working on something important and only want to get alerted for phone calls? Alertify can help you with that.

Why this? Why not that?

For my critical-thinking readers, let me address the question you all have been waiting for - Why Alertify? If you have read so far, at some point you may have thought of some kind of currently existing solutions that can serve the same purpose as Alertify. Well, let me tell you, you are not alone. While researching this problem, my first step was to identify the existing solutions and how they are trying to solve this problem for deaf and hard-of-hearing individuals. One of the most exciting alternatives is the Apple Vision Pro (AVP) which has the capabilities for real-time speech-to-text. Imagine you are a deaf person talking to your friends, the AVP can listen to your conversation and visualize the speech in front of your eyes, somewhat similar to real-time captioning. This is not just limited to speech, but can also detect sounds like running water, someone knocking at your door or the “ting-tong” on your phone. Even better it can display on your screen whenever you receive a new alert on your phones. However, as of now, the tech is very new and you can’t go around wearing this device 10 hours a day without feeling a strain on your eyes. Then comes the smartwatches that people go around wearing throughout the day. They come fitted with vibrating sensors and can alert you about incoming notifications using it. Though they serve as a good utility for various other applications like tracking your health metrics and navigation, they fall short in providing comfort when used for longer periods of time. At home, where the users value comfort the most, they might not prefer wearing any device that alerts them at the cost of their comfort level. This is where Alertify shines. You don’t have to wear any device on your body. A device like Alertify can light up a corner of your room and serve the same purpose while at the same time adding to the aesthetics of your room.

At last, let’s talk about the notification LED that comes inbuilt into some smartphones. It is positioned on the top of your phone and constantly blinks in the event of an incoming notification. In the newer versions, this has been replaced by Always-On-Display or AOD. The major drawback of such an alerting system is that it requires the user to constantly keep their phones in sight if they want to be alerted. What if your phone is inside a drawer or kept at some faraway table? Or maybe it is kept in some other room. To tackle this problem, I plan to enhance Alertify and shift from Bluetooth-based to WiFi-based communication networks. In this manner, a network of interconnected smart lights can be created that can transform your whole home into a big alerting system. So if you are working in the kitchen and your phone is kept in your bedroom, the kitchen lights can change colors to alert you about any important incoming alerts on your phone. This behavior can also be configured by the users as per their preferences. This is what Alertify aims to achieve, making it stand out from other alternatives currently in the market.

The Road Ahead

While Alertify, in its current form, is far away from being used in our day-to-day lives, it is a first step towards making the alerting system inclusive for the deaf and hard-of-hearing individuals. During the process of building the prototype, I decided to use the resources available at my disposal. To make it compact and increase the range of communication any WiFi-enabled Arduino NANO microcontroller can be used. Secondly, a RGB LED in place of multiple single-color LEDs can be used to provide more customization to the end users. This can then be neatly packed inside a smart-bulb-like 3D-printed enclosure along with some AC to DC converters. However, I would like to bring your attention back to why I built this prototype in the first place. Big brands who have perfected their hardware have kept the technology of interfacing with their products private. This prevents developers from building on top of their devices. E.g. if you have a smart bulb of XYZ brand, you need to download the app of the same brand to control your device. Now pinch out the problem a little bit to see the bigger picture. There is a lack of a common mechanism to control digital devices from different brands through a single app. You don’t have a single app to control all your smart bulbs from brands A, B, and C. This problem is not limited to just smart bulbs but also other smart home devices like doorbells, door locks, ACs, smart plugs, and switches. A feature through which you can listen to your phone notifications and change light colors is fairly easy to implement. While brand A may provide support for it through its app, brand B may not. Even though the hardware of both the products may be able to support this functionality. This is why we need open networks for controlling our devices. Imagine if we could have an open network that all these individual devices use to communicate with each other. In this way, instead of using multiple apps, one can prefer using a single app to control all their devices. Not only this, it will allow community-driven solutions to take precedence and drive forward the innovative use-case of such devices. This is the change that can be brought by open-sourcing such systems. While manufacturers can then focus on enhancing their hardware, consumer-focused apps can work on bringing exciting features to their large consumer base. This will in turn help drive up the sales of smart home devices. This is what I envision the future to be and I call this setup, the ONDC or Open Network for Digital Communication.

That marks the end of Alertify. However, I do hope, that this article opens up the gates of your mind into thinking about how inclusivity can be brought into many more things around us. If you want to try this project, feel free to build it on your own. Here is a GitHub repository containing the Arduino code and APK file along with its source code which can be imported into the MIT App Inventor. Let us together build an inclusive world. If you enjoyed reading the article above, do show your support by giving it a like. At last, what do you think about such open networks from a security point of view? Do let me know your thoughts in the comments below.