blog

Push Notifications on Droidscript

NOTE: Proof Of Concept

So, you are developing DroidScript application and your requirement is to send Push-Notifications to user phone. Also, you want to handle correctly those notification messages, so each message would have different outcome: open particular page on your app or do whatever is needed.

Looks like there are some options not to use Google Firebase infrastructure for notifications, but get away with other tool. As this project is more of PoC- there will not be very mature back-end solution or front end. This project is more like showing the way how Push-Notifications can be implemented in you DroidScript app. Obviously, there are more solutions available, and I assume each one witch supports JavaScript should work. But we are learning, so it good to know basics.

This project is to let you understand what it basically needed to have this functionality. No paid tools or licenses are required, event for testing purpose no back-end hardware is required- Public free options can be used.

Basic concept of a project is following: there is Droidscript application, it has Service that runs on background. Service utilise MQTT protocol for receiving messages and creating Android Push-Notifications. Clicks on notification is handled by Main application. Each client should subscribe to unique topic (for PoC I simply used topic notification). Then back-end engine should send messages to required topics.

For MQTT messages, you can fire up your own MQTT Broker or use the Public one. Droidscript MQTT plugin work by communicating with broker user Web Sockets. Wikipedia: WebSocket is a computer communications protocol, providing full-duplex communication channels over a single TCP connection. The WebSocket protocol was standardized by the IETF as RFC 6455 in 2011, and the WebSocket API in Web IDL is being standardized by the W3C

This is not needed if you want to test with public brokers. You can skip this part

If you have chosen to set up your own hardware- please follow this write-up further. For this example I use Mosquitto (by Eclipse) software broker that works on mostly any hardware and operating system.

Eclipse Mosquitto is an open source (EPL/EDL licensed) message broker that implements the MQTT protocol versions 5.0, 3.1.1 and 3.1. Mosquitto is lightweight and is suitable for use on all devices from low power single board computers to full servers.

My hardware is Raspberry Pi 2B+. The same broker can be used in IOT projects, for example with ESP32 to send you sensors value. Here I will cover configuration required for Web Sockets and MQTT Broker. How this can be installed search here. If you are on linux, after installing broker you need to chaneg configuration. Edit following file with a command sudo nano /etc/mosquitto/mosquitto.conf or any other way you like it.

# if you need to have closed system- you need to
# provide were login details are saved
# if allow_anonymous is set to false
password_file /etc/mosquitto/public/public.passwd
listener 1884
allow_anonymous false

# web sockets configuration
listener 9004
protocol websockets

After changing configuration, you should be able to connect to your Broker via MQTT. For checking it, you can use HiveMQ WebSocket Client. Provide your broker IP, port (and login if needed). If it connects succesfully, you can subscribe to topic $SYS/# Here you will get all statistics how your Mosquitto is running.

The next step is to prepare software side. To work with MQTT- plugin is needed that is can be downloaded from droidscript application. Install it

Create new JavaScript application in your DroidScript . Your project must have two files: main file that is named after you project, and another, named “Service.js”. This tells your DroidScript application to run Service.js as Android service and listed for DroidScript messages.

//Called when application is started.
function OnStart()
{
    //Start our service.
    svc = app.CreateService( "this","this", ()={
        // On service ready
        console.log( "Service Ready" );
    });
    svc.SetOnMessage( OnServiceMessage );
 
    //This will cause your service to start at boot.
    //(Set it to "none" if you need to stop it starting)
    app.SetAutoBoot( "Service" );
 
    var id = app.GetNotifyId();
    if( id ) HandleNotification(id);
}
 
//Called when application is resumed.
//(eg. When user returns from home screen)
// or after notification is clickeed while app was open
function OnResume() {
    app.ShowPopup( "On resume!", "Short" );
	var id = app.GetNotifyId();
    if( id ) HandleNotification(id);
}
 
 
function HandleNotification(id) {
    app.CreateNotification().Cancel(id)
    app.Alert( id, "Notification ID" );
}
 
//Called when messages comes from our service.
function OnServiceMessage( msg ) {
    console.log( "Service Message: " + msg );
}

Here is code for your service file. File must be named exactly this way to make it work as Android service

Service.js
app.LoadPlugin( "MQTT" );
//Note: When running background services on newer Huawei phones, you
//may need to go into the Android battery saving settings and enable 
//DroidScript to run in the background if you want background 
//notifications to work when the phone is locked.
 
//Called when service is started.
function OnStart()
{
	//Force service to foreground on newer devices (required).
	if( app.GetBuildNum() > 25 ) {
        app.SetInForeground( "Waiting for notification...");
	}
 
	// my private mosquitto configuration
	var mosquitto_options = {
    	servers:[{
    		host: "bukys.eu",
    		port: 9004
    	}],
    	keepalive: 1800,
    	username: "myUser",
    	password: "myPass",
    }
    // Unique identifier for each device
    if(app.IsAPK())
        mosquitto_options.clientId = "Android-" + app.GetDeviceId()
    else
        mosquitto_options.clientId = "IDE-" + app.GetDeviceId()
 
    // Public mosquitto connection
    // client = mqtt.connect( 'ws://broker.hivemq.com:8000/mqtt' );
 
	client = mqtt.connect( mosquitto_options );
    client.on( 'connect', ()=>{
        client.subscribe( 'notification', {qos:2} );
 
        // send client id as message to topic "clients"
        client.publish("clients", mosquitto_options.clientId)
    });
 
    // create Android notification when MQTT message arrives to subscribed topic
    client.on( 'message',  (topic, message)=>{
        not = app.CreateNotification();
        not.SetMessage(
            "ticker",
            "MQTT "+topic,
            "Msg: " + message,
            "Message: "+ message
        );
        not.Notify( message );
    });
 
    setInterval(()=>{
        client.publish("heartbeat", new Date().toLocaleString())
    }, 60000)
}

Word of warning- android system limits web services. In my POC i've set up message publication each 1 minute to HeartBeat topic, so I would know how service behaves under the hood. On Android 11 Samsung device, at first it sends message each 1 minute, but later gradually increasing time between hear-beets. On my test average is 7 minutes. Looks like it does work correctly and on receiving MQTT message- prepare notification. Sure, if you unlock your device, heartbeat is sent each minute. You may need to play around with various battery optimisation setting for your application.

There are various MQTT protocol settings (Retained and QoS) may ensure that message is received even if device is disconnected upon sending notification message to broker from back-end. Client will get it when it connect to it unique topic.

Log Micropython console text to file

Here is a small snippet, how you can write everything what you get on console to file in your Micropython device. This may help you to debug long running devices.

import io, os
 
class logToFile(io.IOBase):
    def __init__(self):
        pass
 
    def write(self, data):
        with open("logfile.txt", mode="a") as f:
            f.write(data)
        return len(data)
 
# now your console text output is saved into file
os.dupterm(logToFile())
 
# disable logging to file
os.dupterm(None)

Project "SmartHome" build: Central

My personal project to build Smart Home system and learn Micropython by utilising LVGL on ILI9341 display by using ESP32 microcontroller.

Initial idea was to have one SmartHome Central space where I could see all rooms temperature and humidity measures in one place. I'm also planning to add soil moisture sensor (built on my favourite ESP32) and later on- control some of appliances at home. I did not want to go with already built systems (HomeAssistant or Tasmota) as my goal was to learn to build multi-component system by myself.

Central Unit

Source is available on my github repository. Comments and ideas are welcome.

Central unit is ESP32 generic (no PSRAM) with 4mb PSRAM microcontroller with Micropython + LVGL firmware and ILI9341 display. It listenf for messages in MQTT and display innformation on screen. Also weather integration is made: current weather outside and forecast

Wire connection diagram
ESP32 ILI9341
3v3 VCC
GND GND
26 CS
33 Reset
27 DC
23 SDI (Mosi)
18 SCK
32 LED
19 SDO (Miso)
14 T_CLK
15 T_CS
13 T_DIN
12 T_DO
35 (not used) T_IRQ

I've used two different SPI busses, One for ILI9341 and another one for XTP2046 (touch) as i had plenty of pins on my central device.

Proxy unit

Source is available on my github repository.

Proxy unit is also ESP32 generic based device, witch listens to all available BLE advertisement packages. It filters know MAC addresses and forward that messages to MQTT.

BLE thermometers

Probably, this section may be most interesting for various DIY'ers :-)

Once I've stumbled on Youtube video, where person was discussing to DIY or BUY temperature sensor. Probably, you can not DIY cheaper yourself, because this sensor cost 4$. Yes, four dollars! It can be used as stand alone device as it is has its own screen.

Issue was BLE advertisement is encoded. Dear Xiaomi- let us use your devices freely in out projects. Open source is future!

But lucky for us, DIY'ers, AAron Christophel dedicated a lot of his own time and was able to write custom firmware for extra cheap (4$) Xiaomi Thermometer model LYWSD03MMC. Its easy to change firmware in thermometer with any regular Android device. No additional hardware is needed. To change firmware- download bin file from ATC github repository to your phone. Then open web flasher. I've used Android Chrome browser for flashing.

Always consult official repository how flashing procedure is done! It may change over time.

  1. Download firmware file
  2. Enable bluetooth on your device.
  3. Open web flasher page and press button “Connect” and choose your device. If it is the first time you flash firmware- its name will be LYWSD03 or something similar. After flashing- it will be ATC_######. Last six digit will be last six pairs of your device mac address.
  4. Do activation”. It connect to device and calculates Device ID, token and Bind key. Nothing will be changed in firmware.
  5. Choose bin file. Take care of what file you choose. Mistake can brick your device. Anyway, not a huge loss- it's dirt cheap :-)
  6. Start flashing”. It may take 60-100 seconds to complete. It should restart after finishing. If it did not- wait 5 minutes and remove battery from thermometer.

Soil moisture sensors

I'm planning battery (LiFePO4) powered ESP32 device to measure soil moisture of my balcony greens and send data to central unit via BLE advertisement. My choice of LiFePO4 battery was because it's perfect candidate to directly power ESP32 (without any power management devices). More about that in my later posts. Now i'm in progress of building this.

What i have learned

  • How BLE advertisement works;
  • Setup environment, required for Micropython firmware building;
  • What is MQTT and how it works, setup server locally.

Pure python and very simple event management library

In event driven programming events are crucial. My requirements was Micropython platform on ESP32 micro-controller, so i could not use any other events library out-there. I've stumbled one very basic, very simple library and gave it a try… and it worked out-of-the-box for my case. When testing various event management libs out there I've stumbled on something similar, but it did not worked within classes. Python is nothing but classes everywhere :)

I was testing and extended this very simple python script, that perfectly works as main element in event driven development.

Strengths of library:

  • Observer can be declared before event type is created
  • Events can be fired even if there is no observers.
  • Works as pure python script
  • Works on micropython platform
  • Event name can be any string

I'm using this solution for event for timer, that fires each second and observe “ticker” where i need to update clock. I use clock in multiple places in my micropython script and i can get away with just one hardware timer used.

Honorable mentions goes to Pithikos from StackOverflow this answer

I have extended this library with possibility to forget event.

EventObserver.py
class Observer():
    _observers = []
 
    def __init__(self):
        self._observers.append(self)
        self._observed_events = []
 
    def observe(self, event_name, callback_fn):
        self._observed_events.append({'event_name': event_name, 'callback_fn': callback_fn})
 
    def forget(self, event_name):
        for dict_item in self._observed_events:
            for key, val in dict_item.items():
                if val == event_name:
                    self._observed_events.remove(dict_item)
 
 
class Event():
    def __init__(self, event_name, *callback_args):
        for observer in Observer._observers:
            for observable in observer._observed_events:
                if observable['event_name'] == event_name:
                    observable['callback_fn'](*callback_args)

Here is example of how to use this library

example.py
import EventObserver
 
# class object MUST inherit Observer object if you use it within class
class Room(Observer):
    def __init__(self):
        print("Room is ready.")
        Observer.__init__(self)  # DON'T FORGET THIS
        # observer can be declared already in class init method with following line
        # this.observe('someone sneezed', this.someone_sneezed)
 
    def someone_arrived(self, who):
        print("{} has arrived!".format(who))
 
    def someone_sneezed(self, who):
        print("{} has just sneezed!".format(who))
 
 
# Observe for specific event
room = Room()
room.observe('arrived', room.someone_arrived)
room.observe('sneezed', room.someone_arrived)
 
# Fire some events
Event('left', 'John')       # no events are attached to event "someone left" 
Event('arrived', 'Lenard')  # will output "Lenard has arrived!"
Event('sneezed', 'Jenny')   # will output "Jenny has just sneezed!"
 
# Remove known event
room.forget('arrived')
 
# no events based on 'someone arrived' will be fired
'''

Older entries >>

  • blog.txt
  • Last modified: 2020/03/23 00:00
  • (external edit)