Description

Smart Photo Editor is introduced for differently-abled people having hand motor disabilities to use as a good leisure time activity. This is a hand-free mobile application with attractive photo editing features. Users can interact with the application using simple head gestures, eye gaze, and eye blinks which prevent the use of hands. Meantime, the users are given a great experience with new editing features using object segmentation and image processing techniques.

User Interfaces

Main Objective

The main objective of the systems is to provide an opportunity for people with motor disabilities in their hands to access a smart photo editor application that can be used when they feel lonely, bored. As they cannot be busy like abled people due to some limitations, this product will be a good medication for them to use in needy instances. The application is a mobile application and proposed to build focusing on the differently-abled people who don't have hands or some motor disability in their hands. They should be given a hands-free experience to use the application. To fulfil this objective, the system is proposed to implement head, eye gesture recognition and real-time eye gaze detection as Assistive technology. The application needs no help from a third party. It is fully controlled by head, eye gesture inputs and eye gaze. Additionally, the application can get touch inputs that are enabled for ordinary people. Then normal people also can join differently-abled people in editing if desired; sometimes joining another person will make differently-abled people happier.

Specific Objective

There are several sub-objectives to finish a complete hand-free photo editing application for disabled people with hand motor disabilities. They are,

Head gesture recognition to understand user commands:

The head gestures are utilized to record operational commands by a user to the application because the targeted user group is disabled people with hand impairments. We use simple head gestures in the application considering the convenience and comfort of disabled users. The gestures are identified by the application, mapped with the relevant response action, and perform accordingly. Image processing and machine learning are used to implement head gesture command recognition.

Eye gaze-controlled navigation by cursor enabling:

As our application is hand-free, the user cannot use touch inputs to navigate on the screen. Mainly, the users need to navigate to the buttons located in different places on the screen. To ease this, a gaze-controlled cursor is enabled. As it visually depicts the user's gaze point on the screen, it is convenient for the user to navigate on the screen using the cursor. Image processing techniques are used to implement eye gaze tracking.

Detect eyeblink to perform button clicking:

One of the main tasks that users need to perform is clicking buttons. As hands cannot be used, a replacement is introduced for that using eyeblink of the user. The application can recognize the intentional eye blinks of the user. So that, a deliberate eyeblink is mapped to a click operation in the application, making it easy for the disabled users. Image processing techniques and machine learning is used to implement eyeblink detection.

Advanced image-enhancing features:

Our application gives the user an advanced photo editing experience. It has some unique features that are not present in any other existing photo editors. The photo editing options provided in the application are image-light enhancement, image de-noising, and image restoration. A lot of image processing techniques are used to implement these.

Object segmentation-based background customization:

This is a new functionality that is rare in mobile photo editors. The background of an image is recognized by doing object segmentation in the image. After the background and foreground are identified, the application can perform several actions to the background. The users can cut and replace the background, apply blur, splash, and bokeh effects to the identified background. This feature will leverage the standard of our Smart Photo Editor. Mask R-CNN is used for the implementation.