The Development and Challenges of Face Alignment: A survey

Finding the face key-points location or which is also called Face alignment has gotten a lot of interest in the last twenty years because of its wide range of applications in the major of automatic face analysis. Nonetheless, this task has proven an extremely challenging task in an unconstrained environment because of multiple confusing factors, such as occlusions, position, illumination, and expression. However, this task is still an open problem, even though researchers still developing a great many techniques to face the challenges of this problem, in this survey, a critical review of the current literature in the face alignment field had already been presented focusing on those methods tackling the total of this topic's challenges and difficulties under uncontrollable circumstances. Especially, the existing face alignment techniques have been categorized existing referring to more than the type of categorizing. Then represent a survey of the recent related works in face alignment and lastly, we discuss the general steps of the face alignment system.


Introduction
"Fiducial facial points" that also called predefined face landmarks, which is mostly centered on or around the face parts such as the nose, chin, mouth and eyes (fig1).Finding the location facial points, otherwise called face alignment, it has received a lot of attention in the last fifteen years.and there are at least two reasons for that, first reason that face analysis, face makeup, Age estimation, face mask detection, expression recognition, estimation of the head pose, and many other important facial tasks can get benefit from localized facial points in an accurate manner The second reason, in the last years it achieves some level of success, finding the key-points task has a lot of challenges in unconstrained environments for that it's still an so far to be solved in the field of computer vision still attracts the researchers.Face alignment is considered as a very substantial and fundamental intermediary step for many following face analysis such as "biometric recognition" and the analysis and prediction of the mental state of humans.These tasks can be different in the needed landmarks type and the number of them, also the way that it going to be used it.First, a brief explanation of face alignment has been given with some examples of face alignment tasks, challenges, and methods.Second, General face alignment system steps have been stated and explained Lastly, a review of the most recent face alignment algorithms has been made.

Face alignment tasks
Face alignment is a very important step in many facial tasks and application is considered as a stepping stone for many facial vision tasks.Here some specifics of the regular tasks that depend on face alignment: • Face recognition : face alignment can be considered as a step of face recognition system, [2] using face alignment in face recognition applications gives good accuracy even with various head poses [3,4] • Age estimation: the apps that guesstimate the age need a robust alignment system detect the face's landmarks in accrue manner so they can study human faces properties and the size of the measurement for it.it also can be used to model the Age features [5] • Face makeup : face alignment is used in the makeup applications to align the face module with the face of the user by using the projected face points(dots) on the user's face which is have to be very correct and accrue on the face of the user to get the best result [6] These apps, and many others that haven't been mentioned, encourage the construction of a reliable and robust face alignment system.

Challenges
Under unobligated conditions the face alignment problem was treated wonderfully and some of its techniques were very close to human performance, but, this task is overmuch challenging and a huge step from solving, because of the huge level of Variability in faces structures that is caused by either innate dynamic features of the face elements like the nose and eyes.Also could be caused by changes in the environment.In specific, the following factors have an important impact on the face apparition and also have an impact on state of "facial features": • Pose: facial landmarks and features will be different between the images with different poses (e.g., upside down, profile, front face, and one side face) fig. 2 (a).
• Expression: eyes, mouth, eye bows, etc. are sensitive to the change in the face expression for instance when someone laugh it will change the shape of eyes and mouth fig. 2 (b).
• Illumination: the varying in the light intensity, distribution of the source and the light spectra can change the look of the faces remarkably and lead to a lost in the texture of the face components fig. 2 (c).
• Occlusion: occlusion makes a huge challenges in face alignment when it appears in face images which is captured in unlimited conditions for instance the mouth will be hidden by the face mask during the covid-19 period fig. 2 (d).Fig. 2. shows these challenges from Helen database [7], a perfect face alignment system have to be capable to deal with these factors and there challenges.In the past five years, a great development has been noticed in applications of face alignment and the research conducted on it, despite this, there is no survey that collects and compares these studies.The major aim of that research paper is to provide a inclusive and critical review for 2D images face alignment methods and to address the challenges and the difficulties unrestricted environment.

Overview
Face alignment is a long history computer vision problem and it has a big number of approaches to deal with it, with varied levels of effectiveness.Overall, face alignment may be characterized as the challenge of looking across a face image for predetermined face key-points which's known as the face shape too, face alignment process started from rough initial shape and go ahead step by step by estimate face shape till the convergence.Two provenance of inputs over this search process are needed which are the shape information and the facial appearance.The shape information seeks to formally represent the spatial relationships among the positions of face key-points to guarantee that the assessed face key-points can compose the right shape of the face, even though some methods does not use the "shape information" in straightforward way, the combination between these two is so common.[7] Before everything, categorizing the methods is going to help to understand the methods holistically by using the shape information and the appearance information, for that pattern recognition basic modeling principles are followed and the available methods are almost divided into two categories: generative methods and discriminative methods.[8] that is going to be discussed in section 5.

Generative methods:
Which is also called AAM, a generative models has to be built for the appearance and the face shape.These methods formulate the face alignment as an optimization problem in order to find the appearance and the shape parameters and that will generate a face model an example of getting the most appropriate trial face.The face appearance can be represented by local image patches which are centered on the face points or by warped face (whole face).[7]These methods use shape model PDM which is training from AAM manual labels, it's models the global intensity variation of facial appearance which already been shape-normalized in AAM two generative models are being employed, and it need to estimate shape and appearance parameters both while the AAM-based fitting.[9]

Discriminative methods:
Which is also called BAM these methods infer the target location from the appearance of the face directly.That will be done typically via using a global shape model to adjust its predictions in addition to make the independent localregressor or detector learn for each of the facial points, or deducing the whole face shape via learning a Vectorialregression function directly, in the time of which the shape constraint encoded implicitly [7] Use shape model PDM which is training from BAM manual labels Whilst, BAM will learn classification boundary among the right warped photos and wrong warped photos optimally in a much known boosting framework.And by comparing it to AAM it takes better advantage of the facial image which is labeled manually, and the reason for that, is it not learning only from the appearance of right alignment like AAM does, it also learns from the appearance of the wrong alignment.When a Comparison BAM to AAM algorithms has been represented when there are partial occlusion, the BAM algorithm is more disposed to be strong and the reason for that is the local rectangular features and that's why BAM will improve the accuracy, efficiency, and robustness of face alignment in a great way [9] Differences between AAM and BAM has been presented in table 1

Fig. 3-Popular approaches for face alignment methods
The most recent studies on face alignment divided the face alignment methods into three or two categories which are the Holistic method, CLM, and the Regression-based method.Each method's algorithm differs from the other in performance and speed.[10] Look at: table (2)

General face alignment system steps:
A face alignment system will be divided into sup stages [24].Mainly it could be splitted into these parts: the first one face preprocessing then the shape initialization process, and finally the third one is feature extraction and shape prediction repeatedly.This architecture is only to shows the face alignment general steps, however not all components are obligatory in the practical work.For instance, the LBA method [25] does not include the face processing step.But the feature extraction and shape prediction process are important and they always play a big part in face alignment methods.But, the shape initialization and the face preprocessing may be ignored sometimes.At the same time, for all the face alignment systems there is an essential problems such as the balance between the efficiency and accuracy and the augmentation of training data.Fig. 4.

Survey on the last face alignment methods
In this survey some of the most recent face alignment methods has been represented with the dataset which tasted on with the NME on each one in table 3. The face alignment methods in the last recent years have moved towards the Discriminative methods for some reasons has been explained in section 5.

Table 1 . Comparing between AAM and BAM
.