Introduction

The computer vision is the field of research where the machines are given intelligence as humans, and appropriate interpretation is done to analyse accordingly. It provides the necessary information depending on the observations made by the machine. Brains of human enable them to visualize things so easily, but for a machine to work similarly, it requires extraction, analysis of the information and then delivering the information with the help of specific algorithms for automatic visualization. It holds various applications in different fields like agriculture, biometrics, character recognition, gesture analysis, medical image analysis, and so on. One of the emerging areas which use computer vision is eye-tracking, which is measuring, recording and study of eye movements for visual presentation.

The study of eye movements had been started in the late 1800s using direct observations. The movement of the eye includes the intermediate stops for a span of time for the central fovea vision called fixations and movements between those fixations called saccades. Gaze points are the coordinate points (x,y) placed on the stimulus and fixations are multiple gaze points concentrated in a particular region subjected to change with respect to eye tracker configuration. For instance, gaze point for a tracker working at 60Hz will be with an interval of 16.7 milliseconds, and that of 300Hz will be 3 milliseconds apart. The fixations include start and end timestamps, including duration, which is calculated using a specific algorithm. Obsession, perceptibility, mental processing can be inferred from these above calculations. The eye tracker uses infrared light, which is focussed towards the pupil, and the reflection will be calculated with respect to a stimulus, which is called as eye-tracking raw data. This data will be further processed for calculations such as gaze points, fixations, saccades, and so on. This data can be used to inspect the cognitive ability and workload of a particular participant. This technology has a wide range of applications also such in driving, reading, surfing, searching and perceiving things. This has even reached to the industries of medicine and defence along within advertising, packaging, web designing and so on. The scope of research is increasing in areas like psychology and physiology.

This paper discusses the raw data which is processed for the calculation of fixations and saccade using dispersion algorithm and also visualizing fixations using heatmap over the stimulus and also saccades for representing the order of movements between the fixations. This visualization can be used to infer the cognitive ability of different participants and in analysing their Area of Interest.

Literature survey

(Steil, Huang, & Bulling, 2018) proposed a novel fixation detection algorithm where target appearance will be constant and use fine-grained fixation annotations on dataset MPIIEgo Fixation which outperforms velocity- and dispersion-based algorithm especially with respect to a number of fixations. Evaluation on the performance on the basis of evaluation metrics on fragmentation and merge errors. This enhanced the visualization and analysis of the scene image.

(Chung, Kumar, Li, & Levi, 2015) compare six characteristics of eye movement fixation in three conditions such as normal control eyes, non-amblyopic fellow eyes and amblyopic eyes. They proposed meditation analysis for the representation of error magnitude and microsaccade amplitude and also fixation stability. Summarization using box plot and usage of Kruskal-Wallis test along with Post-hoc pairwise comparison is done. The effective method of eye movement determination is using reclaim package along with the Shapiro–Wilk test.

(Caldara & Miellet, 2011) introduced a novel method iMap for computation of fixations. Fixation data is smoothed using a Gaussian kernel to generate 3D fixation maps. In order to assess the significance of fixation spots, robust statistical random field theory (RFT) approach is used. They compared and analysed high- and low-checkers implement different strategies with false information.

(Lans, Wedel, & Pieters, 2011) propose a velocity-based algorithm to detect fixations with specific thresholds and this algorithm is based on control chart procedures and robust minimum determinant covariance estimators (MDC). Binocular-Individual Threshold (BIT) was applied on a dataset, which is huge with different frequencies and concerned with various situations such as reading, viewing the scene and also in supermarket perception. Frequency distributions based on fixations and saccade length was plotted. The algorithm is secure and isolated from the machine.

(Nyström & Holmqvist, 2010) proposed an adaptive algorithm for detecting fixation, saccade and glissade on eye data. They also proposed new velocity-based algorithm identifies glissades as a different class of eye movement. They addressed many limitations of current algorithms and also developed data-driven algorithms. The algorithm is adaptive to noise dependent velocity -based threshold and also was successful in detecting glissades.

(Blignaut, 2009) used dispersion -threshold-based algorithm for the detecting fixations of gaze data from chess players. The optimum threshold value was found using various factors like a number of fixations, spatial dispersion and difference between scan paths. Fixations with radius 1 degree will provide replicable results with the usage of 90 percent gaze data. Different threshold values are taken, and later, one optimum threshold is considered. The threshold values less than .7 degree will result in different fixations than that of less than 1.3 degrees.

(Martinez-Conde, 2006) review neural activity related to fixational eye movements with participants having the visual disease and normal vision. They discussed various difficulties while fixation happens with respect to normal and impairment eye, specifically oculomotor diseases. They also proposed some of the factors which can be applied to the visualization of such fixations. The patients produced neither microsaccade nor any drifts as its amplitude was very negligible.

(Goldberg & Kotval, 1999) evaluated several measures based on eye movement location and scan paths while observing computer interfaces, also presented a framework for the analysis of eye data. They tested the gaze points with respect to spatial and temporal constraints. The poor interface resulted in more saccade with less directed search, more fixations and less interpretability. In the good interface, an instinct determination could be made to check whether the desired feature is available within the same context.

(Goldberg & Kotval, 1999) proposed five algorithms which constitute a classification of various classes. Detecting fixations and saccades are an inevitable part of the study of eye movements, which reduces the difficulty in tracking eye movements. Features are measured for the algorithms. Through these analyses, the most precise and accurate fixation identification found using I-HMM and IDT with the presence of noise in the device. The I-MST also makes the identification robust but considering all algorithms; it is slower. The I-VT has the simplest of all the algorithms, and it starts by estimating point to point speeds for all points in the protocol.

(Lupu & Ungureanu, 2013) explains descriptive studies in the field of eye-tracking, with the features regarding different types of devices, algorithms related to pupil recognition, image processing and filtering of data. The paper was a promising work for future researchers. The paper was a promising work for future researchers. coil method has very high accuracy and good resolution, and it is used for the measurement of torsion actions with minute variations in the design of the coil. It has explained methods like estimating movements with respect to the head and the second approach measures eye position comparative to the environments.

(Singh & Singh, 2012) reviews different methods of tracking eye movements. The main purpose is to encourage human-computer communication based on the eye. Through the analysis, they found none of the existing methods is universally best for all applications. The sclera coil method has very high accuracy and good resolution, and it is used for the measurement of torsion actions with minute variations in the design of the coil. It has explained methods like estimating movements with respect to the head and the second approach measures eye position comparative to the environments.

(Chennamma & Yuan, 2013) describe the involvement of eye movement in the research of Human-Computer Interaction. The intent of the paper was to present a review on the latest technologies in the remote tracking of the eye. This contained the basic terminologies and descriptions, future enhancement in the field of eye-tracking.

(Bojko, 2013) Elucidated the eye-tracking heat maps were very popular and effective in summarization and communication of data. Still, heat maps are often used improper and for imprecise reasons. Moreover, all the reasons are not being represented for actual interpretation. The objective of this paper is a review on several types of heat maps as visualizations in different aspects of visual attention and provides information on when, how to explicate heat maps. Its formation of heat maps and how to transform heat maps by manipulating various display settings. proposed a quantitative measure based on the Voronoi diagram, which separates regions of points based on distance. The method was listed in three conditions resulting in various optical distributions of fixations. It mainly focuses on the spatial feature description of eye movement.

(Wooding, 2002) discussed the application to quantification of similarity traces and the degree of coverage by fixation using fixation maps. The different traces were normalized with the clusters of fixations for comparison. This paper discussed the importance of visualization and communication of large eye movement data sets.

(Bhattacharya & Mishra, 2012) proposed eye-tracking methods with respect to various errors. Those errors are common and include more noise. It is time-consuming and requires manual effort for the correction. The validation of heuristic-based technique and lessening of the mistakes in association with fixations. It can be applied for the reading of objects placed in a sequential manner.

Proposed method

The method followed in our work is creating a GUI in R Studio by using a package called ‘shiny'. Shiny is an open source package for creating applications for an easier and efficient way of implementing the data. Therefore, making the analysis of the data into applications without the use of any other scripting languages.

The data is loaded into the application and then processed to detect fixations and saccades using dispersion algorithm and then visualized the same on the stimulus. The stimulus is various images on which the participant data is recorded.

Algorithm

Start

Step1: load data (start_point, x, y)

Step2: initialize variables

Step3: window start position start =1

Step4: while we move the window by 1, if D > threshold

Step5: window end position

Step6: create a window

Step7: dispersion

D = (max(x) - min(x)) + (max(y) - min(y))

Step8: while: we expand the window by 1 using j

Step9: select window (j - 1) as fixation

Step10: handle the last window if data ends during a fixation

Step11: fixations= data frame (result)

Step12: saccade=movement between the fixations

End

Results and Discussion

The experiments were conducted on ten participants on 12 image stimulus, and the experiments were carried out using an SMI eye tracking device. The raw data depicted in Figure 2 collected by the eye tracking device are processed to generate the fixation and saccades as depicted in Figure 4; Figure 3; Figure 1. The fixations points are then plotted with saccades on the stimulus depicted inFigure 3.

The results show the visualization provided by the R interface for fixation and saccades on the given image stimulus. This visualization can be further processed to understand the viewing behaviour of the participant. The raw data is processed, and fixations are detected using IDT algorithm.

Conclusion

The proposed work provides an R based interface to visualize the fixation and saccades. Through this visualization, the viewing behaviour of the participant can be understood. Highly dispersed saccades and longer fixation duration and dispersed fixation can depict a problem faced by the participant while the participant is trying to view the stimulus. Thus, we conclude that our work focuses on the basic need of understanding the fixation and saccades during an eye-tracking experiment.