A Collaborative Representation Approach to Detecting Error-Related Potentials in SSVEP-BCIs

This study takes advantage of Error Related Potentials, a certain type of neurophysiological event associated with humans' ability to observe and recognize erroneous actions, in order to improve SSVEP-based Brain Computer Interfaces (BCIs). The Error Related Potentials serve as a passive correction mechanism that originates directly from the user's brain. In this paper we propose a novel approach to spatial filtering, based on a supervised variant of Collaborative Representation Projections (CRP) offering a more discriminant representation of electroencephalography signals for detecting Error Related Potentials. This new approach enhances the detectability of Error Related Potentials by projecting the spatial information of signals into a new space where samples of the same class tend to form local neighborhoods. Moreover, the limitations under which the Error Related Potentials positively contribute to the performance of a SSVEP-based BCI are explored. For this reason we also provide a new methodology, namely Inverse Correct Response Time (ICRT), that reliably captures the trade-off, between the gain of the automated error detection and the induced time delay of a BCI system that potentially incorporates Error Related Potentials.


INTRODUCTION
During the last years, Human-Machine Interfaces (HMI) met a rapid growth in both research and application fields [1]. The main reasons of the aforementioned trend lie on a confluence of factors, including: affordable and fast computers, knowledge in neurophysiology, commercial neuroimaging devices, and more sophisticated signal processing and machine learning schemes [19]. Within this context brain-computer interfaces (BCI) extend the frontiers of interaction with contemporary systems.
During the first years of their development, BCIs were dedicated to offering new communication pathways to patients with severe motor disabilities [3]. Nowadays non-medical applications (as in arts, video-games etc.) are making their appearance in order to enhance users' experience by giving access to new interaction methods [18,29]. Moreover several BCI-related studies concerning multimedia-rich environments [8] offering both active, where the user actively operates the system using his mind, and passive BCI systems, where control signals originate from users brain activity automatically and unintentionally [37].
A quite common approach in active BCI systems is to utilize the steady-state visual evoked potentials (SSVEPs). SSVEPs are brain signals that occur in response to a periodic visual stimulation, a flashing light of a certain frequency. The brain consistently produces electrical activity at the same (or multiples of this) frequency mainly located over the occipital regions [2]. By having multiple selection options (e.g. square boxes) flickering at different frequencies, each one representing a different interface choice, the user can select the desired option using SSVEPs and in this way interact with the interface.
Undeniably SSVEPs are among the popular brain-based interaction paradigms not only because they can be reliably detected but also due to their ability to offer a generic interface that can be easily adapted in multiple applications [32]. The majority of SSVEP-based BCIs has been tested in laboratory-like environments with ideal settings (e.g. well sound-proofed rooms, with optimal lightning conditions, etc.). Hence, this is not to be expected in a real world Session 1 Thematic Workshops '17, Oct. 23-27, 2017, Mountain View, CA, USA setting, where such conditions are unlikely in the every-day living of the users. Overall, in a real world setting, detecting brain commands is still an error-prone procedure, forcing the users in unintentional interaction errors. Naturally, the question that rises is if we could leverage Error Related Potentials (ErrPs), specific neurophysiological brain responses associated with ability of humans to monitor and detect erroneous actions, in order to develop a highly responsive interface that would enhance the user experience.
In this work, we present an error-aware BCI based on the SSVEP paradigm which is able to identify and ignore erroneous userinterface interactions based on the detection of ErrP signals. The proposed system is implemented within the context of a brainaccessed web site 1 , where a menu with different options exists that redirects to different web pages. ErrPs in our case, serve as a passive feedback mechanism originating from the user's brain and denoting which user commands were correctly decoded and which need to be redone.
More specific, we propose a novel methodology that stems from collaborative representation based projections (CRP) for spatial filtering in order to increase the accuracy of detecting ErrPs. We show that a supervised variant of CRP is more efficient for ErrP detection. In order to evaluate the performance of such an approach we make use of a novel measure, based on the Utility metric [7], that can easily be adapted in a wide range of BCI systems and corresponds to the inverse of the average time needed to perform one action correctly.
To this end, our study unfolds in two different directions. Firstly we propose a novel spatial filtering method, that is suitable for single trial analysis of Error Related Potentials. Secondly a new BCI evaluation method is presented that quantifies the performance of a system that incorporates an error detection procedure, namely Inverse Correct Response Time (ICRT). Moreover, we are investigating whether ErrPs can be reliably detected in a SSVEP-based BCI and we are demonstrating how they can be exploited in order to improve the overall performance of the BCI in terms of Information Transfer Rate (ITR). This approach uncovers the conditions under which a SSVEP-based BCI system can benefit from the introduction of the error-awareness.

BACKGROUND 2.1 Steady State Visual Evoked Potentials
When a series of identical stimuli are presented at specific frequencies (approximately in range 6-75 Hz) [2], the primary visual cortex of the recipient will generate electrical activity at the same (or multiples of) frequency of the visual stimulus. Besides the significance of SSVEPs in clinical studies, their employment as a basic building block of Brain Computer Interfaces (BCIs) make them a very important tool.
The study of SSVEP-based BCIs has attracted a lot of attention in what refers to the use of algorithms and methods for maximizing the classification accuracy and improving the information transfer rate. The novelties that have been introduced in the literature cover the full spectrum of the Signal Processing module, ranging from signal filtering and artifact removal all the way to feature extraction and classification.
SSVEPs constitute a basic building block of Brain Computer Interfaces (BCIs) by enabling the user to select among several "flickering" interface options, each one corresponding to different command (e.g. selecting among flickering letters in order to type a word). Each command is associated with a repetitive visual stimulus that has distinctive properties (e.g., frequency). The stimuli are simultaneously presented to the user who selects a command by focusing their attention on the corresponding stimulus. When the user focuses on the stimulus, a visible SSVEP oscillating at the target frequency (and its harmonics) is produced. In general, SSVEP-based BCI systems have the advantage of high signal-to-noise ratio [20]. In addition, short or even no training time and few EEG channels are required. On the other hand, viewing flickering boxes can be tiring and disturbing to the user. Finally, SSVEP-based BCIs are usually designed with only a few options (e.g. 5 or 6), since it is difficult to distinguish more frequency components [15].

Error related Potentials
Error Related Potentials, were first introduced in [9] and refer to an electrophysiological response of the human brain when an erroneous action is monitored and/or realized. A typical ErrP signal consists of two components, a negative deflection followed by a positive one [5]. The first one (usually refered to as Ne) has a frontocentral scalp distribution and peaks, approximately, from 50 to 100 ms following incorrect responses. The second component, the errorrelated positivity (Pe), is associated with awareness of erroneous actions and consists of a large magnitude peak. This component, which is subsequent to the Ne, is most distinguishable in the centroparietal brain area. The most prominent findings regarding the localization of this event indicate that the anterior cingulate cortex is mostly activated during erroneous processes [12,13].
Error Related Potentials have been used mostly to improve the user experience provided in various BCI applications. In particular, spellers, a common category of BCI applications, exploit the Error Related Potentials. Authors in [26] took advantage of neural correlations with error awareness so as to achieve higher ITR in a P-300 based BCI. An online adaptation of ErrPs was employed in [28] in conjunction with code-modulated VEPs. Generally the results of integrating ErrP-based correction in a HMI systems are encouraging. Several studies have successfully combined motorimagery based interfaces with ErrP mechanisms that served as an indication to whether ignore certain actions (those classified as erroneous) or not [11,14]. More recently researchers showed that the amplitude of ErrP components is modulated by the severity of the error and that ErrPs can be detected when continuous feedback was presented, which means that discrete feedback presentation is not mandatory [27]. In this work we examine how and when ErrPs can be used in order to offer a significant improvement in a SSVEPbased BCI for a web-site navigation by canceling out erroneous actions automatically.

Spatial Filtering
A common preprocessing step, in EEG signal analysis, concerns spatial filtering, where a transformation is applied to the spatial (channels) domain in order to increase the signal-to-noise ratio. Spatial filters are actually separated in two major categories. The first concerns spatial filters that are not data specific, as the Laplacian filter where the kernel (weights that applied in each channel separately) is fixed [17]. These filters are mostly used is EEG analysis to diminish volume conduction effects, where the same brain activity is recorded by several electrodes. On the other hand, more sophisticated approaches, which are data driven, have been developed advocating different perspectives. A common approach in motor imagery BCIs is the Common Spatial Pattern (CSP) algorithm, which is used in binary EEG classification tasks to simultaneously maximize the variance of one class while minimizing the variance of the other class (and the opposite) [31]. Principal Component Analysis (PCA) has also been used as a spatial filtering method [16]. A more recent approach of spatial filtering was employed in order to enhance prediction of emotional states using low rank estimation [36]. A detailed collection and comparison of the most used spatial filter techniques is presented in [6]. Among the already discussed methods only CSP takes class information into account but it is not suitable for Event Related Potential analysis. CSP method is dedicated into calculating spatial filters that separate signals according to their variance and requires a signal filtered in a narrow band which makes it more appropriate for studying synchronization and desynchronization events (e.g. senso-motoric rhythms).

Subspace Learning
Dimensionality reduction is a wide field of statistical analysis that has drawn a lot of attention. It is quite common in many machine learning and statistics tasks to use dimensionality reduction methods as a feature extraction process especially when they succeed in preserving the underlying manifold structure of the data. The graph embedding methodology, where the data are represented by a graph according to their relationship, offered a unified framework for multiple dimensionality reduction methods (PCA, Linear Discriminant Analysis, etc) [34]. The graph construction is probably the most crucial step in these methods since it should be able to capture and reveal the inherent data relationship. Collaborative representation has been recently proposed as an automatic way for graph construction where data similarity stems from the ability of data to reconstruct each other [35]. Each data point is potentially represented as a weighted sum of the rest data points. The weights used during this reconstruction process represent the similarity of data.
In this context we exploit collaborative representation projections (CRP) in order to create spatial filters suitable for single trial analysis of ErrPs. Since the ErrPs waveform is reproducible across trials -only in certain electrodes-it is not expected that the classic collaborative representation approach will be a suitable method for graph construction. Activity beyond the area(s) of interest (e.g. area over the occipital lobe is not expected to reflect activity related to the perception of errors) will present no consistency due to the chaotic nature of EEG signal and this could lead to a fail of the original CRP method. This fact served as motivation to our supervised CRP approach.

Evaluation Metrics
In order to evaluate the benefit an error prediction system offers to a BCI we need reliable estimators that quantify this gain. The most common metric that is used in the BCI domain is the Information Transfer Rate (ITR). This metric was originally deployed for communications and has a strong theoretical background based on Shanon's information theory [24]. The first BCI related variant of the ITR [33] calculates the amount of information that can be transfered per units of time. However, the classic ITR formula does not take into account the possibility of an error correction mechanism. Recently Dal Seno et.al. proposed not only a more general framework that considers error detection, namely the Utility metric, to calculate the efficiency of a BCI-speller system but also formulated the ITR so as to consider error detection [7]. In our work we had to employ a new evaluation method since the ITR variant that takes error correction into account calculates the theoretical upper bound for a BCI while Dal Seno's approach is exclusively formulated for speller applications. The assumption in [7] that for every erroneous action a correct is needed to cancel the previous (e.g. for each mistyping a backspace is required) does not apply to our case, in which a simple "cancel out" redirects the user to the selection panel and all previous erroneous navigation options are canceled (see section 4.1 more information).

METHODS 3.1 Collaborative Representation Projection
The collaborative representation projection method [35] aims to offer an automated way in order to construct the graph that represents the data relationship. During the graph construction process each data sample is represented as a linear combination of the rest of the samples. Let us denote by X = [x 1 , x 2 , . . . , x n ] the data matrix that contains n m-dimensional samples. The problem corresponds to calculating the optimal weights w i that offer the best reconstruction for each x i In the equation above, where each element w i j of w i * represents the contribution of x j in the reconstruction of x i . The L-2 graph G(X ,W ) is constructed considering that data samples are the vertices, the collaborative weights as the graph weights and q is set to 2. For L-1 graph q should be set to 1.
The constructed graph can now be used in order to calculate a new projection space through a projection matrix P. The L-2 graph represents the reconstruction relationship of the data using weak sparse constraints (a more strict sparsity constraint can be achieved using the L-1 graph). We now aim to find a projection space where local compactness of the graph is minimized. This means that the samples with the ability to accurately represent another sample will preserve that property in the projection space. This criterion is formulated mathematically as Session 1 Thematic Workshops'17, Oct. 23-27, 2017, Mountain View, CA, USA and by setting S L = X (I − W − W T + WW T )X T , where I denotes the identity matrix, equation 2 can be rewritten as: Apart from minimizing the local compactness of the graph (which can also be understood as sparsity preservation) we aim at a projection capable to offer maximum separability of the data. This leads to the maximization of the total covariance, in the projection space, that can be achieved though the total scatter and is expressed by: which can be rewritten in matrix form, by setting It is consistent that for machine learning purposes both of the aforementioned criteria need to be held simultaneously. We should minimize the local compactness and in the meantime we need to maximize the total separability. The final optimization problem is formulated as The solution to equation 6 can be obtained by the generalized eigenvalue decomposition of S T P = λS L P. Finally P * corresponds to the eigenvectors of the largest eigenvalues of the previous problem.

Supervised Collaborative
Representation. The classic CRP algorithm operates in an unsupervised manner assuming that the weights will be able to uncover the underlying relationship of the data. In EEG the desired brain activity is only captured by certain channels while the activity of the rest channels is being inconsistent (regarding a desired stimulus) due to the nature of human brain signals where each brain region is devoted to a different task (e.g. it is not expected to find the reflection of an auditory stimulus in signals recorded over the motor cortex). Consequently, by using the classic CRP approach we would end up with weights that are not meaningful although the collaborative representation error will be sufficient small.
The supervised CRP (sCRP) differentiates from the original version only during the graph construction process. In our study we enforce the data to be represented as a linear combination of the same class samples, that is w i j = 0 if i and j data samples belong to different classes. Equation 1 is used to calculate the weights for samples of the same class. Since the CRP objective, described by equation 6, ensures that locality is preserved in the low dimensional space we end up with a more discriminant representation. Intuitively the samples of the same class tend to create a local neighborhood while the overall scatter, across all classes, is maximized.
As already mentioned, during the graph construction we take into account only the samples of the same class. This reduces the computational cost needed during weight calculation (equation 1), which is the most computationally intensive task since it involves the calculation of an inverse matrix as many times as the number of training samples.

Measuring the efficiency -ICRT
Our objective in this section is to showcase the benefit of having an error detection system in a BCI, with respect to the ITR (i.e. the time that is required to perform an action correctly).
In order to compute the efficiency of an error-agnostic BCI system enhanced with error detection capabilities we compute the inverse of the average time needed for an individual to complete an action (e.g. to select a specific navigation option) correctly by taking into account the initial system's (error-agnostic) accuracy as well as the precision and recall values that correspond to the error detection system. This quantity, that will be referred to as Inverse Correct Response Time, is monotonically related to the ITR of the system (i.e. information transfer per unit of time). Denoting the number of actions to be completed as s, accuracy of the initial system as Acc (without the error detection feature), the duration required for a user to complete an action as t (i.e. for how many seconds will the boxes flicker in SSVEP setup), the recall of correctly interpreted actions (actions that were interpreted by the system as the user intended) as Re(c), the recall of erroneous actions (actions that where miss-interpreted by the initial system) as Re(e) and the time needed for the user to transition from the erroneous state to the initial state (navigation panel) as d, we calculate the time needed to complete s correct actions in an error-agnostic system as Equation 7 sums the time for s actions plus the extra time needed to repeat the erroneous ones till none erroneous is left. Although it is straight forward to calculate the time needed by an error-agnostic system, the calculation of time required in an error-aware system derives from the addition of four subcomponents. Considering the first stage of a simple system, where the initial system classifies the user's intentions with accuracy Acc. Then the error detection system detects the errors with a true positive rate (TP), a false positive rate (FP), a false negative rate (FN) and a true negative rate (TN). In the first case (TP), the user's intention were correctly interpreted by the initial system and the error detection system did not detect any erroneous action (e.g. an ErrP when the selection was presented to the user). These trials do not need to be repeated. In the case of FP, where the initial system falsely interpreted the user's intention and the error detection system did not manage to detect this miss-interpretation, the user needs d time to undo the previous action and t time to repeat the action. In the case of FN, where the initial system correctly classified user's intentions but it was considered falsely as a miss-interpretation by the error detection system, the user just needs t time to repeat the selection, since the canceling of the previous action is performed automatically by the error detection system. Finally, in the case of TN, where the initial system erroneously interpreted the user's intention but the error-detection system was able to capture this miss-interpretation, the user needs t time to repeat the selection action. In all cases, there is an additional time e that is essential for the error detector (in our case this time corresponds to time needed for ErrPs to be elicited), which is added to the time t of each trial.
T T P = Acc · s · Re(c) · (t + e) T F N = Acc · s · (1 − Re(c)) · (t + e) (10) T T N = (1 − Acc) · s · Re(e) · (t + e) (11) Equations (8), (9), (10), (11) describe the amount of time needed during the first pass of action (after the operation of the simple system's classifier) modulated by the error detection system. In order to calculate the total time required by the error-aware system to successfully interpret s actions, we recursively compute the previous equations substituting s with the number of actions that need to be repeated. This recursive computation leads to Eq. 13.
Finally, ICRT is defined to be the number of actions times the inverse of the already calculated total time ICRT = s/T (13) This approach presents a generic framework that quantifies the efficiency of a classifier-based system augmented by an error detection procedure. It can be easily shown that the sums of equations 7 and 13 are geometric series and converge when Acc or Re(c) does not equal to 0 [25].

EXPERIMENTAL SETUP 4.1 Experimental protocol
The experimental protocol relies on the SSVEP-based selection of five boxes in the context of a website with a flickering frequency of 60/9, 60/8, 60/7, 60/6 and 60/5 Hz respectively (these flickering frequencies are a result of the monitor's display rate of 60Hz) [30]. The participants were asked to select one of the five magenta boxes flickering at different frequencies via auditory indication. The boxes would flicker for 5 seconds, then they would stop and a preview of the box that was selected by the system was shown for 2 seconds to the participant by turning the magenta color of the selected box to green (Figure 1). In the case that the previewed box was not the same as the one the participant was asked to observe, we would expect to create an ErrP in the recorded EEG signal a few ms (200-800) after the preview. In order to maintain a similar ratio of correct and error trials for each participant, we opted for a predetermined classifier to select the boxes (with a ratio of 70% correct and 30% erroneous). The participants were not aware of the pseudo-random classifier so as to get a natural response to the unexpected erroneous trials.
Compared to the typical experimental protocols for the detection of ErrPs and despite the fact that the subjects were instructed to confine any eye movement during the SSVEP detection and the feedback stimulations, it is unavoidable that their attention is drawn towards the box that turns green. Thus, during erroneous actions only, there is an unintentional gaze shift towards the box that turns green (i.e. above, bellow, on the left or on the right). Nevertheless, Figure 1: Selection preview for the SSVEP-based interface this issue has been already investigated in [10] leading to the conclusion that the generated ErrPs are not affected by this occasional gaze shift.

Dataset
The signals were captured with the EBN cap (64 electrodes - figure  2) with a sampling rate of 128Hz. Five healthy subjects with prior SSVEP experience participated voluntarily in the study, all male, right handed and between 26-37 years of age with prior experience in SSVEP experiments. Each participant performed a total of 100 trials (20 trials per box), out of which 70 were correct and 30 erroneous. After the start of the selection preview, which lasted 2 seconds, the EEG signal was recorded in order to acquire the brain responses and then the system would redirect the user to the selected option so that the participant could move to the next trial.

Implementation details -Preprocessing
The following choices, regarding the data analysis, have been made. First, zero-phase filtering (1-13Hz so as to include delta, theta and alpha brain oscillatory activity) was applied on the signals. Since the reference electrode of the EEG device we used is near the area of interest the EEG signals have been re-referenced by subtracting the average over all electrodes from each electrodes for each time point, a procedure referred to as common average re-reference. For the ErrPs, the EEG signal between 0.2 and 0.8 seconds after the preview of the selection was used, since this is the most prominent time window for the ErrPs. For outlier detection the function robustcov in Matlab was applied with default parameters. To evaluate the spatial filtering methods 6 electrodes (AF3, AF4, F7, F8, FCz, CPz) were used (figure 2), located mainly over the frontal brain region so as to cover the anterior cingulate cortex. As the default classification method we employed Support Vector Machines (SVMs) using polynomial kernel of 3rd degree. The performance of SVMs (accuracy, precision and recall) was used to evaluate the contribution of each setting. All processing steps (outlier detection, spatial filtering, classifier ) were applied and tested by means of 10-fold cross validation, in a personalized manner, tailored for each participant separately. In order to calculate the spatial filters we treated the EEG as a collection of multiple 6-dimensional vectors (dimension equals to the number of used channels) ignoring the temporal information. Then these 6-dimensional vectors were considered as independent observations for input in the sCRP algorithm. Since sCRP is a computational intensive task during the training process the signals were subsampled, in order to reject redundant information as well as to reduce the training time.

Visual Representation
In figure 3 the average response across all trials for one subject and for the 6 utilized electrodes is depicted. The red line corresponds to the average signal of the erroneous trials, recordings that are expected to contain ErrPs, the blue line to the correct trials and the green line is their relative difference (erroneous minus correct). With visual inspection we can easily find out two peaks at about 350 and 450 milliseconds, only during erroneous responses, over the frontal area (F7,F8 and FCz). The observed contradiction in polarity (peaks at FCz come with a different sign) is justified by the common average re-reference procedure. Comparing the signals with the literature reports we observe that the two components are time delayed about 100 milliseconds which could be justified by the previous condition during which the brain was performing a completely different task and habituated [23]. Although it is clear that the two states (erroneous and correct) have major differences in their average forms, when signal-to-noise-ratio (SNR) is drastically increased, single trial analysis, which is our case, is by far more complicated.

Baseline Comparison.
In order to assess the benefit of the proposed method, that is supervised CRP, as a spatial filtering method we initially compare its performance with a baseline configuration, when no spatial processing is applied. Table 1 presents the average performance, across five participants, of the ErrP detection system. Results are presented for three different spatial configurations: FCz, all utilized electrodes and finally when sCRP is applied as a spatial filter. As we can see, in the first two rows, when the six channels are used the results are significantly better in terms of ICRT, compared to the results of the FCz which is the most prominent electrode for the ErrP detection. This observation servers as an indication that information, essential for ErrP detection, exists in multiple channels (at least those surrounding the area of interest) although it may not always be visible in their average forms. The third row corresponds to the results when sCRP is used as a spatial filter. The trade-off between the Re(c) and Re(e) is captured by the ICRT value which is increased in the sCRP case. We should note that the used Acc value for ICRT calculation is selected so as to match the accuracy of the SSVEP system during the data collection procedure.

State of the Art Comparison.
In order to further evaluate the contribution of sCRP we compare its performance with other, widely used, spatial filtering methods namely PCA [16], CSP [31] and xDAWN [22]. Actually CSP and xDAWN are the current state of art spatial filters in detecting synchronization-desynchronization events and event related potentials respectively [4,21]. Moreover the results of the classic CRP method are presented in order to examine whether supervision offers a positive or a negative effect.
Results of the compared spatial filtering methods are presented in table 2. Comparing the first two rows, that correspond to PCA and CRP, to the rest we observe that all the three supervised methods outperform the unsupervised having the ICRT as the performance criterion. Concerning the supervised methods (rows three, four and five), for the used ICRT configuration (t = 1 and e = 0.25 and Acc = 0.7) CSP performs slightly better while sCRP is marginally in the second place. Actually ICRT is highly sensitive to the accuracy of the SSVEP system. Figure 4, which provides a more accurate way for comparison of all the methods since a wide range of SSVEP Acc is taken into account, shows that as the accuracy of the SSVEP increases so does the efficiency of sCRP. Specifically, when the SSVEP system is able to perform at accuracies of 75% or better (which is the most probable scenario having in mind the actual performance of a SSVEP system) sCRP outperforms all other methods. Also the sCRP is clearly superior in terms of Pr(e) which is probably the most important evaluation score having the users' experience in mind. It should be mandatory for a system to avoid classifying correct actions as erroneous since this kind of miss-classification is expected to cause high frustration.

Assessing the contribution of Error-Related Potentials
Our next objective is to showcase the benefit of having an error detection system in a web-site BCI, with respect to the ICRT. Since the SSVEP system highly depends on the duration of the trial we    present figures that show the profit of our approach in three different scenarios that differentiate in time needed for the SSVEP detection to operate (e was set to 0.25sec in this case, which is the expected time for ErrPs to be elicited). In order to visualize the results, we plot the ICRT of the systems with respect to the accuracy of the SSVEP detection, which serves as an independent variable. The results can be seen in Fig. 5(a-c) for the three duration (t) of SSVEP. As shown by the figures, the ErrP system provides significant benefit in terms of ICRT in the case where the SSVEP-system accuracy is below a certain threshold, which depends on the SSVEP trial length (t). This point of change corresponds to the intersection of the simple SSVEP system (blue dashed line) and the SSVEP-ErrP system (green solid line). The presented results correspond to the sCRP method for the ErrP detection. Apart from the reliability of the both the error-agnostic and error-detection system one more parameter plays a critical role in the performance of an error-aware system. Unavoidably the errorawareness introduces a time delay during each action (in our case the system has to pause till the ErrPs are elicited). The shortest the duration a SSVEP systems needs to operate the more is affected by this delay. Even in the case of a perfect ErrP detector it should not be expected to positively contribute in any SSVEP system. Intuitively, the use of error detection (in our case it is realized by the ErrPs detection) is justified when three conditions are met: a) the accuracy of initial system (in our case an SSVEP interface) is not perfect b) the accuracy of error detection is significant and c) the introduced time-delay is relatively small compared to the time needed for the error-agnostic system to operate.

DISCUSSION
In this paper we investigated the contribution of ErrPs in a SSVEP system under realistic conditions. The presented proof-of-concept shows that the limitation in such combination, between ErrPs and SSVEPs, is bidirectional. System's utility is not only dependent from the SSVEP system accuracy but also from the capabilities of the ErrPs classifier. Although the evaluation was not operated in an online setting this does not limit the significance of the presented results.
In a more complicated interface, where links could flicker for a more user-friendly web browsing experience it is questionable even if these stimulations would be able to produce SSVEPs. This issue is beyond what this paper presents and examines. On the other hand we expect that the mechanism that produces the ErrPs would still be the same and hence the employment of ErrPs would still be beneficial under certain circumstances, that have been already discussed.
In order to enhance the performance of a SSVEP-based BCI we had to employ more sophisticated algorithms for spatial filtering, more appropriate to the nature of EEG signal. The supervised variant of collaborative representation projections was proved to be, in most cases, the best choice in order to increase the classification accuracy of the error detection system. Although the sCRP served as an invaluable tool in our research, there is much more work that needs to be done before it becomes an asset in the multivariate signal analysis. The performance of the proposed method, and hence the overall BCI performance, was evaluated by the ICRT, a generic framework that can easily be adapted in any error-aware system.