A New Motion Illusion Related to the Aperture Problem

A previously unreported motion illusion is described. Oblique lines that drift smoothly on the retina in a vertical direction appear to be displaced laterally. The effect occurs both for moving lines under fixation and for stationary lines under ocular tracking of an external target. Orientation, length, and homogeneity of the obliques affect the magnitude of illusory displacement. We propose that this illusion is associated with a misregistration of the direction of displacement occurring, in lines slanted relative to the axis of their motion, because of the aperture problem.


Introduction
We have observed a compelling motion illusion. If one tracks a dot that moves vertically between two stationary oblique lines, the lines appear to be displaced laterally. They seem to drift inwards when the dot moves towards the convergence point, as in figure la, and outwards when the dot moves away from it, as in figure lb. If the lines are parallel and slanted counterclockwise from vertical, as in figure lc, they appear to drift solidly rightwards when the dot moves downwards, leftwards when the dot moves upwards. Two lines make the effect stronger, but the illusion can be appreciated even with a single line, as in figure Id. In all cases, it is hard to say exactly whether the direction of illusory motion is horizontal or orthogonal to the orientation of the lines.
(c) (d) Figure 1. Different stimulus configurations that give rise to illusory lateral displacement of the obliques. The solid vector represents physical motion; the dashed vectors represent perceived motion. The precise direction of perceived motion is not always clear; it is here shown as horizontal because it has a strong horizontal component.
Though the effect is robust enough to be seen consistently in fairly uncontrolled conditions, as far as we have been able to determine it has not been reported before. In this paper, we present some investigative work on the phenomenon and discuss its relevance for current theories of motion perception.

General method
The method described below was used in all the experiments in which the obliques were stationary and the eye followed a moving target (ie with the exception only of experiment 2). Stimuli were created with an Apple Macintosh LC III computer and displayed on its 14 inch screen. Each stimulus consisted of a pair of black lines (0.12 cd m" 2 ) converging downwards, as in figure lb, and presented on a white background (67cdm~2). Line width was always 2 min of visual angle, while length, orientation, and reciprocal position of the lines varied across experiments. Stimuli were presented in the centre of a window whose width and height were a constant ratio of the horizontal and vertical projection of the lines, respectively, to avoid a confound related to the observation that stimuli elicited weaker illusions when shown on larger fields. Apart from the white window, the screen was black. At the start of each trial a black circular spot (0.3 deg in diameter) appeared at the bottom of the window, between the lines, moved vertically upwards until it reached the top, then reappeared at the bottom and went through the same cycle ten times. To aid smooth tracking, the starting and ending positions of the spot were slightly lower and higher, respectively, than the bottom and top endpoints of the lines, so that the length of the path covered by the spot was always longer (by a factor of 1.2) than the vertical projection of the lines. For the same reason, at the beginning of each cycle the spot was stationary for half a second before its motion began.
The stimuli were presented one at a time in a random order, separated by 30 s intervals. Each stimulus was presented twice in experiments 1, 4, and 5, and four times in experiment 3. Viewing was binocular and from a distance of about 90 cm, in a dimly lit room. Subjects, who were free to move their heads, were asked to track the moving spot and to rate, at the end of each trial, the effect of lateral motion of the lines on a 5-point scale going from 0 (no effect) to 4 (very strong effect).
The distance between the lower (closer) endpoints was 1.27 deg. The vertical projection of each line was kept constant (3.35 deg) across orientations; therefore, line length covaried with orientation, and was respectively 3. 51, 3.64, 3.90, 4.39, 4.86, and 6.79 deg for lines oriented 10°, 20°, 30°, 40°, 45°, and 60° from vertical. We chose to keep constant the vertical projection of each line rather than its length because the section of the tracking-spot path effective in producing displacement would have been much too short for more-tilted lines. The constant ratio between window size and stimulus size was 1.56 in width, 1.14 in height. The tracking spot moved at a speed of 3.12 deg s" 1 .
Five naive subjects and one of the authors took part in the experiment. All had normal or corrected-to-normal vision.

Results
As can be seen in figure 2, the illusion is at its best for lines tilted 10° to 30° from vertical, and decreases steeply as the orientation of the lines deviate further from the axis of pursuit. The effect of line orientation on the strength of the illusion is statistically significant {F 5>25 = 13.11, p < 0.0001).
We wondered whether the effect was specific to stationary lines observed under conditions of ocular pursuit. To address this question we compared the effects of ocular tracking of an external target (with stationary lines) and fixation of an external target (with moving lines) for various orientations of the obliques. This experiment is reported below. 4 Experiment 2: Effect of line orientation-fixation versus ocular tracking 4.1 Method Stimuli were created with an Apple Macintosh Ilfx computer and displayed on its 21 inch screen. The orientation of the lines was either ±10°, ±20°, ±30°, ±40°, ±45°, ±50°, or ±60° from vertical. A control stimulus with perfectly vertical lines was also used. The distance between the lower (closer) endpoints was 0.69 deg. The vertical projection of each line was kept constant (5.88 deg) across orientations; therefore, line length covaried with orientation, and was respectively 6. 06, 6.33, 6.88, 7.76, 8.40, 9.23, and 11.76 deg for lines oriented 10°, 20°, 30°, 40°, 45°, 50°, and 60° from vertical. Since half of the stimuli consisted of moving lines, no proportional window was employed in this experiment.
Two different conditions were run. In the first (pursuit condition), the tracking spot moved at a speed of 4.92 deg s" 1 . This part of the experiment was basically a replication of experiment 1. In the second (fixation condition), the tracking spot was replaced by a stationary, circular fixation spot (0.3 deg in diameter) presented in the centre of the screen. At the start of each trial the converging lines appeared at the top of the screen, just above the fixation spot, and moved vertically downwards, at a speed of 0.41 deg s" 1 , until they reached the bottom of the screen; they then reappeared at the top and went through the same cycle ten times. The stimuli for the two conditions were mixed and presented in a random order, separated by 30 s intervals, within a single session. The rating-scale system was the same as that employed in the other experiments. Each subject participated in four sessions, divided in two blocks of two sessions each. The two authors and two naive subjects took part in the experiment. All had normal or corrected-to-normal vision.

Results
Even when the vertical (0° condition) lines are omitted from the analysis, line orientation has a significant effect on illusion magnitude in both conditions {F 618 = 9.89, p < 0.0001 for the pursuit condition, and F 6fl8 = 18.41, p < 0.0001 for the fixation condition). As can be seen in figure 3, however, the two curves do not overlap. When the obliques are stationary and the eye follows a moving spot, the illusion is strongest for lines oriented 10° to 30° from vertical and rapidly approaches zero for more-tilted lines. These findings replicate those of experiment 1. When the obliques move and the eye fixates a stationary spot, however, the illusion is still quite remarkable for slants up to 50°.
Overall, mean ratings are lower in the pursuit condition. Though this can be taken to mean that, in the experimental setup described here, one condition produced stronger effects than the other, one should not infer that the pursuit condition entails weak effects in general. Both in this and in the previous experiments, we were forced to use relatively short lines, either because longer lines would not have had enough space to move across and/or because longer lines would have touched the borders of the screen when tilted. If longer stationary lines are employed, their apparent lateral displacement under tracking of a spot is in fact impressive.
The results, then, show that the illusory lateral-motion effect is observed both when the obliques are stationary and the eye moves, and when the obliques move and the eye keeps still. The findings in experiment 2 also show that the lines have to be tilted relative to the direction of their displacement. No lateral-displacement effect is observed with lines drifting in the direction of their elongation (0° orientation in figure 3). The results of both experiments 1 and 2 indicate that the strongest illusion is observed for lines that form a small angle with the direction of their displacement, and that the effect virtually disappears for angles exceeding 60°.
In the experiments described above, line length and orientation covaried: as the lines became more tilted they also became longer. Line separation also covaried with line orientation. Since the minimum separation (ie the distance between the closer endpoints of the lines) was kept constant, the maximum separation (ie the distance between the farther endpoints of the lines) increased as the lines became more tilted. Variations in the magnitude of the illusion may then merely reflect differences in line length or separation, rather than in orientation. To address this issue, we decided to keep the orientation constant while varying separately line length and line separation. The experiment is reported below.

Method
In this experiment the length of the lines was either 2, 4, 6, or 8 deg, and the distance between the lower (closer) endpoints was either 0.5, 2, 4, or 6 deg, making a total of sixteen different stimuli. The orientation of all lines was ±20° from vertical. The constant ratio between window size and stimulus size was 1.21 in width, 1.34 in height. The tracking spot moved at a speed of 4.64 deg s" 1 . The stimuli were presented in a random order, separated by 30 s intervals, within a single session. Each subject participated in four sessions, divided in two blocks of two sessions each. Six naive subjects and one of the authors took part in the experiment. All had normal or corrected-to-normal vision.

Results
Though in our experiment we manipulated minimum line separation, maximum line separation varied proportionally for each length of the oblique. In figure 4, we chose to plot illusion magnitude as a function of maximum (ie distance between the farther endpoints) rather than minimum line separation. As can be seen in figure 4, the strength of the effect increases as line length increases (F 318 = 107.39, p < 0.0001) and line separation decreases (F 31S = 5.46, p < 0.01). The interaction between these two factors is also significant (^54 = 2.13, p < 0.05), reflecting the failure of line separation to reach significance for the shortest lines {F 31S = 1.43, ns). However, the flattening of the curve for the 2 deg lines may well be an artefact due to the circumstance that, for lines so short, the effect is very weak even at the smallest separation, and subjects only used the two lower points of the rating scale. Though the effect of line separation is clear only for long lines, it can be seen from the figure, first, that the closer the lines are to the tracked spot the better the illusion is, and, second, that the illusion is still strong for lines quite spaced apart.
It must be said that we are aware that our experiment settles the issue of the effect of line length and separation only in part, since we tested it for just one orientation. We made, however, systematic informal observations, and though we cannot rule out the possibility that orientation somehow interacts with line length and separation, the effect never gets stronger for shorter or more-separated lines, at any line orientation. These data, then, besides indicating a clear effect of line length and separation on the magnitude of the illusion, also answer the question raised by their covariation with orientation in the previous experiments. In experiments 1 and 2, the apparent displacement strength was found to increase and then decrease as the lines became longer, whereas it should have increased steadily if it were reflecting mere variations in length rather than in orientation. (Actually, the longest lines we used in experiments 1 and 2 entailed virtually no effect: see 60° orientation in figures 2 and 3.) A similar reasoning applies to the problem of the covariation between maximum line separation and orientation in the previous experiments: the illusion increased and then decreased steeply as the farther endpoints of the obliques became more separated, whereas it should have remained constant (for short lines) or decreased gently and continuously (for long lines) if it were reflecting mere variations in separation rather than in orientation. In addition, the results of experiment 3 show that the illusion is still quite remarkable for widely separated lines (oriented 20°), whereas in experiments 1 and 2 we used lines of similar length and separation (but different orientation) that produced virtually no effect. Compare, for instance, the data point in figure 3a for the 45° orientation, corresponding to lines 8.40 deg long whose farther endpoints are spaced 12.6 deg apart (rating 0.38), with the rightmost data point in figure 4, corresponding to lines 8 deg long whose farther endpoints are spaced 12 deg apart (rating 2.46). It can be safely concluded, then, that the magnitude of the effect is separately modulated by line orientation, line length, and line separation.
In the experiments described below, we dealt with the relevance of another property of the obliques: homogeneity. The homogeneity issue was addressed in two completely different ways: by placing stationary marks on the obliques, and by providing crossing points between the obliques and a set of additional lines.
6 Experiment 4: Effect of line nonhomogeneity-additional spots 6.1 Method In this experiment we manipulated line homogeneity by placing stationary spots with a diameter of 0.3 deg on the lines. There were four different conditions: no spots (control condition), two spots (at the line endpoints), three spots (two at the line endpoints and one in the middle), and five spots (two at the line endpoints and three uniformly spaced in the middle).
The orientation of the lines was ±20° from vertical and their length was 10.19 deg. The distance between the lower (closer) endpoints was 2.42 deg. The constant ratio between window size and stimulus size was 1.56 in width, 1.14 in height. The tracking spot moved at a speed of 4.64 deg s _1 .
Five naive subjects and one of the authors took part in the experiment. All had normal or corrected-to-normal vision.

Results
As can be seen in figure 5, the strength of the effect decreases steadily as the number of spots increases (F 315 = 49.48, p < 0.0001). With the five-spot lines the illusion virtually disappears.
The results of the experiment, then, show that the lines have to be uniform and continuous. The illusory displacement effect is severely impaired when nonhomogeneous lines are used. (Incidentally, we obtained the same results with dashed or otherwise interrupted lines.) It is interesting to note that subjects almost unanimously reported, for the spotted lines of the experiment reported above, a qualitative change in the effect. While the spots looked stationary, the sections of line between a pair of spots appeared to 'bow', as if the spots were anchoring the lines but could not completely prevent their displacement.

Experiment 5: Effect of line nonhomogeneity-additional lines 7.1 Method
In this experiment we manipulated line homogeneity by placing a set of seven horizontal lines on the obliques. The width of the additional lines was the same as the obliques. They were 9.77 deg long, parallel, and separated vertically by 0.67 deg. We ran three different conditions. In the first, control condition, there were no additional lines. In the second the lines were stationary and located exactly in the middle of the window, so that they intersected the obliques in fourteen points, seven for each oblique. In the third condition the lines moved together with the tracking spot, in the same direction and at the same speed; the tracking spot was placed in the centre of the middle line.
The orientation of the lines was ±20° from vertical and their length was 10 deg. The distance between the lower (closer) endpoints was 2.4 deg. The constant ratio between window size and stimulus size was 1.56 in width, 1.14 in height. The tracking spot moved at a speed of 4.64 deg s" 1 .
Six naive subjects and one of the authors took part in the experiment. All had normal or corrected-to-normal vision.

Results
Compared with the control condition (mean rating = 2.5), the illusion decreases when the additional lines are stationary (mean rating = 1.14) and increases when the additional lines move with the tracking spot (mean rating = 3.36). Both differences are significant (F 16 = 90.25,/? < 0.0001, and F 16 = 13.09,/? < 0.01, respectively).

Discussion
Can the effect we report here be regarded as a rather straightforward case of relative motion? As the eyes (or the lines) move, in fact, the distance between the lines changes over time. The apparent displacement of the lines away from each other may then correspond to the actual (retinal) motion of the lines relative to the tracked (or fixation) spot, acting as a frame of reference. Some of our findings are at odds with this account, however. The extent of actual relative displacement is a direct function of the orientation of the obliques (with the most-tilted obliques resulting in the largest displacement), while the strength of the effect is not (with the most-tilted obliques resulting in the smallest illusion, as shown in experiments 1 and 2). Even more importantly, we see no reasons why the impression of relative displacement should be disturbed by the presence of identifiable features on the lines, whereas this destroys the illusion (as shown in experiment 4).
We think that this illusion may provide further cues about the way in which the direction of motion of translating objects is recovered by the visual system. Consider first the case in which the lines are actually moving downwards. This displacement is veridically recorded at the line endpoints, which specify downward motion when the lineAB moves to the position A'B' (see figure 6). However, at locations other than the line endpoints, such as C, the direction of displacement is ambiguous. Since no point on the line is distinguishable from any other (where is C'?), any motion component parallel to the line remains undetectable; only displacement orthogonal to its contour can be picked up. (This indetermination, known as the 'aperture problem', occurs necessarily whenever motion-detection operations are to be performed over a limited area of a scene, which is indeed the case in the human visual system.) Thus the line endpoints signal downward motion, while the rest of the line signals orthogonal motion.
It has been suggested that, in a moving line, the translation signal (extracted at line endings) and local velocity signals (normal to the orientation of the line) are integrated with different relative weights. This would account for observed biases in the perception of line speed  and direction . The integration idea may be formalised by assuming that perceived motion corresponds to vector V a , which is the weighted average of vectors V T , representing the true translation motion of the line, and V L , representing the local component normal to the line (see figure 7). The crucial point here is that this average vector exhibits a horizontal component which is absent in the true motion vector. It may come as no surprise, then, that the lines will appear to translate downwards and, at the same time, be displaced laterally away from one another. This is indeed what we observed in experiment 2.

Effect of line orientation
The finding that the tilt of the obliques has a marked effect on the magnitude of illusory displacement suggests that differently oriented lines entail different average vectors. As can be seen in figure 7, this is in fact the case. As the orientation of the obliques approaches the horizontal, J£ gets closer to V T ; correspondingly, both its magnitude and its direction change. We found that the lateral displacement effect was strongest for lines oriented 10° to 30°. The average vector length increases continuously for orientations of the obliques from 0° to 90°; it follows that the effect cannot be related to vector magnitude per se. As to direction, the horizontal component of the average vector increases from 0, for perfectly vertical lines, up to a maximum value for lines tilted 45°, and decreases thereafter. Thus the effect is not simply proportional to the magnitude of the horizontal component either. On the other hand, the angular deviation of the average vector from horizontal increases as the orientation of the oblique increases. If what observers perceive is in fact the direction of the average vector, then it makes sense that the illusory horizontal displacement be stronger for small orientations of the obliques.

Ocular tracking vs fixation
The kind of retinal motion information provided by our displays in conditions of ocular pursuit is similar to that occurring when the lines are actually moving. When one tracks the upward-moving spot, its image is displaced very little, if at all, on the retina. The image of the oblique lines, however, moves in the opposite direction, with the velocity that is needed to track the spot. In either case, then, the obliques are sliding downwards on the retina. However, when the eye simultaneously moves upwards (like in our pursuit condition), efferent signals are supposedly used to compensate for retinal displacements. Oculomotor compensation, in simplified terms, is equivalent to subtracting a vector corresponding to the eye movement, say V E , from the vector associated with the object movement (see figure 8). In our case then, if we suppose that the vector associated with the line movement is the average vector V a , the vector corresponding to the final percept will be a residual vector that we call V T , such that V T = V a -V E . If the magnitude of V B corresponded exactly to the velocity of pursuit, the direction of V T would be upwards and perfectly parallel to the line (along the line, in fact). Yet, there is overwhelming evidence (reviewed, for instance, in Mack 1986) that pursuit velocity is underregistered. This implies that only some fraction of the actual eye movement will be subtracted, and that the residual vector may have a direction close to horizontal (outward) supporting the perceived lateral displacement. Note that the exact direction (and magnitude) of V T will depend both on the orientation of the line and on the amount of underregistration of pursuit velocity. Under the conditions of our experiments 1 and 2, the combination of these two factors yielded strongest effects (that is, possibly, a residual vector departing from horizontal the least) for lines tilted 10° to 30°. Incidentally, the hypothesis that under tracking the illusion may be based on the underregistration of pursuit velocity is consistent with an observation that had been puzzling us for a long time: the effect seems to grow stronger and stronger as the tracked spot goes repeatedly through the same cycle. (This does not happen under fixation, where the illusion simply repeats itself over cycles.) Interestingly, it has been reported (Miller 1980) that repetitive pursuit produces a remarkably stronger (by about a factor of three) underregistration than nonrepetitive pursuit.
It should be pointed out that the velocities used in the tracking and fixation conditions were very different (one greater than the other by a factor of twelve). This was done to allow optimal, or close to optimal, effects for each condition, on the basis of a pilot experiment in which we tested the effect of varying speed. As it happens, we observed that the strongest illusions were obtained at extremely slow speeds when the lines moved (the slower the better, provided, of course, motion was still above threshold), and at extremely high speeds when the spot moved (the higher the better, provided, of course, the spot could still be tracked). A detrimental effect of low speed in recovering the actual speed of a tilted moving line has also been found, albeit in rather different conditions, by Castet et al (1993). These authors speculated that as speed is lowered local orthogonal signals become stronger than the translation signal. While the observations we made in the fixation condition confirm and extend their data (they used two speeds only), the reversal of the effect of speed under pursuit seems rather puzzling. We cannot offer any obvious explanation for this difference, apart from suggesting that it might be a consequence of the fact that the residual (underlying perceived displacement in our tracking condition) and the average (underlying perceived displacement in our fixation condition) vectors do not coincide. Note, in particular, that the residual vector is always shorter than the corresponding average vector. Thus, velocities that result in good effects under fixation might simply be much too slow to provide a sufficiently long vector under tracking. Figure 8. The residual vector V x (in grey) is obtained by subtracting vector V E (which is an arbitrarily large fraction of the vector corresponding to the true eye movement) from vector V a . See text for details.

Effect of line length
The influence of line length on the magnitude of illusory displacement is consistent with the vector explanation we are suggesting. The longer the oblique, the stronger the tendency to default towards the local orthogonal signal; in other words, the more weight is given to V L , the more the average vector V a 'slides' towards V^ (and away from the true motion direction; see Castet et al 1993). A detrimental effect of line length on the accuracy in recovering the direction of low-contrast, briefly presented moving lines has been in fact observed by Lorenceau et al (1993), and, in somewhat different conditions, by Gerbino and Bruno (personal communication). Though a smaller influence of V T (owing to the terminators being more peripheral for longer lines) could not be ruled out, Castet et al (1993) suggested that the increased relative salience of V L for longer lines may follow from a process of spatial pooling of the orthogonal signals along the line. Our data on the effect of line separation (experiment 3) indicate that, in fact, the eccentricity of terminators cannot be the whole story. For widely separated short lines, all terminators were more peripheral than for long lines close together; nonetheless, the illusion was much larger for the latter.
It must be pointed out that in Lorenceau et al (1993) the bias towards orthogonal readings only occurred at low contrasts and for very short presentation times, decreasing with longer durations and completely disappearing with a duration of about 500 ms. In our case, however, the apparent displacement is observed at full contrast and for much longer presentation times, and, if anything, becomes more conspicuous as the presentation time increases. If the bias reported by these authors and our motion illusion are supported by the same mechanisms, as seems reasonable, a likely explanation of this difference could be that our lines were much longer. The longest lines they used subtended 1.76 deg, whereas we typically used lines ranging between about 4 and 10 deg, for which the relative weight of the signals extracted at the line endpoints must be much smaller. Our data, then, may suggest that the duration and contrast constraints can be overcome when lines are very long.

Effect of line homogeneity
From the point of view we are adopting here, line homogeneity would of course be essential. For motion parallel to the line to be undetectable, thus leaving, at locations different from the endpoints, the orthogonal component only, it is crucial that no identifiable features appear on the line itself. In figure 6, marking C will specify unambiguously the position of C (defining, in this case, a vector of the same direction and magnitude as those corresponding to AA' and BB'), eliminating the basis of the illusion altogether. Perhaps not surprisingly, a weaker form of the illusion can still be observed if the spots are sufficiently far from each other. In this case the line seems to bow at one or more points, that is, it undergoes some deformation. This should be considered, we think, an additional instance of figure that appears as nonrigid under conditions of simple translational motion (see Nakayama and Silverman 1988).
The presence of additional horizontal lines moving with the tracked spot clearly enhances the effect. Our explanation implies that they may do so by providing identifiable features (the crossing points) that displace outwards. Outward vectors are unambiguously specified when, on the retina, the oblique moves downwards while the horizontal line is stationary. In this case, which corresponds to ideal tracking conditions, the vectors at the crossings are in fact parallel to the lines, ie perfectly horizontal. When the additional horizontal lines are stationary, on the contrary, the illusion decreases. Note that, in this condition, the crossing points displace vertically downwards, specifying vectors of the same direction and magnitude as those corresponding to the line ends.
It is routinely claimed that motion signals coming from distinctive features such as line endpoints (terminators) will enjoy a special status, by virtue of which they will be automatically attributed to the rest of the object-thus solving any local ambiguities. If things were that simple, the illusion we report here would not exist. It is possible, though (as suggested by Hildreth 1984), that the motion of the endpoints does constrain the motion of the rest of the line, but that the propagation of this constraint is spatially limited. This is in fact what would seem to be suggested by our experiment 4 (nonhomogeneous lines).
One secondary aspect of our findings requires a comment. Shimojo et al (1989) have demonstrated that motion of extrinsic (ie accidental because created by occlusion) terminators does not contribute to the perceived direction of a moving grating. Vallortigara and Bressan (1991) have extended this finding to terminators corresponding to the crossings of two superimposed gratings (nodes). In our experiment 5, the nodes produced by the overlapping between the moving horizontal lines and the stationary obliques were obviously extrinsic, but affected the strength of illusory displacement nonetheless. Whether this means that extrinsic terminators can, after all, be used to determine the direction of motion is a question that will be evaluated in a future study.