Abstract
Color and shape are important media for information perception, cognition, judgment, and decision making. However, the combined effects of color and shape on cognitive ability have not been widely investigated. In this paper, the perceptual load paradigm is used to investigate the combined effect on icon cognition. Thirty-four healthy subjects (17 males and 17 females, aged 20–26 years, mean = 24, and SD = 4.51) participated in three behavioral experiments using an in-subject design, in which response time and accuracy data were recorded. In a redesigned behavioral experiment, EEG data were recorded from another 20 healthy subjects (10 males and 10 females, aged 20–26 years, mean = 23, and SD = 4.23). Experiment A used color block, graph, and color graph combination icon as the target stimulus. The results indicated that the combination effect was insignificant at high cognitive load, while the combination effect of color and shape at medium and low cognitive loads promoted icon cognition. In experiment B1, both positive and inverted shapes were used as target stimuli, in which the inverted shapes hid the semantic meaning of the shapes. The results showed that the semantic meaning in the combination effect significantly improved cognitive performance at medium and low cognitive loads. In experiment B2, shapes of different complexity were used as target stimuli. The results showed that reducing shape complexity improved cognitive performance with a single color, but shape complexity had no significant effect on cognitive performance with multiple colors. Experiment C used EEG experiments to verify the results of behavioral experiments. The activation degree of combination icons in the occipital lobe was higher than that of shape icons, indicating that the cognitive mechanism of combination and shape icons was not completely the same. Our study shows that the combined effect of color and shape significantly improves cognitive performance at moderate-to-low cognitive loads, in which color and semantic information play a key role.
1. Introduction
The combined effect of color and shape refers to the reasonable conversion of information in graphics with both color and shape to make them more acceptable to the observer’s mind, improve their cognitive performance, and reduce their cognitive load. For example, in the field of fighter aircraft design, red and yellow icons with warning signs [1] capture attention more easily than icons in other colors so that the pilot can perceive, judge, and make decisions faster. On the instrument panel, icon buttons with different colors affect the operator’s cognitive performance [2]. Important text or graphics marked with special colors enable users to filter and process information more quickly when browsing web pages [3]. Shapes with different colors are seen to produce different degrees of cognitive performance in the user. However, in addition to the individual effect of color and shape, as to the question whether the combination of color and shape affects people’s cognitive performance, the current research has not reached a consensus. Some opinions even differ significantly.
Previous researchers in the field of visual cognition [4] found that the salient features of color and shape can guide bottom-up attention [5]. Scholars have conducted multifaceted research on the influence of color and shape on cognitive performance [6]. In terms of color, many studies have shown that a hierarchical visual perception of color affects people’s search performance and the priority in capturing information [7]. Nuthmann et al. [8] improved the visual search performance of real scenes by using color coding according to the location of the visual area. In terms of shapes, previous researchers have studied the influence of shapes on visual attention from semantics, complexity, concreteness, and structural features [9]. For example, in the experiment of Michalski et al. [10], the participants were asked to select a target object from a group of items (made up of English letters and Arabic numerals) randomly placed on the computer screen during the visual search process, and the participants’ reaction times were then recorded. The follow-up analysis showed that the geometrical factors of the graphical structures played a more important role in the participants’ visual search and operation efficiencies. Luo and Zhou [11], through a similar experiment process, found that a single shape led to better search performance than the combined ones did. To further study the effect on cognitive behavior when color and shape coexist, Zovko and Kiefer [12] performed EEG experiments and studied participants’ attention to objects with both shape and color. In the experiment, target images were presented. In the color induction task, participants had to determine whether the object was red or blue as quickly and accurately as possible; while in the form induction task, they had to decide whether the presented object was circular or elliptical. The result showed that the visual movement of shapes would lead to an electrophysiological reaction in micro-cognition but color would not, and semantic meaning played a key role in the process. However, Adamo et al. [13] used ACSs (attentional control settings) in their experiment, in which participants were instructed to select the target color on one side of the screen (color area) and select the target shape on the other side (shape area), ignoring the nontarget color and shape. They compared the relative efficacy of ACSs from two different feature categories (shape and color) that were associated with different levels within the visual processing hierarchy and found that colors captured attention more easily than shapes. Some scholars found that the cognitive mechanism of color and shape would change at different loads [14, 15]. Lavie and Cox [16] used three levels of time pressure (high load: 200 ms; medium load: 600 ms; low load: 1000 ms) to control cognitive load and employed single-factor variation to detect the cognitive mechanism of color and shape at different loads. They found that, within 1000 ms, color had a greater effect on cognitive capacity and could lead to faster cognition than shapes could. Rokszind et al. [17] controlled cognitive load by adjusting the brightness, saturation, and hue of images and found that excluding colors did not affect the classification of complete stimuli and that colors were recognized faster than shapes when cognitive performance was low. Qian et al. [18] expanded the research scope and probed into the combined effect of color and shape. They analyzed and compared the results from single and multiple-feature items in low and high-resolution conditions while controlling the number of to-be-remembered features. Results showed that the required cognitive capacity for color, shape, and their combined icons was roughly the same, but the cognitive capacity for color exceeded that for shape and the combined icons in the low-resolution condition.
However, current research achievements mainly focus on the effect of color and shape as visual stimuli on the cognitive mechanism. Few studies have been conducted on the combined effect of color and shape and the basic attributes in the combined effect on the cognitive mechanism. Using functional magnetic resonance imaging, Parra et al. [19] and Luria and Vogel [20] asked subjects to remember visual arrays of abstract shapes with single and combined features while undergoing fMRI. After an unfilled delay, they were presented a second array and asked to judge whether the features presented in the two arrays were the same or different. The results showed that the temporary memory of the combined features of color and shape and single features depended on different brain regions and indicated that the combined and single features had different cognitive mechanisms. By combining independent component analysis with the shortest-distance clustering method, Luo and Zhou [21] found that the brain region activated by icons with both color and shape (hereinafter referred to as “combined icons”) displayed characteristics of feature separation and local integration. The above study results establish a theoretical foundation for the feasibility of the combined effect of color and shape. Lee et al. [22] showed that the priority of color and shape would be adjusted according to interference factors in the combined effect. However, in most cases, color took precedence over shapes in capturing perception. Many studies have also found that color, in the combined effect, was better than shape in improving search performance [23].
To sum up, in terms of research objects, previous studies on the combined effect mainly focused on the influence mechanism of the single feature of color or shape in a combined icon or the priority given to color or shape at different loads. However, there were few studies on the combined effects of combined icons on cognitive performance from the perspective of their combined features or the effect of their basic attributes at a deeper level. In terms of the experiment, previous experimental materials cannot effectively separate visual features and semantic information. Therefore, it is difficult to study the independent role and mechanism of the two factors in a combined effect. In this paper, the experimental materials were optimized and the bottom-up visual features were desirably separated from the top-down semantic information in combined icons. This paper adopted the classic perceptual load paradigm [24–28] to study the influence of the combined effect on icon cognitive performance at different cognitive loads and revealed the influence mechanism of basic shape attributes (semantic information and shape complexity) on icon recognition. In experiment A, the combined effect of color and shape conveying some specific meaning on cognitive performance at different cognitive loads was explored. In experiment B1, inverted shapes without semantic meaning were used to explore the role and influence mechanism of semantic information in icon cognition in combined effect. In experiment B2, the cognitive processing mechanism of shape complexity in combined effect was explored by using inverted shapes without semantic information. Experiment C was based on the cranial nerve mechanism of exogenous and endogenous attention, using EEG experiments to verify the conclusions of behavioral experiments.
2. Experiment A: Combined Effect Experiment
2.1. Method
2.1.1. Participants
Thirty-four college students (17 males and 17 females, aged 20–26 years, mean = 24, and SD = 4.51) participated in this experiment. All participants had normal or corrected vision without any color blindness or weakness. None of the participants were informed of the experimental intent or ever participated in similar experiments before. They got paid after the experiment. The participants in experiments A, B1, and B2 were the same.
2.1.2. Materials and Design
The experimental materials were displayed in the middle of the experimental screen. The color materials were composed of 10 hues and white as shown in Figure 1 extracted from the Munsell phase ring. The icon materials (Figure 1) consisted of 12 common icons with clear meanings selected from the icon library. Each icon took the full space of a 15 × 15 mm2 square. They were presented in a rectangle with a size of 72 mm × 54 mm (length × height; the angle of view was ±4.1 horizontally and ±3.1 vertically to the line of sight with the sight distance of 500 mm). The experimental materials were divided into three types of icons (Figure 2), which were combined icons (with both color and shape), shapes (with no color but clear meaning), and color blocks.
(a)
(b)
The experiment was conducted in the human-factor laboratory of the China University of Petroleum with normal indoor lighting conditions (40 W fluorescent lamp). E-prime was used to develop the experiment program. The interface background was black. The distance between the subject and the center of the screen was 490 mm–540 mm. The stimuli were presented on a 20-inch screen with a brightness of 92 cd/m2 and a resolution of 1680 × 1050 dp.
2.1.3. Procedure
The display time of the search set is divided into three levels: 200 ms at high load, 600 ms at medium load, and 1000 ms at low load (hereafter referred to as high/medium/low cognitive load, respectively).
Before the formal experiment, the quantity of experimental materials to be presented was determined by preexperiment. The number of icons was set at 2/4/6/8, and the 3 cognitive loads (high/medium/low) × 3 icon types (combined icons/color blocks/shapes) × 4 quantities (2/4/6/8) within-subjects design were adopted in the preexperiment. A significant difference was found between the error rates of less than four icons and more than six icons (), indicating that the quantity of six icons was too much for the cognitive ability of the participants. Therefore, the number of materials for experiments A, B1, and B2 was determined to be 4.
The 3 cognitive loads (high/medium/low) × 3 icon types (combined icons/color blocks/shapes) within-subjects design were adopted in experiment A. In the same icon type, the icon would not appear repeatedly. The experiment procedure is shown in Figure 3. The experimental stimulus was presented on black background. After reading the prompts, participants pressed any button on the keyboard to start the formal experiment. First, the gray gaze point “+” appeared in the center of the screen for 500 ms, and then the search set popped up. The search sets were designed to pop up randomly with equal chance and displayed for 200 ms, 600 ms, and 1000 ms, respectively. The stimuli were presented 500 ms after the search set disappeared, and the participants were required to search for the stimulus icon in the search set and to make judgments as rapidly and accurately as possible. If the stimulus appeared in the previous search set, the participants would press either “A” or “L” button on the keyboard. The gaze point screen of the next trial started after the subject responded or 3000 ms after no response was made. There were 360 trials in the formal experiment in which three types of shapes appeared randomly with an equal chance at three types of loads. Before the formal experiment, the participants were required to engage in 20 trials. The participants maintained their concentration during the experiment. After 120 trials, the participants rested for 1 min. The entire experiment lasted approximately 20 min.
2.2. Results
The reaction time (RT) data longer than 3 s and shorter than 0.2 s were excluded by the traditional method for excluding cognitive load data [16]. Hence, 1.6% of the data were excluded. The repeated variance analysis (ANOVA) of 3 (cognitive load: high/medium/low) × 3 (icon types: combined icons/color block/shape) was carried out for the average reaction time and the error rate. At different cognitive loads, the RT and error rate for three icon types are shown in Figure 4. The result showed that the main effect of the cognitive load was significant (F[2,33] = 45.50, ); the main effect of icon type was significant, (F[2,33] = 60.78, ), indicating that both cognitive load and icon types significantly impacted the cognitive performance. This finding was consistent with previous studies.
(a)
(b)
To further compare the cognitive mechanism of the combined effect at different cognitive loads, the RT and error rate for the three icon types at different cognitive loads were compared. The results showed the following. (1) There was no significant difference between the combined icons (875.53 ms) and color blocks (867.62 ms) in the RT at high cognitive load (), both of which were significantly faster than those of the shapes (927.26 ms, ). There was no significant difference between the combined icons (23.9%) and color blocks (26.2%) in the error rate (), both of which were significantly lower than those of the shapes (35.7%, ). It could be seen that the cognitive performance for the combined icons and color blocks was better than that for the shapes, while the effect of the combined icons and color blocks on cognitive performance was insignificant. From this, it could be inferred that at high cognitive load, color was the key factor in cognitive coding and that the role of shape information was insignificant. (2) At medium cognitive load, no significant difference was found in RT among the three icons. However, the error rate differed significantly. The error rate of combined icons (18.2%) was significantly lower than that of color blocks (23.5%) and shapes (28.8%, ), indicating that combined icons significantly improved accuracy at medium cognitive load. (3) At low cognitive load, the RT of combined icons (767.52 ms) was significantly shorter than that of color blocks (814.09 ms) and shapes (833.71 ms, ), indicating that the combined effect was significant at low cognitive load, and cognitive performance was significantly improved. No significant differences were found in the error rates of the three icons, so trade-off was unnecessary.
The above findings showed that at medium and low cognitive loads, the combined icons significantly affected reaction time and error rate, which proved that the combination effect was significant. To further compare the effect of the combined effect at medium and low cognitive loads on cognitive performance, ANOVA was carried out for the RT of combined icons. The results showed that the RT of combined icons (852.33 ms) at medium cognitive load was significantly longer than that at low cognitive load (767.52 ms, ), which proved that the combined effect on cognitive performance at low cognitive load was higher than that at medium cognitive load.
2.3. Discussion
In experiment A, the combined icons did not significantly affect cognitive performance at high cognitive load, and color was the key factor in cognitive coding. The combined icons were better than color blocks and shapes in improving icon cognition at medium and low cognitive loads. At medium and low cognitive loads, except for the effect of colors, the effect of semantic information and the complexity of shapes also mattered in the combined effect. To further explore the effect of semantic information and the complexity of shapes on cognitive performance, the shapes were changed to inverted with no semantic meaning in experiment B1 to separate the visual features and semantic information. Experiment B2, which used shapes with different complexities as target stimuli, further probed into the effect of shape complexity in combined icons at medium and low cognitive loads on cognitive performance. Finally, to study the activation degree of the three icons, including combination icon, shape icon, and color icon, on the brain, experiment C used the graph of experiment B2 as the target stimulus, recorded the EEG data during the experiment, and explored the influence of the three icons on the brain.
3. Experiment B1: Semantic Experiment
3.1. Method
3.1.1. Participants
Due to the new outbreak of COVID-19, the subjects were difficult to find, so the participants were the same as in experiment A.
3.1.2. Materials, Design, and Procedure
The details of the procedure were largely the same as in experiment A, with the following differences: the materials used in experiment A were upright icons with a clear meaning. According to previous studies, the subject’s processing of the upright icons was integral, while the inverted icons would interfere with the overall semantic processing of the icons without affecting their visual processing. There were four types of experimental materials (Figure 5): monochrome upright icons, monochrome inverted icons, multicolor upright icons, and multicolor inverted icons. The upright icons have semantic information, and the inverted icons carry no semantic information. The experiment was divided into two groups, using 2 (cognitive load: medium/low) × 2 (icon type: monochrome upright icons/monochrome inverted icons) and 2 (cognitive load: medium/low) × 2 (icon type: multicolor upright icons/multicolor inverted icon) within-subject design.
3.2. Results
Similar to that in experiment A, 1.9% of the data with RTs longer than 2 s and shorter than 0.3 s were deleted, and the repeated variance analysis of 2 (cognitive load: medium/low) × 2 (icon types: monochrome upright icons/monochrome inverted icons) was conducted. Figure 6 shows the reaction times of monochrome icons at medium and low loads.
(a)
(b)
The results of the ANOVA showed that the main effect of the cognitive load was significant (F [1, 33]Error! Hyperlink reference not valid. = 10.21, ), the main effect of semantic information was significant (F [1, 33] = 61.99, ), and the interaction effect between cognitive load and semantic information was insignificant (F [1, 33] = 7.47, ). Least significant difference (LSD) of RT and error rate of semantic information showed that the RT of the monochrome upright icons (883.94 ms) was significantly lower than that of the monochrome inverted icons (962.71 ms; ) and the error rate of monochrome upright icons (21.6%) was significantly lower than that of monochrome inverted icons (25.4%; ), which indicated that the semantic information in the combined icons significantly improved cognitive performance and reduced the error rate.
To further compare the RTs at medium and low cognitive loads, ANOVA was performed. The results showed that the RT at medium cognitive load (883.94 ms) was significantly higher than the RT at low cognitive load (824.92 ms; F [1, 33]) = 15.01, ]), indicating that the semantic information in the combined icons was fully processed at low cognitive load and that the effect on improving cognitive performance was greater than that of medium cognitive load.
To further explore the effect of semantic information on cognitive performance when the icon colors were different, the repeated variance analysis of 2 (cognitive load: medium/low) × 2 (icon type: multicolor upright icons/multicolor inverted icons) was carried out on RT, and the results (Figure 6) were consistent with those of the monochrome icon experiment. The main effect of semantic information was significant (F [1, 33] = 36.21, ), and the RT of the multicolor upright icons (809.94 ms) was significantly shorter than that of the multicolor inverted icons (884.56 ms). It thus indicated that when the icon colors were different, the semantic information in the combined icons still significantly improved cognitive performance. In the same way, repeated-measures ANOVA was performed on the error rate, and the main effect of semantic information was not found to be significant (F [1, 33] = 28.29, ), so there was no trade-off between the RT and accuracy rate.
3.3. Discussion
In experiment B1, we further separated the two factors of semantic information and graphic visual features in the combined effect. The experimental results revealed that the semantic information in the combined icon significantly improved the cognitive performance of the icons at medium and low cognitive loads. Semantic information, as a higher-level perceptual attribute, can improve the top-down cognitive processing ability of the icon itself, making the cognitive performance of icons significantly higher than that of shapes without semantic information.
4. Experiment B2: Shape Complexity Experiment
4.1. Method
4.1.1. Participants
The participants were the same as in experiment A.
4.1.2. Materials, Design, and Procedure
The experimental content and procedure were the same as in experiments A and B1, with the following differences: to reduce the experimental error caused by the change of experimental materials, the adopted high-complexity shapes were the inverted shapes in experiment B1, and the low-complexity shapes were simple geometric figures selected from the high-complexity shapes. Four test materials were provided (as shown in Figure 7): monochrome low-complexity icons, monochrome high-complexity icons, multicolor low-complexity icons, and multicolor high-complexity icons.
The 2 (cognitive load: medium/low) × 2 (icon type: monochrome low-complexity icons/monochrome high-complexity icons) and 2 (cognitive load: medium/low) × 2 (icon type: multicolor low-complexity icons/multicolor high-complexity icons) within-subject design was adopted in the two sets of experiments.
4.2. Results
Similar to that in experiment A, 2.8% of the data with RTs longer than 2 s and shorter than 0.3 s were deleted. The repeated variance analysis of 2 (cognitive load: medium/low) × 2 (icon type: monochrome low-complexity icons/monochrome high-complexity icons) was performed for the RTs. The ANOVA result showed (Figure 8) that the main effect of the cognitive load was insignificant (F [1, 33] = 31.84, ), indicating that cognitive load did not significantly affect the task difficulty when the icon colors were the same. The main effect of shape complexity was significant (F [1, 33] = 53.77, ). The interactive effect between cognitive load and shape complexity was insignificant (F [1, 33] = 5.65, ).
(a)
(b)
The RT and error rate for the shape complexity were compared after the experiment. The results showed that the RT for the monochrome low-complexity icons (803.99 ms) was significantly shorter than that of the monochrome high-complexity icons (952.13 ms; ) and the error rate for the monochrome low-complexity icons (15.4%) was significantly lower than that of the monochrome high-complexity icons (25.2%; ), indicating that the decrease in the shape complexity would significantly improve cognitive performance in monochrome combined icons. To further compare the RTs at medium and low cognitive loads, ANOVA was carried out. The results showed no significant difference between the RTs at medium and low cognitive loads (F [1, 33] = 16.12, ), indicating that the complexity of combined icons at medium and low cognitive loads was not sensitive to the cognitive influence of shapes.
To further study the effect of the shape complexity on cognitive performance when the icon color was different, the 2 (cognitive load: medium/low) × 2 (icon type: multicolor low-complexity icons/multicolor high-complexity icons) repeated-measures ANOVA was carried out for RT. The results showed (Figure 8) that the main effect of the shape complexity was insignificant (F [1, 33] = 35.80, ), indicating that the shape complexity in the combined effect did not significantly affect cognitive performance when the icon color was different. Repeated-measures ANOVA was carried out for the error rate. The results showed that the main effect of cognitive load was insignificant (F [1, 33] = 17.44, ) and that the main effect of shape complexity was not significant (F [1, 33] = 31.52, ) either. Therefore, no trade-off was necessary between the RT and accuracy.
4.3. Discussion
In experiment B2, the effect of semantic information in the combined icons on cognitive performance was excluded. When the icons had only one color, the reduction of the shape complexity would significantly improve the cognitive performance. However, when the icons had multiple colors, the shape complexity did not significantly affect the cognitive performance. It can be inferred that when the icons had only color information but no semantic information, color gained a cognitive competitive advantage, so the effect of color on improving cognitive performance was greater than that of shapes without semantics.
5. Experiment C: EEG Experiment
5.1. Method
5.1.1. Participants
20 college students (10 males and 10 females, aged 20–26 years, mean = 23, and SD = 4.23) participated in this experiment. All participants had normal or corrected vision without any color blindness or color weakness. All participants were not informed of the experimental intent and had never participated in similar experiments before. They got paid after the experiment.
5.1.2. Materials, Design, and Procedure
The experimental content and procedure were the same as those in experiment A, with the following differences. (1) To obtain more accurate EEG results, the low-complex shapes of experiment B2 were selected as the shape material. The color material was presented on the entire screen, and 4 colors were randomly selected to appear in sequence (see Figure 9). (2) The participants sit in a closed and quiet EEG/ERP laboratory. During the EEG experiment, the participants wore a wireless EEG cap during the entire experimental task. The EEG equipment was placed in the front center of the head, and 24 standard dry electrodes in the parietal and occipital regions were collected. Data were collected according to the improved international 10–20 electrode placement system. All impedances were maintained at 2000 k, and the EEG signal was amplified, sampled at a rate of 250 HZ, and band-pass filtered between 0.5 HZ and 100 HZ in the acquisition software [29].
5.2. Results
Preprocess the EEG data and use MATLAB to analyze continuous EEG. Remove 10% of the data with eye or other prominent artifacts [30]. Using the conventional P300 component inspection method [31], the P300 component was analyzed from the selected 15 electrode positions (FPz、FP2、FP1、F3、F4、Fz、Cz、C3、C4、Pz、P3、P4、Oz、O1、O2). 15 electrodes was divided into 5 areas: forehead (FP1, FPz, FP2), frontal lobe (F3, Fz, F4), central (C3, Cz, C4), parietal lobe (P3, Pz, P4), and occipital lobe (O1, Oz, O2). The P300 component was measured in a 200–400 ms time window, and 3 (icon type: color block, combined icon, shape) × 2 (electrode position: forehead, frontal lobe, center, parietal lobe and occipital lobe) two-factor repeated measurement analysis of variance was carried out for the P300 amplitude. The main effect of electrode position was significant (F = 372.15, ), the main effect of icon type was significant (F = 49.92, ), and the interaction effect between the two was not significant. After the multiple comparisons of the P300 amplitude of the electrode position, the results showed that the P300 amplitude caused by the frontal lobe position (Oz) was significantly higher than that of other positions (), and the P300 waveform of the Oz electrode and the topographic map of the brain are shown in Figure 10.
(a)
(b)
The analysis results of the P300 components showed that the activation degree of the occipital region of the color block and the combined icon was significantly higher than that of the shape. The P300 wave amplitude of the color block was larger and the average voltage was higher, indicating that the color recognition time was shorter than that of the shape. This result verified the conclusion of the behavioral experiment: colors were recognized before shapes to improve cognitive performance.
Comparing the three topographic maps in Figure 10, it was found that the brain regions activated by the combined icon and the single feature shape are as follows. (1) The combined icon activated the left posterior temporal and right posterior temporal brain regions more than the shape, and the combined icon activated a large area of the brain during 200–400 ms. This shows that in the cognitive process of the icon, as the feature quantity increases, more brain regions will be activated. This conclusion is consistent with the previous conclusions and verifies the conclusion of the behavioral experiment: the brain cognition mechanism of the combined and the shape icons is not exactly the same. (2) Both the combined icon and the shape icon activated the occipital lobe, and the activation degree of the combined icon in the occipital lobe area was higher than that of the shape.
6. Conclusions
From the perspectives of visual features and semantic information, we studied the cognitive processing mechanism of the basic attributes in combined effect. The conclusions we arrived at from the research supplemented the previous research results. The present study found that the lower the cognitive load, the greater the combined effect on improving cognitive performance, and the role of color in the combined effect on improving cognitive performance is greater than that of shapes. The result also confirmed the research conclusion of Adamo et al. [32] that color was better than shapes in improving cognitive performance. Meanwhile, as the cognitive load decreased, the role of semantic information in the combined effect on improving cognitive performance gradually increases. The conclusion supplemented the views of Zovko and Kiefer [33].
Through the research of experiment A, it was found that color could significantly improve cognitive performance and reduce cognitive load under high cognitive load. Combined icons had higher cognitive performance than single-feature icons under medium and low cognitive loads. In the previous related results, scholars mostly used the attentional blink paradigm [34] or the exogenous cueing paradigm [35]. In the attentional blink paradigm, the research object was used as a target stimulus, and the processing mechanism was controlled by subjective consciousness. In the exogenous cueing paradigm, the target stimulus caused a significant attention capture effect under different loads, and the processing mechanism was affected by external distractions. This research adopted a more rigorous and precise experimental design than previous ones, mainly including the following. (1) The cognitive capacity of different experimental materials was different. The number of experimental materials was determined at 4 through preliminary experiments, which made the experimental materials more rigorous. (2) The expectation of the target icon of participants was reduced by randomizing the appearance of icon types. (3) The icon types under different cognitive loads appeared in a mixed rather than a classified pattern, which reduced the participants’ expectations of cognitive loads. The above measures made the icon type and cognitive load unpredictable, which was more life-like, thus ensuring that the result was more stable and more convincing to reflect the processing and cognitive mechanism of the combined icons.
The results of experiment A were shown as follows. (1) Under high cognitive load, the combined effect of color and shape had no significant effect on cognitive performance, and color was the key factor in cognitive coding. (2) Under medium and low cognitive loads, the combined effect of color and shape significantly improved cognitive performance. By comparing the RT under medium and high cognitive loads, we found that shape response was processed more under medium cognitive load than under high cognitive load, which indicated that the cognitive processing of color information was faster than that of shape, and the cognitive processing of shape consumed more resources or longer time. By comparing the error rate under medium and high cognitive loads (under high cognitive load, there was no significant difference between the error rate of combined icons and color blocks; however, under medium cognitive load, the error rate of combined icons was lower than that of color), shape under medium cognitive load had been cognitively processed, so the recognition rate of the combined icons was improved, or in other words, the combined effect was produced. The following could be seen. (1) Some information of shapes needed more time to be processed. Once recognition and processing were completed, the accuracy rate would be significantly improved. (2) Under high cognitive load, color could be used as a single feature for coding design. However, under medium and low cognitive loads, icons with more feature information led to a higher accuracy rate than icons with less feature information. By comparing the RT under low and medium cognitive loads (there was no obvious difference in RT of shapes, combined icons, and color blocks under medium cognitive load, and the RT of combined icons was lower than that of shapes and color blocks under low cognitive load), we found that although the cognitive processing time of shapes was longer than that of color, when the information processing time was sufficient, the RT of shapes decreased, and combined icons required less RT than shapes or color blocks, which revealed that multiple feature information of the combined icons had a cross effect on improving cognitive performance. Color and shapes were both important to improve accuracy, which needed time to be cognitively processed. Once the cognitive processing was completed, the accuracy would increase.
To study the role of semantic information in the combined effect on cognitive performance under medium and low cognitive loads, the bottom-top visual features and the top-down semantic information effect were separated from the combined icons by inverted icons in experiment B1. According to previous studies, upright icons (including semantic information) were processed as a whole (with visual features and sematic features), while inverted images interfered with people’s semantic processing of stimuli without affecting the processing of visual features of stimuli. Experiment B1 indicated that the lower the cognitive load was, the greater the role of sematic information would play in improving people’s cognitive performance. This conclusion was consistent with that of experiment A (the lower the cognitive load, the more obvious the combined effect would be). Therefore, such improvement could be attributed to the semantic information itself. Under low cognitive load, the semantic information in the combined effect significantly improved cognitive performance, and the effect of semantic information on improving cognitive performance under low cognitive load was better than that under medium cognitive load. It could be inferred that as the cognitive load decreased, the role of semantic information in the combined effect on improving cognitive performance gradually increased. However, when the experimental materials were increased from 4 to 6 under medium cognitive load, the semantic information significantly affected cognitive performance. The main reason for such result could thus be presumed that the excessive cognitive load led to the higher consumption of cognitive resources, or too much semantic information significantly increased cognitive load as cognitive resources increased. The specific reasons need further study.
In the shape complexity tests of experiment B2, the effect of semantics on cognitive performance was excluded from combined icons. The results showed that the decrease of shape complexity led to certain cognitive performance improvement, but such improvement was less than that of color information. Under medium and low cognitive loads, when the icon only had color feature without semantic information, the complexity of the shape had no significant effect on cognitive performance, while color had a significant effect on cognitive performance. This indicated that color had gained a cognitive competitive advantage. Therefore, the effect of color on improving cognitive performance is greater than that of shapes without semantics.
Experiment C verified the results of the behavioral experiment using EEG experiments. The P300 components and topographic maps of the combined icon, color block, and shape were compared and found. (1) The activation degree of color block and combined icon in the occipital region was significantly higher than that of shape, indicating that the color recognition time is shorter than that of the graphic, and the color is recognized before the graphic to improve cognitive performance. (2) The combined icon activated more brain areas such as the left posterior temporal and right posterior temporal brain regions than shape, and the combined icon was activated in a large area of the brain during 200–400 ms. This shows that in the cognitive process of the icon, as the feature quantity increases, more brain regions will be activated. This conclusion is consistent with the previous conclusions and verifies the conclusion of the behavioral experiment: the brain cognition mechanism of the combined icon and the shape is not exactly the same. It is inferred that the memory neural mechanisms of complex visual information and simple visual information are different.
In sum, in combined effect, color and shape information have different cognitive mechanisms under different cognitive loads. (1) The combined effect is insignificant at high load, and color is the main factor in cognitive coding. (2) Under medium and low cognitive loads, the combined effect significantly improves cognitive performance. In combined icons, the cognitive performance will be greater when the shapes become simple and the semantic meanings are clear. Based on the above cognitive mechanism, optimized design strategy can be adopted for human-machine interface design, i.e., in an emergency, color coding alone shall be used to convey information, which can significantly improve cognitive performance. Under normal circumstances, the combined icons or images with semantic meanings are more conducive to users’ cognition. In addition, this study also has shortcomings. The EEG study is only one of the reliable physiological monitoring methods. It is believed that the use of fMRI (functional magnetic resonance imaging), infrared technology, etc. will also produce interesting results. Due to the impact of the epidemic, the number of participants in the experiment is small and all are students at school, so the results of this study only reflect the visual cognition of college students. In the future research, we will increase the number of samples in the experiment and explore the coding mode of visual elements in the combined icons based on shape semantics and visual characteristics.
Data Availability
The relevant experimental data used to support the findings of this study have not been made available because this experiment is one of our internal projects; it involves physiological data from the general population and is a part of our whole project, so the experimental data are confidential.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
This project was supported jointly by the National Natural Science Foundation of China (grant no. 52075547), the Ministry of Education of Humanities and Social Science Project (grant no. 16YJCZH037), and the Fundamental Research Funds for the Central Universities (grant no. 18CX02087A).