Abstract
With the improvement of people’s cultural level, more and more museums are being built or renovated. The design of lighting products for museums is a specialized field that requires designers to take into account a variety of factors, such as safety, presentation, and maintainability. As museum lighting systems meet the needs of conservation, visitor experience, and maintenance, the traditional design process is limited by the experience of the designer and the actual situation of the museum, and the actual light conservation effect of the exhibits is difficult to quantify. We have designed a digital-twin-based intelligent lighting system for museums, which can facilitate museum managers to quantify and manage the light life of exhibits while providing a more immersive viewing experience and recognition effect for visitors.
1. Introduction
Lighting design is an important part of museum exhibition design [1–3]. Without the existence of light, the audience will not be able to appreciate the form, material, and color of the exhibits [4, 5]. Without the change of light, the exhibition space will lose its vitality and artistic conception [6]. Because the light is directly related to vision, the construction of light environment is also what designers need to pay attention to when planning the exhibition space and the modeling performance of exhibits. Excellent museum lighting should first have the function of cultural relics’ protection [7, 8], and the lighting should filter out harmful infrared and ultraviolet rays. At the same time, due to the damage of visible light to some exhibits, it is necessary to consider the quantity of the exhibits. In addition, the design also needs to choose the light source with appropriate color temperature and color rendering so that the exhibits can be clearly viewed by the audience, and the colors of the exhibits can be restored to the maximum. For the lighting of the whole exhibition area, we should control the glare, contrast, uniformity, and other parameters to create a soft and comfortable light environment so that the audience can immerse themselves in it and enjoy it.
Lighting serves the function of museums [9–11]. Museums [12] have three basic functions: “collection and preservation of cultural relics,” “protection and research of exhibits,” and “education and appreciation.” In order to reduce the damage of optical radiation to exhibits, most countries have formulated standards to provide reference values of illumination and limit annual exposure of different exhibits. However, in order to create a good visual environment for the audience, we need to improve the illumination of the exhibits. Due to the reduction of environmental light [13], the ability of recognition will decline, so will the human visual ability, people will not be able to identify the information that the exhibits need to transmit [13]. To solve this contradiction and realize the balance between exhibits’ protection and viewing experience are the key points of museum lighting design. In order to make the museum, lighting environment not only provides a good visual experience for the audience but also minimizes the exposure of exhibits in the exhibition cycle; the lighting design must comply with the principles of safety, reliability, economy, advanced technology, energy saving, and convenient maintenance. In addition, many artificial intelligent technologies [14–17] have been used in some display applications [18–20] and also used in the smart city’s applications [21–24].
2. Lighting Design of the Museum
2.1. Museum Lighting Design Requirements
In order to solve the contradiction between the aforementioned viewing experience and the protection of exhibits, some museums use infrared induction technology; the lights are on when people come, and the lights are off when people go. In this way, the exposure time is reduced, and the illumination of exhibits can be appropriately improved. Some studies have shown that the exposure time can be controlled when the illumination is properly increased, and no additional damage will be caused to the exhibits. At the same time, lighting design not only pays attention to the experience of the audience but also needs to consider the experience of managers. With the development of Internet of Things technology and mobile devices, some designers provide mobile terminal management software of lighting equipment for museum managers, realizing intelligent management. Traditionally, owners need to operate lighting equipment switch and maintenance on-site. Now, it can be processed through the mobile terminal.
However, the single induction control, fixed illumination adjustment logic, and simple management software cannot meet the growing needs of museum lighting design. Therefore, it is of great practical significance to design multisensor integrated interactive control lighting products and intelligent management software. We designed a museum intelligent lighting system based on digital twin, which realized the interaction between the audience and the museum lighting equipment through a variety of sensor devices and procedures such as target detection, position detection, and illumination detection, dynamically adjusted the best lighting brightness for the audience, exhibition area, and exhibits, and provided quantitative management of exhibits’ health for museum managers.
2.2. Problems in the Lighting Design of Existing Museums
At the beginning of the design, we chose a large comprehensive museum for field investigation, evaluated the existing display design of two exhibition halls, and summarized the existing problems in lighting in order to design a lighting system that can take into account the exposure and display effect of the exhibits; see Figure 1. We sum up the following four problems: ① The glare of the basic lighting is serious, and the point light source used for the basic lighting in the exhibition hall produces a lot of light spots and glare on the glass cover of the exhibits. When analyzing this aspect, the first thing is to interpret the angle of natural light. When the light is sufficient, the exhibits will have the optimal visual effects. ② The guiding effect of key lighting is insufficient (see Figure 2), and the brightness of some wall washing lighting is too high, and the range is too large, which affects the local lighting (see Figure 3). For artificial lighting, we can use the halogen lamps at 45° angle, and the exhibits can meet the general viewing requirements of the audience. ③ The local illumination brightness is insufficient, and the focus range is uneven, which leads to poor display effect and unreal appearance. In addition, the smallpox part of the display space and the light irradiation of the wall facade are coordinated, and the design of the color sense of the ground is more dark. For the precious exhibits displayed in textiles, clothing, paper painting and calligraphy, and other vertical forms, the angle of light must be more rigorous. ④ During the opening period of the exhibition, all the lighting brightness is constant, and there is still a certain amount of exposure when there is no audience to watch the exhibits in part of the time.
When light angles vertically to 30° of the exhibit, viewing the surface of the vertical artifacts does not produce any glare for the best viewing experience, but the light source is best extended to make the edge of the shade softer (see Figure 4).
In view of the above problems, our research objectives include the following: the lighting design system can adjust the exposure level of exhibits according to whether the exhibits are being watched; that is, when the exhibits are being watched, it can provide the best illumination for the exhibits; when there is no audience around the exhibits, it can enhance the guiding effect of key lighting and, at the same time, reduce the basic lighting and local lighting to the minimum safety limit.
2.3. Related Work at Home and Abroad
The balance between exhibits’ protection and viewing experience is the core of museum lighting design. The problem of light damage in museum lighting is not a new topic in the field of museum lighting at home and abroad, but the research on museum lighting design is still in the exploratory stage. At present, there are still many problems and deficiencies in this field. The theoretical research and practice of museum lighting without light damage are still at a low level, and a complete design system has not yet been formed. Although some museums began to use infrared induction control lighting and mobile terminal software control lighting schemes, these schemes are still too simple. With the development and improvement of light source equipment, interactive control mode, and digital twin theory in recent years, some new technologies worthy of designers’ adoption have emerged. Through theoretical exploration and practical verification, we have applied some research results to our design scheme, mainly involving the following three aspects.
The space light environment of the museum can be divided into basic lighting, key lighting, local lighting, and atmosphere lighting. This paper summarizes the following common space elements and lighting methods: display cabinet space is used to display small precious cultural relics, suitable for local lighting; large-scale cultural relics and art exhibition space, suitable for the key lighting mode; wall washing space is used for paintings, murals, thangka, and other exhibitions, suitable for key lighting methods.
According to the research results of many museums at home and abroad, all sensitive exhibits adopt artificial lighting, which combines general lighting with key lighting, rather than uncontrollable natural lighting. Traditional light sources mainly include halogen lamp, fluorescent lamp, and metal halide lamp, which have their own advantages and disadvantages. However, the survey also found that the LED light source, as a rising star, has the characteristics of energy saving and environmental protection, long service life, adjustable, convenient maintenance, and so on, which make it widely used in the lighting market. In recent years, after the white LED light source came out, it developed rapidly, which provides conditions for the LED light source to be used in indoor lighting. And the relevant research data show that a large number of museums represented by the Palace Museum have generally realized the application of the LED light source in lighting design, and the proportion of museums and art museums considering using the LED light source instead of the traditional light source is as high as 75%.
We finally decided to use the LED light source in this design for the following reasons: small size of LED lamps, easy installation, and debugging; no thermal and chemical damage to the exhibits; the color temperature and brightness of the LED can be easily controlled; high reliability and low maintenance cost. Because the LED basically does not contain infrared light and ultraviolet light, this design selects the LED light source with high color rendering index and only considers the illumination and exposure time to control the exposure of the exhibits in the low color temperature lighting mode.
2.4. Multisensor Fusion and Interactive Control
The intelligent Internet of Things control system connects all kinds of household appliances through the network to realize the functions of data collection, centralized monitoring, and management. The powerful network communication function is an indispensable part of the intelligent Internet of Things system. At present, the mainstream communication technologies used in the control system of intelligent Internet of Things are as follows: fieldbus technology, whose wiring adopts the bus structure, which is mainly composed of three basic parts: power supply, twisted pair, and function module; the power line carrier technology is to transmit analog or digital signals at high speed using carrier mode; wireless technology is a networking mode with WLAN technology as the main connection mode. Wireless technology is usually classified according to the wireless communication protocol, such as wireless Ethernet, Bluetooth, ZigBee, and LoRa. At the same time, the wireless Internet of Things control system also needs to meet the requirements of low power consumption, stability, easy expansion, and grid connection.
In this design, the main hardware devices are LED light source, temperature sensor, humidity sensor, PM fine particle sensor, camera, gateway, and industrial computer. Due to the scene limitation of museum exhibition area, in this design, LED light source, temperature sensor, humidity sensor, PM fine particle sensor, and camera of each exhibition area are connected to the gateway equipment of each exhibition area through the WiFi communication protocol, and the gateway equipment of each exhibition area is summarized to the local area network of the industrial control computer through wired Ethernet.
In order to reduce the exposure of exhibits (see Figure 4), some museums use the infrared sensor to control the light source switch. However, the sensing range of the infrared sensor is limited, and it cannot accurately perceive the real-time position and moving track of the audience in the exhibition area, so its control logic is too simple. Therefore, in this design, the camera is used to capture the overall picture of the exhibition area, and the target detection algorithm is used to detect the audience position algorithm and predict its viewing path, so as to achieve the effective control of lighting and smooth transition. Target detection is one of the key technologies of intelligent video image recognition and individual target behavior recognition. At present, with the development of deep learning technology, target detection algorithm has entered the research track of rapid development. In recent years, a number of excellent deep learning target detection algorithms such as R-CNN, Fast R-CNN, SSD, and YOLO have emerged. These algorithms have excellent detection effect and performance in some open-source target detection video image datasets. Among these algorithms, YOLO algorithm is a rising star. It is a target detection algorithm based on deep learning, which was published by Joseph et al. in a paper published in 2016. Compared with other algorithms, the biggest advantage of YOLO algorithm is its fast detection speed, which is a necessary condition for target detection engineering. Since 2016, the latest version of YOLO algorithm is YOLOv4 algorithm released in 2020. For the original YOLOv1 algorithm, in addition to maintaining the advantages of fast detection speed, great progress has been made in accuracy. Because of the excellent accuracy and speed of YOLO algorithm, it is very suitable for this system to detect the position and trajectory of the museum audience, so we use the most widely used YOLOv3 algorithm to develop the light source interactive control program of this system.
2.5. Twin Digital Platform
The concept of the digital twin platform has been born for decades. When it first appeared, it was defined as product production and real-time virtual presentation. However, due to the shackles of the technical level at that time, this new concept did not get enough attention. With the development of intelligent devices in recent years, sensor technology, hardware and software technology, and computer computing performance have been greatly improved. The concept of the digital twin platform has also been further developed, especially in the real-time monitoring of products and devices in industrial scenes. In 2019, by building the digital twin platform of the exhibition hall and cooperating with the UWB sensor to obtain the real-world indoor positioning data of the audience, some researchers have realized the functions of real-time positioning, hot spot analysis, and pedestrian trajectory tracking in the exhibition hall scene (ref. [5]). In our system, we use the computer vision scheme to get the audience’s position in the exhibition hall through the target detection algorithm, which saves the cost of preparing a large number of positioning tags for the audience.
3. Design Details of the Museum Interactive Lighting System Based on the Twin Digital Platform
3.1. Lighting Control and Illumination Detection Hardware
Our design is suitable for a variety of museum exhibition space. In order to better reflect the application effect of this design, we decided to find a real exhibition area of a museum, design the lighting scheme after remodeling, and apply our intelligent lighting system to explore the improvement effect of the existing museum after reconstruction.
The lighting control equipment is mainly composed of the controller, communication module, and LED drive circuit. The controller is STM32F103C8T6 MCU, and the communication module is an ESP8266 wireless network chip. After the equipment is connected to the power supply circuit of LED lamps in the exhibition hall, it can realize the switch and brightness adjustment and receive the light adjustment instructions from the server through the wireless local area network.
The hardware part of illumination detection equipment is mainly divided into the controller, communication module, sensor module, and power supply module. The controller is STM32F103C8T6, the communication module is an ESP8266 wireless network chip, the illumination detection sensor is MAX44009 light intensity sensor, and the power supply module is low-voltage DC power supply. All the environmental data detection devices are sent to the server through the wireless LAN network and stored in the digital twin platform database (see Figure 5).
3.2. Pedestrian Detection and Twin Digital Platform
In this design, we need to install a webcam at the top of the exhibition area which can capture the largest panoramic image and transmit the video stream to the server through Ethernet. The main configuration of the server used in this system is 8-core 16-thread CPU, RTX2080Ti GPU, and 32 GB memory, Gigabit Ethernet card.
The system in this paper uses the method based on deep learning to detect pedestrian targets. In the field of pedestrian target detection, YOLOv3 is widely praised as a good choice. With its excellent characteristics, YOLOv3 can effectively solve the scaling, background clutter, and morphological diversity of pedestrian targets (see Figure 6).
The pedestrian target detection dataset used in this system is divided into two parts, one is the open-source data pedestrian dataset, and the other is the video segmentation dataset, including pedestrian targets captured in the exhibition area of Nanjing Museum and nearby roads. After finishing the dataset, we get 902 images containing pedestrian targets and their corresponding XML annotation files. Another dataset is the video taken by ourselves in the exhibition area of Nanjing Museum and nearby roads. We use labelling annotation software to annotate the captured images and finally sort out 1000 images containing pedestrian targets and their corresponding XML annotation files. Through the synthesis of the above two datasets, a dataset containing more than 2000 pedestrian images is obtained. Following the basic principles of deep learning training and testing, the dataset needs to be divided into two parts: training set and test set. The pictures of the training set and test set cannot be repeated. In the experiment designed in this paper, the author randomly divided the comprehensive pedestrian dataset into the training set and test set according to the ratio of 8 : 2 and then trained and tested the neural network. Finally, the weight obtained from the training can be used to collect the video in the exhibition area to detect the audience.
After detecting the audience in each frame of the video stream (see Figure 7), the program developed by the open computer vision library can calculate the geometric center point of the audience in the picture. Through the conversion between the camera shooting angle and the plane coordinates of the exhibition area, the position of the audience in the exhibition area can be calculated, and the nearest light source can be adjusted to the preset optimal viewing illumination. The illuminance of the distant light source decreases according to the preset value, and only the minimum safe illuminance of the distant exhibition area is maintained.
The above programs are written by using Python programming language, and the audience position information and light source illumination control information are stored in the database for the digital twin platform to read.
3.3. Exhibition Health Twin Digital Platform
The digital twin platform of this system uses Blender software to model the scene and exhibits, and the model is imported into Unity 3D software to develop the whole system (see Figure 8). The digital twin platform is developed with C# programming language, and the client can develop the web version/PC version/APP version according to the requirements. The illumination data can be saved locally or hosted in the cloud, such as Baidu’s time series database, which is determined according to the needs of remote management and the networking environment of the site.
For the users of this system, that is, the owner of the museum, we can use this platform to intuitively monitor the exposure of each exhibit in the exhibition cycle, which can not only grasp the health status of the exhibits but also flexibly adjust the exhibition time accordingly. Compared with the traditional fixed exhibition period in which the manager has no knowledge of the exposure of the exhibits, after the introduction of this system, the manager can quantitatively analyze the exposure of the exhibits and then make a more reasonable decision on the exhibition period.
4. Human-Computer Interaction Design Based on Motion Capture Technology
The previous pedestrian detection algorithm focuses on the detection of pedestrian distribution in a large area and then adjusts the illumination of basic lighting and key lighting. For the local lighting near the exhibits, it needs to identify and adjust the viewing state of the audience near the exhibits.
The recognition of the audience’s viewing state can be summarized into the category of action interaction in human-computer interaction. In this paper, the most typical action of “leaning to watch” is selected as an example to study, and a set of human-computer interaction schemes based on motion capture technology is designed.
Because this research is in the stage of conceptual design, Kinect DK hardware of Microsoft company and its supporting body tracking SDK are used in the technical prototype. Several Kinect DKs need to be installed near the exhibition stand to ensure that the pedestrians around the exhibits are within the camera’s field of vision. Kinect DK transmits RGB and depth data to the aforementioned server in the venue through the optical fiber.
Kinect DK device based on Microsoft can track multiple human bodies at the same time. Through the development of body tracking SDK, the data of each human body in the field of vision can be obtained, including the time-dependent sequence between each picture and the moving bones and the number of human bodies detected in each picture. Based on this, we can determine whether there is an audience near the booth.
After confirming the audience is close to the exhibits, it is necessary to judge whether they are in the viewing position. The most important part of each human body data is the joint data. Joint position and orientation are estimated based on the depth learning framework of the global depth sensor. The unit of position is mm, and the direction is expressed by Euler angle. The position and direction of each joint constitute its own joint coordinate system, and all joint coordinate systems are absolute coordinates relative to the 3D coordinate system of deep camera data.
If the light source is too close to the object, the surface of the object will reflect high-reflection light. If the angle of the light is too vertical, the projected shadow will become longer and the edges will become sharper. When the smallpox part uses a translucent light source, the place where it produces the strongest reflected light is most likely the overhead part of the visitor. If the visitor looks directly at the showcase, the reflected light will enter the viewer’s view (see Figure 9).
After realizing the accurate recognition of the posture of “looking down,” we can determine whether an exhibit is in the state of being watched. When the exhibits are being viewed, the background adjusts the local lighting of the exhibits to the best viewing illumination; when the exhibit is not viewed, the background adjusts the local lighting of the exhibit to the lowest detectable illumination (see Figure 10).
5. Conclusion
Based on the field investigation of museum lighting design, this paper finds some problems and shows how target detection technology, digital twin technology, and human-computer interaction based on motion capture technology are applied to museum lighting design from the perspective of exhibition exposure protection. The intelligent lighting system designed in this paper can ensure the exhibition experience and reduce the unnecessary exposure of museum exhibits. At present, there are still limitations in our research work and design scheme. For example, we have not studied a more detailed light radiation control scheme for specific exhibits and only protect them at the actual exposure quantization level, which is also a research vacancy in the world. In addition, as the workload of the project team is focused on the theoretical research and design of the intelligent lighting system, no equipment test is installed on the spot, and there are bound to be many detailed problems in the application process of large exhibition space, complex structure, and crowded exhibition scenes, which will be discussed in the follow-up research process. The human-computer interaction scheme based on motion capture technology studied in this paper can be further studied in the future in order to become a set of low-cost and portable motion capture schemes. Motion capture has great potential in game animation, film and television animation, virtual image, and other fields. However, in the current commonly used motion capture schemes, the cost of equipment is very high, and the implementation site is limited. The author will continue to study on the basis of the existing achievements in this paper in order to form a more mature human-computer interaction scheme and motion capture scheme [25].
Data Availability
The datasets used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that there are no conflicts of interest.
Acknowledgments
This work was supported by the Jiangsu Province University Philosophy and Social Science Research 2019 Major Project, Human-computer interaction design research based on artificial intelligence technology (Project no. 2019 SJZDA118), the Higher Education Research Project of Nanjing Institute of Engineering in 2020, Research on Cultivating Path of Artificial Intelligence Design Applied Talents (Project no. 2020YB17), and Philosophy and Social Science Research in Colleges and Universities of Jiangsu Province, China (Project no. 2018SJZDA015).