Abstract
This study determined the value of mixed-reality (MR) technology for doctor-patient communication, preoperative planning, intraoperative navigation, and tumor localization in treating patients with breast cancer. Fifty-eight patients with breast space-occupying lesions (16 benign and 42 malignant) who underwent breast lumpectomy at the People’s Liberation Army General Hospital of China were included in this study. The patients were randomly divided into the MR group and the computed tomography (CT) group. In the MR group, a 3D reconstruction of whole-breast ultrasound was used to localize the spatial position of the breast lesion and was combined with the 3D reconstruction of breast MRI to determine the lesion boundaries. To improve the preciseness of surgery, a postoperative survey was conducted. The MR group exhibited a higher level of patient knowledge regarding the disease, treatment, and diagnosis (26.207 ± 1.698 points) than the CT group (19.228 ± 4.889 points) (T = 7.033; ), and patient satisfaction with surgical results, treatment confidence, and communication methods (4.448 ± 0.572) was also higher than that of the CT group (3.172 4.448 ± 0.572) 0.602) . In addition, doctors were significantly more satisfied with surgical planning and intraoperative localization when the MR technique was used (T = 8.273; ). The use of MR technology in lumpectomy has improved patients’ understanding of surgical procedures and surgical results and has achieved positive results. This technique may provide clinical benefits.
1. Introduction
Breasts are important organs that are of great significance to women [1, 2]. However, breast cancer is a serious threat to women’s health. According to the World Health Organization’s International Agency for Research on Cancer [3], the number of new breast cancer cases worldwide reached 2.26 million in 2020, surpassing lung cancer to become the most commonly diagnosed cancer in the world. Although substantial progress has been achieved in the treatment of breast cancer, patients express significant concern regarding the disease. In general, most younger patients with breast cancer meet the requirements for breast-conserving surgery. However, due to a lack of patient knowledge regarding breast lumpectomy, plastic and reconstructive surgery, and surgical outcomes, several patients undergo radical mastectomy or total mastectomy, leading to irreversible changes [4–6]. Surgical planning based on mixed-reality (MR) technology allows for the accurate navigation of breast lesions and the visualization of the anatomical structure of the breast [7–12]. This technology allows surgeons to perform more precise operations and patients to gain a better understanding of their own condition, increasing their confidence in treatment, which can help patients in choosing surgical treatment methods [13–15].
MR technology has been applied to breast lumpectomy in the past. Amini and Kersten-Oertel [16] developed an MR system that uses Microsoft HoloLens to project a three-dimensional (3D) hologram of images created using breast magnetic resonance imaging (MRI) onto the patient, allowing the surgeon to accurately identify tumor location during the surgical process. Gouveia et al. [12] were the first to use a digital, noninvasive method for the intraoperative localization of tumors in patients with breast cancer. Their 3D, digital breast model and augmented reality technology were combined to guide breast-conserving surgery with the use of preoperative markings using carbon tattooing. Allison et al. [17] proposed Breast3D, a fully functional mammographic X-ray image analysis system that reconstructs MRI and computed tomography data in extended reality, establishes visualization models, and is portable to different MR head-mounted displays including Magic Leap. Invernizzi et al. [18] designed a set of guidelines for the early diagnosis of breast cancer-related lymphedema. These guidelines integrate augmented reality tools with clinical examination findings for breast cancer-related lymphedema and use a 3D laser scanner and tablet to conduct a digitally assisted assessment. MR technology has gradually developed into an important research field with potential applications in clinical settings.
In this study, whole-breast ultrasound images were fused with breast MRI and applied to specific surgical cases using MR technology. This study determines the value of this clinical application and discusses its clinical uses.
2. Materials and Methods
2.1. Patients
Fifty-eight patients with space-occupying breast lesions who were treated at the Breast Surgery Department of the People’s Liberation Army General Hospital between January 2021 and May 2021 were included in this study. The patient inclusion criteria were as follows: aged between 18 and 50 years, underwent surgical treatment of the breast at the study institution, had a maximum transverse nodule diameter of <50 mm, underwent preoperative MRI and ultrasound (using an automated breast volume scanner (ABVS)), communicated clearly, were fully informed (as verified by the patients’ families), and provided written informed consent prior to ABVS. Male patients and those who had breast hyperplasia or cystic, inflammatory, or other nonsolid space-occupying lesions were excluded from this study. In addition, patients who did not undergo preoperative MRI and ABVS or those who underwent needle biopsy were excluded from the study. Patients who were enrolled in the study but were subsequently found to not meet the inclusion criteria or those whose actions or surgery did not meet the study protocol were rejected from the study. The mean patient age was 38.2 years (range, 25–45 years), and the transverse diameters of the nodules were 8–48 mm. Invasive ductal carcinoma was diagnosed in 38 patients (65.5%), fibroadenoma in 12 patients (20.8%), and ductal carcinoma in situ in 8 patients (13.7%). Five patients with multifocal lesions and two patients with diffuse disease were excluded from the study.
The patients were randomly divided into the conventional group and the MR group according to the natural order of visits. The conventional group received the traditional abstract oral narration and description of diagnosis and treatment (preoperative communication, preoperative planning, and intraoperative positioning). Patients in the MR group received three-dimensional reconstruction and MR technology based on ultrasound and MRI images, which were applied to preoperative communication, preoperative planning, and intraoperative navigation and positioning of patients (doctors relied on three-dimensional images to display the condition, explain the diagnosis and treatment of patients, and visualize the body surface in the three-dimensional space during surgery). The number of patients, mean age, ratio of left to right breasts, tumor size (including length, width, and depth), educational level, annual income, and marital status were not significantly different between the computed tomography (CT) and MR groups (Table 1).
This study was conducted in accordance with the Declaration of Helsinki. This study was approved by the Ethics Committee of the People’s Liberation Army General Hospital. This study was registered at clinicaltrials.gov (number) on January 1, 2019.
2.2. ABVS Ultrasound System
Siemens S2000 ABVS was used to perform automated continuous tomography with a transducer length of 20 cm, a scanning depth of 20 cm, and a slice thickness of 0.525 mm. A sealed silicone sink with adjustable stands was designed and developed [19]. This sink uses the ultrasonic permeability and deformation properties of water to ensure that the unilateral breast being examined could maintain contact with the transducer attached to the ABVS ultrasonic robot arm with the help of the water surface. This ensures that the scanning range of the transducer covered the entire unilateral breast and that the breast was exposed to as little external pressure as possible. Lesion deformation due to external pressure may result in changes in tumor location and morphology. A silicone film at the bottom of the sink provided a seal, while the outer layer of the silicone film and the breast surface were coated with a coupling agent to maintain the connection. Image segmentation, reconstruction, and head-mounted displays were developed by Beijing Weizhuo Zhiyuan Technology Co., Ltd (Beijing, China).
2.3. MR-Based Lumpectomy
Tumor segmentation was performed using the patient’s MRI data to create a 3D reconstruction of the tumor. Feature points were marked on the skin of the patient’s breast. Relationships between the tumor and feature points were obtained using ABVS and used to construct the 3D model and its corresponding feature points. Based on the transformation matrix obtained with the feature points, point cloud registration of the 3D model was performed, and reality and virtual reality data were fused. The MR-based lumpectomy process is outlined in Figure 1.
2.4. 3D Reconstruction and Localization of the Breast Tumor
Tumor segmentation was performed using the random walker algorithm and preoperative breast MRI data. The random walker graph was established using the imaging data and defined as , where V is the set of vertices in the graph and E is the undirected edge set of vertices in the graph . is the connection relation between vertices and . The edge weight reflects the degree of similarity between adjacent pixels. As tumors exhibit significant contrast, the edge weights were defined using the Gaussian weighting function based on the grey value [20] given bywhere is the grey value at the vertex and is adjusted by the parameter.
Using the edge weight, the probability of walking was obtained from unlabeled pixels to each seed. Each pixel was then reassigned with the label to which it had the highest probability of arriving to achieve image segmentation. As previously described [21], the process of solving these probabilities was transformed into a Dirichlet problem:where A is the incidence matrix of the edges and vertices, C is the diagonal matrix, and the diagonal elements are the corresponding edge weights. The discrete harmonic function x that minimizes must be satisfied. As is a positive semidefinite matrix, has a unique minimum and is decomposed as follows:where and correspond to the probabilities of the labeled and unlabeled pixels, respectively. The differential of with respect to was solved, and the extrema was found using zero: . Thus, the solution to the Dirichlet problem was given by , where the sum of all probabilities satisfying any vertex was one, such as .
In addition, tumor localization was performed using the Siemens S2000 AVBS breast transducer. The transducer scanning range was set to 20 × 20 cm, which includes most unilateral breast sizes in a normal human body. The sealed silicone sink described above was used to account for the mismatch between the flat surface of the ultrasound transducer, the convex surface of the breast, the deformation of the breasts, and certain space-occupying tissues due to the pressure applied by using the ABVS transducer during examination. This allowed for level contact with the transducer attached to the ABVS ultrasonic robot arm and ensured that the scanning range of the transducer covered the entire unilateral breast while minimizing the changes in tumor location and morphology caused by external pressure.
2.5. Point Cloud Registration
The 3D reconstruction model and breast tumor registration are important prerequisites for MR. In this study, the point cloud method was used for the 3D reconstruction model and a coordinate system conversion was implemented through the construction of the H conversion matrix. In general, after ABVS-based tumor localization, only rotations and translations existed between point cloud data points. Therefore, the H matrix was defined as follows:where scale factor ; was the rotation matrix, which was defined as
The translation matrix was given by
Therefore, was defined as the spatial coordinates of the 3D model, and the transformed matrix must satisfy the following equation:
After eight groups of feature points were marked on the patient’s skin, ABVS was used to measure the patient’s tumor location. The corresponding eight groups of matching points were constructed on the 3D model. The translation and rotation parameters of the H matrix were obtained through the transformation of the matching points. The H matrix was then used to fully convert the point cloud of the model to the patient’s physical signs.
2.6. Outcomes
The main outcomes of this study were doctor-patient communication, preoperative planning and discussion, and localization of the tumor using MR technology. Doctor-patient communication was assessed using a postoperative survey regarding communication time (time used by doctors to explain individual patient cases and answer patient questions), patient knowledge (knowledge of illness, treatment plans, prognosis, and potential complications), patient acceptance of surgical outcomes, patient confidence in treatment, and patient satisfaction with communication methods. Preoperative planning and discussion were assessed using postoperative survey items regarding the preoperative planning discussion (specific location and length of the surgical incision) and the doctor’s satisfaction with the preoperative planning.
In the MR group, tumor localization was achieved using markers. Localization was achieved by comparing the location of the center point on the actual intraoperative breast tumor image with the surface marking of the tumor central point based on 2D pre-operative ultrasound. Differences up to 1 mm were considered accurate. For all the patients, the patients had reached clinical recovery after follow-up one month after surgery, and no related complications occurred.
2.7. Statistical Analysis
All statistical analyses were performed using SPSS version 19.0 statistical software (manufacturer name and location) and GraphPad statistical graphing software (manufacturer name and location). Normally distributed continuous data are presented as the mean ± standard deviation. The t-test was used to compare these data. Statistical significance was indicated as .
The testing methods above were all completed by unchanging examining physicians at our hospital, while manual segmentation and the measurement of images were performed by designated imaging physicians and 3D reconstruction technicians.
In order to evaluate the accuracy of registration and fusion of the MR system, we took ten reconstructed breast models for 3D printing and selected five marker points on each printed model. The marker point on the nipple, M0, was the center of the circle, and four marker points, M1, M2, M3, and M4, were placed on the arc every 90°, with an equal spacing of 3 cm, as shown in Figure 2. Since the fusion error of the tumor could not be measured directly, we used the fusion error of these five marker points for systematic evaluation. The five marker points were used for point cloud registration with the outer surface of the reconstructed breast model. After registration, the minimum distance between the marker points and the reconstructed skin was measured as the fusion error. Table 2 shows the obtained fusion error results of the five marker points through 3D printing, registration, and fusion. The average fusion error of our proposed method is between 0.93 mm and 1.57 mm, and the standard deviation (SD) is between 0.44 mm and 1.05 mm.
3. Results
3.1. Doctor-Patient Communication
The time used by doctors to explain individual patient cases was not significantly different between the groups (survey scores: CT group, 3.724 ± 1.437 vs. MR group, 4.241 ± 0.786; T = −1.701; ).
The time used by doctors to answer patient questions was rated as significantly higher in the MR group (survey score: 4.379 ± 1.374) than in the CT group (survey score: 2.483 ± 1.122) (T = −5.759; ) (Table 3).
Similarly, the time used by doctors to respond to patient questions was significantly longer in the MR group (survey score: 4.448 ± 1.429) than in the CT group (survey score: 2.103 ± 0.976) (T = −7.296; ) (Table 3). The total communication time was significantly higher in the MR group (survey score: 12.069 ± 2.103) than in the CT group (survey score: 8.379 ± 2.499) (T = −6.084; ).
Patients in the MR group exhibited significantly more knowledge regarding their illness (MR group, 4.517 ± 0.575 vs. CT group, 3.448 ± 0.827; T = −5.714; ), the treatment plan (MR group, 4.724 ± 0.455 vs. CT group, 3.172 ± 0.693; T = −6.623; ), and prognosis and subsequent treatments (MR group, 4.448 ± 0.572 vs. CT group, 3.000 ± 0.02; T = −7.917; ). The reported satisfaction with the surgical outcomes was not significantly different between the two groups (MR group, 4.276 ± 0.797 vs. CT group, 3.207 ± 0.819; T = −5.038; ). Patients in the MR group had significantly higher postoperative treatment confidence (survey score: 3.862 ± 0.789) than patients in the CT group (survey score: 2.759 ± 0.739) (T = −5.494; ) (Table 3). Patients in the MR group reported a higher level of satisfaction with communication methods (survey score: 4.724 ± 0.455) than patients in the CT group (survey score: 3.138 ± 0.693) (T = −10.34; ).
The total patient knowledge and satisfaction score was significantly higher in the MR group (26.207 ± 1.698) than in the CT group (19.448 ± 4.889) (T = −7.033; ).
3.2. Preoperative Planning
The actual location of the surgical incision was significantly closer to the location determined in the preoperative plan in the MR group (survey score: 4.551 ± 0.572) than in the CT group (survey score: 2.607 ± 0.875) (T = −9.891; ) (Table 4). The actual length of the surgical incision was significantly closer to the length determined in the preoperative plan in the MR group (survey score: 4.276 ± 0.591) than in the CT group (survey score: 3.655 ± 0.669) (T = −3.742; ).
The doctors’ overall satisfaction with preoperative planning was significantly higher when MR was used (survey score: 4.448 ± 0.572)than when CT was used (3.172 ± 0.602) (T = −8.273; ).
4. Discussion
The use of MR technology during breast lumpectomy surgery was explored in this study. The MR system improved preoperative doctor-patient communication, surgical planning, and image visualization for patients with breast cancer compared to CT.
The results of this study indicate that more time was devoted to doctor-patient communication in the MR group. Although a certain amount of time is added, this time is relatively fixed and limited. In addition, the MR method resulted in a higher level of patient knowledge, increased patient confidence in the treatment, and enhanced patient understanding of future treatments and complications, all of which are important factors for patients with breast cancer. The resection of diseased tissue during breast cancer surgery does not equate to the end of treatment or patient suffering; it may be interpreted by the patient as the first of a series of painful treatments and suffering, as pathological and psychological rehabilitation are needed after surgery. Therefore, it is necessary to increase awareness regarding the disease, which will enable patients to determine the optimal treatment plan for their individual conditions and needs based on a complete understanding of the relationship between the disease and the treatment method. Unlike European, American, and other Western countries, the rate of breast-conserving surgery for women in some countries is relatively low, although it has increased recently. Currently, breast cancer patients rely on doctors’ advice when deciding a surgical method to undergo. However, the patient’s subjective intentions can only be achieved when patients are able to make rational and autonomous decisions regarding surgical procedures. This study highlights the need to improve patient disease knowledge in the clinical setting and enhance the expression of the patient’s subjective goals of care.
The doctors’ survey responses indicate that the three dimensionality of the anatomical structure and the visualization of spatial properties provided by MR positively impacted preoperative planning. As the 3D reconstruction was based on ultrasound images, the virtual 3D image could only display unilateral breast structures. While satisfactory intraoperative navigation was achieved regarding tumor location due to high agreement with the anatomical body position, the resulting visual experience lacked images of the contralateral breast tissues. Therefore, the 3D construction of breast MRI images was fused with ultrasound images to improve patient understanding of the visual images and the intraoperative accuracy for the localization of the tumor. Based on the results of this study, this technique allowed for the maximum presentation of available imaging data.
This study is not without limitations. Positional deviations due to the HoloLens equipment may have occurred. Positional deviation refers to the deviation of measurements of stationary objects when viewed from different angles using HoloLens. This deviation is approximately 2 mm, which limits the precision that can be achieved during surgery. However, a positional deviation of 2 mm is within the acceptable range for breast surgery.
Although this technique is currently being developed, it may revolutionize surgical methods. Appropriate standards and quality measures must be developed to evaluate the application of this technique and to enhance its clinical value.
Data Availability
The data that support the findings of this study are available from the corresponding author on request.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.
Authors’ Contributions
Yongfeng Zhang and Yuanyuan Lu contributed equally to this work.
Acknowledgments
This work was supported in part by grants from the National Natural Science Foundation of China (81771835), the Non-Profit Central Research Institute Fund of Chinese Academy of Medical Sciences (2020-JKCS-008), and the Fundamental Research Funds for the Central Universities (FRF-DF-20-05).