Surgical operations in the future
– Information Rich

To create a support platform for intelligent surgeries, we are working to combine artificial intelligence (AI) and information and communication technology (ICT) to support surgeons' visual and cognitive processes with information that is presented with perfect timing and optimal efficiency during operations.

Caution: Images of laparoscopic and robotic surgeries appear hereafter. Some people may find it difficult to view images of actual surgeries, so please use your own judgment about proceeding.

The focus of healthcare services is shifting from quantity to value, meaning healthcare that maximizes the value for patients. For example, in the case of surgery, safety is a universally sought value, and in some cases patients might place a higher value on reduced complications or avoiding further surgeries. If such goals are achieved, patients can look forward to fewer days of post-surgical hospitalization.

So far, university institutes and medical device manufacturers have focused their innovation on developing safer, more efficient surgeries. Such innovations achieved in the past few decades can be divided into three stages. The first stage was the development of laparoscopic surgery, which replaced conventional open surgery with the insertion of various instruments, including endoscopes, forceps and energy devices, through small holes made in the patient's abdomen. The procedure is considered to be minimally invasive because it places less burden on the patient than open abdominal surgery.

The second stage of innovation was the introduction of new devices and procedures in response to first-stage innovations. Endoscopes serving as the eyes of the surgeon transitioned from high definition to 4K. Three-dimensional observation systems enhanced device orientation awareness and infrared fluorescence imaging enabled the enhanced observation of dynamic changes in lymph nodes and blood flow. In addition, great advances were made with treatment instruments, known as energy devices, which serve like extensions of the surgeon's hands, allowing opposing actions to be performed simultaneously, such as making a fast incision and quickly sealing it at the same time to prevent bleeding.

The third stage of innovation introduced advances in system integration and the operation of multiple devices from universal controller. It is anticipating to contribute to more efficient surgical teamwork and the elimination of human errors. In addition, advances in robotic assisted surgery led to dedicated consoles for operating manipulators equipped with articulating arms on their tips. Device control became so precise that it surpassed the accuracy of surgeons manipulating forceps and energy devices with their own hands.

Multiple devices operated from one screen

Robotic assisted surgery

Despite such advances, however, the goal of true surgical innovation has not been fully achieved. Looking to the future, the next essential innovation can be summed up with a concept we call Information Rich. This is a new world of surgery that will be realized through the combination of ICT and AI. Read on to see exactly what this means.

The first key is sharing the knowledge of experienced surgeons. It is said that mastering accurate incisions comes with experience. During an operation, a surgeon makes judgments about various conditions and first draws the incision line in his/her head. This type of experience-based knowledge is also known as tacit knowledge. If tacit knowledge could be converted into data, it would enable the knowledge of experienced surgeons to be transferred relatively easily to other surgeons.

Here's our idea of how to do this. First, surgery images are collected from many experienced surgeons. Then, incision lines made in specific scenarios are converted into digital data. Thereafter, the data is used to train a neural network to draw accurate incisions according to various situations, allowing the neural network to function as AI. During a surgery in which a doctor must make an incision, AI will assess the situation and display what it determines to be the ideal incision line on a monitor. Finally, the surgeon will use this as reference information and then proceed with the incision. (The AMED Project group under the direction of Professor Inomata at Oita University, Japan is now researching this method.)

The second key to realizing the next great surgical innovation is step-by-step information control during surgeries. The procedures of many surgeries are standardized and can be divided into small steps. For each step, surgeons require great amounts of information, such as CT scans taken prior to the surgery, various test data and treatment history. Because the information varies step-by-step during the operation, surgeons currently must verbally direct the team to prepare and present the needed information, or view numerous monitors set up in the operating room.

If there were a system that could follow the progress of a surgery, it would become possible to create an environment in which information could be provided to surgeons with great efficiency. The key will be the use of AI that has been optimized to understand surgical steps so that it knows which step is taking place at any given time during an operation. The AI could then direct the information control system to present the surgeon with information as needed and on the most appropriately positioned monitor.

To turn these ideas into reality, medical equipment from different manufacturers will have to be interconnected and be able to exchange information. This presents a number of challenges, including the standardization of IoT environments (network connections between devices); the collection of clinical data for AI and the implementation platform; and guidelines for the certification and approval of medical devices with AI functionality.

These are not challenges that can be solved by a single company. If many people share the same vision, however, they should be able to move forward together to realize this next great surgical innovation. In the Japanese medical device industry, which is regarded as being strong in diagnostic devices but weak in therapeutic devices, the timing is perfect for collaboration between industry, government and academia to unleash the collective power of Japan.

ANSWERS BEYOND SIGHT

Technology Innovation Office will lead X (cross) Innovation Projects.
If you have any comments, suggestions or inquiries, please let us know.