Welcome - 李 敏 - 教师个人主页.pdf

http://pih.sagepub.com/content/early/2014/05/06/0954411914533679 DOI: 10.1177/0954411914533679 2014 Proceedings of the Institution of Mechanical Engineers, Part H: Journal of Engineering in Medicine Review Article Intra-operative tumour localisation in robot-assisted minimally invasive surgery: a review Min Li Department of Informatics, King’s College London, UK Hongbin Liu Department of Informatics, King’s College London, UK Allen Jiang Department of Informatics, King’s College London, UK Lakmal D Seneviratne Department of Informatics, King’s College London, UK; College of Engineering, Khalifa University of Science, Technology and Research, Abu Dhabi, U.A.E Prokar Dasgupta Medical Research Council (MRC) Centre for Transplantation, Guy’s Hospital, UK Kaspar Althoefer Department of Informatics, King’s College London, UK Helge Wurdemann Department of Informatics, King’s College London, UK Corresponding author: Min Li, Email: min.m.li@kcl.ac.uk, Tele: +44(0)2078482902 Abstract Robot-assisted Minimally Invasive Surgery has many advantages compared to conventional open surgery but also certain drawbacks: it causes less operative trauma and faster recovery times but does not allow for direct tumour palpation as is the case in open surgery. This article reviews state-of-the-art intraoperative tumour localisation methods used in Robot-assisted Minimally Invasive Surgery and in particular methods that employ force-based sensing, tactile-based sensing, and medical imaging techniques. The limitations and challenges of these methods are discussed and future research directions are proposed. Keywords Haptic feedback, minimally invasive surgery, palpation, tumour localization Introduction Minimally Invasive Surgery (MIS), also called laparoscopic, or keyhole surgery, was introduced in the mid-1980s and has since been widely performed worldwide increasingly replacing open surgery. MIS is performed through small incisions (Trocar ports) ranging from 3 to 12 mm in diameters 1 while open surgery is carried out using a single large incision. MIS has many advantages over open surgery, including improved therapeutic outcome, shortened postoperative recovery, lesser immunological stress response of the tissue, reduced tissue trauma, lower postoperative pain, and less scarring. However, MIS also carries some drawbacks: clinicians need to cope with motion constraints due to small key holes, limited vision of the operative site, reduction of intuitiveness, and the absence of direct tissue interaction 2. To solve the motion constraint problems, surgical robots have been developed in a master-slave configuration which separates the surgeons and the patient completely and augments the distal dexterity of the tool. The da Vinci system (Intuitive Surgical, Inc) is a successful example of this approach. Robot- assisted Minimally Invasive Surgery (RMIS) has enabled surgeons to achieve more successful outcomes and has been utilised in a variety of procedures from relatively routine ones such as prostatectomy 3, cholecystectomy4, and cystectomy5, to the more complex coronary artery revascularization and mitral valve repair6,7. Limited vision of the operation site in MIS has been resolved with the application of high-definition 3D vision systems. The sense of touch however (kinaesthetic force and tactile sensations) is still quite limited. In general, the use of the term “tactile” describes the mechanical stimulation of the skin 8. The force exerted on soft tissue can only be estimated by observing the tissue deformation. Tactile information during tool-tissue contact is absent. During open procedures, surgeons can access affected organs directly which allows them to identify tumours and their boundaries through hand-soft tissue interaction, in other words through manual palpation, and to ensure that tumours have been removed in their entirety. Manual palpation can be conducted by non-prehensile motions, such as pushing and lifting, and prehensile motions, such as grasping and seizing9. Surgeons investigate the force-displacement response to acquire distributed tactile information. Tissue areas that are stiffer than the surrounding tissue can be recognised as abnormal tissue and, therefore, as possible tumours10,11. Lack of direct palpation in RMIS may lead to insufficient tumour excising. Thus, it would be greatly beneficial to develop a real-time intra-operative tumour localization method which is safe, effective, precise, and user-friendly, whose components can be sterilized and which can be easily integrated within existing systems to conduct the palpation procedure at the master side of an RMIS device 12,13. Researchers have proposed methods that can obtain partial force and tactile information 2 to mimic the function of palpation during robotic-assisted surgical procedures. Intra-operative Computed Tomography (CT), Magnetic Resonance Imaging (MRI), Ultrasound (US) imaging is also introduced. Previous research survey articles reviewed the applications of force and/or tactile sensing and/or feedback techniques in RMIS. However, intra-operative tumour localisation in RMIS has not been reviewed in detail. The aim of this article is to present a comprehensive review of recent research achievements in intra-operative tumour localisation methods for RMIS, address their limitations and propose future directions of research. First, force-based sensing is reviewed. Tactile-based sensing is discussed then. The following section provides a review of medical imaging techniques. Challenges and future possible research directions are addressed in the last section. Intra-operative tumour localisation using force-based sensing Direct force feedback architectures At present, tele-robotic systems14 used in RMIS cannot provide direct force feedback to the surgeon. The clinician operating a master-slave surgical robotic system is not able to discern the material properties of soft tissue by using the surgical tool on the slave side. To achieve transparency, that is a match between the indentation forces applied at the tool tip and the feedback as well as between the positions of master and slave15, would require an entire redesign of existing surgical systems, such as the da Vinci and Titan Medical Amadeus16. Given the absence of direct force feedback in current tele-robotic systems14, visual force feedback is considered as the only option17. However, research described by Mahvash et al. shows a better performance of direct force feedback over visual force feedback using a colour bar in tumour identification18. Bilateral control is the basic method used to integrate direct force feedback in robotic surgery 19,20. Instead of a simple two-port model, bilateral control has been extended to a four channel architecture, which considers not only the difference between the master and slave forces but also the positions. DLR (German Aerospace Centre) developed a 7-DoF MiroSurge surgical robotic system providing bimanual force feedback based on a bilateral control scheme21. Manipulation and force feedback are provided by two input devices Sigma.7 (Force Dimension Inc., Nyon, Switzerland) on the master side. Using this system, the user can distinguish between instrument collisions and tool-tissue interactions21. In other studies, Tavakoli et al.20 developed and evaluated a force feedback method which helps users to distinguish tissue stiffness when probing them remotely. They used strain gauges and a load cell attached to the end of a surgical tool. Employing a PHANToM 1.5A force feedback device (Sensable Technologies Inc.) and implementing a bilateral tele-operation control scheme, the researchers provided direct force feedback of bending and torsional moments and the contact force between the tool and tissue. However, system instability is caused by uncontrollable jitters generated by small errors and delays when the transparency increases. Surgery has a low tolerance for this type of inaccurate behaviours. The trade-off between force feedback transparency and system stability is a significant barrier of direct force feedback since it is not possible to successfully apply both position and force control using the aforementioned bilateral control scheme 22. Instead, acceleration-based bilateral control achieves high transparency and manoeuvrability performing position and force control simultaneously and using the common variable between position and force – acceleration19,23. This control type has been utilised in a 1-DoF master-slave forceps surgical robot24 and in a multi-DoF haptic endoscopic surgery robot19. In order to distinguish between different tissue stiffness, further research regarding this application is needed. Force sensing strategies Currently, no commercially available, multiple DoF force sensor meets the dimensional constraints for potential use in MIS through Trocar ports (less than 12 mm in diameter) 1,25. Although the Nano-17 (ATI, Industrial Automation), a commercial 6-DoF sensor system with a diameter of 17 mm, can be sterilized, it cannot be used in standard MIS. However, this sensor is frequently utilized in MIS-related research studies12,26–28. Other specialized force sensors include a 6-DoF force/torque sensor for the DLR telesurgery scenario MiroSurge, for instance. An additional 1-DoF gripping force sensor is integrated to the gripper, which has a annular cross section with a diameter of 10 mm. Sargeant et al. 29 developed an MRcompatible 6-DoF force/torque sensor based on the Steward Platform that obtains intensity modulated light using linear polariser materials and fibre optic guided light. This MR-compatible sensor has a height of 10 mm, diameter of 11 mm, and weight of 0.6 g which meets the MIS requirements. If the sensor is positioned outside the patient, there would be no size restraints and sterilisability issues in regards to said sensor. However, the sensor measurement may be influenced by joint actuation or by the friction between the tool and the Trocar. Alternatively, by measuring contact forces without any force sensor issues related to MR-compatibility, size, sterilisability, and cost will not arise30. For snake-like robots, force sensing could be achieved by the kinematic analysis31. In another context, Mahvash et al.18 estimated contact forces by using the current that is applied to the actuators of the slave robot during remote palpation experiments. However, the sensitivity of these methods is lower than force sensor implementations. Recently, Beccani et al.32 proved the feasibility of a wireless uniaxial indentation palpation method using a 1-DoF magnetic device. This method eliminates direct physical connection through the Trocar port, and, thus, force data is not distorted by friction or joint actuation. Another choice is estimating forces and providing direct force feedback using a further developed bilateral teleportation controller, like acceleration-based bilateral control33. A force sensor is also not needed. Force sensing strategies for tumour localisation are summarised in Table 1. Tissue property acquisition using uniaxial indentation An alternative to direct force feedback is the acquisition of displacements and applied forces in real time and their combination with tissue models to estimate tissue property. The feasibility of conducting separate point uniaxial compression to acquire tissue stiffness distribution information and localise lung tumours utilising a force-sensitive probe is discussed by McCreery et al.34. Yamamoto et al.26 supported surgeons with a graphical overlay to distinguish hard and soft tissue. Real-time tissue stiffness visualisation was established using a Hue-Saturation-Luminance (HSL) representation on the image of tissue surface, where the Hue value represents the stiffness at a contact point and the Saturation value is calculated based on the distance from the palpated point (see Figure 1). Since the tumour tissue is typically stiffer than healthy tissue35, a surgeon can use the colour information provided by the map to distinguish abnormal tissue regions from healthy areas. Later, this graphical overlay method was improved to an interoperable interface which provides augmented visual feedback using threedimensional graphical material property overlays as well as virtual fixtures with haptic feedback36. (a) (b) Figure 1. A coloured stiffness map displays the material property and a colour bar displays the stiffness range (left). Slave-side hardware, including the artificial prostate tissue, the slave manipulator, and the stereo-camera (right)36. Rolling indentation probes for continuous palpation Individual discrete uniaxial indentation may be time intensive for rapid and efficient tumour localisation in cases where the tissue area to be investigated is large12. Hence, lateral movement of finger tips or hands over the organ’s surface is more beneficial for palpation37. In this case, reacting forces vary when moving over abnormal and normal tissue if the indentation depths remain the same. The rolling indentation approach for tumour localisation has been proposed by Liu et al.12,38,39. Instead of a discrete point uniaxial compression test, conducting a rolling indentation over a tissue surface using a force-sensitive wheeled probe can acquire the stiffness map rapidly along fixed trajectories continuously. A force distribution matrix can be obtained, which illustrates the tissue’s elastic modulus at a given indentation depth assuming that the investigated tissue is linear elastic, isotropic, homogeneous, and incompressible40. The resultant forces fr acquired by the force/torque sensor (fx, fy, and fz) at each sampled point are used to generate the Rolling Mechanical Image (RMI), which depicts the geometrical tissue stiffness distribution as shown in Figure 2. An air-cushion force sensitive indentation probe41 - a concept similar to rolling indentation - was also designed to locate stiff tissue. In real applications, it is challenging to maintain a constant indentation depth during the scan. Hence, a stiffness probe, which is able to measure the reacting force and indentation depth at the same time, will be essential. Wanninayake et al. 42,43 proposed an airfloat stiffness probe, which meets these requirements. Indentation depth and surface profile variations can be measured. However, some improvements should be undertaken to fulfil the requirements of RMIS with respect to miniaturization for instance. (a) (b) Figure 2. (a) The rolling indentation probe and (b) the stiffness map44. Sangpradit et al.28,45,46 developed an inverse FE modelling method for the identification of the stiffness of a tumour at a known depth and the estimation of the depth of the tumour with known mechanical properties using rolling indentation. The FE model is established with an initial guess of the soft tissue parameters. The force-displace curve generated by the FE model is compared to real data. The Newton- Raphson method is applied to adjust tissue parameters and minimise the error. The results show that locations and depths of embedded nodules can be identified accurately. Ahn et al.47 uses mechanical property characterisation with FEM-based inverse estimation for a robotic sweeping palpation method. The comparatively long computational time is the main barrier of this method. Intra-operative tumour localisation using tactile-based sensing Tactile information is significant in palpation in order to mechanically display properties of tissue regions48. It enables surgeons to investigate a tissue area rather than a specific point as is the case when using force sensors. Tactile sensors consist of an arrangement of force sensing elements which enable the surgeon to receive information of the internal structure of the tissue by determining pressure spatial distributions. Ideal tactile sensors are reliable, sensitive, firm, small, and low-cost. Tactile-based tumour localisation is summarised in Table 2. Imitated tactile sensing palpation and visualisation systems A graphical representation of force or tactile data is a low-cost and effective method for intra-operative palpation and diagnosis applications. The commercialised tactile array systems Pressure Profile Systems, Inc.49 and TekScan, Inc.50 provide these graphical displays which are capable of showing the pressure distribution over a tissue surface. However, colour-coded tissue stiffness maps only represent local relative stiffness differences and do not transfer absolute stiffness information to the surgeon. Hence, surgeons should rely on their expertise of haptic properties in order to correctly judge the corresponding tissue when using this system51. Grasping palpation. A common approach for tumour localisation is to grab tissue with a grasper or hand (prehensile motions). Schostek et al.51 developed a 10 mm disposable laparoscopic grasper with a mounted 32-element tactile sensor which conveys tactile information visually. The grasper is low-cost, entirely encapsulated in silicone rubber, and can withstand high grasping forces are the main benefits. Najarian et al.52 and Dargahi et al.53 equipped endoscopic graspers with miniaturised PVDF-sensing (Polyvinylidene Fluoride) elements with a graphical visualisation. The resulting data includes tissue stiffness and stress distribution on the tissue/grasper interface. An average discrepancy of about 10% was achieved between the evaluation experimental outputs and the known tactile properties53. However, the developed sensing array of 8 elements, which is limited by the size of the grasper, only covers a small tissue area. This is a problem when internal stiffness information of big organs is required. Non-grasping palpation. Egorov et al.54,55 created a mechanical imaging system for breast and transrectal prostate examination. The feedback provides a real time 2D pressure response pattern and a summary mode with a 3D reconstruction. The Breast Mechanical Imager (BMI) designed by Egorov et al. has a 16×12 array of pressure sensors (Pressure Profile Systems, Inc., Los Angeles, CA) covering a 40 mm×30 mm area of the scan head. Obviously, further miniaturisation is needed in order to make it suitable for RMIS. Two pressure sensor arrays were integrated in a Prostate Mechanical Imaging (PMI) transrectal probe: a probe head pressure sensor array for prostate imaging and a probe shaft pressure sensor array for sphincter imaging. The probe head pressure sensor array consists of 16×8 sensors (Pressure Profile System) covering 40×16 mm2. The shaft pressure array also has 16×8 sensors with a total slightly larger size of 60×20 mm2. In 84% of studied cases, the system was able to reconstruct 2D cross-sectional and 3D images of the prostate. The PMI system was able to determine malignant nodules in 10 of 13 patients with biopsy-confirmed malignant inclusions. Trejos et al.56 and Perri et al.57,58 developed and enhanced the Tactile Sensing Instrument (TSI) to a more advanced Tactile Sensing System (TSS) by adding a visualization interface (see Figure 3). This system now visualises a real-time updated pressure map of the contact surface between the tactile sensor (4×15 elements) and the organ surface. Both interaction force data and the colour-coded pressure map (tactile data) are provided to the clinician. This study concludes that sustained applied forces exceeding 6 N would cause visible and irreversible bulk damage to the liver. Using a capacitive sensor array, Miller et al.59 constructed a similar Tactile Imaging System (TIS) for the localisation of tumours during MIS. The advantage is that a vision-based algorithm localises the probe and a live video is overlaid with a registered pseudo-colour map of the measured pressure distribution (3×12 sensing elements) at the tracked probe location. The surgeon can locate tumours by scanning the surface of the organ using the probe and observing the change in pseudo-colours of the distribution map overlaid on the laparoscopic image. Figure 3. The Tactile Sensing System (TSS)57. Palpation using tactile feedback devices Using tactile feedback devices to interpret the stiffness distribution of the soft tissue may provide a more intuitive reception of tissue stiffness information51. However, tactile feedback devices are much less well developed than tactile sensing17. Limited understanding of human tactile receptors makes the development of tactile feedback devices a challenging task. Research of tactile interfaces is still in the early stages60. Currently, there are several types of tactile feedback display techniques including pins tactile display48, vibrotactile61,62, pneumatic activated tactile display63, microfluidic activated tactile display64, surface acoustic waves65, focused ultrasound66,67, electrorheology68,69, and magnetorheological fluid70. Most existing tactile technologies and devices are expensive, large, imprecise, and non-portable, and cannot be used in real haptic interaction, especially in MIS and in training procedures 60,71,72. The lack of commercially available tactile devices also limits current research of intra-operative palpation in RMIS. There are two main simulation types available for utilising tactile feedback devices for tumour identification: Tactile feedback using movable components. Ottermo et al.73 presented a system capable of remote palpation equipped with a tactile sensor (total size: 24×8 mm2, 2×2×0.5 mm3, 30 piezoelectric sensor elements in a 3×10 pattern) and a tactile display (with mounted 4×8 tactels (TACTil Element)). Force distribution is simulated by tactel height modification which creates skin deformation. A study comparing graspers with and without tactile feedback to each other proved that the grasper with tactile feedback can be helpful for hardness discrimination. Kim et al.48 developed a planar distributed tactile display for organ palpation. It has a 5×6 pin array with a total size of 40×20×23 mm 3. The 30 stacked actuators are piezoelectric bimorphs. As is the case in Ottermo et al.73, the height modification is used to simulate force distribution. The experimental results showed that the addition of tactile feedback display significantly improved precision of perception of the shape and stiffness of objects. Tactile feedback using materials with variable stiffness. The use of rigid movable elements to simulate force distribution in palpation improves tumour identification results, but does not give the user a direct stiffness feeling. Hence researchers have investigated approaches to simulate stiffness directly. The viscosity of electrorheological (ER) fluid can be controlled by the application of an electric field. Similarly, the rheological properties of magnetorheological (MR) fluid will change when subjected to an external magnetic field. Khaled et al.68 described a tactile actuator array using Electrorheological (ER) fluid. Liu et al.70 proposed a single MR fluid-based tactile element: as the applied magnetic field changes, so does the sensed surface profile. Goto and Takemura69 presented a tactile bump display which uses ER fluid. Although the original intention of this tactile feedback device is to improve the accuracy and precision of touch typing, it also has the potential to be modified and applied in tumour localisation. Mansour et al.74 presented a device which can display both the stiffness distribution and surface shape of an object. It consists of an Elongation Spring (ES) for displaying shape and a Stiffness Spring (SS) for displaying stiffness. A finite element analysis of selected parameters proves and validates the design concept. Pneumatic and micro fluidic activated tactile displays also illustrate shape and stiffness at the same time. Culjat et al.63 developed a pneumatic balloon tactile display. Balloon deflections display the shape/height, while air pressures inside display the stiffness. This device can be easily attached to an existing commercial robot-assisted surgery system, such as the da Vinci. Here, commercial single-element piezoresistive force sensors (FlexiForce, Tekscan) were used for psychophysics experiments. The results revealed that their tactile feedback can reduce grasping force in robot-assisted surgery. Although it has not made inroads in tumour localisation as yet, this application shows great potential. Similar to pneumatic activated tactile displays, microfluidic activated displays also exert the force on the finger tip by using the inflation of a tactile layer. Tactus Technology, Inc.64 developed a deformable tactile layer panel which can be integrated in a touch-screen device to provide transparent physical buttons. These buttons can be disabled and will recede into the screen where they become invisible. This has potential to be used in tactile feedback for palpation. Medical imaging techniques Imaging registration Sophisticated pre-operative imaging techniques such as CT, MRI, and US imaging are often used for preoperative tumour identification. They provide accurate and highly detailed multidimensional images. However, sometimes they are not able to distinguish between tumour and oedema fluid, especially in the case of small size formations75. Moreover, accurate rigid registration of the tumour’s position is challenging as it is often different to the one recorded in the preoperative scan due to the movement of organs and the deformability of the soft tissue during surgery76,77. Image registration is commonly used to transform preoperative images to the intra-operative tumour positions. Non-rigid transformations have a high degree of freedom and are capable of accommodating the local deformations that occur during surgery. This registration method, which can estimate the most likely deformations, has therefore been introduced as a way of mapping the pre-operative functional information into the intra-operative space. Deformable tissue models have been developed such as specialised non-linear finite element algorithms and solutions for real-time estimation of soft tissue deformation78. Compared to intra-operative palpation, the performance of pre-operative imaging techniques are moderate. Schipper et al. 79 compared the pulmonary nodule detection rates between intra-operative lung palpation and pre-operative CT imaging. The results show that a significant number of malignant pulmonary nodules that were detected intraoperatively were not identified on preoperative imaging. Intra-operative imaging helps identify any residual tumour tissue and leads to a significant increase in the extent of tumour removal and survival rates. However, the quality of intra-operative images is often degraded compared to pre-operative images. Co-registration of pre- and intra-operative images could be a solution but is not straightforward due to tissue deformation, different acquisition parameters, resolutions, plane orientations, and computational time constraints80. Challenges include discontinuities and missing data in the registration algorithms due to retraction and resection, and time requirements of intra-operative registration. Rigid registration is more common because it is relatively faster compared to the non-rigid registration80. Non-rigid registration methods are still at an experimental stage and cannot be used as yet in practical applications. Registration uncertainty has also been considered 81. By providing registration uncertainty information, the confidence level of surgeons in the registered image data can be increased, would be helpful in decision making. Real-time elastography Elastography, also known as elasticity imaging, is a technique to calculate and visualise various elastic parameters of soft tissue from different tissue stimuli, such as US, CT, MRI, or optics82,83. Elastography involves mapping the strain of the soft tissue induced by applied stress is the concept of elastography 84. Stiffer tissue experiences lower strains. In palpation, the Young’s modulus (E), or shear modulus (µ) describe the elastic properties. In general, there is a simple linear relation between the Young’s modulus and the shear modulus: E=3µ for soft tissue. US elastography can evaluate tissue stiffness in real-time, and has been applied to tumour identification in breast tissue85, prostate86, liver87, and pancreas84. However, expertise of the surgeon is still essential to interpret the image properly. Combining real-time elastography with haptic actuators will allow remote palpation and solve this problem37,68. Khaled et al.68 developed an integrated haptic sensor/actuator system based on ultrasound real-time elastography and ER fluids. The results of elasticity images were combined to reconstruct virtual objects on the haptic actuator array, which allows users to palpate patient’s organs while imaging. Hence, specialised personnel are not required to understand the images. However, disadvantages include high computational expenses 83, limited acquirable characteristics of linear elasticity such as Young’s elasticity and Poisson’s ratio88. Also, there is a limited depth for measurements of US. Discussions and future directions Tactile-based sensing The force and tactile-based sensing methods and technologies reviewed above can be used to support surgeons during tumour removal procedures. However, sensing array sizes are limited by the small mounting surface of surgical tools. This results in relative data variations over a large tissue area. Multiple discrete indentations need to be performed, a fact which increases the palpation time. The tumour detection result may be affected by the higher contact stress which appears at the edge of the sensor array when it is indented on the soft tissue. It is noted that this issue has been largely ignored in the research presented in the literature review. Graphical sensory substitution techniques are more common than other tactile actuators. One major disadvantage of these techniques is that tissue stiffness maps can only represent relative stiffness differences and pinpoint tumours on the tissue surface without providing depth information. Perception is not intuitive, so surgeons find it difficult to receive a sufficient impression of the actual stiffness. Hence, these techniques are no substitute for manual palpation. Feedback modality combinations Direct force feedback does not convey tactile information and thus is not useful for identification of exact tumour boundaries. Graphical material property overlays on the other hand could be beneficial for tumour identification. The combination of force feedback and tactile feedback could enhance the perception and improve the performance of tumour localisation in future research. Mahvash et al.18 pointed out that force displays could be based on real-time intra-operative patientspecific tissue models rather than the current measured force. Tumour identification during RMIS can rely on intra-operative palpation of virtual tissue which is generated by rapid tissue property estimation based on in-vivo tests. Also, force displays based on such tissue models would enable the acquisition of quantitative information of abnormal tissue localization. Palpation with haptic feedback on a virtual tissue is superior to direct haptic feedback as it avoids the complex control between the master robot side and the slave robot side. Based on this virtual tissue real-time generation for intra-operative palpation, pseudo-haptic feedback can be used to enhance the perception of palpation on the virtual tissue. Pseudo-haptic feedback has already been used in medical applications. Bibin et al. 89 introduced a medical simulator called SAILOR for training of Anaesthesia with neurostimulation in virtual environment. Pseudo-haptic feedback was utilised to give touch feedback of organs located under the skin. The implemented algorithm changes the speed of the cursor movement as function of the height of the picture pixels90. Further, the size of the cursor also varies to improve the pseudo-haptic sensation91. Li et al.92 proposed the concept of pseudo-haptic feedback for soft tissue simulation and localisation of abnormal tissue. Augmenting haptics with pseudo-haptics and its use for intra-operative palpation needs to be explored. Multi-fingered palpation Among clinicians, multi-fingered palpation is more common than single-fingered palpation. Some attempts have been made at simulating multi-fingered palpation93–97. However, these multi-fingered palpation simulations used complex and expensive feedback systems. Moreover, no comparison study between single-fingered palpation and multi-fingered palpation has been conducted yet. Multi-fingered palpation feedback can be adapted to intra-operative palpation using real-time generated virtual tissue. Alternatively, a specialised multi-fingered probe and corresponding feedback actuator need to be developed to inspect the surface stiffness of tissues for direct force feedback. Indentation depth measurement Indentation depth measurement is crucial for stiffness calculation. For rolling indentation palpation, it is essential to maintain a constant indentation depth throughout the palpation activity. This could be achieved by pre-registration of the surface. However, it might be time consuming and the accuracy may be affected by errors introduced. Thus, a real-time indentation depth measurement is needed. Although some sensors with the capability of indentation depth measurement have been developed e.g. air-float palpation probe43, some improvements should be undertaken to fulfil the requirements of RMIS with respect to miniaturisation for instance. Three-dimensional (3D) reconstruction techniques could be used in tissue surface contour acquisition for indentation depth measurement. Izadi et al. used a moving Microsoft Kinect for real-time 3D reconstruction and interaction98. To make it more suitable for minimally invasive intra-operative purposes, endoscopic cameras should be used for 3D reconstruction. Once the original, unindented surface is reconstructed, the indentation depth can be calculated based on the distance between the current indenter position and the closest triangle planar on the mesh of the original reconstructed contour. To compensate any tissue shift or deformation, the surface reconstruction process can be repeated several times. Conclusions This article reviewed current engineering solutions for intra-operative tumour localisation. Overall, most existing engineering solutions for intra-operative tumour localisation are still at an experimental stage and have not been tested in-vivo. Further research in this field needs to address the main problem of how to acquire accurate tissue stiffness data and convey useful information to the surgeon. So far, no fast and robust intra-operative solution has been established for clinical use. There are a number of approaches that need to be investigated further if we want to improve user experience and emulate manual palpation as much as possible: employment of multi-fingered actuators can be effective, and combination of tactile with kinaesthetic feedback, of pseudo-haptic with real haptic feedback, and of graphical with haptic displays are all promising methods. Table 1. Summary of force sensing strategies for tumour localisation Approach Challenges Example Properties Measuring contact forces with force sensors Size, sterilisability, cost, and MRcompatibility. Nano-17 (ATI, Industrial Automation) Does not meet the dimensional constraints for potential use in MIS through Trocar ports (less than 12 mm in diameter)1,25 A 6 DoF force/torque sensors for the DLR telesurgery scenario MiroSurge An additional 1-DoF gripping force sensor is integrated in the gripper, which has a annular cross section with a diameter of 10 mm. On the one hand, size limitations and sterilisability of the Reference 12,26–28 21 Measuring contact forces without force sensor used sensor are negligible, if the sensor is positioned outside the patient. On the other hand, friction between the Trocar and the tool or by joint actuation affects the measurement. An optical multi-axis force/torque sensor 6-DoF force/torque MR-compatible sensor. Diameter: 11 mm, height: 10 mm weight:0.6 g A wireless indentation palpation approach using a magnetic device Since a direct physical connection through the Trocar port is redundant, the force data is not distorted by friction or joint actuation. Sensitivity and accuracy A state observer is used to estimate environment force using the current applied to actuators Not as accurate as force sensor Bilateral teleportation controllers Transparency achieved is limited. Acceleration based bilateral control High transparency and manoeuvrability 99 32 18 33 19,24 Further research regarding distinguishing between different tissue stiffness is needed. Kinematic analysis of a snake-like robot The flexible continuum robot has intrinsic force sensing ability. Average force sensing errors: 0.34 g, standard deviation: 0.83 g. 31 Table 2. Summary of intra-operative tumour localisation using tactile-based sensing Approach Sensor Feedback Disposable laparoscopic grasper with tactile sensing 10 mm disposable laparoscopic grasper with a 32element tactile sensor Graphical visualisation Endoscopic grasper with tactile sensing Endoscopic graspers are equipped with miniaturised Graphical visualisation Properties In-vitro palpation experiments Phantom organ Animal organ Main benefits: low-cost, entirely encapsulated in silicone rubber, and withstanding high grasping forces. No Yes The developed sensing array of 8 elements, which is limited by the size Yes In-vivo palpation experiments Reference Yes 51 No No 52,53 PVDF-sensing elements with a graphical visualisation of the grasper, only covers a small tissue area. Breast Mechanical Imager (BMI) A 16 ×12 array of pressure sensors (Pressure Profile Systems) covering 40 mm×30 mm Graphical visualisation Prostate Mechanical Imaging (PMI) transrectal probe Probe head pressure sensor array: 16×8 sensors (Pressure Profile System) covering 40 mm×16 mm. Shaft: 16×8 sensors covering 60 mm×20 mm. Graphical visualisation Tactile Sensing System (TSS) Tactile sensor (4×15 elements) Tactile Imaging System (TIS) A remote palpation instrument Area-based haptic palpation simulator HAptic Sensor Actuator Further miniaturisation is needed in order to make it suitable for RMIS Yes Probe head pressure sensor array for prostate imaging and probe shaft pressure sensor array for sphincter imaging. Yes Graphical visualisation Provides both contact force data and the colourcoded tactile data. No 3×12 sensing elements (Pressure Profile System) Graphical visualization A live video overlaid with a registered pseudocolour map of the acquired pressure distribution. No 2×2×0.5 mm3×30 piezoelectric sensor elements in a 3×10 pattern and a total size of 24×8 mm2 A tactile display (a tactile display with 4×8 tactels (TACTil Element) Using rigid movable elements to simulate force distribution in palpation. Yes Null A 5×6 pin array with a total size of 40×20×23 mm piezoelectric bimorphs Designed for training residents how to perform diagnosis or surgery Yes A tactile actuator array Simulates stiffness directly No Ultrasound real- No No 54,55 No Yes 54,55 Yes No 56–58 Yes No 59 No No 73 No No 48 No No 68 System (HASASEM) time elsatography using ER fluid allows users to conduct palpation while imaging and conducting a biopsy Magnetorheolo gical (MR) fluid based tactile display Null A single MR fluid-based tactile element Simulate stiffness directly. Miniaturisation is needed. No A multi-modal tactile display device Null Two springs: the Elongation Spring and Stiffness Spring Displays both surface shape and stiffness. No Pneumatic balloon actuators Commercial single-element piezoresistive force sensors (FlexiForce, Tekscan) A pneumatic balloon tactile display Can be mounted directly onto the hand controls of the da Vinci surgical robotic system No No No 70 No No 74 No No 63 Acknowledgement This work was partially supported by the National Institute for Health Research (NIHR) Biomedical Research Centre based at Guy's and St Thomas' NHS Foundation Trust and King's College London; the European Commission’s Seventh Framework Programme in the framework of EU project STIFF-FLOP [287728]; and the China Scholarship Council. The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health. References 1. Bicchi A, Canepa G, de Rossi D, Iacconi P, Scilingo E. A sensor-based minimally invasive surgery tool for detecting tissue elastic properties. In: Proceedings of IEEE International Conference on Robotics and Automation. 1996. p. 884–888. 2. Puangmali P, Althoefer K, Seneviratne LD, Murphy D, Dasgupta P. State-of-the-Art in Force and Tactile Sensing for Minimally Invasive Surgery. IEEE Sens J [Internet]. 2008 Apr;8(4):371–381. Available from: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=4453913 3. Wilson T, Torrey R. Open versus robotic-assisted radical prostatectomy: which is better? Curr Opin Urol. 2011;21(3):200–205. 4. Ji W, Zhao Z, Dong J, Wang H, Lu F, Lu H. One-stage robotic-assisted laparoscopic cholecystectomy and common bile duct exploration with primary closure in 5 patients. Surg Laparosc Endosc Percutaneous Tech [Internet]. 2011;21(2):123–126. Available from: http://www.embase.com/search/results?subaction=viewrecord&from=export&id=L361640602 5. Zehnder P, Gill IS. Cost-effectiveness of open versus laparoscopic versus robotic-assisted laparoscopic cystectomy and urinary diversion. Curr Opin Urol [Internet]. 2011;21(5):415–419. Available from: http://dx.doi.org/10.1097/MOU.0b013e3283490582 6. Currie ME, Romsa J, Fox SA, Vezina WC, Akincioglu C, Warrington JC, et al. Long-term angiographic follow-up of robotic-assisted coronary artery revascularization. Ann Thorac Surg [Internet]. 2012;93(5):1426–31. Available from: http://www.ncbi.nlm.nih.gov/pubmed/22342992 7. Masroor S, Plambeck C, Dahnert M. Complex repair of a Barlow’s valve using the Da Vinci robotic surgical system. J Heart Valve Dis [Internet]. 2010;19(5):593–595. Available from: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=2 1053737 8. Van Erp J, Kern T. ISO’s Work on Guidance for Haptic and Tactile Interactions. Haptics Percept Devices Scenar [Internet]. 2008;:936–940. Available from: http://www.springerlink.com/content/u25hw52p50488180 9. Ullrich S, Mendoza J, Ntouba A, Rossaint R, Kuhlen T. Haptic pulse simulation for virtual palpation. Bild für die Medizin 2008 [Internet]. 2008 [cited 2011 Dec 2];:187–191. Available from: http://www.springerlink.com/index/M3P215656TM311V8.pdf 10. De Gersem G. Reliable and enhanced stiffness perception in soft-tissue telemanipulation. Int J Rob Res [Internet]. 2005 Oct 1 [cited 2011 Dec 2];24(10):805–822. Available from: http://ijr.sagepub.com/cgi/doi/10.1177/0278364905057861 11. Brown JD, Rosen J, Kim YS, Chang L, Sinanan MN, Hannaford B. In-vivo and in-situ compressive properties of porcine abdominal soft tissues. Stud Health Technol Inform [Internet]. 2003;94(January):26–32. Available from: http://www.ncbi.nlm.nih.gov/pubmed/15455858 12. Liu H, Noonan DP, Challacombe BJ, Dasgupta P, Seneviratne LD, Althoefer K. Rolling mechanical imaging for tissue abnormality localization during minimally invasive surgery. IEEE Trans Biomed Eng [Internet]. 2010 Feb;57(2):404–14. Available from: http://www.ncbi.nlm.nih.gov/pubmed/19789104 13. Trejos AL, Jayender J, Perri MT, Naish MD, Patel RV, Malthaner R a. Robot-assisted Tactile Sensing for Minimally Invasive Tumor Localization. Int J Rob Res [Internet]. 2009 May 19 [cited 2013 Mar 20];28(9):1118–1133. Available from: http://ijr.sagepub.com/cgi/doi/10.1177/0278364909101136 14. Tavakoli M, Patel R V. Haptics for Teleoperated surgical robotic systems. World Scientific Publishing Co.; 2008. 15. Lawrence DA. Stability and transparency in bilateral teleoperation [Internet]. In: 1992 Proceedings of the 31st IEEE Conference on Decision and Control. IEEE; 1993. p. 624–637.Available from: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=258054 16. Okamura AM. Haptic feedback in robot-assisted minimally invasive surgery. Curr Opin Urol [Internet]. 2009 Jan [cited 2011 Oct 29];19(1):102–107. Available from: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=2701448&tool=pmcentrez&rendertype=ab stract 17. Hamed A, Tang SC, Ren H, Squires A, Payne C, Masamune K, et al. Advances in Haptics, Tactile Sensing, and Manipulation for Robot-Assisted Minimally Invasive Surgery, Noninvasive Surgery, and Diagnosis. J Robot [Internet]. 2012 [cited 2013 Nov 28];2012:1–14. Available from: http://www.hindawi.com/journals/jr/2012/412816/ 18. Mahvash M, Gwilliam J, Agarwal R, Vagvolgyi B, Su L-M, Yuh DD, et al. Force-Feedback Surgical Teleoperator: Controller Design and Palpation Experiments. 2008 Symp Haptic Interfaces Virtual Environ Teleoperator Syst [Internet]. 2008 Mar;(Figure 1):465–471. Available from: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=4479994 19. Tanaka H, Ohnishi K, Nishi H, Kawai T, Morikawa Y, Ozawa S, et al. Implementation of Bilateral Control System Based on Acceleration Control Using FPGA for Multi-DOF Haptic Endoscopic Surgery Robot. IEEE Trans Ind Electron [Internet]. 2009;56(3):618–627. Available from: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=4639511 20. Tavakoli M, Aziminejad A, Patel R V, Moallem M. Methods and mechanisms for contact feedback in a robot-assisted minimally invasive environment. Surg Endosc [Internet]. 2006 Oct [cited 2011 Sep 25];20(10):1570–1579. Available from: http://www.ncbi.nlm.nih.gov/pubmed/16897288 21. Konietschke R, Hagn U, Nickl M, Jorg S, Tobergte A, Passig G, et al. The DLR MiroSurge - A robotic system for surgery [Internet]. In: Proceedings of the IEEE International Conference on Robotics and Automation (2009). IEEE; 2009. p. 1589–1590.Available from: http://ieeexplore.ieee.org/stampPDF/getPDF.jsp?tp=&arnumber=5152361 22. Okamura AM. Haptic feedback in robot-assisted minimally invasive surgery. Curr Opin Urol [Internet]. 2009 [cited 2011 Dec 6];19(1):102. Available from: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2701448/ 23. Ohnishi K. Real world haptics and telehaptics for medical applications. 2010 IEEE Int Symp Ind Electron [Internet]. 2010 Jul;:11–14. Available from: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=5637758 24. Katsura S, Iida W, Ohnishi K. Medical mechatronics—An application to haptic forceps. Annu Rev Control [Internet]. 2005;29(2):237–245. Available from: http://linkinghub.elsevier.com/retrieve/pii/S1367578805000325 25. Trejos AL, Patel R V, Naish MD. Force sensing and its application in minimally invasive surgery and therapy: a survey. Proc Inst Mech Eng Part C J Mech Eng Sci [Internet]. 2010 Jan 1 [cited 2011 Nov 23];224(7):1435–1454. Available from: http://pic.sagepub.com/lookup/doi/10.1243/09544062JMES1917 26. Yamamoto T, Vagvolgyi B, Balaji K, Whitcomb LL, Okamura AM. Tissue property estimation and graphical display for teleoperated robot-assisted surgery. 2009 IEEE Int Conf Robot Autom [Internet]. 2009 May;:4239–4245. Available from: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=5152674 27. Samur E, Sedef M, Basdogan C, Avtan L, Duzgun O. A robotic indenter for minimally invasive measurement and characterization of soft tissue response. Med Image Anal [Internet]. 2007 Aug [cited 2011 Aug 29];11(4):361–73. Available from: http://www.ncbi.nlm.nih.gov/pubmed/17509927 28. Sangpradit K, Liu H, Dasgupta P, Althoefer K, Seneviratne LD. Finite-element modeling of soft tissue rolling indentation. IEEE Trans Biomed Eng [Internet]. 2011 Dec;58(12):3319–3327. Available from: http://www.ncbi.nlm.nih.gov/pubmed/21257372 29. Sargeant R, Liu H, Althoefer K. An MRI Compatible Optical Multi-Axis Force / Torque Sensors Robotic Surgery. In: The Hamlyn Symposium on Medical Robotics. 2012. p. 79–80. 30. Karadogan E, Williams RL, Howell JN, Conatser RR. A stiffness discrimination experiment including analysis of palpation forces and velocities. Simul Healthc J Soc Simul Healthc [Internet]. 2010 Oct [cited 2011 Dec 2];5(5):279–88. Available from: http://www.ncbi.nlm.nih.gov/pubmed/21330810 31. Xu KXK, Simaan N. An investigation of the intrinsic force sensing capabilities of continuum robots. IEEE Trans Robot [Internet]. 2008;24(3):576–587. Available from: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=4542874 32. Beccani M, Natali C Di, Rentschler ME, Valdastri P. Wireless Tissue Palpation: Proof of Concept for a Single Degree of Freedom. In: IEEE International Conference on Robotics and Automation. 2013. p. 703– 709. 33. Gwilliam JC, Mahvash M, Vagvolgyi B, Vacharat A, Yuh DD, Okamura AM. Effects of haptic and graphical force feedback on teleoperated palpation. 2009 IEEE Int Conf Robot Autom [Internet]. 2009 May;:677–682. Available from: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=5152705 34. McCreery GL, Trejos AL, Naish MD, Patel R V, Malthaner RA. Feasibility of locating tumours in lung via kinaesthetic feedback. Int J Med Robot Comput Assist Surg [Internet]. 2008;4(1):58–68. Available from: http://onlinelibrary.wiley.com/doi/10.1002/rcs.169/abstract 35. Wellman P, Howe R. Breast tissue stiffness in compression is correlated to histological diagnosis [Internet]. 1999 [cited 2013 Aug 22]. Available from: https://biorobotics.harvard.edu/pubs/1999/mechprops.pdf 36. Yamamoto T, Abolhassani N. Augmented reality and haptic interfaces for robot‐assisted surgery. Int J Med Robot Comput Assist Surg [Internet]. 2012 [cited 2013 Jul 25];8(November 2011):45–56. Available from: http://onlinelibrary.wiley.com/doi/10.1002/rcs.421/full 37. Hamamoto K. Investigation on virtual palpation system using ultrasonic elasticity imaging. [Internet]. In: Proceedings of the 28th IEEE EMBS Annual International Conference. 2006. p. 4873–4876.Available from: http://www.ncbi.nlm.nih.gov/pubmed/17946267 38. Liu H, Li J, Poon Q, Seneviratne LD, Althoefer K. Miniaturized force-indentation depth sensor for tissue abnormality identification during laparoscopic surgery [Internet]. In: International Conference on Robotics and Automation (ICRA). 2010 [cited 2013 Jan 22]. p. 3654–3659.Available from: http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=5509440 39. Noonan DP, Liu H, Zweiri YH, Althoefer KA, Seneviratne LD. A Dual-Function Wheeled Probe for Tissue Viscoelastic Property Identification during Minimally Invasive Surgery [Internet]. In: IEEE International Conference on Robotics and Automation. Institute of Electrical and Electronics Engineers Inc.; 2007. p. 2629–2634.Available from: http://dx.doi.org/10.1109/ROBOT.2007.363862 40. Liu H, Li J, Song X, Seneviratne LD, Althoefer K. Rolling Indentation Probe for Tissue Abnormality Identification During Minimally Invasive Surgery. IEEE Trans Robot [Internet]. 2011;27(3):450–460. Available from: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=5746537 41. Zbyszewski D, Challacombe B, Liu H, Seneviratne L, Dasgupta P, Murphy D, et al. Air-cushion forcesensitive probe for soft tissue investigation during minimally invasive surgery. J Endourol Endourol Soc [Internet]. 2009;23(9):1421–1424. Available from: http://dx.doi.org/10.1109/ICSENS.2008.4716568 42. Wanninayake IB, Althoefer K, Seneviratne LD. Air-float stiffness probe for tissue abnormality identification in soft tissue palpation. In: The Hamlyn Symposium on Medical Robotics. 2012. p. 52–53. 43. Wanninayake I, Dasgupta P, Seneviratne L, Althoefer K. Air-float Palpation Probe for Tissue Abnormality Identification During Minimally Invasive Surgery. IEEE Trans Biomed Eng [Internet]. 2013 May 20;(c). Available from: http://www.ncbi.nlm.nih.gov/pubmed/23708764 44. Liu H, Puangmali P, Zbyszewski D, Elhage O, Dasgupta P, Dai JS, et al. An indentation depth-force sensing wheeled probe for abnormality identification during minimally invasive surgery. Proc Inst Mech Eng H [Internet]. 2010 Jan;224(6):751–763. Available from: http://www.ncbi.nlm.nih.gov/pubmed/20608492 45. Sangpradit K, Liu H, Seneviratne LD, Althoefer KA. Tissue identification using inverse finite element analysis of rolling indentationof Rolling Indentation. In: IEEE International conference on Robotics and Automation ICRA. 2009. p. 1250–1255. 46. Liu H, Sangpradit K, Li M, Dasgupta P, Althoefer K, Seneviratne LD. Inverse finite-element modeling for tissue parameter identification using a rolling indentation probe. Med Biol Eng Comput [Internet]. 2013 Sep 15 [cited 2013 Oct 30];Available from: http://www.ncbi.nlm.nih.gov/pubmed/24037385 47. Ahn B, Kim Y, Kim J. New approach for abnormal tissue localization with robotic palpation and mechanical property characterization. 2011 IEEE/RSJ Int Conf Intell Robot Syst [Internet]. 2011 Sep;:4516–4521. Available from: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=6094786 48. Kim S-Y, Kyung K-U, Park J, Kwon D-S. Real-time area-based haptic rendering and the augmented tactile display device for a palpation simulator. Adv Robot [Internet]. 2007 Sep 1;21(9):961–981. Available from: http://openurl.ingenta.com/content/xref?genre=article&issn=01691864&volume=21&issue=9&spage=961 49. Pressure Profile Systems, Inc., LosAngeles, CA U. Pressure Profile Systems, Inc. [Internet]. 2012;Available from: http://www.pressureprofile.com 50. TekScan, Inc., South Boston, MA U. TekScan,Inc. [Internet]. 2012;Available from: http://www.tekscan.com 51. Schostek S, Ho C-N, Kalanovic D, Schurr MO. Artificial tactile sensing in minimally invasive surgery - a new technical approach. Minim invasive Ther allied Technol Off J Soc Minim Invasive Ther [Internet]. 2006 Jan [cited 2011 Dec 2];15(5):296–304. Available from: http://www.ncbi.nlm.nih.gov/pubmed/17062404 52. Najarian S, Dargahi J. A novel method in measuring the stiffness of sensed objects with applications for biomedical robotic systems. Int J Med Robot Comput Assist Surg [Internet]. 2006 [cited 2011 Dec 7];2:84–90. Available from: http://onlinelibrary.wiley.com/doi/10.1002/rcs.75/abstract 53. Dargahi J, Najarian S, Ramezanifard R. Graphical display of tactile sensing data with application in minimally invasive surgery ´de sensation Affichage graphique des donn ees ` la chirurgie tactiles avec application a minimalement invasive. Engineering. 2007;32(3). 54. Egorov V, Ayrapetyan S, Sarvazyan AP. Prostate Mechanical Imaging: 3-D Image Composition and Feature Calculations. IEEE Trans Med Imaging. 2006;25(10):1329–1340. 55. Egorov V, Sarvazyan AP. Mechanical Imaging of the Breast. IEEE Trans Med Imaging. 2008;27(9):1275–1287. 56. Trejos AL, Jayender J, Perri MT, Naish MD, Patel R V., Malthaner R a. Experimental evaluation of robot-assisted tactile sensing for minimally invasive surgery. 2008 2nd IEEE RAS EMBS Int Conf Biomed Robot Biomechatronics [Internet]. 2008 Oct;:971–976. Available from: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=4762808 57. Perri MT, Trejos AL. New tactile sensing system for minimally invasive surgical tumour localization. Int J Med Robot Comput Assist Surg. 2010;6(March):211–220. 58. Perri M, Trejos A, Naish M. Initial evaluation of a tactile/kinesthetic force feedback system for minimally invasive tumor localization. Mechatronics, IEEE/ASME Trans [Internet]. 2010 [cited 2011 Dec 6];15(6):925–931. Available from: http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=5607308 59. Miller AP, Peine WJ, Son JS, Hammoud MDZT. Tactile imaging system for localizing Lung nodules during video assisted thoracoscopic surgery. Proc 2007 IEEE Int Conf Robot Autom [Internet]. 2007 Apr;:2996–3001. Available from: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=4209545 60. Coles TR, Meglan D, John NW. The Role of Haptics in Medical Training Simulators: A Survey of the State of the Art. IEEE Trans Haptics [Internet]. 2011 Jan;4(1):51–66. Available from: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=5453367 61. McMahan W, Gewirtz J. Tool contact acceleration feedback for telerobotic surgery. IEEE Trans Haptics [Internet]. 2011 [cited 2013 Jul 25];4(3):1939–1412. Available from: http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=5871614 62. McMahan W, Romano JM, Abdul Rahuman AM, Kuchenbecker KJ. High frequency acceleration feedback significantly increases the realism of haptically rendered textured surfaces. 2010 IEEE Haptics Symp [Internet]. 2010 Mar;:141–148. Available from: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=5444665 63. Culjat M, King C-H, Franco M, Bisley J, Grundfest W, Dutson E. Pneumatic balloon actuators for tactile feedback in robotic surgery. Ind Robot An Int J [Internet]. 2008 [cited 2011 Dec 2];35(5):449–455. Available from: http://www.emeraldinsight.com/10.1108/01439910810893617 64. Tactus Technology I. Tactus Technology [Internet]. 2012;Available from: http://www.tactustechnology.com/index.html 65. Nara T, Takasaki M, Maeda T, Higuchi T, Ando S, Tachi S. Surface acoustic wave tactile display. IEEE Comput Graph Appl [Internet]. 2001;21(6):56–63. Available from: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=963461 66. Gavrilov LR, Tsirulnikov EM, Davies IA. Application of focused ultrasound for the stimulation of neural structures. Ultrasound Med Biol. 1996;22(2):179–192. 67. Iwamoto T, Tatezono M, Shinoda H. Non-contact Method for Producing Tactile Sensation Using Airborne Ultrasound. EuroHaptics. 2008;5024:504–513. 68. Khaled W, Reichling S, Bruhns OT, Boese H, Baumann M, Monkman G, et al. Palpation imaging using a haptic system for virtual reality applications in medicine. Stud Health Technol Inform [Internet]. 2004 Jan;98:147–153. Available from: http://www.ncbi.nlm.nih.gov/pubmed/15544261 69. Goto M, Takemura K. Tactile Bump Display using Electro-rheological Fluid. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). 2013. p. 4478–4483. 70. Liu Y, Davidson R, Taylor P, Ngu J, Zarraga J. Single cell magnetorheological fluid based tactile display. Displays [Internet]. 2005 Jan [cited 2011 Jul 27];26:29–35. Available from: http://linkinghub.elsevier.com/retrieve/pii/S0141938204000873 71. Liu A, Cleary K, Tendick F, Kaufmann C. A Survey of Surgical Simulation: Applications, Technology, and Education. Presence Teleoperators Virtual Environ. 2003;12(6):599–614. 72. Eltaib M. Tactile sensing technology for minimal access surgery––a review. Mechatronics [Internet]. 2003 Dec [cited 2011 Aug 27];13(10):1163–1177. Available from: http://linkinghub.elsevier.com/retrieve/pii/S0957415803000485 73. Ottermo M V, Stavdahl Ø , Johansen T a. A remote palpation instrument for laparoscopic surgery: design and performance. Minim Invasive Ther Allied Technol MITAT Off J Soc Minim Invasive Ther [Internet]. 2009 Jan [cited 2011 Dec 2];18(5):259–72. Available from: http://www.ncbi.nlm.nih.gov/pubmed/19711224 74. Mansour NA, El-bab AMRF, Abdellatif M. Design of a Novel Multi-Modal Tactile Display Device for Biomedical Applications. In: IEEE RAS/EMBS International Conference on Biomedical Robotics and Biomechatronics. 2012. p. 183–188. 75. Li J, Zirjakova J, Yao W, Althoefer K, Dasgupta P, Seneviratne LD. A passive robotic platform for three-dimensional scanning of ex-vivo soft tissue. In: Advances in Reconfigurable Mechanisms and Robots. 2012. p. 477–485. 76. Carter TJ, Sermesant M, Cash DM, Barratt DC, Tanner C, Hawkes DJ. Application of soft tissue modelling to image-guided surgery. Med Eng Phys [Internet]. 2005;27(10):893–909. Available from: http://discovery.ucl.ac.uk/169703/ 77. Risholm P, Golby AJ, Wells W. Multimodal image registration for preoperative planning and imageguided neurosurgical procedures. Neurosurg Clin N Am [Internet]. 2011;22(2):197–206. Available from: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=3080117&tool=pmcentrez&rendertype=ab stract 78. Wittek A, Joldes G, Couton M, Warfield SK, Miller K. Patient-specific non-linear finite element modelling for predicting soft organ deformation in real-time: application to non-rigid neuroimage registration. Prog Biophys Mol Biol [Internet]. 2010;103(2-3):292–303. Available from: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=3107968&tool=pmcentrez&rendertype=ab stract 79. Ellis MC, Hessman CJ, Weerasinghe R, Schipper PH, Vetto JT. Comparison of pulmonary nodule detection rates between preoperative CT imaging and intraoperative lung palpation. Am J Surg [Internet]. 2011;201(5):619–622. Available from: http://www.ncbi.nlm.nih.gov/pubmed/21545910 80. Gao X, Navkar N V, Shah DJ, Tsekos N V, Deng Z. Intraoperative Registration of Preoperative 4D Cardiac Anatomy with Real-time MR Images. In: Proceedings of the 2012 IEEE International Conference on Bioinformatics & Bigengineering (BIBE). 2012. p. 11–13. 81. Risholm P, Pieper S, Samset E, Wells WM. Summarizing and visualizing uncertainty in non-rigid registration. Med Image Comput Comput Interv [Internet]. 2010;6362:554–561. Available from: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=2976974&tool=pmcentrez&rendertype=ab stract 82. Mariappan YK, Glaser KJ, Ehman RL. Magnetic resonance elastography: a review. Clin Anat [Internet]. 2010 Jul [cited 2013 Feb 13];23(5):497–511. Available from: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=3066083&tool=pmcentrez&rendertype=ab stract 83. Ophir J, Alam SK, Garra BS, Kallel F, Konofagou EE, Krouskop T, et al. Elastography: Imaging the elastic properties of soft tissues with ultrasound. J Med Ultrason [Internet]. 2002 Dec;29(4):155–171. Available from: http://www.springerlink.com/index/10.1007/BF02480847 84. Jr JE, Mauad FM, Muglia VF, Caetano E, Sebastião J, Kemp R, et al. Intraoperative application of realtime tissue elastography for the diagnosis and staging of pancreatic tumours. In: Health Care Exchanges (PAHCE). 2011. p. 8–10. 85. Raza S, Odulate A, Ong EMW, Chikarmane S, Harston CW. Using real-time tissue elastography for breast lesion evaluation: our initial experience. J Ultrasound Med Off J Am Inst Ultrasound Med [Internet]. 2010;29(4):551–563. Available from: http://www.jultrasoundmed.org/cgi/content/abstract/29/4/551 86. Miyagawa T, Tsutsumi M, Matsumura T, Kawazoe N, Ishikawa S, Shimokama T, et al. Real-time elastography for the diagnosis of prostate cancer: evaluation of elastographic moving images. Jpn J Clin Oncol [Internet]. 2009 Jun [cited 2013 Feb 13];39(6):394–8. Available from: http://www.ncbi.nlm.nih.gov/pubmed/19359330 87. Inoue Y, Takahashi M, Arita J, Aoki T, Hasegawa K, Beck Y, et al. Intra-operative freehand real-time elastography for small focal liver lesions: “visual palpation” for non-palpable tumors. Surgery [Internet]. 2010;148(5):1000–1011. Available from: http://www.ncbi.nlm.nih.gov/pubmed/20363009 88. Misra S, Ramesh KT, Okamura AM. Modeling of tool-tissue Interactions for computer-based surgical simulation: a literature review. Presence (Camb). 2008 Jul;17(5):463. 89. Bibin L, Anatole L, Bonnet M, Delbos A, Dillon C. SAILOR: a 3-D medical simulator of loco-regional anaesthesia based on desktop virtual reality and pseudo-haptic feedback. In: ACM Sysmposium on Virtual Reality Software and Technology (VRST). 2008. p. 97–100. 90. Burkhardt J. Feeling bumps and holes without a haptic interface: the perception of pseudo-haptic textures [Internet]. In: Proceedings of the ACM SIGCHI. 2004 [cited 2011 Dec 7]. p. 239–246.Available from: http://dl.acm.org/citation.cfm?id=985723 91. Lécuyer A, Burkhardt J-M, Tan C-H. A study of the modification of the speed and size of the cursor for simulating pseudo-haptic bumps and holes. ACM Trans Appl Percept [Internet]. 2008 Aug 1 [cited 2011 Sep 19];5(3):1–21. Available from: http://portal.acm.org/citation.cfm?doid=1402236.1402238 92. Li M, Liu H, Seneviratne LD, Althoefer K. Tissue stiffness simulation and abnormality localization using pseudo-haptic feedback. In: IEEE International Conference on Robotics and Automation. Saint Paul, Minnesota, USA: 2012. p. 5359–5364. 93. Langrana NA, Burdea G, Lange K, Gomez D, Deshpande S. Dynamic force feedback in a virtual knee palpation. Artif Intell Med [Internet]. 1994;6(4):321–333. Available from: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=7 812426 94. Dinsmore M, Langrana N, Burdea G, Ladeji J. Virtual Reality Training Simulation for Palpation of Subsurface Tumors [Internet]. In: IEEE Virtual Reality Annual International Symposium. IEEE Computer Society; 1997. p. 54–60.Available from: http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=583044 95. Daniulaitis V, Alhalabi MO. Medical Palpation of Deformable Tissue using Physics-Based Model for Haptic Interface RObot ( HIRO ). In: Information Systems. 2004. p. 3907–3911. 96. Kawasaki H, Takai J, Tanaka Y, Mrad C, Mouri T. Control of multi-fingered haptic interface opposite to human hand [Internet]. In: Proceedings 2003 IEEERSJ International Conference on Intelligent Robots and Systems IROS 2003. Ieee; 2003. p. 2707–2712.Available from: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=1249279 97. Endo T, Kawasaki H, Mouri T, Doi Y, Yoshida T, Ishigure Y, et al. Five-fingered haptic interface robot: HIRO III. World Haptics 2009 Third Jt EuroHaptics Conf Symp Haptic Interfaces Virtual Environ Teleoperator Syst [Internet]. 2009;4(1):458–463. Available from: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=4810812 98. Izadi S, Kim D, Hilliges O, Molyneaux D, Newcombe R, Kohli P, et al. KinectFusion: Real-time 3D Reconstruction and Interaction Using a Moving Depth Camera [Internet]. In: The ACM Symposium on User Interface Software and Technology (UIST). ACM; 2011. p. 559–568.Available from: http://dl.acm.org/citation.cfm?id=2047270 99. Sargeant R. An investigation of the use of linear polarizers to measure force and torque in optical 6DOF force/torque sensors for dexterous manipulators [Internet]. In: IEEE International Conference on Robotics and Automation. 2012 [cited 2012 Sep 13]. p. 1422–1427.Available from: http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=6225067