Gordon Cheng, Technical University of Munich/Germany
End-to-end smart robot skin
Since the beginning of robotics, enabling a sense of touch that is similar to human sense has been the aspiration for every roboticists. Despite all efforts during the past 30 or so years, the solution to date robots with the true sense of touch are still missing. In this talk, I will present all the necessary challenges in our attempt to an end-to-end solution to enable robots with the complete sense of touch. The talk will cover a human multi-modal tactile sensor; the self-organisation of large numbers of connected tactile sensors; the optimized handling of a large amount of tactile information; to the sensitive control of robots covered with smart tactile skin. Several examples will be given during this talk.
Gordon Cheng holds the Chair of Cognitive Systems at Technical University of Munich (TUM). He is Founder and Director of the Institute for Cognitive Systems in the Department of Electrical and Computer Engineering at TUM. He is also the coordinator of the CoC for Neuro-Engineering - Center of Competence Neuro-Engineering within the department. He is also involved in a number of major European Union Projects.
Over the past years Gordon Cheng has been the co-inventor of approximately 20 patents and is the author of approximately 300 technical publications, proceedings, editorials and book chapters.
Nicolaus Correll, University of Colorado/USA
Towards materials that make robots smart: gesture recognition in tactile sensing skins
The materials biological systems are made of are very different than the links, gears and joints that we use to design robotic systems. Instead, they tightly integrate sensing, actuation, computation and communication using appropriate receptors, muscles and a nervous system. In this talk, I'll describe recent progress on tactile sensing skins that can sense proximity, contact and force and are capable of differentiating between a number of social touch gestures (tap, rub, etc.) and obstacles. Rather than sending this information to a central processing location, sensor signals are processed locally and only high level information is shared in an event-based fashion. Possible applications are increasing safety of manipulating arms in unstructured environments and new forms of human-robot interaction. As sensing and computation are local and fully distributed, the resulting skins are theoretically scale-free and can be manufactured in any shape or size. Yet, routing power and communication remain hard problems, and I describe our current efforts in robotic materials that rely on wireless communication and wireless power.
Nikolaus Correll is an Associate Professor at the University of Colorado at Boulder, and founder of the Interdisciplinary Research Team on Multi-functional Materials Robotic Materials Inc. Nikolaus' background is in swarm intelligence and swarm robotics with a PhD fromEPFL, Switzerland, and a post-doc at MIT CSAIL. Nikolaus is the recipient of a 2012 NSF CAREER award, a NASA Early Career Faculty Fellowship and the Provost's Faculty Achievement award.
Ravinder Dahiya, University of Glasgow/UK
Large Area Flexible Electronic Skin for Robotics
The miniaturization led advances in microelectronics over 50 years have revolutionized our lives through fast computing and communication. Recent advances in the field are propelled by applications such as robotics, wearable systems, and healthcare etc. through More than Moore technologies. Often these applications require electronics to conform to 3D surfaces and this calls for new methods to realize devices and circuits on unconventional substrates such as plastics and paper. This lecture will present the approaches that are being pursued (over different time and dimension scales) for obtaining distributed electronics and sensing components on flexible and conformable substrates, especially in context with tactile or electronic skin (e-skin) for robotics and prosthetics. These approaches range from distributed off-the-shelf electronics, integrated on flexible printed circuit boards to advanced alternatives such as e-skin by printed nanowires, graphene and ultra-thin chips, etc. The technology for such sensitive flexible (and possibly stretchable) electronic systems is also the key enabler for numerous emerging fields such as internet of things, human-robot cooperative settings, and mobile health etc. This lecture will also discuss how the tactile skin research could unfold in the future.
Ravinder Dahiya is Professor of Electronics and Nanoengineering and Engineering and Physical Sciences Research Council (EPSRC) Fellow in the School of Engineering at University of Glasgow. He is the Director of Electronic Systems Design Centre (ESDC) and the leader of Bendable Electronics and Sensing Technologies (BEST) group. His group conducts fundamental research on high-mobility materials based flexible electronics and electronic skin, and their application in robotics, prosthetics and wearable systems.
Prof. Dahiya has published more than 200 research articles, 4 books (3 at various publication stages), and 12 patents (including submitted). He has given more than 100 invited/plenary talks. He has led many international projects including those funded by European Commission, EPSRC, The Royal Society, The Royal Academy of Engineering, and The Scottish Funding Council.
He is Distinguished Lecturer of IEEE Sensors Council and is on the Editorial Boards of Scientific Reports (Nature Group) and IEEE Sensors Journal. He has also served on the editorial board of IEEE Transactions on Robotics. He was the Technical Program Co-Chair (TPC) for IEEE Sensors Conference in 2017 and is continuing in this role for IEEE Sensors Conference 2018.
Prof. Dahiya holds EPSRC Fellowship and in past he received Marie Curie Fellowship and Japanese Monbusho Fellowship. He has received several awards and most recent among them are: 2016 IEEE Sensor Council Technical Achievement Award, the 2016 Microelectronic Engineering Young Investigator Award, and International Association of Advanced Materials (IAAMM) Medal for the year 2016. In 2016, he was included in list of Scottish 40UNDER40.
Personal website: www.rsdahiya.com
TEDx talk: ‘Animating the Inanimate World’ (https://www.youtube.com/watch?v=h7yY7ExYAB4)
Ali Javey, EECS Department, UC Berkeley/USA
Wearable Sweat Sensors
Wearable sensor technologies play a significant role in realizing personalized medicine through continuously monitoring an individual’s health state. To this end, human sweat is an excellent candidate for non-invasive monitoring as it contains physiologically rich information. In this talk, I will present our recent advancements on fully-integrated perspiration analysis system that can simultaneously measure sweat metabolites, electrolytes and heavy metals, as well as the skin temperature to calibrate the sensors' response. Our work bridges the technological gap in wearable biosensors by merging plastic-based sensors that interface with the skin, and silicon integrated circuits consolidated on a flexible circuit board for complex signal processing. This wearable system is used to measure the detailed sweat profile of human subjects engaged in prolonged physical activities, and infer real-time assessment of physiological state of the subjects.
Ali Javey received a Ph.D. degree in chemistry from Stanford University in 2005, and was a Junior Fellow of the Harvard Society of Fellows from 2005 to 2006. He then joined the faculty of the University of California at Berkeley where he is currently a professor of Electrical Engineering and Computer Sciences. He is also a faculty scientist at the Lawrence Berkeley National Laboratory where he serves as the program leader of Electronic Materials (E-Mat). He is the co-director of Berkeley Sensor and Actuator Center (BSAC), and Bay Area PV Consortium (BAPVC). He is an associate editor of ACS Nano.
Javey's research interests encompass the fields of chemistry, materials science, and electrical engineering. His work focuses on the integration of nanoscale electronic materials for various technological applications, including low power electronics, flexible circuits and sensors, and energy generation and harvesting. He is the recipient of MRS Outstanding Young Investigator Award (2015), Nano Letters Young Investigator Lectureship (2014); UC Berkeley Electrical Engineering Outstanding Teaching Award (2012); APEC Science Prize for Innovation, Research and Education (2011); Netexplorateur of the Year Award (2011); IEEE Nanotechnology Early Career Award (2010); Alfred P. Sloan Fellow (2010); Mohr Davidow Ventures Innovators Award (2010); National Academy of Sciences Award for Initiatives in Research (2009); Technology Review TR35 (2009); NSF Early CAREER Award (2008); U.S. Frontiers of Engineering by National Academy of Engineering (2008); and Peter Verhofstadt Fellowship from the Semiconductor Research Corporation (2003).
Roland Johansson, Umeå University/Sweden
Fast and accurate geometric tactile processing during object manipulation
The populations of first order tactile afferent neurons that innervate the inside of the human hand signal soft tissue transformations that occur when the hand interacts with objects, thus providing moment-to-moment information about the contact state between the object and the hand. A critical aspect of dexterous object manipulation is that the relevant spatiotemporal tactile information is sufficiently accurate and that it can be used quickly and efficiently by the brain. My talk addresses encoding and use of tactile afferent information in the control of manual dexterity. In particular, I discuss how the peripheral organization of first order tactile neurons in humans might promote rapid and automatic processing of geometric tactile information required to control fingertips in fine manipulation tasks. The emphasis is on population coding and representation of object location and orientation within fingertips based on objects’ edge-based features.
Key Words: Human, Hand, Touch, Tactile system, Neurons/physiology, Fine motor skills, Object manipulation
Pruszynski, J. A., Flanagan, J. R. and Johansson, R. S. (2018). Fast and accurate edge orientation processing during object manipulation. Elife, 7. doi:10.7554/eLife.31200
Pruszynski, J. A. and Johansson, R. S. (2014). Edge-orientation processing in first-order tactile neurons. Nat Neurosci, 17(10): 1404-1409. doi:10.1038/nn.3804
Johansson, R. S. and Flanagan, J. R. (2009). Coding and use of tactile signals from the fingertips in object manipulation tasks. Nature Reviews Neuroscience, 10(5): 345-359.
Johansson received his MD/PhD 1978 at Umeå University, and in 1988 he became a full professor in physiology at the same university. His research primarily addresses the organization and function of the neural mechanisms that give our hands their extraordinary sensorimotor skills, with focus on the hands’ tactile sensory innervation and on multimodal sensorimotor control mechanisms crucial to dexterity. In addition, he has contributed with pioneering research in human orofacial neurophysiology and muscle physiology. During his career, Johansson has received several academic honors and awards and since 2004 he is a member of the Royal Swedish Academy of Sciences.
John Rogers, Northwestern University and Simpson/Querrey Institute/USA
Soft Electronic and Microfluidic Systems for the Skin
Biological systems are mechanically soft, with complex, time-dependent 3D curvilinear shapes; modern electronic and microfluidic technologies are rigid, with simple, static 2D layouts. Eliminating this profound mismatch in physical properties will create vast opportunities in man-made systems that can intimately and non-invasively integrate with the human body, for diagnostic, therapeutic or surgical function with important, unique capabilities in biomedical research and clinical healthcare. Over the last decade, a convergence of new concepts in materials science, mechanical engineering, electrical engineering and advanced manufacturing has led to the emergence of diverse, novel classes of 'biocompatible' electronic and microfluidic systems with skin-like physical properties. This talk describes the key ideas and presents some of the most recent device examples, including wireless, battery-free electronic 'tattoos' with applications in continuous monitoring of vital signs in neonatal intensive care; and microfluidic/electronic platforms that can capture, manipulate and perform biomarker analysis on microliter volumes of sweat, with applications in sports and fitness.
Professor John A. Rogers obtained BA and BS degrees in chemistry and in physics from the University of Texas, Austin, in 1989. From MIT, he received SM degrees in physics and in chemistry in 1992 and the PhD degree in physical chemistry in 1995. From 1995 to 1997, Rogers was a Junior Fellow in the Harvard University Society of Fellows. He joined Bell Laboratories as a Member of Technical Staff in the Condensed Matter Physics Research Department in 1997, and served as Director of this department from the end of 2000 to 2002. He then spent thirteen years on the faculty at University of Illinois, most recently as the Swanlund Chair Professor and Director of the Seitz Materials Research Laboratory. In 2016, he joined Northwestern University as the Louis Simpson and Kimberly Querrey Professor of Materials Science and Engineering, Biomedical Engineering and Medicine, with affiliate appointments in Mechanical Engineering, Electrical and Computer Engineering and Chemistry, where he is also Director of the newly endowed Center for Bio-Integrated Electronics.
He has published more than 650 papers, is a co-inventor on more than 100 patents and he has co-founded several successful technology companies.
Takao Someya, TUM-IAS Hans Fischer Senior Fellow & University of Tokyo/Japan
Smart skins connecting cyberspace and human
The human skin is a large-area, multi-point, multi-modal, stretchable sensor, which has inspired the development of an electronic skin for robots to simultaneously detect pressure and thermal distributions. By improving its conformability, the application of electronic skin has expanded from robots to human body such that an ultrathin semiconductor membrane can be directly laminated onto the skin. Such intimate and conformal integration of electronics with the human skin, namely, smart skin, allows for the continuous monitoring of health conditions. The ultimate goal of the smart skin is to non-invasively measure human activities under natural conditions, which would enable electronic skins and the human skin to interactively reinforce each other. In this talk, I will review recent progresses of stretchable thin-film electronics for applications to robotics and wearables. Furthermore, the issues and the future prospect of smart skins will be addressed.
Takao Someya received his Ph.D. degree in Electrical Engineering from the University of Tokyo in 1997. From 2001 to 2003, he worked at the Nanocenter (NSEC) of Columbia University and at Bell Labs and Lucent Technologies as a Visiting Scholar. Since 2009, he has been a professor of the Department of Electrical and Electronic Engineering at the University of Tokyo as well as a Global Scholar at Princeton University. Furthermore, he currently serves as the Project Leader of the NEDO/JAPERA Project and as a Research Director of a JST/ERATO Project since 2011. At TUM, Takao Someya is hosted by Gordon Cheng from the Department of Cognitive Systems.
Benjamin C.K. Tee, National University of Singapore
Human-Inspired Artificial Somatosensory Systems
There is a clear trend towards a living environment where humans, connected devices and robots live in synergy together. Intelligent devices and robots are augmenting human abilities and assist in a myriad of applications, such as health diagnostics, surgery and predictive analytics. Developing materials with sensitive yet robust mechanical properties is important to achieve seamless integration and digitally augmented activities. I will discuss a few key strain-engineering technologies and possible strategies to engineer damage robustness into devices, such as new self-healing materials and devices. In addition, I will also discuss our recent progress in developing new scalable electronic skin platform technologies for more tactile-aware and perceptive AI robots with applications in healthcare and home settings.
Dr. Benjamin C.K. Tee is appointed President’s Assistant Professor in Materials Science and Engineering Department at the National University of Singapore. He obtained his PhD at Stanford University and was selected as a Singapore-Stanford Biodesign Global Innovation Postdoctoral Fellow in 2014, where he applied a needs-driven approach to innovation in healthcare technologies. He has developed and patented multiple award-winning materials and sensing technologies in electronic sensor skins. He is named one of the prestigious MIT TR35 Innovator (Global) in 2015, and is one of the 2017 Singapore National Research Foundation (NRF) Fellow. He currently leads a multi-disciplinary team to develop new materials and sensor devices technology. His research group aims to develop technologies at the intersection of materials science, mechanics, electronics and biology, with a focus on sensitive electronic skins that has tremendous potential to advance global healthcare technologies in an increasingly Artificial Intelligence (AI) and Robotics future. He can be reached at www.benjamintee.com.
Nitish V. Thakor, Singapore Institute for Neurotechnology, National University of Singapore/Singapore
E-dermis: Multi-layer, multi-receptor tactile array with neuromorphic algorithms detects texture, shape and provides a range of sensory perceptions
The E-dermis concept is to mimic the epidermis of the human skin.We developed a 3-dermis with a multi-layer piezo-fabric to mimic different receptors in the epidermis: these include the fast adaptive (FA), slow adaptive (SA), and free-ending receptors. Each of these receptors was modeled to produce neuromorphic signals, which essentially generated stochastic spike trains in response to applied pressure and shaped (from round to pointy, spanning the range of soft to painful) perceptual ranges. From the spiking activity from the sensor/receptors on the tactile array, we produced identification of different textures and shapes using a fast Extreme Learning Machine algorithm. Together with bioimetic sensor, neuromorphic encoding, and machine learning algorithm, we show that our E-dermis is shown to detect different textures and, shapes. We incorporated the E-dermis in a prosthetic hand to provide sensory perception that spanned light touch and pressure to pain to the amputee.This work opens up the possibility of providing a range of sensory perception in prosthesis and robotics.
Dr. Nitish Thakor is a professor of biomedical engineering and neurology at the Johns Hopkins University School of Medicine. He also has an appointment in the Johns Hopkins Department of Electrical and Computer Engineering. He conducts research on neurological instrumentation, biomedical signal processing, micro and nanotechnologies, neural prosthesis, clinical applications of neural and rehabilitation technologies, and brain-machine interface.
Dr. Thakor directs the Laboratory for Neuroengineering and is also the director of the NIH Training Grant on Neuroengineering. He received his undergraduate degree from the Indian Institute of Technology, in Bombay, India. He earned both a M.S. in and a Ph.D. in biomedical engineering from the University of Wisconsin-Madison. Dr. Thakor joined the Johns Hopkins faculty in 1983.Dr. Thakor is a co-author of more than 250 refereed journal papers and is currently the editor-in-chief of Medical and Biological Engineering and Computing.