Controlling and Simulating Soft Robotic Systems: Insights from a Thermodynamic Perspective

Soft robots are machines, and like all machines their function is to convert energy from one form into another to perform tasks. One key figure of merit for machines is efficiency: defined as the ratio of task-oriented-work out to total-energy in. All soft robots convert stored energy (from e.g. batteries, pressurised gas, chemicals) into task-oriented work (picking up objects, locomoting, jumping). These systems are complex hybrids of: chemical; mechanical; pneumatic; hydraulic; and electrical components. This complexity makes it difficult to analyse and to measure their total efficiency and to identify the sources of energy loss between chemical, electrical, and mechanical domains. As the field of soft-robotics matures, the design-flow process will shift from one in which building is central, to one in which simulation takes precedence. That is: a shift from an empirical experimental methodology and towards a well-characterised engineering workflow. At this point questions such as “For how long will this robot run on a 2000 mAh battery?” will need to be answered, and predictive capabilities will become paramount as designers need to understand: 1.) the large scale deformations inherent to soft robotic systems; and 2) the transduction of energy in these complex, dissipative, systems to enable them to design an efficient and well controlled system. In this perspective piece we discuss one possible predictive approach: a framework which uses port-based modelling. This approach uses bond-graphs and, the recently developed, port-Hamiltonian theory to provide a step-by-step system for analysing hybrid, multi-domain, soft robotic systems. We discuss how this framework could be applied to controlling and optimizing soft robotic systems for energy efficiency, and thereby increasing their utility. An energy-based approach is useful as a domain-free linker in analysing complex systems; the use of ports promotes a clear distinction between energy conservation and dissipation and facilitates the analysis of efficiency. In addition the parallels with hardware description languages and object oriented programming will make it easier for engineers to design, for soft robots, control systems which optimise for efficiency.


Introduction
Soft robotic systems are complex hybrids that use sources and sinks of chemical, mechanical, pneumatic, hydraulic, and electrical energy. They are machines which have been designed to perform tasks by converting energy from one form (storage) into another (actuation). The fact that they have elements which cross multiple domains, and that they are often made using materials which are capable of very large deformations means that it is difficult for system designers to predict and control their motion; and to measure their total efficiency. The control systems for this class of robot are typically open-loop, and due to the complexity of analysing energy transduction from storage to actuation, figures of merit, such as total cost of transport, are very difficult to calculate [1][2][3] .
In this perspectives piece, we suggest that a port-based framework, based on bond-graphs 4 and port-Hamiltonian theory 5 , will be a useful tool for analysing the thermodynamics underlying hybrid, multi-domain, soft robotic systems. Using this approach will enable designers to identify where energy is lost between source and actuation, across the chemical, mechanical, pneumatic, hydraulic, and electrical domains, and will pave the way for application of energy-based control methods in soft robotics.

Design Paradigms for Hard vs Soft Robotic Systems
Hard-bodied and soft-bodied robots are designed using completely different methodologies, as summarised by Figure 1. One key figure of merit for these machines is efficiency: defined as the ratio of task-oriented-work out to total energy in. The design methodology and corresponding ability to predict efficiency are very closely linked.
The design flow for hard-robots (Figure 1a) is one in which simulation is paramount. The system is built using standardised and well-defined components and rigid links. Testing the physical robot is the last step in the design loop, and can almost always be replaced with a simulation. The system is controlled by making a predictive model, often based on defining the Jacobian of the system and then applying well-established methodologies using inverse kinematics. The efficiency of this class of robotic systems is relatively easy to predict, it relies on the composition of efficiencies of each of the well-defined components (motors, links, end effectors) in the system. This design flow has parallels with Hardware Description Languages (HDL) and Object Oriented Programming (OOP)-which hardware and software engineers use to design and to program computer processors. Using these types of paradigms, designers of complex hard-bodied robotic systems use well defined, and well characterised blocks.
In stark contrast, the design-flow for soft-robots (Figure 1b) is centred on building systems, and uses an empirical, experimental-science iterative loop. Soft robotic systems are often built using composite, and multi-domain blocks that are not completely characterised. The system control is often open-loop and locomotive actuation sequences are experimentally determined. Significant efforts towards simulating soft material systems has been made by Cotin and Lipson, resulting in programs such as SOFA 6 and VoxCAD 7-9 respectively. Testing the physical device is the key iteration in the design loop and, due to the complexity of simulating large-deformations, and their strong dependence on interaction with the environment, this loop cannot (yet) be completely replaced by simulation.
This design flow has parallels with early microprocessor designs which were laid-out by hand in an empirical, revisionist, iterative process. All soft robots convert stored energy (from electrochemical cells [10][11][12][13] , pressurised gas [14][15][16][17] , energy-dense chemicals 3,18-20 ) into taskoriented-work (grasping and lifting 21-23 , locomoting 10,11,16,17,24,25 , jumping 19,20 , swimming 13 ). In order for system designers to predict how long such a system will run on a given amount of stored energy they need to understand and model the forms of energy storage and dissipation, only then will they be able to control these systems efficiently.
These types of questions, about efficiency, about longevity, about recharging, are common to all sub-fields of mobile-robotics, but are particularly interesting when asked about those robots which: 1.) are bioinspired 3 , as they raise questions about how living systems store and recover energy, and insights into how these mechanisms could be improved; 2.) use direct, chemical to mechanical , actuation 19,20 , as the multi-domain physics is complex; or 3.) interact synergistically with biology, e.g. a human body 26 , as the coupling between living and non-living components presents great challenges in control and in safety.
The robotics community is starting to make strides towards developing robots that approach the minimum costs of transport (MCoT) for animals, as defined by Tucker 1 . One characteristic that is common to all robots, soft or otherwise, is the conversion of energy from one or more storage elements to one or more dissipative elements. These dissipative elements include those which perform task-oriented work, and those which dissipate heat-resulting in the irreversible loss of energy to the environment.
Robotic systems can, therefore, be analysed in terms of fundamental quantities: energy, work, and heat. The MIT cheetah robot, for example, is a robot which has been designed with energy in In order to use elastic structural and actuating elements most effectively, designers of soft systems will need to understand two grand challenges: 1.) Characterising soft robots in terms of energy transformation, calculating energetic "figures of merit", and identifying sources of energy loss; and 2.) Analysing, modelling, and simulating the whole-body mechanics and dynamics of large scale deformations in soft robotic systems.
In this paper, we offer our perspective on the first of these two grand challenges in soft robotic systems: how to characterise complex, multi-domain hybrid systems. We suggest that a modelling, analysis, and control framework built upon bond-graphs and port-Hamiltonian theory will be of great utility in the future of soft robotics. The framework also provides a link to the second grand challenge, which we do not tackle here. Bond-graphs and port-Hamiltonian theory describe the flow of energy through a system. They are tools for understanding the thermodynamics, and predicting the dynamic behaviour, of complex systems.
The Role of Thermodynamics in Robot Design-Energy: Storage, Transduction, and Dissipation Thermodynamics is often overlooked when designing tethered, factory-based, robotic systems as it is a secondary consideration to the task in hand-rapid, precise manipulation in a structured, well-defined, and people-free, environment. In contrast, when designing field robotics-where the task is reconnaissance, or transportation-analysis and predictions for storage, transduction, and dissipation of the energy within the system is paramount.
There are well established modelling approaches which focus on energy. For instance, Hamiltonian mechanics is a reformulation of classical Newtonian mechanics which takes as its starting point the Hamiltonian-a quantity which generally corresponds to the total energy contained in a mechanical system. From this single quantity it is possible to derive differential equations which govern the motion of the system. Although initially applied only to discrete (lumped-parameter) systems, Hamiltonian field theory has generalised the concepts of Hamiltonian mechanics to continuum systems. More recently, these ideas have been extended by port-Hamiltonian theory 5 , which generalises Hamiltonian mechanics further to the case of multidomain, dissipative, mixed discrete-continuum systems with inputs and outputs. We believe this theoretical framework provides an ideal tool for the analysis of soft robotic systems.
In two of our previous papers-on hybrid hard and soft robots 11 , and on using explosions to power soft robots 19 -we began to introduce the idea of analysing the efficiency and capabilities of these robots by discussing what was known about the transduction of energy in each system.
In this paper we use these systems to illustrate the application of a generalised framework for describing energy flow and dynamics for these types of robot. Once expressions for the total energy, power transfer, and dissipation in a robotic system have been formulated using this approach, they can be used for control and to optimise for efficiency. In the supplemental information we work through two examples which relate to the two systems shown in Figure 3: a rigid link robotic arm, and a soft-continuously deformable octopus tentacle. In the supplemental information we use the same energy-based analytical tools to derive the equations of motion for each of these two systems.

Modelling of Complex Systems-A Brief Introduction to Bond-Graphs
Bond-Graphs: In the 1950s at MIT Henry Paynter developed pictorial representations of interacting energetic elements, bond-graphs, as a way of modelling complex systems 4 . These graphs are a way of representing the flow of power in systems and they allow designers to test their assumptions and to draw relationships between interacting elements across multiple domains. Bond-graph theory 32 centres on the concept of ports that connect effort and flow variables, which together carry power. Table 1 shows some flow and effort variables for multiple domains.
In Bond-graph diagrams "ports" between system-blocks are connected with half-arrow bonds showing the usual direction of power-flow. The properties of a port are most easily understood by considering basic electrical circuit theory, in which the flow variable is current (I), and the effort variable is voltage (V). Loss would be characterised by a dissipative elements representing Ohmic (I 2 R) heating. The ideas however, have direct analogues across multiple physical domains, see Table 1 for examples.

Word Bond-Graph Analysis of two Soft Robotic Systems
Bond-Graph Analysis of a Hybrid Soft Robotic System: Figure 2a (derived from S6a) shows a word Bond-graph multi domain block diagram for a hybrid soft robotic system. In this system energy is derived from the electricity grid and task-oriented energy is dissipated by the interaction of the wheels and the pneunets with the environment. The global efficiency of this system can be calculated as the ratio between the sum of the electrical input energy and the mechanical output from the reaction of the wheels and pneunet with the environment. This bondgraph analysis allows us to identify what we need to know in order to simulate this system. For example we do not know how to simulate the link between the power into a pneunet actuator (from pressure and volumetric flow rate), and the power out (force and velocity). This analysis also reveals what type of sensors we would need to deploy in the system in order to monitor the flow and dissipation of energy. If we consider Table 1, we can see that there are a variety of parameters which designers would not routinely include in a non-energy based system controller.
This type of insight is critical in the design of energy optimal systems. Figure 2b (derived from S6b) shows a word Bond-graph multi domain block diagram for a soft robot which is powered by explosions. The global efficiency of this system can be calculated as the ratio between the sum of the electrical and chemical input energy and the mechanical output from the reaction of the pneunet with the environment. Following this process, as before, we find that we do not know how to simulate the link between power into an exploding soft actuator (spark voltage and current, and enthalpy and mass flow rate) and the power out (force and velocity). We can measure the energy-in (chemical potential, and mass flow), and the useful-work out (potential energy developed in jumping), so therefore we have a measure of the energy which is dissipated in the system, but predicting or simulating this block is complex.

From Bond-Graph to Dynamics: the Port-Hamiltonian Approach
In the previous two sections we discussed how the construction of a word bond-graph can help in identifying sources of energy loss and evaluating the efficiency of a system. Bond-graphs can also be powerful tools for taking a high-level description of a system to the point of mathematical analysis and simulation. To begin this process, we must detail the energetic transformations occurring within each block of an abstract word bond-graph. This is a recursive process, in which blocks are replaced with more detailed bond-graphs. The aim is to arrive at a system description in which each block represents a fundamental energy storage, transport, transduction, or dissipation element with a well-defined constitutive relationship between its power-conjugate flow and effort variables. Several excellent examples of how to reduce word bond-graphs to a minimal set of fundamental elements covering many physical domains can be found in the literature 28 .
Having produced a detailed bond-graph describing a system, it is possible to apply the tools of port-Hamiltonian theory to derive the differential-algebraic equations governing the dynamics of the system 28 . This theory extends energy-conservative Hamiltonian mechanics to the case of multi-domain, dissipative systems with inputs and outputs. The central quantity in this theory is the Hamiltonian-which generally represents the total energy stored within a system and can be constructed by consideration of the energy storage elements in a bond-graph. In Hamiltonian mechanics, this quantity is usually a sum of kinetic and mechanical potential (e.g. gravitational, elastic) energy. In port-Hamiltonian theory, it may equally contain terms for chemical energy or energy associated with electric and magnetic fields. These elements are interconnected mathematically via a power-conservative Dirac structure, whose form may be derived from the energy transport and transduction elements appearing in a detailed bond-graph. This structure allows the individual stores of energy represented in the Hamiltonian to interact, but also connects them to dissipative elements which irreversibly remove energy from the system as heat.
The fundamentals of energy storage, transport, transduction, and dissipation are common to both hard and soft robotics. Thus, the tools of port-Hamiltonian theory can be applied equally well to either. In the case of an entirely lumped-parameter system, for example a traditional robot with discrete electronics and rigid mechanical elements, the port-Hamiltonian dynamics equations take the following form: In a completely distributed-parameter system they take the form: In the lumped-parameter case, x is a vector of state variables which are derived from the bondgraph's flow and effort variables, and H is the Hamiltonian. In the distributed-parameter case, x is now a vector of field variables (e.g. the mechanical displacement and momentum fields for a deformable soft body) which depend on spatial coordinates X, and H is the Hamiltonian density-usually the energy density. In both cases J is a skew-symmetric map representing the power-conservative Dirac structure of the system, and R is a perturbation to J which allows for dissipation. Combinations of lumped-and distributed-parameter elements, a situation commonly encountered in soft robotics, are equally well treated by the port-Hamiltonian approach.
The procedure for deriving these equations of motion from a given bond-graph is systematized, and can even be carried out algorithmically 16 . This means that the roboticist is able to focus on an intuitive, pictorial representation (the bond-graph) of the system being designed. Furthermore, once a detailed bond-graph has been constructed for a given subsystem, it can be reused several times. So long as constitutive equations for the elements in the system can be provided, the difficult work of deriving the system's dynamics is taken care of.
The port-Hamiltonian theory can also be put to good use in developing controllers for complex systems. For instance, there has been much success in using passivity or energy-shaping control to alter the static and dynamic behaviour of port-Hamiltonian systems 28,29 . Given the generality and strong physical basis of this theoretical approach, many of the control techniques can be readily applied to soft, continuum systems 30 .

Gaining Insights by Using an Energy-Based Analytical Framework
The step-by-step framework we are proposing allows system designers to start with a complex system and move towards an energy-based system controller, this approach is composed of six steps: 1.) Writing the word bond-graph; 2.) Refining to a detailed bond-graph; 3.) Minimising the bond-graph; 4.) Developing the port-Hamiltonian and Dirac structures; 5.) Deriving the equations of motion; and finally 6.) Coding the system controller.
A limitation, which we identified by constructing both of the bond-graph analyses which we presented earlier in this paper, is that the block representing the pneumatic actuator has not yet been represented mathematically. This abstract-block represents a complex interplay of elements: 3D viscoelastic polymers are subjected to surface pressures from a compressible gas while dynamic reaction forces appear due to interaction of the whole network with a surface. On breaking this system into more basic energetic components, we realise that we do not know how to model the storage and loss of energy in the viscoelastic polymer. The construction of bondgraphs for these systems and progression towards simulation has highlighted exactly what empirical work is critical for the analysis of the system; we need to develop explicit constitutive relations for the viscoelastic polymer.
Note, however, that this need not stop us from proceeding to analysis, simulation, and control. If we approximate this constitutive relation-for instance by assuming infinitesimal strains and a linear material response 31 -we can begin using the tools of port-Hamiltonian theory to derive the dynamics for the entire, multi-domain system, and thus begin computational analysis. All models are based on theories, they require us explicitly to state our assumptions and they allow us to test our understanding. The approach of building bond-graphs and moving towards simulation by making successive approximations can help us find out where we should focus our future efforts on theoretical and empirical work.

Implications for Future Robotic Systems
In this perspectives-piece we have discussed how the use of bond-graphs and port-Hamiltonian theory generalises domain-specific knowledge and allows engineers to analyse complex and hybrid systems. We hope to popularise a mature framework for addressing energetic concerns in soft robotics, and we expect that it may also be used, systematically, to derive equations governing coupled, multi-domain dynamics. Using the insights gained from this type of holistic system overview-and one based on energy-engineers will be able to use elastic, energystoring, structural and actuating elements most effectively in future soft robotic systems. Bond-graphs are clearly a useful tool for conceptualising a system at various levels of abstraction. The application of port-Hamiltonian theory requires us to make quantitative modelling decisions, and to identify those areas in which idealisation or empirical analysis is most needed; the methods we have discussed here offer a significant step towards incorporating mathematical analysis, simulation, and control into the design flow of complex soft robotic systems. Table 1: Variables used in Bond-graphs and Port-Hamiltonian theory which describe the flow and effort which together carry power transfer between a range of domains, including thermal, mechanical, pneumatic, chemical, electrical, and magnetic.  task-oriented-work performed. The effort and flow variables used are described in Table 1. system (Octopus). These two systems can perform the same task-gripping-but each uses completely different mechanics, control systems, friction models, etc… Control paradigms that have been developed for rigid bodied systems such as the PUMA robot (e.g. deriving the Jacobian and computing the inverse kinematics) have little or no relevance to soft-bodied systems that have more characteristic in common with the octopus arm. The task-oriented-work performed by each system-such as gripping and lifting an object-is, however, the same.