Experimental learning of a lift-maximizing central pattern generator for a flapping robotic wing

In this work, we present an application of a policy gradient algorithm to a real-time robotic learning problem, where the goal is to maximize the average lift generation of a dynamically scaled robotic wing at a constant Reynolds number (Re). Compared to our previous work, the merit of this work is two-fold. First, a central pattern generator (CPG) model was used as the motion controller, which provided a smooth generation and transition of rhythmic wing motion patterns while the CPG was being updated by the policy gradient, thereby accelerating the sample generation and reducing the total learning time. Second, the kinematics included three degrees of freedom (stroke, deviation, pitching) and were also free of half-stroke symmetry constraint, together they yielded a larger kinematic space which later explored by the policy gradient to maximize the lift generation. The learned wing kinematics used the full range of stroke and deviation to maximize the lift generation, implying that the wing trajectories with larger disk area and lower frequencies were preferred for high lift generation at constant Re. Furthermore, the wing pitching amplitude converged to values between 45{\circ}-49{\circ} regardless of what the other parameters were. Notably, the learning agent was able to find two locally optimal wing motion patterns, which had distinct shapes of wing trajectory but generated similar cycle-averaged lift.