DRL - Cow Herding with Virtual Fences

From DRLWiki

This project seeks to develop tools for automatically generating dynamical models and control strategies for groups of animals based on recorded tracking data. It is envisioned that the resulting techniques will be useful for modelling and controlling a broad class of biological group phenomena including cow herding, bird flocking, insect swarming, and human crowd behavior. Specific applications are numerous. Notably, these tools would enable the monitoring and control of groups of people to alleviate foot traffic congestion, or to identify anomalous behavior. They would also enable the control of livestock to minimize environmental damage from over-grazing. In addition, such methods could be directly useful for the control of groups of autonomous mobile robots. Methods from nonlinear control, system identification, and statistical learning theory are currently being adapted to these purposes.

Figure 1. Agent-Agent interaction force magnitude.

[edit]

Model Description

We have proposed a simple difference equation model with four important features. Firstly, each animal is given individual dynamics to enforce kinematic constraints and basic physical laws. Secondly, a force-like contribution is applied to each animal from its interaction with each of the other animals in its group. This force law is designed to accommodate a principle common to many multi-body dynamical systems: forces between bodies are repulsive at close distances and attractive at far distances (see Figure 2). Thirdly, a force contribution is applied to each animal from its interaction with the environment. The force is a function of the animal's position, inducing a force-field over the environment. This force-field is parameterized in such a way that closed streamlines (orbits) are possible (see Figue 3).

Figure 2. Agent-Agent interaction force magnitude.

Figure 3. Agent-Environment interaction force-field.

Lastly, unknown elements of an animal's decision-motive processes are modelled as a force-like random disturbance.

[edit]

Systems Identification with Least Squares Fitting

The model structure discussed above has the convenient property that it is linear in its unknown parameters. We use this fact to perform a least squares regression using measured data from real animals to estimate the unknown parameters. This work is described in detail in .

We also have used a K-means classification algorithm to categorize stretches of animal data into higher-level states Image:SchwagerCEA06.pdf. Each state will then have an associated set of regressed model parameters. We plan to model the transition among these higher-level states as a Markov model and identify the state-transition probabilities from the data using maximum likelihood techniques. Control

Several strategies exist for the control of multi-body dynamical systems [3], [4]. Specific control laws depend on what actuation is available. Experiments have been carried out on herds of cattle using collars that emit a sound to stimulate the animal's movement in a particular direction. Developing useful control actuators for groups of animals posses a significant practical challenge.

[edit]